Org Tables Changed My Life
Summary
TLDRThis video delves into the powerful table functionality of Emacs' Org mode, showcasing its capabilities for tabular data entry, spreadsheet-like manipulation, and advanced features. The host demonstrates basic table creation, data manipulation, and the integration of calculations and references within Org mode. They also explore Org Babel for executing code snippets and generating dynamic content, illustrating practical applications such as financial calculations and data visualization with R programming language, all within the versatile Org mode environment.
Takeaways
- ๐ Org mode offers powerful table functionalities similar to HTML and Markdown for tabular data entry.
- ๐งฎ Org mode integrates spreadsheet capabilities for data manipulation directly within the editor, including references and calculations.
- ๐ Users can easily manipulate tables in Org mode by adding, removing, or moving rows and columns using keyboard shortcuts.
- ๐ Org mode supports advanced features like passing tables to an interpreter and using a programming language for more complex data handling.
- ๐ก The video demonstrates basic table creation, manipulation, and the use of spreadsheet functions within Org mode.
- ๐ข Org mode allows for calculations in tables using specific keyboard commands, such as 'C-=' for column calculations and 'C-c *' for field-specific calculations.
- ๐ References to other tables or ranges within a table can be made using a combination of row and column indicators, enhancing data linkage and calculation accuracy.
- ๐ Org Babel enables the execution of code blocks within Org files, allowing for integration with various programming languages for more advanced data processing.
- ๐ The script showcases practical applications of Org mode tables, including financial calculations and data visualization with programming languages like Python and R.
- ๐ The video mentions 'Simx', an Emacs setup tailored for scientists and engineers, as a potential starting point for those new to Emacs and looking for specialized functionality.
- ๐ The speaker encourages viewers to explore Org mode's capabilities further, especially for automating personal and professional tasks, and to provide feedback for additional content.
Q & A
What is the main topic discussed in the video script?
-The main topic discussed in the video script is the table functionality in org-mode, which includes creating and manipulating tables, performing calculations, and integrating with programming languages for advanced features.
What is org-mode and why is it useful?
-Org-mode is a mode in Emacs for organizing and editing notes, lists, and projects. It is useful because it allows users to create tables, manage tasks, and even write documents with powerful features like spreadsheet functionality and integration with programming languages.
How can you create a table in org-mode?
-You can create a table in org-mode by initializing it with a horizontal bar and then pressing the Tab key to start entering data. You can also use a vertical bar followed by 'a' and Tab to create a header.
What are some advanced features of org-mode tables mentioned in the script?
-Some advanced features of org-mode tables include the ability to manipulate tables by adding or removing rows and columns, performing calculations with spreadsheet functionality, referencing other tables, and using org-babel to execute code within an org file.
How can you perform calculations within org-mode tables?
-You can perform calculations within org-mode tables using the 'control equals' command to enter a calculation for a column, or 'control U control equals' for a specific field. You can also use references to other cells or tables for more complex calculations.
What is org-babel and how is it used in the context of org-mode tables?
-Org-babel is a feature of org-mode that allows you to execute code blocks within an org file. It can be used to generate results from programming languages like Python or R, which can then be used as input or output within org-mode tables.
How can you reference data from one table to another in org-mode?
-You can reference data from one table to another in org-mode by using the 'remote' keyword followed by the table name and the cell reference, such as 'remote:costs@2$2' to reference the second row and second column of a table named 'costs'.
What is the significance of the '$' and '@' symbols in org-mode table calculations?
-In org-mode table calculations, the '$' symbol is used to reference a column, and the '@' symbol is used to reference a row. This allows for precise calculations based on specific cells within a table.
How can you import data into an org-mode table?
-You can import data into an org-mode table by using the 'table-import' command or by selecting a region of comma-separated or tab-separated values and using the 'control vertical bar' command to convert it into a table.
What is the practical application of org-mode tables discussed in the script?
-The script discusses using org-mode tables for practical applications like financial calculations, daily life calculations, and even generating graphs or plots using programming languages integrated with org-babel.
Outlines
๐ Org Mode Table Functionality Overview
This paragraph introduces the video's focus on Org Mode's table functionality, which allows users to enter tabular data akin to HTML and Markdown. It highlights Org Mode's spreadsheet capabilities for data manipulation within the editor, including adding, removing, and referencing tables. The speaker emphasizes the advanced features like passing tables to interpreters for complex tasks, and the overall utility of Org Mode for note-taking and data management. The video promises to cover basic table creation, advanced manipulation, spreadsheet functionalities, and interpreter usage.
๐ Deep Dive into Org Mode's Table Manipulation and Calculations
The speaker discusses advanced table manipulation techniques in Org Mode, such as moving fields and rows with keyboard shortcuts, and copying and pasting data within tables. They demonstrate how to generate tables from comma or tab-separated values and introduce the powerful spreadsheet functionalities, including column-wise calculations and cell-specific formulas. The paragraph also covers referencing cells and ranges within tables for dynamic calculations, showcasing Org Mode's ability to handle complex data operations seamlessly.
๐ Leveraging References and Calculations in Org Mode Tables
This section explores the use of references for dynamic calculations across different tables in Org Mode. The speaker explains how to create references using a combination of row and column indicators and perform operations like summing values from referenced cells. They also touch upon the use of Calc, Emacs's built-in calculator, for arithmetic and algebraic operations within tables. The paragraph concludes with an introduction to org-babel, a feature that allows executing code blocks in various programming languages directly within Org files.
๐ Practical Applications and Org Babel Integration
The final paragraph presents practical applications of Org Mode's table functionalities, such as integrating with SQL databases for data manipulation and visualization. The speaker demonstrates how to filter, export, and import data using SQL queries within Org tables. They also showcase the integration of Org Babel with programming languages like Python and R for more complex data analysis and graphical representation. The video concludes with a mention of Simx, an Emacs setup tailored for scientists and engineers, and an appreciation for the channel's supporters.
Mindmap
Keywords
๐กOrg Mode
๐กTables
๐กSpreadsheet Functionality
๐กData Manipulation
๐กCalculations
๐กReferences
๐กInterpreter
๐กKelk
๐กOrg Babel
๐กSQL Light Database
Highlights
Introduction to org mode's table functionality for tabular data entry, similar to HTML and markdown.
Emacs and org mode's powerful spreadsheet functionality for data manipulation and calculations.
The ability to reference and calculate data within org mode tables for financial management and other uses.
Advanced features in org mode that allow tables to be passed to an interpreter for more complex tasks.
Basic table creation in org mode using simple horizontal bars and tab navigation.
Creating headers in tables for better data distinction using vertical bars and tab.
Manipulating tables by moving fields and rows with arrow keys and modifiers in Emacs.
Adding and deleting rows and columns in tables with specific key combinations.
Copying and pasting data within tables using control space and rectangle selection.
Generating tables from comma-separated or tab-separated values using control and vertical bar.
Importing tables from CSV files using table import feature in org mode.
Performing calculations in tables using control equals and column references.
Entering specific field calculations with control U and the use of dollar signs for row references.
Using references for advanced calculations, combining row and column references for complex formulas.
The integration of calc, Emacs's built-in stack-based calculator, for advanced mathematical operations in tables.
Executing code within org files using org-babel for generating results and automating tasks.
Practical application of org mode tables with programming languages for database manipulation and data processing.
Using org-babel with R programming language to generate graphical plots from table data.
The potential of org mode for automating financial and daily life calculations, as well as generating graphs and plots.
Invitation for viewers to learn more about advanced features of org mode and its practical applications.
Transcripts
how's it going you guys I'm here today
to talk to you once again about org mode
specifically we're going to talk today
about their table functionality tables
in org mode can be used for what you'd
expect in most other markup languages
like HTML and markdown for entering
tabular data you know kind of like your
name your age all that sort of stuff now
another thing that you kind of get with
emac and org mode that I find quite
powerful is the spreadsheet
functionality you can also manipulate
the data um using the editor itself so
delete adding removing different tables
rows fields that sort of stuff but you
also get spreadsheet functionality for
doing references and calculations which
I find super helpful for doing my
finances as well as the usual sort of
small level uh table functionality that
You' kind of hope for when you're
referencing other tables you can do a
lot of that there um and in addition you
can do some really Advanced features
where you take these tables you can pass
them to an interpreter use a programming
language of your choice to do some more
advanced features and then spit that out
as a that you can use later this is one
of the things that I find extremely
powerful with emac and let you kind of
come up with ideas write it all down so
you can reference it later and then get
the results that you want and not really
think too much about what's the best
tool for the job you can kind of just
work with it let the data flow as you
want and reference it later tables are
one of those things that I myself have
used more and more over the years and
have slowly fallen in love with it's
been one of those things that I think
org really gets right and hopefully by
the end of this video you guys will feel
the same way we'll start off with some
very basic functionality just designing
a table then we'll move into more
advanced stuff like manipulating the
table using emac and then we'll move
into references calculations and the
spreadsheet functionality and then
finally we'll move into The Interpreter
level stuff that's a bit more advanced
and extremely powerful now without any
more delays let's get into the video so
first off let's look at creating a table
so tables can be initialized using a
quick little just horizontal bar like so
and if you just hit tab boom you've got
another table just started so usually
what I like to do is I usually go ahead
and start entering some data so let's
say we want to make our heading line
name and age and then I can hit Tab and
it will start letting me enter in data
now often you want to make like a little
header so it's a bit nicer because if I
just entered in Gavin uh you guys don't
need to know my age I'm I'm for all you
know I'm
100 uh and then I could keep going but
usually you want something to kind of
distinguish these two and so to do that
you can do a vertical bar and then a and
hit Tab and it will sort of fill that in
for you and create a little header so
that's basically what I say here and you
can navigate these guys using tab kind
of as you'd expect and then you can use
enter um to kind of uh go to the next
level now moving on to a more advanced
feature we have the ability to actually
manipulate these tables um so for
example if we wanted to say move things
around you can use shift and the arrow
keys to kind of move around a field and
we'll go ahead and undo that and you
could do the same obviously with any of
these fields now you can also use meta
to kind of move say an entire column
around or an entire row
around um and in addition you can use
shift in combination with meta with down
to add one and then with up to delete it
my up key is broken so I can't show that
um and then you can do some extra stuff
with rows as well so right would add a
row left would delete a row or sorry
column which is quite powerful and quite
helpful for sort of the general
manipulation stuff obviously it depends
on where the cursor is but yeah so
that's kind of the main workflow for
sort of manipulating the table is using
the arrow keys with different modifiers
you can obviously do the regular sort of
marking deleting and moving stuff around
as you'd expect now another thing when
it comes to modifying the different data
is that sometimes you want to copy paste
and so the way that this is done is
actually pretty interesting so you can
do control space here and kind of move
your cursor around and so you can do
kind of row and column selection like
this so while it does show that I'm
going across two lines I'm actually
selecting from this little rectangle
right here and so I can do crl c crl x
and then any of these Keys above us
let's do alt W and so as you can see it
said that I copied and so now if I do
contrl c crl xrl y it will paste that
data um and I can kind of do say for
example if I did here and went down to
here and did crl C CR X Alt W and copied
like we just did and then do crl c CR XR
y um we can copy and paste just the
column right here um so this can be very
helpful and very powerful if you guys
want to do some more like modifications
and moving things around without uh
disrupting the actual layout too much
now I'll admit I don't use this all the
time but it is helpful on the occasions
where I kind of feel like I need it and
now let's move on to generating tables
generating tables is actually a really
cool feature Within emac there's a bunch
of different ways to do it this is just
two of them so uh one of the ways is
using control and then the vertical bar
and so you'll see it right down here in
my like what I'm typing so you can kind
of turn anything that's comma separated
or tab separated into a table um so this
is really useful like I mentioned if I
just copy pasted say a list from a
website or something like that I can uh
select the region do contrl c vertical
bar and it will turn that into a table
now another example is using U Altex or
table
import and then I can just go to results
this is just an example CSV that I have
right here this was kind of a bad spot
to import it um let's undo that and
import it here so we'll just get the
same thing that we did last time we
import it and you as you can see right
here we imported uh the entire table and
we can kind of navigate it as you'd
expect and uh move things around as
you'd expect um yeah and so this is kind
of a nice way to kind of take like a CS
V or a tsv like I mentioned here tab
separated value and kind of import it as
you did hopefully okay now we can get
into the actual calculations which is
one of the really cool things about org
mode so there's a few different ways to
enter them but we'll go through them one
at a time so there's control equals
which will let you enter a calculation
for a column so right now uh this is
pretty arbitrary but let's go ahead and
just change this to 10 12 and then let's
make this column some sort of a
calculation so what we can do is we can
do contrl C
equals and then we can say that it is
that this column is column one plus 10
hit enter and it will apply that for the
column if we go and remove that cuz that
doesn't really matter there we go now we
get the sort of calculation that you'd
expect so each of these guys plus 10 and
so the dollar sign is how you specify a
row now you can also do an extra
calculation say if you want to do
control U contrl C equals then you can
make it specific to the field and say
let's just make this field equal to 10
minus
20 um which equals
-10 um and as you can see they're all
added down here into this table FM which
I think is short for table format is
kind of what they're going for and so
this is kind of used to do some
calculations and kind of just display
the results up here now you can kind of
uh recalculate them if you wanted to
modify them so let's say move minus this
to 10 and then we do contrl c contr c or
if you want to just do it specific to a
field you can do contrl c star which is
useful but really these sort of General
things I'm just showing off for the sake
of it we'll get into some more advanced
and actually useful uh ways to sort of
modify these values in just a second so
when we move on to references we
actually get some really Advanced
features and so this kind of makes the
actual calculations a bit nicer so as
you can see right down here I make a
small reference to the second row in the
second column remember that dollar sign
indicates that this is the column and
the at symbol indicates that this is a
row now there's a few different ways
that you can manipulate them as I
mentioned here you can do contrl c uh
quote single quote contrl C single quote
will give you the current calculation so
right now we have this set to calculate
to one but we can instead say that we
want to do plus and then let's say we
want to do at one uh dollar sign two and
so it will indicate where that is now
instead let's like go ahead and change
that to the right and then down and so
now it will be one plus that entry and
we can do contrl c contrl c to save that
and there we go and when we recalculate
it using contrl C control C you'll see
that it is computed now just like before
we can edit it once again and say maybe
we want to uh change this to a 10 or 101
I guess there we go we'll make it 10 and
then contr C contr C and then and we can
re-evaluate it I find this sort of a
workflow to be much better than the uh
one I talked about before um but it's
most effective when you're using it with
references I find now if you ever get
confused you can do a contrl c uh curly
brace this will kind of give you a bit
of a layout and kind of breakdown of
what you're looking at if you want to
know what it is for one of the field you
can do contrl c question mark and like I
said before contrl c contrl c aligns
things once again here's a little
breakdown of how references work so at
references a row the dollar sign
references a column if you want to
reference something say like very last
element in a range and you can use the
greater than symbol and then if you want
to specify a range you can use two dots
so here's a really good example now an
alternative notation instead of having
to control C quote or any of these other
things you can actually just do a colon
equals to enter in a calculation so here
we're saying that we want the second row
in the First Column so that's this guy
all the way to the third row in the
second column so we're selecting this
region right here all right and so if I
just hit tab boom that expands and we
get the actual calculation and as you
can see it's one right here if we go
through you'll notice that it lines up
with this one this guy this guy this guy
and we created a range from that and as
you can see that calculation ends up
down here so this is a really quick way
to just sort of write some quick
shorthand if you just want to do like a
quick calculation like we were talking
about before like 10 + 20 tab you get
the calculation kind of like what you'd
expect now what if we want to reference
other tables now this is a really
awesome feature I use this all the time
for the accounting stuff that I was
talking about before so let's say we
have uh some things and costs so we say
that car costs x amount of money house
costs x amount of money and so we could
have some calculations down here but in
this case we don't and then we can
actually reference it so here we use
remote to indicate that this is a
reference we're referencing costs which
comes from Up here we labeled this table
as costs and then we're saying that we
want the element right here all the way
to this element which uh really is only
two things that I could have said but I
created a range from it so second row
all the way to the third row both in the
second column and then I am using vs sum
to sum these results and as you can see
right here if I um remove that or let's
just remove that and recalculate it
you'll see that the results align and we
can sort of uh change this to
and recalculate and we get the updated
result now if you're wondering where uh
all of the powers of adding subtracting
and everything come from it's using a
thing in emac called Kelk um which you
guys can find in the info if I go ahead
and take a look at it it's a stack-based
calculator um it's very powerful I'm
pretty sure it even has like Taylor
series generation a bunch of different
stuff like generating Graphics as you
can see basically anything you see here
arithmetic it can do linear algebra um
it could do a lot of advanced math stuff
that you guys can put in your tables
obviously I don't take advantage of that
all the time um but from time to time it
can be super useful now when Kelk isn't
enough this is when I reach out to org
Babble org Babble is a way to sort of
execute code from within an org file I
haven't made a video on it but there's
lots of videos out there for those of
you interested um but there's pretty
cool stuff that you can do with it so
let's go ahead and do a really simple
example using a table so the really
simple one is just generating results so
here I have a source block using python
and I say return a uh list containing
the values from zero all the way up to
five and so if I hit control C control C
to execute it I'll get a range of 0 1 2
3 and four now we can also uh take data
and pass it in so for example I have a
table here I have named it note that I
didn't use the table name this is just a
generic name that we can use this is how
you can label basically everything in
org mode is using the hash plus name and
so we named it messages and they're from
and to somebody and then we actually
take that and as you can see right here
with VAR we're actually passing that in
as a variable called data and we're
actually using emac list this time not
just python um so for those of you guys
that don't have python installed very
impressive that you don't have installed
but uh you can actually do this with
just regular emac list that will work
everywhere um and so basically what I do
here is I iterate over that data which
we passed in up here and we go through
and we go through each row and we print
from Two and we get the first element
that's given to us which is Jim and the
second element which is John all right
and so if we go ahead and execute that
you'll see we get the results of from
Jim to John from Tim to Drake from Griff
to step uh questionable names there at
the end hey just coming in to interrupt
the video to talk about today's sponsor
brilliant brilliant is an online
learning platform when it comes to
learning a lot of us jump to things like
tutorials and lectures and really miss
out on being able to absorb everything
that we're being told this is where
brilliant really stands out it gives you
interactive real world examples that you
can think through and problem solve and
really retain the information that
you're learning in this video I show you
how you can use org tables and integrate
them with different programming
languages and interpreters if this is a
New Concept to you I highly recommend
taking a look at the computer science
fundamentals course from them that goes
a bit into the basic ideas of
programming that a lot of new
programmers can learn a lot from and you
can start applying what we talk about at
the end of this video in those Advanced
features as you go if furthering your
career knowledge or education interests
you I highly recommend giving a look at
brilliant to try everything brilliant
has to offer now for free for a full 30
days check out brilliant.org gaven
freeorn first 200 of you will get 20%
off brilliant's premium annual
subscription once again thanks to
brilliant for sponsoring this video okay
so let's think of a practical
application for those of us that do
programming so in this case let's say we
have a database and in this case I have
one that is an SQL light database
containing multiple restaurants so if I
do contrl c contrl c we'll see that we
can get the results like we were doing
before as a table now this is a little
messy so we can go sort of clean this up
by executing this which will actually
filter out by the important data now
this is quite good and we can kind of
like we were saying before manipulate
these uh different guys as you'd expect
so kind of adding a row removing a row
if we want to filter down to maybe just
like the location there you go now while
exporting these results is useful
sometimes you want to take them in as
input so let's go ahead and do that with
another SQL light database so we have a
table right here it says Who and the
task that they have we have two tasks
for gaven that's me and then some other
tasks for some other people now in this
case I'm going to uh set a variable and
we're going to set that variable to be
called some data and it's going to take
our table as we can see up here called
test table uh and we do some extra
settings I'm not really going to get dig
into those and we say that the DB is
just a temporary database that we'll be
using and we'll say that we're going to
drop this table if it already exists um
just for this Demo's purpose and then we
will import it and so now if we contrl C
control C on this we will see that it's
able to select who and task where the
who is Gavin and so as you can see right
here it selected just the ones that were
tasks for me now just to prove that I'm
not making this up we can go ahead and
use emac to actually open this and if if
we open up the database you can see that
it's all the original data that we
originally entered in you can do a lot
of this stuff with basically any SQL
light database um as well as any SQL in
general database um there's a bunch of
information both these links that you
can find in these show notes and finally
we'll move on to another Advanced
feature using the org Babel integration
um so this is a bit more of a breakdown
of what's going on here so in this case
we're going to use the r programming
language to do some mathematical stuff
do note that the ES s package is what's
used to give us our source code
functionality once you have the ESS
package installed you can also set up
the org Babble functionality for R by
just uh executing this eisp and here
let's go ahead and dig into it so here
we have a table which has each month of
the year and then what the temperature
was and so we can kind of plot this out
so here we're going to once again take
that table up there turn it into Data
we're going to say that our results will
be a graphical file we'll say that that
file is going to be a PNG um and that
way we will export it I think that's
what the exports was I can't actually
remember what I entered that part in for
we'll plot it and then we give it a
legend for the plot and we can hit crl c
contrl c and then you can use as I
mentioned down here contrl C CR x contrl
v to preview it and as you can see we
generated a very nice graph um to get
the data that we wanted I know that gnu
plot also can do this sort of stuff but
if you guys wanted to do statistics and
stuff um R is there and you can use
something like gnu octave if you wanted
to do uh linear algebra that maybe Kel
can't do um or anything like that
there's a lot of functionality out there
if this interest you guys uh be sure to
check out simx I think that's how they
expect you to pronounce it um which is
basically an emac setup meant for uh
scientists and Engineers um which is
really cool I've looked at that in the
past obviously for me it doesn't really
make sense because I'm so used to emac
but uh for those of you that aren't it
might be a very interesting starting
point so where does this functionality
leave us well in the end we have quite a
lot of power that we can use to automate
our lives do things like Financial
calculations daily life calculations
very advanced stuff for maybe generating
a graph or a plot if you guys want to
learn more about how I use it maybe how
I do some of my finances um and maybe
some more advanced features that you can
do with org mode and tables be sure to
let me know down in the comments now
before I let you all go I wanted to give
a big shout out to my supporters on
patreon and GitHub sponsors you guys
mean the world to me and you guys have
done so much to support the channel and
kind of keep keep things going uh in
between sponsors which I really
[Music]
appreciate
5.0 / 5 (0 votes)