Org Tables Changed My Life

Summary

This video delves into the powerful table functionality of Emacs' Org mode, showcasing its capabilities for tabular data entry, spreadsheet-like manipulation, and advanced features. The host demonstrates basic table creation, data manipulation, and the integration of calculations and references within Org mode. They also explore Org Babel for executing code snippets and generating dynamic content, illustrating practical applications such as financial calculations and data visualization with R programming language, all within the versatile Org mode environment.

Takeaways

• 📊 Org mode offers powerful table functionalities similar to HTML and Markdown for tabular data entry.
• 🧮 Org mode integrates spreadsheet capabilities for data manipulation directly within the editor, including references and calculations.
• 🔄 Users can easily manipulate tables in Org mode by adding, removing, or moving rows and columns using keyboard shortcuts.
• 📑 Org mode supports advanced features like passing tables to an interpreter and using a programming language for more complex data handling.
• 💡 The video demonstrates basic table creation, manipulation, and the use of spreadsheet functions within Org mode.
• 🔢 Org mode allows for calculations in tables using specific keyboard commands, such as 'C-=' for column calculations and 'C-c *' for field-specific calculations.
• 🔗 References to other tables or ranges within a table can be made using a combination of row and column indicators, enhancing data linkage and calculation accuracy.
• 📈 Org Babel enables the execution of code blocks within Org files, allowing for integration with various programming languages for more advanced data processing.
• 📚 The script showcases practical applications of Org mode tables, including financial calculations and data visualization with programming languages like Python and R.
• 🌐 The video mentions 'Simx', an Emacs setup tailored for scientists and engineers, as a potential starting point for those new to Emacs and looking for specialized functionality.
• 🎓 The speaker encourages viewers to explore Org mode's capabilities further, especially for automating personal and professional tasks, and to provide feedback for additional content.

Q & A

What is the main topic discussed in the video script?

The main topic discussed in the video script is the table functionality in org-mode, which includes creating and manipulating tables, performing calculations, and integrating with programming languages for advanced features.

What is org-mode and why is it useful?

Org-mode is a mode in Emacs for organizing and editing notes, lists, and projects. It is useful because it allows users to create tables, manage tasks, and even write documents with powerful features like spreadsheet functionality and integration with programming languages.

How can you create a table in org-mode?

You can create a table in org-mode by initializing it with a horizontal bar and then pressing the Tab key to start entering data. You can also use a vertical bar followed by 'a' and Tab to create a header.

What are some advanced features of org-mode tables mentioned in the script?

Some advanced features of org-mode tables include the ability to manipulate tables by adding or removing rows and columns, performing calculations with spreadsheet functionality, referencing other tables, and using org-babel to execute code within an org file.

How can you perform calculations within org-mode tables?

You can perform calculations within org-mode tables using the 'control equals' command to enter a calculation for a column, or 'control U control equals' for a specific field. You can also use references to other cells or tables for more complex calculations.

What is org-babel and how is it used in the context of org-mode tables?

Org-babel is a feature of org-mode that allows you to execute code blocks within an org file. It can be used to generate results from programming languages like Python or R, which can then be used as input or output within org-mode tables.

How can you reference data from one table to another in org-mode?

You can reference data from one table to another in org-mode by using the 'remote' keyword followed by the table name and the cell reference, such as 'remote:costs@2$2' to reference the second row and second column of a table named 'costs'.

What is the significance of the '$' and '@' symbols in org-mode table calculations?

In org-mode table calculations, the '$' symbol is used to reference a column, and the '@' symbol is used to reference a row. This allows for precise calculations based on specific cells within a table.

How can you import data into an org-mode table?

You can import data into an org-mode table by using the 'table-import' command or by selecting a region of comma-separated or tab-separated values and using the 'control vertical bar' command to convert it into a table.

What is the practical application of org-mode tables discussed in the script?

The script discusses using org-mode tables for practical applications like financial calculations, daily life calculations, and even generating graphs or plots using programming languages integrated with org-babel.

Outlines

00:00

📊 Org Mode Table Functionality Overview

This paragraph introduces the video's focus on Org Mode's table functionality, which allows users to enter tabular data akin to HTML and Markdown. It highlights Org Mode's spreadsheet capabilities for data manipulation within the editor, including adding, removing, and referencing tables. The speaker emphasizes the advanced features like passing tables to interpreters for complex tasks, and the overall utility of Org Mode for note-taking and data management. The video promises to cover basic table creation, advanced manipulation, spreadsheet functionalities, and interpreter usage.

05:01

🔍 Deep Dive into Org Mode's Table Manipulation and Calculations

The speaker discusses advanced table manipulation techniques in Org Mode, such as moving fields and rows with keyboard shortcuts, and copying and pasting data within tables. They demonstrate how to generate tables from comma or tab-separated values and introduce the powerful spreadsheet functionalities, including column-wise calculations and cell-specific formulas. The paragraph also covers referencing cells and ranges within tables for dynamic calculations, showcasing Org Mode's ability to handle complex data operations seamlessly.

10:02

🔗 Leveraging References and Calculations in Org Mode Tables

This section explores the use of references for dynamic calculations across different tables in Org Mode. The speaker explains how to create references using a combination of row and column indicators and perform operations like summing values from referenced cells. They also touch upon the use of Calc, Emacs's built-in calculator, for arithmetic and algebraic operations within tables. The paragraph concludes with an introduction to org-babel, a feature that allows executing code blocks in various programming languages directly within Org files.

15:03

📈 Practical Applications and Org Babel Integration

The final paragraph presents practical applications of Org Mode's table functionalities, such as integrating with SQL databases for data manipulation and visualization. The speaker demonstrates how to filter, export, and import data using SQL queries within Org tables. They also showcase the integration of Org Babel with programming languages like Python and R for more complex data analysis and graphical representation. The video concludes with a mention of Simx, an Emacs setup tailored for scientists and engineers, and an appreciation for the channel's supporters.

Keywords

💡Org Mode

Org Mode is a document and information organizer for Emacs, which is a powerful text editor. It is designed for note-taking, project planning, and authoring with a fast and effective plain-text system. In the video, Org Mode is the central theme, with the speaker discussing its table functionality and advanced features for data manipulation and calculations, which are essential for organizing and managing information.

💡Tables

In the context of the video, tables refer to the tabular data structures within Org Mode that allow users to organize information in rows and columns. The script mentions using tables for entering data like names and ages, and the speaker demonstrates how to create and manipulate tables, emphasizing their importance in the Org Mode ecosystem.

💡Spreadsheet Functionality

The term 'spreadsheet functionality' in the script refers to the capability of Org Mode to perform calculations and data manipulation similar to traditional spreadsheet software. The speaker highlights this feature as powerful for tasks such as financial calculations, showcasing how Org Mode can handle complex data operations within a plain-text environment.

💡Data Manipulation

Data manipulation involves altering and organizing data within tables. The script discusses how Org Mode allows users to add, delete, and move rows and columns in tables, as well as copy and paste data. This is crucial for users who need to dynamically adjust their data structures without losing the integrity of the table format.

💡Calculations

Calculations in Org Mode refer to the ability to perform mathematical operations within tables. The script explains how users can use Org Mode's built-in functions to add, subtract, and reference values for computation. This feature is exemplified by the speaker when demonstrating how to add a constant value to a column or perform more complex calculations using references.

💡References

In the script, 'references' are used to link data from one table to another or within the same table. The speaker explains how to create references to specific cells or ranges of cells, allowing for dynamic calculations that update automatically when the referenced data changes. This is particularly useful for creating interconnected data sets and maintaining consistency across a document.

💡Interpreter

The 'interpreter' keyword in the script refers to the ability to pass table data to an external programming language for advanced processing. The speaker mentions using an interpreter to extend Org Mode's capabilities, such as generating graphs or performing statistical analysis, by leveraging the power of languages like Python, R, or Emacs Lisp.

💡Kelk

Kelk, mentioned in the script, is a stack-based calculator in Emacs that Org Mode can utilize for its calculations. It is a powerful tool that can handle a wide range of mathematical operations, from basic arithmetic to more complex functions, enabling Org Mode users to perform advanced calculations within their tables.

💡Org Babel

Org Babel is a feature within Org Mode that allows for the execution of source code blocks in various programming languages. The script describes how Org Babel can be used to generate results from code, such as creating a list of numbers or filtering a database query. It is a demonstration of how Org Mode can be extended for more complex tasks beyond its native capabilities.

💡SQL Light Database

In the context of the video, an 'SQL Light Database' is a simplified version of a traditional SQL database, which is used to manage and query data. The script includes an example where the speaker uses Org Babel to interact with an SQL Light Database, demonstrating how to filter, import, and export data using Org Mode's capabilities.

Highlights

Introduction to org mode's table functionality for tabular data entry, similar to HTML and markdown.

Emacs and org mode's powerful spreadsheet functionality for data manipulation and calculations.

The ability to reference and calculate data within org mode tables for financial management and other uses.

Advanced features in org mode that allow tables to be passed to an interpreter for more complex tasks.

Basic table creation in org mode using simple horizontal bars and tab navigation.

Creating headers in tables for better data distinction using vertical bars and tab.

Manipulating tables by moving fields and rows with arrow keys and modifiers in Emacs.

Adding and deleting rows and columns in tables with specific key combinations.

Copying and pasting data within tables using control space and rectangle selection.

Generating tables from comma-separated or tab-separated values using control and vertical bar.

Importing tables from CSV files using table import feature in org mode.

Performing calculations in tables using control equals and column references.

Entering specific field calculations with control U and the use of dollar signs for row references.

Using references for advanced calculations, combining row and column references for complex formulas.

The integration of calc, Emacs's built-in stack-based calculator, for advanced mathematical operations in tables.

Executing code within org files using org-babel for generating results and automating tasks.

Practical application of org mode tables with programming languages for database manipulation and data processing.

Using org-babel with R programming language to generate graphical plots from table data.

The potential of org mode for automating financial and daily life calculations, as well as generating graphs and plots.

Invitation for viewers to learn more about advanced features of org mode and its practical applications.

Transcripts

how's it going you guys I'm here today

to talk to you once again about org mode

specifically we're going to talk today

about their table functionality tables

in org mode can be used for what you'd

expect in most other markup languages

like HTML and markdown for entering

tabular data you know kind of like your

name your age all that sort of stuff now

another thing that you kind of get with

emac and org mode that I find quite

powerful is the spreadsheet

functionality you can also manipulate

the data um using the editor itself so

delete adding removing different tables

rows fields that sort of stuff but you

also get spreadsheet functionality for

doing references and calculations which

I find super helpful for doing my

finances as well as the usual sort of

small level uh table functionality that

You' kind of hope for when you're

referencing other tables you can do a

lot of that there um and in addition you

can do some really Advanced features

where you take these tables you can pass

them to an interpreter use a programming

language of your choice to do some more

advanced features and then spit that out

as a that you can use later this is one

of the things that I find extremely

powerful with emac and let you kind of

come up with ideas write it all down so

you can reference it later and then get

the results that you want and not really

think too much about what's the best

tool for the job you can kind of just

work with it let the data flow as you

want and reference it later tables are

one of those things that I myself have

used more and more over the years and

have slowly fallen in love with it's

been one of those things that I think

org really gets right and hopefully by

the end of this video you guys will feel

the same way we'll start off with some

very basic functionality just designing

a table then we'll move into more

advanced stuff like manipulating the

table using emac and then we'll move

into references calculations and the

spreadsheet functionality and then

finally we'll move into The Interpreter

level stuff that's a bit more advanced

and extremely powerful now without any

more delays let's get into the video so

first off let's look at creating a table

so tables can be initialized using a

quick little just horizontal bar like so

and if you just hit tab boom you've got

another table just started so usually

what I like to do is I usually go ahead

and start entering some data so let's

say we want to make our heading line

name and age and then I can hit Tab and

it will start letting me enter in data

now often you want to make like a little

header so it's a bit nicer because if I

just entered in Gavin uh you guys don't

need to know my age I'm I'm for all you

know I'm

100 uh and then I could keep going but

usually you want something to kind of

distinguish these two and so to do that

you can do a vertical bar and then a and

hit Tab and it will sort of fill that in

for you and create a little header so

that's basically what I say here and you

can navigate these guys using tab kind

of as you'd expect and then you can use

enter um to kind of uh go to the next

level now moving on to a more advanced

feature we have the ability to actually

manipulate these tables um so for

example if we wanted to say move things

around you can use shift and the arrow

keys to kind of move around a field and

we'll go ahead and undo that and you

could do the same obviously with any of

these fields now you can also use meta

to kind of move say an entire column

around or an entire row

around um and in addition you can use

shift in combination with meta with down

to add one and then with up to delete it

my up key is broken so I can't show that

um and then you can do some extra stuff

with rows as well so right would add a

row left would delete a row or sorry

column which is quite powerful and quite

helpful for sort of the general

manipulation stuff obviously it depends

on where the cursor is but yeah so

that's kind of the main workflow for

sort of manipulating the table is using

the arrow keys with different modifiers

you can obviously do the regular sort of

marking deleting and moving stuff around

as you'd expect now another thing when

it comes to modifying the different data

is that sometimes you want to copy paste

and so the way that this is done is

actually pretty interesting so you can

do control space here and kind of move

your cursor around and so you can do

kind of row and column selection like

this so while it does show that I'm

going across two lines I'm actually

selecting from this little rectangle

right here and so I can do crl c crl x

and then any of these Keys above us

let's do alt W and so as you can see it

said that I copied and so now if I do

contrl c crl xrl y it will paste that

data um and I can kind of do say for

example if I did here and went down to

here and did crl C CR X Alt W and copied

like we just did and then do crl c CR XR

y um we can copy and paste just the

column right here um so this can be very

helpful and very powerful if you guys

want to do some more like modifications

and moving things around without uh

disrupting the actual layout too much

now I'll admit I don't use this all the

time but it is helpful on the occasions

where I kind of feel like I need it and

now let's move on to generating tables

generating tables is actually a really

cool feature Within emac there's a bunch

of different ways to do it this is just

two of them so uh one of the ways is

using control and then the vertical bar

and so you'll see it right down here in

my like what I'm typing so you can kind

of turn anything that's comma separated

or tab separated into a table um so this

is really useful like I mentioned if I

just copy pasted say a list from a

website or something like that I can uh

select the region do contrl c vertical

bar and it will turn that into a table

now another example is using U Altex or

table

import and then I can just go to results

this is just an example CSV that I have

right here this was kind of a bad spot

to import it um let's undo that and

import it here so we'll just get the

same thing that we did last time we

import it and you as you can see right

here we imported uh the entire table and

we can kind of navigate it as you'd

expect and uh move things around as

you'd expect um yeah and so this is kind

of a nice way to kind of take like a CS

V or a tsv like I mentioned here tab

separated value and kind of import it as

you did hopefully okay now we can get

into the actual calculations which is

one of the really cool things about org

mode so there's a few different ways to

enter them but we'll go through them one

at a time so there's control equals

which will let you enter a calculation

for a column so right now uh this is

pretty arbitrary but let's go ahead and

just change this to 10 12 and then let's

make this column some sort of a

calculation so what we can do is we can

do contrl C

equals and then we can say that it is

that this column is column one plus 10

hit enter and it will apply that for the

column if we go and remove that cuz that

doesn't really matter there we go now we

get the sort of calculation that you'd

expect so each of these guys plus 10 and

so the dollar sign is how you specify a

row now you can also do an extra

calculation say if you want to do

control U contrl C equals then you can

make it specific to the field and say

let's just make this field equal to 10

minus

20 um which equals

-10 um and as you can see they're all

added down here into this table FM which

I think is short for table format is

kind of what they're going for and so

this is kind of used to do some

calculations and kind of just display

the results up here now you can kind of

uh recalculate them if you wanted to

modify them so let's say move minus this

to 10 and then we do contrl c contr c or

if you want to just do it specific to a

field you can do contrl c star which is

useful but really these sort of General

things I'm just showing off for the sake

of it we'll get into some more advanced

and actually useful uh ways to sort of

modify these values in just a second so

when we move on to references we

actually get some really Advanced

features and so this kind of makes the

actual calculations a bit nicer so as

you can see right down here I make a

small reference to the second row in the

second column remember that dollar sign

indicates that this is the column and

the at symbol indicates that this is a

row now there's a few different ways

that you can manipulate them as I

mentioned here you can do contrl c uh

quote single quote contrl C single quote

will give you the current calculation so

right now we have this set to calculate

to one but we can instead say that we

want to do plus and then let's say we

want to do at one uh dollar sign two and

so it will indicate where that is now

instead let's like go ahead and change

that to the right and then down and so

now it will be one plus that entry and

we can do contrl c contrl c to save that

and there we go and when we recalculate

it using contrl C control C you'll see

that it is computed now just like before

we can edit it once again and say maybe

we want to uh change this to a 10 or 101

I guess there we go we'll make it 10 and

then contr C contr C and then and we can

re-evaluate it I find this sort of a

workflow to be much better than the uh

one I talked about before um but it's

most effective when you're using it with

references I find now if you ever get

confused you can do a contrl c uh curly

brace this will kind of give you a bit

of a layout and kind of breakdown of

what you're looking at if you want to

know what it is for one of the field you

can do contrl c question mark and like I

said before contrl c contrl c aligns

things once again here's a little

breakdown of how references work so at

references a row the dollar sign

references a column if you want to

reference something say like very last

element in a range and you can use the

greater than symbol and then if you want

to specify a range you can use two dots

so here's a really good example now an

alternative notation instead of having

to control C quote or any of these other

things you can actually just do a colon

equals to enter in a calculation so here

we're saying that we want the second row

in the First Column so that's this guy

all the way to the third row in the

second column so we're selecting this

region right here all right and so if I

just hit tab boom that expands and we

get the actual calculation and as you

can see it's one right here if we go

through you'll notice that it lines up

with this one this guy this guy this guy

and we created a range from that and as

you can see that calculation ends up

down here so this is a really quick way

to just sort of write some quick

shorthand if you just want to do like a

quick calculation like we were talking

about before like 10 + 20 tab you get

the calculation kind of like what you'd

expect now what if we want to reference

other tables now this is a really

awesome feature I use this all the time

for the accounting stuff that I was

talking about before so let's say we

have uh some things and costs so we say

that car costs x amount of money house

costs x amount of money and so we could

have some calculations down here but in

this case we don't and then we can

actually reference it so here we use

remote to indicate that this is a

reference we're referencing costs which

comes from Up here we labeled this table

as costs and then we're saying that we

want the element right here all the way

to this element which uh really is only

two things that I could have said but I

created a range from it so second row

all the way to the third row both in the

second column and then I am using vs sum

to sum these results and as you can see

right here if I um remove that or let's

just remove that and recalculate it

you'll see that the results align and we

can sort of uh change this to

and recalculate and we get the updated

result now if you're wondering where uh

all of the powers of adding subtracting

and everything come from it's using a

thing in emac called Kelk um which you

guys can find in the info if I go ahead

and take a look at it it's a stack-based

calculator um it's very powerful I'm

pretty sure it even has like Taylor

series generation a bunch of different

stuff like generating Graphics as you

can see basically anything you see here

arithmetic it can do linear algebra um

it could do a lot of advanced math stuff

that you guys can put in your tables

obviously I don't take advantage of that

all the time um but from time to time it

can be super useful now when Kelk isn't

enough this is when I reach out to org

Babble org Babble is a way to sort of

execute code from within an org file I

haven't made a video on it but there's

lots of videos out there for those of

you interested um but there's pretty

cool stuff that you can do with it so

let's go ahead and do a really simple

example using a table so the really

simple one is just generating results so

here I have a source block using python

and I say return a uh list containing

the values from zero all the way up to

five and so if I hit control C control C

to execute it I'll get a range of 0 1 2

3 and four now we can also uh take data

and pass it in so for example I have a

table here I have named it note that I

didn't use the table name this is just a

generic name that we can use this is how

you can label basically everything in

org mode is using the hash plus name and

so we named it messages and they're from

and to somebody and then we actually

take that and as you can see right here

with VAR we're actually passing that in

as a variable called data and we're

actually using emac list this time not

just python um so for those of you guys

that don't have python installed very

impressive that you don't have installed

but uh you can actually do this with

just regular emac list that will work

everywhere um and so basically what I do

here is I iterate over that data which

we passed in up here and we go through

and we go through each row and we print

from Two and we get the first element

that's given to us which is Jim and the

second element which is John all right

and so if we go ahead and execute that

you'll see we get the results of from

Jim to John from Tim to Drake from Griff

to step uh questionable names there at

the end hey just coming in to interrupt

the video to talk about today's sponsor

brilliant brilliant is an online

learning platform when it comes to

learning a lot of us jump to things like

tutorials and lectures and really miss

out on being able to absorb everything

that we're being told this is where

brilliant really stands out it gives you

interactive real world examples that you

can think through and problem solve and

really retain the information that

you're learning in this video I show you

how you can use org tables and integrate

them with different programming

languages and interpreters if this is a

New Concept to you I highly recommend

taking a look at the computer science

fundamentals course from them that goes

a bit into the basic ideas of

programming that a lot of new

programmers can learn a lot from and you

can start applying what we talk about at

the end of this video in those Advanced

features as you go if furthering your

career knowledge or education interests

you I highly recommend giving a look at

brilliant to try everything brilliant

has to offer now for free for a full 30

days check out brilliant.org gaven

freeorn first 200 of you will get 20%

off brilliant's premium annual

subscription once again thanks to

brilliant for sponsoring this video okay

so let's think of a practical

application for those of us that do

programming so in this case let's say we

have a database and in this case I have

one that is an SQL light database

containing multiple restaurants so if I

do contrl c contrl c we'll see that we

can get the results like we were doing

before as a table now this is a little

messy so we can go sort of clean this up

by executing this which will actually

filter out by the important data now

this is quite good and we can kind of

like we were saying before manipulate

these uh different guys as you'd expect

so kind of adding a row removing a row

if we want to filter down to maybe just

like the location there you go now while

exporting these results is useful

sometimes you want to take them in as

input so let's go ahead and do that with

another SQL light database so we have a

table right here it says Who and the

task that they have we have two tasks

for gaven that's me and then some other

tasks for some other people now in this

case I'm going to uh set a variable and

we're going to set that variable to be

called some data and it's going to take

our table as we can see up here called

test table uh and we do some extra

settings I'm not really going to get dig

into those and we say that the DB is

just a temporary database that we'll be

using and we'll say that we're going to

drop this table if it already exists um

just for this Demo's purpose and then we

will import it and so now if we contrl C

control C on this we will see that it's

able to select who and task where the

who is Gavin and so as you can see right

here it selected just the ones that were

tasks for me now just to prove that I'm

not making this up we can go ahead and

use emac to actually open this and if if

we open up the database you can see that

it's all the original data that we

originally entered in you can do a lot

of this stuff with basically any SQL

light database um as well as any SQL in

general database um there's a bunch of

information both these links that you

can find in these show notes and finally

we'll move on to another Advanced

feature using the org Babel integration

um so this is a bit more of a breakdown

of what's going on here so in this case

we're going to use the r programming

language to do some mathematical stuff

do note that the ES s package is what's

used to give us our source code

functionality once you have the ESS

package installed you can also set up

the org Babble functionality for R by

just uh executing this eisp and here

let's go ahead and dig into it so here

we have a table which has each month of

the year and then what the temperature

was and so we can kind of plot this out

so here we're going to once again take

that table up there turn it into Data

we're going to say that our results will

be a graphical file we'll say that that

file is going to be a PNG um and that

way we will export it I think that's

what the exports was I can't actually

remember what I entered that part in for

we'll plot it and then we give it a

legend for the plot and we can hit crl c

contrl c and then you can use as I

mentioned down here contrl C CR x contrl

v to preview it and as you can see we

generated a very nice graph um to get

the data that we wanted I know that gnu

plot also can do this sort of stuff but

if you guys wanted to do statistics and

stuff um R is there and you can use

something like gnu octave if you wanted

to do uh linear algebra that maybe Kel

can't do um or anything like that

there's a lot of functionality out there

if this interest you guys uh be sure to

check out simx I think that's how they

expect you to pronounce it um which is

basically an emac setup meant for uh

scientists and Engineers um which is

really cool I've looked at that in the

past obviously for me it doesn't really

make sense because I'm so used to emac

but uh for those of you that aren't it

might be a very interesting starting

point so where does this functionality

leave us well in the end we have quite a

lot of power that we can use to automate

our lives do things like Financial

calculations daily life calculations

very advanced stuff for maybe generating

a graph or a plot if you guys want to

learn more about how I use it maybe how

I do some of my finances um and maybe

some more advanced features that you can

do with org mode and tables be sure to

let me know down in the comments now

before I let you all go I wanted to give

a big shout out to my supporters on

patreon and GitHub sponsors you guys

mean the world to me and you guys have

done so much to support the channel and

kind of keep keep things going uh in

between sponsors which I really

[Music]

appreciate