FFmpeg Filtergraph

Summary

The video script provides an in-depth explanation of FFmpeg filtergraphs, a concept often confusing for developers. The presenter introduces basic FFmpeg commands, focusing on how filtergraphs are used to manipulate video streams, including scaling and concatenating. The script emphasizes the importance of stream order, options formatting, and handling multiple inputs and outputs. Through examples, the presenter highlights the differences between video and audio stream handling, how to avoid common mistakes like dangling streams, and introduces advanced concepts like filter chaining for continuous stream editing.

Takeaways

• 🔧 FFmpeg's filtergraph can be confusing, especially for new users, but it is key to applying filters to media streams.
• 📊 The most common FFmpeg command example involves scaling a video to a specified resolution (e.g., 1280x720) using the `-vf` option.
• 📁 `-i` stands for input, followed by a file path, while `-vf` refers to the video filter applied to the video stream.
• ⚙️ Filter parameters in FFmpeg require an equals sign (`=`), and multiple parameters are separated by colons (`:`).
• 🔀 Filters can accept multiple options, which can be reordered or skipped, but named options improve clarity.
• 🔗 The `-filter_complex` option works across all streams (video, audio, subtitles) and requires more explicit input/output stream naming.
• 🧩 Streams in FFmpeg filtergraphs are enclosed in square brackets and must be defined before use as inputs; outputs create new streams.
• 🎞️ Filters can have multiple inputs (e.g., overlaying or concatenating videos) and outputs (e.g., duplicating streams using the `split` filter).
• 🔄 Streams need to be fully utilized; dangling streams without a clear output will prevent FFmpeg from running.
• 🔊 Filters for video and audio are handled separately, requiring duplication of the filter logic for each stream type, often separated by a semicolon.

Q & A

What is the FFmpeg filtergraph, and why is it important?

The FFmpeg filtergraph is a series of filters applied to video or audio streams to manipulate them. It’s important because it allows users to modify streams, like scaling video or applying effects, in a powerful and flexible way.

What does the '-i' option in FFmpeg do?

The '-i' option specifies the input file for FFmpeg. It is followed by the file path of the media you want to process.

What is the purpose of the '-vf' option in FFmpeg?

The '-vf' option stands for 'video filter'. It allows the application of a filtergraph to video streams in the input file.

How do filter options work in FFmpeg?

Filter options in FFmpeg are provided after the filter name, using an equals sign to assign values to filter parameters. If the filter has multiple options, they can be separated by a colon.

What is the difference between '-vf' and '-filter_complex' in FFmpeg?

'-vf' only applies to video streams, whereas '-filter_complex' can be used for any type of stream, including audio and subtitles. It allows more complex filter graphs with multiple inputs and outputs.

What does the 'scale' filter in FFmpeg do?

The 'scale' filter resizes a video to specified dimensions. For example, 'scale=1280:720' changes the video resolution to 1280x720 pixels.

How are input and output streams handled in FFmpeg filtergraphs?

In FFmpeg, streams (input or output) are encapsulated in square brackets. Input streams must already exist, while output streams are implicitly created when named.

What happens if a stream is left 'dangling' in FFmpeg?

If a stream is output by a filter but isn’t used or assigned to anything, it is considered 'dangling'. FFmpeg will flag this as an error and the filtergraph won’t run.

What is 'chaining' in FFmpeg filtergraphs?

Chaining refers to applying multiple filters consecutively to the same stream. This is done by adding another filter without separating them with a semicolon.

Why might someone use the 'split' filter in FFmpeg?

The 'split' filter duplicates a stream, creating multiple copies of it. This can be useful for applying different filters to the same stream or for creating different outputs from the same input.

Outlines

00:00

🎥 Understanding FFmpeg Filtergraph Basics

This paragraph introduces the concept of FFmpeg filtergraphs, noting that it is a tricky subject for developers. The speaker emphasizes the practical use of color-blind friendly slides and code-friendly fonts. A common example of FFmpeg usage—scaling a video to 1280x720—is used to explain basic syntax. The -i flag represents input, -vf denotes video filter, and options like width and height are defined with an equals sign and separated by colons. The paragraph also notes how multiple FFmpeg commands can achieve the same result using variations in the filter options.

05:03

🔄 Handling Filter Options and Stream Complexity

This section delves deeper into filter options, explaining that while you can skip option names when the order is default, having multiple options requires specifying names. The flexibility of using names in any order, or mixing named and unnamed options, is discussed. The introduction of -filter_complex allows operations on multiple streams (video, audio, subtitles), and the explanation becomes more complex with the need to be explicit about input and output streams. The structure of filtergraphs is explained, focusing on input/output streams as variables. The paragraph also explains how naming inputs/outputs creates new streams and highlights the possibility of multiple inputs and outputs using filters like overlay and split.

Keywords

💡FFmpeg

FFmpeg is a powerful multimedia framework used for video, audio, and other multimedia file processing. It provides a wide variety of functionalities, including encoding, decoding, and filtering. In the video, FFmpeg serves as the core tool being taught to developers, with a focus on its unique filtergraph system and command-line usage.

💡Filtergraph

A filtergraph in FFmpeg refers to a sequence of filters connected by inputs and outputs, allowing for complex media transformations. In the video, filtergraphs are emphasized as a crucial concept because they enable the manipulation of video streams, such as scaling, concatenation, and overlaying.

💡-vf

-vf stands for 'video filter' in FFmpeg commands and is used to apply a filtergraph to a video stream. The video discusses this as a basic way to use FFmpeg, highlighting that this option only affects video streams, unlike more advanced options like -filter_complex.

💡Filter options

Filter options are parameters provided to filters in FFmpeg commands. These options modify the behavior of the filter and are delimited by colons, with each option being defined as a key-value pair. The video shows examples of how filter options are used to set video width and height, and explains how they can be skipped or ordered differently.

💡Stream

In FFmpeg, a stream is a sequence of video, audio, or subtitle data within a multimedia file. The video explains how streams are treated as variables within filtergraphs and are always encapsulated by square brackets. Each stream can be manipulated individually, and their order matters for operations like concatenation.

💡Scale filter

The scale filter in FFmpeg resizes a video stream to a specific width and height. In the video, the speaker introduces the scale filter early on as a simple example, showing how it can be used to resize a video to 1280x720 resolution. This filter is a common starting point for learning FFmpeg.

💡Concatenation

Concatenation in FFmpeg refers to merging multiple video or audio streams into one. The video provides an example of concatenating three video streams into one, explaining the importance of stream indexing and how changing the order of indexes changes the final output.

💡Filter_complex

-filter_complex is an FFmpeg option used to apply filtergraphs to multiple streams (video, audio, and subtitle). Unlike -vf, it can handle complex filtering scenarios, such as overlaying or concatenating streams. The video emphasizes the need for this option when multiple types of streams are being processed together.

💡Semicolon (;) in filtergraph

A semicolon in a filtergraph is used to separate filters that act on different streams. In the video, the speaker explains that semicolons are necessary when addressing multiple streams (e.g., video and audio) separately in a filtergraph. This ensures each filter operates on its intended stream.

💡Chaining

Chaining refers to applying multiple filters in sequence to the same stream within FFmpeg. The video introduces this concept as an advanced filtering technique, where one filter's output becomes the input for the next filter. This is useful when performing multiple edits, such as scaling and cropping, on the same video stream.

Highlights

FFmpeg filtergraph is often the most confusing part for developers.

Apology for using bright colors in slides to ensure colorblind compatibility.

Ensuring fonts distinguish between similar characters like 1, i, and l in code sections.

Explanation of basic FFmpeg command: -i for input, -vf for video filter, and scaling video to 1280x720.

FFmpeg filters use equal signs to assign options, and options are delimited by colons.

Commands with alternative formats yield the same results despite varying syntax.

-filter_complex applies to all streams, not just video, unlike -vf which is video-only.

Stream names are encapsulated by square brackets in FFmpeg to identify inputs and outputs.

Filters can have multiple inputs and outputs, like overlaying images or using the split filter.

The order of files in the filter matters, and changing it alters the final output.

Stream variables are either created or used when addressed in FFmpeg.

A dangling stream (unused output) causes FFmpeg to prevent the filter graph from running.

FFmpeg supports both video (v), audio (a), and subtitle (s) stream types.

Chaining filters in FFmpeg allows adding multiple filters to a stream without separation.

The use of semicolons separates distinct filters, while chaining keeps modifying the same stream.

Transcripts

The FFmpeg filtergraph

I taught this to multiple groups of devs in my  

previous startups and this part  was usually the most confusing,  

it could be because it’s the first thing that’s  really ffmpeg-specific in what we’re teaching here

I’ll try to incorporate as  many examples as possible  

and I’m leaving extra slides for questions

First of all an apology for the weird  bright colors in my presentation

I test my slide for color blind compatibility  so it's certainly not an aesthetic choice  

it's about being practical and inclusive

I also try to make sure that I use fonts that  distinguish 1 from i and l in all the cases  

where I'm using code and if i missed  it anywhere please send me a message.

Back to the topic, the most common example  you’d usually find for FFmpeg usage is a  

single command using a single  filter, something like this -

This example which basically just scales your  video to 1280 by 720 and even though I made  

this one up, I’m sure that there are plenty of  people who had this as the google search result

So obviously, the -i stands for input

it’s followed by a file path

-vf stands for video filter and  is followed by a filtergraph

this entire thing

and this filter,

this specific filtergraph only applies  to video streams in the previous,  

in the provided inputs that's  the what the -v stands for

Scale is the filter name and  you see equals 1280 by 720

whenever you’re providing a filter with  parameters or “options” as they’re called  

in the documentation you need to add the equals  sign, and the options are delimited with a colon

What you see here are 4 more ffmpeg commands  that are alternatives to the one we just saw 

and all of these commands not only have the  same results, they do the exact same thing

Filter options have a default order,  

meaning you can skip the option name  (as we did in the previous slide)

no option name anywhere

But if the filter you’re using has  more than one option, those options  

always have a name, which is what  we’re seeing in the first variant

w for width and h for height

In the second variant, we can see that you  can use any order of options when using names

Or as we see in the third option, mixing  

is also possible though I don’t recommend  it because it can be confusing or unclear

Lastly, instead of using -vf, which works only  on video streams, I’m using -filter_complex,  

which works on every stream

See that now I have this additions, this is  

because when we're using the -filter  _complex we have to be more explicit

It can run on every stream so these are  the input stream and the output stream

The stream names are always  encapsulated by square brackets

I know this suddenly just got a  bit more confusing, bear with me

this mock of an example is  still just a single filter

the rules are very simple and I'm going  to read them out loud - I know it helps

each filter in the graph is  compromised of an input block in blue  

the filter itself and the outer block in red

these inputs and outputs are always streams

these are our so-called variables  and it's really important that you  

think of them as streams because  that's how they actually work

our variables or streams are always  encapsulated by square brackets (imagine this)

and when you address a variable  as an input it must already exist  

but when you name your output  you're implicitly creating it

meaning that it shouldn't already  exist, if it already exists and wasn't  

used I'll elaborate on that later then the  program will shout at you and wouldn't run

filters can have multiple inputs, for example,  overlaying an image on a video or concatenating  

multiple streams those are examples for  filters that would have more than one input

and they can also have multiple  outputs for example - the split filter

lets you duplicate the stream

it's a lot more useful than you might think

and that would have multiple outputs

when you provide your filter with  options you have to add the equal sign

those options should also be a pair of option  name and value delimited by an equal sign  

and those pairs should be delimited by a colon

so in this example what we're doing  is concatenating three video streams  

into one video stream and then outputting  that stream into a new video file

you can see that our files are  addressed by the stream index

the order matters here

you see i have this file and this file this  this file the order in which I named them  

is going to assign their index later

if I change the indexes around  here I would have basically just  

change the order of the concatenated output

`v` stands for video,obviously

and the other possibilities are  `a` for audio and `s` for subtitles

you might have noticed in the  map section that I'm implying  

you can have multiple outputs in one command

that's because you can

but it's not a very common thing to do and I  don't think we should touch on that in this lesson

the other thing is that you can't  have a dangling stream, I mean you  

if you output a string with one of your filters  and you aren't doing anything with it - dangling

ffmpeg will think you made  a mistake probably forgot  

about it and will prevent that graph  from running it has to have a purpose

last thing that we should fix here is that  we don't have any audio, so let's do that

It wasn't so scary, 

so basically you just duplicated our  filter and we are now addressing the audio

because these two filters are completely  separate in the way that they work,  

we need to separate them and  that's why this semicolon is here

There's also an option not to use a semicolon when  you want to keep on editing the same stream over  

and over adding another filter, and another filter  - that's called chaining and also our next slide