How Blurs & Filters Work - Computerphile

Summary

This video script delves into the concept of filters in image processing, contrasting common social media filters with professional techniques. It explains kernel convolution as the foundation for effects like Gaussian blurs and edge detection, demonstrating how different kernels can achieve various image transformations. The script also touches on the practical applications of these filters in both professional and everyday contexts.

Takeaways

• 🔍 The speaker distinguishes between the common understanding of 'filter' on social media platforms and the technical use of filters in image processing, emphasizing the latter's role in computer vision work.
• 📷 Filters in image processing involve low-level operations like Gaussian blurs, edge detection, and color adjustments, which are different from the aesthetic filters found on Instagram or Facebook.
• 👨‍🏫 The core technique discussed is 'kernel convolution,' which is fundamental to various image effects including blurring, sharpening, and edge detection.
• 👉 Kernel convolution is performed by overlaying a small grid of numbers (the kernel) over an image and transforming the image based on the values in the kernel, creating a new output image.
• 🌟 A kernel is typically smaller than the image and is positioned so that each pixel is at the center of the kernel for processing.
• 🔢 The process involves multiplying corresponding values from the image and the kernel, summing them up, and then normalizing by dividing by the kernel's total value to prevent brightness changes.
• 🌀 A mean blur is achieved by using a kernel where all values are equal, effectively creating an average of the surrounding pixels.
• 📏 Gaussian blur is a common and preferred method that uses a normal distribution to assign weights to the pixels, with more emphasis on the central pixel, thus preserving edges.
• 📉 The standard deviation in a Gaussian blur determines the spread of the bell curve, affecting the kernel size and the degree of blurring.
• 🖼️ The speaker mentions practical applications of these techniques, such as noise reduction in images before further processing or aesthetic adjustments in photography.
• 🌐 High-level filters seen on social media apps are essentially combinations of these low-level image processing techniques, creating effects like sepia tones or vintage looks.

Q & A

What is the general understanding of a filter in the context of social media apps like Instagram?

In social media apps, a filter is typically a combination of various low-level image processing techniques such as blurs, contrast changes, and color adjustments that alter the appearance of an image when applied.

What are some of the low-level image processing techniques mentioned in the script?

The script mentions Gaussian blurs, edge detection, and kernel convolution as some of the low-level image processing techniques used in day-to-day work.

What is kernel convolution and why is it significant in image processing?

Kernel convolution is a technique where a small grid of numbers, or a kernel, is passed over an image to transform it based on the values of the kernel. It is significant because it is the core of many effects like Gaussian blurs, mean blurs, and edge detection used in computer vision.

How does the size of the kernel affect the convolution process?

The size of the kernel determines the area of the image it influences. A smaller kernel will affect a smaller area around each pixel, while a larger kernel will cover a broader area, potentially affecting the image more significantly.

What is the purpose of normalizing the result of kernel convolution?

Normalizing the result of kernel convolution by dividing by the total value of the kernel ensures that the image does not get brighter or darker, maintaining the overall brightness of the image.

What is a mean blur and how does it differ from a Gaussian blur?

A mean blur is a type of kernel convolution where all values in the kernel are equal, resulting in a simple average of the pixel values within the kernel's area. It differs from a Gaussian blur, which uses a normal distribution to weight the pixel values, giving more importance to the central pixel and less to the surrounding ones, thus preserving edges better.

Why is the Gaussian blur considered more controlled and edge preserving than a mean blur?

The Gaussian blur is more controlled and edge preserving because it assigns higher weights to the central pixel and gradually reduces the weight for pixels further away, which helps to maintain sharp edges and avoid over-blurring.

What is the standard deviation in the context of a Gaussian blur and how does it affect the blur?

In the context of a Gaussian blur, the standard deviation determines the spread of the normal distribution used to weight the pixel values in the kernel. A larger standard deviation results in a wider bell curve, affecting a larger area of the image and creating a more diffused blur, while a smaller standard deviation results in a tighter curve, affecting a smaller area and creating a more focused blur.

How does the script describe the process of applying a kernel to an image?

The script describes the process as placing the kernel over each pixel of the image, with the pixel of interest in the center of the kernel. The values of the image and the kernel are multiplied together, summed up, and then normalized to produce the new pixel value.

What are some practical applications of using blurs in image processing mentioned in the script?

Some practical applications mentioned in the script include removing noise from images for further processing, improving the visual appearance of images by blurring them, and creating effects such as background blur in photographs.

How does the script address the issue of applying a kernel at the edge of an image?

The script suggests that at the edge of an image, one could ignore the pixels outside the image, wrap the image around, duplicate the edge pixels, or perform a smaller averaging for the corners, acknowledging that the decision depends on the specific requirements of the image processing task.

Outlines

00:00

🌌 Understanding Image Filters and Kernel Convolution

This paragraph explains the concept of image filters, particularly focusing on kernel convolution, which is a fundamental technique in image processing. The speaker clarifies that filters in apps like Instagram involve low-level processing techniques such as Gaussian blurs, edge detection, and color adjustments. The core idea is that filters process an image to produce an output, and kernel convolution is a method where a small grid of numbers (the kernel) is used to transform the image. By changing the numbers in the kernel, various effects like blurring, sharpening, or edge detection can be achieved. The explanation includes a step-by-step process of how kernel convolution works, emphasizing the importance of the kernel's size and values in determining the effect on the image. The paragraph also touches on practical applications, such as noise reduction and background blurring, and discusses the handling of image edges during convolution.

05:01

🔍 Exploring Gaussian Blur and Its Impact on Image Processing

In this paragraph, the focus shifts to Gaussian blur, a popular and commonly used filter in image processing. The speaker discusses the importance of the normal distribution in creating a Gaussian blur, explaining that the standard deviation of the distribution determines the width of the bell curve and thus the kernel's size and effect. The paragraph illustrates how a larger standard deviation results in a more extensive blur, while a smaller one prioritizes the central pixel. The speaker demonstrates the application of Gaussian blur on a test image, comparing it with a mean blur. The comparison highlights how Gaussian blur preserves edges better than mean blur by giving more weight to pixels closer to the center. The paragraph concludes by mentioning that complex filters seen in apps like Facebook or camera apps are essentially combinations of these low-level image processing techniques, such as grayscale conversion, blurring, and adding borders.

Keywords

💡Filter

In the context of the video, a 'filter' refers to a process used in image processing to alter the appearance of an image. This can include effects like blurring, sharpening, or color changes. The script mentions that filters in apps like Instagram involve low-level processing techniques. Filters are central to the video's theme as they are used to demonstrate the concept of kernel convolution and its applications in image processing.

💡Kernel Convolution

Kernel convolution is a fundamental technique in image processing that involves applying a small grid of numbers (the kernel) over an image to transform it. The script explains that this technique is the core of effects like Gaussian blurs and edge detection. It is used to demonstrate how different effects can be achieved by using various kernels, making it a key concept in understanding image processing.

💡Gaussian Blur

Gaussian blur is a type of image smoothing that uses a Gaussian function to reduce image noise and detail. The script describes it as a common and preferred blur due to its edge-preserving properties. It is achieved by using a kernel that follows a normal distribution, where the weights decrease as you move away from the center of the kernel, thus prioritizing the central pixel.

💡Mean Blur

Mean blur, as discussed in the script, is a simple form of image blurring where each pixel in the output image is the average of the pixels under the kernel. The script uses a 3x3 kernel with all values set to 1 to demonstrate this effect, showing that it results in an even distribution of the image's values, which can be used to remove noise.

💡Edge Detection

Edge detection is a process in image processing used to identify and highlight boundaries between different regions in an image. The script mentions that kernel convolution can be used for edge detection by applying a specific type of kernel. This is an important application of image processing that helps in tasks like object recognition and feature extraction.

💡Kernel

In the script, a 'kernel' is defined as a small grid of numbers used in kernel convolution to process an image. The values in the kernel determine the effect applied to the image, such as blurring or sharpening. Different kernels can be used to achieve various visual effects, making the kernel a crucial element in image processing.

💡Normalization

Normalization in the context of kernel convolution refers to the process of scaling the output so that the image does not become brighter or darker. The script explains that after multiplying the kernel values with the corresponding image values and summing them, the result is divided by the total value of the kernel to ensure normalization.

💡Standard Deviation

The script discusses standard deviation in relation to Gaussian blur, where it determines the spread of the bell curve in the normal distribution. A larger standard deviation results in a wider kernel, affecting how the blur is applied across the image. It is a key parameter in controlling the intensity of the blur effect.

💡Weighted Average

A weighted average, as mentioned in the script, is a type of average where each value in the set is multiplied by a weight. In the context of kernel convolution, this is how the effect of each pixel under the kernel is calculated, with the weights determined by the kernel values. This concept is crucial for understanding how different effects like blurring are achieved.

💡Image Processing

Image processing is the overall theme of the video, encompassing techniques like blurring, sharpening, and color adjustments. The script delves into how these effects are achieved through low-level processing techniques like kernel convolution. It is the broader context in which the specific techniques discussed in the script are applied.

💡Noise Reduction

Noise reduction is a common application of image processing filters, as mentioned in the script. Blurs, such as mean and Gaussian blurs, are used to remove noise from images, making them appear smoother. This is particularly useful in preparing images for further analysis or to improve visual quality.

Highlights

A filter in image processing is different from social media app filters.

Filters involve low-level processing like blurs, contrast changes, and color changes.

Kernel convolution is a core technique in image processing.

Kernel convolution uses a grid of numbers to transform an image.

Different kernels can perform various effects like blurring, edge detection, or sharpening.

The kernel is generally smaller than the image and is placed over each pixel.

Kernel convolution involves multiplying corresponding image and kernel values and summing them up.

Normalization is done by dividing by the total value of the kernel.

A mean blur is achieved when all values in the kernel are 1.

Mean blurs are used to remove noise and can be found in image processing software.

Gaussian blur is a common and controlled type of blur that preserves edges.

Gaussian blur uses a normal distribution to weight the kernel values.

A small Gaussian kernel might be 3x3 with values like 4, 2 on the edges, and 1.

Gaussian blur prioritizes the pixel of interest and reduces weight as you move away.

The size of the Gaussian kernel increases with the standard deviation.

Edge handling in convolution can involve ignoring edge pixels or wrapping the image.

Complex filters like bloom, sepia, or vintage effects are combinations of low-level image processing techniques.

Transcripts

What I would understand to be a filter is perhaps slightly different from what people who use Instagram would describe as a filter.

Usually in an app, or a camera phone app, or Facebook, or some other thing where you can apply some filter,

it's gonna actually be a combination of lots of low level processing,

of various types you know, blurs, contrast changes, colour changes.

What I tend to use in my day to day work would be the low level things, Gaussian blurs, edge detection, this kind of stuff.

But really they're all filters and they all take some image, process it and come up with some output.

Today we're gonna look at a simple technique called kernel convolution,

and that is kind of the core of Gaussian blurs and mean blurs and edge detection and lots of other things.

And it's a fairly simple technique that we use a lot around here with computer vision work.

Kernel convolution is just a process where we take a small grid of numbers and we pass them over the whole image,

transforming it based on what those numbers are.

And by using different numbers in the kernel, we can perform blurs, or edge detection or sharpen, unsharpen,

basically any effect we like.

So, I'll first describe kernel convolution and then we'll look at a couple of examples of what sort of kernels that we see a lot of.

Kernel convolution works by, if this is our test image, it's 5x5 and this is our kernel which is 3x3.

So, generally speaking the kernel will be smaller than the image, and usually actually quite small.

And what we do, this is my movable kernel that we've come up with.

What we do, basically, is for every pixel in our image, we put our kernel over it, so that the pixel is in the centre.

So let's look at this pixel here, number 64, we put our kernel over it

and then we are looking at the 3x3 grid, centred around that pixel.

And we take whatever value is in our kernel,

multiply it by 17, and then the top value, multiply it by 14, and the top corner value, multiply it by 13.

So we take each corresponding image value and kernel value, we multiply them together, in pairs, and we sum the whole thing up.

And then finally, we normalise by dividing by the total value of our kernel to make sure that it doesn't get brighter or darker.

So it's a bit like averaging like we did in...

It's exactly, it is averaging, yep.

In a way, it's just a big weighted average, or if all the numbers are same, an actual average.

So, in fact that's the first one we can look at.

So, if all of our values in our kernel are 1, then that's a mean blur.

So, if you go into Adobe Photoshop, or any other image processing package, and you go for blurs,

blur filters, and mean, that's what it's gonna be doing.

This is a kernel of size 3.

Usually we'll do it, obviously, centred around a single pixel, so, odd sizes, so, 3, 5, 7 and so on.

So, what we're gonna do here , is we're going to go

17 multiplied by 1, plus 14 multiplied by 1, and so on, and divide the whole thing, by the sum of this, which is 9, and that will basically take an average.

And then we'll overwrite that pixel with the output of that kernel convolution.

The only thing to make clear is that we should output these to a different image, because if we overwrite them as we go, it's gonna mess up the maths as we go down.

I use blurs to remove noise from images before I process them in other ways for my job.

Some other people might try and remove noise from images just to make them look better, or you might try and blur the background out in a photograph or something like that.

And you can achieve a lot of effects just by convolving a kernel over an image.

You move it around for every pixel and when you look at whatever window, that is the size of your kernel.

The only other thing is, that if you're at the edge, then you've gotta make some decision as to what you do here.

So you could ignore those ones and they wouldn't contribute at all, or you could wrap the image around, or you could duplicate the edge ones, you could do lots of different things.

Generally speaking, I would just ignore them, and then do a slightly smaller averaging for the corners.

That'll mean that the blurring around the edges is ever so slightly less than the blurring elsewhere in the image.

But, just in the last edge pixel in a 5 to 10 megapixel image, probably won't make much of a difference.

So that's a very very simple kernel, if we go for a slightly more complicated kernel, we can go for the normal distribution of the Gaussian blur.

So Gaussian blur is extremely common, probably, I suppose the most common blur.

It's a little bit more controlled and edge preserving than a mean blur and so a lot of people like it.

So a normal distribution is a bell curve.

So, if we have our axes there and we draw a bell curve, and the standard deviation of this curve,

So, a standard deviation is essentially the average distance from the mean of all the points,

will determine how wide this bell curve is. So a really large standard deviation will have a really large bell curve.

OK, it should be symmetrical.

And a really small standard deviation will have a really tight bell curve, which is essentially really prioritizing the weights in the middle.

Now, if we weight these numbers based on the values from our normal distribution then, we get a Gaussian blur.

An example of a very small Gaussian kernel would be a 3x3.

So these values are gonna be floating point decimal values, but we just simplify it a little bit.

So 4, 2 on the edges and 1.

Now the crucial difference between this and the mean blur is that we're really prioritising those in the middle.

The further away you get from the pixel of interest, the less weight you have in the combined average.

And that's important because it basically means that you're not gonna be blurring too much. Whereas an edge, let's say a sharp change in intensity,

as you approach that edge, you're not gonna be taking too many of those pixels.

And this Gaussian blur can obviously get a lot bigger.

So, in your image processing program, you might have an option that says standard deviation or radius,

And what that's really referring to, is the standard deviation of the normal distribution that produced this kernel.

And also the width of the kernel.

So, we have to increase the size of our kernel, as the radius of the Gaussian function increases.

This is a very small normal distribution. If this was a bell curve, it's really steep.

If we had a bigger standard deviation, then we're gonna need a bigger window to be able to hold enough information.

This isn't near big enough.

So let me draw a test image and then we'll do a blur on it and we'll see how it works.

So, okay, so this is a good sized image. This image is pretty basic, it's an edge.

So we've got an area of 50 intensity and an area of 100 intensity.

So if we just put our mean one here as well just for comparison.

So if we do our mean first, so we put that on let's say this 50 here which is a lot on the edge.

Okay, so we put our kernel on there and we're gonna be doing

50 times 1, plus, 50 times 1, plus 100 times 1 and so on.

So it's going to be *counting* 600.

And then we divide that by nine,

and that gives us 66.6.

Okay, and we'll round it to the nearest integer value.

For a gaussian blur, with this gaussian blur here, we're gonna be doing

50 times 1, plus, 50 times 2, plus, 100 times 1 and so on, okay.

And that will be a value of, this is where we cut...

So, it's 1000, these add up to 1000 divided by 16, which is 62.5.

So, ever so slightly closer to these 50 values on this edge than perhaps the mean blur was.

It's subtle, but the edge has been slightly preserved and generally speaking we're giving more weight to the numbers really really close to us.

These are very low level filters, these are sort of filters I will use in my day

day to day work to alter images that I need to analyse.

But, on Facebook, and you know, on all these camera apps for your phone,

You'll see a lot of other, perhaps more complicated filters, bloom, you know, sepia,

vintage ones that make it look like some camera from the 1800 or something like that.

And really what they're doing is nothing hugely complicated. The're just chaining together low level image processing.

So, a grayscale filter, maybe a bit of blur, and then some kind of border to make it look good and things like this.

So, really, these kinds of filters are at the core of a lot of the stuff that you see.

Even of you might not know it.