Some New Features in Python 3.11

Summary

This video discusses new features in Python 3.11, including enhanced tracebacks for better error debugging, improvements in handling asynchronous tasks with task groups, and the introduction of exception groups for managing multiple exceptions simultaneously. The presenter demonstrates these updates with code examples, highlighting how Python 3.11 makes debugging and handling asynchronous processes more efficient. Additional features like performance improvements and updates to Python's typing module are briefly mentioned. Viewers are encouraged to explore these updates and share their thoughts on the usefulness of these new additions.

Takeaways

• 😀 Python 3.11 introduces improved tracebacks, offering more precise error location details compared to Python 3.10.
• 🔍 With Python 3.11, you can easily spot where a ZeroDivisionError occurs in a complex expression, saving debugging time.
• 🚀 The new Task Groups in async programming allow asynchronous tasks to be grouped and executed immediately, unlike the old gather method which waits for task completion.
• 🛠️ Task Groups don't require gathering results explicitly, making them ideal for cases where the result doesn't need to be returned.
• 📊 If results from tasks are needed, using asyncio.gather is more straightforward, but Task Groups offer a simpler syntax for non-result-driven tasks.
• 🔗 Python 3.11 introduces exception groups, allowing multiple exceptions to be raised and handled together, which is especially useful for asynchronous programming.
• ⚡ With the new `accept*` syntax, multiple exceptions within a group can be handled, unlike the traditional method that stops after the first match.
• 📝 Exception notes are a new feature that lets you attach additional information to exceptions, providing more context during debugging.
• 💡 The script demonstrates how to use exception groups and notes by showing how to manage multiple failed URL requests with informative error handling.
• ⏩ Python 3.11 includes performance improvements, making it faster globally without requiring changes to existing code, along with enhanced type hints.

Q & A

What is the main focus of the video?

The video covers new features introduced in Python 3.11, focusing on improved tracebacks, task groups, exception groups, and performance improvements.

How has the traceback feature improved in Python 3.11?

In Python 3.11, the traceback provides more precise information about where an error occurs, highlighting the exact part of the expression causing the issue, such as showing which variable is causing a zero division error.

What are task groups in Python 3.11, and how do they differ from using 'asyncio.gather'?

Task groups are introduced in Python 3.11 as a more efficient way to manage asynchronous tasks. Unlike 'asyncio.gather', task groups start tasks immediately without needing to create a list of tasks, and the 'async with' block does not return until all tasks are completed.

Can you get the results of tasks in a task group? How?

Yes, you can get the results of tasks in a task group by appending each task to a results list and later accessing the results using a list comprehension. This approach gives you similar functionality to 'asyncio.gather'.

What are exception groups, and when would you use them?

Exception groups allow multiple exceptions to be raised and handled at the same time, which is particularly useful in asynchronous programming where several tasks can fail for different reasons. They enable you to manage multiple exceptions in a single block.

How do 'accept *' blocks work in Python 3.11?

'Accept *' blocks in Python 3.11 allow multiple exception handling blocks to run for different types of errors within the same try-except statement. Each applicable exception is handled separately without exiting after the first match, unlike traditional except blocks.

What is the purpose of the 'add_note' method in Python 3.11?

The 'add_note' method allows developers to attach additional information to exceptions without needing to create custom exceptions. For example, you can add extra details like URLs or context-specific notes to the exception for debugging purposes.

What other features are included in Python 3.11 besides task groups and exception groups?

In addition to task groups and exception groups, Python 3.11 brings speed improvements, enhancements to the typing module, and other performance optimizations that benefit developers without requiring code changes.

Are there performance improvements in Python 3.11, and do they require code changes?

Yes, Python 3.11 includes speed improvements, but these enhancements do not require code changes. Developers can continue writing code as usual and benefit from the performance boost automatically.

Why might a developer choose to use task groups instead of 'asyncio.gather'?

A developer might choose task groups over 'asyncio.gather' because task groups start tasks immediately, simplifying code by removing the need for creating and appending tasks to a list. They are particularly useful when task results are not needed.

Outlines

00:00

🐍 Python 3.11: Enhanced Tracebacks

In this paragraph, the video introduces a new feature in Python 3.11: improved tracebacks. The speaker compares the error reporting between Python 3.10 and 3.11 by running a piece of code that divides numbers, triggering a zero division error. In Python 3.11, the traceback is more precise, highlighting the exact part of the expression causing the error. This new feature improves debugging efficiency by pinpointing where the issue occurs, saving developers time when identifying the cause of an error.

05:01

⚡ Running Asynchronous Tasks with Task Groups

This section discusses two methods for running asynchronous tasks in Python: the traditional `asyncio.gather` approach and the new `asyncio.TaskGroup` introduced in Python 3.11. The speaker walks through an example where tasks are executed concurrently. The `TaskGroup` method starts tasks immediately when added, eliminating the need for task lists or explicit `gather` calls. While `gather` makes it easier to return results, `TaskGroup` provides a cleaner way to run multiple tasks, especially when the results are not needed. This new method simplifies managing async operations in Python.

10:02

💥 Managing Multiple Exceptions with Exception Groups

In this part of the video, the speaker introduces Python 3.11's support for handling multiple exceptions with 'Exception Groups.' This feature is particularly useful in asynchronous programming, where many operations can fail for different reasons. The example demonstrates how to catch and handle multiple exceptions simultaneously using `ExceptionGroup`, as well as how the new `except*` syntax allows handling several exceptions in one try-except block. The speaker also introduces 'Exception Notes,' which lets developers add additional context to exceptions for better error diagnostics.

⚙️ Python 3.11 Performance Improvements and Other Features

The final part discusses general improvements in Python 3.11, including performance boosts that speed up code execution without requiring any special modifications by developers. While the speaker acknowledges that they couldn't demonstrate the performance gains due to the nature of their code, they mention that Python 3.11 will deliver noticeable performance increases. Additionally, the speaker briefly touches on improvements in the typing module, which makes using type hints easier and more powerful for developers. The video ends by encouraging viewers to share their opinions on Python 3.11's new features.

Keywords

💡Python 3.11

Python 3.11 is the latest version of the Python programming language at the time of the video's recording. This version introduces several new features and improvements, including better error tracebacks, enhanced support for asynchronous programming, and overall performance boosts. The speaker demonstrates these features to showcase the version's efficiency and debugging advantages.

💡Traceback

Traceback refers to the error output in Python that shows where an error occurred in the code, which helps in debugging. Python 3.11 offers more detailed tracebacks, indicating the exact location of errors, such as a division by zero, making it easier for programmers to identify and resolve issues without scanning all potential problem areas manually.

💡ZeroDivisionError

A ZeroDivisionError occurs in Python when a division operation has a divisor of zero, which is undefined in mathematics. In the video, this error is used to show how Python 3.11 provides more detailed tracebacks, pinpointing the specific location in an expression where the division by zero occurs, thus making debugging more efficient.

💡Asynchronous Programming

Asynchronous programming, or async, allows tasks to be run concurrently without waiting for each to finish before starting the next. The speaker discusses enhancements in Python 3.11, including the new 'TaskGroup' feature, which simplifies running and managing asynchronous tasks compared to the previous 'gather' approach. This is beneficial in scenarios where tasks need to be run independently, like multiple API requests or parallel computations.

💡TaskGroup

TaskGroup is a new class in Python 3.11 that enables the grouping of asynchronous tasks in a simpler and more organized way. Unlike traditional 'gather' methods, TaskGroup automatically initiates tasks and waits for all to complete without the need for a separate task list. This feature improves code readability and reduces setup requirements for asynchronous operations.

💡asyncio.gather

'asyncio.gather' is a method used in Python for managing multiple asynchronous tasks, running them concurrently, and waiting for all of them to complete. In the video, 'asyncio.gather' is contrasted with the new TaskGroup approach, with the former requiring a task list and the latter handling tasks more dynamically, thereby simplifying async management in Python 3.11.

💡Exception Groups

Exception groups allow handling multiple exceptions simultaneously within Python, which is particularly useful for asynchronous programming where various tasks might fail for different reasons. In Python 3.11, this feature enables programmers to group related exceptions together, making it easier to handle complex error scenarios by processing them all at once.

💡Exception Notes

Exception notes in Python 3.11 allow developers to add custom notes or messages to exceptions. This can provide extra context about the error, such as the URL that caused a failure. Exception notes help developers understand the specifics of an error without having to create custom exception classes, simplifying error handling.

💡Performance Improvements

Python 3.11 includes general performance improvements that make it faster than previous versions. These optimizations benefit Python programs without requiring any changes to the code, and though the speaker doesn't give specific metrics, they note that all Python scripts will see an automatic boost in speed, particularly in computation-heavy tasks.

💡Typing Module

The typing module in Python provides support for type hints, enabling developers to specify data types in code, which helps with readability and error detection. Python 3.11 introduces enhancements in the typing module, making it easier to define and work with types, which is especially beneficial for large codebases and team projects that require clear documentation and structure.

Highlights

Introduction to Python 3.11's new features, available now and recently released.

Improved tracebacks in Python 3.11, offering more precise error locations for better debugging.

Example of improved traceback: detecting exact zero division location in a complex expression.

Explanation of new async features: creating groups of asynchronous tasks more efficiently.

Introduction to the 'task group' feature in async IO, which simplifies task grouping and management.

Comparison between async IO gather and the new task group; task group eliminates the need to store tasks in a list.

Illustration of running async tasks using 'task group' for automatic task initiation and grouping.

Drawback of task groups: cannot retrieve results directly, unlike with async IO gather.

Solution for retrieving results when using task groups, by appending tasks to a results list.

Introduction to exception groups: managing multiple exceptions simultaneously, useful for async programming.

Demonstration of exception groups, using a URL loading function where multiple requests may fail.

Using 'accept star' to handle multiple exceptions in one block, which processes all applicable exceptions.

New feature: adding exception notes to exceptions for additional context, e.g., URL causing an error.

Overview of Python 3.11's general performance improvements, benefiting all Python code without additional setup.

Mention of new typing module enhancements in Python 3.11, which facilitate using type hints.

Transcripts

hey everyone in today's video I want to

go over some of the new features of

python 3.11 that I found interesting

it's available now it was released a

couple of days ago at the time of

recording this video so you'll be able

to do everything that I'm showing in

this video today

the first feature has to do with

improved tracebacks so here I have some

code it basically just assigns the

letters a to J to the numbers one

through ten and then it divides all of

these here just like you see and it

prints out the result so let me go ahead

and run this just so you can see what it

looks like and it gives me a really

small number now if I were to change

this to have a zero somewhere let's say

five and I change that to zero and I run

this using python 3.10

we see I get this error message right so

it's telling me that there's a zero

division error somewhere in this

expression I'm dividing by zero but it's

not clear where now with python 3.11 if

I run this so python 3.11 is my default

now so I still see the zero division

error but it's giving me more precise

information on where the error is

occurring and we see it's happening

after the D

and before the E so the division there

so I'm trying to divide by zero there so

it's saying that e is zero so one two

three four five and one two three four

five yes so e is zero and that's the

part that's failing if I change this

back to five

and I change let's say the 9 to 0 which

means the I in this case will be zero

and run it again we see now this little

arrow is pointing to the division sign

right before the I which means the I

itself is zero this is going to save you

time when you're debugging because it's

going to show you exactly where

something is occurring so you won't have

to kind of look at everything in the

area you'll have a better idea of where

to start your investigation when you're

debugging so I think this is a great

addition to python 3.11 the next thing I

want to talk about are different ways of

creating groups of asynchronous tasks

so right now I have this example of the

old way of doing it or you know the the

standard way of doing it here I'm

here I have this async function which

will basically print start and end and

return finish along with some number

that I pass in and it's just going to

sleep for however many seconds I pass in

so I want to run this multiple times in

a group so the typical way you do this

is with gather so you create a list of

tasks so I have this task list and I

just create a bunch of tasks from one to

five well really one to four and they're

calling this async function that I've

defined up there and then I'm adding

them to this task list so I have the

list of tasks and then I await

um this list of tasks by using gather so

then gather will run all these tasks at

the same time and it will wait for them

all to finish and when it finishes then

it Returns the results from the entire

group of them so this is fine I've used

this before of course you can change

this user list comprehension but the

idea is the same you just basically

create a list of tasks and then you

await those tasks using gather so now

there's a new way of doing it and I'm

going to copy this one and I'm going to

modify it to support the new approach so

I'm going to call this run with groups

so let's change this to groups I can get

rid of the tasks here because I no

longer need a tasks list and what I want

to do instead is I want to use async

with

and I want to call the class called

task group inside of async IO

so async with async IO and then I can

assign this to a variable so let's call

this TG for task group and then with

this task group whenever I add a task to

this group it will immediately start

running and this async with block won't

return until all the tasks in the group

have finished running so let me go ahead

and indent this let me show you how it

works so the loop here looping over one

to four

still appears but instead of creating a

task like this what I can do is I can

say TG dot create task and then the rest

of the API is similar so instead of

using

uh the standard great task from async IO

I'm using the one from task group and

then I'm creating a task and then I

don't need to append it I don't need to

append it to a list here if I'm not

interested in the results and then I

also don't need to run async gather

anymore because like I said these will

start executing right away so let me go

ahead and run this

so task groups so right now this is

going to use the gather and we see it

starts them all at the same time and

then they slowly appear for the other

ones because the first one takes one

second and the last one takes four

seconds right so now if I do this with

group

so whoops just like that and I run it we

see it works just the same it starts

them at the same time and it returns and

I see I have this print results here

because I didn't remove it but let me go

ahead and just run that again

we see start one two three four and then

in one two three four and then it's done

but I don't have the results for this so

this is a little bit of a limitation

um it depends on if you want the results

from your tasks being run oftentimes you

don't you just want to make sure that

they run and don't fail but if you do

want the results here's what you can do

so you can do results equals an empty

list

and then you can append each task to the

results so results a pin task

and then outside I can print the results

so print and then I'll use a list

comprehension for this so task.results

or actually results singular uh four

tasks in results

and you can call this tasks as well and

now if I run this

I see it still works the same but at the

end I get the results from this so if

you want the results I think async

io.gather

is a little bit more clear on how it

works but if you don't care about the

results then task group is definitely a

lot better the last thing I want to talk

about are exception groups and exception

notes so before I get into that let me

just show you this function that I have

here I have this function called load

URLs and I have a list of four URLs out

of the four only one is valid what I'm

going to do is I'm going to Loop over

these four URLs and then I'm going to

try to send a get request to them but

because only one is valid only one

should work but because I have an

exception here it's going to fail on the

first one because the first URL is

invalid so let me just go ahead and run

this exception groups and we see I

immediately get an error so

what I can do is I can use a new feature

called exception groups and exception

groups allow you to pass multiple

exceptions at the same time so this is

useful in cases in like async

programming where you can have many

things running and many things can fail

for different reasons you can group all

those exceptions together and deal with

them as a group so this example that I'm

showing isn't very realistic but I think

it's clear to show you how exception

groups work and the way that I'm going

to do this is first I'm going to create

a list that will hold a bunch of

exceptions so exceptions equals an empty

list and what I'm going to do here is

inside of the exception block here

instead of raising I'm going to append

the exceptions to this exceptions list

so exceptions dot append e and now when

I run this

we see it runs but uh all the exceptions

were suppressed because I appended them

to this list so now that I have a list

of exceptions the way I can raise the

entire group is by using exceptional

groups so to do that

outside of the loop here what I want to

do is I want to raise exception group

just like that you can give it a name so

I can say request errors is the name

and then you have to give it a list of

exceptions

so just save that and I'll run it again

and now the results

have multiple exceptions and if I go to

the top we see exception group Trace

back

and then it just gives you information

about the entire group so raise

exception group request errors and then

we see three sub exceptions because

three out of the four

URLs are in ballot so now how do you

handle this in the exception blocks the

way you do this is first let me put a

try here is instead of using accept

you're going to use accept star

and the way that accepts star is

different from accept is multiple except

Stars can run in one try except block

group so typically when you have try

except when it hits the first except

that applies only that one will run and

the whole block exits after that but now

with accept star multiple except Stars

can run

so let me go ahead and bring in the

exceptions that

could happen here so we have

requests.exceptions dots uh invalids

schema is one

and what it does is it catches the

entire group so it's going to create

like a subgroup for you and it's going

to group all of the invalid schemas into

a sub exception group so that's why I'm

calling the EG for exception group

and I just want to print uh let's say

prints uh invalid schema

and then I'll do the same thing for the

other two so

um one is going to be missing schema

and I just put missing schema here and

then the last one is going to be

connection error

so connection

error

and then we can say

connection error here

so now when I run this let's see what

happens

we see invalid schema missing schema and

connection error so those print

statements all ran even though it

matched multiple of them if I only had

accept without the star then it would

have stopped on the first one and then

the other two wouldn't appear you

wouldn't see them

so now let me just show you the last

thing that I wanted to talk about which

are exception notes which are pretty

simple so inside of exception you have

the ability to add a note to each

exception that you're dealing with so

here e now has this method called at

node so any exception and what you can

do is you can add extra information onto

the exception so typically what you

would do is you would like create your

own exceptions that support additional

information but now you don't have to do

that you can just use e dot add note and

you can add any extra information you

want so what I'll do is I'll add the URL

here

for the notes

and inside of the print statement here

what I'll do

is I'll print out the note information

and I'm going to use a list

comprehension here because this is an

exception group and I can have multiple

exceptions in this example I only have

one per group but I could have multiple

exceptions per group so I'm going to

Loop over them so four

e n e g dot exceptions so each exception

in the exception group and then here I

want to take e and then Dunder notes and

I just want the first note right so it

puts all the notes here

um in this Dunder notes and because I

only have one node per exception I'm

just getting the first one here and I

can just copy this and paste it to the

other ones

and I can run it

and now we see for the invalid schema we

have HTTP SS which is invalid schema a

missing schema so there's just nothing

there for pretty printer.com and then

connection error when I do pretty

printed.coms with an s on the end it

can't connect there and then for the

other one that is correct nothing

happens because there's no exception

there so like I said this isn't really a

realistic example but I do think it

shows the case where you can have

multiple exceptions in one thing and

then you can handle all of those

exceptions at the same time so there are

some other features in Python 3.11 that

I didn't cover like a big one has to do

with speed improvements I couldn't think

of a good way to demonstrate that

because most of the code that I write is

limited by input and output not

necessarily like the speed of the code

execution so the speed changes probably

won't affect me but it is something

that's going to

um globally affect all Python scripts so

you don't have to do anything different

to benefit from the performance

increases you just write your code as

normal use Python 3.11 and there's going

to be some speed increase you don't have

to switch to something like Pi Pi to get

an increase like you just use it like

you are right now and you're going to

benefit from increased performance and

there are also a bunch of other things

added to the typing module to make a

writing typing it's a lot easier there

are a few of them I don't want to cover

them in this video but if you're using

type hints in Python then you'll

probably benefit from the new type hints

and features around type pins that were

added in Python 3.11 so check out the

link in the description below it's going

to talk about like the difference uh

changes to Python 3.11 and in the

comments below let me know which

features you like which features you

don't like

um do you think these are good additions

to python do you think they're not

really necessary let me know in the

comments so that's it for this video uh

feel free to give me a like if you like

this video and if you haven't subscribed

to my channel already please subscribe

so thank you for watching and I will

talk to you next time