Some New Features in Python 3.11
Summary
TLDRThis video discusses new features in Python 3.11, including enhanced tracebacks for better error debugging, improvements in handling asynchronous tasks with task groups, and the introduction of exception groups for managing multiple exceptions simultaneously. The presenter demonstrates these updates with code examples, highlighting how Python 3.11 makes debugging and handling asynchronous processes more efficient. Additional features like performance improvements and updates to Python's typing module are briefly mentioned. Viewers are encouraged to explore these updates and share their thoughts on the usefulness of these new additions.
Takeaways
- 😀 Python 3.11 introduces improved tracebacks, offering more precise error location details compared to Python 3.10.
- 🔍 With Python 3.11, you can easily spot where a ZeroDivisionError occurs in a complex expression, saving debugging time.
- 🚀 The new Task Groups in async programming allow asynchronous tasks to be grouped and executed immediately, unlike the old gather method which waits for task completion.
- 🛠️ Task Groups don't require gathering results explicitly, making them ideal for cases where the result doesn't need to be returned.
- 📊 If results from tasks are needed, using asyncio.gather is more straightforward, but Task Groups offer a simpler syntax for non-result-driven tasks.
- 🔗 Python 3.11 introduces exception groups, allowing multiple exceptions to be raised and handled together, which is especially useful for asynchronous programming.
- ⚡ With the new `accept*` syntax, multiple exceptions within a group can be handled, unlike the traditional method that stops after the first match.
- 📝 Exception notes are a new feature that lets you attach additional information to exceptions, providing more context during debugging.
- 💡 The script demonstrates how to use exception groups and notes by showing how to manage multiple failed URL requests with informative error handling.
- ⏩ Python 3.11 includes performance improvements, making it faster globally without requiring changes to existing code, along with enhanced type hints.
Q & A
What is the main focus of the video?
-The video covers new features introduced in Python 3.11, focusing on improved tracebacks, task groups, exception groups, and performance improvements.
How has the traceback feature improved in Python 3.11?
-In Python 3.11, the traceback provides more precise information about where an error occurs, highlighting the exact part of the expression causing the issue, such as showing which variable is causing a zero division error.
What are task groups in Python 3.11, and how do they differ from using 'asyncio.gather'?
-Task groups are introduced in Python 3.11 as a more efficient way to manage asynchronous tasks. Unlike 'asyncio.gather', task groups start tasks immediately without needing to create a list of tasks, and the 'async with' block does not return until all tasks are completed.
Can you get the results of tasks in a task group? How?
-Yes, you can get the results of tasks in a task group by appending each task to a results list and later accessing the results using a list comprehension. This approach gives you similar functionality to 'asyncio.gather'.
What are exception groups, and when would you use them?
-Exception groups allow multiple exceptions to be raised and handled at the same time, which is particularly useful in asynchronous programming where several tasks can fail for different reasons. They enable you to manage multiple exceptions in a single block.
How do 'accept *' blocks work in Python 3.11?
-'Accept *' blocks in Python 3.11 allow multiple exception handling blocks to run for different types of errors within the same try-except statement. Each applicable exception is handled separately without exiting after the first match, unlike traditional except blocks.
What is the purpose of the 'add_note' method in Python 3.11?
-The 'add_note' method allows developers to attach additional information to exceptions without needing to create custom exceptions. For example, you can add extra details like URLs or context-specific notes to the exception for debugging purposes.
What other features are included in Python 3.11 besides task groups and exception groups?
-In addition to task groups and exception groups, Python 3.11 brings speed improvements, enhancements to the typing module, and other performance optimizations that benefit developers without requiring code changes.
Are there performance improvements in Python 3.11, and do they require code changes?
-Yes, Python 3.11 includes speed improvements, but these enhancements do not require code changes. Developers can continue writing code as usual and benefit from the performance boost automatically.
Why might a developer choose to use task groups instead of 'asyncio.gather'?
-A developer might choose task groups over 'asyncio.gather' because task groups start tasks immediately, simplifying code by removing the need for creating and appending tasks to a list. They are particularly useful when task results are not needed.
Outlines
🐍 Python 3.11: Enhanced Tracebacks
In this paragraph, the video introduces a new feature in Python 3.11: improved tracebacks. The speaker compares the error reporting between Python 3.10 and 3.11 by running a piece of code that divides numbers, triggering a zero division error. In Python 3.11, the traceback is more precise, highlighting the exact part of the expression causing the error. This new feature improves debugging efficiency by pinpointing where the issue occurs, saving developers time when identifying the cause of an error.
⚡ Running Asynchronous Tasks with Task Groups
This section discusses two methods for running asynchronous tasks in Python: the traditional `asyncio.gather` approach and the new `asyncio.TaskGroup` introduced in Python 3.11. The speaker walks through an example where tasks are executed concurrently. The `TaskGroup` method starts tasks immediately when added, eliminating the need for task lists or explicit `gather` calls. While `gather` makes it easier to return results, `TaskGroup` provides a cleaner way to run multiple tasks, especially when the results are not needed. This new method simplifies managing async operations in Python.
💥 Managing Multiple Exceptions with Exception Groups
In this part of the video, the speaker introduces Python 3.11's support for handling multiple exceptions with 'Exception Groups.' This feature is particularly useful in asynchronous programming, where many operations can fail for different reasons. The example demonstrates how to catch and handle multiple exceptions simultaneously using `ExceptionGroup`, as well as how the new `except*` syntax allows handling several exceptions in one try-except block. The speaker also introduces 'Exception Notes,' which lets developers add additional context to exceptions for better error diagnostics.
⚙️ Python 3.11 Performance Improvements and Other Features
The final part discusses general improvements in Python 3.11, including performance boosts that speed up code execution without requiring any special modifications by developers. While the speaker acknowledges that they couldn't demonstrate the performance gains due to the nature of their code, they mention that Python 3.11 will deliver noticeable performance increases. Additionally, the speaker briefly touches on improvements in the typing module, which makes using type hints easier and more powerful for developers. The video ends by encouraging viewers to share their opinions on Python 3.11's new features.
Mindmap
Keywords
💡Python 3.11
💡Traceback
💡ZeroDivisionError
💡Asynchronous Programming
💡TaskGroup
💡asyncio.gather
💡Exception Groups
💡Exception Notes
💡Performance Improvements
💡Typing Module
Highlights
Introduction to Python 3.11's new features, available now and recently released.
Improved tracebacks in Python 3.11, offering more precise error locations for better debugging.
Example of improved traceback: detecting exact zero division location in a complex expression.
Explanation of new async features: creating groups of asynchronous tasks more efficiently.
Introduction to the 'task group' feature in async IO, which simplifies task grouping and management.
Comparison between async IO gather and the new task group; task group eliminates the need to store tasks in a list.
Illustration of running async tasks using 'task group' for automatic task initiation and grouping.
Drawback of task groups: cannot retrieve results directly, unlike with async IO gather.
Solution for retrieving results when using task groups, by appending tasks to a results list.
Introduction to exception groups: managing multiple exceptions simultaneously, useful for async programming.
Demonstration of exception groups, using a URL loading function where multiple requests may fail.
Using 'accept star' to handle multiple exceptions in one block, which processes all applicable exceptions.
New feature: adding exception notes to exceptions for additional context, e.g., URL causing an error.
Overview of Python 3.11's general performance improvements, benefiting all Python code without additional setup.
Mention of new typing module enhancements in Python 3.11, which facilitate using type hints.
Transcripts
hey everyone in today's video I want to
go over some of the new features of
python 3.11 that I found interesting
it's available now it was released a
couple of days ago at the time of
recording this video so you'll be able
to do everything that I'm showing in
this video today
the first feature has to do with
improved tracebacks so here I have some
code it basically just assigns the
letters a to J to the numbers one
through ten and then it divides all of
these here just like you see and it
prints out the result so let me go ahead
and run this just so you can see what it
looks like and it gives me a really
small number now if I were to change
this to have a zero somewhere let's say
five and I change that to zero and I run
this using python 3.10
we see I get this error message right so
it's telling me that there's a zero
division error somewhere in this
expression I'm dividing by zero but it's
not clear where now with python 3.11 if
I run this so python 3.11 is my default
now so I still see the zero division
error but it's giving me more precise
information on where the error is
occurring and we see it's happening
after the D
and before the E so the division there
so I'm trying to divide by zero there so
it's saying that e is zero so one two
three four five and one two three four
five yes so e is zero and that's the
part that's failing if I change this
back to five
and I change let's say the 9 to 0 which
means the I in this case will be zero
and run it again we see now this little
arrow is pointing to the division sign
right before the I which means the I
itself is zero this is going to save you
time when you're debugging because it's
going to show you exactly where
something is occurring so you won't have
to kind of look at everything in the
area you'll have a better idea of where
to start your investigation when you're
debugging so I think this is a great
addition to python 3.11 the next thing I
want to talk about are different ways of
creating groups of asynchronous tasks
so right now I have this example of the
old way of doing it or you know the the
standard way of doing it here I'm
here I have this async function which
will basically print start and end and
return finish along with some number
that I pass in and it's just going to
sleep for however many seconds I pass in
so I want to run this multiple times in
a group so the typical way you do this
is with gather so you create a list of
tasks so I have this task list and I
just create a bunch of tasks from one to
five well really one to four and they're
calling this async function that I've
defined up there and then I'm adding
them to this task list so I have the
list of tasks and then I await
um this list of tasks by using gather so
then gather will run all these tasks at
the same time and it will wait for them
all to finish and when it finishes then
it Returns the results from the entire
group of them so this is fine I've used
this before of course you can change
this user list comprehension but the
idea is the same you just basically
create a list of tasks and then you
await those tasks using gather so now
there's a new way of doing it and I'm
going to copy this one and I'm going to
modify it to support the new approach so
I'm going to call this run with groups
so let's change this to groups I can get
rid of the tasks here because I no
longer need a tasks list and what I want
to do instead is I want to use async
with
and I want to call the class called
task group inside of async IO
so async with async IO and then I can
assign this to a variable so let's call
this TG for task group and then with
this task group whenever I add a task to
this group it will immediately start
running and this async with block won't
return until all the tasks in the group
have finished running so let me go ahead
and indent this let me show you how it
works so the loop here looping over one
to four
still appears but instead of creating a
task like this what I can do is I can
say TG dot create task and then the rest
of the API is similar so instead of
using
uh the standard great task from async IO
I'm using the one from task group and
then I'm creating a task and then I
don't need to append it I don't need to
append it to a list here if I'm not
interested in the results and then I
also don't need to run async gather
anymore because like I said these will
start executing right away so let me go
ahead and run this
so task groups so right now this is
going to use the gather and we see it
starts them all at the same time and
then they slowly appear for the other
ones because the first one takes one
second and the last one takes four
seconds right so now if I do this with
group
so whoops just like that and I run it we
see it works just the same it starts
them at the same time and it returns and
I see I have this print results here
because I didn't remove it but let me go
ahead and just run that again
we see start one two three four and then
in one two three four and then it's done
but I don't have the results for this so
this is a little bit of a limitation
um it depends on if you want the results
from your tasks being run oftentimes you
don't you just want to make sure that
they run and don't fail but if you do
want the results here's what you can do
so you can do results equals an empty
list
and then you can append each task to the
results so results a pin task
and then outside I can print the results
so print and then I'll use a list
comprehension for this so task.results
or actually results singular uh four
tasks in results
and you can call this tasks as well and
now if I run this
I see it still works the same but at the
end I get the results from this so if
you want the results I think async
io.gather
is a little bit more clear on how it
works but if you don't care about the
results then task group is definitely a
lot better the last thing I want to talk
about are exception groups and exception
notes so before I get into that let me
just show you this function that I have
here I have this function called load
URLs and I have a list of four URLs out
of the four only one is valid what I'm
going to do is I'm going to Loop over
these four URLs and then I'm going to
try to send a get request to them but
because only one is valid only one
should work but because I have an
exception here it's going to fail on the
first one because the first URL is
invalid so let me just go ahead and run
this exception groups and we see I
immediately get an error so
what I can do is I can use a new feature
called exception groups and exception
groups allow you to pass multiple
exceptions at the same time so this is
useful in cases in like async
programming where you can have many
things running and many things can fail
for different reasons you can group all
those exceptions together and deal with
them as a group so this example that I'm
showing isn't very realistic but I think
it's clear to show you how exception
groups work and the way that I'm going
to do this is first I'm going to create
a list that will hold a bunch of
exceptions so exceptions equals an empty
list and what I'm going to do here is
inside of the exception block here
instead of raising I'm going to append
the exceptions to this exceptions list
so exceptions dot append e and now when
I run this
we see it runs but uh all the exceptions
were suppressed because I appended them
to this list so now that I have a list
of exceptions the way I can raise the
entire group is by using exceptional
groups so to do that
outside of the loop here what I want to
do is I want to raise exception group
just like that you can give it a name so
I can say request errors is the name
and then you have to give it a list of
exceptions
so just save that and I'll run it again
and now the results
have multiple exceptions and if I go to
the top we see exception group Trace
back
and then it just gives you information
about the entire group so raise
exception group request errors and then
we see three sub exceptions because
three out of the four
URLs are in ballot so now how do you
handle this in the exception blocks the
way you do this is first let me put a
try here is instead of using accept
you're going to use accept star
and the way that accepts star is
different from accept is multiple except
Stars can run in one try except block
group so typically when you have try
except when it hits the first except
that applies only that one will run and
the whole block exits after that but now
with accept star multiple except Stars
can run
so let me go ahead and bring in the
exceptions that
could happen here so we have
requests.exceptions dots uh invalids
schema is one
and what it does is it catches the
entire group so it's going to create
like a subgroup for you and it's going
to group all of the invalid schemas into
a sub exception group so that's why I'm
calling the EG for exception group
and I just want to print uh let's say
prints uh invalid schema
and then I'll do the same thing for the
other two so
um one is going to be missing schema
and I just put missing schema here and
then the last one is going to be
connection error
so connection
error
and then we can say
connection error here
so now when I run this let's see what
happens
we see invalid schema missing schema and
connection error so those print
statements all ran even though it
matched multiple of them if I only had
accept without the star then it would
have stopped on the first one and then
the other two wouldn't appear you
wouldn't see them
so now let me just show you the last
thing that I wanted to talk about which
are exception notes which are pretty
simple so inside of exception you have
the ability to add a note to each
exception that you're dealing with so
here e now has this method called at
node so any exception and what you can
do is you can add extra information onto
the exception so typically what you
would do is you would like create your
own exceptions that support additional
information but now you don't have to do
that you can just use e dot add note and
you can add any extra information you
want so what I'll do is I'll add the URL
here
for the notes
and inside of the print statement here
what I'll do
is I'll print out the note information
and I'm going to use a list
comprehension here because this is an
exception group and I can have multiple
exceptions in this example I only have
one per group but I could have multiple
exceptions per group so I'm going to
Loop over them so four
e n e g dot exceptions so each exception
in the exception group and then here I
want to take e and then Dunder notes and
I just want the first note right so it
puts all the notes here
um in this Dunder notes and because I
only have one node per exception I'm
just getting the first one here and I
can just copy this and paste it to the
other ones
and I can run it
and now we see for the invalid schema we
have HTTP SS which is invalid schema a
missing schema so there's just nothing
there for pretty printer.com and then
connection error when I do pretty
printed.coms with an s on the end it
can't connect there and then for the
other one that is correct nothing
happens because there's no exception
there so like I said this isn't really a
realistic example but I do think it
shows the case where you can have
multiple exceptions in one thing and
then you can handle all of those
exceptions at the same time so there are
some other features in Python 3.11 that
I didn't cover like a big one has to do
with speed improvements I couldn't think
of a good way to demonstrate that
because most of the code that I write is
limited by input and output not
necessarily like the speed of the code
execution so the speed changes probably
won't affect me but it is something
that's going to
um globally affect all Python scripts so
you don't have to do anything different
to benefit from the performance
increases you just write your code as
normal use Python 3.11 and there's going
to be some speed increase you don't have
to switch to something like Pi Pi to get
an increase like you just use it like
you are right now and you're going to
benefit from increased performance and
there are also a bunch of other things
added to the typing module to make a
writing typing it's a lot easier there
are a few of them I don't want to cover
them in this video but if you're using
type hints in Python then you'll
probably benefit from the new type hints
and features around type pins that were
added in Python 3.11 so check out the
link in the description below it's going
to talk about like the difference uh
changes to Python 3.11 and in the
comments below let me know which
features you like which features you
don't like
um do you think these are good additions
to python do you think they're not
really necessary let me know in the
comments so that's it for this video uh
feel free to give me a like if you like
this video and if you haven't subscribed
to my channel already please subscribe
so thank you for watching and I will
talk to you next time
5.0 / 5 (0 votes)