Your Quarterly Linus Torvalds Linux Rant
Summary
TLDRThe video script discusses the development journey of Bcachefs, a file system for Linux that claims to combine the speed of ext4 with the reliability of Btrfs and ZFS. It details the challenges faced during its long development, including feedback from the Linux kernel community and the necessity for wide-scale deployment to validate its claims. The script also touches on the importance of following kernel development procedures and the human aspect of developers, highlighting the project's eventual merge into the Linux kernel after years of refinement.
Takeaways
- ๐ฎ BcacheFS is a file system for Linux that has been in development for a long time and has recently been integrated into the Linux kernel.
- ๐ BcacheFS uses the Copy-on-Write (CoW) model, similar to Btrfs and ZFS, which is efficient in terms of space and performance.
- ๐ BcacheFS aims to combine the speed of ext4 with the features and reliability of Btrfs and ZFS.
- ๐ The journey of BcacheFS began in 2015, with the first attempt at integration into the Linux kernel in 2018, and it faced several challenges and delays.
- ๐จโ๐ป The developer of BcacheFS has been working silently and diligently, focusing on coding and improving the file system.
- ๐ง The process of integrating BcacheFS into the Linux kernel involved addressing feedback, fixing bugs, and adhering to kernel development procedures.
- ๐ BcacheFS has seen significant development, including the addition of features like snapshots and 'badass' snapshots, which are similar to those in Btrfs.
- ๐ค The integration of BcacheFS into the Linux kernel required cooperation and adherence to the community's standards and procedures.
- ๐ป The debate over BcacheFS's integration highlighted the importance of following the kernel's development process and the role of maintainers in ensuring quality.
- ๐ฎ The future of BcacheFS in the Linux kernel depends on its ability to work within the kernel's release schedule and address the concerns of the community.
Q & A
What does the term 'Btrfs' stand for?
-Btrfs stands for 'ButterFS,' which is a file system that has been in development for a long time and has recently been integrated into the Linux kernel.
What is the significance of the term 'COW' in the context of file systems?
-In the context of file systems, 'COW' stands for 'Copy on Write,' a strategy used to save space and improve performance by only copying the modified data, as seen in file systems like Btrfs and ZFS.
What are some claims made by the developers of Btrfs regarding its capabilities?
-Btrfs claims to offer a balance of features, speed, and reliability, aiming to be as scalable as ext4 and as reliable as file systems like Btrfs and ZFS.
Why is wide distribution and deployment important for validating file system claims?
-Wide distribution and deployment across thousands of users are crucial for validating file system claims because it allows for real-world testing, uncovering potential issues, and ensuring the file system performs well under various conditions.
When was Btrfs first announced, and what was the initial reception like?
-Btrfs was first announced in 2015. The initial reception was cautious, with a small user base of enthusiasts willing to try out this experimental file system.
What was the status of Btrfs development in 2016, and were there concerns about its progress?
-In 2016, Btrfs was still in development with a small user base. There were concerns about its progress due to the developer's silence and lack of updates on the project's status.
What were some of the challenges faced during Btrfs's attempts to be upstreamed into the Linux kernel?
-Btrfs faced challenges such as the complexity of the code base, the need for proper patch submission, and ensuring that the patches were reviewable and did not introduce issues with other parts of the kernel.
Why is it important for developers to follow the kernel development procedure when submitting patches?
-Following the kernel development procedure ensures that the patches are in line with the standards and expectations of the Linux kernel community, making the review process smoother and reducing the likelihood of conflicts or issues.
What is the significance of the 'Linux next' tree in the context of kernel development?
-The 'Linux next' tree serves as a testing ground for future Mainline kernel patches, allowing developers to test and refine their code before it is considered for inclusion in the main kernel.
What was the final outcome of Btrfs's inclusion in the Linux kernel, and what were the developer's reactions?
-Btrfs was eventually merged into the Linux kernel after a long development process and several attempts. However, there were disagreements and concerns raised by kernel maintainers about the size and scope of the patches, as well as the developer's adherence to kernel development procedures.
Outlines
๐ฎ Introduction to BcacheFS: The COW File System
The paragraph introduces BcacheFS, a file system with a long development history that has recently been integrated into the Linux kernel. It highlights BcacheFS's claim to offer a balance of features, reliability, and speed, comparing it to other file systems like ext4, ZFS, and XFS. The concept of copy-on-write (COW) is explained, which is a key feature allowing for space-efficient snapshots of data. The paragraph also touches on the challenges of proving these claims, emphasizing the need for widespread adoption and testing among many users.
๐ฐ๏ธ The Long Road to BcacheFS Upstream Integration
This section delves into the timeline and challenges of BcacheFS's development and integration into the Linux kernel. It discusses the initial announcement in 2015, the quiet period of development without much public update, and the first attempt at upstreaming in 2018. The paragraph outlines the iterative process of feedback, refinement, and resubmission for review, highlighting the developer's persistence and the community's involvement in shaping the file system's integration.
๐ง Refinements and Review Challenges for BcacheFS
The paragraph discusses the technical and procedural hurdles faced during BcacheFS's review process for inclusion in the Linux kernel. It mentions the need for splitting the code into reviewable patches, the importance of providing change log texts, and the broader implications of making changes that affect other parts of the kernel. The narrative illustrates the back-and-forth between the developer and the kernel maintainers, emphasizing the importance of following established procedures and community standards.
๐ Navigating Technical and Procedural Critiques
This section continues the narrative of BcacheFS's journey towards the mainline kernel, focusing on the technical debates and the necessity of adhering to kernel development protocols. It highlights the importance of having a verifiable chain of trust, such as a PGP key, and the need for the developer to demonstrate a collaborative approach within the kernel community. The paragraph also underscores the significance of proper testing and the developer's responsibility to ensure code็จณๅฎๆง and compatibility.
๐ ๏ธ Final Push and Ongoing Development of BcacheFS
The final paragraph summarizes the ongoing development and integration efforts for BcacheFS, including the resolution of issues and the eventual merge into the mainline kernel. It discusses the developer's response to critiques, the importance of following the kernel's release schedule, and the necessity of keeping changes minimal and focused during the release candidate phase. The paragraph concludes with a reflection on the human aspect of kernel development, acknowledging the passion and dedication of developers while also recognizing the importance of rules and procedures in maintaining the integrity of the Linux kernel.
Mindmap
Keywords
๐กFile System
๐กBcacheFS (Bcash FS)
๐กCopy-on-Write (CoW)
๐กUpstreaming
๐กLinux Kernel
๐กSnapshots
๐กBenchmarks
๐กRelease Candidate
๐กLinus Torvalds
๐กDevelopment Procedure
Highlights
Linux distributions typically default to file systems like ext4 or Btrfs, with others like ZFS and XFS also in use.
Bcachefs, recently upstreamed after long development, has a dedicated fan base and the tagline 'the COW file system for Linux that won't eat your data'.
Bcachefs claims to combine the speed of ext4 with the features and reliability of Btrfs and ZFS.
Copy-on-write (COW) is a key feature in Bcachefs, similar to how it's used in Btrfs and ZFS, optimizing performance and space usage.
Bcachefs development began in 2015 and has been a topic of interest for Linux kernel discussions.
In 2016, despite doubts about the project's continuation, the developer silently worked on it without much public update.
The first attempt to upstream Bcachefs into the Linux kernel was in 2018 during the 4.18 release cycle.
By the end of 2018, Bcachefs was close to being ready for upstreaming, with a to-do list that continued to shorten.
In 2020, Bcachefs aimed for Mainline kernel integration but faced issues with code submission and review.
Bcachefs's development has been marked by large patch sets and extensive discussions within the Linux community.
The Linux kernel's maintainers have emphasized the importance of following proper development procedures for new contributions.
Bcachefs's journey to the Linux kernel has been long, with several attempts and setbacks, reflecting the complexity of kernel development.
In 2024, there were disagreements over Bcachefs patches, highlighting the ongoing challenges in its integration.
The developer of Bcachefs has been advocating for its reliability and trustworthiness compared to other file systems.
The Linux kernel community's feedback and review process have been critical in shaping Bcachefs's development.
The story of Bcachefs underscores the human aspects of open-source development, with developers having different opinions and occasional conflicts.
The video concludes with a call for more user testing and real-world deployment to validate Bcachefs's claims and performance.
Transcripts
everyone needs a file system or two most
Linux dist is default to either ext4 or
butterfest with a scattering of others
on things like ZFS and xfs but these are
not the only file systems out there
there is one that has been in
development for a very long time that
very recently got upstreamed that has a
very dedicated fan base that being bash
FS using the tagline the cow file system
for Linux that won't eat your data
as you can tell by the capitals here
we're not referring to the animal this
means copy on right another example of
these is Butters and ZFS you've probably
heard that snapshots aren't a full
backup of your data the way they work is
they only copy the modified data this
saves a lot of space it is a lot more
performant this is copy on right copy on
right can mean different things in
different context but in the case of a
file system this is how it's being used
now bcash FS has some pretty big claims
it claims to be as feat F and have the
reliability of things like butter fs and
ZFS whilst also having the speed of
things like ext4 Now features are
something you can easily go and list out
when it comes to reliability and speed
yes you can do benchmarks and you can do
tests amongst like small groups of users
but to really give those claims any
weight you need wide District deployment
across thousands and thousands of users
and a few updates back it finally got
upstreamed into the Linux Kel after a
very very long time of development and
then during the 61- rc5 cycle as lus 12
olds tends to do every so often he went
off on a bit of a rant a bit of a rant
that usually when things like this
happen it's not like a one-off event
that cuses it it is something that
builds up over time it is little things
here and there and then eventually
enough is enough and this is only the
first of the emails so let's talk about
how we got here and to do that we need
to go all the way back to the start to
when bcash FS was first announced in
2015 now judging by just how many
articles fonics has made about bash FS I
think it's fair to say that Michael
arabel is at least a little bit of a fan
or is at least somewhat interested in
seeing where this project goes because I
don't think there is a topic on fonics
that he has talked about this much
besides just general Linux kernel stuff
now way back in 2016 there were
questions about whether the project was
still going because the developer was
just silently working on it he wasn't
posting about it he wasn't like o here
here's what's happening on Twitter
here's this and that he was just writing
code making a file system and doing his
job which honestly I kind of wish that
more people did I feel like a lot more
stuff would get done I'm talking about
myself as well I shouldn't be on social
media as much as I am and whilst it was
certainly coming along in 2016 it had a
very small user base of very crazy
people that wanted to use a very
experimental file system that was only
at that point a couple of months into
development it wouldn't be for a little
bit longer that the first attempt at
upstreaming the code base came that was
in 4.18 in
2018 and like many early attempts at
trying to Upstream code like this it had
some discussion it had some feedback
there were people that were saying hey
maybe do this maybe do this or try out
this instead for the most part it was
still quite a while away from being
ready but he just wanted some feedback
on whether he was going in the right
ction and from what I can see he
basically was and towards the end of
2018 he gave a status update in the
project what was done what was close to
being done what was still very much a
work in progress it was a fairly
extensive status update the to-do list
continues to get shorter and bugs
continue to get fixed it was getting
very close to the point where
upstreaming was legitimately possible at
least from the developers perspective
with hopes to possibly Upstream the code
base sometime in mid 2019 from his
perspective it seemed like things were
in a really good state it seemed like
the US he had were fairly happy with it
and it seemed like it was becoming at
least good enough to be an experimental
file system upstreamed into the kernel
there is a lot of code in the kernel
that is still a work in progress but
it's good enough to the point where it's
not going to cause issues for other
things in the konel but here is the
problem it doesn't matter how long
you've been a developer for how much
experience you have with a code base how
much experience you have with a specific
kind of work most people are really bad
at estimating timelines especially when
they are completely in control of what
goes into that timeline so whilst maybe
if he'd stuck to the absolute minimum
set of what he wanted to do mid 2019 was
possible the next time that started
being talked about was the end of 2020
this was being submitted for review in
the 5.11 release window but again every
time you try to do something in a really
complex code base there's always
something that you do that isn't done
entirely the correct way and this person
here points it out please excuse my
ignorance if I miss things things in
other discussions but if this is what's
expected to be reviewed why the
submission is not splitted into
reviewable patches usually what you
would see is instead of one giant block
of here's all the code it would be patch
one patch two patch three so on and so
forth as an example the first time he
went up for review here we have patch
one of two and then the other one is two
of two this makes it a lot easier to
review a specific commit and say okay
this is what you should change about
this one here's something here okay
here's something about the other commit
instead of just replying to one Mega
thread and then trying to work out
exactly what things people are trying to
refer to even with that being the case
though like all the times before it he
got some feedback and then went back to
working on things this time working on
faster boot times doing a core feature
rework and hopefully maybe if the review
goes well it could be merged soon and
this time when he submit it for review
this time he made sure it was split out
into separate individual patches B
cashif Fest status update it's done
cooking let's get this sucker merged
this time being a set of 12 separate
patches but this time having a bit of a
different problem this should say I know
not you know here is a post from Greg
cower Hartman you know I reject patches
with no change log text at all you
shouldn't rely on other maintainers
being more laxed you need to describe
why you're doing this at the very least
as I sure do not know and this is making
a change outside of just the bcash fs
stuff if he was just changing his stuff
it wouldn't be as big of a deal but when
you're touching things outside of your
own work you need to explain why you're
doing that because that's going to
affect other people who now don't know
why you're changing it either so again
got some feedback went back to work on
things by 2021 it had butterfest like
snapshots it also had something being
called badass snapshots and again was
time to aim for Mainline Kel integration
surely 2022 was going to be the
year 2022 was not the year surely 20123
was going to be the year bcash FS a new
cow f system features too many to list
known bugs too many to list this time it
has ballooned even further now turning
into a patch set of 32 patches and
turning into the biggest discussion for
bcash FS up to this point some of the
patches don't have any comments but some
of them have giant threads under them
with tons of different people all
getting involved to talk about what is
being done here being such a big patch
set and a patch set that a lot of people
just hadn't actually reviewed yet before
it was a fairly small subset of people
there were a lot of issues that were
discovered some old kernel bugs that
were being returned some weird usage of
types that don't really make sense in a
kernel context the Kel does things in a
very specific way and just other bugs
that maybe didn't get spotted earlier
this is not a problem
with bcash FS this happens with every
single big patch set there is a lot of
things that need to be ironed out to
make it Linux kernel ready need I say it
again he took the feedback went back to
working and resubmitted the work for the
6.5 cycle and that discussion didn't
last 6.5 it stretched out into 6.6 and
there was a couple of reasons for that
one is it was still an absolutely giant
patch that this was not going to change
it touched a lot of systems it did a lot
of things it just took a really long
time to review leading to a lot of
people wanting to give their feedback on
the topic but
also uh there were a couple of issues
with um getting into fights with people
in the thread but people fighting in a
Foss project and especially fighting in
the lkml that's pretty normal stuff it
wouldn't be a normal day in Linux if
people were not just fighting over
random things sometimes things that
don't matter sometimes things that
actually do have some relevance to the
code base but usually not and eventually
lonus horal did comment on the thread
because there were some technical issues
as well or more like issues with the
developer following kernel development
procedure and kind of trying to jump
ahead of the queue no way am I pulling
that without a single sign tag and a pgp
key with a chain of trust you've been
around for long enough that having such
a key shouldn't be a problem for you so
make it happen there are a few other
issues that I have with this and
Kristoff did mention a big one it's not
been in Linux next Linux next is
basically the testing ground for future
Mainline Kel patches you want it to be
there so people can go and mess around
with things make sure it all works make
sure the Kel still compiles but before
you bring it anywhere near a Mainline
contribution I don't know why I thought
it had been it's just such an obvious
thing for any new I want this merged
Upstream tree these kinds of I'll just
ignore all basic rules kinds of issues
do annoy me I need to know that you
understand that if you actually want
this Upstream you need to work with
Upstream that very much means not
continuing this I'll just do it my way
you need to show that you can work with
others that you can work within the
framework of upstream and that not every
single thread you get into becomes an
argument this by the way is not
negotiable if you feel uncomfortable
with that basic notion you'd better just
continue doing development outside the
main Kel tree for another decade I know
it can scare some people away I know
some people are intimidated by it but
this is why every big project needs a
Linus tvols needs someone who is a fair
person but is somebody is not following
procedure has the power to just say no
you are going to do this correctly I
don't care how good your code is I don't
care why you think it's such a great
addition if you don't follow procedure
we don't need your code now it'd be very
different if he was a new developer that
didn't really understand Colonel lingo
the way the colonel did things
understandable but at this point he's
been a kernel developer for a very long
time he should understand the procedure
by now now it turns out he did try to
submit the code to Linux next the issue
is it wouldn't build against Linux next
so it effectively wasn't in the tree
anyway so as you could probably guess it
didn't make its way into
6.6 however it did move into the Linux
next tree it was able to build it did
clean up a lot of the external patches
so it wasn't touching as many systems
outside of bcash FS anything it needed
to touch touch those were handled in
external patches and it was getting much
much closer to actually being merged so
once again it was submitted for
6.7 and this time it actually got merged
but the merger is not the Final
Destination this is just the starting
line so come March of 2024 there was
some disagreement about some patches
that bcash FS was trying to admit Kent
was approaching this in a very engineer
like mindset where he wanted to turn a
part of the bcash Fest file system into
a generic Library function that other
FAL systems were able to make use of now
doing this within the context of bcash
FS totally fine no one would really care
about it but when you're trying to do
something that is going to interact with
other file
systems now it becomes more of an issue
the make random BFS code as a library
function stuff I looked at decidedly is
senseless and ended up meaning that I'm
not pulling this without a lot more
explanation and honestly I don't think
the explanations would hold water keep
it your own code where it belongs don't
try to make it some generic Library
thing and if you do make a library thing
it needs to be much more explained have
much Seer naming and fewer disgusting
and completely nonsensical interfaces
and no find finding the other file
system to share this kind of code is not
sufficient to try to claim it's a sane
interface and sane naming but the main
deal breaker is the insane math this was
a system that was being very over
engineered it wasn't being engineered
wrong right totally fair how it's being
engineered the problem
is if anyone else need to touch it in
the future it's
like why did you write it like this like
trying to make any modifications to it
would be an absolute Nightmare and with
that it brings us to the present the
August patch set bcash FS fixes for
6.11
rc5 now when we're in the RC period This
is a release candidate by this point we
are well into the development window and
the general rule for the colonel is
development is done before the
development window after the window
opens save it for next time the only
things that are being done are fixing
bugs that are showing up for this
development period not fixing old bugs
that you happen to discover in some
random bit of code fixing things that
are relevant to new code being submitted
also by rc5 things should be pretty much
mostly cleaned up and um nobody wants to
deal with 1,300 insertions
671 delici at this point yeah know
enough is enough the last pool was
already big this is too big it touches
non bcash ofs stuff it's not even
remotely some kind of regression at some
point fix something just turns into
development and this is that point
nobody sayane uses bcash fs and expects
it to be stable so every single user is
an experimental site the bcash fs
patches have become these kinds of lots
of development during the release Cycles
rather than before it to the point where
I'm starting to regret merging bcash FS
if bcash FS can't work sanely within the
normal Upstream kernel release schedule
maybe it shouldn't be in the normal
Upstream konel this is getting Beyond
ridiculous and Kent disagreed with the
framing eh Universal consensus has been
that bcash Fest is definitely more
trustworthy than butterfest in terms of
will this file system ever go
unrecoverable or lose my data I've seen
many reports of people who've put it
through the same situations where
butterfest Falls this is the fal system
you're all going to want to be running
knock on wood in just a year or two and
lest times back in I'll believe that
when there are major distes that use it
and have lots of varied use but it
doesn't even change the issue that you
aren't fixing a regression you are doing
new development to fix some old problem
and now you are literally editing non
bcfs files 2 enough is enough with Kent
saying what is to be gained by holding
back fixers if we've got every reason to
believe that the fixers are solid again
from lus what is to be gained by having
release rules in a stable development
environment I wonder but seriously
thinking that I changed a thousand lines
there's no way they introduce new bugs
is the kind of thinking that I do not
want to he from a maintainer what planet
are you from stop being obtuse now like
most developers on big projects Kent has
a bunch of automated testing now the
problem is when you have automated
testing somebody has to write the
testing and sometimes the testing has
bugs and sometimes the testing misses
things that's just every code base not
like again not just specific to this
code base every single code base has
this problem and the bigger the code
base the bigger the problem K bugs
happen the number of bugs that happen in
bug fixes is in fact quite High you
should see the stable tree discussions
when people get heeded about the
regressions introduced by fixers this is
for example why stable has the rule of
fixers being small which does get
violated but it is at least a goal it
cannot be bigger than 100 lines with
context because small fixes are easier
to think about and hopefully they have
fewer problems on their own it is also
why my development happens before the
merge window rule exists if you have to
do development to fix an old problem
it's for the next merge window exactly
because new bugs happen we want
stability but your poor requests haven't
been small fixes and I admit I've let it
slide you never saw the last po request
when I side did a get fetch and went
through every commit just to see and
then did the pull for real this time I
did the same and came to the conclusion
that no this was not a series of small
fixes anymore and to be fair to Kent he
does have a solid reply I know look file
system development is as high stakes as
it gets normal colal development you
screw up you crash the system you lose
some work you reboot people are annoyed
but generally it's okay in F system land
you can corrupt data and not find out
about it until weeks later or worse I've
got stories to give people literal
nightmares hell that stuff has fueled my
own nightmares for years you know how
much gray my beard has now which is why
I spent many years of my life building a
code base and development process where
I can work productively where I can not
just catch but recover from pretty much
any mistake imaginable because peace of
mind is priceless now toold did not end
up replying to this
and I it's fair like it's a fair point
right but at the end of the day Kent is
not the only developer in the colel
there is thousands of other lines of
code tens of thousands of other lines of
code that could dealt with every single
patch cycle
and unless there is a good justification
for it as I said earlier every big
project needs a lonus tvol need someone
who can put their foot down and say this
is the way we're doing it get over it
honestly I love these Kel development
stories because it just makes you
remember that the people developing
Linux are people they're not development
machines they are people that have their
own priorities some of them maybe don't
follow all the rules some of them get a
little bit heated sometimes people just
break character and like you know what
I'm done I'm not going to play the nice
guy today I need to say enough is enough
and honestly I think it's important for
everybody to remember this because it
doesn't just app to Linux it applies to
every other project out there the people
involved in this are people but let me
know your thoughts down below if you
happen to be a konel developer and
happen to have some fun konel
development story I would love to know
let me know about it and if you like the
video go like the video and if you
really like the video and you want to
become one of these amazing people over
here check out the patreon scribes liay
Linked In the description down below
that's going to be it for me and
honestly I kind of want to try out B Cas
FS to see if it's as good as he says it
is
[Music]
[Music]
w
Browse More Related Video
5.0 / 5 (0 votes)