So You Think You Know Git - FOSDEM 2024

Summary

Scott Chacon, GitHub co-founder and Git expert, shares his insights on helpful Git configurations, old and new Git features, working with large repositories, and recent GitHub developments in this technical talk. Touching on handy commands like blame and log for debugging, useful options like force-with-lease for safer force pushing, Microsoft's optimizations for huge repos, and GitHub's merge request workflows, Chacon arms the audience with tips to enhance their workflows regardless of their Git proficiency level.

Takeaways

• 😀 Scott Chacon, GitHub co-founder, gave a technical Git talk with tips and tricks
• 📝 Shared helpful Git config tips like aliases and conditional includes
• 💡 Covered useful but lesser-known Git commands like blame ranges and reflog
• 🔍 Discussed capabilities for handling large, complex repos introduced by Microsoft
• 🚀 Talked about new Git features like commit signing via SSH and maintenance commands
• ✨ Demonstrated virtual branches in GitButler for advanced concurrent workflow
• 🤝 GitHub allows enforcing team workflow rules around merging and commit signing
• 📈 Auto-merge and merge queues help streamline collaborating on GitHub pull requests
• 🕵️ GitHub exposes pull request refs that can be fetched like remote branches
• 🍻 Overall, shared advice from his 15+ years using Git aimed at making workflows smoother

Q & A

What is the difference between 'git blame' and 'git blame -w -C -C'?

The default 'git blame' command shows who last modified each line of a file. 'git blame -w -C -C' ignores whitespace changes and detects code movement, so it tracks code authorship more accurately if code has been reorganized.

How can you view the commit history for a specific function using Git?

You can use 'git log -L<function name>' to show the commit history for the code block containing the specified function. Or you can specify a line range with 'git log <path>:<start>:<end>'.

What is 'git reflog' useful for?

'git reflog' shows a log of all repository reference updates, so you can use it to find "lost" commits after doing risky operations like resetting or rebasing. It allows you to get back to a previous state.

What benefits does 'git maintenance' provide?

'git maintenance' runs background tasks to optimize repository performance, including commit graph generation, loose object collection, and incremental repacking. It speeds up operations without any active effort.

How can you speed up 'git status' on an enormous repository?

Enable the file system monitor with 'core.fsMonitor' so Git uses a daemon to watch for file system changes instead of having to stat every single file on each status command.

What is 'force with lease' in Git?

'git push --force-with-lease' does a safer force push by first checking that the remote reference matches the expected value, avoiding overwriting commits that have been added remotely in the meantime.

How can you sign Git commits with SSH instead of GPG?

Recent versions of Git allow SSH-based commit signing. You can configure this by setting 'commit.gpgsign' to point at your public SSH key instead of a GPG key.

What are some advantages of partial cloning?

Partial cloning allows filtering out file contents to make an initial clone much faster for enormous repositories. Needed data can be downloaded on-demand later, although some operations become slower.

What merge types can you enforce on GitHub?

GitHub repository settings allow enforcing requirements like only allowing merge commits, only rebasing, squash merging, signed commits, and more. This helps teams establish consistent workflows.

How can you access open pull requests locally?

GitHub advertises refs for every open pull request that you can fetch directly without adding remotes. This allows checking them out and merging locally.

Outlines

00:00

🎤 Introduction and Background

Scott Chacon, co-founder of GitHub, introduces himself at Fosdem, setting the stage for a technical talk on Git. Despite the large audience suggesting a keynote, Chacon clarifies the focus will be technical, discussing Git's evolution and his involvement. He highlights his contribution to the Git community through his book 'Pro Git' and the development of a new Git client, GitButler. The talk aims to refresh knowledge on Git, introduce new features, and share insights on Git's ongoing development.

05:01

🔧 Exploring Git's Capabilities

Chacon delves into practical Git usage, starting with configurations for a more efficient workflow. He offers insights into aliases for common Git commands, enhancing ease of use. The discussion transitions to 'oldies but goodies', focusing on lesser-known, yet powerful Git commands and options for enhancing productivity and understanding Git's handling of data. Through examples, Chacon demonstrates how these commands can significantly impact the Git experience, especially in terms of data interrogation and manipulation.

10:01

👓 Advanced Git Insights

The talk progresses to more advanced Git topics, including the usage of 'Git blame' and 'Git log' for detailed code investigation. Chacon introduces commands that allow for a granular analysis of changes and contributions within a repository, emphasizing the utility of these tools in understanding code evolution and responsibility. He also covers options to refine 'Git blame' output, showcasing how to bypass cosmetic changes to identify substantial contributions accurately.

15:02

🛠️ Enhancing Git Workflow

Further enhancing Git workflows, Chacon discusses strategies for resolving merge conflicts with 'reuse recorded resolution' and introduces new Git commands for organizing and reviewing branches efficiently. The section covers practical tips for making the Git experience smoother and more productive, including the 'force with lease' option for safer remote updates and SSH keys for commit signing, offering alternatives to GPG keys.

20:03

🚀 Git for Large Projects

Addressing challenges in large-scale Git repositories, Chacon shares Microsoft's contributions to Git for handling massive projects, like Windows. He introduces tools and techniques, such as Scalar and Git maintenance tasks, designed to improve performance and manageability in large repositories. This part emphasizes the importance of prefetching, commit graphs, and file system monitoring in maintaining efficiency and speed in Git operations for large projects.

25:03

🌐 Strategies for Large Repos and Mono Repos

Exploring further into large and monolithic repositories, Chacon discusses the tools and strategies Microsoft developed for managing such projects efficiently. He covers prefetching, commit graph generation, and file system monitoring, all aimed at optimizing Git's performance for enormous codebases. The talk also touches on partial cloning and sparse checkouts as solutions for dealing with large repositories, reducing the overhead of data management and streamlining workflow.

30:03

💡 GitHub Innovations and GitButler

Chacon transitions to discussing recent GitHub innovations and his work on GitButler. He highlights GitHub's new features, such as auto-merging and merge queues, designed to streamline project management. Chacon introduces GitButler's virtual branches concept, allowing simultaneous work on multiple branches, illustrating his vision for more flexible Git workflows. He also addresses audience questions on range diffs, submodules, and SSH signing, providing insights into Git's complexities and potential improvements.

35:04

🤔 Audience Q&A and Final Thoughts

In the closing segment, Chacon engages with the audience, addressing questions on Git's design choices, such as the default behavior of 'force push' and the potential for improving submodule handling. He reflects on the direction of Git development, advocating for features that allow working on multiple branches simultaneously and automatic work recording. Chacon's responses underscore the evolving nature of Git and its adaptation to modern development workflows.

Keywords

💡Git

Git is an open source distributed version control system that is used to track changes in source code during software development. It is one of the core topics of the video, as the speaker provides various tips and tricks for using Git more effectively.

💡GitHub

GitHub is a web-based platform for hosting Git repositories. It allows developers to collaborate on open source projects by providing tools for managing Git workflows. The speaker mentions how GitHub has introduced new features related to pull requests and merge strategies.

💡Commit

A commit in Git captures a snapshot of the project's files at a point in time. Commits are central to Git's version control capabilities. The video discusses commit signing using GPG and SSH, auto-merging commits in GitHub, and optimizations like commit graphs.

💡Branch

Branches represent independent lines of development in a Git repository. The speaker highlights useful commands like 'git branch --column' to view branches visually and 'git branch --sort' to customize branch sorting.

💡Merge

Merging takes changes from one branch and integrates them into another branch. The video covers merge strategies like rebasing vs merging, configuring allowed merge types on GitHub, and resolving merge conflicts.

💡Fetch

Fetching in Git retrieves the latest changes from a remote repository but does not alter the local state. Optimizations like background prefetching and commit graphs speed up fetches.

💡Push

Pushing sends local Git commits to a remote repository like GitHub and updates the remote references. The video discusses 'git push --force-with-lease' for safer force pushing.

💡Sparse Checkout

Sparse checkouts allow cloning only specific subdirectories from a repository instead of the entire repo. This optimizes working with massive mono-repos containing millions of files.

💡Alias

Aliases set up shortcut commands in Git by mapping a word like 'st' to 'git status'. The speaker recommends aliases to save typing for frequent Git operations.

💡Maintenance

Git maintenance automates background tasks like loose object collection and repacking. Running 'git maintenance start' makes many Git operations faster over time.

Highlights

Git has around 40 core commands that are commonly used, with many additional niche or legacy commands

Git blame has options to ignore whitespace changes or code movement when blaming lines to see the original author

Git log can filter history based on a file path or function name to only show relevant commits

The reflog tracks every change to HEAD, allowing you to revisit any previous state even after destructive operations like reset

The 'rerere' feature can automatically record and reuse merge conflict resolutions to save effort during rebasing workflows

Recent options like '--column' improve Git branch's default output formatting for better visualization

Force with lease performs a safer force push by first verifying the remote history before allowing overwrites

SSH commit signing provides an alternative to GPG that can integrate with GitHub's verification UI

Git maintenance handles background tasks like garbage collection to keep repositories optimized

The commit graph caches relationships between commits to enable fast log operations in huge repositories

The file system monitor tracks working directory changes in the background to speed up status checks

Partial cloning allows selectively downloading commits/trees or individual directories from enormous repositories

GitHub now allows enforcing team standards around merging, commit signing, and more

GitHub exposes pull request branches as remote references that can be fetched/merged without added remotes

Virtual branches allow working on multiple lines of work simultaneously without constant switching or stashing

Transcripts

How is our name,

uh last name pronounced?

Chacon. Chacon?

Yeah.

Okay.

Boy, the pressure is on now.

I really feel like I have to deliver here,

so it's nice to have this great full house.

Thank you all for coming out. It's great to be at Fosdem.

So this is not I feel like this

should be kind of a keynote type thing with this nice big audience,

but it's really going to be a bit of a technical, you know, boring talk.

So really, really, you know, bear with me.

So just to introduce myself a little bit, if you don't know,

my name is Scott Chacon.

I was one of the co-founders of GitHub.

I assume most of you know, GitHub at this point.

It's been a little while.

I used to do a lot of talks and go around and try to convince people to use Git

and why Git was cool and now it's, you know, it's been it's

been a while since then, but it's a very different type of talk.

So as I've been giving talks, what I've kind of learned is that

there are some people that have never used anything actually raise your hand.

If you've never used anything that's not Git?

Wow. I feel so old.

All right.

I also I wrote this book called Pro Git.

So if you go to the website and you read the book there, that's my book.

It's open source, Creative Commons license.

So so a lot of times if you search for something on Google or something with Git

it'll hit this, which is always embarrassing

if I forgot something and then read about it

in my own book that I wrote, which happens more than more than zero.

And now I'm working on a Git client called GitButler.

So if you're interested in that, I'll talk about it a little bit later on.

We just opened it on GitHub, so I'll get into it a bit.

But what I really want to go through is Git.

So how many of you use get on the command line?

Okay, so yeah,

fucking nerds.

So how

far with it?

I so, so how well do you know Git right.

Actually throw out a number if you think you know

how many commands how many Git commands actually exist.

And somebody said 400,000.

So there's about 145 Git commands that you can type Git something right.

Most a lot of them are useless.

A lot of them are not useless but are useful for scripting

or in the background or something like that.

There are 40 uh 82 ports on commands.

There's 44 main ones.

Normal stuff that you would do add, commit, push, pull, rebase,

stuff like that.

There's 11 manipulators, so config and ref log

and things that you don't use that often, but kind of give you more more metadata.

Bunch of interrogators, some inter actors.

Most people don't use these, right?

I mean send email you might if you're working with a mailing list with Git

but P4 or SVN or CVS or there's a lot of CVS import,

there's a lot of those types of things that are around almost legacy.

There's a lot of sort of legacy stuff.

And then there's 63 plumbing commands.

So there's a lot of stuff like cat file and things like that that you can use.

You can use the script a lot of times, but for the purposes of this

talk, that's not what I'm going to be talking about.

I'm not particularly interested in the plumbing commands.

I'm not going to go into the the sort of nitty gritty details of Git,

I figure if you're scripting something, if you're using LibGit 2

or something like that, you already know most of this stuff.

That's that's not really the point of this talk.

The point of this talk is

that I've started talking about Git more often with GitButler stuff.

And I've realized that stuff that I took for granted that I thought kind of

everybody knew because it's been around so long, a lot of people don't know.

So this is really kind of a refresher.

And to talk about some stuff that's that's new.

So there's a bunch of stuff that's new

for those of you that are more old school that have been using

Git for a really long time, you may not pay attention to what's new in Git.

So I want to talk a little bit about that as well.

There's still an average of nine commits a day.

There's 10,000 commits in the last three years.

Do you know anything that's been introduced in Git in the last three years.

Right. Like probably not.

Probably most of what you've done, you learned, you know, ten years ago

or seven years ago, whenever you started learning Git

and you don't know a lot outside of that.

So that's kind of the point.

This is my friend Nick.

He actually introduced me to Git when we worked at Reactrix together in 2005.

So this is when Git was five months old when he had his first patch in

and he told me about it. So Git is old enough to drink.

Not in the United States because we're stupid, but here certainly.

So what has changed?

What is new, what is interesting?

So here's our agenda.

I'm going to talk about some helpful config stuff.

If you don't already know, most of you probably do.

I'm going to talk about some oldies but goodies, some stuff

that's been around for a long time that you just may not know.

I'll talk about some new stuff

that's been introduced in the last few years

that you may not have noticed crept in because there's not really a Git

newsletter that most of you guys probably pay attention to.

And then I'm going to talk about some big repo stuff,

which is actually very interesting to me that most of you may not know about, but

may not also have a need for any time soon.

But I find it very interesting.

And then then I'm going to talk about

some new GitHub stuff, some stuff GitHub has introduced since I left

that I recently kind of discovered and I find very interesting.

So I want to make sure you know about it too.

The style of this talk.

I want to call a shotgun buffet so I'm just going to throw

a bunch of stuff at you.

Take what you might find interesting.

If there's something that might make your daily life a little bit easier.

A little bit better. Please.

You know, you also don't need to take notes.

I put these slides on our blog at GitButler, so if you go to get by the dot

com and go to the blog, this whole slide deck is on there.

If, if there's something you want to reference later.

So just keep an open mind.

Let the shotgun buffet hit you in the face and see if there's something

interesting at the end. Right.

So the first thing is how to configure stuff, right?

So I'm going to give you a whole bunch of commands

that have little options, little command flags and stuff

that you may not know about when you learn them.

You probably don't want to type them out of your time.

So you probably want to do a set up

an alias of some sort if you don't know how to do it

or if you haven't done this type of thing before, it's fairly straightforward.

You just say alias, dot,

whatever you want to type, and then if you type, Git that thing.

So if I type Git stash with two A's, it'll do stash, dash, dash all.

I don't have to type dash dash all every time.

So what I like to do is kind of set up a lot of things of how I like it to work,

and I'll show you a bunch of different things.

You might want to Alias, but that's how you do that.

If you didn't know and dash dash global makes it so that all of your Git

repositories have that and not just the one that you're in.

I actually like dash dash all for stash as well.

This is kind of a side note, but if you use dash

and you stash something, it kind of leaves it in a weird state.

Dash, dash, all will take away sort of all of your untracked files

and kind of do what I thought Git stash would do by default, but doesn't.

So just a sidenote.

The other thing you can do is you can tell Git.

If you say get in this case BB, you can say

run some other script and pretend like it's a Git command, right?

So you can script something.

And the way you have to do that is by putting a bang at the beginning

and then it says, okay,

I'll give this sort of a shell prompt and I'll run it as you can even put actual

bash scripts on the command line that way and it'll run that.

So this is something that I put in a just a while ago

to try to give me a better branch output that does head behind information

and orders them by last commit and even uses the Git description,

which most of you probably don't know that you can put a description in the branch,

but it actually lists it out.

Otherwise it doesn't do anything with that information, which is kind of funny.

Okay.

The next config thing that I'll show you

and then and then we'll get into the really fun stuff is include.

If so, most of you might know actually is there water around here somewhere?

I'm going to the rate I'm speaking.

Okay, great.

Git config.

So there's an include if directory directive.

So normally how it works is you're working in your local stuff

and you run something and it needs to say,

okay, I'm doing the commit, what's my email address, right?

Or should I use GPG signing or something like that?

And normally what it does is it looks in your local .git slash config

and then if it doesn't find a value, it backs up into your global config

sort of .git config in your home directory.

But what you can do is you can set up something in between, right?

So it actually has a third place to look in between those things.

So in this case I'm saying anything that's under the OSS directory,

use this Git config so I can sort of overwrite

my email address for my open source stuff

and anything that's in the work directory, use this Git config.

And so I can, I can say here's my, my work information for any of my work stuff,

which is kind of nice to be able to have a little bit more flexibility.

So if we go in here and we look at this Git config,

I'm, I'm saying let's GPG sign stuff and use my

other email address and if I run, user.email.

Outside of that it says here's the email to use.

If I run, user.email inside of that subdirectory,

then it it changes the default, right?

So if you want to have sort of here's all this stuff that that applies

to everything underneath this, then go and do that.

So that's the include if stuff.

Okay so just the config stuff now let's go into oldies but goodies.

I'm actually I might pull the audience a couple of times because you're all nerds

and I really want to know who knows some of this stuff

and who doesn't know some of this stuff because again, I'm not, I'm not.

I've been doing this for a while and I don't always know.

So who uses get blame? Okay,

Who uses get blamed Dash?

Does anybody know what that does?

Okay, so

it just blames a portion of the file.

So normally you run, Git blame and it blames the entire file and pages it.

If you know the line range, then you can say just blame this line

range and A it makes it faster, but B, it makes it more, more understandable.

Right. Because it's not it's not showing so much.

And the other cool thing is that you can do exactly the same thing with Git log.

So I don't know if anybody's ever used this, but this is super cool.

If you know a function and you want to know

sort of the history of that function, you can run,

Git blame and get sort of the one liners or you can say Git log

and give it the the the range in the path of the file

and it'll only show you the commits that Git blame would have shown you right.

But in order so you can kind of get a little bit

more of a story of how that function kind of came together.

Thank you.

You are welcome.

Boy, you're going to love the rest of this talk.

The other cool thing you can do

is you can say, Git log dash L and then give it like a function name.

And it will try to figure out what that block is, hueristically

and do the exact same thing.

So if you it depends on the language.

And I actually don't exactly know what that heuristic is,

but you can say for example file, right it they'll say, okay, this is class file

and so it automatically gives me the line range for that

and just gives me the story of what that is.

All right.

So that is Git blame.

The other stuff that Git blame does and that if you use a GUI

or if you use actually GitHub and this is particularly embarrassing

because I wrote this implementation at GitHub.

So when I shit on GitHub for this, it is entirely on me.

But GitHub and most of the GUIs don't ignore whitespace by default

when when it runs Git blame and there's more options you can give Git blame.

So you can also say dash C which says detect code movement.

So if you delete a function and then move it somewhere else

in the file, it will remember that and it will kind of ignore that movement.

Right? When you when you blame that thing.

So so it won't credit you if you move the function as owning that

if you do it two times.

I don't know how many people

know about command line options where you give it multiple times

and it does different things, but it is arguably not the best UX I've ever seen.

And if you give it

two times it says or the commit that created that file,

if you give it three times it says or any commit at all.

So it's much more expensive, but it can really help

you get if you if you have I mean, it's not that often you're running Git blame.

You might as well do this every time. Right.

Because it takes more time.

But it's much smarter about trying to ignore stuff that doesn't matter.

So, for example,

if we go back to this one, it looks like mostly Kiril did everything.

If I run, Git blame dash WCCC and then give it the same thing,

it gives me very different output. Right.

And and importantly so because Kiril moved this at some point

but I told it ignore that like follow all the renames

and all the movement and it turns out that Kiril actually

had very little to do with any of this code. Right.

And so blame by default, a lot of times if your refactoring stuff

will not end up with the person

who wrote that line of code, but you can tell it to do that.

And it's a I would set up an alias for this.

Okay.

And you can also see what file it was, right.

So this file was changed multiple times

and we can see it moving around and who edited it when it was named.

What file.

Okay, another one.

Git log dash S how am I doing on time? I'm doing okay.

Okay. Git log dash S pickax.

So this is a way that you can say filter all my Git

log output to anything that has this regular expression or the string in it.

So if you removed a variable

or added a variable, anything with that, that that string in it,

it will just show you that this is a really nice way of seeing

when something was changed that maybe Git grep wouldn't find because it was removed

from the code base and you want to know when it was removed from the code base.

The other thing is reflog.

So this is a log of your references.

If you haven't used this, it will do it by default on head

and you can run Git reflog and it will show you okay, you pulled

and you did the reset before that and you did check out from a branch

to another one and you to rebase and here's all the steps of the rebase.

But if you ever want to run like Git reset and you think it's going to mess stuff

up, don't be too afraid because you can run, Git reflog

and see kind of what every step of that was.

And you can always get back to where you were.

So whenever the head is pointing at something, it has a log of that.

Another real

small thing, and then we'll move on to new stuff, Git word diff.

So if you run Git diff, it will give you by default a line based

diff, which sometimes, especially if you're, you know, using tailwind

can be really difficult to figure out.

What was the one class that was kind of added in in the middle of this line.

If you run Git diff --

word diff, it will do it as a word diff instead.

And so it makes it much easier to see if you know it's a word

sort of a word based diff, what the difference is.

Okay.

And then the last one, I'm just going to say the last one because I can't remember

how many there are is re re re, which is reuse recorded resolution.

How many people have this turned on and know what it is. Okay.

So the number of people don't know what it is.

If you have a merge conflict you can tell Git remember this

merge conflict and remember how I fixed it and keep that in memory.

It actually keeps it on disk as a as a hash.

But if I see it again it can automatically fix it for me.

Right.

So if you're rebasing stuff and you're hitting

sort of the same merge conflicts over and over again,

you can turn this on and you only have to fix it one time

and every time after that that Git sees it, it will automatically fix it for you.

And so the way you do that is by saying we re re re enabled.

I would just turn this on.

There's almost no reason not to do this.

It's not super expensive disk wise.

And when you get a merge conflict, it says, hey

I recorded the pre image for this this conflict and when you fix it

it says, okay, I recorded the resolution and when you get the same conflict again,

it says, Hey, I staged this using the previous resolution.

So I saw the same thing and I just fixed it automatically for you.

So it's very nice of you cherry picking a lot of if you re basing a lot.

Okay, so that's oldies but goodies that's been around in Git for-

all of those things have been around in Git for a really long time.

How many learned something new in that section out of curiosity?

I can fantastic.

Okay, so now there's some new stuff.

So now nobody's going to know this.

So one, there is a new option to get branch called Dash Dash column,

which if you've ever run Git branch you'll see it

just kind of lists them out in alphabetical order,

which is one of my least favorite things about Git branch,

because that's not usually what I want to see.

But there's some new options recently that they have.

One is column so you can do dash

dash column or you can set a global config column.ui auto

and then if it sees this sort of list instead of sort of paging it as one line,

it will try to put it into columns graphically for you.

And the other one is branch.sort.

So you can sort your branches by default by in this case reverse commiterdate.

So I want to see the ones that have had the last commit first

right the most recent commit first.

And if I run Git branch now it actually puts them in the columns for me.

Right. So it's a little bit easier for me to kind of kind of see what this is.

So I said this on I really like this this Git branch output much better.

The other kind of funny thing is that there's a new git command

called Git column that just takes input and puts it into columns.

It literally doesn't do anything else.

So if you want to put anything into columns now Git helps you.

All right.

Another show of hands who uses force with lease?

Okay, good.

So force with lease is the worst named and most useful option

that I think it has.

But I'll go through a real quick explanation of what it does.

So this is relatively common.

Let's say that I'm Scott and I'm working with somebody named Tom and I edit a file.

Can you guys with that? It's okay, right?

And I edit a file and I commit the file and I push it to the server

and then Tom pulls from the server and he gets that file change

and he changes the file as well

and he commits it and he pushes it to the server. Fantastic.

We've now collaborated, except Scott doesn't have that change yet.

And Scott just went to lunch and had a beer and thought, You know what,

I really mess this up. I should have done a little bit differently.

And so he goes in and modifies the file and amends the commit.

Right?

Or rebase is his changes or something like that changes the history

and he force pushes.

Fantastic.

So now Tom's change is gone, right?

I just forced pushed over because I didn't think anybody had even seen that.

In the meantime, if you rewind instead and you say force with lease,

what it will do is it will check to see what it thought the reference was.

And if it doesn't match, it won't push it. Right.

Because it seems like it's stale.

So you have to fetch first just to make sure you're up to date

and then you can push

if you push and it's the same, it will do the force push, right?

As long as your reference was the last one that's on the server.

So it's kind of a safe force push.

So I just use this.

If you if you do rebasing or amending or something like that, use force

with least it's much safer than force

and it does the same thing in the right circumstances.

Okay.

Another thing, I'll go through this real fast.

Signing commits who signs commits.

wow. you are nerds.

Yeah.

That nobody I know who signs like usually it's

somebody corporate because the company kind of makes them do it or whatever.

But who signs it with SSH instead of GPG okay, not very many.

So there's now a new thing.

Some people, not me, because I'm pretty smart, but other people

have sometimes a problem getting GPG to work and kind of,

you know, reliably and so, but all of us have SSH

and in openness and say

as of a couple of years ago I think they released a new feature

where you can sign arbitrary data and get a signature out of it.

And so that has been incorporated in Git And so now you can

use SSH to sign your Git commits in the way that you used to.

You can also do what GPG But it does exactly the same thing.

So if you want to use a SSH instead of GPG,

you can say it's actually kind of funny because it's not called signing, it's

the GPG format is SSH and you can

say, Here's where my public key is or here's where my signing key is.

And then if you run, Git commit dash s.

This is what a normal Git commit looks like.

Although I left out the message here, if I put that in though,

if I edit a file and I commit and I sign it and I look at what

that that file looks like, now I have this, this GPG sig.

Right.

But it's actually generated from SSH But the cool thing is that GitHub

you can upload your public key to GitHub

or even use the same one that you push with and then GitHub mark

it as verified or GitLab or you know, if you have your own server or whatever,

if you could do SSH commit verification, it will verify.

And then if I you know, I mean, this is kind of the problem with with,

you know, why we're doing a SSH signing, right,

is because you can put some other email address and do a commit and push it

and then GitHub will say, Hey, this one's verified

because I have the public key and I match it to the email address.

This one is unverified.

And so if you're using GitHub or GitLab, these things will work.

But yeah, if you're signing your keys then or you're signing your commits with GPG,

you can sign them with SSH

and a lot of a lot of things will work that way as well.

Um little known and nobody in this room probably will use this.

But you can also put one guy, one guy

in the front uses this, but you can sign your pushes as well.

So you can sign the sort of ref updates that you push to a server.

GitHub and GitLab do not support this, so most

most people don't use it,

but I think kernel.org supports it and there's a handful.

Or if you do your own, you could do this and then you can create

sort of like an audit log or a trust log or something like that

that has sort of the ref, the ref updates and the and the signature.

So that is doable, but you kind of have to run your own thing

if you want to use that type of thing.

The next thing is Git maintenance.

Who use it, Who's run Git maintenance?

man.

All right.

So all you have to do, everyone do this today.

Go into whatever repository you work on the most and run Git maintenance start

because what Git maintenance start does is it adds.

I mean, really what it does is it does this it just adds this to your Git

config file and it says strategy incremental.

I'll show you what that means in a second.

But what it really does is it adds a cron job.

Forget to run in the background every hour, every day

and do maintenance tasks on that repository.

Right.

And most of the time

that's just going to make lots of things faster in a way that you'll never notice.

So run it

because it's just going to make everything faster

and you won't have to worry about anything.

So if you've ever run like a Git command and you see like, you know,

it's garbage collector, it's running.

Git GC It's like garbage

collecting or pruning loose objects or doing something like that.

It's doing that but in the background.

So you don't

it doesn't have to tack it on to another command that you run

because there's something running in the background.

So this is what strategy incremental does.

It does mostly these four things.

It does commit graph generation, which I'll talk about in a second

pre fetch every hour, which I'll talk about in a second

loose object collection and incremental repacks of of your packs.

But really all that means is shit just gets faster.

Right.

So if you want to run it and just have it go in the background instead of after

sequentially after you run to Git command, you can do that, Git maintenance start.

That's all you have to do. Okay. So now

stuff that

I'm actually I'm curious how many people in this room

work on a repository that is you would qualify as enormous

by enormous would you say more than half

a million files in your working directory?

Okay. Okay, good.

Then there's seven people in this audience

that are going to find the next 20 minutes useful. So.

So whether you like Microsoft or not, I have a little bit of a bias

because they bought GitHub and that was nice for me. But

whether you like them or not, they have put a ton of effort

into large repository and monorepository supporting git

because they moved Windows to it and Windows is enormous.

So Windows is, you know, 3.5 million files on disk, 300 gigabytes to clone.

Like think about starting the job and trying to clone off of a Git repos-

like GitHub or something, 300 gigabytes.

For reference, the Linux kernel is four and a half.

Right?

So it's it's it's a, it's a chore.

And so what they did is that they put a lot of resources

into making large repositories and monorepositories

work relatively well in the Git ecosystem in the last five years.

And I want to show you the tools that they did.

So the first thing they did was they tried to do a virtual file system

and so you wouldn't have sort of actually all of the contents of the files.

It would just be sort of on demand and it would go to a special server

that they had and it request sort of, you know, file contents as it needed it

in order to do something.

But it looked like you had all the files until you actually tried to access them

and then it pulled them on demand and then they thought

it's actually not necessary.

And they created this project called Scalar and Scalar

tried to do a lot of the things that I'm going to actually

they did all of the things I'm going to tell you about now.

But then over time, all of that stuff actually just got shifted into Git core.

And so now they a scalar is just a wrapper around that in the in

the git for windows binary

but everything's baked into Git and you can use it on any on any system.

So now it's just all in Git, so what is that.

So there's a couple of things.

One is prefetching which I talked about with Git maintenance.

If you run Git maintenance, start every hour, your system will fetch

from the server whatever project that you're on,

it will not update your references, so it won't actually do what it does

when you do a Git fetch in that origin main updates,

but it will get all the data and it will keep the references to that data

so that when you go and you run, Git fetch,

that it's not getting almost any data.

It almost always seems to me, even if you haven't fetched in three days,

if you go and you run, Git fetch, it's going to be like, Yep, I already have

all the data essentially in no op Here's your new reference, right?

And so it just makes Git fetch seem fast because it's doing it every hour in

the background all the time.

You can look at this if you run, Git maintenance start

and after an hour you go and you look at your references,

you look at, you know, the sort of for-each-ref list out of the

but you can also just list dot Git slash refs or whatever

there's a new thing called pre fetch

all of the Git commands ignore this very specifically so if you run Git

log dash dash all it will just 100% pretend

that these don't exist which it won't do for any of the refs,

but it pulls all of them down so that it doesn't see

any anything that it pulled down.

And it just does it in the background.

It just makes fetches and pulls very, very fast.

The next thing that they do is something called commit graph,

which also Git maintenance will run for you and will build for you.

But let's look at an example of a large repository.

So this is the Linux kernel.

If I pull down the Linux kernel, it has 1.2

million commits if I run Git log dash, dash, graph dash.

dash graph is a little bit difficult because it has to walk all of the objects

in order to figure out how to render even the first one.

Right?

So it has to walk the entire tree because it doesn't know what it's going

to find to be able to show you sort of even the beginning of the tree.

So it tends to take a really long time.

It can't incrementally do it,

but you can run, Git Git commit graph, right.

And it takes a while, but it is incremental.

So it's only the first time and then it can kind of add to that.

And then you run it again and it takes almost no time.

Right?

Because what it's doing is it's

sort of caching an index of all of the commit objects so that it can do commit

operation or commit graph operations very, very fast.

Now, you don't really want to go in and run, Git commit graph, write all the time.

But there's two different things you can do.

One is run the maintenance thing that I talked about,

which will do it automatically.

The other is this you can set this fetch, write,

commit graph and it will write it on fetch.

So every time you kind of get new data, it'll write the difference update

and then all of your log operations are faster if you have a big repository.

The next thing is file system monitor so you don't have to do untracked cache.

That's just caching on track files.

But first monitor, if you've ever run Git status

in a very, very large repository like chromium or something like that,

if you have 400, 500,000 files, it takes a really long time, right?

Because the way that Git figures out what has changed is it stats

every file in your working directory and that can be relatively expensive.

So FS Monitor instead launches a daemon that looks at the file system and watches

for inode events and updates of cache in memory that says

this file was changed because I saw it get changed with my file system watcher.

And so if you set that that config value, then the next time that you run Git

status, Git will notice that the thing is running or not running.

And if it's not running, it'll launch it.

And the next time you run, Git status after that it'll be very fast.

So this is a status on the chromium project,

on a, on a bare check out, which is again 470,000 files and it takes 10 seconds.

Right.

So if every time you ran Git status and took 11 seconds, it'd be a little bit

annoying if you turn this on and you run Git status again.

It's still relatively slow, but the third time you do it, it's

very fast because now it's just running it out of the file system.

Watcher.

So again, if you have a really large repository,

you can turn on the file system. Watching

this are

some sort of fake statistics that GitHub did where they did synthetic

loads of million files and 2 million files,

and it just gets worse and worse and worse to do add and status commands.

But with the files at the monitor, it's very fast.

Okay. Thank you.

And last is partial cloning.

So if I clone the, let's say,

the Linux repository, it takes a while because it's four and a half gigabytes.

If I say filter, I can filter out blobs so it downloads everything except blobs.

And so all of those are left on the server, but it downloads

all the commits and trees and it has to do two different fetches for this.

One is the gigabyte and a half of commits and trees, and then the other one is

the 243 megs of the working directory blob so that it can put something in my work.

So it just takes the data I need for the very last commit,

but I can still run graph operations and all of that

or I can say no trees

and I don't know why it's blob none but tree zero.

But you've used git for a while so you understand

and that just doesn't bring any trees down either.

It's relatively rare to use this except for in CI build type stuff.

But if you want to take a really large repository and make it make it

so that it's a relatively fast download,

like if you're working on the Windows repository or something like that,

what most people will do when Microsoft does is they filter out

the blobs, they do this, this blobless one, and

and then and then Git will automatically Git new data as it needs new data.

Right. So it's almost like a virtual file system.

And again, this goes down to 2 seconds instead of instead of,

what, 40 4 minutes, four and a half minutes.

Right.

The bad part is that if you blame, it doesn't have all of the data

for every blob file.

And so it needs to fetch it on demand as you run a blame.

So instead of a blame looking like this, it goes like this, right?

So it just keeps pulling next versions down in order to run the blame.

So if I look at sort of the difference of blame from Linux on the full

check out is 4 seconds on a file and and on the blobless one is 45 seconds.

So it does slow some stuff down and there's a lot of other stuff.

If you're really interested in large repository stuff, there's multipack

indexes, the reachability bitmaps, there's geometric repacking.

GitHub has talked about a lot of this a blogged

very well about a lot of this if you're interested in it,

but it tends to be you have to have huge repositories.

And then the last thing that I'll talk about in big repo is model repo stuff,

which is sparse checkout.

So how many use a mono repo at work or so?

Okay. Okay. Not bad. Not bad.

Thank you for the data. Zero.

So sparse checkout is you can clone with no checkout

and then say here's the directories I want and only those ones.

Also, you can say I want a sparse index so only they go in the index.

And if you have a ton of of subdirectories and you only really need to work

with three of them, you can say

give me my just my commits and my commits in my trees and none of the blobs

and just check out these, this one directory, these five directories

or something like that and then just grab the data as you need them.

Right.

And it makes really, really large repositories much, much easier.

So another example we have Git Git status.

It takes 5 seconds

and I run this, Git sparse, check out and set, build and base.

Those are the two subdirectories I want in my chromium build and then I run Git

status and it's much, much, much faster because it's looking at much less data.

But those are the only two that are in my working directory as well.

And when I do commits, it looks like the rest of the data is there, right?

Like it doesn't pull them out, but I can just work with some subsets

and it even tells you I'm only working with 2% of my track files.

Okay, last thing on our agenda.

Actually, I do have one one secret thing real fast,

but GitHub stuff.

So there's a bunch of GitHub stuff that you may not know about.

There's allowed merge types.

So if you go into your GitHub settings,

you can say I only want merge commits or I only want rebasing.

And then you get in a conversation with your coworkers and you fight.

And like people have actual physical arguments and you figure out

are you a rebase shopper or you merge shop, we're having beers later.

So if anybody wants to have that argument with me, I'm more than happy to do that

merge.

So and but you can also do squash merging,

which is arguably a good sort of middle ground for this as well.

And so all these things and then when you go and on GitHub, you can you can

now sort of make sure that your repository sticks to that, right?

So you can say enforce these standards when you hit the sort of merge button,

it will only give you the options that you have allowed it to give you.

But if you try to push, say, a merge, you can tell it

don't accept pushes with merge, commits or you know,

however however you actually want to do it,

you can make GitHub enforce a lot of the stuff now, which is kind of cool.

I actually didn't know that until recently.

You can also auto merge.

So if you guys ever if you turn on this this requirements

for some sort of status and you have some CI, okay

and you have some CI and you say allow auto merge,

there's a button that says I will merge it as soon as it passes CI

But it's kind of nicer than going back and checking status and merging it later.

So I use this all the time.

There's also a

merge queue which if you have a lot of commits,

will kind of stack them all together and merge them at once.

There's a require a linear history.

If you really like rebasing,

you can force GitHub to not accept pushes that have merge commits in them.

You can require signed commits which goes back to the SSH thing and so on.

And then the last thing on GitHub is the PRs that have heads.

So if you run Git LS remote on GitHub repository, all of your pull requests

actually have refs that you can pull down and you can merge from.

Most people don't know that most people think you have to add.

Like if somebody sends you a pull request, an open source project

that you have to add that is a remote and fetch from that and merge that in.

But GitHub actually advertises all of these refs

that you can pull from or fetch from directly, which is kind of cool.

So you can put in something like,

you know, give me all my pull requests as actual refs and then every time

you fetch it it'll pull all of those down as local references, which is cool.

Okay.

And then very last thing I wanted to talk really fast about GitButler,

which is something that, that I'm working on with my team over here, which is we

have this new thing called virtual branches,

so we can work on multiple branches simultaneously

instead of having to sort of stash stuff and switch branches.

And so you can have sort of each lane as a branch.

So this is one branch and another branch, maybe a bug fix,

like you're working on a feature and you see a bug fix,

you don't have to commit it into the feature branch, right?

You can kind of and you don't have to switch.

You can drag it over in another lane and committed separately and each

one can kind of go into its own PR as though that was the only thing on disk.

But it's a very kind of nice way of working and it's open.

So if you go to, to, to, to GitHub you can take a look

and and see if that's something that is interesting for you.

And that is the end of my talk.

Thank you very much. And let's drink.

Okay. So I wanted to have some time for Q&A.

If there's any questions or something that I can do, there's a question over here.

Yeah.

So since you mentioned you do merges, I cannot help asking this.

So what about range diff does

GitButler does it And why does GitHub not do it.

Why does GitHub not do Git range diff?

so, I almost put a thing on range diff in here, but

it's a very kind of specific workflow where you don't use it

unless you're doing a lot of sort of rebasing series

and you want to compare sort of a rebase series

to another series to see if something changed.

So yeah, if you, if you do that, check out range diff,

I have no idea why GitHub doesn't do that except I think that

you know, GitHub like a lot of services

like they try to say here's what we think

a workflow should be or what a good workflow would be.

And they build tooling around that and try not to do everything

that could possibly like, like every workflow that they see in order

to try to make it less confusing for people.

So I think it's more that people in GitHub don't use that type of workflow like,

like at least when I was there, I can't speak for the last

eight years or whatever,

but like we did not do a lot of rebasing, we did a lot of merging.

And so if you're doing merges all over the place, then you don't need

range diff like range diff doesn't really make sense in that

in that workflow because you're not comparing,

you know,

what is this this patch series compared to this patch series

and did I miss something? Is that's my guess

So another question.

We've been working a lot in my team with some sub modules

and frankly it's not a nice thing to work with.

Google has for large project projects introduced repo,

but is there any like work that you know of is being done on sub modules?

Because I'm keeping track of refs of updated.

sub modules is frankly Yeah it's a great question.

Why do some modules suck everywhere is essentially

the question and you know it's funny because they always have

like I've been working on this, as I said, I've been involved

heavily in the Git project for more than 15 years probably,

and there's not a single even meetup of Git developers where they're like,

We shouldn't have put sub modules in there.

Like they it's and there's a lot of different

I think what a lot of people are moving towards is mono repo, which is mono

repo stuff

where you just have

everything in one repository so you don't have

to worry about a lot of the things you did sub module stuff for,

but it doesn't cover every case.

And even actually a lot of the stuff that I talked about,

like the sparse checkouts and the sparse index stuff I think don't work

well if you have sub sub sub modules because just nobody uses them

that works on on Git and so they, they don't optimize optimize anything

but yeah I honestly don't know from from a the companies that I've talked to

in the last couple of years most people that they care about that

either use patch package management type stuff for that that type of problem said

or they have them on a repo

and they can start using some of these tooling,

although GitHub is not good at mono repo support.

So that that that becomes the problem now. Right.

Like locally it's fine but now all your issues

are in one repository and all of your you know.

Yeah but yeah.

Hey thanks. So is this SSH signing, is

it possible to specify more than one key?

Yes. Okay.

Yeah. Yeah, you can.

I mean, when you're signing it or verification like the keyring

when you're signing, I mean, you have to choose a one

to sign the commit with, if that's what you're asking.

But you can.

Is that what you're asking?

I don't know what you're asking.

Yeah. Okay.

So my question would be why can't force was least be the default force.

Yeah.

I mean, the thing is why can't force at least be the default?

It probably would be a good idea.

Or to have the more aggressive version be the the longer option.

I think it's more just that Git is always backward compatible, right Like that.

That's always like, like for, for 19 years now, there's like almost nothing

that I could do.

I could have scripts that I wrote in in 2005.

They would probably still work with, with, with the current version of Git.

And it's a great thing about Git right?

Like, it's actually very, very nice that they really don't deprecate very much

and they make sure everything's always backward, backward compatible that

gets into trouble.

There's actually stuff I didn't really talk about,

like there's like Git switch instead of check out.

That's to just being able to switch branches and not do

other stuff like, like a, like revert file contents and things like that.

So they kind of add

more to the user interface in order to try to make it less confusing.

But then there's just now more commands that do the overlapping things.

So but yeah, it's, it's because they don't want to be backward

compatible on anything and they just assume

you're going to set up aliases for new stuff. That's cool.

I think in that case, it would be great if it would be configurable over

kind of thing. So sorry it might be.

I'd have to check if you go to like to get config

man page, there's 10,000 things that you can configure.

It might be possible.

I don't know if you can overwrite what force does to force with

least by default if you actually push, put it on the command line,

but you could probably change what the default

behavior of Git push with no option is.

Right. But that's not usually what you want to do.

So it may be there may not be one.

I'd have to look or you'd have to look. Actually, you look.

Yeah. Thanks for the talk.

And first of all, Git squashed.

No other options are allowed.

But my real question is, if you're like back at the Gates

development team, what would be the direction that you would like to

see Git moving More like we have a lot of features for mono repos.

What are the things that you think Git is missing?

what a great question.

I mean, to some degree

I have a little bit of a cheat in that I'm writing a Git client

and so everything that I'm putting in the Git

client is stuff that I kind of wish Git did.

So I would like to be able to work on more than one branch at a time

without being able to essentially have more than one index of more than one head

that I'm working on simultaneously. I think.

I mean, we're building that and I find it very valuable.

So that is one thing.

The other would be recording everything you do

that it's a little bit frustrating that I can lose work in Git,

but I can't lose work in,

you know, Google Docs or something that just records shit all that.

Like just save a C R D T of every file all the time.

Like, how expensive is that? Right?

Like, it'd be kind of nice to say, show me what my working

directory or this file or something look like on Tuesday at 10 p.m..

Right. Like you can't do that.

You can't Git won't save anything you don't commit really.

Right. Or stash which is basically a commit.

But why not?

Like there's no there's no reason for a modern version control system.

That's not it's not network based, It's not disk bound.

Right. Like just save everything.

It'd be nice to just say, watch this, like, watch this directory and make sure

I never lose anything in it and help me Craft commits when I'm ready to share.

So those would be two big things that we'll probably

I mean, we've built both of those to some degree,

but like even in core Git, I think it would be nice to have to have there's

Git hasn't changed it was written

for people that are that are submitting patch series the mailing list.

Right and it's very very good at that

and I think GitHub

kind of bastardized that a little bit to really go towards the pull request

rather than than talking about patch series

and have having communications with the patch series

which has a different set of of strengths and weaknesses.

But like I said, it's not it's backwards compatible.

It hasn't changed really the user interface fundamentally for,

which is why a lot of you guys didn't know a lot of the stuff that I talked about.

Like you, you just use the same thing all the time.

I just use the same thing all the time. So.

So yeah, it's, it's a little bit difficult to go

in a massively different direction for those reasons, I think.

But there's lots of cool stuff we could do with version control, I think.

Okay, one more maybe.

How much time do we have? One more.

So we all have the command line.

But one question.

Do you ever use any visual tools?

Do I use visual tools?

So I use obviously I use GitButler.

I've been using it for three or four months now, which I enjoy a lot.

The from and I use, you know, Visual Studio code with VIM

key bindings because I'm kind of a nerd, but then also not

but I'm partial partially nerd, partially executive.

So I have to do Visual Studio code with key bindings.

But, but before that, I think is a more interesting question,

which was what did I use?

What I mean what do you guys use GUIs versus the command line I used GUIs

always for interactive adding because I just find it much much faster

to do interactive adding through almost any GUI than it is through

the actual interactive ad script that Git provides.

I would love to do rebasing more drag and droppy or something like that.

But, you know, I can also edit the stupid script and,

you know, put put pic and stuff all over the place.

That's right.

But for the most part it's interactive, adding

that's the thing I spend a lot of time in.

And when I do takes,

it's worth the context which to go over to a GUI and say,

I can make this much faster and much more specific

than trying to go through, you know, like hit five

to choose the next file grouping and add them and stuff like that.

So it's really it's whatever's fastest.

I think everybody uses the command line with Git still because not because it's

it's the best thing that could possibly be because but because it is fast, right?

In almost anything you do, it's faster than hitting a button

that then calls that script. Right.

And so I always use the command line for stuff,

unless it's something like interactive ad, we're really having a do we add

something to the experience in a way that makes what I'm trying to do faster?

Okay, that's it.

Hopefully I'll see you for drinks later.

Thank you very, very much. It's been a pleasure. Bye.