Is the AI Boom Real?
Summary
TLDRThe author traveled to Japan and the U.S. to learn about the AI chip industry. Much buzz surrounds Sam Altman raising huge funds for a new AI chip venture. The concept of 'scaling laws' suggests that with more data and compute, AI models like GPT continue improving. This drives massive investment. Nvidia faces competition but responds aggressively. Giants like Microsoft vertically integrate to cut costs. OpenAI's ChatGPT generated $2B revenue, but it's unclear if consumer demand can sustain the AI boom. The industry hopes Microsoft's Copilot subscription catches on. If not, some assumptions need rethinking. AI's main impact could just be improving ad targeting for giants like Google and Meta.
Takeaways
- 😲 The chip venture Sam Altman is working on could involve up to $1-7 trillion in investments over several years, largely for data centers and manufacturing
- 📈OpenAI's 'scaling laws' show that combining more data and compute leads to better AI model performance
- 👀Nvidia is aggressively updating its AI chips to stay ahead of new competitors
- 🚨Tech giants like Microsoft and Google want to cut costs by making their own custom AI chips
- 🤔It's unclear if demand from mainstream consumers will sustain the massive investments expected in AI
- 🙌ChatGPT is one of the fastest growing consumer tech products ever at over $2 billion annual revenue
- 🔎All eyes are on Microsoft's Copilot subscription to indicate enterprise demand for AI
- 💰AI promises automation that could greatly boost advertising revenue
- 😕But AI may just make rich companies even richer, not necessarily transform industries
- ✈The author plans more trips to learn about and discuss the AI landscape
Q & A
What was the purpose of the author's recent travel to Japan and the United States?
-The purpose was part sight-seeing and part learning tour to understand more about the AI and AI chip boom landscape outside of Taiwan.
What figure is Sam Altman said to be seeking for investments related to AI chips?
-Initially it was reported to be billions, but more recently a figure of $1-7 trillion was reported, likely referring to total investments over multiple years related to manufacturing, real estate, power, etc.
What concept is driving the massive investments forecasted in AI chips?
-The 'scaling laws' concept that combining more data and compute leads to better AI model performance. This concept helped drive advances like GPT-3 and GPT-4.
How is Nvidia responding to increased competition in AI chips?
-By aggressively increasing the pace of updates to their AI accelerator lineup, releasing new products as often as every 6 months.
How might vertical integration by tech giants like Microsoft impact Nvidia?
-In the medium to long term, as tech giants develop more advanced custom AI chips, it could reduce Nvidia's market share. Nvidia aims to stay ahead by leading on performance and cost.
What evidence is there so far that AI products are driving real consumer demand?
-There are some signs of promise like ChatGPT's reported $2 billion annualized revenue run rate, but it's still early. Microsoft's Copilot subscription will be an important test case.
Where else might AI have a major financial impact?
-In advertising and automation, where AI could greatly boost revenues and efficiencies for major platforms like Google and Meta.
What does the author see as a disappointing possibility for AI?
-That its biggest impact is automating and optimizing existing industries rather than enabling wholly new products and companies, making the rich richer rather than producing new wealth.
What approach does the author recommend for embedding AI capabilities?
-Recommend embedding AI into existing successful products to maximize its impact, using AI as an enabling technology rather than a standalone consumer product.
What are the author's plans going forward regarding AI?
-The author plans to make more frequent trips to the US and other regions to have interesting conversations with people in the AI space and keep abreast of the latest developments.
Outlines
🤔 The Trillion Dollar Question
Paragraph 1 discusses the recent news around Sam Altman raising massive investments for an AI chip venture, with figures ranging from billions to trillions of dollars over several years. It questions the feasibility of such a high investment level and compares it to the entire semiconductor industry revenue. The key theme is the uncertainty around the scale of investment needed to realize Altman's vision.
📈 Scaling Laws - The New Moore's Law?
Paragraph 2 introduces the concept of scaling laws that have driven advancements in AI like GPT. It notes the parallels between scaling laws and Moore's Law in driving industry roadmaps and investments. However, it also highlights arguments against the perpetual viability of scaling laws.
💸 Is the AI Boom Financially Sustainable?
Paragraph 3 questions whether the current AI boom, despite rapid progress, is financially sustainable in the long run. It examines early revenue data for ChatGPT and other potential indicator products. The conclusion is cautious optimism, noting AI's potential as an enabling technology, but uncertainty around transformative mainstream adoption.
Mindmap
Keywords
💡Scaling Laws
💡ChatGPT
💡Nvidia
💡Moore's Law
💡Microsoft
💡ASICs
💡Advertising
💡Vertical Integration
💡Consumer Demand
💡Killer App
Highlights
Sam Altman and his team have talked to TSMC, people at Abu Dhabi, and the US Government about a trillion dollar chip venture
The trillion dollar figure may be a negotiating tactic to set high expectations for future talks
The scaling laws concept is driving massive investments - more data and compute leads to better AI model performance
The scaling laws could have a similar rallying effect in AI to what Moore's Law had on the semiconductor industry
Nvidia is aggressively updating their AI accelerators to fend off startup competitors
Nvidia ships GPU designs before fully testing them using advanced modeling tools
Tech giants like Google and Microsoft pose the biggest threat to Nvidia through custom ASICs
Microsoft is pushing hard on vertical integration for inference chips and networking
It's unclear if the AI boom is financially sustainable without compelling consumer products
ChatGPT grew quickly, but few products have development and operating costs like AI
All eyes are on Microsoft's Copilot subscription to indicate consumer demand
The killer app for AI could just be improving ad targeting and sales
If AI mainly helps the wealthy get richer, it may not justify massive investments
AI could have more impact by supercharging existing products versus creating new ones
Ads enhanced by AI automation generate tens of billions in revenue already
Transcripts
For the past three weeks I have been traveling through Japan and the United States.
This trip has been part sight-seeing and part learning tour with the goal
of understanding more about the AI and AI chip boom landscape outside of Taiwan.
Now that I am finished with the big conversations, I wanted to sit down and share a few thoughts.
## The Trillion
There’s been a lot of news about a chip venture thing that Sam Altman
of OpenAI is working on for the AI industry.
The news came slowly. First there was the report - released at around the
time he was temporarily ousted from OpenAI - that he was raising money
from Middle East investors for some chip venture. Back then it was just "billions".
Since then, you have also gotten news that Sam Altman and his team have talked to TSMC,
people at Abu Dhabi, and the US Government.
Just as I was leaving Tokyo for San Francisco, the WSJ broke the report with
that eye catching $1-7 trillion figure that has got everyone buzzing. It was one of the
first things that I started asking people about when I arrived in Silicon Valley.
Many people agreed with me that the number is a bit too much. Perhaps
it can be a negotiating tactic to set expectations for future talks. Once you
start talking trillions then hundreds of billions no longer feels as substantial.
Then you have the Information adding a bit of context to Altman's remarks, saying:
> But in reality, Altman privately has told people that figure represents the sum total of
investments that participants ... would need to make, in everything from real estate and
power for data centers to the manufacturing of the chips, over some period of years.
The ecosystem concept makes a lot more sense. Total semiconductor sales in 2023 were about $520
billion. Total capital expenditures - according to the trade group SEMI - were about $140 billion.
But even if we were to assume that this trillion is spread out over five years
and diluted with real estate expenses and power,
that is still another COVID-like step function upwards in capital expenditure.
The semiconductor industry is a conservative one. Many of the people there are old-heads
who have seen many a cycle of booms and busts downstream. Sam isn’t really the first guy to
knock on a foundry’s door saying that his use case is going to change the world.
## Scaling Laws
The concept driving this investment is something called the "scaling laws".
The name dates to a 2020 paper posted by OpenAI titled "Scaling
laws for neural language models". I am not going to go over the fine details,
but the gist is that if we combine more data and compute, we get better results (i.e. less loss).
This was one of the core ideas that helped OpenAI make the GPT-series a
reality. Ilya Sutskever, one of the company's cofounders,
mentioned something like this in an appearance on the No Priors podcast back in November:
> I was very fortunate in that I was able to realize that the reason neural networks of
the time weren't good is because they are too small. So like if you tried to solve a vision
task with a neural network with a thousand neurons, what can it do? It can't do anything.
> It doesn't matter how good your learning is and anything else.
But if you have a much larger network then it can do something unprecedented.
GPT-3 is big. GPT-4 was even bigger, performing far better. And GPT-5, whenever it comes out, will
be even bigger. So far there are no indications that the scaling laws have broken apart.
I find the parallels with the semiconductor
industry's Moore's Law very interesting. The two "laws" do not say the same thing,
but they might have similar effects on their respective industries.
In the 1980s and 1990s, Moore's Law became the rallying cry of the entire semiconductor
industry - the metronome keeping time for every company from Santa Clara to Tokyo to Hsinchu.
There is a chance that the Scaling Laws can make a similar impact on the AI industry. A simple set
of easily understandable rallying cries that drive R&D roadmaps for whatever years to come.
There are a variety of arguments against the Scaling Laws. For instance, people have
commented that we have basically pulled all the existing data across the entirety of the Internet.
But there are ways around such things, if the money is there to make it happen. Physics problems
have cropped up ahead of Moore's Law, and the industry derived new engineering solutions:
the High-K Metal Gate, FinFET, DUV Lithography, Dry Etch, Ion Implantation, SEM, and so on.
The bigger question is whether the money is there to continue
driving this investment. More on that later.
For a way deeper review of the technical issues behind scaling,
I recommend Dwarkesh Patel's post - "Will scaling work?" His podcast is quite excellent too.
## AI Chips & Nvidia
Where there are profits, there are competitors coming out with ideas to take them.
There has been a lot of ink spilled on Nvidia and their competition - particularly the
multi-pronged attack on Nvidia's AI accelerator profits. I don't want to add too much to this.
I do question whether the Nvidia fortress is going
to be as assailable as it might at first seem. Jensen is responding to the threat
by aggressively ramping up the annual updates to their accelerator lineup.
For more information on this, I refer you to
the report from SemiAnalysis. It goes into much more detail. Dylan's great.
But this relentless pace reminds me of the days of old Nvidia during the Graphics Cards Wars,
when Nvidia released a new product every six months to the market. It
worked for them then. Why not try it again now for the AI Chip Wars?
Nvidia can maintain this rollout speed because - as Jensen implied
during his appearance at the Acquired podcast - they ship before they test.
They use the latest computer software design and emulation tools to model
and ship new GPU designs to the market without first fabbing a physical chip:
> The reason why we needed that emulator is because if you figure out how much money that
we have, if we taped out a chip and we got it back from the fab and we started working
on our software, by the time that we found all the bugs because we did the software,
then we taped out the chip again. We would’ve been out of business already.
The Nvidia teams will try to tape out "perfect chips" as Huang said,
but this inevitably will cause problems for customers. For instance, deployments
with drivers and other affiliated software that won't work all that well at first.
But the Nvidia brand can take a hit like that,
whereas it is unlikely that the AI chip startups will be able to. It takes several
years to build a team and then get that team good enough to ship a working product.
This high speed of iteration will be rough on customers. Many have no choice but to buy
what is available, but it stings to spend tens of thousands of dollars on a machine,
only to have a vastly better one coming out just a short time later.
## Giants and Verticalization Nvidia is less concerned about the startups and
even the established silicon players than they are about the tech giants.
The tech giants - Microsoft, Google, and the like - are the ones driving the current investment
spend in AI today. They are also the ones with the most incentive to cut Nvidia out of their margin.
They would do this using custom-designed chips or ASICs. Large companies have
sought these since the good old days of the computer - commissioning an ASIC to
replace an entire board of discrete electronic components to save money.
An example of this would be the Apple IIe,
a third-generation version of the Apple II PC powered by a custom ASIC.
A more modern example would be the Google TPU, which right now is in its fourth iteration.
It makes up a big part of Google's compute advantage over OpenAI and the other AI labs.
Microsoft seems to be the giant pushing hardest on vertical integration. CEO Satya Nadella said
in the Q2 2024 earnings call that most of the usage in Azure AI services is coming from
inference rather than training. Those activities are easier to push to a custom-designed ASIC.
In addition, Microsoft has been working on vertically integrating other parts
of the database stack. For instance, the Information's recent report that
they are working on a network card to shuttle data between servers.
This type of vertical integration implies either that the scale of the AI database stack is so
large and costly that every penny counts/will count. Or that the growth in the industry has
petered out, leaving players to compete on price. These scenarios both feel weird to consider.
What does this vertical integration trend mean for Nvidia? It will take some time
for the other tech giants to ramp up their AI chip designs. Apparently the first Google TPU
was a very bare-bones product. So in the short term, things will be as they are.
But as those chips get better in the medium to long term, it makes sense that Nvidia
push as hard as they can to always win the performance-cost crown. Who knows.
## Financially Sustainable?
I want to move away from cost. I think the cost benefits of vertical integration are significant
enough to matter. Now, I want to move to the other side of the financial equation.
One of the really big questions that I wanted to answer coming to the United States was to learn
more about whether or not this OpenAI boom is Real. Real, with a capital R.
There has been so much progress in the AI industry in the past year. For instance, the
OpenAI text to video Sora model. The improvement in the models' output quality has been impressive.
But even if the technology is amazing, it does not seem like a hit product by itself. We need
to embed them into compelling products. Are there indications of such products breaking
through into the public? Who is making money from actual consumers with them?
The massive investments required by Moore's Law and the semiconductor industry were
driven by real demand from various downstream industries. First the military, then consumer
electronics like radios and calculators, then the PC, smartphones, and then cloud computing.
These were all things that ordinary people wanted. To me,
it feels unlikely that the truly large investments in AI will happen
unless those ordinary consumers start buying these services in a major way.
The Financial Times did report that ChatGPT hit a
$2 billion run-rate revenue in December 2023. That is basically the only solid
piece of news indicating that people are paying real money for an LLM service.
Now, we should note that it is still early in the
generative AI boom. ChatGPT is a little over a year old.
But I also want to note that the iPhone was first released in June 2007.
About two years later in 2009,
Apple sold 20 million iPhones for like $13 billion in revenue. Growth continued,
with 40 million iPhones sold the year after that, and then 72 million and 125 million after that.
Okay fine, the iPhone is the greatest consumer technology
product that history has ever seen. It is a difficult example to live up to.
ChatGPT is still one of the fastest growing consumer tech products in history,
depending on what you think about Threads. And people are still
using it a lot - Altman recently said that ChatGPT generates 100 billion words a day.
And Facebook took a while to produce revenue - at first investing to build up the audience.
Perhaps ChatGPT is doing the same Aggregator approach, gathering users to monetize later.
But few products cost as much to use. AI probably has shorter a leash than we think.
Other than ChatGPT, the one product that everyone in the industry seems
to have their eye on is the Microsoft Copilot subscription service. We want
to see if this $20 a month offering catches on with people and enterprises.
If it does, then we are really off to the races ... the sky really is the limit ... we are so back
... and so on, insert bombastic metaphor here. If not, then we have to adjust some mental models.
## Conclusion
There is one last thing that I should mention. Maybe where the
LLM revolution will be most Real is when it is embedded into the products we know today.
For instance, a few people I spoke to insisted that I am overlooking the
impact that AI automation will have in supercharging advertising sales.
Like Google's Performance Max ads, which use AI to automate the creation, deployment,
and targeting of new ads. Ads like Performance Max apparently generate tens of billions of dollars.
Or how when Apple disabled cookie tracking in 2021 with the App Tracking Transparency thing.
Meta/Facebook at first said that this would cost the company over $10 billion in sales.
But over time, Meta somehow managed to claw back its targeting accuracies using AI,
collecting data from its Conversions API.
So maybe the killer app for AI is just more ads.
But Ben Thompson and others have been saying for a
while that AI is an enabling technology - a technology for making the rich richer,
rather than making a whole new class of rich people like the PC or electrification was.
So for me, if this is all that it is,
then it is a bit disappointing. But it makes the AI boom nevertheless real.
Anyway, these are my reflections from my trip to the United States. I hope to make more frequent
trips in the future and have more interesting conversations with people in the space.
If you are interested in having a chat,
shoot me an email. Maybe I will be coming to town again and can have that chat in person.
Browse More Related Video
Nvidia's meteoric rise to $3 trillion | About That
OpenAI's New Model Releases LEAKED | Sam Altman talks about AGI, UBI, GPT-5 and what Agents will be
GOOGLE entra a gamba tesa per spazzare via ChatGPT (ma inciampa su una demo Fake)
Why Microsoft has a 'meaningful ways to run,' analyst discusses
FIX - Up 100% In 4 Months - Building Data Centers As Fast As They Can!
AI News: The AI Arms Race is Getting Insane!
5.0 / 5 (0 votes)