I tried Vibe Physics. This is what I learned.

Summary

The video explores the capabilities and limitations of current large language models (LLMs) in developing new physics theories, using the Millennium Problem of the Navier-Stokes equation as a test case. The host evaluates GPT5, Claude Opus 4.1, Grok 4, and Gemini variants, highlighting their strengths in research and idea critique but weaknesses in conceptual accuracy, notation consistency, and genuine innovation. GPT5 performs best, while Claude lags significantly. The video emphasizes that AI is useful for literature review and brainstorming but cannot replace human expertise in creating novel physics theories. It also briefly introduces Incogn, a service for protecting personal data online.

Takeaways

• 🧠 AI is being used to explore new physics theories, but results are often either novel or correct, rarely both.
• 📌 The Navier–Stokes existence and smoothness problem serves as an example to test AI's capabilities in theoretical physics.
• 🔍 GPT-5 showed the best performance among tested models, understanding the problem roughly and suggesting reasonable steps.
• 🤖 Grov 4 provided moderate assistance, pointing out known links and offering pseudo-code, but with limited practical usefulness.
• 💡 Gemini 2.5 was creative but confused key concepts and often concluded the problem was impossible due to misunderstandings.
• ⏳ Gemini Deep Think was slow, mostly rephrased text, and declined to perform novel abstract reasoning.
• ⚡ Claude Opus 4.1 responded quickly but produced wordy, often incorrect content and misunderstood basic concepts.
• 📚 AI excels at literature research, explaining related work, and brainstorming, but struggles with generating genuinely new, correct physics ideas.
• -
• ❌ Common AI issues include confusing similar concepts, switching notations mid-response, and producing plausible-looking but incorrect arguments.
• -
• 🔧 Current AI models are best used for background research, idea criticism, and literature review, rather than replacing human physicists.
• -
• 📱 AI is becoming effective at practical tasks like automating the removal of personal data from databases, highlighting its utility beyond research.

Q & A

What is the main focus of the video transcript?

The main focus is exploring how large language models (LLMs) perform when tasked with developing new physics theories, specifically in the context of the Navier-Stokes millennium problem.

What is the Navier-Stokes millennium problem discussed in the video?

It asks whether solutions to the Navier-Stokes equations, which describe fluid and gas dynamics, can develop singularities from regular initial conditions under finite forces.

Why does the presenter believe singularities in Navier-Stokes solutions might exist?

The presenter suggests that quantum physics may be necessary to prevent singularities, and since Navier-Stokes equations are classical, they might allow blowups.

What approach did the presenter consider for using AI to explore this problem?

The idea was to link solutions from general relativity with the Navier-Stokes equations using a suitable coordinate system and stress-energy tensor, leveraging Penrose's singularity theorem to identify potential blowups.

Which AI models were tested in the video, and how were they evaluated?

GPT-5, Claude Opus 4.1, Grock 4, Gemini Pro Ultra Extra Super Deep Think, and Gemini 2.5 were tested. They were evaluated based on their understanding, reasoning, ability to suggest next steps, and usefulness for generating new physics ideas.

What were GPT-5's strengths and weaknesses according to the transcript?

GPT-5 roughly understood the proposed idea, suggested reasonable steps, and could dig up relevant solutions. However, it had minor misunderstandings about forcing in the Navier-Stokes problem and needed clarifications.

Why did Gemini models underperform in the task?

Gemini models were slow, vague, and often confused fundamental concepts, such as time-reversal symmetry and energy definitions. They also had inconsistent self-confidence and sometimes dismissed the task as impossible.

What common limitations were observed across all AI models in physics reasoning?

Models often confused similar physical concepts, switched notation or topics mid-response, and could not reliably generate genuinely new physics ideas, instead producing plausible but often incorrect arguments.

What are AI models currently good for in physics research?

They are effective for literature review, background research, explaining related concepts, and critiquing ideas if prompted carefully.

Why does the presenter believe AI is not yet a replacement for human physicists?

Because AI struggles with abstract, high-level theoretical reasoning, consistently makes conceptual mistakes, and cannot reliably develop new, correct physics theories, making human expertise essential.

How did Claude Opus 4.1 perform compared to other models?

Claude was the fastest to respond but produced verbose, low-quality text and made basic conceptual errors, ranking lowest in usefulness for theoretical physics exploration.

What is the presenter’s humorous observation about AI outside physics?

The presenter notes that AI is currently better at making scam calls than solving complex physics problems.