Lecture 3: Pretraining LLMs vs Finetuning LLMs

Summary

This lecture explores the two critical stages of building Large Language Models (LLMs): pre-training and fine-tuning. Pre-training involves training on massive, diverse, unlabeled datasets, enabling LLMs to learn general language patterns and perform multiple tasks like translation, summarization, and question answering. Fine-tuning refines the pre-trained model on labeled, domain-specific data to optimize performance for specialized applications such as legal AI, customer support chatbots, or financial research tools. The session highlights real-world examples, explains the differences between foundational and task-specific models, and emphasizes the importance of computational resources, illustrating how companies leverage LLMs for practical, high-accuracy solutions.

Takeaways

• 📝 LLMs (Large Language Models) are AI models trained on massive text datasets to understand and generate human-like language.
• 📚 Pre-training involves training an LLM on a huge and diverse corpus of data, including Common Crawl, WebText2, books, and Wikipedia.
• ⚡ Pre-trained models, also called foundational models, can perform a wide range of tasks such as text completion, translation, summarization, sentiment analysis, and question answering, even if not explicitly trained for them.
• 💰 Pre-training LLMs requires enormous computational resources and cost, e.g., GPT-3 pre-training cost was approximately $4.6 million.
• 🎯 Fine-tuning adapts a pre-trained LLM to a specific domain or application using labeled data, making the model specialized and more accurate.
• 🏢 Companies like SK Telecom, Harvey AI, and JP Morgan fine-tune LLMs to improve task-specific performance, such as customer support, legal assistance, or financial analysis.
• 📊 Fine-tuning can be categorized into instruction fine-tuning (using instruction-response pairs) and classification fine-tuning (using text with labeled categories).
• 🛠️ Pre-training typically uses unlabeled data in an unsupervised learning setup, while fine-tuning requires labeled data for supervised learning.
• 💡 The workflow for building LLMs includes three main steps: data collection, pre-training to create a foundational model, and fine-tuning for task-specific applications.
• 🚀 Fine-tuned LLMs are essential for production-level applications in industries and startups, whereas pre-trained models alone may suffice for general educational or personal use.
• 📖 The lecture emphasizes understanding the difference between pre-training and fine-tuning, with pre-training providing broad capabilities and fine-tuning adding domain-specific precision.
• 🔜 Future lectures in the series will introduce Transformers and the 'Attention Is All You Need' paper, preparing for hands-on coding with LLMs.

Q & A

What are the two main stages of building a Large Language Model (LLM)?

The two main stages are pre-training and fine-tuning. Pre-training involves training on a large, diverse dataset to build a general-purpose foundational model, while fine-tuning refines the model for specific tasks or domains using labeled data.

What is the difference between a pre-trained model and a fine-tuned model?

A pre-trained model, also called a foundational model, is trained on a large corpus of general data and can perform multiple tasks without task-specific training. A fine-tuned model is further trained on labeled, domain-specific data to optimize performance for a particular application or task.

Why is pre-training usually done on unlabeled data?

Pre-training is done on unlabeled data because the task, such as next-word prediction (autoregression), does not require labels. The model learns general language patterns and semantic relationships from a massive, diverse dataset.

Why do companies like SK Telecom and JP Morgan fine-tune LLMs even if pre-trained models like GPT-4 already exist?

Pre-trained models may lack domain-specific knowledge, such as customer support conversations in a specific language or proprietary company data. Fine-tuning ensures that the model provides accurate, context-specific responses tailored to the company’s requirements.

What types of tasks can a pre-trained LLM perform without fine-tuning?

A pre-trained LLM can perform tasks like text completion, translation, summarization, question answering, sentiment analysis, and linguistic acceptability, even if it was originally trained only for next-word prediction.

What is the role of labeled data in fine-tuning?

Labeled data provides explicit instruction-response pairs or classification labels for the model. This data is used to guide the LLM toward performing specific tasks accurately, such as email classification (spam vs. non-spam) or legal case analysis.

Can you explain instruction fine-tuning and classification fine-tuning?

Instruction fine-tuning involves providing input-output examples (like English-French translation pairs) to guide the model’s responses. Classification fine-tuning involves labeled examples for categorization tasks, such as labeling emails as spam or non-spam.

Why is pre-training computationally expensive?

Pre-training requires massive datasets and substantial computational power, often involving powerful GPUs or specialized hardware. For example, pre-training GPT-3 cost approximately $4.6 million due to the size of the data and model parameters.

What is meant by 'raw text' in the context of LLM training?

Raw text refers to unlabeled text data from diverse sources such as the internet, books, Wikipedia, and forums. It does not contain predefined labels or annotations and is used for pre-training foundational models.

How does pre-training a model on next-word prediction enable it to perform other tasks?

Training on next-word prediction helps the model learn language structure, grammar, and semantic relationships. This underlying knowledge enables the LLM to generalize to other tasks such as summarization, question answering, and translation, even without direct training for those tasks.

What are some practical applications of fine-tuned LLMs in different industries?

Fine-tuned LLMs can be used for customer support chatbots (telecommunications), AI legal assistants (law firms), financial analysis tools (banking), educational tools for generating quizzes, classification systems, summarization assistants, and translation applications.

Why is the term 'pre-training' used instead of just 'training'?

The term 'pre-training' is used because it refers to the initial stage of training a model on a large general dataset before fine-tuning it for specific tasks. It emphasizes that further refinement (fine-tuning) is usually required for production-level applications.