Agent Factory Recap: Reinforcement Learning and Fine-Tuning on TPUs

When to Consider Fine-Tuning

Timestamp: 3:13

We started with a fundamental question: with foundational models like Gemini becoming so powerful out of the box, and customization through the prompt can often be good enough, when should you consider fine-tuning? 

Fine tuning your own model is relevant when you need high specialization for unique datasets where a generalist model might not excel (such as in the medical domain), or when you have strict privacy restrictions that require hosting your own models trained on your data.

The Model Lifecycle: Pre-training and Post-training (SFT and RL)

Timestamp: 3:52 

Kyle used a great analogy inspired by Andrej Karpathy to break down the stages of training. He described pre-training as “knowledge acquisition,” similar to reading a chemistry textbook to learn how things work. Post-training is further split into Supervised Fine-Tuning (SFT), which is analogous to reading already-solved practice problems within the textbook chapter, and Reinforcement Learning (RL), which is like solving new practice problems without help and then checking your answers in the back of the book to measure yourself against an optimal approach and correct answers.