aifromzero.dev
All 15 modules live

Fine-Tuning in Production — Build Anvil, a Tool-Calling Specialist Model (from data to the Hub)

Take one open model and one dataset all the way through the modern post-training pipeline — SFT, LoRA/QLoRA, DPO, GRPO, task-grounded evaluation, quantization, and serving — and ship a tool-calling specialist to the Hugging Face Hub, on a free GPU.

15 modules Intermediate Python (no ML background needed) Free, no paywall No GPU required
Start with Module 1 Companion labs ↗

About this course

This is a free, 15-module course for Python developers comfortable with LLM APIs who want to drop below the API and train their own model. Across the series you forge Anvil — a Qwen3-4B base model hammered into a production-grade tool-calling specialist — through the whole modern post-training pipeline: curate the data and apply chat templates, supervise-fine-tune it, make it fit a free 16 GB GPU with QLoRA, speed it up with Unsloth, prove it beats the base model with task-grounded evaluation, align it with DPO (and the ORPO/KTO/SimPO family), push its reasoning with GRPO and verifiable rewards, merge and quantize it, serve it with vLLM/Ollama, and publish it to the Hub with a complete model card.

Vendor-neutral and verified against the current stack (trl 1.6.0, transformers 5.12, peft 0.19) — the v1 idioms only (processing_class, max_length, stable DPO/GRPO), not the dead ones most tutorials still ship. The default path is free to learn on a Colab/Kaggle T4; a couple of heavy labs (GRPO) document an honest rented-GPU cost. Every lab lives in the companion repo with an offline test suite that runs the real trainers on a tiny model — no GPU, no network — so you can verify the harness before spending a credit.

Syllabus

  1. 01 When (Not) to Fine-Tune: The Post-Training Landscape (Fine-Tuning in Production, Module 1) 16 min read Read
  2. 02 Data Is the Model: Curation and Chat Templates (Fine-Tuning in Production, Module 2) 14 min read Read
  3. 03 Your First Fine-Tune: SFT from Scratch (Fine-Tuning in Production, Module 3) 16 min read Read
  4. 04 LoRA: Train 1% of the Weights (Fine-Tuning in Production, Module 4) 14 min read Read
  5. 05 Make It Fit a Free GPU: QLoRA from Scratch (Fine-Tuning in Production, Module 5) 14 min read Read
  6. 06 Go Faster: Unsloth and Config-as-Code (Axolotl) (Fine-Tuning in Production, Module 6) 13 min read Read
  7. 07 Is It Any Better? Task-Grounded Evaluation (Fine-Tuning in Production, Module 7) 14 min read Read
  8. 08 Align It: Preference Tuning with DPO (Fine-Tuning in Production, Module 8) 15 min read Read
  9. 09 The DPO Family: ORPO, KTO, SimPO — and When to Use Each (Fine-Tuning in Production, Module 9) 14 min read Read
  10. 10 Teach It to Reason: GRPO and Verifiable Rewards (Fine-Tuning in Production, Module 10) 16 min read Read
  11. 11 Free Capability Boosts: Model Merging with mergekit (Fine-Tuning in Production, Module 11) 13 min read Read
  12. 12 Shrink It for Serving: GGUF, AWQ, GPTQ (Fine-Tuning in Production, Module 12) 15 min read Read
  13. 13 Serve It: vLLM, Ollama, and TGI (Fine-Tuning in Production, Module 13) 15 min read Read
  14. 14 Ship It: Publishing, Model Cards, and Reproducibility (Fine-Tuning in Production, Module 14) 14 min read Read
  15. 15 Capstone: Anvil, End to End (Fine-Tuning in Production, Module 15) 17 min read Read