Fine-Tuning vs RAG vs Prompting: Choosing the Right Tool
From the course Fine-Tuning and Customizing Open-Source LLMs: LoRA, QLoRA and Self-Hosting
Built-in AI Professor Exclusive
Ask anything about the lesson and get an instant answer. The AI Professor knows the course content and helps you learn more effectively.
Before you touch a single line of training code, you need to answer a deceptively simple question: does your problem actually require fine-tuning at all? A large share of "we need a custom model" requests are solved faster, cheaper, and with less operational risk by better prompting or by retrieval-augmented generation (RAG). Fine-tuning is powerful, but it is also the option with the highest ongoing cost in engineering time, data curation, and maintenance. This lesson gives you a decision framework so you deploy the right tool for the right job.
The three techniques, precisely defined
Prompting (including few-shot examples and structured system prompts) changes model behavior purely through the input context. You do not modify any weights. You steer the model with instructions, examples, output-format specifications, and role framing. It is instant to iterate on and requires zero training infrastructure.
RAG (retrieval-augmented generation) adds an external knowledge source at inference time. You embed your documents into a vector store, retrieve the most relevant chunks for a given query, and inject them into the prompt. The model reasons over facts it was never trained on. RAG is the standard answer to "the model does not know my private or fresh data."
Fine-tuning updates the model weights (fully, or with parameter-efficient adapters such as LoRA) on your own examples. It changes the model's behavior and style durably: tone, formatting conventions, a domain-specific reasoning pattern, a niche output schema, or a language/register the base model handles poorly. It does not, by itself, reliably teach new facts.
The mental model that resolves most debates
Use this heuristic:
- Prompting changes how you ask.
- RAG changes what the model can see.
- Fine-tuning changes how the model behaves by default.
A frequent and expensive mistake is trying to fine-tune facts into a model. If your requirement is "the assistant must answer questions about our 4,000-page internal policy handbook and stay current as it changes weekly," that is a RAG problem, not a fine-tuning problem. Fine-tuning on those documents will produce a model that confidently paraphrases and hallucinates, and every handbook update would force a retrain. RAG lets you update the knowledge base by re-indexing, with no training at all.
Conversely, if your requirement is "every response must follow our strict clinical-note format, in a terse professional register, even under adversarial input," prompting may get you 90% of the way but drift on edge cases. Fine-tuning bakes the behavior in so you no longer spend hundreds of tokens per request re-teaching format, which also cuts latency and cost per call.
Comparison table
| Dimension | Prompting | RAG | Fine-tuning |
|---|---|---|---|
| Changes weights? | No | No | Yes |
| Teaches new/fresh facts | Weakly | Strongly | Poorly |
| Teaches style/format/behavior | Moderately | Weakly | Strongly |
| Time to first result | Minutes | Hours to days | Days to weeks |
| Upfront cost | Very low | Medium | High |
| Per-request token cost | High (long prompts) | High (retrieved context) | Low (behavior is internal) |
| Inference latency | Higher (long context) | Higher (retrieval + context) | Lower |
| Freshness / update speed | Instant | Fast (re-index) | Slow (retrain) |
| Data needed | A few examples | A document corpus | Curated example pairs |
| Best at | Quick steering, prototyping | Grounded factual answers | Consistent behavior, niche tasks |
Cost, latency, and freshness trade-offs
The token economics matter more than people expect. If you rely on a 1,500-token system prompt with ten few-shot examples on every call, you pay for those tokens on every request forever, and they inflate latency. Fine-tuning can move that behavior into the weights, shrinking prompts dramatically. Over millions of requests, the training cost amortizes quickly. This is a classic reason to fine-tune even when prompting "works."
Pick up exactly where you left off
Create your free account in under a minute, then pick the option that fits you best:
What's next in this lesson
- Combining them (the usual production answer)
- Measure before and after, always
- A worked example and common anti-patterns
- Decision checklist
- Sequencing the three techniques in a greenfield project
- Token economics: a worked example
- Three extended scenarios and the right call
- When RAG and fine-tuning genuinely conflict
Everything you'll learn in this course
1 When to Fine-Tune: Foundations & Decision Framework 3 lessons
- Fine-Tuning vs RAG vs Prompting: Choosing the Right Tool Reading now 50 min
- Anatomy of an Open-Source LLM 52 min
- The Open-Source Licensing Landscape 48 min
2 Data: The Foundation of Fine-Tuning 3 lessons
- Dataset Design: Formats, Structure, and Chat Templates 52 min
- Data Quality, Curation, and Synthetic Data 51 min
- Legal and Ethical Fine-Tuning Data 50 min
3 Parameter-Efficient Fine-Tuning (PEFT): LoRA 3 lessons
- How LoRA Works: Low-Rank Adaptation Explained 52 min
- LoRA Hyperparameters: Rank, Alpha, Target Modules, Dropout 51 min
- Full Fine-Tuning vs PEFT: Trade-offs 48 min
4 QLoRA and Quantized Training 3 lessons
- Quantization Fundamentals and bitsandbytes 50 min
- QLoRA in Practice: Fine-Tuning on a Single GPU 52 min
- Memory Math: VRAM Budgeting for Training 49 min
5 The Fine-Tuning Toolchain 3 lessons
- Hugging Face Stack: transformers, PEFT, and TRL 51 min
- Unsloth: Faster, Memory-Efficient Fine-Tuning 49 min
- Axolotl: Config-Driven Fine-Tuning at Scale 50 min
6 Instruction Tuning & Alignment 4 lessons
- Supervised Fine-Tuning and Chat Templates 52 min
- Preference Optimization: DPO and Beyond 51 min
- Common Pitfalls: Catastrophic Forgetting and Overfitting 49 min
- Reinforcement Fine-Tuning: RLHF, GRPO, and Verifiable Rewards 50 min
7 Evaluation and Quantization for Inference 3 lessons
- Evaluating a Fine-Tuned Model 51 min
- Inference Quantization: GGUF, GPTQ, and AWQ 51 min
- Merging Adapters and Exporting Models 48 min
8 Self-Hosting and Serving 4 lessons
- Ollama: Local Serving Made Simple 49 min
- vLLM: High-Throughput Production Serving 51 min
- TGI and Choosing a Serving Stack 49 min
- Serving Many Adapters: Multi-LoRA and Multi-Tenant Inference 50 min
9 Production, Cost, and Case Studies 3 lessons
- Cost and Hardware Planning 51 min
- Deploying to Production: Monitoring, Scaling, and Safety 52 min
- Case Studies: Real-World Fine-Tuning Projects 50 min
10 Final Quiz — Fine-Tuning Open-Source LLMs 1 lessons
- Final Assessment — Fine-Tuning and Customizing Open-Source LLMs 50 min
Everything you need to learn effectively
Interactive quizzes
Check your knowledge at the end of every lesson with scored quizzes and feedback.
Personal notes
Save notes on every lesson, accessible anytime from your dashboard.
Scheduled reviews
Revisit lessons exactly when it matters, at the right intervals — so you remember for the long term.
Progress & Achievements
Track your progress, unlock achievements, and visualize what you've learned.
Bookmarks
Save the lessons that matter and find them instantly when you need them.
Questions & Answers
Ask questions right on the lesson and get answers from our team.
Good to know before you start
How do I get access to the course?
You can read the beginning of the first lesson for free, right on this page. For the course you create an account, pick the subscription that fits — a single course or a bundle — and get access immediately after your payment is confirmed. Everything happens 100% online.
Can I cancel my subscription anytime?
Yes. Cancel anytime, straight from your account, in just a few clicks. Your access stays active until the end of the period you have already paid for.
What does the subscription for this course include?
All 30 lessons in the course, interactive quizzes, the AI professor built into every lesson (select any passage and it explains it on the spot), personal notes, automatically saved progress, and content updates included.
Is there a fixed learning schedule?
No. You learn at your own pace, on any device. Lessons are structured step by step, and the platform saves your progress automatically, so you can pick up right where you left off — anytime.
Ready to unlock all the content?
Just this course — €99 / month, VAT included — or every IT Pro course, with smart quizzes and the full AI Professor, in the bundle at €399 / month, VAT included.
