RAG vs Fine-Tuning vs Prompt Engineering: Optimizing AI Models

High-level summary

The video compares three ways to improve LLM responses — Retrieval‑Augmented Generation (RAG), fine‑tuning, and prompt engineering — explaining how each works, their strengths, costs/limitations, and when to combine them.

Retrieval‑Augmented Generation (RAG)

Pipeline: retrieval → augmentation → generation

What it is

• A query triggers a search of an external corpus. Relevant passages are retrieved, appended to the prompt, and the LLM generates an answer using that enriched context.

Core technology

• Vector embeddings: convert documents and queries into numeric vectors.
• Semantic search: finds conceptually similar documents even without keyword matches.
• Vector database: stores embeddings and supports similarity search.

Benefits

• Provides up‑to‑date and domain‑specific facts.
• Extends an LLM’s knowledge without retraining.

Costs and limits

• Added latency and compute per query (the retrieval step).
• Infrastructure and storage for embeddings and the vector DB.
• Embedding/indexing costs and a more complex architecture.

Fine‑tuning

What it is

• Continue supervised training of a pretrained model on a focused dataset (input–output pairs) so the model’s internal weights adapt to a domain.

Core technology

• Backpropagation on additional labeled examples; requires curated training data.

Benefits

• Produces deep domain expertise baked into the model.
• Very fast at inference since no retrieval step or external DB is required.

Costs and limits

• Needs many high‑quality training examples and significant GPU/compute resources.
• Ongoing maintenance (retraining to update knowledge).
• Risk of catastrophic forgetting (loss of some general capabilities).

Prompt engineering

What it is

• Crafting prompts (instructions, examples, formatting, role/specifiers, chain‑of‑thought cues) to better activate patterns the LLM already learned — no model changes or extra data retrieval.

Core practice

• Use examples, explicit formatting, step‑by‑step or “think aloud” instructions, and constraints to guide model attention.

Benefits

• Immediate results with no backend or infrastructure changes.
• Flexible and low‑cost to try.

Costs and limits

• Trial‑and‑error process.
• Cannot add truly new facts or up‑to‑date information beyond the model’s training.
• Limited by the model’s existing knowledge and capabilities.

Practical guidance — when to use which

• Start with prompt engineering for quick improvements, formatting fixes, or clarifying intent.
• Use RAG when you need current facts or access to proprietary/evolving documents without retraining.
• Use fine‑tuning when you need consistently deep, repeated domain expertise (e.g., specialized support or firm policy) and can invest in training and maintenance.
• Common pattern: combine all three. Example — legal AI:
  • RAG retrieves recent cases,
  • fine‑tuning encodes firm‑specific policies,
  • prompt engineering enforces legal‑document formats.

Actionable steps (compact)

1. RAG

   • Embed corpus.
   • Store embeddings in a vector DB.
   • Perform semantic search per query.
   • Append retrieved context to the prompt.
   • Call the LLM to generate the answer.

2. Fine‑tuning

   • Collect curated input–output pairs.
   • Train on the focused dataset.
   • Validate for desired domain behavior.
   • Deploy and plan periodic retraining to keep knowledge current.

3. Prompt engineering

   • Provide a role/specifier and explicit constraints.
   • Include examples and the desired output format.
   • Use chain‑of‑thought or step‑by‑step prompts when helpful.
   • Iterate and refine based on outputs.

Main speaker / sources

• Video presenter / narrator (unnamed in subtitles).
• Example person used in the explainer: Martin Keen (illustrative case).