...customize a model
Jump to navigation
Jump to search
| AI Solver map |
|---|
|
Your current page is in bold
0 · Start
1 · Predict & analyze
tables, numbers, events
Predict values
Predict categories Forecast time series Cluster / segment Find outliers Make recommendations Find key drivers 2 · Understand unstructured data
text, documents, images, audio
3 · Generate & converse
4 · Decide & act
5 · Build it right
applies to every path
|
YouTube ... Quora ...Google search ...Google News ...Bing News
- AI Solver ... Fine-tuning ... RAG ... Embedding ... ...choose a model ... ...evaluate and trust AI
Use this page when a general model is close but not good enough for your task. Climb the ladder one rung at a time, and only when your evals show a gap the current rung cannot close.
The customization ladder
- Prompt and context engineering (hours) — clear instructions, a few good examples, an output schema, the right reasoning effort, and the right context: documents, tool results and memory. Most projects stop here.
- Retrieval (RAG) (days) — when the model lacks knowledge, the knowledge changes, or you need citations → ...answer questions from my data
- Tools and skills (days) — when the model must act or calculate: APIs and MCP connectors, code execution, packaged skills → ...build an agent
- Fine-tuning (days to weeks) — when you need consistent behavior, format or tone, domain style, or a smaller model that matches a bigger one on your task → Fine-tuning
- Supervised fine-tuning with parameter-efficient methods (LoRA, QLoRA): typically hundreds to thousands of high-quality examples
- Preference tuning (e.g., DPO): when you have pairs of better and worse answers
- Reinforcement fine-tuning: when answers can be checked automatically (math, code, extraction against a key); the model learns from a grader
- Distillation and compression (weeks) — train a small "student" on a large "teacher's" outputs; then quantize or prune → ...deploy AI
- Continued pretraining or training from scratch (months) — new languages or very specialized domains at large scale; rarely worth it today
Rules of thumb
- Fine-tuning teaches form and skills more reliably than facts. For facts that change, use retrieval.
- Fix the data before the model. A few hundred excellent examples beat thousands of noisy ones.
- Always compare against the best prompted baseline on the same eval set; fine-tuning that does not beat it is cost without benefit.
- Combine rungs: a fine-tuned small model with retrieval is a common production pattern.
- Classic deep learning follows the same idea: start from a pretrained network and fine-tune it on your images or audio (transfer learning).
Quick picks
| Symptom | Try |
|---|---|
| Wrong format or tone | Better instructions and examples → supervised fine-tuning |
| Missing or outdated facts | RAG |
| Cannot take actions | Tools / MCP connectors |
| Good but too slow or expensive | Distill into a smaller model; route easy requests |
| Inconsistent on checkable tasks | Reinforcement fine-tuning with a grader |
| Unknown language or highly specialized jargon | Continued pretraining (last resort) |
Real-world examples
- Build or adapt? — in 2023 Bloomberg trained BloombergGPT, a 50-billion-parameter model on financial and general text. BloombergGPT | arXiv A later study found that a general model (GPT-4) was competitive with, and on several tasks better than, domain-specific financial models, though fine-tuned models still won on others. Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? | arXiv Try the higher rungs before building your own model.
- Adapters on the device — Apple's Foundation Models framework lets app developers add small LoRA adapters to its on-device model for app-specific tasks. Foundation Models framework | Apple Developer
Watch out
- Overfitting — a fine-tuned model can lose general skills; keep a broad regression test set.
- Data rights and privacy — only train on data you may use; remove personal data you do not need.
- Maintenance — every new base model means re-running (and maybe redoing) your fine-tune.
Learn more
- PEFT: parameter-efficient fine-tuning | Hugging Face
- TRL: transformer reinforcement learning (SFT, DPO, RL) | Hugging Face
- LLM course | Hugging Face
- Fine-tuning ... RAG ... Embedding
Where next? ◀ Build it right · ...choose a model · ...answer questions from my data · ...deploy AI · ▶ ...evaluate and trust AI