P1425 min
SFT for Agents
Most agent developers never need to fine-tune a model. But when prompt engineering hits its ceiling — inconsistent tool calls, wrong output formats, domain-specific failures — SFT (Supervised Fine-Tuning) becomes the next lever. This lesson teaches fine-tuning from the agent developer's perspective: not training models from scratch, but making existing models better at your specific agent tasks.
SDK Focustool schemasoutputFormatevaluation harnessstructured output
The Decision Framework
Before fine-tuning, exhaust cheaper alternatives. These rough USD ranges cover experiment-scale training spend only, excluding engineering, data preparation, evaluation, and ongoing inference:
text
Level 1: Better prompts Training spend: ~$0 Time: hours
→ System prompt, few-shot examples, chain-of-thought
Level 2: Better tools Training spend: ~$0 Time: hours
→ Clearer tool descriptions, stricter schemas, validation
Level 3: Better orchestration Training spend: ~$0 Time: days
→ Retry logic, fallback models, output parsing
Level 4: Fine-tuning (SFT) Training spend: ~$50-500 Time: days-weeks
→ Custom model weights for your specific task
Level 5: Preference training Training spend: ~$500+ Time: weeks
→ Preference-aligned behavior (DPO, GRPO, SimPO, KTO, ORPO)When Fine-Tuning Makes Sense
| Signal | Example | Why SFT Helps |
|---|---|---|
| Consistent format failures | Model keeps returning markdown instead of JSON | SFT bakes the output format into weights |
| Tool selection errors | Model calls search when it should call database_query | SFT learns tool routing patterns |
| Domain vocabulary gaps | Medical/legal/financial terms misunderstood | SFT teaches domain-specific language |
| Latency requirements | Need faster responses than few-shot prompting allows | Smaller fine-tuned model replaces large model + long prompt |
| Cost at scale | Paying for 2000-token system prompts on every call | Fine-tuned model needs minimal prompting |
When Fine-Tuning Doesn't Help
- Fresh knowledge gaps — Use retrieval when facts must stay current
- Capability gaps — SFT can shape behavior but may not create missing base capabilities
- Rare edge cases — Add targeted examples only after measuring whether they change behavior
- Rapidly changing requirements — Prompts are easier to update than retraining
Tool-Calling Data Construction
The hardest part of SFT for agents is building high-quality training data. You need examples of correct tool usage in realistic scenarios.