When you want to customize how an AI model behaves — to make it respond in a certain tone, focus on a specific domain, follow a particular format, or avoid certain topics — you have two fundamentally different levers to pull: prompting and fine-tuning. They operate at completely different levels and are suited to completely different situations.

Most people who interact with AI have only ever used prompting, sometimes without thinking of it as such. Fine-tuning is less commonly understood, often misdescribed, and frequently reached for when it isn't actually the right tool. Understanding both clearly — and knowing when to use each — is increasingly valuable as AI gets woven into more products and workflows.

Prompting: Guiding the Model at Runtime

A prompt is everything you send to the model as input. For most users, that's just a question or instruction. But for developers and power users, prompts have more structure: a system prompt (background instructions the model sees before anything else) plus the actual conversation.

Through prompting, you can tell the model who it is, what it should and shouldn't do, what format to use, what persona to adopt, and what knowledge to prioritize. This all happens at inference time — each time the model is called — and it changes nothing about the model's underlying weights.

The model you're prompting today is the same model someone else is prompting with completely different instructions. Your prompt shapes behavior within a conversation; it doesn't persist beyond it, and it doesn't affect anyone else's experience with the same model.

Prompting shapes behavior at runtime — nothing about the model changes. Fine-tuning updates the model's weights, permanently altering how it behaves.

Fine-tuning: Changing the Model Itself

Fine-tuning is a form of continued training. You start with a pretrained model and run additional training on a curated dataset — typically examples of the specific input-output behavior you want. The training process updates the model's internal weights to make it more likely to produce outputs like the ones in your training examples.

The result is a new, distinct model. The base model's general capabilities remain (you haven't overwritten its training from scratch), but it has been nudged — sometimes significantly — toward the style, format, domain, or behavior you encoded in your training data.

Fine-tuned models are persistent: the behavior is baked in and doesn't require you to include long instructions in every prompt. A fine-tuned customer support model will respond in your company's voice without you having to tell it to every time.

When to Use Each

The right choice depends on what you're trying to accomplish.

Use prompting when:

  • You need flexibility. Prompting is trivially updatable — change a line of text and you've changed the behavior. Fine-tuned models require a new training run to update.
  • You're iterating quickly. Prompt engineering takes minutes. Fine-tuning takes hours to days and costs real money.
  • The behavior you want is something a clear, well-written instruction can capture. "Always respond in three bullet points" or "You are a helpful assistant for a law firm; always recommend consulting an attorney" are prompt jobs, not fine-tuning jobs.
  • You need to inject factual knowledge. Fine-tuning doesn't reliably transfer specific facts — the model may learn a general style but hallucinate the details. For factual grounding, use retrieval-augmented generation (RAG) instead.

Use fine-tuning when:

  • You need consistent style or format that would otherwise require a very long system prompt every time. Fine-tuning can embed that style into the model, reducing prompt length and improving consistency.
  • You're working in a specialized domain with its own terminology and conventions — medical, legal, scientific, technical — and the base model doesn't handle it well.
  • You have high-volume inference where prompt length adds meaningful cost. A shorter prompt (because behavior is baked in) reduces tokens and thus cost at scale.
  • You need the model to produce outputs in a very specific structure or follow rules the model keeps breaking despite clear prompting.

Few-Shot Prompting: The Middle Ground

Between zero-shot prompting (just giving instructions) and fine-tuning sits few-shot prompting: including a handful of examples in the prompt itself. You show the model two or three (or ten) examples of the input-output pattern you want, and it generalizes from those examples.

Few-shot prompting is often astonishingly effective. For many tasks where you might reflexively reach for fine-tuning, a carefully chosen set of three to five examples in the prompt gets you 80-90% of the way there at a fraction of the effort. Before investing in fine-tuning, it's almost always worth trying few-shot prompting first.

The downside: examples take up tokens, and at scale those tokens cost money. If you're running millions of queries with twenty examples each, the token cost may eventually justify fine-tuning to bake the behavior in.

What Fine-tuning Cannot Do

This is perhaps the most important thing to understand, because it's widely misunderstood: fine-tuning does not reliably teach a model new facts.

If you fine-tune a model on your internal documents, hoping it will then "know" those documents, you will likely be disappointed. The model may pick up stylistic patterns from your documents, or learn that certain question types tend to appear in certain contexts, but it will not reliably memorize and accurately recall the factual contents. When pushed, it will hallucinate.

This is a consequence of how fine-tuning works: it adjusts weights to make certain output patterns more probable, but it doesn't store facts the way a database stores rows. For factual knowledge injection, the right tool is RAG — retrieving the relevant documents at query time and including them in the context — not fine-tuning.

Fine-tuning excels at style, format, and behavior. RAG excels at knowledge. Prompting is the first thing to try for both. Understanding this division prevents a lot of expensive mistakes.

The Practical Decision Tree

If you're trying to customize an AI's behavior, here's a sensible order of operations:

  1. Prompt first. Can a clear system prompt and a few examples get you what you need? Try this before anything else. For most use cases, the answer is yes.
  2. Add few-shot examples. If zero-shot prompting isn't working, add two to ten examples of the behavior you want. This often closes the gap dramatically.
  3. Use RAG if you need factual grounding. If the model needs to answer questions from specific documents, retrieve those documents at query time rather than trying to bake them in through training.
  4. Fine-tune if you've exhausted the above. If you have a persistent style/format/behavior problem that prompting can't solve, or you need to reduce prompt length at scale, fine-tuning makes sense. But it's a bigger investment and should be a deliberate choice, not a first instinct.

The AI landscape has a tendency to make fine-tuning sound more magical and necessary than it is. The models available today are so capable that thoughtful prompting handles an enormous range of customization needs. Most of the time, the better prompt is more valuable than the fine-tuned model — and it costs a lot less to iterate on.