An example is a specification you do not have to describe
Three lines each, and they have pinned down the output format, the vocabulary, the granularity and the tie-breaking — all things that would take a paragraph of prose to say and would still be ambiguous.
The trailing Category: matters: ending a prompt mid-pattern makes the next token overwhelmingly likely to be the thing you want, because the model is completing a sequence.
Classify the message. Reply with one word.
Message: the door arrived with a crack down the middle
Category: damage
Message: I still have not received my order
Category: delivery
Message: the measurements on the invoice are wrong
Category: billing
Message: {{input}}
Category:What examples are good and bad at
The rule of thumb: if you can state it in one sentence, state it. Examples are for the things that are easier to show than to describe — and for the edge cases, which is where most of their value actually is.
| Good at | Bad at |
|---|---|
| Fixing the output format | Teaching a rule with many cases — say the rule |
| Showing the granularity of a judgement | Adding knowledge the model does not have |
| Demonstrating edge cases and how to handle them | Long tasks — each example costs its tokens on every call |
| Establishing tone and level of detail | Anything a schema can enforce more cheaply |
Choosing the examples
- Cover the classes. If one category never appears in the examples, it will be chosen far less often than it should be. Label bias is real and it is measurable.
- Include the hard cases. An example of a message that is genuinely ambiguous, with the answer you want, is worth five easy ones.
- Keep them consistent. Two examples that contradict each other are worse than none: the model has to guess which convention you meant.
- Make them correct. A typo or a wrong label in an example propagates into the output, and it is very hard to spot afterwards.
- Use real inputs. Invented examples are tidier than production data, which is exactly the problem.
Order matters more than it should
Models are sensitive to the order of examples, and to the order of options in a list — a well-documented effect called recency bias. With classification examples, the last label shown is chosen slightly more often; with multiple choice, the last option is favoured.
Two defences, both cheap: balance the examples so no label sits last by default, and shuffle deterministically by a seed so at least the bias is stable and testable rather than accidental. If a reordering changes your evaluation score noticeably, you have discovered that the prompt is fragile, which is worth knowing before production does.
Dynamic selection
Retrieving the nearest examples instead of using a fixed set is measurably better on varied inputs — it is retrieval-augmented prompting, and it reuses the machinery from the embeddings lesson.
The cost is a retrieval step per call and a pool to maintain. Start with a fixed, well-chosen set; move to selection when the evaluation shows the fixed set failing on a class of inputs it does not cover.
const pool = /* a hundred labelled examples */;
const query = embed(input);
const chosen = pool
.map((example) => ({ example, score: cosine(query, embed(example.input)) }))
.sort((a, b) => b.score - a.score)
.slice(0, 3)
.map((x) => x.example);