Course 17 · Craft
AI engineering
Building on a model you do not control.
The parts of an LLM application that are engineering rather than magic: tokens, sampling, retrieval, tools, evaluation and cost — all of them computed and checked in the page.
0 of 24 lessons finished — nothing yet
The syllabus
24 lessons,
in the order they build on each other.
What a model actually does
Next-token prediction, and the five consequences that explain almost every surprise.
Tokens
Why a word is not a token, what that costs, and the cases where it changes the answer.
Sampling
Temperature, top-k and top-p — the knobs, computed by hand, on a real distribution.
The context window
A budget in tokens: what goes in, what gets dropped, and how to decide.
Writing the prompt
Structure, specificity and the instructions that actually change the output.
Examples
Choosing and ordering the demonstrations, and when they beat an instruction.
Structured output
A schema, a validator, and what to do with the reply that does not fit it.
Tool calling
The declaration, the loop, and validating arguments you did not write.
Agents
The loop with a budget: termination, state, and the four ways it goes wrong.
Embeddings
Vectors, cosine similarity, and exactly what "similar" turns out to mean.
Chunking
Size, overlap and boundaries — the decision that quietly decides retrieval quality.
Vector search
Nearest neighbours, approximate indexes, and measuring recall.
Keyword search
BM25, implemented — and the queries where it beats embeddings outright.
Hybrid retrieval
Fusing two rankings without comparing their scores, then reranking.
Retrieval-augmented generation
The whole pipeline, and the six places it fails in production.
Context engineering
Ordering, compression and the middle of a long prompt that nobody reads.
Evaluation
A dataset, a metric and a number — exact match, F1 and what they miss.
Using a model as a judge
Rubrics, pairwise comparison, and the biases that make a judge useless.
Evals in the pipeline
A gate on every change: drift, flakiness, and the golden set.
Cost and latency
Token accounting, caching, batching, and where the time actually goes.
Streaming
Parsing server-sent events, partial JSON, and cancelling properly.
Reliability
Timeouts, retries with backoff, fallbacks and idempotency for a service that fails.
Injection and data handling
Prompt injection, untrusted content, PII and output filtering.
Practice: a RAG pipeline you can measure
Chunk, index, retrieve, answer — then an eval harness that proves a change helped.
Keep going
The other courses.
HTML
The document, the elements and the meaning behind them. Everything else on the web is layered on top of this.
CSS
Selectors, the box model, layout with flexbox and grid, and motion that has a reason to move.
JavaScript
Values, functions, the DOM, events and asynchronous work — written so you can debug it later.