Embeddings

A point in a few hundred dimensions

An embedding model maps a piece of text to a fixed-length list of numbers — typically 384 to 3 072 of them — chosen so that texts used in similar ways land near each other. That is the whole idea, and the qualifier matters: near means "appears in similar contexts", not "means the same thing".

The vector is opaque. No dimension corresponds to anything you can name; only the relationships between vectors carry information, which is why every operation you will do is a comparison.

Cosine similarity

Cosine measures the angle, ignoring length — which is what you want, because a long document and a short one about the same subject should be close. Dot product alone would favour whichever is longer.

In practice: normalise once, then use the dot product. For unit vectors the two are identical, and the dot product is a great deal cheaper when you are doing it a million times.

Absolute values are not meaningful. 0.82 is not "82 % similar"; it is only comparable with other scores from the same model. Thresholds must be calibrated per model, on your own data.

cosine(a, b) = dot(a, b) / (|a| × |b|)

 1.0   the same direction
 0.0   unrelated (orthogonal)
-1.0   opposite

What similarity actually captures

The negation case is the one to remember: a vector search cannot reliably tell "is broken" from "is not broken", because they appear in almost identical contexts. Anything where a single word flips the meaning — not, except, before, without — needs a keyword filter or a re-read, not a similarity score.

The asymmetry problem, and what fixes it

A question and its answer are written differently, so their embeddings are further apart than you would like. Three standard responses, in increasing order of effort:

• Instruction-prefixed models. Many models take "query: …" and "passage: …" prefixes and were trained to bridge the gap. Use them if your model supports them — it is free.
• HyDE. Ask the model to write a hypothetical answer, embed that, and search with it. Answer-to-answer similarity is much better than question-to-answer.
• Embed a generated question. For each chunk, generate the question it answers, and index that alongside the text.

And the simplest answer of all: pair vector search with keyword search, which has no asymmetry problem at all. That is the hybrid-retrieval lesson.

Practical notes

The embed in these labs is a stand-in: a deterministic hashed bag of words. It captures word overlap, not meaning, so "car" and "automobile" are unrelated to it. Every operation built on it — cosine, ranking, thresholds, fusion — is identical to what you would write against a real model, which is the part these labs are teaching.

• Cache them. Embedding is a network call and the result is deterministic for a given model and text. Key the cache on the model and a hash of the text.
• Re-embed everything when you change model. Vectors from two models are not comparable — not "slightly off", meaningless.
• Batch. Embedding 1 000 texts in one call is far cheaper and faster than 1 000 calls.
• Watch the dimensions. 3 072 floats is 12 KB per chunk; a million chunks is 12 GB. Several models support shortening the vector (Matryoshka) with a small accuracy loss.