Vector search

Exact search is a loop

That is the whole algorithm, and for up to a few hundred thousand vectors it is genuinely fine — it is O(n × d), it vectorises well, and it is exact. A great many production systems that reach for a vector database would have been better served by this loop and a normalised array.

Normalising at index time is the one optimisation worth doing up front: for unit vectors the cosine is the dot product, and you have removed two square roots from every comparison.

function search(query, index, k) {
  const q = normalise(embed(query));

  return index
    .map((entry) => ({ entry, score: dot(q, entry.vector) }))   // unit vectors: dot = cosine
    .sort((a, b) => b.score - a.score)
    .slice(0, k);
}

Approximate nearest neighbours

The knob is always the same shape: search more, get better results, take longer. HNSW calls it ef_search; IVF calls it nprobe. It is not a constant you can set once — the right value depends on your data and your tolerance, and the only way to choose it is to measure.

Recall is the number to measure

Run it over a few hundred queries, take the mean, and you have the one number that says whether your index configuration is acceptable. Anything above about 0.95 is usually indistinguishable in the final answer; below 0.8 you are losing documents your retrieval was supposed to find, and no amount of prompt work will recover them.

The failure this catches is silent. An approximate index that drops the right chunk does not error — it returns four plausible neighbours, the model answers from them, and the answer is confidently wrong.

/* recall@k: of the true top k, how many did the approximate index return? */
const truth = exactSearch(query, index, 10).map((r) => r.id);
const got = approximateSearch(query, index, 10).map((r) => r.id);

const recall = got.filter((id) => truth.includes(id)).length / truth.length;

Filter first, where you can

Metadata filtering is not an optimisation, it is usually a correctness requirement: tenant isolation, permissions, date ranges, document types. Never rely on similarity to keep one customer’s documents away from another — filter, explicitly, before or during the search.

Beware the post-filter: retrieving the top 10 and then removing what the user may not see can leave you with two results, or none. Filter first, or ask the index for enough candidates that the filtered set is still large enough.

/* a pre-filter: cheap, exact, and it shrinks the search */
const candidates = index.filter((e) => e.tenant === user.tenant && e.date >= since);
const results = search(query, candidates, 10);

Choosing a store

• Under ~100 000 vectors: an array in memory, or pgvector in the database you already have. Do not add infrastructure for this.
• Already using Postgres: pgvector with an HNSW index, and your filters are plain SQL. This covers a great deal of production.
• Millions, with heavy filtering and scale requirements: a dedicated vector database earns its keep.
• Whatever you choose, store the original text and the metadata alongside the vector. A vector alone is useless — you need the text to put in the prompt.