Grounded Tours is an AI assistant for a tour operator. The operator is fictional, but the problem is a real one: a customer asks something like “a relaxed food trip in Italy in May under £2,000, and what if I cancel three weeks before?”, and the answer has to come from the operator’s own booking conditions, FAQ and itineraries, with a numbered source for every fact.
That makes search the part everything else depends on. The model can only cite what the search hands it. If the right passage isn’t in the results, the best case is “I couldn’t find that”, and the worst case is a confident answer about the wrong cancellation fee.
This post is about the search: full-text and vector search combined in a single Postgres query, how I measured it, and the first version that made things worse.
Two Kinds of Questions
The documents are 51 markdown files, split into 459 passages. Customers ask about them in two very different ways.
Some questions carry an exact term. “CXL-42” is a cancellation tier code. “Lagazuoi hut” is one night on one itinerary. Keyword search is built for this: the word is either in the passage or it isn’t.
Other questions share no words at all with the answer. “Is there internet access on the trip?” should find the FAQ entry headed “Is there wifi?”. Keyword search has nothing to match there, and that’s exactly what embeddings are for: they place passages with similar meaning close together, whatever the words.
So I built an evaluation set before choosing anything: 30 queries, each with the passage that should come back, in four groups (10 exact terms, 12 paraphrases, 5 mixed, 3 near-duplicates). Two numbers per run. Recall@5 asks whether a correct passage is in the top 5. MRR (mean reciprocal rank) scores 1 for a correct passage in first place, 0.5 for second, and so on, so it also rewards putting the answer at the top.
Full-text search alone, on the 12 paraphrases, scored an MRR of 0.08. It found one of them.
Why Both Live in Postgres
The tours and departures were already in Postgres, so I kept the documents there too. pgvector adds a vector column type and distance operators, and Postgres has had full-text search for years. One database means one place to back up, one connection pool, and filters (like “only policy documents”) applied in the same WHERE clause as the search itself, with no second service to keep in sync.
The table holds both representations of each passage. Trimmed from migrations/005_chunks.sql:
CREATE TABLE chunks (
-- ...
heading_path text NOT NULL,
content text NOT NULL,
embedding vector(768) NOT NULL,
-- Kept up to date by Postgres on every insert and update. Never written by our code.
tsv tsvector GENERATED ALWAYS AS
(to_tsvector('english', heading_path || ' ' || content)) STORED
);
CREATE INDEX chunks_tsv_idx ON chunks USING gin (tsv);
CREATE INDEX chunks_embedding_idx ON chunks USING hnsw (embedding vector_cosine_ops);
The generated tsv column means the application never has to remember to update it. The headings go into it too, because a passage’s heading is often the best keyword it has (“Is there wifi?” lives in the heading, not the body).
One Query, Two Lists, One Ranking
The hard part of hybrid search is merging. Full-text search gives you a ts_rank_cd score. Vector search gives you a cosine distance. They’re on different scales and mean different things, and any formula that adds them together needs weights you’ll be tuning forever.
Reciprocal Rank Fusion avoids that by ignoring the scores completely. Each list only contributes positions: a passage gets 1 / (60 + rank) from every list it appears in, and the totals decide the final order. The 60 is the constant from the original RRF paper, and it stops the top position from dominating.
Here’s the shape of the query, trimmed from search_documents.sql (the real one also handles a document-kind filter and a mode switch used by the evaluation):
WITH query AS (
SELECT websearch_to_tsquery('english', $1::text) AS q
),
fts AS (
SELECT c.id, row_number() OVER (ORDER BY ts_rank_cd(c.tsv, query.q) DESC, c.id) AS rank
FROM chunks c CROSS JOIN query
WHERE c.tsv @@ query.q
ORDER BY rank
LIMIT $6::integer
),
vec AS (
SELECT id, row_number() OVER (ORDER BY distance, id) AS rank
FROM (
-- ORDER BY distance ... LIMIT directly on the table is the shape the HNSW index can serve.
SELECT c.id, c.embedding <=> $2::vector AS distance
FROM chunks c
ORDER BY c.embedding <=> $2::vector
LIMIT $6::integer
) AS nearest
),
fused AS (
-- Only positions are used, never the raw scores.
SELECT id, sum(1.0 / (60 + rank)) AS score
FROM (
SELECT id, rank FROM fts
UNION ALL
SELECT id, rank FROM vec
) AS both_lists
GROUP BY id
)
SELECT ... FROM fused f JOIN chunks c ON c.id = f.id
ORDER BY f.score DESC, c.id
LIMIT $5::integer;
Each side takes 40 candidates, and the merged list returns the top 8 to the agent. The c.id tiebreaker keeps results deterministic, which matters more than it sounds when you’re comparing two evaluation runs.
There’s one Postgres detail worth knowing if you filter. An HNSW index finds the nearest neighbours first and filters afterwards, so with a filter like “only itineraries” it can come back with fewer rows than you asked for. pgvector 0.8 added iterative scans, which keep searching until enough rows pass the filter. From server/src/search/documents.ts:
await client.query('BEGIN');
// With a kind filter, an HNSW index scan may find too few matching rows (it looks at the
// nearest neighbours first, then filters). Iterative scan keeps looking until it has enough.
// SET LOCAL lasts only until the end of this transaction.
await client.query("SET LOCAL hnsw.iterative_scan = 'relaxed_order'");
SET LOCAL inside a transaction matters here: the setting dies with the transaction, so it can’t leak into whatever query the pooled connection runs next.
The First Version Lost to Vector Search
My first version didn’t use websearch_to_tsquery. That function requires every word of the question to match, and the comment I committed with the first version explained why I avoided it: almost no passage contains every word of a whole question. So it built an OR query instead. Any word matched.
It sounded reasonable. Then the evaluation ran:
| Search mode | Recall@5 | MRR |
|---|---|---|
| Full-text only (all words) | 0.53 | 0.48 |
| Vector only | 1.00 | 0.97 |
| Hybrid, any word (first version) | 0.90 | 0.81 |
| Hybrid, all words | 1.00 | 0.98 |
Hybrid search was worse than one of its own halves. And the per-query results showed where: on the paraphrases. “My suitcase never showed up at arrivals” was vector search’s first result, and the any-word hybrid didn’t have it in the top 10 at all. Same for “I can’t walk far and use a wheelchair”.
The cause is how RRF rewards agreement. With any word allowed, a passage only had to share one ordinary word with the question to make the keyword list, so that list was mostly noise. And a passage that shows up in both lists, even in mediocre positions, beats the best passage of a single list. The arithmetic: 5th place in both lists scores 2 / 65, about 0.031, while first place in only the vector list scores 1 / 61, about 0.016. Noise that appeared twice outranked the right answer that appeared once.
The part I didn’t expect: the any-word version made full-text search on its own better (MRR 0.65 against 0.48). The same change improved one list and broke the fusion.
The fix was the function I’d avoided. Requiring every word makes the keyword list short and exact for codes and place names, and empty for most paraphrases. An empty list isn’t a problem in RRF. It just means vector search decides alone, which is what you want when the words don’t match. Hybrid went to 0.98.
Both runs are committed in the repo’s eval/results folder (the -or-words files are the first version), and the commit that fixed it moved the reasoning into a comment at the top of the query, next to the line it explains.
Results
On the final version:
- Paraphrases: MRR 0.96 for hybrid, against 0.08 for full-text alone.
- Exact terms: MRR 1.00 for hybrid, against 0.95 for vector alone. “CXL-0 charge” comes back first instead of second.
- Overall: recall@5 1.00 and MRR 0.98, against 0.53 and 0.48 for full-text alone.
The tests run against a real Postgres with pgvector in GitHub Actions, using deterministic fake embeddings so they need no network. One of them checks the fusion arithmetic directly, so a future refactor can’t quietly change what “score” means. From server/test/search.test.ts:
it('scores with Reciprocal Rank Fusion: the sum of 1 / (60 + rank)', async () => {
const hits = await searchDocuments(embeddings, { query: 'single supplement', mode: 'hybrid' });
for (const hit of hits) {
const expected =
(hit.ftsRank ? 1 / (60 + hit.ftsRank) : 0) +
(hit.vectorRank ? 1 / (60 + hit.vectorRank) : 0);
expect(hit.score).toBeCloseTo(expected, 10);
}
});
There are tests for the fix too: full-text mode must return only passages containing every word, and a question made only of stop words must return nothing rather than an error.
Trade-offs
To be fair to vector search: on this data it was already very good on its own. Hybrid beats it on exactly one query, “CXL-0 charge”, which moves from second place to first, and that single query is the whole gap between 0.97 and 0.98. With 30 queries I wouldn’t read more into it than that. What hybrid buys here is a safety net on exact codes and names, the queries where a near miss means the model reads about the wrong fee first.
A dedicated search engine would give me BM25 ranking and more control over how codes like CXL-42 are tokenized. For 459 passages it would also give me a second service to deploy, monitor and keep in sync. Postgres was the right call at this size, and the query would be the first thing I’d revisit if the documents grew by a couple of orders of magnitude.
Not every table needs an index, either. The 40 tours also have embeddings, for ordering by “vibe”, and there’s no vector index on them. EXPLAIN ANALYZE shows a full scan of all 40 rows in 0.24 ms, and the result is exact, where HNSW is approximate.
Lessons Learned
Build the evaluation before you trust the feature. The whole retrieval set is one small JSON file: 30 queries, each with the passage it should find. It’s the only reason the first version didn’t stay. Without it I’d have had a hybrid search that felt more sophisticated and performed worse.
A plausible comment isn’t a measurement. The any-word reasoning was sensible. It was also wrong, and only a number could show that. I built this project with Claude Code as a pair programmer, setting the scope and architecture and checking every step, and this is the habit that made that work: anything the code claims, the evaluation checks.
Fusion rewards agreement, so watch what can agree. If one list is noisy, RRF doesn’t average the noise away. It promotes it.
When not to do this: if your users always search with exact terms (product codes, error messages), plain full-text search is simpler and you can skip the embeddings entirely. If they only ever ask in natural language, vector search alone may be enough. Hybrid earns its extra query when you get both kinds, as a booking assistant does.
The code is in the grounded-tours repository, and the live demo runs this exact query (it’s on a free tier, so the first visit can take up to a minute to wake up).