How retrieval works
When a visitor asks your chatbot a question, Promptly doesn't send your whole knowledge base to the AI model. It retrieves the handful of most relevant passages and answers from those. Understanding this pipeline is the difference between writing content the bot can find and writing content that sits in the index untouched.
Embeddings: search by meaning
Every chunk of your knowledge base is converted into an embedding — a 1024-dimension vector that captures its meaning — using the multilingual BAAI/bge-m3 model. The visitor's question is embedded the same way, and Promptly finds the chunks whose vectors sit closest to it.
Because this matches meaning rather than characters, "how long until my order shows up?" can match a passage headed Shipping times even though the two share no words at all. The model is multilingual, so a question asked in one language can match content written in another — useful if your site is English but your customers aren't.
🖼️ [Image] — A diagram: a question and several passages plotted as points, with the nearest ones highlighted as matches.
Hybrid search: meaning plus exact words
Meaning-based search has a blind spot. Ask for part number X-4471B and a vector search will happily return passages that are about part numbers, because that's what the question means. So Promptly runs two searches over your knowledge base for every question:
- Keyword search — Postgres full-text search over the chunk text, which is what reliably finds exact terms: SKUs, model numbers, product names, error codes.
- Vector search — meaning-based matching over the embeddings above.
Each produces its own ranked list. They're merged with Reciprocal Rank Fusion (RRF), which scores each passage by its position in each list rather than by raw scores — that way two incomparable ranking systems can be combined fairly, and a passage that lands near the top of either list survives into the final set. Both methods carry equal weight.
The practical effect is that the same knowledge base serves both "I pasted the exact part number" and "I described my problem in my own words," without you having to write two versions of anything.
Reranking: a second, closer look
Fusion is fast but shallow — it only knows about positions in two lists. So the top candidates then go through a reranker (the multilingual bge-reranker-v2-m3 model), which reads each candidate passage together with the question and scores how well it actually answers it. This catches passages that ranked well on surface similarity but don't really respond to what was asked, and promotes ones that do.
Reranking is part of the Chat module — it isn't something you enable or pay extra for. If the reranking service is briefly unavailable, retrieval falls back to the fusion order rather than failing the conversation, so you may see slightly less precise answers rather than an outage.
How enrichment widens the net
The metadata generated during enrichment is indexed alongside your text, giving retrieval extra ways to connect a question to the right item:
- Hypothetical questions are embedded, so a visitor's phrasing can match a generated question even when it matches nothing in your prose.
- Summaries are embedded, so broad "what do you do about X?" queries have something appropriately broad to match.
- Keywords strengthen the keyword side of hybrid search.
This is why an enriched knowledge base noticeably outperforms the same content raw.
What actually reaches the model
By default the top 10 chunks survive to the answering step, and only those are given to the model as context. That number is configurable per workspace, and it's a genuine trade-off rather than a bigger-is-better dial: more chunks mean broader coverage but more tokens per answer, more competing material for the model to weigh, and a slower reply.
The bot then composes its answer from those passages, not from general world knowledge. If nothing relevant came back, it returns your configured fallback message instead of improvising — Promptly applies grounding checks to keep answers tied to your content. That's the no-hallucination rule in mechanical terms: an answer the retrieval step didn't supply is an answer the bot won't give.
What this means for your content
- If it's not in the knowledge base, the bot cannot answer it. No amount of prompt tuning substitutes for the missing paragraph. Add it — Adding knowledge.
- Write the question, not just the topic. A heading phrased the way a customer would ask gives both search methods a target. "Logistics" gives neither one much.
- Keep identifiers verbatim. Paraphrase is handled for you; exact strings are not invented for you.
- One idea per section. Chunking splits long text, and a chunk that mixes three unrelated policies matches all three questions weakly instead of one strongly.
- Gaps are measurable. Questions that retrieved nothing are logged as Content Gaps in Analytics — that list is your content backlog, already prioritised by real demand.
🎬 [Video] — Asking the widget a question and seeing which knowledge items the answer was grounded in.
Where retrieval sits in the bigger loop
Retrieval is one step, not the whole story. Intent routing decides whether a message needs a knowledge lookup at all — a greeting doesn't. Multi-step reasoning can chain a retrieval with a catalog lookup or a custom tool before answering. And because that planning happens through tool calls, the model behind your key has to support them — see Model roles.