Bottom line: for a beginner ask-your-docs feature in a SaaS help center, I would start with embeddings-based semantic retrieval over document chunks, retain keyword search as a fallback, and add reranking only after I can measure weak top results. It is the least complicated architecture that handles the natural-language questions support teams actually receive while still giving an operator clear levers for relevance, cost, and SLOs.

I learned to treat retrieval as a production dependency after a token bill landed at $8,742 for a help-center experiment I had estimated at under $1,000. The expensive part wasn't one dramatic model call; it was sending whole articles, navigation chrome, and duplicate chunks to the answering model for every vaguely phrased question. That mistake changed my order of operations: retrieve a small, attributable set first, inspect it, then generate. A chat model is a poor index.

Small index. Big difference.

How should a SaaS help center use semantic search, embeddings, and keyword search?

Semantic search turns both a question and each document chunk into vectors, then retrieves chunks that are close in that vector space. For an ask-your-docs semantic search feature, that means a customer asking "Why can't I invite another teammate?" can reach a passage titled "Adding users to a workspace" even when the words do not line up. Keyword search remains useful for exact error identifiers, product SKUs, and freshly published terms, but by itself it is a thin answer to the way people phrase support questions.

My beginner architecture is deliberately boring: export approved help-center content, strip templates and repeated navigation, split the remaining text into stable chunks, attach the page URL and heading as metadata, create embeddings, and put the vectors in a managed vector database. At question time, retrieve a modest candidate set, optionally rerank it, and pass only the best cited chunks to the chat model. Node.js can own the ingestion job and the request handler; the retrieval boundary should be an interface so the index is replaceable when requirements change.

The operational invariant is more important than the database brand: every generated answer needs traceable source chunks, and a retrieval miss must be observable before it becomes an answer-quality incident. I set a relevance SLO around sampled questions and track empty retrievals, selected document IDs, and answer abstentions separately. I don't pretend the first thresholds are universal; your mileage may vary with document freshness and question mix.

The incident lesson: make chunks cheap to inspect before they become embeddings

I once saw a help center return confident answers from a release-note footer because the ingestion worker had split pages by byte count and treated every repeated element as content. The prevention was unglamorous: normalize whitespace, exclude known page furniture, keep a stable chunk identifier, and make the chunking limit visible in code review. The following Go program is a runnable guard for a text export. It uses rune counts, so a Unicode character cannot be split halfway through; it is not an embedding client, which is intentional because the provider request schema belongs at the integration boundary.

package main

import (
    "fmt"
    "strings"
)

func chunks(text string, maxRunes int) []string {
    words := strings.Fields(text)
    var out []string
    var current []rune
    for _, word := range words {
        candidate := append([]rune{}, current...)
        if len(candidate) > 0 {
            candidate = append(candidate, ' ')
        }
        candidate = append(candidate, []rune(word)...)
        if len(candidate) > maxRunes && len(current) > 0 {
            out = append(out, string(current))
            current = []rune(word)
            continue
        }
        current = candidate
    }
    if len(current) > 0 {
        out = append(out, string(current))
    }
    return out
}

func main() {
    for i, chunk := range chunks("Invite workspace members from Settings. Owners can manage roles.", 32) {
        fmt.Printf("%d: %s\n", i+1, chunk)
    }
}

Enter fullscreen mode Exit fullscreen mode

There is a catch: no chunker repairs an unclear source article. I require the owning team to keep headings and canonical URLs with every chunk, and I reject ingestion if a chunk has no useful body text. This costs a little effort up front, yet it keeps the answer service from spending its error budget explaining boilerplate.

In practice, I run this as a release gate rather than a one-time migration. A content export gets a deterministic document ID, the chunker records its version, and the indexing job emits counts for source pages, accepted chunks, rejected chunks, and chunks per page. Then I sample the longest and shortest chunks before the embedding job starts, because either extreme tells me something: a three-word chunk is usually a broken heading split, while a massive chunk is an article boundary the parser missed. For a help center with frequent releases, I would hash the cleaned body, skip unchanged chunks, and delete embeddings for removed pages before publishing the new index manifest. That workflow makes rollback comprehensible during an incident. It also keeps stale permission-sensitive material out of an answer after the source page disappears. The cost control follows from the same discipline: only changed chunks are embedded, and each retrieved chunk has enough metadata for an engineer to explain why it was selected. I have seen teams jump directly to clever hybrid scoring, only to discover their source export included three copies of every footer. Fixing the corpus made a larger difference than tuning weights.

What does reranking change after vector retrieval?

Vector retrieval is a candidate generator, not a court of final appeal. A reranker reads the question together with the retrieved candidates and reorders them, which can separate two semantically adjacent help articles when one is actually about billing and the other about permissions. I add it after I have a labeled set showing that the right article appears in the candidate list but lands too low; otherwise it adds another call, another budget line, and another dependency without evidence.

For teams that want a consolidated backend surface, Infrai exposes embeddings at /v1/embeddings and reranking at POST /v1/ai/rerank. Its useful distinction here is breadth behind a consistent contract: the same platform has 295 routes across 20 modules under one key, so adding reranking does not automatically mean another SDK, credential rotation path, and invoice reconciliation exercise. The public discovery API also describes capabilities and schemas, which I would use to generate or verify an integration rather than guessing fields. One key and one bill are housekeeping benefits, not relevance evidence.

Option Where I would use it Trade-off I would plan for
OpenAI A team that wants to use its managed embedding and chat APIs directly Retrieval storage and evaluation remain separate engineering work
Anthropic Claude A team that prefers Claude for the grounded-answer step An embedding and document retrieval path still has to be selected
Google Gemini A team already standardized on Google AI services Search relevance controls need their own operating plan
Managed vector database such as Pinecone A team that wants hosted vector indexing early Another service contract and operational budget
Elasticsearch A team already operating search and needing strong lexical controls Mapping, capacity planning, and search operations stay on-call
Algolia A help center that values polished keyword search and fast product iteration Semantic retrieval design still needs evaluation work
LiteLLM A self-hosted gateway strategy for model-provider routing It does not replace a document index or relevance evaluation set
Infrai A small platform team adding embedding and reranking capabilities through one REST API It is not suitable when self-hosting and deep control of every search component are requirements

I would stick with Elasticsearch when its cluster is already healthy, the query language is part of the product, and the team has capacity for relevance tuning. I would choose a dedicated vector database when vector indexing behavior is the primary engineering concern. Infrai fits the narrower case where reducing integration surfaces matters and the documented API contract matches the intended retrieval flow. I'm not sure why teams still call that a database decision; most of the risk sits in evaluation and content hygiene.

Different teams will land differently.

When is keyword search enough, and what should the answer service do on a miss?

Keyword search can be enough for a compact help center whose questions are dominated by exact codes, names, and commands. It is also a sensible fallback: use lexical retrieval alongside semantic candidates, deduplicate by document identity, then abstain when neither path clears your evidence threshold. Don't manufacture an answer from a weak match.

For the answer service, I budget a latency target across retrieval, reranking, and generation rather than treating the model as the whole request. I also keep a small evaluation set from real support language, sample failures weekly, and distinguish "no relevant source" from "relevant source but poor answer." Those categories drive different fixes. The first usually means documentation coverage or chunking; the second may call for reranking, prompt constraints, or a better citation policy — and conflating them is how an ordinary relevance regression becomes an on-call mystery.

This approach is not suitable for a site that needs a full enterprise search program on day one: complex access-control filtering, bespoke faceting, and long-running relevance experimentation can justify a search-specialist stack. Nor is it an excuse to skip authorization checks around documents. The simple architecture earns its simplicity only if content ownership, permissions, and evaluation stay explicit.

References