A vector index is necessary, not sufficient
Technology
A vector database answers one question well: what is semantically near this? That is the first move in retrieval, and on its own it is not the whole job. Nearness is not relevance, and an index cannot tell you when it has missed.
Pinecone holds the embeddings behind the assistant on this site. Content from our CMS is chunked, embedded, and indexed there, then reranked before anything reaches the model. This page is about where a managed vector index earns its cost, and where a Postgres table would have done the job. More on that architecture.
Retrieval
What happens between the question and the answer
- Chunk at boundaries, not at character counts
Splitting text by length produces chunks that start mid sentence and end mid table. We split at headers and logical sections, and every chunk carries a prefix naming the document and the section it came from, so the embedding is anchored to a place in the corpus instead of floating free.
- Retrieve wide, then rank hard
The index returns candidates, not answers. We pull a wide candidate set and combine semantic similarity with keyword search, because an exact product ID and a general question are different queries that fail in different ways. Neither one covers for the other.
- The reranker does the last mile
A first pass compares two frozen vectors, one of which squashed a whole paragraph into a single number. A cross encoder scores the query and the passage together, and late interaction models go token by token, which is how you catch the word not that flips a sentence.
The limits
Where a managed vector index earns its cost
- Vector only search has a ceiling
Semantic similarity typically stalls between 60 and 70 percent precision recall. That is not a tuning problem you can prompt your way past. If your target is above 90 percent, treat the vector index as one input to ranking rather than as the retrieval layer itself.
- Most projects do not need a separate store
Keeping facts and vectors in one engine removes the sync pipeline that causes data lag and drift. Published benchmarks put Postgres with pgvector and pgvectorscale well ahead on latency at fifty million vectors. Move to a specialized index above a hundred million, and only with a team to run the sync.
- A second index is another thing to keep true
A managed index is another source of truth. Every content change has to reach it, every deletion has to propagate, and permissions have to be enforced somewhere real. Budget for the reindex job and the drift it causes, not only for the query latency you were sold on.
The build
Where we run it
Writing
How we think about retrieval
Pinecone runs behind the assistant on this site, and by our own rule it does not have to. The corpus is our own CMS content, nowhere near a hundred million vectors. It has been there since the first version of the chatbot, and moving it would cost more than running it. That is a legacy reason, not an architectural one.
What a managed vector index buys you is not accuracy. It is not having to run the index. The accuracy comes from what happens before the query and after it: chunking at real section boundaries, pairing semantic search with keyword search, and letting a cross encoder decide the final order.
If your retrieval is stuck in the sixties, the fix is almost never a bigger index. It is the chunking, the hybrid query, and the order things come back in. Start a conversation.