RAG, explained like we build it.

By Mokshify Engineering · Reviewed by the team · Updated 17 Jul 2026 · 7 min read

The problem

A language model knows what it was trained on - not your price list, your policies, or what changed last Tuesday. Ask it anyway and it answers confidently from nothing. Retrieval-Augmented Generation fixes the knowledge problem without retraining: fetch the relevant truth first, then have the model answer from it, citing where it looked.

The pipeline, honestly

The diagrammed version lives in our gallery; here is what each stage actually decides:

Ingestion & chunking. Documents split into retrievable pieces. The unglamorous truth: chunking strategy (size, overlap, respecting structure like headings and tables) moves answer quality more than model choice. Bad chunks cannot be prompted away.
Embedding & storage. Chunks become vectors. We store them in pgvector inside the product’s existing PostgreSQL (ADR-003) - one database to operate, transactions with your product data, and no second store until scale genuinely demands one.
Retrieval. The query finds its passages. Pure vector similarity misses exact identifiers (invoice numbers, drug names, statute references) - hybrid retrieval (vector + keyword) is our default for anything operational, with metadata filters (tenant, date, document type) applied before similarity, which is also where tenancy isolation gets enforced.
Synthesis with citations. The model answers from the retrieved passages, each claim traceable to its source. Citations are not decoration - they are the difference between an answer a professional can act on and one they must re-verify.
Evaluation. The stage that separates products from demos - big enough to be its own article.

Advantages & disadvantages

Advantages: current knowledge without retraining; answers grounded in your documents; per-tenant knowledge isolation; auditability via citations. Disadvantages: retrieval quality caps answer quality (garbage retrieved, garbage generated); latency adds up across stages - async patterns matter; the index must be kept in sync with source documents, which is an ordinary data-engineering job people forget to budget.

When you do not need RAG

If answers rely on general knowledge, or the whole corpus fits comfortably in a context window and rarely changes - skip the pipeline and pass the documents directly. RAG earns its complexity at real corpus scale, real change rates, or real access-control needs. We say this in week-one feasibility spikes, sometimes to our own commercial disadvantage.

Technology selection

Python/FastAPI close to the models (ADR-002), pgvector for storage, provider-agnostic model access (ADR-008), Redis for caching frequent queries - the same boring platform, deliberately.