← Back

RAG Essentials

·Bryan Lai
TechnicalRAGLLM

RAG Essentials

RAG is search plus context.

If search is bad, the answer will be bad.

The model cannot recover a document it never sees.

Rules

  1. Fix search before blaming the model.
  2. Lock down permissions before indexing company data.
  3. Clean the messy sources: email, Slack, docs, meetings, code.
  4. Start with text and tables.

Implementation

  1. Use one document shape. Every source should map to fields like title, body, type, owner, date, and permission.
  2. Index more than one source. Drive, Slack, tickets, docs, and code often hold different pieces of the answer.
  3. Respect company language. If the company has weird acronyms, product names, and customer terms, the search system must learn them.
  4. Split by domain. Product manuals, sales policy, and legal docs should not all use the same retrieval recipe.
  5. Route the query. A policy question should go to policy docs. A product question should go to product docs.
  6. Re-index when data changes. Old vectors point to old meaning.
  7. Use synthetic Q&A when data is thin. Generate likely questions from important docs, then test whether search finds the right source.
  8. Collect feedback. Clicks, dwell time, thumbs up, thumbs down, and "this did not answer my question" all matter.
  9. Log everything. Inputs, retrieved chunks, final answers, user feedback, model version, and timestamps.

When users say no, split the failure:

  1. Did search fetch the wrong source?
  2. Did the model use the right source badly?
  3. Did the user ask the wrong question?

Bottom line: bad retrieval kills good models.