RAG Starter Kit

Architecture

How it works

1. 01 / 06

   Ingest

   Documents (PDFs, DOCX, email exports) go through the FastAPI backend, get chunked with LlamaIndex, embedded with OpenAI's text-embedding-3-small, and stored in Postgres — each chunk's embedding lives in a pgvector Vector(1536) column next to the source document and page number it came from.

2. 02 / 06

   Ask

   The Next.js chat UI sends the question to FastAPI, which hands it to a LangGraph retrieval node — a small state graph rather than a single hard-coded function, so the retrieval logic can grow branches later without a rewrite.

3. 03 / 06

   Retrieve

   The node embeds the incoming question and runs a similarity search against pgvector to pull back the chunks most relevant to it.

4. 04 / 06

   Synthesize

   Those chunks go to the LLM with instructions to answer only from what was retrieved and attach a citation — source document and page — to every claim, so nothing in the answer is unsourced.

5. 05 / 06

   Trace

   Every step of that call — retrieval, synthesis, latency, token counts — is logged to Langfuse, so any single run can be pulled up and inspected after the fact.

6. 06 / 06

   Gate

   Before any change ships, a DeepEval golden set of 20-30 real question/answer pairs runs against the retrieval pipeline in CI. If precision drops below threshold, the build fails and the deploy is blocked — the eval isn't a report, it's a gate.