This runs the offline pipeline: it reads the knowledge-base documents, splits them into overlapping fragments (chunks), turns each one into a vector (embedding) with the OpenAI API, and saves everything to a vector index on disk. This runs once, or whenever the documents change.
Why might a rebuilt index disappear on the public demo?
If you're running this on your own machine, the server is a single long-lived process with
a writable disk — click "Build index" and every question you ask afterward uses the fresh
index immediately, saved to index/ for next time too.
If you got here through the landing page's "Start the demo" button, your rebuild is
isolated to your own demo session — it never touches the shared index on
disk, and (when the public deployment has session storage configured) it's durable for 24
hours regardless of which serverless instance answers your next question, then it expires
automatically. That's a real guarantee, not best-effort.
Calling the API directly without a session (or on a deployment without session storage
configured) is different: the public deployment on Vercel runs as
serverless functions
with a read-only filesystem — so a rebuild there can't be saved to disk at all. Instead it
stays active in memory for that one server instance: your very next
question is likely to land on the same instance and see it (Vercel tends to reuse warm
instances under light traffic), but that's a best-effort, not a guarantee — a follow-up
question could land elsewhere and see the stable pre-built index instead. That's not a bug,
it's what "serverless" means; it's exactly why this demo also ships a pre-built index and a
pre-computed evaluation snapshot that
work regardless of which instance answers you.
index/ for next time too.Paste your OpenAI API key above first.
Click any box below to see exactly what it produced.
Output:
Documents and their chunks
Documents will appear here once you start building the index.
Embedding batches
Each chunk is sent to the OpenAI API in batches of up to 100.
Raw event log
This runs the online pipeline: your question becomes a vector, gets compared against every chunk in the index by cosine similarity, the best matches (top-K) are assembled into a prompt, and the LLM answers citing its source.
Build the index first, in the "Build the Index" tab.
Click any box below to see exactly what it produced.
Output:
Is this an AI agent that iterates until it's "done"?
Not in this mode. The pipeline above runs top to bottom exactly once, no loop-back: one embedding call, one search, one prompt, one generation call. The model never decides to search again, ask a follow-up, or judge whether it has "enough" context — that decision doesn't exist here. This is classic (non-agentic) RAG.
It's built with the raw OpenAI Python SDK — no LangChain, no LangGraph, no agent framework of any kind.
Switch to Agentic
RAG above to see a real version of this: retrieve() exposed
as a tool
the model can call zero, one, or several times, deciding for itself when it has enough
information before answering (the
ReAct pattern).
That trades predictability and cost for flexibility.
Your question, turned into a vector
The embedding has 1536 dimensions — the first 32 are shown here as a color strip by magnitude, just to make "it's a vector" tangible.
Similarity against every chunk in the index
Bar length = cosine similarity with your question. The highlighted ones are the top-K picked to build the context — everything else gets discarded.
Vector map — where your question landed
Every sphere is a chunk in the index, positioned in the same 3D space as your question (the gold sphere). The dashed lines connect it to the real top-K chunks — the ones actually used as context — highlighted in blue. Hover any sphere for its exact similarity (tap on touch devices, then use the button to make it the new reference point — a fresh search from that chunk's own stored vector, no extra OpenAI call needed). Drag to rotate, scroll or pinch to zoom, right-click-drag or two-finger-drag to pan.
The map will appear here once a search runs.
The assembled prompt
View full prompt (context + question)
Assistant's answer
Raw event log
Same question, different pipeline: the model has a retrieve_context tool
and decides for itself — via native OpenAI tool calling, no LangChain/LangGraph — whether
to call it zero, one, or several times (reformulating the query in between) before it has
enough context to answer. Bounded at
4 iterations so it always terminates. Each call
fetches fewer chunks than Classic mode's top-K, on purpose — it's what actually gives the
model a reason to search again.
The step-by-step trace will appear here once you ask a question in this mode.
Assistant's answer
Run summary
Raw event log
Classic vs. Agentic — same question
Appears once you've asked the exact same question in both modes.
Browse the raw material: pick a document, see exactly how it was cut into chunks (including the overlap between them), and — once the index is built — inspect the real vector stored for each chunk. Every code panel below shows the actual Python source that ran, fetched live from the server, not a copy that can drift out of sync.
1. What's actually stored in "the database"
2. Pick a document
Document
View raw text
3. How it was split into chunks
4. See the code
Real evaluation numbers for this exact index (not illustrative placeholders), the actual latency of the questions you've asked this session, and a glossary of RAG/AI-engineering concepts — including which ones this small demo deliberately doesn't need.
Evaluation metrics
The tiles below show a real run of the 10-question golden dataset against this exact index — either the snapshot committed to the repo, or a fresh live run if you click the button (~30 real OpenAI API calls: 10 question embeddings + 10 answers + 10 judge calls, takes roughly a minute, needs your own key on the public demo).
Build the index first, in the "Build the Index" tab.
Recall@K — per question
Faithfulness — per question
Raw event log
Embeddings vs. keyword search
Same questions, same index — the only thing that changes is how each chunk is scored:
cosine similarity between embedding vectors (semantic) vs. counting shared words after
dropping common Spanish stopwords (lexical, src/keyword_retriever.py). Rows
where they disagree are usually cases where the right document uses different words than
the question, or shares vocabulary with the wrong one.
Per question
Raw event log
Where the time goes
No question asked yet this session — ask one in the "Ask a Question" tab.
Concepts glossary
RAG (Retrieval Augmented Generation)
Instead of relying only on what the model memorized during training, RAG retrieves relevant text from an external knowledge base at request time and feeds it to the model as context before it answers. Every tab on this page demonstrates one half of that: retrieval (Build the Index, Explore) and generation (Ask a Question).
Chunking & embedding model
Documents are too long to embed or feed to an LLM whole, so they're split into
smaller, overlapping pieces (chunks), each turned into a vector by an embedding
model (text-embedding-3-small here). Chunk size and overlap are a real
design decision — see the Explore tab to see it happen on real documents.
Golden dataset
A hand-written set of representative questions paired with the source document each
one should retrieve, used to measure retrieval quality instead of guessing. This
demo's 10-question dataset lives in eval/golden_dataset.json and drives
the Recall@K score above.
LLM-as-a-judge
Using a second LLM call to grade the first one's output — here, checking whether a generated answer is actually supported by the retrieved context or invented something. Cheap and scales better than manual review, but imperfect: the judge can be wrong too, especially on borderline cases.
Fine-tuning vs RAG
Fine-tuning bakes new knowledge into the model's weights through additional training; RAG keeps the model frozen and injects knowledge at request time via retrieval. This demo uses RAG because the knowledge base needs to stay trivially editable (edit a markdown file, rebuild the index) and every answer needs to cite its exact source — much harder to guarantee with fine-tuning.
ReAct pattern (Reason + Act)
A prompting pattern where the model alternates between reasoning about what to do next, taking an action (like calling a tool), observing the result, and deciding whether to act again or answer — repeating until it decides it has enough. Used in this demo's Agentic RAG mode — ask something there and watch the trace.
Tool calling in agents
Giving an LLM a set of callable functions with typed arguments so it can decide
during generation which ones to invoke and with what input. This demo's
Agentic RAG mode exposes
retrieve() as exactly this kind of tool — the model calls it on its own
terms, zero or more times, instead of it always running first unconditionally like in
Classic mode. Real source in src/agentic_rag.py, via the Explore tab's
code viewer.
Not covered here, and why — these are real, useful concepts, just not ones a small local demo like this one has anything to compute:
- DORA metrics (deployment frequency, lead time, change failure rate, MTTR) measure software delivery performance for teams shipping continuously to production — this demo has no CI/CD pipeline or deployment history.
- CodeScene / code ownership analysis mines git history across many contributors to flag hotspots and knowledge silos in large, long-lived codebases — this is a single-author educational project.
- EU AI Act high-risk classification is a legal framework for classifying real production AI systems (a real healthtech assistant plausibly would need it) — there's no live regulatory classification to compute for a local synthetic demo with fake data.