I built CiteWise to solve a problem I kept running into: LLMs are great at answering questions, but you can't trust an answer if you can't trace it back to a source. So I built a full-stack RAG (Retrieval-Augmented Generation) system that doesn't just answer questions from your documents — it shows you exactly which passage backed each claim, and then grades itself on how honest it's being.
Architecture,
Under the hood, it's a FastAPI backend that handles document ingestion, chunking, and retrieval, paired with a React + Tailwind frontend for uploading files and chatting with your docs in real time. I kept the embedding layer fully local using sentence-transformers, so there's no dependency on a paid embedding API — ChromaDB handles vector storage and similarity search on your own machine. The only external call is to the LLM itself, routed through OpenRouter, and I added a safeguard in the config so the app literally refuses to start if you try to point it at a paid model — the whole thing is designed to run on free tiers only.
Evaluation Layer,
The part I'm most proud of is the evaluation layer. Every answer gets scored on three axes — faithfulness (does it hallucinate?), answer relevance (is it actually on-topic?), and context precision (did retrieval pull the right chunks?) — using an LLM-as-judge approach combined with embedding similarity. I also built a benchmark suite that runs entirely offline with mocked fallbacks, so tests and CI never depend on network access or API keys.
Full Stack,
Along the way I worked through the full stack: REST API design with Pydantic validation, retry/backoff logic for flaky LLM calls, Docker + Docker Compose for deployment, and a three-panel React UI for upload, chat, and live evaluation results.
Tech Stack,
Python, FastAPI, LangChain, ChromaDB, sentence-transformers, React, Vite, Tailwind CSS, Docker, Pydantic, pytest.