05Production RAG system
Overview
Production-grade Retrieval-Augmented Generation platform. Upload any PDF, chat with it using hybrid search (dense + sparse + RRF fusion), and get cited answers. The system auto-generates evaluation datasets on ingestion, runs quality harnesses (exact match, token F1), and tracks per-query observability — latency percentiles, token usage, and cost. Built with FastAPI, Qdrant, PostgreSQL, and a React + TypeScript frontend. Demonstrates the full RAG lifecycle: ingestion, retrieval, generation, evaluation, and monitoring.
The challenge
A useful retrieval system needs more than a chat interface. Ingestion quality, retrieval behavior, citations, evaluation, latency, and cost all need to remain visible after deployment.
The approach
AuditRAG covers the complete retrieval lifecycle, from PDF ingestion through hybrid search and cited generation to automated evaluation and per-query observability.
01Process
How it works
- 01
Ingest
Parse, segment, and prepare uploaded documents.
- 02
Retrieve
Run dense and sparse search, then fuse the rankings.
- 03
Answer
Generate a response grounded in cited source material.
- 04
Evaluate
Measure answer quality and inspect operational signals.
02Decisions
Key decisions
- 01
Combine retrieval signals
Dense and sparse retrieval are fused with reciprocal rank fusion instead of relying on one search strategy.
- 02
Evaluate during ingestion
The system generates evaluation material as documents enter the pipeline, making quality checks part of the workflow.
- 03
Make operations inspectable
Latency percentiles, token usage, and cost are tracked alongside answer quality and citations.
Technologies
- Python
- FastAPI
- Qdrant
- PostgreSQL
- React
- TypeScript
- OpenAI
- Anthropic