Evaluating RAG Without Ground Truth: Production Pitfalls of LLM-as-a-Judge
When deploying enterprise Retrieval-Augmented Generation (RAG) systems, the hardest operational problem is often not retrieval tuning or prompt engineering; it is evaluation. In academic settings, benchmarks like MS MARCO or HotpotQA provide human-annotated ground-truth labels for every query. In production enterprise applications, users submit tens of thousands of unique, domain-specific queries daily across messy internal documentation, leaving engineering teams without curated ground-truth labels. ...