Skip to content
Enigma Campus

Track

Build a RAG system with an evaluation set

Rebuild the ESRS-to-XBRL tagger step by step: write the evaluation set first, then retrieval, reranking and cited answers, measuring every change.

  • Intermediate
  • 9 lessons
  • 1 h 59 min
0 of 9 done0%
Start lesson 1

Module 1Start with the evaluation set

  1. 01Why the evaluation set comes firstExplain why a RAG system without an evaluation set can't be improved on purpose.
  2. 02Hand-tag twenty real casesBuild a small, honest evaluation set from public ESRS disclosures.
  3. 03Score a baselineWrite the scoring script and record the first, deliberately simple number.

Module 2Retrieval

  1. 04Chunk the documentsSplit the taxonomy into chunks that keep each definition whole.
  2. 05Embed and store with pgvectorStore embeddings in Postgres and query the nearest chunks.
  3. 06Rerank the candidatesAdd a reranker and see whether the number moves.

Module 3Answers with sources

  1. 07Prompt for cited answersReturn tags with the passages that support them.
  2. 08Refuse when the sources don't sayTeach the system to say it doesn't know, and score that too.
  3. 09Measure, then shipCompare every version on the evaluation set and ship behind FastAPI.