Lesson 0914 min
Measure, then ship
Goal: Compare every version on the evaluation set and ship behind FastAPI.
Why this step
Placeholder body for the interface. Each lesson opens with the problem this step solves and the decision behind it, so the code that follows has a reason. The real text arrives with the track.
Build it
The code for this step is small on purpose. Write it yourself first, then compare it with the reference commit below.
def score(cases: list[Case], answer: Answerer) -> float:
"""Share of cases where the top tag matches the hand-tagged one."""
hits = 0
for case in cases:
result = answer(case.disclosure)
hits += result.tags[0] == case.expected_tag
return hits / len(cases)Check it
Run the evaluation script. The number should match the one at the end of this lesson, or you'll know exactly which step to look at.
Checkpoint
This lesson ends on this commit. Compare your work with it:
f0c7d26api: serve the tagger with FastAPI
$ git diff f0c7d26