Lesson 0712 min
Prompt for cited answers
Goal: Return tags with the passages that support them.
Why this step
Placeholder body for the interface. Each lesson opens with the problem this step solves and the decision behind it, so the code that follows has a reason. The real text arrives with the track.
Build it
The code for this step is small on purpose. Write it yourself first, then compare it with the reference commit below.
def score(cases: list[Case], answer: Answerer) -> float:
"""Share of cases where the top tag matches the hand-tagged one."""
hits = 0
for case in cases:
result = answer(case.disclosure)
hits += result.tags[0] == case.expected_tag
return hits / len(cases)Check it
Run the evaluation script. The number should match the one at the end of this lesson, or you'll know exactly which step to look at.
Checkpoint
This lesson ends on this commit. Compare your work with it:
47ab9c0answer: return tags with citations
$ git diff 47ab9c0