Skip to content
Enigma Campus

Lesson 0312 min

Score a baseline

Goal: Write the scoring script and record the first, deliberately simple number.

Why this step

Placeholder body for the interface. Each lesson opens with the problem this step solves and the decision behind it, so the code that follows has a reason. The real text arrives with the track.

Build it

The code for this step is small on purpose. Write it yourself first, then compare it with the reference commit below.

eval/score.pypython
def score(cases: list[Case], answer: Answerer) -> float:
    """Share of cases where the top tag matches the hand-tagged one."""
    hits = 0
    for case in cases:
        result = answer(case.disclosure)
        hits += result.tags[0] == case.expected_tag
    return hits / len(cases)

Check it

Run the evaluation script. The number should match the one at the end of this lesson, or you'll know exactly which step to look at.

Checkpoint

This lesson ends on this commit. Compare your work with it:

c51d0aaeval: score a keyword baseline

$ git diff c51d0aa