R.O.A.D. Barbados Historic Handwriting Challenge
'/%3e%3c/svg%3e)
Helping Barbados
$25 000 USD
2 months left
Optical Character Recognition
Natural Language Processing
1049 joined
182 active
Start
Jul 03, 26
Close
Oct 04, 26
Reveal
Oct 04, 26
Help Please
12 Jul 2026, 06:27 · 3
Can you share the exact wer.py/cer.py implementation used for scoring, or clarify why WER Weighted values exceed 1.0? Trying to align local validation with leaderboard scoring
WER weighted = mean word-level Levenshtein distance per line.
CER weighted = mean char-level Levenshtein distance per line.
public_score = 1 - 0.5 * (WER_weighted / 12 + CER_weighted / 55)
from rapidfuzz.distance import Levenshtein def public_score(y_true, y_pred): assert len(y_true) == len(y_pred), "Inputs must have the same number of lines" wer_weighted = sum(Levenshtein.distance(ref.split(), pred.split()) for ref, pred in zip(y_true, y_pred)) / len(y_true) cer_weighted = sum(Levenshtein.distance(ref, pred) for ref, pred in zip(y_true, y_pred) ) / len(y_true) score = 1 - 0.5 * (wer_weighted / 12 + cer_weighted / 55) return wer_weighted, cer_weighted, scoreThanks @J0NNY!
That's cool. If you look in the code of their Starter code, you will see a file: eval_metrics.py which have an import at the top
import os
import sys
import yaml
import pandas as pd
SCRIPT_DIR = os.path.dirname(__file__)
EVAL_DIR = os.path.abspath(os.path.join(SCRIPT_DIR, "..", "..", "evaluations"))
if EVAL_DIR not in sys.path:
sys.path.insert(0, EVAL_DIR)
from wer import compute_weighted_wer
from cer import compute_weighted_cer
....
It says go to the SCRIPT_DIR which is the current directory, go up twice and enter an evaluations folder. This evaluations folder likely contain the wer and cer modules being loaded above because an exhaustive search of the starter downloads yields nothing.
Can someone tell me where to find these evaluation files?