ProbabilityBench provides an evaluation framework with exact, instance-level class probabilities for text-based domains like credit risk and peer review. By defining the full data-generating process, it allows for direct measurement of model uncertainty via KL divergence.
HOW THIS AFFECTS YOU
●
researcherYou can precisely measure how well your model's reported probabilities align with true conditional probabilities.
●
policyThis provides a more rigorous way to evaluate model calibration in high-stakes domains like credit risk.