Moral Entropy Framework for Auditing Bias in Ethical Labeling
September 21, 2026
This Bayesian framework decomposes annotator disagreement into aleatoric and epistemic uncertainty to audit consensus rules. By modeling the full posterior over true labels, it reveals biases in standard aggregation methods like the any-annotator rule using metrics such as Brier score and ECE.
HOW THIS AFFECTS YOU
●
researcherYou can use this to better quantify whether disagreement in your datasets stems from irreducible ambiguity or noisy labels.