Preregistration Protocol Mitigates p-Hacking in LLM-Based Research

A new protocol suggests preregistering experiments and specifying eligible future models to prevent researchers from tuning prompts or parameters to achieve desired results. By running confirmatory analyses on the first released model post-preregistration, researchers avoid hacking against existing configurations.