The SHAPE framework evaluates LLM mathematical reasoning by analyzing semantic spaces and mathematical heuristics within Chain-of-Thought trajectories. Findings indicate that the specific heuristics used by a model are better predictors of final answer correctness than traditional CoT metrics.
HOW THIS AFFECTS YOU
●
researcherYou can use these semantic and heuristic lenses to better understand why certain models fail complex reasoning tasks.