[arXiv]score: 0.13

GPT-5.1, Gemini 3 Pro, DeepSeek-V3.2 Show Large Provider-Level Methodology Bias

June 26, 2026

Prompting three LLMs with 1,000 arXiv CS research questions and comparing their methodology suggestions to actual paper methods reveals provider choice drives 3–5x more divergence than dataset type or evaluation metric. The study uses Jensen-Shannon divergence across a shared taxonomy to quantify structural differences in default methodological tendencies.

HOW THIS AFFECTS YOU

●

builderIf you're building research-assistance tools, provider selection shapes output structure more than prompt content — worth auditing which model family aligns with your target methodology space.

●

researcherModel family matters more than task type when LLMs suggest research methods, so model choice introduces systematic methodological bias in AI-assisted research workflows.

read original ↗arxiv.org

← back to feed