[arXiv]score: 0.31

High Model Variance in LLM-Based Textual Analysis of Financial Data

September 28, 2026

Using seven LLMs to score S&P 500 earnings transcripts revealed an average cross-model rank correlation of only 0.52. Discrepancies in sentiment, risk, and clarity scores are model-specific rather than caused by text ambiguity, significantly impacting the reliability of downstream statistical inference.

HOW THIS AFFECTS YOU

●

builderYou cannot assume consistent results across different model providers when building automated text-to-variable pipelines.

●

founderBuilding financial tools requires rigorous cross-model validation to ensure your product's metrics are not merely artifacts of a specific LLM.

●

investorBe skeptical of alpha generated solely from LLM-derived sentiment, as the signals may be highly model-dependent.

read original ↗arxiv.org

DAILY DIGEST

you don't check 9 sources — we do. one email every morning, read in 2 min. free. unsubscribe anytime. privacy

← back to feed