Accuracy Is Not Enough: A Framework for Bias Detection in AI-Driven Investment Workflows
As large language models move deeper into investment workflows, drafting equity research, scoring trade ideas, and summarizing earnings calls, the risk they introduce isn’t primarily getting the numbers wrong, but the quiet, systematic propagation of bias: confirmation, overconfidence, recency, and dozens of other distortions that compound invisibly across a firm’s decision-making.
A growing body of research has proposed LLM-based detectors to catch this exact kind of bias, typically reporting a single accuracy score on a held-out test set as proof the detector works. In a new paper accepted at the 7th ACM International Conference on AI in Finance (ICAIF ‘26), Domyn researchers Fabrizio Dimino, Abhinav Arun, Mahesh Kumar, Bhaskarjit Sarmah, and Stefano Pasquali explore why this single number is not enough.
The paper introduces a 25-bias, four-family taxonomy built specifically for investment workflows, paired with a 5,012-input benchmark independently labeled by financial subject-matter experts. In addition, the team introduces a forced-execution detection framework in which every selected bias produces an auditable trace rather than waiting on a model to decide, on its own, whether anything is worth flagging — a design built for the kind of per-decision governance that regulators now expect of AI in production. Along with it, a five-axis sensitivity protocol tests whether a detector’s verdict survives the trivial rewordings that any real analyst would actually use, which benchmarks usually miss.
Evaluating seven detectors across both measures, the team of researchers uncovers a split that headline accuracy scores conceal entirely: the most accurate detector in the study is not the most robust to rewording, and the most accurate detector also happens to be the one that collapses hardest under a stricter confidence threshold. In short, accuracy and deployability turn out to be two different questions. For anyone procuring a bias detector for a regulated workflow, a high accuracy score isn’t enough on its own, robustness and calibration have to be checked too.
The paper breaks down the full findings, including exactly where each detector fails, axis by axis.
