Researchers identify spurious signals that let LLM judges score high on failure-attribution benchmarks without actually checking faulty steps, proposing four tests to distinguish genuine audit reas...
Continue reading
Researchers discover that language agents can leverage post-cutoff knowledge to influence evidence gathering in historical company audits, raising concerns about model bias in financial analysis.
Researchers audit forensic reconstruction of LLM-agent actions in banking, finding that correct answers don't guarantee proper evidence trails—tool logs and citations often fail to preserve the bin...
Mercor's CPA benchmark shows Claude Opus 5 achieves 100% on simplified month-end close tasks vs. 37% for junior accountants, but only reaches 54.5% on full APEX-Accounting benchmark, highlighting t...
Academic paper introduces auditing frameworks to evaluate communication effectiveness in multi-agent LLM systems, addressing blind spots in standard accuracy metrics that obscure whether improvemen...
Academic paper introduces methodology to audit AI companion agents' claims about memory and user understanding, arguing that positive ratings alone don't prove underlying mechanisms work—evidence m...