Updated Financial Touchstone benchmark with 2,967 Q&A pairs from 495 annual reports tests 20 open-weight and proprietary models; Chinese AI labs' models show improved financial text comprehension b...
IntelliAudit uses retrieval-grounded multi-agent LLMs to automate IT audit evidence evaluation by synthesizing heterogeneous organizational data across policies and records—addressing the semantic ...
New research introduces FinRank, a benchmark for evaluating AI systems on financial question answering over SEC filings, focusing on evidence provenance rather than answer correctness alone.
Researchers introduce A²E (Agent Auditing Engine), an end-to-end evaluation framework for assessing LLM agent capabilities across domains, addressing the gap in systematic auditing of complex agent...
Academic researchers present A²E, a systematic evaluation framework for auditing LLM-based agents, addressing gaps in capability testing critical for enterprise AI deployments in regulated domains ...
New Accounting Graph Transformer enables joint 12-month forecasting of 13 financial KPIs from limited accounting history, addressing a core pain point for small business planning and risk workflows.
Research on role-differentiated LLM agents for audit resource allocation shows that distributing triage decisions across multiple agents can paradoxically worsen demographic bias outcomes compared ...
Research paper introduces trace-grounded parametric profiling to expose video language model failures in event counting tasks—a core requirement for automated audit and bookkeeping workflows.
arXiv paper argues top-k embedding retrieval fails on financial statements and audit reports; proposes interpretable agentic operations for structured financial document analysis.
Researchers propose interpretable agentic operations over traditional embedding-based retrieval for financial documents, showing top-k methods fail on 780-page government reports where 86.8% of con...
Academic paper proposes methods to detect errors in autonomous data analysis agents used in audits, addressing reliability gaps in unsupervised error detection systems.
Academic scoping review of 4,259 documents reveals AI audits focused on individual models miss integration risks—drawing lessons from aerospace safety-critical auditing for accounting and complianc...
Researchers introduce FinReportBench, an expert-grounded benchmark measuring LLM-generated financial report quality for institutional use, identifying gaps in identity, compliance, sourcing, and vi...
FRAMES framework helps LLM agents improve at policy-bound tasks like document auditing by evolving skills from existing assets while preserving auditability and rule consistency—addressing the chal...
Research reveals systematic validity problems in automated benchmarks evaluating agentic AI systems, with audits finding flaws in 7 of 10 popular benchmarks—raising questions about safety certifica...
Academic paper proposes optimal allocation strategy for auditing LLM agent fleets with limited human budget, identifying miscalibration thresholds where confidence-based auditing underperforms rand...
Study reveals accounting firms investing in AI spend 15-30+ hours weekly verifying AI work, contradicting job-loss narratives and highlighting hidden compliance costs of automation.
Mercor and Ramp introduce APEX-Accounting, a benchmark measuring whether AI agents can execute core accounting tasks like reconciliation, variance analysis, and month-end close.
Researcher proposes adapting food safety's HACCP methodology to create an auditable self-control standard for AI, bridging compliance gaps in automated systems used by auditors and accountants.
Researchers distilled DeepSeek V4 Flash into a 120B open-weights model scoring 83.61% on FinanceReasoning benchmark, outperforming Kimi K3 and Inkling, while confirming censorship did not transfer ...