AI Outperforms Outsourced Accountants on Transaction Coding, Powers Solo Fina...

There may be errors in spelling, grammar, and accuracy in this machine-generated transcript. Blake Oliver: [00:00:04] So who is in charge of the IRS? Who has the legal or administrative authority? I mean, this all ties into that settlement that Trump got for himself, because who actually can sign that? I mean, if there's no confirmed IRS commissioner, is that even valid? David Leary: [00:00:23] Coming to you weekly from the OnPay Recording Studio. Blake Oliver: [00:00:30] Hey, everyone, and welcome back to the podcast. This is your weekly roundup of news in the accounting profession. I'm Blake Oliver. David Leary: [00:00:37] And I'm David Leary. Blake Oliver: [00:00:38] David, our two top stories are tied together this week. I've got a story about a controller at a SaaS company. Runs an entire finance team. Just him. He's the finance team using automation AI to do it all. They do. David Leary: [00:00:57] 50 weeks ago we had a firm of one. And now you're having a finance team of one. Blake Oliver: [00:01:02] They do 50 million a month in revenue through their marketplace. And this guy is doing it all by himself. I think it's a great case study. And you, David, have been digging into some recent studies about the capabilities of LMS like cloud and ChatGPT and how they perform in different accounting tasks, bookkeeping, and more advanced accounting milestones. David Leary: [00:01:25] Let's just say. Blake Oliver: [00:01:26] They're getting better and better, better and better. So we'll start with those two. I also have some follow up about the Trump Weaponization Fund and the IRS. Also former guest of the podcast, Christina Ho, who was a board member at the PCAOB, is starting an audit powered or an AI powered audit firm that's going to audit private companies and then maybe public companies as well. The big four are aware of this threat. Kpmg is going to Silicon Valley and partnering, possibly investing in AI companies that they view as a disruptive threat to their business model. We've got all that and more. But first, David, let's thank our sponsors. David Leary: [00:02:12] Our sponsors. This week. We have on pay the value builder system and our cost seg. Are you tired of payroll headaches getting in the way of client experience? You want to deliver manual workflows, creating bottlenecks, compliance, nightmares, and endless support calls that go nowhere. There's a better way for your team and your clients on pay. Is the payroll partner that accountants and bookkeepers actually love? Why? Because it's easy to use, packed with value, and backed by support that actually supports you. Their team gets rave reviews for being fast, expert, and actually reachable when you need them on pay. Handles the heavy lifting. You get a dedicated onboarding support coordinator who sets up worker profiles and transfers year to date data from previous providers, all at no extra cost. They're seamless. Quickbooks and Xero integrations eliminate manual journal entries, and they support any type of business you serve. Farms, restaurants, nonprofits, you name it on page can handle the unique requirements without adding complexity. And on page pricing is simple to everything your clients expect, from multi-state filings to off cycle payroll is included. No hidden fees or surprises. To book a demo. Head over to The Accounting Podcast dot ProAdvisor that is Accounting Today dot promo forward slash OPAY, and I actually had to use on page support yesterday. Blake Oliver: [00:03:26] How'd that go? David Leary: [00:03:27] So I had my daughter come and work with us at the AME conference in Palm Springs, and I had a pair. So I paid her. And apparently, you know, now that she's kind of an adult, she went and created a new bank account in her own, but never changed the account numbers in on pace. So I sent her paycheck to her old closed bank account and it was painless. I got on chat with on pay and they just confirmed, do you want us to send it to the new numbers? I said yes, and they reprocessed her payroll and she texted me this morning she got paid, But yeah, it was super easy to do. They just they knew there was a problem. They just needed confirmation from me for the next step. It was great. Blake Oliver: [00:04:03] I love hearing that. Thank you for being a big supporter of the show. All right. Let's talk about this controller at school. The company is called school. It's a SaaS business. They do online communities, educational communities, SKOOL. And the controller of school was on a webinar hosted by the Controllers Council recently. And he talked about how he runs the entire finance function for this startup by himself. No humans other than him, just AI agents. School handles over 5 million in monthly spend and nearly 50 million a month runs through its marketplace. The controller went on vacation for two weeks, and he came back and he was expecting a huge mountain of work He had 2000 transactions that had piled up while he was gone, but his automations had already coded, categorized, approved and synced almost all of them into their ERP. He had to review only 67 transactions by hand, and those were the tricky ones out of the 2000 that needed the human judgment. The entire cleanup after his two week vacation took him about 30 minutes, he said on the webinar. Yeah. Go ahead. David Leary: [00:05:25] Can you repeat the numbers? So he had he had to manually fix 67 of 2000. Blake Oliver: [00:05:29] 67 out of 2000 transactions. Are you like 3%? Is that right? David Leary: [00:05:36] No, it's less than that. Right. Blake Oliver: [00:05:38] Took him 30 minutes. How does he do it? He runs seven specialized AI agents, plus an admin agent that checks the work of the others and enforces the controls. And he basically describes that his job has changed from doing the work to reviewing the work. And so now he has time to do forecasting, cash management and strategy instead of processing transactions. So that is James Agius, A GIUS he's the financial controller at school. And that was on a controller's council webinar. And that may sound crazy, but I have heard about this. I have experienced just how much AI and automation is helping us in our accounting here at earmark, which we do ourselves still, because we want to be deep in the weeds on it, deep in the weeds, know what's going on. Um, and I'm curious how this fits with what you've been looking at, David, this week when it comes to how the AI models, the, the major ones collide and ChatGPT are performing with every new iteration. We've got some new data from digits. David Leary: [00:06:52] Yeah. Blake Oliver: [00:06:52] So ramp. Right? David Leary: [00:06:54] Yeah. So take your example. That guy there, he's now at 96.6%, which is pretty believable, right? Because for the first time ever, the main models. So the main, when I say main models, this is going to be your off the shelf clod open, uh, open AI or ChatGPT, uh, Google Gemini. The, the models off the shelf are now beating according to digits human accountants. So digits has been comparing human outsourced accountants across 2000 transactions to get them categorized over time. And this is the first time ever all the base models are now beating human accountants, but the base models still are not anywhere near. Your guy is your guy was at 96.6%. The base models are like 79%, 79.4, 79.5, 79.9. Uh, clod. Opus 4.8 is now at 80.7%, and then digits itself is coming in at, according to their study is coming in at 97.8%. So just slightly higher than what? Uh, I forgot the guy's name. This one person. Blake Oliver: [00:07:57] James at. David Leary: [00:07:57] School finance team, right? Yeah. Blake Oliver: [00:07:59] And so, so I just want to point out here on the chart, like I'm looking at this right now, you've got it on the screen. Human outsourced accountants are able to code correctly 79% of transactions in digit study correctly the first time. And they're saying that they can do close to 98%. But that like the other models, just without being wrapped in anything else, are able to do it at like between 79 and 80% also. So they're basically on, they're on par with, with humans. David Leary: [00:08:31] Out of the box with nothing. And, and the conclusion maybe on this is that this is kind of a commodity. Now, the basic categorization, um, transaction classification is kind of a basic thing now that everybody's just going to get for free from these models. Blake Oliver: [00:08:48] If you can plug it into your GL, then? Yeah. I don't see why it couldn't just do that. David Leary: [00:08:53] But then you look at the example, the I forgot the finance guy's name that you just talked about. He, James was able to probably take things and because he's got a closed system, tweak it a little bit more. And that's how he's getting those higher percentages. Blake Oliver: [00:09:08] Yeah. More context. Right. Yeah. Giving it source documents, giving it more information. It's all about the context. So you've also got data from ramp here. They also have been doing a study. David Leary: [00:09:19] And we'll talk about this in a moment too. But ramp also is comparing. So they launched basically a thing called they're calling it stack. And think about it. It's like AI agents that plug into your current accounting stack at your firm and can talk to QuickBooks and other tools you use and do work accounting work for your firm. And they've measured that against the models itself. So ramps coming in at like 65.8%, it's different type of transactions they're doing versus just categorizations, but they're beating the model. So the the purpose built things. So ramp and digits are beating the models. But the models now are starting to beat humans for that kind of work. Blake Oliver: [00:09:57] Did. So did ramp say like what the performance is versus the human performance on these. David Leary: [00:10:02] Ramp never actually um says that they're they're not measuring what huma