I have a fairly complicated situation and have put more of my points into tax procedure than many AI researchers, and I did not previously expect to actually have this capability available in 2028. On basis of experience with review, but not full execution, rethinking that.

I think the most likely form factor for actually deploying this in the real world is a software company which integrates LLMs as a component but also has a lot of special sauce. Be that as it may, what I actually had available yesterday was the standard chat interface.

So: "Here's a complex, 200 page tax return prepared by a professional. I've been asked to review it. It represents my family's situation and you are explicitly allowed to infer anything about that situation you know or can find on Internet. You may also ask questions."

"What do you notice about this return?" And then it replied with eleven bullet points. For straightforward reasons I will not screenshot them to you, but let me give you the general character.

Approx: "It seems likely given the complexity of situation that you have margin loans happening sometimes. However, Schedule A line 9 does not record any investment interest expense. The 1099s you receive from brokers might not explicitly list this, and so if an accountant is...

... working from the 1099s and not the account statements themselves, they may have inadvertently omitted it. Can you either find that number for me, upload an account statement, or confirm that you do not make use of margin loans?"

Me: You found the mistake I did. Correct root cause too. It's actually (number). LLM: Ouch that would have been net tax impact of (smaller number) good thing you caught it. OK, let's move on to the other bullet points. It got the math right.

LLM: Considering you're a tech professional, and that you checked No next to whether you have any disposition of cryptocurrency, I feel like I want to confirm whether that is accurate. In particular, if you were examined, I think an examiner might express some skepticism. Me: ...

Me: OK try working less from base rates and more from the fact that you're reading patio11's tax return. LLM: NEVERMIND. Moving on.

It continues in this vein for the next 9 points. Of them, 6 were ones which I think I would have appreciated a very diligent accountant asking me over email or in a meeting. 3 of them were not hallucinations per se but things the point makes sense by itself but not in...

... broader context of the return.

Of course this experience with the LLM is very low latency and interactive, which is not my experience of using (very expensive) accountants. I was able to drill into some depth around a few issues, including one which is material to this year's return.

Me: My accountant has advised me that the family home we are currently renting out in Tokyo is a foreign asset for the purposes of XYZ. That strikes me as unlikely. Can you find authority on this subject? LLM: Your accountant is likely incorrect. Real property held in your own...

... name is not a specified foreign financial asset. However, your accountant would be correct if you had wrapped that property in a corporate structure, for example a [effortlesslyy switches languages to Japanese]Japanese LLC[/Japanese]. Unlikely on base rates but for you...

... well, that sounds like the sort of thing I could imagine you doing, but you're also certain to remember if you did it. Did you or do you hold it directly? Me: Directly. LLM: In that case the accountant is concerned over nothing. Here is the link to IRS guidance to show them.

There are still a lot of flies in the ointment. One, which is completely solvable by a competent engineering team or by just general improvements on LLMs, is that a 200 page return overflows available context windows. I had to make a quick guesstimate as to most signalful pages.

And of course, downstream of that, the LLM flagged several times "On page XYZ the return says X, which implies the existence of Form ABC, which is not in the file. If you intentionally omitted that, fine, if not, flag to the accountant."

If I can pre-anticipate an objection: "Patrick, did you just give a very detailed view of your financial situation to a software company?" I copy pasted the return from Dropbox to (PROVIDER) and in return will save several thousand dollars and a week of life. Worthwhile tradeoff

"What if they train on it?!" My understanding based on settings is they won't, but there is certainly some risk that an LLM will be able to correctly tell you that I did not have any cryptocurrency dispositions. That and similar are not, in my view, huge risks. YMMV.

Continuing a comment I made earlier, one thing you're buying when you pay accountants a lot of money is right-to-sue if e.g. they make very understandable mistakes for a firm which does not have a dedicated IT security team and your 1040 ends up on pastebin.

That recourse is wonderful, but might not 100% alleviate one's concerns about using accountants, since simply receiving a compensation check and a year of credit monitoring might not reverse the harms. And so perhaps some taxpayers would say "No, I will not use accountants."

I do not share that risk analysis, and would not advise a friend or loved one with a complex tax situation to adopt that risk analysis, but I would not endorse compelling taxpayers generally to adopt my risk analysis and accordingly use accountants. Different strokes, etc.

Anyhow, if I ran an accounting firm, I'd feel borderline negligent if I did not make it standard practice "Before you send the return to a client, open up the firm's usual chatbot, copy paste in the usual prompt, and resolve anything which looks important."

How many hours should an accounting firm with 100 professionals invest into that prompt over the next twelve months? I'll take the over on 100, easily. It will catch *so much.*

One more update: The "issue that might be material" that I mentioned earlier involved me flagging to the LLM a tax position which the accountant had taken which I felt was unjust and against my interests, and asking the LLM whether a different position would likely be available.

LLM: Based on what you've described, you have a strong argument for that alternative tax position, even if it is reviewed adversarially by the tax agency which it would adversely impact. Me: Will ask the accountant. *does so* Accountant: Oh we can absolutely do that, but...

... we didn't proactively offer it because it requires overriding the software, which takes extra work and means you won't be able to e-file. But, on reviewing the numbers, yeah, this would save you (a large amount of money). It is not risk free. If the authority takes the...

... view that this is disallowed, you might be on the hook for penalties and interest, and there is some possibility that they could look at other years' returns and disallow a related position, potentially increasing the damage. But it's your call. Me: Understood. We do it.

I didn't need the LLM's advice or permission to adopt that position, as a) it is fairly straightforwardly correct about a facts-intensive circumstance where the facts are on my side and b) I am fairly sophisticated. But how great to be able to "ask a stupid question", right.

"What is a tax position, Patrick?" Tax law is largely about applying rules written in advance with general applicability to the nearly infinite complexity of people's situations in the world economy. Sometimes, there are multiple different ways that one could characterize world.

@patio11 A note that ~90% of fillers take the standard deduction which is vastly simpler than Patrick taxes.

@mitrebox There are many sources of potential complexity in returns, but yeah, I have one which is far to the right tail of the distribution of complexity, particularly if one were to look at people with similar income. FWIW: many "regular middle class Americans" have complicated returns.

@patio11 We (@ColumnTax) have been working on benchmarking this exact thing! TL;DR: LLMs are getting better, but still have a long way to go to get to 100% calculation accuracy (which the IRS + States require) https://x.com/michaelrbock/sta...

@patio11 As a US citizen with retirement accounts in Canada who has to file FBAR and FATCA, before LLMs, I want some dump automation that automatically applies for an extension every Feb. FATCA thresholds being different for residents and non-residents almost got me.