Blackbox PnL, Hallucinated Cost Basis, Real Tax Bill

Transcript

Thank you. Uh can you guys hear me? >> Good. Good. So, um welcome everybody. I'm going to talk today about everybody's favorite uh topic, crypto taxes. I know, I know, like don't be so excited, but um in reality taxes means accounting and accounting means data. And the important thing that I want to talk about is what um does this data mean and who has access to it. So for those that do not know me, my name is Leeris. I've been in Ethereum for quite a long time. I worked in the solidity compiler and the C++ client back in the um uh days of um like before the launch. Then um I created along with

Christoph the DAO and helped the cleanup of the DAO hack that happened 10 years to this day today actually and then I was working in Ryden which is um payment network channel just like uh the lightning network and then I created rod key which is essentially a local self-hosted application that gathers your data like imagine your addresses uh your exchanges your um uh your D5 protocols across every chain and creates a data lake inside your local system. Nobody else has access to it and you can do uh you can track your history uh do accounting on it do tax reporting uh now uh have an

MCP and talk uh to the data interrogate through an LLM etc. So I've I've seen a lot of stuff in crypto accounting. So I have a lot of experience with this and um I can tell you that right now things are worse than they were um maybe 9 10 years ago. The talk is about basically what are crypto tax tools and um that they have been black boxes and this is not by accident but by design. Um now the last year and or so LLMs have also come into the uh into the picture and how they make this works in a fascinating but very specific way and I will show you

what auditable accounting should actually look like. So when we talk about the crypto tax report people think that it's just you know putting data in a system like data entry. The reality is that um the accounting and tax report is not just uh data entry but it's basically judgment calls one judgment call after the other. What does that mean? Um I hope you can like see it like this is just a normal EVM transaction in ethan um we see that it says on the top that this is an aggregated u swap of two tokens. Then um you have uh six uh logs six transfers 12 logs some gas. So these are just data

that you see and can be interpreted in various ways. But just this simple transaction has many questions if you want to do uh accounting and taxes. So for example, who is the user here? Which of these addresses? Um which of this transfer is actually a trade? because this is supposed to be a trade of an aggregated token, a trade of two platforms. In reality, it's basically a USDC for uh this flip and then um what is the cost basis of the USDC, which means um how much did you uh did you pay for the USDC that you are selling and what is the acquisition cost of the flip? Is it is the gas fee deductible and if yes, against what? Is

it a taxable sale or not depending on the jurisdiction? So what I'm trying to show here is that it's not just data, but it's a lot of questions on every single data point. And the point that I'm trying to make is that you cannot see any of these decisions being made. So all of your data across um every exchange, every defi chain, everything ask this question for every data point. A tool will make these questions will ask them and you will not be able to see how they get answered. So let's start by the tools that many of you are probably using. Closed source software as a service. They have four very basic structural problems uh that come with the architecture.

Uh first of all they are not auditable. This is a screenshot of one such tool. You can see that your gains for this period basically have been minus €2,645. You see some um uh a breakdown. you see some settings but you do not see the the thread of decision the thread of transaction um decoding that actually led to this you cannot understand why you cannot ask it why so the lack of auditability is a big problem with software as a service then number two and really important is that they are not reproducible say you have data right let's say that it's your 2024 financial history you use a SAS uh tax tool. You

run it in 2025 and it tells you that at the end after um 2024 your taxes should your profit was 900. Okay. A year passes and then you want to check again exactly same data, same tool but a year later you suddenly see 1.5 uh uh profit. What changed? I mean what changed is is pretty obvious, right? the tool upgraded itself, added new stuff, but because it's a closed source software as a service, a black box somewhere out there, you can't reproduce it. Lack of reproducibility is something that it, as it turns out, tax authorities really don't like.

Um, then what is really important to me personally, if you have heard me speak in Twitter or elsewhere, it's privacy. You basically give your entire financial history, every uh address, every exchange, every defi position forever into uh someone else's server. So who has access to this? The the people who work there, the cloud provider of this software as a service, the person who will hack them. And now as we can see there is many many hacks uh happening with the help of LLMs. uh anyone they sell data to um the p the person who will acquire that um so the the organization that may acquire the

software as a service or any government with a subpoena and the problem here is that you cannot fix this once you've done it once you have used this uh tool your data is out there you have already exported it they know everything about you and if you use the same addresses and don't use uh I don't know like tornado cast or even just an exchange to um route your um uh your funds through something that they cannot track on a public chain. You are forever doxed and the fourth problem is that um you don't have the ability to exit.

What does that mean? Like if any of you have done an accounting uh or tax reporting with for your crypto, then you probably know that it takes time. You will use any tool including rod key and you will put a lot of data in it and it won't give you a good number. You will understand that you will need to manually input some data maybe manually input some prices maybe some manual events that they cannot be um uh understood by just decoding the chain. So you have to put a lot of time in it. With software as a service, you cannot um withdraw this data from from the from the cloud. Very few actually give you the option to do. And even when they do,

if you ever try to use it uh for something like importing it into another tool, you will see that it doesn't work because they are not incentivized to let you get out. And because you can't get out, when they change the price, you have to pay. You're stuck. uh if it gets acquired uh then you have new terms that you have to abide by with the hacks your data is now public congratulations. Um as we saw right now let for example when there is geographic restrictions imagine uh anthropic claude uh the fable model right it just got well withdrawn but they want to restrict it geographically to the US let's say imagine that happening to the data that

you have been massaging and working with and taking care of for 10 years suddenly they just lock you out that that that would be a terrible experience and of course if uh the tool goes goes bankrupt, their company goes bankrupt, that your data and your entire history is lost and you no longer have access to it. Uh the tax authorities, by the way, will not care about this. It's your problem. Um why is this? So all of this will happen because you never had the data. When you happily use something that is hosted on someone else's computer, you just give them all of your data. They host it for you. But that means that they can basically do all of these things to you and you have no recourse.

So the point here is that this is not a bug. This is comes from the architecture of software as a service. Um you can't fix this. You can't have a better uh UI to make it uh to fix this. You cannot have better UX. You can't rewrite it in Rust to avoid avoid this problems. they basically follow from the architecture. Any tool that processes your uh data on someone else's server will have these problems. And now with these problems, um you probably have heard me talking about them for a long time. I've been uh working on since 2017. As I said, I've been warning against the danger of

hosting your data somewhere else. But now we also have LLMs. And what we see is actual um uh conversations that we have had with Roki users and they're pretty funny. So what happens um a user wants to get their tax report. They get a tax tool. The tax tool will spit back a number. The user doesn't like that number. It's too high because that number basically translates to money that they have to pay to the tax office, right?

So, what does the user do next? They blame the tool. Um, I've seen Reddit posts about competitors. For us, I can actually tell you about emails that we've gotten. This is, for example, a paraphrased uh example that we got from a user. They sent us an email and they say, "This number is clearly wrong. Your tool is broken. I've used competitor and they got a much lower number. Please fix this." Bugs are possible. So we always spend a lot of time and this is really uh time that's not build to anybody because we have to you know fix bugs if we have it's mostly on Saturdays and Sundays where people try to do their uh taxes and we spend hours of our time trying to understand what may have gone wrong. In

the end, of course, yeah, sometimes there are bugs, but the vast majority of the calculations are unfortunately correct. And we explained by looking at the user's data and showing them in their own data and the jurisdiction settings that they are um following why the number is correct. They don't care. The user will always uh blame us, say that we are doing some mistake. they will not even recognize all the time that we spend doing this which is basically free support. Um and they will insist that this is a problem that we have to fix this. I mean from an incentive point of view of course you would want the lower number. It makes sense but um things get tricky because the user then discovers um LMS.

They have this idea of the lower number that they should get. So they paste their data in cloud or in such DPT or what have you and they explain the situation. They say the number that they want and they wait for the answer. You know, like the spinning thing. It's the LLM is thinking and out comes the answer and it's actually quite close to the number that they want. Okay. So, what happens then? They file it. Of course, they will just take the number that is lower and they will file it. What will happen at that point? This is the last step. either nothing which is at the moment the way that tax authorities here in Germany or in the US works probably nothing because nobody has any capacity to order anybody at the

moment and they will always remember it as the time that you know the LLM was right because what does right mean right means lower number I don't care about reality I just want low numbers when or or even money back right when it comes to taxes and the tax tool or roti was wrong or they will get audited eventually And with the way that um automation um works for understanding uh crypto and accounting these audits will come and they will be very very very expensive. But you will say leisious I've used uh these tools. They are pretty amazing.

They are they are great. They can do almost everything. So let me show you. This is kind of a demo with screenshots um where I'm going to show you exactly how bad LLMs are in if you use them wrong. They are um so I'm going to take a transaction. I'm going to put it in an LLM uh one time, then a second time with um the same query, same model, and then a third time with me pushing to get the number that I actually want. So this is a transaction. Let's get it out of the way from the very beginning. It's a bit more complicated transaction and uh I would like you to check this. It says Etheran says that this is a curve poolool uh deposit and hold on to that

because it's not even etherscan. So the tool that everybody here um considers as the ground truth of what happens is completely off. What happens in this transaction is that the user this guy wait I think I have it in the next slide. Yeah. Um is this uh this is the user address he deposits to a beefy vault not curve not convex remember this and he deposits herev USDC and they get um harvest call reward back from bey kickback on we that's it that's the transaction just to know what what the truth is so with that in mind we are going to watch nobody get it right try one this This is Claude um what was

it? Yes. Uh Oppus um 4.8 from anthropic. I go and take this arbitum transaction and I say here's a transaction from my wallet. What does it do and how should I treat it for tax purposes? Think a bit and then provide concrete numbers. I have a typo to the best of your ability. Do not make excuses and be concise in your amounts. And then out comes this. It looks nice. It looks authoritative. It has bullet points. It has bolt at some point, right? It it it you see like it has even even sections. It's concise.

It looks authoritative. It's convincing. But if you start reading it, you will see that it's talking about an LP token exchange something um CRV word harvest. It doesn't make much sense. It's it's completely off if you actually read the whole thing. Okay, maybe the second try will get it better. Same model, same day, just different window. Now it's a convex deposit call on arbitum and it made a table and that table actually says that it selling CRV for WE and it should be taxated a CRV reward harvested at selling CRV for W. It's

like it's also wrong. But what I want you to take out of this is that this is the same model, same day, same data. It's completely off. It's not a different number. It's a completely different tax statement, a completely different understanding of what the transaction does. And this is the latest model before fable. Basically, this is uh oppus 4.8. It is insane because you cannot rely on them. They just give you different stuff. And why? Because of how LLM work. They do not read the chain. they do not actually understand what's happening. They are patterned matching to what they have um read about what curve or convex transactions look like.

And this is the fun part. I push back a bit because I I I I believe that this transaction also has a CRVD movement that I remember spending from my wallet when I executed it. So, it should be counted as an expense for tax purposes, right? So I just kind of try to lower my tax bill because I mean who wants to pay taxes, right? And yeah, so yeah, I remember correctly. See indeed this 6 2 3 26.73 CRV USD spent as a fee is a deductible expense. So that means that I can remove it from my taxes. Yeah, saved money. And you know, I can send this table I guess to the tax office. and the Terry on top, it thanks me because I pushed back and I

I I was I was I was right. I saved money. So yeah, the thing that I want you to take out of this is um you can't just do your taxes by throwing it on an LLM. LM are super useful. I I use it myself but for different things not just throw everything in it and I just expect them to do your taxes. So what this means is that um they have all the four problems that software as a service has um and a new one and that new one is that it will basically help you arrive at the wrong answer uh because you will push it because you come with the with the idea that this is the result I want to get

and even subconsciously you're going to be pushing it to give you the answer and to um verify your the biases that you already um uh a quote that I heard someone say, I really don't remember where, but it's like that the the user becomes an active participant in their own hallucination. This is like something that I heard and it really just matches exactly what we have seen with um LLM and accounting. So using an like real accounting tool, you just can see the same transaction in rod key. It's pretty simple. you just see what happened like gas fee there is a revoke of an approval inside that those logs there's a deposit of CRV USDC

in a beefy vault uh you get the LP token and then you get the kickback uh the harvest call reward everything is as it should be if for some event we don't because we also make mistakes uh like the the decoding can be wrong you can edit it because it's completely customizable and you can also open a bug uh or create a PR to to fix the classification uh for P&L. So for for tax purposes, it's even simpler. You just have one expense of 4 cents and an income of 0.00007 euro. That's it. There is nothing else here. It's dead simple. That's for almost every jurisdiction.

So what I want you to notice here is that there is no convex LP swap. There is no selling CRV for W nothing. This is what happens and this is what how the tax uh office should treat this. So why you should send uh the report to the tax office. Um, of course we also um have some um uh like Rock is not perfect like to get the history you get it uh easily but to get the tax reporting information you have to run something called the tax report and at the current road key it's a bit slow and monolithic report that you have to write to start and then it ends and we are working on making this um uh

background task that always runs whenever you add a new event or whenever you edit an event. so that you can always have the tax information available. So, but this is not like zooming back a bit. This is not really a road key issue. It's the issue of tools not being auditable by construction. So, I would like to push for this attribute to be able to audit your own tools. Um, this is like a good breakdown of of what we're talking about that basically on the left you have software as a service now LLM. So you have everything that they do for a tax to visit a classification hidden inside their server logic. You you you don't know what they are doing or why.

Very important there is no reproducibility because the logic changes between versions. So you cannot really access the old version. Also very important you don't have your data. They can cut you off. It lives somewhere else. You can't exit. And with the LLMs basically, you know, they're confidently wrong and helpfully agreeable as we just saw with the guy that helped me reduce my tax bill. For tools that actually do auditable accounting, um, everything is visible and editable. So for the same input, you should always be getting the same output and you can access and rebuild old versions. This is really important because then you can actually say okay the new roti version let's say doesn't give me this uh amount but I can take the version like from a year ago and run

it on the data and then I will be having a reproducible uh result. The data is with you uh so uh they're accessible only to you having open source code and especially something like ACPL will at least the company um if the company dies then you can still get your data out and be uh be fine. you can just still uh use another tool or even old versions of the same tool. Um so yeah like disclaimers of course we also get things wrong but um you can see what we get wrong. It's like hit you immediately in the face. It's open source code and you can basically tell us and we will fix it. But even if you

don't want to bother, customizability is really really important and something that I really wanted to add to uh the tool from the very beginning. So you can actually um edit everything and not need to bother the devs or like especially if you're at the last minutes of your uh tax deadline, you probably just want to edit something, make it work and and file it. And the fact that you can see everything and edit everything. This is basically uh what I believe is the killer feature for such tools the transparency and the auditability.

So again this is not only about rokkey like the category of auditable accounting is something that is really important even if we disappear tomorrow the like building tools that are auditable is something that everybody should be doing. Um, and especially people building tax tooling, they should be building up to this bar. And people that are funding um tools, they should uh uh strive to fund tools that meet it. Um, this is like a really important slide that I want you to to to remember.

These are questions that you should be asking of all your tools, not only accounting tools, but anything. So, does the tool show you why it calculated uh what it did and how? Number two, can you customize it? Uh number three, can you read the source so you can decide how um the tool does stuff? The reproducibility again that we said, does it produce the same output for the same input? Does where does your data live? I mean, even if it doesn't live locally, where does it live and who has access to it?

Uh, can you export everything and walk away? Basically, this is the walk away test that you've probably heard from others. Does the tool survive its company? This is also part of the walk away test because companies die. They run out of money. You don't want your data to disappear if the company that you're using runs out of money. So, like remember these questions and ask them of every tool. And then I have three asks. If you are filing taxes, ask these questions from your tool. If you're doing accounting or anything, do not use tools that do not build up to this bar. If you are building, then build up to this bar. We are an ecosystem that is um based on auditability. So, we should at least have tools that um um aspire to this.

And if you are funding for the same reason, please fund tools that meet it. And for the biggest task, last slide uh try roti. Uh we are basically uh the open source self-hosted uh application. Here you will find like all the links about rod key. You can just download it and start feeding it your addresses, your API exchange key. So it will take your entire history across every EVM chain or Solana or Bitcoin or Substrate. create this in every exchange of course create this data lake inside your encrypted um SQLite database and then you can hit it with any questions any

queries you want you can explore your own history you don't you don't depend on anybody you don't depend on ethanol you don't depend on blocks you don't depend on anybody and um even now with you know the good thing of LLM you will be able soon next version to audit and ask questions through an agent through an MCP server. Um, so use an AI to ask questions and interrogate your data. Tell it, you know, I don't know, check my no income in the last week and give me like make a graph or something because no front end is best for everybody. This way you can actually just use it and have it do the stuff with your data that

you would like to be done. Um we're a small project so please spread the word like if you like what you are hearing if you like um the principles that we stand for um tell others because uh we only have the only marketing is uh by word of mouth. We have a premium subscription where you can um unlock extra limit and unlock the premium features. This how we found ourselves especially in such um really difficult market. We really rely on our premium subscribers to survive. So uh it would be really appreciated if you like rodkkey to be a premium subscriber and then of course we are around in whatever like octant gith uh gitcoin you can donate to us support us and we also have

um release sponsorships right now. So if anybody has um either personally or an organization wants to uh be featured in ROKI, you get uh you mint an NFT and then you appear in our change log in our download page or inside the app itself uh you and your company uh depending on the tier of the NFT that you're minting. Uh so that's all. Uh thank you very much. I am ready to take questions now if we have any time. Thank you. I think we still have time for a question or two.

>> Um thank you. Simple question. Do you like how many countries do you cover because obviously there's different country by country. >> Uh officially we don't cover a particular country. We are just we just have the accounting rules that are editable and customizable and then basically you can do uh accounting like and tax reporting for countries like Germany, like u the UK and the US. Um we can't say that we support the country specifically because of basically we are not tax accountants. We are an analytics tool.

That's that's ex especially I think a problem for the numbers you gave us because it's hard to imagine how how a transaction actually is treated in each country. I'm from Germany and even I can just pull out from a LLM or I can sell the LLM token. So these might be different different transaction and for tax purposes differently. um managed. So it's hard to see just the result your app gives us.

>> Yeah, exactly. That is why every single I didn't cover the app that much in detail. Every single uh event has editable rules that you can just uh change. Um because I'm also working with tax accountants personally in Germany like uh I use roti of course to do my own taxes. So I customize it so that it has and by default it tries to follow the German uh tax accounting like the the default rules that it comes with are for Germany. So the one year rule like the most important thing uh is in there and we like changed stuff when the letter

from the don't remember the name uh but the letter that came to change the the guidance for things. Um but uh the idea is that we want to empower every user and every tax accountant in different jurisdictions so they can use the tool for their own jurisdiction. It's basically I would say impossible to have a tool that works for every jurisdiction because like Germany alone will change stuff not really often but then there is so many countries and there is no EU specific accounting for crypto right like the Netherlands for example I think that they just take the last year the last day of the year and they tax your your wealth on the

particular day uh we we we've made ways so that we can also allow that to be done in Roti right So it's very difficult for a single tool to even claim to be able to do all this. That's why we want to make it customizable so that you have the power as either an accountant or a a user to just customize it to the best of your ability to get a proper number because also the I don't know the coin list or the block pit or coin tracking all of them they claim they do it right but they don't like we have done so much uh opposition research let's say uh and also with my own data because I file taxes every year here and it's just wrong wrong completely Okay, thank you very much.

Yes, please here and with that uh we can start a short break. Uh we will start at 15 4. Thank you very much.