soldb CLI tool for debugging Solidity and EVM rollups - Roman Mazur

Transcript

Okay. So, I'm Roman from walnut.dev. Uh, we're working on debuggers, compilers, and my talk is about debugging solidity. So, this is from the recent uh solidity survey. It came out in April this year. There's this question like what are the biggest problems with solidity or which feature would you want to see next and what stroke me? No.

Okay. What struck me was the fact that better debugging tools are second. Uh yeah, I'm not sure if you see the light, but they're right here. Uh so solidity is around 10 years old and still developers are asking for this kind of basic thing such as debugging. So my talk is split into three parts. Unfortunately I can see that there are no animations uh but we'll survive. So the in the first part I'll show you the best debugger you've ever seen for solidity.

Um I'm very keen for feedback either here on stage or afterwards. So if you have some thoughts or if you don't like it or if you like it, please come and speak. We're very keen to improve it. Uh but as we're building this debugger and trying to like solve this problem of solidity not having great debugging tools, we discovered some challenges. So I'll speak about that in my second part. And um lastly, I'll share some improve some proposal from improvements because those challenges um they are not just for us but once solved uh they might improve debugging across the ecosystem.

Okay. So let's first check how debugging looks today on solidity. Okay. So this is like a setup of a engineer. On the left they can see they have the app they're working on. They're building a D app. Here is a uh anvil running. My example is on Foundry stack but uh we would like to explore also making it uh work for HUD Hut. And on the top right there is VS Code.

So this is the contract this developer is working on. And here is how they can debug things today. So they would like click in the D app. They would initiate some transactions. They would see them logged here in this terminal. At some point they will see something like this like the transaction reject was rejected. So now what this developer needs to do they need to think which action caused this failure. They need to go back to UI try to replicate like do the same same set of steps and then go to the code put some logs.

Yeah, that was this put some logs and try to replicate this whole thing hoping that the execution is going to hit these lines and they will see this um logged in the terminal and that they will be able to fix it. So I asked JPT like at what point did software engineers get tired of console log debugging and started implementing visual step-by-step debugging. This is general stuff not just solidity. And the answer not surprisingly between 60s and 70s people were using console logs around 70s 80s that's where first visual debuggers started to pop up. So I think

with Solidity we are somewhere somewhere here like we're we're on the verge of going from console logs into visual style debugging. Okay. So now the demo same setup. Uh this is the UI here. VS Code. We don't need the um terminal running. I mean, Unveil is running in the background, but the developer doesn't need to have it on their screen all the time. As you can see, this is in debug mode. So, there is a uh indication that the debugger is uh launched. It's using Soul DB, our CLI based debugger.

Uh what I'm demoing is a VS Code adapter plug-in. And take a look at this section here. uh this debugger is attached to anvil and it's listening for transactions. So this this can serve as a little uh local explorer. So when the deb when the developer is making actions here in the UI, the transactions are going to get locked. So and you can see the icon, it it indicates whether the transaction failed or succeeded.

And you can see like the hash and the entry point decoded. So now let's go through the same flow as before and um see how the debugging would look here. So we're going to make some actions in the UI. So the the developer is like clicking vote, going next, vote. Now new transaction popped up. Everything is looking good again. Clicking vote. Oh, something weird happened like there is an error, failed transaction and the popup did not disappear. It should have.

So what can we do? Well, as you might expect, we can just go and put a break point to that place. The the developer doesn't need to replicate the UI sequence second time because the transaction is already logged. and they can okay the the break point was set and they can go down and there is this little play button and they can replay the transaction locally.

Uh so um you know they can debug it. So here I can hit play and now we are in debugging mode and we can go next and next. Uh obviously there are variables so it's all like what you would expect. So so this is the demo. It's available on GitHub. You can go try it yourself. I'm not going to continue with those steps but I'll talk about them in my second part of the talk which is the challenges. All right. So just to sum up this is the debugger. It works locally and remotely.

You can attach it for to your local node, but you can also attach it to if you're building a roll app or uh if you want to debug something on chain, you can give it a custom RPC. It works with local source code, but we can also pull code from sourceify. So, this really would enable debugging anything on any chain uh if the source code is a available on source. Okay. Now the challenge the challenge is in mappings. So um I didn't tell you at the beginning but we tried to build this debugger um based on e debug which is the the new format only on the e debug.

We didn't want to use the source maps generated uh by uh older versions of the compiler and the mappings are just not there yet. So I'll talk about that now for a second. So very high level how debuggers work. Uh the compiler takes solidity code and it generates two outputs. One is the executable and second is debug information. Okay. And the debugger needs to consume both of those files and do mapping.

This debug info this this file is critical for debuggers. like it needs to contain a lot of information that was lost throughout the compilation. Uh and it's not great today when it comes to debug info generated by soulc. But what what do I mean by that? So I'll skip these two slides. Okay, one more uh one more point about uh debug info. So we're seeing new compilers pop up in this ecosystem.

uh there is the new SOLC C, the old SOL C, solar and so X and all of them need to generate this debug information so that debuggers such as Tenderly, Sol DB which is the one we are talking about here today or EDB a new cool debugger out there so that all of them can consume it and it's very important for this debug info to be standardized and here I would like to make a give a shout out to Nidita for his work on this. Like this is very important for debuggers like to have one standard that every compiler is following otherwise we would need to do custom integrations with every

compiler which would be very painful. So so this is this is very very good and very important. Now I told you that the Eid debug generated by Soulc today is not very good. So like the the question is how not very good like can we quantify it? Can we give it a number? Uh so that like can we build one like one single KPI and run it through and generate something like this compiler with this version uh generated E debug at level five or something like that like something similar to what we see in rollup stages, right? this L2B team came

up with really nice simple numbers 012 so that all of the rollup apps now can aim for the same goal. So like it would be nice to have something like that for each debug and uh we tried to come up with something uh it's pretty early but should give an idea. So this is the second repo I I would like to show to you. It's called solidity compiler benchmarks. And what it does is it takes as an input it takes a list of contracts. Currently there is only five but we can configure it to take like many contracts. It can take everything from sourcify for example or

like some some subset of contracts from there compile it and then run some checks. And we implemented two checks so far. One is the line coverage checks and in here we basically look at the at each line in the source code in solidity and we check if this line is represented in E debug generated. If yes, it's good. If no, points down. Then we do second check which is variable location tracking. So similar we look at the source code at each variable and we check if it's represented in in debug.

We have support for all three compilers and multiple versions. Of course, it's CL C cli based so it can be integrated uh into like some automation. So with each new compiler version we can generate a little report so that if if there is a regression in each debug uh the development team know that they they need to improve because it's important to realize that it that debug info generation is a it's an ongoing thing. There is work on compiler on improving it but with each improvement the debug for also needs to catch up.

Okay. So I'll now show the uh output that is generated by this benchmarking tool. So this is how it looks. You call it bench. py compilers and you pass all of the compilers you want to analyze. So here we're passing the latest sol but also the legacy one so x and solar and it's analyzing five contracts which is coming from some config file. And it does this to checks. And here you can see the quality of u its debug. So if we look at the latest compiler with via AR,

uh the line coverage is 80%. It should actually be 100 because uh one of those five contracts is using Pragma with some old compiler version. So that that that doesn't count. So line coverage is pretty good with ED debug on the latest. Everyone else, everyone else is zero. When it comes to variable location tracking, everything is zero at the moment. So that's the state of E debug. Uh just to sum up, it's a very important standard but it would be good to improve it a bit.

the generation I mean uh okay so just to summarize the community wants better debugging tools like we see users or developers asking for better debugging tools and we also see teams like us building debuggers so like the interest is there uh solidity compilers currently emit not amazing debug info And I would like to urge to align on some benchmark so that we can at least talk about improving it. And you know I understand it's going to take a while

but I think this could be a good start. We propose two checks but obviously more can be implemented. We would love to get into discussions on this and to contribute. That's my talk. Thank you very much. Here are all the links. >> [applause] >> Thank you so much, Roman. That was a very insightful talk. Um, we have two questions from the app, but in the meantime, if you haven't posted your questions yet, please uh scan the QR code to post your questions. The first one is, what RPC node requirements does Walnut have?

Uh we're using so for the demo that I just uh showed uh it's using anvil. So it it was just built on anvil. It's that's the current situation. >> Okay. Any plans to support whim or neo-wim? >> Uh yes, good question. Uh so the way it is architected right now is there's this CLI based tool called soul DB which is kind of like GDB. It's CLI based. It's very low level, but that's where the brain of the debugger is. That's where most of the work relies sits. Um, and then when it comes to Visual Studio Code or Neo Vim, we just need to build

adapters, but that's much simpler when compared to the brain. So, um, like we can prioritize it. Uh, if someone like whoever posted a question, hit me up. But it's it's not that crazy to do that to to like implement it. >> Sounds good. Do you have workflows for debugging readonly operations like ETH call? >> I'm not sure I understand the question. >> Does the person who asked this question want to clarify? That's Nick. [laughter] >> As opposed to transactions, right?

>> Yes. But debugging like >> ah okay okay. Uh so the question was not to debug transactions but uh things like balance off. Uh no we have not thought about that. Sorry. >> Yeah. >> Thank you so much Roman. That will be all. >> Thanks a lot.