ARGUZZ: Testing zkVMs for Soundness and Completeness Bugs

Transcript

All right. Um, yeah. Hi. Uh, this is joint work with um my colleagues um Kristoff and Maria from Teov. So um why should we fuzz uh CKVMs? Well, there's essentially there's two main reasons. Um one of them is uh that they're incredibly complex. There's a lot of optimizations people are doing implementing to get them to be even faster. Um and that of course has a possibility to introduce bugs and of course they will also uh become very critical in the near future for the correctness and security of upcoming blockchains uh including the Ethereum

L1. Um so there are two main um bugs. The the most critical bugs are uh soundness and completeness bugs. And the soundness bugs in simple terms is essentially when a ZKVM uh accepts an invalid execution. And the completeness bugs are essentially kind of a dual uh where uh a ZKVM would reject a valid execution. And as you can imagine, both of these bugs are critical and they could result in catastrophic financial and reputational damage for the teams building the ZKVMs, but also anybody that's building on top of them, right?

Um, and uh, in this talk, I'll give a quick overview of how we use fuzzing to detect those. Um, yeah, fuzzing is a testing technique that's been used uh, widely in industry to find bugs in very complex systems. So, Argus is the first um fuzzer that allows you to detect soundness and completeness bugs in ZKVMs and it already supports uh six different ZKVMs uh and found 11 critical bugs in three of them. Uh so, in particular, it found three uh soundness bugs and eight uh completeness bugs in these VMs we tested. And we also found uh we also got some bug bounties for the bugs we found.

Um and the whole infrastructure uh builds on um another fuzzer we built um a while ago called circus which is targeting these uh circuit processing systems like uh for instance Circum or Noir and it found also a lot of bugs uh 20 uh soundness and completeness bugs in uh four different systems. And if you want to learn more about this uh fuzzer um check out my uh talk at Defcon last year um or you can also look at the research paper we um published very recently at CCS which is a top uh security conference.

So now let me try to give you a very uh brief overview of uh how Argus uh works. So the first thing um that Argus does is it will generate a rust program uh R with an expected outcome E and uh this is one of the key innovations uh how we generate this uh rust program is is one of the key innovations in in Argus but unfortunately I won't have time to cover this in detail here. Um once we have the program uh we can then um go to the input generator which generates inputs for this program and um with these two um yeah with these two we can then actually run the the program on

the ZKBM. If the ZKVM crashes we um we find a completeness bug. Um they are shown in orange in on this slide. Uh the soundest bugs are shown in red. Um yeah, if the ZKM doesn't crash, we uh get a proof P and we also get an output O. So then we can actually use the ZKVM's verifier to check if the proof uh is valid. Um if the the verifier crashes or if it rejects the the just the proof that was just generated, then that would again be a completeness bug. uh if the provers um the proof sorry if the verifier

accepts the proof uh and the output is different from the one that um we expect then that um constitutes a soundness bug. So now you can already with this very simple um pipeline you can already uh find both soundness and completeness bugs but essentially we we went one step further by introducing uh so-called like adversarial uh ZKVM. So this is the the second key innovation in in Argus. Basically, we we took uh existing ZKVMs and kind of patched it to introduce some subtle faults during the execution that will allow us to generate invalid proofs. So with this adversarial ZKVM, we can then again generate another

output and another proof and we can again invoke the normal verifier um to see if it crashes. If it doesn't crash, we can again check if the the output that we get from from the the adversarial ZKM is different from the expected output in which case we found a soundless bug. And this is actually how we found all three soundless bugs in um that we reported. So uh unfortunately I don't have more time to to go into details but uh please feel free to reach out um to ask yeah uh about more details on on Argus or if you want to uh fuzz your ZKVM uh let me know and talk to me.

Thanks. >> [applause]