The Future of ZK Tooling: Insights from powdr

Transcript

So we want to provide high performance zik proofs and at the same time provide seamless developer experience. We basically we see a future where we don't have to do anything different just because you want to have a zikk proof for your rust program. I think a lot of the current efforts in zikvms are very focused on ethereum. OpenVM is the first integration that we have with powder. So any ZK proofs that are made with OpenGM can automatically make make use of auto priompiles. [Music] The term will sound familiar to any developer writing code day in and day

out. But imagine how much simpler life could be if those pre-ompiles were automatically synthesized. Yes, that's what we are going to explore today. End to end auto pre-ompile. Welcome back to ecosystem project demo. This is episode 26 brought to you by EC Institute. I am Puja Ranjan joined by Akash from our team. Today we are talking about powder, a project introducing an approach to automatically synthesize pre-ompiles for ZKVMs. It shares end-to-end proof generation, performance number and aims to make developers life easier. What is this project and how is it impacting the

Ethereum ecosystem? To unpack it all, we are joined by Leo. Leo, welcome. >> Thank you. Great to be here. >> Leo, please tell our listeners a bit about yourself and your journey in the Ethereum space before we get into the presentation. >> Yeah, sure. Yeah, my name is Leo. I'm one of the founders of Powder. So this is a project that started within the Ethereum Foundation in 2023 and late last year we spun out as our own organization and yeah basically trying to push for better ZK tooling in the ZK ecosystem. Before that me and the other founders we were working at the foundation for a long time. I first joined as a solidity core developer and then I was leading the full verification

team for for the longest part of my my time at the >> awesome. So you already have experience of core development and things that come on the way of code developers. All right. Uh this is pretty interesting background. Thank you so much for sharing. I understand there is a small presentation. So let's uh get into the presentation right away. >> Yes. Yeah. So this is a very short presentation on the main things we're doing at powder and with powder. So the high level idea of what the things we want to do as a as a group as a team is to improve ZK tooling or improve developer tooling in the Z ecosystem at the same time as we want to improve proof generation performance which is also a lot of people's goals. So we want

to do these things at the same time. So we want to provide high performance zk proofs and at the same time provide seamless developer experience. And a lot of the work we do is kind of in in the middle of we don't do for example zigg prover work prover code ourself or zik research or cryptography research proceed we work more on the compiler layer where we we have access to different programming languages and different VMs different paradigms and we do this connection from high level code or circuit code to zik provers and zik provers have different types that can be optimized in different ways So basically we built this bridge that lets people make ZK proofs in a highly performant

way without compromising developer experience. So of course this is a common topic in Zik VMs. That's one of the main reasons ZKVMs exist. So people didn't want to write ZK circuits anymore because they're difficult to write, they're difficult to audit, they're difficult to maintain. Um and at the same time you need a team of experts. It doesn't scale a lot if you're just writing circuits all the time. With cicms, you can basically make proofs for Rust or or even Go programs. You can reuse a lot of the code you already have. So you don't have to rewrite things from scratch in the new DSL or new language ecosystem. With that come different challenges. So ZK proof performance has higher requirements when you're making proofs with ZKVMs as opposed to just standalone circuits. But

yeah, so there's a lot to do in the intersection of these two things. And this is where the main chunk of our work exists. And the the two main things we work on are auto pre-ompiles and wmir. So the main thing currently is auto pre-ompiles. So what does that mean? So zikvms they for example if you take into account risk 5 zvms you can make proofs for risk 5 programs which usually come from rust or C++ but the basem can have a lot of overhead in proving because it has to be generic over the instruction set. And if you're thinking about things that like Ethereum needs a lot when you're verifying Ethereum block, there are things like catchak hash, ECDSA, signature verification for every single transaction that you have to do. These

are pretty expensive things to prove if you're just going through risk 5 code. So usually if you did this in a circuit, you would have personalized or like customized code that verifies those things specifically and then you get better performance out of this. In in ZKVMs, these are pre-ompiles. These are not the usual Ethereum pre-ompiles. They're so-called pre-ompiles. These are handwritten circuits that are more than running the same thing over the software version that gets compiled to to risk 5. And these circuits are usually they have a similar developer experience to the old style circuits people don't don't want to write anymore now that ZKVMs exist. You still have to write them manually. They're big. They're difficult to write, difficult to audit, difficult

to maintain. The process of writing these circuits doesn't really scale. So you have to do it manually. You need expert knowledge. So it's not very productive to just keep writing these things by hand. With auto pre-ompile which is a library that we built with inside powder which has a different toolkit for a KVM acceleration. This module basically builds it synthesizes these pre-ompiles from the guest program automatically. So the developer doesn't have to write pre-ompiles. These circuits are generated automatically just from the program you're trying to prove. So if we take a RS program that's trying to do catchup computation even if it doesn't have the manual pre-ompile doing catchup the manual circuit this module will see that catch is is a heavy piece of computation and this benchmark

and we'll synthesize a circuit using the original code but then optimizing it a lot that does the same thing but in a much more optimized way for the ZKVM and this approach is agnostic to ASA it's agnostic to ZKVMs it's agnostic to the program it can accelerate any guest program that you want to make a proof for. And here are a few links. So this is the main repo where we have everything and these are two blog posts we wrote on first on the technique itself and how it works and then on integration that we did with OpenVM which is a high performance AKVM that has access compiles as well and then we published some performance numbers which I can also show here. So just wanted to

quickly mention this. This is the blog post where we go through the technique um have a benchmark on catch. So you can see for example here. So this row here is what you get if you just make a proof for the risk 5 program that does catch. There's no manual pre-ompile. It's just Rust doing catchup. You can see it's not very optimized. It's very slow compared to the other rows. And this row here is when you add the manual pre-ompile. This is about a couple of thousand lines of Rust code, complicated circuit, difficult to audit and so on, but of course it's very performant. So it reduces the time to make this proof by 12x basically. So you can see it's it it's worth it, right? So if you're doing

a lot of catch computation, which Ethereum does, you will want to have the circuit anyway. And in the latest iteration we have or that we ran the benchmarks with auto pre-ompiles, you can see that it actually accelerates the benchmark even more than the original manual circuit. So this circuit here does the same job as the circuit here. It's not exactly the same. They have a few differences but mostly and you can see that it's actually it provides more acceleration than the manual circuit that was written before. So this is basically like the in simple terms what we want this approach to do. You want to it's two things basically provide better security developer experience and performance to the alternative which is

writing manual pre-ompiles but at the same time there are manual pre-ompiles written for things that a lot of people care about which is catchda and so on but if you have a rust program that has some heavy computation that not a lot of people have therefore people haven't written pre-ompiles for them yet you're going to have a hard I'm writing the pre-ompile yourself. So this approach is especially powerful for cases where people have not written manual pre-ompiles for us. So you can see here that it's actually 13 times almost 13 times faster than the pure software version. And the main things that is actually agnostic. This is not specific to catch. If you put different gas programs that don't have manual

acceleration by manual pre-ompiles the manual circuits, you still get some acceleration without actually doing anything. So and the reason why we do this kind of optimization is that a lot of the optimizations that are performed in ZK space they come from the cryptography side of things. So you have faster provers faster research that leads to newer research that leads to faster provers and new techniques. But there's also a lot to be done on the compiler side on the program side that often gets kind of overlooked because we have Rust we have a little VM how come these programs are not optimized? they are optimized for what they're compiled for which is not zkms. So when we use these programs in zkms the cost model is

completely different. So the cost model of zkm is very different from the hardware where these vms these programs are going to run. So by combining the compile optimization knowledge with the the specific domain knowledge from ZKVMS and the circuits, we see that there's there's a lot that can be done to accelerate these proofs on top of any performance improvements we get from the prover. So this doesn't replace the prover. So these things compound and these are orthogonal to prove improvement. So you can actually do both at the same time and improve many times the performance of of the prover. So the other main project that is slightly unrelated but eventually will be related is this project called wmir which means

write once memory IR the name is not final but we're going with this for now and it's a compiler from was to a zk friendly ISA and you can see this as for example a an alternative to the risk 5 pipeline that a lot of zvms use. So instead of compiling rust to risk 5 to and transpiling to whatever the language of the zkm is we compile to wasm which we can reuse all the compile infrastructure for then we compile to the zk friendly ISA that is one step lower than wir but the main point here is that it has a special memory region which is right once and this is inspired by tyrom which can bring a lot of performance improvements together with

the compiler work because it's another example of compiling a program for what it's going to be used or for where it's going to be used. So it doesn't make sense to compile was to a right one's memory if the ZKVM doesn't have a right one's memory. It also doesn't make sense for the ZKVM to have a right one's memory if the compiler has no idea about that and doesn't make use of this. So the idea here is that we have or we will have at some point the right one never implemented in the ZKVM. At the same time the compiler knows about that. So we can actually take WASM and optimize WM specifically for the ZKVM with right memory case which can give a lot of

different performance optimizations that are just not available in risk 5 for different reasons. And with this, we're also working on integration with OpenVM. And because OpenVM is pretty modular and it's easy to extend with different instructions, the new new instruction sets basically. So we're writing this new ZK friendly instruction set, which is really only a few more instructions than OpenDM currently has with risk 5. Then we can compile Rust or C++ or go to WASM to Omir to this ISA. And the idea is that we get a lot of we get zk proof performance improvements by having a better complete pipeline with more

efficient zk primitives and a compiler and an optimizer that knows about these and can can make use of them instead of just being a completely generic pipeline that doesn't know about things that can be optimized if you are compiling for a zikvm. So this is the link to the IR itself and the ISA and this is the integration with OpenVM. So yeah, so you can see a lot of our work or most of our work not always is basically working on compilers, building compilers, adjusting compilers and optim and optimizers from a high level programming languages to provide the best developer experience for people that want to make proofs for their programs and at the same time trying to make this as optimized as

possible so that the zikk proofs are very performant. >> Thank you. Thank you so much for sharing these. I mean you took me down the memory lane. I remember hearing about Ewasm team back in 2019 but beyond that it was not too much into active discussion when we are discussing Ethereum and upgrades. It's really interesting to learn that team was working in the background and now we have contributors coming out with individual project ideas and trying to support the ecosystem in multiple ways.

I also love the approach agnostic approach. So it's not going to be limited to Ethereum only. We can look into like the growth of the web 3 ecosystem al together. So that is great. Thank you once again for sharing that. You have covered a lot of things. My questions could be repetitive but I want to just have some better understanding of things where we are from a highle perspective. So my first question to you is about the main motivation. You talked about multiple ideas. It could be security. it could be better developers experience and performance for prover.

What do you think was the main motivation to design the project powder? >> So the main motivation I would say is the lower part here the seamless developer experience. So that was at least a driving force because there's also a bit of a delay and so if you think about how programming languages are used now that people are coding C++, Rust, Python, Go etc. And you don't see that much or that often anymore that people writing pure assembly code to optimize whatever they are trying to do as they used to 20 30 40 years ago. I don't know exactly. Um but this still is the case in Z development because it takes a while for the research to become production software to become developer

experience and so on and obviously a lot of normal software development has been done decades ago. So it's natural that ZK tooling is is a bit delayed, but we don't see why it has to be the case forever. And I'm sure a lot of people agree with this that there are a lot of people working on the same goal. And yeah, so we basically we see a future where you don't have to do anything different just because you want to have a zik proof for your Rust program. So it doesn't matter if I'm encoding Rust to run on my server, to run on my laptop, to run on a GPU farm or whatnot or to make zigg proofs or I don't know whatever other use case. You shouldn't have to it still is the case. I think this is this is still a very futuristic

thing. But um I do believe that one day we won't have to developers won't have to care anymore that they're doing for ZK proofs or to run natively or to run the phone or etc. they will just write code and you get a zig proof as as outcome. >> Very interesting. So if I have to understand about this project, it seems like auto pre-ompile is one of the most powerful feature of it. When preparing this auto pre-ompile, who was the main target audience in mind? Meaning was it the code developer or was it the guest developer? Assuming that people want to have projects open source and invite a lot of participation. Yes, on both sides it's basically both users and ZKVM developers.

So the the direct users of the code are the ZKVMs because they need to integrate the module. They need to basically use auto pre-ompiles in their internal circuit building. But there's different ways. So for example in OpenVM because they have a modular and extensible framework it's actually possible for direct users of OpenVM to use auto pre-ompiles without having to change anything in OpenVM. So and this is not only related to auto pre-ompiles related to OpenVM's architecture itself. So they've built it in a way that you can like you can import the normal base open VM and you can extend it with your own circuits if you want with your own instructions with your own pre-ompiles and as part of these this can also be

the auto pre-ompiles right so there's two sides here so on one side I would say it's for end users that want to accelerate programs that don't have pre-ompiles for yet and that they don't have the expertise or or they don't want to spend the built in pre-ompiles manually. So they can just plug that in they should get acceleration just like that. The other side is ZKVM developers and this is a bit more like cutting edge related. So it's like how can we extract the maximum performance we can out of the whole system. So can we use a mix of auto pre-ompiles, manual pre-ompiles, software version. It's almost like a piece of research as well. So, how can

we extract as much as we can from this proof? And this currently is especially important for Ethereum real-time proving for the L1. So, there is this I'm not sure how decided it is. It feels very decided, but I'm not sure. I don't speak for Ethereum protocol, but it might be the case that in a couple years there's this Ethereum snarkification going on where instead of having a lot of validators, we're going to have ZK proofs that are verified by a lot of people. These proofs are very heavy to make. So currently the ZKVMs that do provide real-time proving they use a lot of GPUs and they use a lot of manual pre-ompiles to achieve the desired performance.

Anything we can do to improve that further even if it's a little bit is already very um very important for Ethereum because there's still a lot of improvements we have to make to get to the point where Ethereum would be smartified. We're not there yet. Um so in that sense autorel can also be seen as a as a piece of research that's further accelerating the the state-of-the-art in zigiums. >> That is cool to know. I mean it is on the road map or on the horizon for the ecosystem. So perfect. We are going in the right direction. It seems like the project is definitely going to help out in saving time and coding time. Do you think that it is also going to significantly optimize uh in economical sense? the proofs would be more

economical now. >> Yeah, I think the I think this rel like the time and the the computational effort you take to make a ziki proof translates directly to costs, right? So these these machines are not cheap or I guess some of them are but like especially if you think about the other problems people would have to solve with Ethereum sanctification. So currently to get the real-time proving let's say we need 10 GPUs, 20, 40 GPUs. Not everyone is going to have that. So currently one of the things that exists for Ethereum is homeers, right? Home validators. So this will likely not be possible if we require many GPUs to become approver for for Ethereum necessarily exactly what

you asked, but just to draw a parallel between the cost. So if we need 40 GPUs, it's only the cost of having the machine or renting the machine, but it's overall the cost of also building the whole thing right. So with pre-ompiles, you also spend a lot of developer time actually writing the code, auditing the code, maintaining the code and this is both time and costs and time and direct like financial costs. The former it's quite direct. So you want to generate more proofs, you need to spend you need to spend more time in these machines.

It's going to cost more either by owning the machines or by renting servers or GPU farms. Um yeah, so I would say there's these two sides these two economical sides to to the question. >> Yeah, that is interesting. Of course, I think as the technology is evolving, we are trying to make it more faster, more economical so that it can be more decentralized at the end of the day. Having these technologies in place are helpful. My next question to you is with respect to the applications and maybe the early adopters by any chance you came across any project team or any lowhanging application ideas in your mind that you would like to share with people or you have met any early adopter

of this technology yet? Yeah. So for now we I think a lot of the current efforts in ZKVMs are very focused on Ethereum. So yeah, it feels like a repeated answer, but it just is the case that there's so much money and effort being put on um real-time ZK proving for Ethereum block verification which already happened before. So you already had ZKL2s before that had similar proofs to Ethereum verification. But now there's a really big this really big effort in making proofs for L1. Yeah. So, OpenVM is the first integration that we have with powder. So, any ZK proofs that are made with OpenVM can automatically make make use of auto pre-ompiles. And so, for example, the

ETH proofs initiative, I think OpenVM is not there yet and we're not there either, but it is one of the goals we have is to basically support Ethereum certification by providing proofs for it proofs as well. But yeah, so the I would say for now the basically the OpenVM ecosystem is the easiest way to get access to autops. >> That's pretty cool. In terms of challenges or maybe uh the blockers, what are the biggest challenge or blockers that you think are stopping the adoption of ZK as such in the present scenario?

>> Um I think one thing it's a difficult topic. Um there's a lot of very smart people working on it. But it's still a challenging field in itself. The research is complicated. Implementations are complicated. It can take a while for a paper to translate into the code. And many times when it does translate into code, it doesn't actually work in practice as people would expect. Um this actually happens a lot. And then when a paper comes out, you're not sure. there's a lot of promises but you're not sure if it's actually going to translate into practical performance or different features. So I think that combined with the lack of tooling and it's kind of like a cycle in a way because the reason

why tooling is not more established in parts is because you're not sure which Zack is going to win. So do you focus on starks? Do you focus on snarks? Do you focus on folding or different things? And it's not easy to make tooling that is generic over all the possible ZK pieces of technology. So you kind of have to bet on something. And the moment that you do you you try to build better tooling for that specific niche which is one of the things we're doing. So most of what we do actually works well with the air arithmetization used by Starks which is what OpenVM and other ZKVMs use for what they call the core or app

proofs. So for example if you wanted to translate auto compiles for um like a snark folding scheme it would not be trivial because we went on the air stark path. Yeah I think it's it's kind of like a cycle of it's a difficult field. You don't know which zik attack is going to win. you don't know what papers are going to work. So you refrain from making super generic tooling which is also difficult to build in the first place. I think it's just a that's delayed. I think tooling is is definitely getting better every year. We have better and better tooling betters.

I mean circuits are are back in a way with great projects focusing on circuits again but only ZKVMs which is also nice to see for client side proving for example. Yeah. I think a mix of of a few different reasons. Yeah, and I think Eat Proof team is doing great in having these uh calls weekly by weekly. As it grows, people will also start thinking about different tooling that can support the ecosystem as it is growing. Well, I would like to pick your brain on the EIP proposal uh which is um of a block level access list. Yeah, EIP 7928.

We had a meeting this morning and I was intrigued by hearing that we can perhaps use BAL for ZK trace generation and it will be a nice idea to do that though we understand that production may not be in momentum because ZK trace is like a little far from actually being produced on mainet but I want to hear from you what do you think about this proposal and do you think this would help uh the work that you guys are doing here? >> I'm not familiar with the AIP so I would have to take a look. So what's the number?

>> 7928. I mean it came on general discussion that's why I thought of checking it with you. Like it seems like using BAL would be making a proofing easier for ZK proof. >> Oh is it for for to do parallel trace? >> Yes, I guess so. >> Right. Yeah, that makes sense. I mean succinct did something. I don't know if it's ah maybe it's actually the same thing. I don't know if it's exactly the same thing, but when SP1 first presented the real-time proving that they had, they had it on top of like a parallel block execution, which basically enabled the proofs to be parallelized a lot better on GPUs.

So because if you have to sequentially execute everything to build the witness data and then spawn parallel proofs you lose that there's like there's time that you could have gained by pipelining the trace generation before like while spawning different provers which is what they did I think for their first real time proving um benchmarks it might have been this this EIP I haven't been following EAP too closely but I think it might be the same. I don't want to say that it is, but if it's not this one, it's probably similar related one, >> right? Uh yeah, I do not want to get into the technical details of it. I was just wondering if it uh was even

discussed in the community that people are trying to bring such proposal on Ethereum mainet may be helpful for ZK people. >> Yeah, I think again if it is the same thing that I'm talking about or similar then it would definitely help help a lot. >> Very cool. Well, my next question to you is about community contribution. How do you think that u general community may able to contribute to powder's road map or even in the adoption? Anything as a call to action that you would like to share with them? >> Yeah, I think integration and benchmarking is a really good way to get feedback on what people want and need.

So, it's difficult to say up front which optimizations we should be implementing or which features we should be implementing. And this is ultimately an infrastructure project. So it's difficult to go and ask people exactly what because we don't necessarily talk or work with end end users frequently. So a lot of the people we talk and work with are ZK developers or ZKVM developers or researchers. Often it's it's difficult to be accurate about what's the very next thing we should be working on. And usually these things come from benchmarking but benchmarking requires integration. And we are definitely interested in integrating these two different techniques to different provers. So for example, one idea of me is to integrate different

provers that may support redness memory or for example circuit languages could be an idea as well as well with auto pre-ompiles. It can integrate with almost all ZKVMs or different projects. Doesn't have to be ZKVMs. It can also be ZKDSLs. It can be circuit languages. I think this could be very useful as well. And even related to what we asked before about adoption, one thing I forgot to mention, it's not only Ethereum or web 3, there's a lot of work outside of web 3 or blockchain that goes in a similar direction. I don't know if you've seen the the Google results from a few months ago. So they have uh some ZK proof support in the Google wallet now. And this was done completely inhouse by two

researchers at Google. and we had contacts with your foundation people with different ZK researchers and the tools that had been built before just didn't work for their use case which was age verification on the phone. So they wanted to make proofs that you were older than 18 on your phone and for this they needed a very specific setup. They wanted to be able to parse your passport or driver's license or something locally on the phone and make a hash of it and make a signature verification. That's all. And with existing things that we had in the ZKVM world, even the circuit world just wasn't enough for them, they did reuse some literature ZK protocols, but not the tooling that we had for other things. So I think the adoption is

actually much larger than when we think about ZK in terms of web 3 or Ethereum or Bitcoin or L2s. Yeah. >> Yes. Of course, sometimes we can have tunnel fish and just focus on Ethereum ecosystem, but there are other development and adoption going on. Glad to know about this one. Well, I would like to hear from you. How do you see powder evolving compared to other ZK projects? There are multiple projects and I understand that ZK is going to be one of the most active conversation projects and applications those are being developed. So, what do you see?

Where do you see powder to be standing there? >> Yeah. So we do want to keep basically going in this direction. So seamless developer experience has always been our focus and the main thing we want to provide for people also based on the expertise that we have in the team. At the same time yeah also aiding high performance zip proofs. So I think what I see mostly is to continue providing better and better tooling to get to this goal here where people don't even notice they're making Zika proofs and it just happens. they just open some software somewhere and it just happens that it makes a zip proof for them happening either on a server on their phone. This is I guess technical implementation

detail but that's a seamless part for developers and users. So I think this is the part where Ziggy proofs just become a library just become an API call that you make and then you get a proof and someone else can verify it and I think this is the direction we want to to keep pushing and at the same time we love to collaborate with different teams. We have collaborated and we're still collaborating with different teams working on different things like ZKVMs or VMs or Ethereum protocol or different compilers and this is basically how how is it. to continue pushing the seamless developer experience direction while collaborating with other teams that are building different parts of of the puzzle.

>> Very interesting. You have briefly mentioned how you want to see the ecosystem getting involved around zero knowledge. If everything goes right, what does success look like for powder in like 5 to 10 years from now? >> That's a very good question that I would like to have the answer for. I would say that the one of the powder compilers in the powder toolkit is one of the main compilers being used by web 3 and non-web3 developers out there. >> Maybe one of the most used compiler >> hopefully. Yeah.

>> Very well. Thank you so much, Leo, for sharing all this information about this project. It seems pretty interesting. It seems that it's definitely going to ease off burden of developers who are sometimes kind of reinventing the wheel and trying to get into the complex codes of pre-ompilers. This auto pre-ompile might relieve them and help them do the coding faster and we can see the project simpler and sooner. >> Yeah, thank you. Uh thanks for the chat and thanks for the the great questions. >> Very well. I'm so excited for like seeing the adoption growth and we hope that new application which are coming across web 3 should also adopt it. Thank you so much for your time once again and uh we hope to see you around.

>> Thank you. >> All right dear listeners you will find all the links and resources in the episode description. If you enjoyed this conversation please share it with another developer. You know helping your fellow builder is how knowledge spreads. That's how we can make ours ecosystem even more robust. We'll be back with another exciting ecosystem project demo. Until then, keep learning and keep building. Keep sharing your knowledge with people and keep shaping the future of Ethereum. Cheers everyone.