ZK12: Mechanism Design of a Decentralized Prover Network - Norbert Vadas

Transcript

[Music] hi everyone well two minutes ago I thought I was going to speak alone thanks a lot for coming so my name is Norbert uh I'm doing protocol research at gvulot and uh there was a slight change in the title um I'm going to talk about the mechanism design of the of aover network but actually that prover Network is a decentralized cloud specifically optimized for ZK we call it ZK Cloud that's going to be the end product the permissionless decentralized product and um yeah um so that's kind of how the initial title transition to this

um kickoff slide here um okay let's start off um before I go into details I would like to talk a little bit about why decentralization matters um The Proving Market is uh quite susceptible to centralization and that is because it involves like it's heavy on economics for example also if we look at the meev market or like block building on ethereum you see that it involves uh a lot of money you know heavy on economics and because of that it's actually a couple of um uh specialized Builders who are building most of the blocks on ethereum similar

way the expect that the proving Market can be equally centralized if not more um and because of that it's super important to talk about decentralization to design protocols that consider these type of uh centralization risks because centralization in itself is not just you know an issue because there is one entity or a limited number of entities producing proofs it's an issue because it increases the risks of liveness it increases the risk of like unhealthy cost structures or like centralized entities becoming like dominant in defining cost structures um it increases the life the the risk of censorship for

example or it may also have some scalability issues or Landing some scalability issues so uh the solution is from our perspective and that's how uh um we started building gavot is we need to rely on mechanism design principles that can address these risks um and that can address um these concerns one of the key decision points I want to talk about more is the workload allocation mechanism and the reason why this is important is because you can build a decentralized network it can be per like you can allow permissionless entry for anyone um anyone can spin up aover node and join the network but if at the end

of the day your workload allocation mechanism is such that it can be easily um like it's it's very susceptible to centralization and I'll talk a little bit more about that then you may end up losing out or like spoiling all the advantages that the permissionless entry or like the trustless execution would bring and let's let's go into more details what I what I what I mean by this so couple of design considerations when we talk about workload allocation I'm just listing here a couple of the most popular or known ones maybe auctions let's start off with auctions

so auctions are a good tool to decrease costs um on the other hand I don't believe auctions are the holy grail and I'll talk about that uh later on as well on the one hand it is because in in an auction basically the market will start forming the cost structures and like the how like how at the endend of the day how much the the proving will cost on the other hand if um protocol econom like proper economics are designed at protocol level around the cost of proving you don't necessarily need an auction in simple words if you can make proving cheap enough within your

protocol you don't need to to uh apply an auction to bring it even further down and I'll I'll we'll get there my main issue with auctions is that any Rich entity Super Rich entity let's say one of the big names Amazon Google whoever if they would like to come and become dominant um in the ZK proving space maybe they could afford underbidding everyone for years even doing proofs at loss until the whole competition is killed the decentralized permissionless network I built is basically down uh to a couple of huge entities winning all the rights to prove uh the jobs or like to complete the jobs

and then even if my intention was was to build a decentralized network my workload allocation mechanism allowed it to be kind of you know the the advantages of that to be spoiled so PO undercutting is real uh it's happening on like it's been happening on Mina's saret place for example it's a little different story I don't want to go into details but like entities started offering proofs for free on the marketplace which killed the whole economics and you know it's uh it's um unhealthy Dynamics order book based mechanisms are such that also richest entities could absorb the demand at any

cost if they can afford to absorb that demand and again create healthy unhealthy economic Dynamics proof racing is something uh I think it's also important to talk about we seen um like proof of work type of mining the most powerful entities can easily create uh or become dominant there as well and again someone has to pay for redundancy so it's never going to be the cheapest option obviously um stake based Lottery that's a more interesting already um more interesting thing Lottery hints at some more neutral solution uh to to allocate workloads you know it's kind of includes Randomness on the other hand um

since it's stake based again the richest entities will have proportionately higher shares like higher chances of being selected by that lottery so it again somewhat limits um the the decentralized network that one might want to build if it's paired up with this type of uh workload allocation mechanism and then we've got Randomness at the end of the list um that's as of now um I believe that's the most neutral and fair workload allocation um obviously you cannot prevent a large entity to spin up thousands of nodes in your network and thus increase the chances of them being selected for the

jobs even in a random network but that's just the property of permissionless decentralized blockchains so it's not like something you can uh you can um uh avoid on the other hand for them to do that it may just include a whole much more um Capital to to somehow tweak um the network to their favor so that carries the least risk of centralization uh a pure random selection and that's what we also voted for at our end uh please wouldn't then the last two option be kind of equalent no because Randomness doesn't take into consideration the amount of stake you have uh by in a way you you have money

you can buy more Stak equally if you have more money you can spin up and put more Capital yeah uh actually that's good point it they are somewhat similar in that sense we would need to do a bit more research on what which one would cost more uh it depends on stake size and it could it it could even depend on for example whether I could imagine delegated taking also being involved kind of like involving uh retail in running prover nodes and then you know delegating stake on behalf of the provs something like that so it's not the actual prover who would uh yeah so there could be some different aspects

there but yeah that's a good that's a valid point that needs to be uh researched a bit more please I can so with decentralization it's obviously as he hinted it's hard to get P true decentralization so do you guys break that down in into properties that you want because for example you could have a healthy Network where there's just one or two parties and if they ever go down someone comes in but then perhaps it's bad if those go down and no one comes in so that's like I'm going to talk about that yeah there are properties uh like much broader properties built around decentralization I just wanted to start off with this one so basically uh I just wanted to go

through the previous slide because it's worth for all of us to be aware of these risks and whenever we make decisions on where to Outsource proving because it seems the industry like that's the main um the industry is lean leaning towards outsourced proof generation and when projects or we make decisions on where to do that it's good to be aware of uh different trade-offs that we may come across uh when we select certain uh proof providers so the core principles that we've been following they are decentralization focused and they are like these decisions ensure that no one can control uh where proving happens um

it can offer like with these decisions and protocol level principles we can increase liess guarantees we can in increase sensorship resistance um add trustless properties reach a fair distribution or a fairer distribution of workloads and rewards and like all this adds up to a more incredibly neutral type of network that can serve the industry well uh I do believe that uh proving should be fast cheap and decentralized and that's kind of the Core V Vision with which we've been building the ZK Cloud uh at gavot um yeah so why decentralized Cloud basically as mentioned the industry is

leaning towards third party proof generation but most of the proof providers currently are actually using AWS or gcp um and we've talked about the risks that centralized entities May mean I don't want to go into details of that again but um with the Zay Cloud basically we are Distributing proving workloads across a permissionless Global Network of Prov nodes um we apply an underlying blockchain um to make that workload allocation and reward distrib ution um fair and balanced and also we needed an underlying blockchain because this is how we are able to uh

design economics flexibly so the main goal of gavot one of the key design goals of galot was how can we make proving ORD at least an order of magnitude cheaper than what it is now and uh for that we we had to turn away away from any existing like building on ethereum building on an L2 building on Solana whatever because then we would have inherited the economic structures of the underlying Network so it's a cosmos based chain uh nothing fancy um it's very minimal we have no smart contract State uh basically very minimal V execution limited number of operations it's purely uh optimized for

um processing proofs um and generating uh generating imprs and processing proof requests so we on a very high level we are aggregating proof demand and thus reaching a higher Hardware utilization for uh for our node Network um it's scalable because it can accommodate like it can horizontally scale without the economics being basically affected and um we have protocol level um execution guarantees fullback mechanisms um to ensure liveness um sensorship resist Etc um it's a basically what is okay as we get closer to what is ZK Cloud so it's

basically a decentralized proving and compute layer that is fast cheap uh and yeah I already mentioned decentralized it's fast because we have um high performance compute nodes optimized for ZK and we add on top of this node and network level orchestration um to optimize performance and also as I already mentioned we aggregate workloads from all across the industry I'm going to talk a little bit more on the details of this how we actually like what it boils down to at the end of the day but with this we are able to reduce costs um for all participants basically and it's decentralized because we have permissionless entry for uh both of both

types of our nodes both the validator and the approver nodes uh in the network so that's going to be in the ZK cloud the architecture has three main layers a universal proving layer an orchestration layer and then a a decentralized compute layer for the proving layer let's start with that so it's why is it Universal it's Universal because it supports any proof system so we don't have our own product ZK VM zkm whatever we don't write our own circuits we don't have our own proof system uh we have no preference for d L anything that can be basically compiled into an e binary could be run on gvulot um because of

this we are basically able to support any prover out there even any prover Network that exists or prover Marketplace or directly any project with zkp demand um and the integration is flexible like we are predeployment you can um as a user uh people can use the predeployment so do does the blockchain verify the

approves yes um I'm going to talk about that but actually it's a good question so we can discuss that here as well I don't need to be like super strict on the on the content and don't want to be so we do verification internally anyway because uh that's how we know the Prov did the job properly and based on that can we only pay the rewards to our approvers so when you deploy a approver program on gavot you need to deploy the corresponding verifi program as well so we use when you send a proof request to your own program for example referencing the hash of your Prov program um with the input Etc the proof is generated you can already fetch the proof and have it

settled wherever you wish to if you don't want to accept our verification because probably like ethereum would have a much higher Economic Security behind that but if you um if you want like if the verification or the security level for the verification on gelot is sufficient for you you can wait for us to do the verification as well fetch the result and then uh you got a verified proof we need that as I mentioned so we are going to do that anyway uh because that's how we pay the angle I'm curious about is the execution angle so basically the nodes in blockchain have to run arbitrary binaries basically yes

yes and I'll go into details like who is doing verification who is doing proving how do we select these nodes for the different tasks Etc regarding orchestration as mentioned it's a it's a blockchainbased mechanism managing workload distribution uh reort distribution we've got uh orchestration within the nodes um to optimize Hardware utilization Etc but among nodes like within the network as well um this way it's not just Hardware utilization or increased Hardware utilization but at the end of the day paired up with the aggregation from the industry and like being able to process kind of any proof request for any proof

system it brings actually just higher revenue for our Pro nodes as well much higher Revenue potential which with much simpler maintenance because they don't need to maintain 10 different prover nodes in 10 different networks but they simply maintain one node in gabot that Ma just uh where the network agregates the workloads and um yeah we've got built-in execution guarantees I'm going to talk about that later on as well and then as I mentioned we have a compute layer at the bottom permissionless entry for GPU nodes CPU nodes and also custom Hardware um the fabric team uh is also here they are building this general

purpose um Processing Unit seems to be very interesting so we want to allow easy integration of those exotic Hardware as well in the network and all of this is optimized for um ZK compute so we've got I think I mentioned the Dual node architecture so validators and provs um validators are randomly selected uh like the leader is randomly selected for every block um it orders transactions into blocks Etc it's the usual thing that validators do they form consensus there are the normal incentives I I don't think um I wouldn't go into more details regarding that

however um a little more details on on block content maybe I already mentioned that programs come in two varieties so you deploy Prov programs and the corresponding verifier as well um we do have plans later on to do verification solely uh if there is demand for that in that case the proof would be generated elsewhere and we just just do the verification but initially we'll start off with proving and verification together the blocks on gelot will include basically deployment transactions approver program deployment transactions um proving results verification results and as I mentioned it's a minimal blockchain no smart contract State barely any re-execution

there and regarding the approver nodes that's more interesting I think so when Prov nodes join the network we do an initial pretty heavy capacity verification so we don't want self-reporting we don't want to fetch their Hardware data um we give them test workloads which they need to complete within some time constraints and if they are able to do that they will be sufficiently large that we will know they do have the proper Hardware uh to be able to serve our users and customers and it's not just initial capacity verification but we are going to verify their constant availability and capability of generating proofs uh

through like random regular allocation of these types of test tasks as well um so a random Prov is selected for every workload the randomness will be calculated using the previous blocks hash we're going to be have a one second block time approximately or even a bit lower uh optimally than that so basically it's not really prec calculable per se um and then a subset of prover nodes sorry okay and then a subset of prover nodes is selected to do the verification so the Prov nodes are doing the proving and the verification as well obviously you cannot be selected or you won't be

selected to verify your own proof but uh there is one node generating the proof and a subset of the of the Prov set that is going to do the verification you can add redundancy because they do have a fullback mechanism so it means you send a request we allocate it to aover node the node fails based on the fullback mechanisms it will be fullback mechanism it will be right away reallocated so we want to make sure you don't need to restart an auction or re make take any action from your angle or from your side uh but we want to uh we want that to be baked into the protocol but if you don't want to run the risk of

having to wait double the time if the first prover fails doesn't deliver the proof and then you need to wait again yeah one quick thing you get 50% of your fees back if you need to wait another time so we we we do acknowledge that it's it's a disadvantage to the user so we we give back a large portion of the fees for in case that would happen but uh if you don't want to wait or run this risk you can increase redundancy and have two three nodes working initially on your request that is going to have a bit higher costs but it shouldn't be that big of an impact we'll see the cost structure soon um yeah please sorry you you mentioned that you didn't want to go

into many details about the consensus under when right I'm not that familiar with with the cosmos I guess uh but uh so the if uh depending on the security assumption right the consensus if it's a a state Bas or whatever right uh wouldn't a leader be able to sort of affect the outcome of the randomness coming from the as of the block that is proposing in a way by reordering you know censoring some uh transaction or proof that you can control little bit what is going to be the next round MH

that's a good point honestly I haven't really thought about that but uh um he will now yeah yeah by the way he's our integration and Integrations engineer Brian so he may feel free to chime in if you if you've got some information on those no that's something we we will be uh I I need to think through and we'll consider that when we are uh designing that like just if you're I don't know yeah it's a on the like it's a normal Comet bft based consensus the way it works is that uh in general in Cosmos uh and I yeah that's just high level uh it's a stake based Randomness so the

amount of stake self stake and delegated stake to the validators will Define their chances of of being selected as a leader we do want to move away from that and remove the the stake based uh connection just to make it random but we'll consider that thanks a lot for the for the input and for the uh for the question because it's useful and we'll need to think about that coming back to the provs so we are designing so-called custom prover sets as well where you could create small subsets of provs if you have some specific need for for example if you want to plug in custom Hardware not all of our provs would have that so this

custom PR set in that case the random selection would run among the nodes who have the proper Hardware or if you have some external software dependencies then in this custom prover set the prover nodes would need to run that external software as well to be able to generate the proof for you and also uh if like to facilitate easy uh working with data storage or availability Solutions uh like custom Pro sets will enable us to give a whole bunch of flexibility uh around that to the users a quick road map and then we are going to more practical details so we've had the devet up and running since March uh this year

it's been generating about 2.1 2.2 million proofs since March uh it's been steady it has the full proving pipeline deploying programs generating proofs verifying them it's real proofs for free currently it's even at this moment it's running it's free of charge for anyone um it's permissioned because we have a set of node operators who were running or who are running those nodes but uh and it doesn't have the economics and the consensus yet however the core element of the entire proving pipeline is already there and it's been it's been running stable we are launching fire starter in a couple of

days time it's a scalable production grade Network it's permissioned because we run all the validators but we are already onboarding prover nodes into the network so the proving node approver node side is already distributed decentralized but we run all the validators so it's still permissioned we are going to have an incentivized permissionless test net in Q4 and probably late Q4 early uh next next year q1 we are going to have the fully permissionless decentralized ZK Cloud running um yeah actually I mentioned the stats maybe something worth mentioning people have

been um uh deploying about 390 prover programs on the network some of these may be the same program but as they were working on optimizing things redeploying them and starting benchmarking on that as well take about 150 those some of them are some of them are uh like we have a couple of them predeployment deploy different programs and then start

benchmarking because the underlying Hardware is similar and yeah I have super curious no go go for it no I'm happy for the questions for yourself but I would love uh maybe not know whether you you have some you know less some during this uh death because I'm sure that must have I think it's going to be the next slide or something like that very valid question yeah yeah key learnings from the Thomas right here yeah the main thing is uh like two core things production workloads will require execution guarantees and fullback

mechanism so that's not something any Prov Network can can spare and you know like if there is an L2 obviously currently l2s are are the biggest demand uh drivers for for ZK so in that sense a sequencer cannot afford not receiving aov so we do have we do need those guaranties and fullbacks and again significant compute resources are needed for production grade or like for production workloads be it a polygon Z KVM be it scroll STC whichever we talk about so you need efficient workload orchestration there that's something we are also focusing on

and then again we had Nanos VM so the prover programs uh the OS images were run uh in Nanos VMS in our devet we are moving away from uni kernels and Nanos going with containers and Linux VM that's so that's something so we are we are moving from from Nanos to Linux VM and we are moving away from OS images to containers to make sure that developers have a much smoother experience because again that's something that turned out to be uh like super like Anyway super important of the fascinating yeah it is what it is um yes

so let I want to talk a little bit about what's coming with fire starter so fire starter is the endtoend implementation of ZK Cloud so when I'm talking about fire starter take it as like it's the ZK Cloud but we run it in a permissioned fashion currently so we call it uh Fire Starter it's going to have decentralized Cloud infrastructure underneath optimized for CPU and GPU workloads it can scale to thousands of prover nodes if need be um and it's compatible as I mentioned already with any prover program so you can deploy any arbitrary program you you want or you use or you

can use the predeployment ones we have yeah I think I missed starkware linear uh there I think I mentioned the authors please yeah I'm just fascinated with arbitrary binary execution so like I'm just thinking about the scenario where I deploy a program and I want to cause other provs to look flaky so it's sort of program that at some point I'm the only one who could actually execute the proofs and um yeah I I guess like do you think that's fine you think the incentives are kind of fine there or no I I understand I understand your point so when you start so the the thing is okay from a

practical perspective that's a that's a valid very valid question by the way from a practical perspective you could do that but you deploy your program when you start sending proofs to your program it's going to cost you money obviously you may earn with your Prov note but it's going to cost you money but why would any of us in this room would want to like we would not know about your program being deployed or like your hash ID basically except for you probably no one really would start sending requests to that and if you want to spoil that it will cost you money as well I mean obviously also it will cost you from the other side so you run your own prover node which will have incur costs in

itself then you will incur cost by sending a bunch of requests so at the end of the day I'm not sure if it would be worth or like how how realistic or like how impactful such an attack would be um I guess there's two sides of the point if you like well just for what you said if you are putting a binary and it's like okay it's kind of a little malicious you know eventually I can just turn off like make people seem likey then um okay fine they just don't opt in but if they don't opt in then how do I bootstrap a new Prov program you know I'm just a small Network I want my Pro programed in your Cloud how would you have to just convince provs to White

list it I mean all the provs in the global PR set all of our provs will be able to accept provs for any program in the network so you can with a custom prover set you could you have the option to limit whoever like whichever provs are included in the in the in the set or in the prover set and thus included in the random selection but in in general all provs will be doing proving for all workloads in the network right so that's where I I so there's no op in in general yeah so in general you can make a

program seems fine for a while and maybe some people are tricked by using it and then you turn on the fact that you're now the only Pro calls out to the network and like oh it just you can't yeah you can't call okay so so your idea is awesome I love to see someone try that um but the way these things are run they're run in isolated VMS with no network access so you would have a really hard telling that you're on your PR well would be it could it could have a back door have a private key and then after a certain clock time it's malicious yeah I mean it's it's not impossible obviously I mean nothing possible it's it's it would be incredibly difficult to pull that off yeah I'm happy if the answerers just

doesn't seem perfect yeah I think it's really comes down to is the fact you probably wouldn't make a whole lot of money off to doing that and the amount of effort you have to put into doing it at all just although please no but the these things are very important for us as well because we we'll consider it when designing further at fine tuning further the network um but there is no Bing basically into the type of uh provs that one can register to Z Cloud right it's completely open and permission sense yes um a quick comparison from between the devet and Fire

Starter uh yeah the main difference is it's scalable uh we are already onboarding nodes Prov nodes um it's got random selection it's incentivized if you want to run CG nodes on gabot it's an in incentivized network uh we've got fullback mechanisms the consensus in place it's ready for production use cases even to serve like the largest workloads currently out there maybe I don't know proving ethereum blocks would be one of the largest uh largest jobs out there but uh with proper orchestration and like hundreds of nodes in the network um it can be pulled off easily we are going to have a free tier

on Fire Starter as well and different payment options but I think it's becoming quite interesting now so ZK cloud and as a permissioned version of ZK Cloud Fire Starter it's we feel it's really changing the game in terms of and removing one of the biggest bottlenecks uh in ZK adoption kind of and that is cost this is a summary of what we've been talking about I don't think I'd I'd point out any particular um parts of this rather keep on heading towards the actual cost structures so couple of things to summarize Fire Starter we use containers that run on Linux VMS um no

need for developers to learn new tools etc etc it's highly scalable even to thousands of nodes if there is sufficient uh demand and it's very ideal as I mentioned for parallel workloads for example risk zero segmentation and like processing several segment proofs and then the aggregation tree built on top that can be easily uh orchestrated on the network or that will be efficiently orchestrated in the network also SX proof tree where each client side transaction uh lands with a client side proofs and then that is turned into a honk proof on the server side already then the public circuits are executed over that and then the aggregation Tree

starts basically we've been participating in the in the attec contest a month ago or so actually we won the custom integration category we tied in the number of proven blocks with three other teams because all four of us were able to prove every single block but we had the uh lowest average proving time so we won in terms of proving speed in that category as well anyway so that's we've got quite some experience on how cont anybody could yeah anybody could come in and yeah exactly because the sequencer was was pushing the the the blog content basically to any participant who was please yeah I was

curious as aover how do I if you're just running a single binary how do I horizontally scale like if I get like theis zero segment proving like how do I horizontally scale that work or is the network managing that tree the network is managing that tree in a sense that data transfer C it's going to be maybe you you have more like we are going to have something that we call workflows where you can Define complex complex task structures so to say and like in Risk zero there is an Executor who breaks down any proving job into segments but they don't need to wait until the end so the the proofs can already be started for the initial

segments and then but you need the segments for for to complete the aggregation um so that's something that we um will like we bake into the network to do that type of orchestration so that it can be like workloads can be or tasks within workloads can be spread across multiple uh nodes and then and then aggregate it uh for example Fortec we've been using 40 prover agents which means that it was basically 40 entities proving uh like one transaction per entity because the tube circuit to convert the the client side proof to a h

proof that's the heaviest it takes like four minutes based on Aztec benchmarks as of now so you need for minute for every single client transaction uh and that's just the first circuit to be run on the client side proves so yeah we' were running like 40 proving agents uh uh to complete that what do the network requirements pardon what is like the iio like what is do you know that like uh one minimum 10 10 gbit Optimum 10 is highly okay cool yeah makes sense for DC and any decent data Cent have well more

yeah um yeah I wouldn't really highlight anything I already mentioned credits here you need to buy credits to be able to start proving on gavot like on Fire Starter one credit is one USD uh yeah I started talking about that let me let me um go to this slide and then I'll go back to the actual life cycle of approving workload so a little bit about the pricing of Fire Starter it's a pay as you go pricing model so you pay for the resources you use and for the time you use it um as I mentioned one credit is one USD our as a general um like summary our GPU nodes are going to have 2 49ts each 64

cores and 380 gigs of RAM so that's one GPU Noe a CPU node will have uh 96 cores and 768 gigs of RAM that's the minimum requirement for folks to join the network as a CPU or a GPU node the cost per hour on the GPU node if you want to use the full capacity of that node is 84 cents per hour that translates to 600 bucks a month you get 247 full capacity 249 PS 64 cores 380 gigs of RAM for 600 bucks a month if you ever wanted to use it constantly obviously with the random selection and the orchestration it's not

going to land at one single node it's just an expressive number uh that shows like the total cost of like what this hourly cost translates to per month for a CPU node it's 440 bucks a month U but it's actually 61 cents per hour if you want to use the full capacity now when you submit a proof request to gavot to Fire Starter as I mentioned you reference the hash ID of the approver program and the verifier program that you want your inputs to be executed on then you set the parameters for resource allocation how many CPU cores you want to work on your request how much RAM you

want to be allocated how many gpus because the GPU nodes will have two each so you can select one or or you want both to work on it and then you define the running time um because you would know like we would expect users to know the prover program and the execution time the approximate execution time with the resource allocation that they have chosen actually we would expect users also to kind of do benchmarking when they start using the network you know based on their their actual priorities do they want to prioritize for Speed allocate more resources pay a little more it's still going to be cheap but pay a little more or they want to add

lower resources and pay even less um because let's say proving time is not is not a priority so yeah these are the parameters time is consider with like one VI or one how you measure the running time running time is just minutes like how how long should our prover nodes run the prover program with your inputs so if you allocate let's let's take this example here if you allocate if you want 32 cores 190 gigs of RAM and one GPU to be allocated on your uh to your request and you expect that with this resource allocation in 5 minutes your proof should be done then

that's what you would pay like 3.5 cents basically how do I know that it will be enough five minut let's say that I have a laptop and my laptop is not that powerful to no I mean I think you would start sending a couple of test requests to the network like if if this workload costs three cents then in in a dollar or two you would be able in 10 bucks like with with a couple of dollars you would be able to do do a bunch of uh a bunch of test workloads to see okay if I set this amount of uh Hardware to be allocated to my job then what's the optimal uh proving time something like that yeah and as soon as we have some retroactive data as well because we will

have some data you know how long it takes to run a workload with some inputs on with some Hardware um allocation on a certain workload or sorry on a certain program so we are going to um planning to help with our our users with that data like on average how much you should set uh as a as a running time for example those could be in the cards how do I know that sorry do I know that appr is running the PO specification promis me how do you know yeah for example the the this there are like 192 gab available during the run extion the thing is

it's I think it's more tied to the proving time because if you set your proving time in relation to the resources allocated and the prover will not like won't allocate that many resources then in 5 minutes they wouldn't be able to pull off this this proof for you and we would kick them out because they failed to deliver a proof for you and SL that like you know it's it's kind of that's why it's important you would know your your resource allocation and the proving time based on that resource allocation and with that if that's properly set our approvers wouldn't be able to trick you really because if they allocate if they

if if they allocate half of it then 5 minutes won't be enough and they just fail to deliver a proof and they risk to be kicked out to be punished etc etc or even worse get paid for the work I mean yeah I me that's that's the least benar minutes before submitting something that pardon it's like I have to certify that my Benchmark before submit to if there is one shat approver in the network when it will be permissionless that starts behaving okay this proof I will just do nothing or claim that it took like 8 minutes instead of five how can I claim yeah but you should have taken five minutes of eight no but they cannot deliver you a proof in 8 minutes like the the pro like

running the program stops at five if they are not able to deliver a proof then they they will be punished like the protocol will take care of that the fullback mechanism kicks in new Prov selected randomly they will work on the proof and get you the proof done in five minutes and and uh uh with the proper allocation if the SEC it's an interesting question if the second prover also fails to deliver the job yeah they should complain about me that's yeah actually so I need a certification about my then they may they then the prover program may actually be uh malicious so like if the first prover fails the second one succeeds we know it was the provers

failure if all provs we allocate there fail then there may be an issue with the program itself uh but it's a good point I'll take note of that because that may be a valid request from users of the network no let's say because I cheated as a user yeah I mean uh in that case like exactly because of that it's a very sensitive area for us how we punish the nodes because there is a very fine line whether our nodes are not delivering or whether it's you who have deployed a a malicious Pro program on to the network and then we would uh I mean in those cases we may even block certain programs

from from the network but anyway don't want to go into those those details either would the this price this this this price function is static then and not yeah it's USD based like we denominating it in USD but you will pay probably in the token of the of the platform but you pay like how much it translates to yeah what what kilowatt rate did you use to calculate this cu the on the last side that Hardware requirement looked like it was above oh sorry the actual like kilowatt price well I think that that's something I I won't be able to tell you by heart now yeah it's a it's a much

more it's a much more complex thing because okay run that it then I won't yeah I'm trying yeah I know he talked about this and I think what he did is he took the average of the more common areas you would run into not China H yeah but it's not uh it's not super accurate obviously but it's no but I I think what you are pointing to is like this is not going to if you are a node operator this is not going to be your income that's the user fee and we are going to have Network subsidy to make sure that you know there

are two things we want cheap proofs for the user and profitable operation for the node operator and the Gap has to be bridged somewhere there are different elements that can Aid in this like Network subsidy will be one but as soon as we reach economies of scale with proper ex like orchestration and Hardware utilization and even like aggregating workloads and bringing Network utilization to 85 90% And like constant operation there we basically start getting to a level where the economies of scale the aggregation the high Network utilization the proper workload orchestration so that we maximize the use of the of the

individual nodes as well uh the network subsidy may be just one of the elements and not the dominant element for you to become a data center where power is really expensive you probably should be in another line of work yeah se but as I just centralized set of they're going to be operating across the globe in a lot of different regions and have different her costs for power costs and that they don't have any ability to influence the price function then you're just pushing it all into one optimal loan so yeah no that is true which then you have then you actually have GE Geographic centralization which is like a geographic risk of like you know one one volcano and you're gone yeah it doesn't matter oper that point they're all the

same city yeah they like the economics push them into one physical C but actually that's a very good uh point because it just reminds me if any prover node wants to rent a server on AWS and run it as a prover node on G they will never be profitable because on top of this price whatever is added it's not going to be enough to cover those I don't the g6x any yeah we're using the uh the 180 core 786 gig gcp instances to some uh poly testing and they're what like they're

like 1,800 bucks a month yeah or even more or even more yeah um yeah so if if anyone would like to run a pro node these are a little bit more detailed specs of of the GPU and CPU node it's an incentivized Network we are onboarding prover nodes even now um and there is actually um a QR code on the next massively stupid way no no no go for it just imagine that I'm playing cyber pun whatever with now you know GPU right do I get like a notification maybe you should stop into this grouping and then you can resume your game or how does it no I mean the desktop never meet the minimum requirements ah because it's

unless you're running like a you see a threader or something you're not getting a and actually you wouldn't remain in the network too long because as I mentioned like we are going to not just initially but randomly periodically later on as well send test workloads you need to be available and be able to like within a 2 three second uh time you need to explicitly accept the job that has been allocated to you if you don't accept it we treat it okay you are unavailable for that I mean there could be consequences for unavailability that's one thing but

right away without even the fullback mechanism kicking in if you don't explicitly accept it we're going to allocate it to another node um Etc so there are those mechanisms ingrained to make sure that uh like ad hoc Pro a note failure that could any time happen they are just have some constraint at there and they're just not available um so we we take care of the reallocation um right away yeah and um we also have uh some size of our own cluster you can and rent approver note from us as well it's not going to be a huge cluster like a dominant sized cluster of the network obviously because we want to uh build a decentralized

network but if you want to run aover node get the the incentives that comes with it uh and rent a server for for that you can even run from us you can scan the QR and there is a a form uh where you can see a bit more details um o nodes whether you run your own there uh or you you use uh or you rent from us that's going to be uh those are going to be receiving the same proval rewards and maybe as a summary yeah maybe I should have um planned it this way as a quick summary how a life cycle of approving workload looks I'd say um okay so you have your program deployed let's finish off with

this one the user sends a run transaction that lands with a validator it sends the transaction to the mol it's going to be included by the by the leader in the next block then assigned to um uh aover node the prover node delivers the proof at that point already the user can fetch it and use it for settlement and verification uh elsewhere and then um The Proving result is going to be available for verification in the M Pool so the randomly selected subset of Prov nodes will pick up the job vote for the validity of the proof as soon as all the votes are in um the verification

results again land in the men poool land in a block as soon as it's finalized we are Distributing rewards and if you want to fetch the proof only at this point in time then then that's where the proof um would be shared with you yes so um and you get the ch should buy exactly the timing maximum or or or just the the actual running time based on the the timing maximum uh that's that's what it would be so for example for this one 32 cores 190 gigs one GPU and running for 5 minutes that would be 3.5 cents for you even though the the the proof will take

four minutes gets the full or for example CPU workload like for the polygon zkm you would need the entire capacity of this node so that's an example for that basically 96 cores 760 gigs of RAM but only running for 10 minutes then you would pay one 16 of the hourly cost like 10 cents it's like you abstracting from theing job literally you are paying for time the most interesting thing is is that one thing is that the cost the workload fees are are low on the other hand we remove your being responsible for the

idle time and your paying for the idle time when you rent your own sh for example then you you pay a whole lot for the idle time and U yeah that's a that's a meme that our CEO temo posted yesterday so I thought let me put this in here fire starter is coming in a couple of days finally uh it's there we've been pending the scope as we were developing it so it took a bit more time than we initially anticipated but ZK is going to be Dirt Cheap soon uh learn more at gavot and we are happy to connect in case you need uh cheap proofs you want to do benchmarking on the network or you want to run aover

node whatever you down for that to pleas I have a question regarding the so are you passing inputs to the to the actual Pro note pardon so when you have to compute the proof you know usually you can also pass some inputs and this is usually private and in a permissionless scenario how can trust currently we do validity proing so there is want the witness to be public or to be known externally we don't have privacy at the moment with our custom appr sets you are going going to be able to set up uh for

example plugin tees and do your proving in a trusted execution environment so the custom Pro sets will enable you to do that as well but currently on fire starter and the first version of ZK Cloud that's going to produce you validity proofs it's a very good point and thanks for for asking that so it's not a privacy protocol VR building so if I want to generate as a knowledge proof that I know the private key to certain address I need to Reeve the private key oh no no no wait a second I think we are like let's say you want a proof to be generated for ZK sync for for a block on ZK sync my custom yeah that's or your your custom prover so the you don't need

privacy per se but your your private keys are not going to be revealed oh one minute thank you sorry thanks how you can generate the proof you don't generate ZK proof you generate so you generate yeah okay I have one question pleas what about but we can take it offline and come back to that no I'm sorry I I was poting to you sorry go for go for it about very large inputs do I have to send them into your blogs as no it's not going it's not going to be included in the blog per se but it's it has to be available on a on a public address from where our approver nodes selected will download the input yeah so it's not yeah

it's not going to be baked into the BL yeah thank you very much guys thanks for coming [Applause]