ZK10: ZPU: The Zero-Knowledge Processing Unit - Yuval Domb

Transcript

[Music] please welcome you've all from [Applause] inyama hi um thanks Alex so um the topic of my talk is zpu the hardware path to verifiable everything okay my mission uh today is to convince you that is a problem and that the solution is Hardware we will start by reinforcing that proving is complex continue to show that Hardware is necessary for uh moving forwards and conclude with some architectural

exploration of how we at ingonyama Envision that Hardware so uh zkp applications span two distinct groups client side and server side the two are quite different in many aspects client side is mostly concerned with protecting one one's uh privacy and server side is mainly concerned with uh trustless service providing from an implementation perspective client side is very power sensitive while server side is also cost sensitive we focused on the more imminent one server side and the reason that uh we think server side is more imminent is because it is uh essential to the scaling of blockchains

so when we think of zpu we think mostly Serv side so verifiable everything is not a slogan it's truly the Holy Grail of service providing you may assume that providers want us to need to trust them and that may have made sense once upon a time but it's becoming less and less so and the primary reason is that trust comes with liability so as we know zkps are small they are sublinear often constant and this is very efficient for communication for storage verification costs are also typically low but the problem is that proof

generation is extremely complex representation costs are high as the arithmetization is an inflating process the handling of mathematical structure such as pols hashes elliptical curves and others um are extremely expensive and large finite fields are computation Ally unfriendly so the common perception is that we have no problem a survey on Twitter for X asked um and this is I think ZK validator survey they asked two questions what would be the highest growing segment for thisk for for ZK

this year and the answer was ZK for scaling and I I think that since we're talking about blockchain scaling that makes sense but when asked what what does ZK need most to succeed The Forum answered more talent and in fact faster protocols came in last to me it's a little bit surprising so perhaps there is no problem let's take a look at ZK evm the prover used for evm transaction Evolution on ethereum Z2 currently zika rollups are latency limited delaying transaction finality moreover although they reduce transactional fees they don't reduce

them sufficiently for very small or very complex transactions if you disagree just ask yourselves why does e ethereum not scale significantly yet so how does one measure the effic efficiency of a zkp implementation well cost can be measured in power improving time in Silicon area data center area performance can be measured in security and that's often the soundness of a proof we can talk about the zero knowledge aspects or postquantum resistance finally we want high performance at low cost and we need to

normalize that to the size of the problem so if we choose security as the performance metric and power as the cost metric then optimization is actually minimizing the power per security so the obvious performance metric is security but why choose power as the cost metric the graphite plotted is a two-dimensional grid of Time Versus area so actually As you move up the vertical axis you minimize time and as you move uh to the right on the horizontal AIS you minimize area and

as you can see the the idea is that time and area are interchangeable for example if we have a chip that performs 10 calculations in 10 seconds we can use 10 of those chips to perform the same calculations in one second this naive tradeoff thing is described by the dotted Blue Line If the two operating points on the line edges are achievable then any point on the line is also achievable and that can be done simply by something we call time sharing but what we would like to do which is not trivial is actually to improve both time and area concurrently and that's the convex Green Line since power is approximately proportional to the product of time and

area minimizing power power is equivalent to minimizing both time and area concurrently so we can achieve our optimization goal by minimizing a single parameter power so let's let's revisit the rollup example to explain how power comes into play to do this we need to do a slight diversion to blockchain first of all so a blockchain is secure and this is just one parameter of of blockchain security I'm not going to go through all of them uh when so it's secure when singular consensus is maintained so if a if the network diverges into two different blockchain versions security is lost

maintaining this uh singular consens requires that the slowest participant receives the previous block before the next block is published and this puts a limitation on block size and block rate and subsequently it makes the blockchain space scarce and it raises the price per transaction and note that this is all related to the economics of blockchain the running costs of blockchain or the validator costs also affect the price but they actually put a lower bound on the cost per transaction now rollups are used to alleviate the economic cost rollups can be viewed as a way to bch many L2 transactions into a single L1

transaction to enjoy the L1 security level for a batch of L2 transactions we packaged the L2 batch with a zkp with a zero knowledge proof that is submitted to L1 the second column on the table right shows a ZK rollup that submits a single proof per batch while the third one shows aggregation of proofs as you can see the security cost per transaction is amortised over the total number of L2 transactions that were compressed into a single L1 transaction the trouble is that proving costs incurred by the ZK rollup do not amortise over the transactions but they

rather accumulate so this means that although blockchain economics are great L improved by ZK rollups the real transactional cost Rises moreover this cost varies great varies greatly with the complexity of the evm transaction and is directly related to the power required to generate its proof zkp relies on many expensive mathematical Primitives such as MSM NTI hashes and more measuring the complexity of these Primitives according to various metrics indicates why so much power is being utilized so in this table I marked in red the boxes where complexity cost

is high and the takeaway is that more than half of the box boxes are red I'm going to be a little bit critical of us in this slide but don't take anything personally it's it's a criticism about um where I think our current efforts are being spent versus where they should be And I stress that this is my opinion and it's based on my limited perspective so the first problem is that we spend too much time and effort on abstraction while abstraction if done well is very desirable for the developer the developer is not the client other than that abstraction often

hides things if implemented badly abstraction hides complexity in a way it encour it encourages non-transparency and over time layers upon layers of abstraction create unnecessary complexity that is very high to remove efficient software engineering is essentially a bottomup process and unfortunately that's the opposite of abstraction so while abstraction does have its virtues it should never dominate over performance in places where efficiency is crucial the second bullet is I think perhaps a misconception that security on its own cells okay so while we all desire secure

Services I would say that we would all be willing to Forfeit security for discounted prices just ask yourselves if you would be willing to pay more commission to approve the security of a credit card transaction that you're doing now there are Niche applications where security is a must but the market that we're aiming at is much larger so what are ways to minimize power power per security so I claim that there are two and only two ways to achieve this one is to improve the hardware and two is to improve the algorithm improving the hardware can lower the power by using more efficient physical

processes algorithms can lower the power by requiring less operations to achieve similar end results so algorithms that uh improve power usually develop over time the evolution of such algorithms is aided by something we tend to call outof thebox thinking but really all human thinking is limited to our uh Collective experiences so when we think of out of the box thinking we just slightly stretch our generalization Beyond linear and this is why in order to advance we need to create new experiences so the evolution of new algorithms depends on new hardware that enables new trial and error experiences and this process is C C available

Hardware induces new algorithms which in turn push to more advances in newer hardware and this is the primary reason why Hardware is a necessary part of the algorithmic development process and let's examine two um historical examples of these the first is neural networks neural networks was developed by molik and pet in 1943 and advances were slow a neural network were actually considered an intractable able method in machine learning the neuron Network winter that arguably lasted until 2007 ended when gpus finally arrived at the scene and this kicked off um the current ERA of

exponential growth in dnns the visualization that I uh gave in the in the slide is actually a graph depicting the number of academic papers published in the neural network area as a function of time and notice how the trend changes from linear to exponential right right around 2007 second example is of ldpc codes that dominate LTE standards they actually dominate a lot of high data rate standards nowadays so ldpc stands for low density parity check and it's a family of iteratively decodable near capacity achieving error correcting codes they were developed by Robert Gallagher in his 1963 PhD dissertation and he actually says in his work that

the codes are an impractical construction and they're only useful for information theoretical analysis now the incredible growth in Wireless connectivity experience from the late 80s triggered the need for better codes and especially when high data rate communication entered the scene in uh in the early '90s in 1993 a new family of codes called turbo codes was introduced as a aristic construction with iterative decoding that achieved rates close to capacity and AIC technology was Advanced enough to implement these codes they immediately entered Wireless standards providing significantly better spectral utilization that was possible up until that point and then it took another

three years until the Impractical ldpc codes were rediscovered and they were now sufficiently practical and those codes are better than the TBO codes for many reasons um and none of this would have happened without the hardware being performant enough to make them practical so there are three main types of off-the-shelf Hardware platforms being used for zkp today sorry uh CPU doesn't parallelize well but it's very easy to program code can quite easily be exported between CPU platforms and 64bit arithmetics is supported

natively GPU conversely is highed to program but parallelizes very well GPU code can also somewhat be exported across platforms and it supports only 32 be arithmetics fpga is probably the most generic of the three but is also the most difficult to program code can be exported but much of the lengthy compilation process must be repeated fpgas typically support fix Point arithmetics much better than floating Point although some uh differences exist all of these Hardware types are high in power to security for ZK P applications as part of our mission in leading the zkp

infrastructure Revolution we are currently invested in ingonyama in existing off-the-shelf Hardware platforms and we've actually got two front ends icle in blaze for GPU and fpga so how should we go about new hardware for zkp let's start with what we consider not to do so this is a slide of of certain things that we've seen or heard um so first of all we think that greed or Farms of gpus and fpgas are not very constructive they enable High proving throughputs but they do not improve cost since their power remains unchanged um we've also read about

overly flexible zkp A6 but in our opinion they're inherently no different to gpus and we've read about very applications specific specific zkp A6 but we think that they will not withstand the test of time as they're too inflexible to um and and they actually prevent exploration so when building CPU the zkp seek we would like to exploit things that we're in agreement at the same time as leaving room for exploring things that have not yet been settled when examining Hardware

platforms on a performance versus flexibility grid we argue that CPUs gpus and fpga are all very similar and they're actually clustered on the bottom right um they're all flexible and their general purpose Hardwares but they utilize high power at the top left we placed very application specific Solutions suggested in various papers and this cluster of solutions is very performant but very flexible we envision zpu to be of slightly higher power than the very customized solution but much more flexible than them and I will show you how this is possible

so first let me show you how we think of the software Hardware ecosystem at ingonyama let's begin with the top application that typically runs on an OS the application consists of tasks that are candidates for acceleration tasks for acceleration get routed to corresponding Hardware accelerators via interfaces we've made a distinction between API and TPI TPI is Task programming interface and it interfaces directly with a specific hardware for example IAL and GPU we think of the API layer as a unified interface between the OS and multiple tpis if we examine in Gama's ecosystem

we see that the unified API accesses distinct Hardware platforms we listed CPU as one of the platforms primarily in order to note that a CPU can also act as a co-processor and anyone who's been following the advances in CPUs knows that GPU and fpga are currently are main goto Hardware platforms for acceleration and hence icle in place and in the future we envision another TPI that will act as the interface to zpu allowing seamless integration with existing applications running over the UniFi API so here's a paral list of what we think is important for zpu architecture the least is prioritized

based on our experiences over the last two years so we think first and foremost we believe that zpu must support large large integer arithmetics natively in existing Hardware this is probably the most power consuming Atomic mechanism for example think of how many computations and data movements need to happen in both GPU and fpga to achieve a single 256bit modular multiplier why worry about carry logic and katuba and why settle for Montgomery and Transformations secondly since zkp is based on pols polinomial manipulation is Elementary hence the need for ntity supporting a basic 2x two kii butterfly natively is

invaluable since arithmetics are based on large integer Fields wide buses large memories and high iio bandwidth are favorable since individual a cores are larger now that they support wide integer arithmetics natively like we said a fewer of them suffice and then we can actually use Mimi Vector processing it becomes realizable rather than simdi which is what we've got now large constant memories are in Advantage for supporting large setups and finally we do believe that a lot of stuff that works for GPU should be inherited by zpu the top level architecture and this

is just what we think now right um so multicore s so comprising of multiprocessors before we dive into the multiprocessor architecture know that we are strong Believers in the S so concept whether zpu is used as the main chap in the C system or as a coprocessor we think a non-ch CPU is desirable allowing management tasks to remain internal Global memory is a dedicated CPU memory with DDR or hbm with possible external mapping via cxl the global memory management unit potentially consisting of L2 caching mechanisms and Global interconnect allows fast low power switching between

MPS go down and level to the MP the MP encapsulates a group of processing elements each MP has dedicated interconnect an mmu that potentially consists of an L1 caching mechanism the interconnect is a low power full WID multiplexer allowing any deterministic data remapping between P the MP is a Mimi processor allowing PS to perform different instructions simultaneously this is a significant differentiator from existing Hardware an MP is capable performing self-contained subprograms solely on its own the lowest level hierarchy and perhaps the most interesting is the

PE each PE is a 2X two kii accumulator we call it CTA its biggest single clock instruction performs a 2x2 finite field K coyi butterfly including accumulation of the city CT outputs to accumulation registers all other instructions are subsets of the CTA note that since the p has three inputs and two outputs it is capable of Performing two operations concurrently for example it can accumulate the left input in the left accumulator while performing a Mech operation over the right two the P has short interconnect routes for relaying its outputs directly to its inputs for very low latency feedback the

P can execute small self-contained subprograms limited to its memory size note that the PE is completely deterministic and does not utilize caching finally the PE supports single clock operations at a native width of 384 bits but this can be partitioned in a SD way for example to six 64bit um operations concurrently let me show you a simple analysis and we'll end with this of how zp efficiency can improve upon currently available Hardware take for example nvidia's 3090 TI it's a 628 mm squ chip

the process is 8 nanometer and it runs on a 4 50 watt Supply including active cooling the 3090 has approximately 10 Terra operations per second considering an operation to be a 32-bit integer multiplication Now using the 3090 to perform a single 377 bit modular multiplication for bls2 37 377 requires splitting this you know the 377 bits to 12 digits and using Montgomery's transformation this requires approximately 5 * 12 squared operations per each single

modular multiplication now both analytical analysis and measurements on icle and complete competing platforms resulted in one gig Ms of of these multiplications per second for approximately 30 watts of power now what we did is we used something called open road which is an open source synthesizer in place and Route tool with a 7 nmet package called ASAP 7 and we were able to construct the same 377 bit fully pipelined multiplier running at 500 Mega samples per second the multiplier is a bar multiplier it's based on katuba and it's based on our current uh xylene sulur scale plus

design without any special cust customization I me this took really like a few hours most of them just the the Synn synthesis um now without any optimization or tuning we were able to build the same multiplier achieving one giga multiplications for only two Watts that's a times 15 and additionally the area the area required for the multiplier was a mere 0.23 mm squ we estimate that customization and tuning can result in more than 100x Improvement in power to the current GPU now if we take the power to four watts and the area to 1 mm squar just to

account for extra circuitry which exists we can get 110 G multiplications per second on the same 450 Watts power for less than 20% the area of the 3090 now the power disspation would of course be very challenging potentially impossible and we probably not opt for such a a large chip but we would go for something much smaller but this is just to give you some idea of how much can be approved just by customizing the basic multiplier building block Okay so I'll just summarize and then I'll take some questions um I guess I try to reinforce that proving is computationally hard um and it's

economically preventing zkp widespread use I showed that complexity can be improved by uh hardware and algorithms and finally I presented some of our thoughts on the zpu hardware and now it potentially is at least 100 times more efficient than existing Solutions thank you awesome thank you very much You' all um we're going to open it up to questions and I'll bring the mic to whoever has a question does anyone have a oh you have a mic over there okay uh does anyone in the audience have a question we've got about five minutes before we've got to go to the next

one hey uh thanks for the talk um for sort of the speed of advances in algorithms and and schemes how do you guys decide think going and on what properties of half experation to sort of focus on in particular again how do we decide what what properties to focus on cuz like each algorithm or scheme could have specific things that could speed up the the proving of right so I guess it's a little bit of uh like Risk mitigation right you want to get as customized as you can but you want to keep it open so that it's usable for as many things as possible um so we're basing it on our experience and

we are still not sure that the timing is right right so this is what we think now but I think that when we actually start building zpu things will potentially change um this is the picture that that we're experiencing right now uh any more question questions from the audience okay cool I actually have one question um so I'm going to I'm going to uh take advantage of my of my mic here um so the a couple of concrete examples you gave were around multiscalar multiplication and I think you mentioned NTT too um how do you all consider you

know the other steps in generating an endtoend proof such as circuit synthesis and witness Generation Um and how does the architect you described uh support you know a variety of different types of potential circuits that you'd want to generate and a I guess by the virtue of it being a zpu it could support multiple kind of back and improving systems yeah so um I I guess we consider uh circuit synthesis um perhaps as as an offline process um so we don't uh actually at this time have any special hardware for it but I guess for certain proven systems maybe that is not true um and and maybe it's

something to uh look into amazing thank you very much this is a great presentation let's give you all a round of applause thank [Applause] you