Episode 267: ZK Hardware Sessions with Zprize Pt. 2

Transcript

[Music] welcome to zero knowledge I'm your host Anna Rose in this podcast we will be exploring the latest in zero knowledge research and the decentralized web as well as new paradigms that promise to change the way we interact and transact online [Music] this week we continue on the topic of ZK Hardware through our interviews with participants in the recent zprice Hardware acceleration competition my co-host for these interviews was Alex Pruden CEO of elio and the creator of the zprize initiative we aired part one of our two-parter last week and I do

recommend listening to that one for context before you jump into this one in that episode we heard from Luke from polychain one of the Architects for the prizes as well as the hardcamel team who competed and won in a few of the Z prize categories in this episode we hear from the team at xilinx which is part of AMD Our Guest Swati and Hamid offer us insights from the perspective of an fpga manufacturer and for our final interview we chat with Nile Emmert also a competitor in the zprize competition Nile competed in one in the MSM for gpus and MSM for wasm competitions it's been really interesting to have a look at the hardware level of the snark stack and get to meet those folks who specialize in this through these interviews I've learned all about areas of optimization

specifically the underlying math and Engineering techniques our guests presented as well as the ways in which we might see Hardware acceleration complement all the work that's happening on the protocol and software side something we cover quite heavily on the show I also just think zprize is a really cool Initiative for the community and I hope we get to see more things like this in the future now before we kick off the episode I do want to direct you to the ZK jobs board there you'll find jobs from Top teams working in ZK if you're looking for your next opportunity be sure to check it out alternatively if you're a team looking for great talent be sure to post your job on the ZK jobs board today I've added the link in the show notes now Tanya will share a little bit about this week's sponsor

today's episode is sponsored by ingonyama ingonyama's mission is to improve the performance of zero knowledge provers by designing Hardware optimized for ZK computation and now they're proud to announce their latest project icicle icicle is an open source Cuda library that enables gpus to accelerate zero knowledge Primitives like MSM NTT and ecntt using icicle in konyama built a GPU version of the Dank sharding builder for the ethereum foundation which is now also open source the ingonyama team is excited to share their toolbox with developers and committed to maintaining and improving icicle for as long as it's useful to the community as always the emphasis remains on Superior Dev experience and ease of use for ideas and discussion around the code visit the

icicle Channel and the ingonyama Discord server where team members and fellow developers await we've added the link in the show notes so thanks again ingonyama and now here's our episode foreign who lead product management for fintech and blockchain in the Adaptive embedded Computing Group which is part of AMD welcome to the show thank you Anna thanks Anna I would love to hear a little bit about the two of you Hamid maybe we can start with you what got you interested in this field in general sure it was about maybe three or four years ago about 2018 2000 early 2019 which is pretty late if you think about this

space because the whole blockchain Revolution started for almost 10 years now but for the most part I kind of didn't pay much attention to it until somewhat by accident I started reading some of the white papers at my previous job and I was totally mesmerized it was you know the mass was beautiful but also the concept really was still revolutionary I realized that for generations for as long as human existed for every transaction we always needed you know a third party whether we get married or we buy a house or we go to a bank and then we don't need that we can eliminate trust so language story short I was just talking to everybody about it and then one of the friends alerted me to an opportunity at zoiling's not now

acquired by AMD about this new practice they are creating to look at the production space more closely and I jumped on it and joined silence and ever since we have been involved with different kinds of blockchain projects and I'm pretty excited about it nice had you been working in Hardware before like what were you actually you know before you got interested in the blockchain part of things what were you doing yeah no absolutely my own background is Hardware I did my PhD at University of Waterloo and then I worked as a hardware developer engineer for a number of years and then on my previous job I was with Marvel semiconductor and I was in charge of Designing products for in vehicle networking autonomous cars which was a pretty interesting area as well so this

was probably the only thing that could have separated me from that job but yeah I had been in the hardware space overall I would say for 10 15 years cool Swati I'd like to hear a little bit about you as well what first got you interested in this field yeah so my background is in Engineering also I was an engineering manager at scilinks when I enrolled myself in a part-time MBA program at UC Berkeley and it was the the Student Activities there and the courses that were offered there that got me intrigued in the space at first and fortunately for me AMD xilinx became interested in that space around the same time and a group was being formed which was going to explore the space so it became clear to me that this group was going to be working on some of the most

interesting and impactful problems at the company in my opinion at least so Express an interest in joining and that's how I became part of the group cool so you had already been working there in Hardware that's right what was it about Hardware that got you excited in the first place like why Hardware I think it was just the most fundamental piece of innovation that you can have for an application and I think that's why fpga has attracted me as well is because you you can really optimize things at the absolute bit level you know so at the lowest level so I think that's what got me interested in the space cool can you guys talk a little bit about xilinx and how the company came to be uh I know you mentioned that you know it's now part of AMD but you

know maybe how that process worked and yeah can you just give us a little bit of background about the company sure um so design links has been founded for for a few decades now um and until it was acquired by by MD in I think it was February of last year officially so the focus of the company has been on fpgas basically it's field programmability array devices and um the main characteristics of fpgas are you can design any circuit you want at the hardware level at the gate level and implement it and then if you decide to change your Hardware you can go in and reprogram your Hardware which is you know very different compared to how you program a CPU or you know a GPU and also

it's very different how you do it on Asic I would say it's probably closer to an Asic with the big difference being that when you're designing an Asic you are basically stamping the hardware with a certain characteristics and in the fpga you can keep doing that over and over again almost like a creating a sketch and then changing it and in that sense it's pretty powerful because you can have really detailed optimization at the bit level at the gate level and also your memory hierarchy how you talk to memory how these different pieces talk to each other really optimizing and then 10 minutes later you can go and change it and then have a totally new hardware now if you compare that with CPU and gpus so these are fixed hardware

architectures and here you have all the flexibility you want obviously all the benefits come with cost as well so for one thing it's relatively harder to program naturally because you have to understand the hardware description languages like very very large and vhdl and also in terms of silicon itself if you compare with an Asic you're going to be needing much more silicon area because at all those flexibility come at the cost so for a lot of you know I guess a startups specifically that are trying to come up with an idea and they think it's going to be an excellent idea to create a chip with people typically a stone from an fpga and design their circuit to make sure it does what it

says it does and then after a certain time you can go and kind of create your own piece of silicon basically So when you say sort of like you fix the I guess it's the algorithm like you've made your decision is that the point where you would then turn it into an Asic is that the sort of creating the fixed version so think about creating the circuit if you want to create a circuit you have a state machine you have a massive truth table and and then you are going to implement that in terms of gates you are going to have a bunch of ands and ORS and and that type of thing and that creates a circuit that's an Asic you go in you go to a Foundry you say these are the gates I want these are how they are connected they are that's how they are

connected to the memory and then uh you go and you have a mask and you go create a circuit and that comes back and it has a fixed architecture so after the circuit comes back you cannot change the configurations it's exactly like you have a stamp something you cannot change it now with fpga all of that is configurable so each gate you can decide what the function of that gate is so if you think about it we have a lot of tiles and each tile is a flexible gate so you decide what the content of that through stable is and that's how you define your search at your estate machine and then if you realize that you made a mistake or the protocol is changed or you want to make a change you can go in and change the definition of each of those Gates and now you have a

whole new circuit and there are a lot of different pieces of resources in an fpga and shortly maybe you can give a little bit of my kind of more in-depth in terms of what are the resources in an fpga yeah absolutely and I was thinking about the question that Anna asked about you know application specific integrated circuits in general once standardization occurs in a new area of research or a new technology we typically see Asics emerge so A6 optimized for a very specific application they can run that at higher speeds at lower power so once we see standardization in areas like AI for instance if we saw fpga is used for a while initially and then Asics emerged and we see that typically with new

technology fpgas tend to be used when technology is changing rapidly and Protocols are changing and it doesn't make sense yet to invest a lot of money and time into building an application specific integrated circuit that's optimized for a very specific protocol um and as Hamid mentioned fpgs consist of reconfigurable blocks but they also have some fixed blocks such as dsps or Digital Signal processors which are optimized for specific functions such as multiplication for instance which may be very expensive to do with the configurable logic in the fpga other blocks such as protocols like pcie which are common to use are also hardened within the current generations of fpgas

that we have today it was AMD not doing that before why did it want to to buy the company yeah so so basically if you think about the fpga market it's a it's a very specialized Market the two companies that dominated the fpga market was one was Altera that was acquired by Intel and the other one was the islands that was acquired by AMD so now AMD and Intel are basically dominating I would say 85 to 90 percent of the fpga market the nature of this Hardware I would say is fairly complex providing all this flexibility comes with a lot of complexity as well so it's not a trivial task to create an fpga and very few companies in the world have the

expertise can you talk a little bit about you know so we're talking about fpgas and we've emphasized flexibility a lot but you know many people may be kind of ignorant about what that means you know because they're thinking to themselves well I'm listening to this podcast and my you know my laptop has a CPU which is of course also very flexible right so can you just give us a sense for and and this could this could be a blockchain you know for blockchain applications or just for General applications like what kind of performance or cost benefits does do you see when you introduce an fpga over just using a run-of-the-mill CPU so if you think about the main question that people commonly ask you know CPUs versus gpus versus fpgas and everybody has a CPU and GPU at home and I would

dare to say 99 don't have an fpga at home and hopefully we can change that one day but um it comes very natural to them and the question is what's different if you think about CPU that's fairly easy CPUs they perform sequentially so for a lot of tasks that you have to finish one task and then go to the next and for a lot of tasks that really need massive parallelization then that's not going to be a good choice right so GPU so people understand okay I have a CPU you know I need to use GPU if I want to have a parallel you know problem that requires a lot of parallel computation and we are going to use the GPU so what's the difference between GPU and fpga so if you look at gpus yes you get massive

parallel compute power which is great and frankly solves a lot of problems but the architecture itself is again fixed so each of the cores are fixed the memory hierarchy is fixed so you have to work within that boundaries and that poses a certain limitations also another thing to keep in mind especially when it comes to Hardware acceleration when you're thinking about the system and you want to accelerate it um sometimes you see that with the GPU for example you can accelerate a specific function but then how you move the data and you create the whole system it creates other bottlenecks and as a result the overall system doesn't have as much performance as much benefit and

we have seen this um time and time again where people you know say oh I found a solution this one function I can do it 100 times faster on a GPU and they do it but then they create the whole system and the whole system is not that much faster and kind of defeats the purpose now with fpgas you have such a control in terms of telling the hardware to each gate what to do and how these gates are connected that you can create a whole system on an fpga and optimize the whole system also because you don't have fixed architecture you can decide what's your bit weights how you talk to memory how you move it it gives you really a huge amount of power to customize but

obviously with that power comes responsibility as well because you see that we have seen teams again implement the same design on the same fpga and one solution is a hundred times faster because you have all the flexibility in the world you can kind of shoot yourself in the foot but you can also be very smart in in terms of how you design something so I would say that's an area that is right for Innovation you know specifically when it comes to zero knowledge we are engaged with a huge number of startups that are trying to use Hardware acceleration and then we see sometimes some startups try to do something but not the right way and then how what the choice of fpga itself also is important because as short is that

each fpga has a slightly different combination of resources and how you bring all of that to Bear makes a big impact in terms of what's your final outcome yeah that's I think that's a really important point and that was a great explanation about fpgas gpus and CPUs and one thing I on that last point that you just mentioned I wanted to call out was you know it could also matter whether or not you're accessing the GPU whether you're hosting the fpga yourself or whether you're accessing it as part of a cloud instance so many people probably know that Amazon or maybe some people may know that Amazon hosts fpga so you can you can get access to it via their Cloud via AWS but you guys for the purposes of zprize uh actually gave the competitors the physical Hardware the

c1100 cards I believe that that xilinx manufacturers and sells uh for the individual competitors to use so removed at least for the purposes of zprize you know that extra variable of needing to consider hey what limitations are being introduced by a cloud provider so it all goes back to the point that you were making is you know you can zoom in on one single part of the problem and consider it from a very low level but if you don't consider the whole end-to-end system well then it may just become irrelevant so um yeah I think that was that was a great point I know you've already mentioned this quickly but I was hoping you could actually tell me what do each of these acronyms actually stand for when you

mention fpga's Asics CPU like all of them I think it would be helpful for audience absolutely fpga stands for field programmable gate array so as the name suggests it's a silicon device that's programmable by a user in the field to the desired functionality CPUs stands for central processing unit and I don't think that needs an explanation gpus stands for graphics processing unit and a6r application specific integrated circuits so it's a silicon device that does one thing very well so it's optimized to a very specific application do you think you could actually put these on a spectrum from one side to another like is that how you often think of it to some extent if you think about

this I would say gpus and fpgas are for the most part it can be on one side CPUs on the other side but again you know if you think about fixed architectures both CPUs and gpus have fixed architecture fpgs don't have a fixed architecture but for GPU and fpga both of them are good at parallel processing so I guess there are some similarities and some differences got it one of the ways that I think about this is also access and ease of use so most people have access to CPUs and they have access to sdks and compilers and ways to program on a CPU that they're familiar with so that's probably the easiest way to start once they hit bottlenecks with the CPU

typically the next choice is gpus and that's because there's easier access to gpus as well since they've been used in rendering and Graphics processing for a while now the companies that are involved in providing gpus have done a very good job of making the sdks available and and open and develop the community around that so again the challenge of developing an application for a GPU tends to be a little bit easier then moving on to fpgas once you've hit a bottleneck with gpus you can optimize further on an fpga fpgs are relatively easy to access now and amd's eye links have been doing quite a bit at least that's one of the goals of our team is to make fpgas accessible and we've launched you know certain products

that are available on our website and you know can be purchased for instance so access is is relatively easier now in terms of the tool chain it used to be more difficult to program fpgas you had to be well versed in in a hardware description language in HDL that's no longer the case so now you could still write a program in CC plus plus and have that be implemented as Hardware you probably wouldn't see the same level of performance with that and optimizations with that but it's possible to start there and then finally I think at the end of the spectrum our application specific integrated circuit so when you can determine exactly what you'd like to optimize and bring that to Market in a way that's that's competitive and you

know you want to create the best solution that's out there in terms of speed power efficiency cost then A6 might be the way to go cool so we saw this in Bitcoin I mean Bitcoin mining for example is a relatively simple application and you sort of saw this linear development right with like okay we've you know Satoshi is mining onto CPU people mind on gpus there was actually an error where people used fpgas for Bitcoin mining um but that very quickly gave way to Asics now as we see the emergence of zero knowledge cryptography or just you know cryptographic applications in general and the need for for greater degrees of specialization people are again turning to fpgas similar like what we saw in Bitcoin mining now this is obviously much more complex than

something like simple Bitcoin mining but do you think fpgas are just inevitably an intermediate step in the development of an Asic or do you kind of think of them as being having more longevity in terms of zero knowledge applications versus something like Bitcoin mining that's the General life cycle of an fpga as you mentioned about the Bitcoin and we see that in a lot of different applications where people have an idea or there is a protocol and they want to accelerate they start with an fpga and eventually when the Dos settles they transition to an Asic and and we believe that in the case of zero knowledge we are going to have similar pattern um the main question here is what does

intermediate means right especially as we move to a smaller process nodes creating an Asic is becoming more costly um say if you want to create an Asic in the you know three nanometer node or you know even a five nanometer node you are going to be spending tens of millions of dollars to create an Asic and obviously if if you make any mistakes or the choice of protocol changes or any of these hard wiring should be done differently you have to do another Asic so you have to go and spend under tens of millions of dollars now instead of doing that you can you know stay on an fpga until the industry matures and then be 99 sure that if you create an Asic that's going to last for a few years

then you can transition um to an AC what we have seen specifically as it relates to zero knowledge early on we saw some of the startups talking to us wanted to jump into an Asics and some of them still do but they realize that even some of them after looking at the space and how every day there is a new protocol and there is another optimization that jumping to an Asic to optimize a function or you know whatever they perceive as to be the main building blocks of ZK might not end up being the blocks that matter and they might end up with you know tens of hundreds of millions of dollars of investment that you know with a piece of Hardware that does nothing basically for them so yeah so the intermediate is it going to be five years or three years we

think it's going to be fairly long before people can actually come with A6 that's going to have longevity and that's the pitfall that we advise a lot of startups that get engaged with us to really make sure they know at one point they want to make that that massive investment they go to their VCS and they give certain promises and we just warn them that just be mindful of all these pitfalls of kind of jumping too quickly into an Asic well and I think listeners of this show would you know can probably pick up on the speed at which development happens in zero knowledge I mean a few years ago you know we we talked about relatively simple proof systems that couldn't weren't very flexible and now you know there's like a menagerie of different

proof systems of Planck and Marlin and Nova and Fry and Starks you know it's just you know and every year there's like more and more of these that come out and to your point you know once you invest in basically hard wiring and Asic it can never be changed right and so you either have to throw that away and start over or you know if you used something like an fpga for example you could you know it gets some optimization benefits without having to invest overly up front yeah even even if you think about if you want to do an Asic to create certain libraries or function you know you think about how people you're using kcg and now people talk about fraud and how these need different underlying functions even so so again you know not to kind of over emphasize the point but

people have to be extremely cautious when they think about an Asic uh because frankly we have seen a startups to raise money and fail because um they were overtly ambitious in terms of what they think they can can be achieved and frankly it might make sense to develop your whole system and get to some critical math the way I I um I talk to a startups I advise them is what's your business model don't focus on margin because the main reason frankly to go from an fpga to an Asic is the chip itself is going to cost less probably you can get similar to same level of power efficiency and by the way that's another advantage of fpgas over gpus because you can really design your architecture you can make it as power

efficient as an Asic almost you know I would say 90 percent so don't worry about the margin focus on proving your business model and then get to some critical mass with fpgas and then if you can prove that and you have customers or you have business models or you're going to do proof as a service or you know go to market go to market first make sure people are paying for what you are offering and then let's talk about raising money and you know making that 100 million dollar investment to do an Asic makes sense I have sort of a last question that brings us back to the zprize topic but yeah what was your take of C price and why did you decide to support it actually so we had a huge um tradition of supporting the startup ecosystem

um I think that's a zoiling strategy and we have been a big supporter of hackster oil um so this is um something we do kind of on a regular basis on areas that we think has great promise and when we formed the group and you know slot it did a lot of research on this we identified zero knowledge as one of the key enablers of blockchain Technology we believe that this could really finally help blockchain to break out of content its shell and you know a scale and really get Mass adopted because a lot of applications really require this underlying technology so that's you know we think it's going to be very disruptive and at the same time it's very compute intense which selfishly is good for us

um so so naturally we said okay we want to support the ecosystem yeah and um we had Romy actually from tzk approached us and said hey we have this competition that um if you guys want to support the community you can participate provide hardware and also we want to get people to get using fpgas more as you said you know everybody has a CPU and GPU at home people usually don't have fpgas so that was a great opportunity to kind of introduce fpgas to the community and I have to say we were pretty surprised in terms of attendance um it was more than what we expected the interest was more than we expected and even some of the traditional guys that we deal with in you know in the regular Financial Services were they also

participated as you guys know and and we think this is something that probably we're going to continue in terms of supporting this ecosystem and we are very excited about it cool I know you did a lot of research in different future areas um of of high potential within blockchain and web 3 and ZK was one of the areas can you just talk a little bit more about some of the applications in web3 that you you all AMD xilinx have looked at where fpgas could be applied so Alex as you mentioned initially fpgas were used to to mine Bitcoin in early early days and which kind of a short-lived and then later on naturally people start using fpgas to mine ethereum anything that we call like memory card that kind of proof of work

and when our team was formed that was our main task because our management saw that we are selling a bunch of fpgas and they were not sure why and we started that and um it turned out that fpgs can be very efficient in terms of mining ethereum especially toward the end of I guess ethereum mining life before it went to proof of a stake a lot of miners had some constraints in terms of how much power they had access to so say you had the mining operations in you know some kind of far-reaching location and they had access to some fixed amount of hydro and then they wanted to increase capacity we realized that with fpgas we could have mined the same thing at you know a 10 maybe like a fraction

of the power maybe 25 percent of the power and then they could increase their capacity about four times so that became an interest and obviously as mining kind of went away ethereum and proof of a stake we started looking at different areas so one was Zero knowledge that we realized that it's going to be very impactful but beyond that another area was distributed as storage so we are engaged with protocol labs and foil coin to see see how we can accelerate that protocol as well interesting enough a big piece of foil coin as well is zero knowledge in a way it's kind of very similar algorithm that they have in the in the back end of their their protocol so that was another area and then another area that we did

some research on was more on the Enterprise side so obviously being a big company there is a lot of pressure to focus on Enterprise and we did focus on Enterprise um we have some work published on hyper Ledger fabric as well but I think looking at all these areas um I think the issue with some of the more Enterprise type blockchains is it's kind of loses some of its blockchain characteristics that make blockchain so great so we take some of those away and it becomes an Enterprise so we realized that at the end of the day even for Enterprise applications probably makes sense if the security comes from layer one which is a more public blockchain and then you can build Enterprise applications on top of that which incidentally to be able to do that you

need certain level of privacy and zero knowledge again comes into play so that's why our team is is very bullish on zero knowledge as an enabler of a lot of these Technologies even when it comes to an Enterprise application hmm I want to say thank you for coming on the show and sharing with us you know all about fpgas giving us real insight into how those work where they stand and how they can be used in ZK stuff thank you Anna I appreciate for having us we had a great conversation thanks Anna for having us yeah I really enjoy the conversation thank you for being here and thank you for your support of not only the Z prize but the zero knowledge ecosystem [Music] now we're here with Nile Emmert president of yurd software welcome Niall

hello thank you now what was your path to Hardware what got you started in this field I guess it goes back to my PhD which I started in about 2007. I had been interested in big number arithmetic and um I had been running a software company for a number of years and decided to go back to grad school to do a PhD and so this is one of the things that I had found interesting and I ran into an old professor of mine and he said yeah why don't you come and do a PhD with me so we started applying for grants for the National Science Foundation the NSF and we got awarded two grants actually and um that was the beginning of my work we

did a big number arithmetic on gpus which is the same sort of computations that are used in cool so that's maybe a bit different from the other interviews that we've done so far which is like you're coming right from the GPU World um have you ever touched fpgas or have you only ever worked with GPS I have only been on the software side doing gpus and CPUs so I have no hard experience with fpgas I mean I have a bit of background um from things I've read and um things like that but uh no actual experience a question I have there is like what is it to work on gpus then like in the GPU world like gpus are

fixed you're not changing the hardware directly or anything like that if you're directly interfacing the GPU like what are you doing like if you're trying to optimize something how do you actually optimize it yeah that's a um that's an interesting question so it is a completely different world from fpgas um gpus are more of a software environment so you can think of it as kind of like having lots and lots of CPU cores um that you can do something incredibly parallel on okay so there are only certain kinds of software that runs well in a GPU and fortunately this is this is one of the types of things the GPU tends to be like a really high throughput

engine so it's not very good at latency it's it's much better at throughput your PhD work was in the you know in the area of big number arithmetic and can you kind of talk about uh how gpus and specifically this height or this you know High throughput platform of a GPU specifically applies to or how you applied it in the research for these algorithms to do big number arithmetic more you know faster or more performantly so I I think the main reason that gpus are so good at this is that the gpus are originally of course used for for graphics and one of the main operations that Graphics has to do is a lot of

these floating Point operations for 3D manipulations of scenes but it turns out that the same Hardware that is being used for floating Point operations can be used for integer arithmetic so the same actual Hardware inside the GPU is doing doing both of this stuff so what that means is that gpus are really fantastic at integer arithmetic as well now um Nvidia has been working for many generations of gpus and I think in the early days they thought that the only thing that was important was the floating Point math but as the generations have gone on they went through like Kepler then Maxwell Pascal then to Volta then to Ampere as they progress through these Generations they

realize that the integer arithmetic was really important for something somewhat surprising for for addressing math for calculating the addresses for for data to load but it turns out that we can sit on top of that same arithmetic that they've that they've made extremely fast and use it for big numb computations and that fits in perfectly with the with the Z Prime work so backing up a little bit can we actually talk about your research area big number arithmetic so what I think to you know to the average person they might just think of arithmetic as like one plus one big number arithmetic applies it's with big numbers but maybe can you just give us a sense for like what does that mean what what is it to have a PhD focused on big number arithmetic so normally computers work on

on numbers that are that are in the range of 32 bits or 64 bits like if you have an Intel processor that's really the size that it it manipulates in a single instruction but for the this elliptic curve cryptography and most of these crypto algorithms they're dealing with numbers which are which are much larger than that they're actually it depends on what curve you're using but for many curves it's uh it's like 384 bits so that's actually represented as a sequence of these 32-bit numbers so when we talk about big number arithmetic we're really talking about operations that work sort of like doing long multiplication that you used in school or long division or long Edition where you have multiple digits but each of

these digits is in fact a 32-bit quantity and that's the way that these things are represented and processed on the on the GPU oh cool I see so now I understand what you said about the gpus and their ability to parallelize computation is effectively the equivalent of like if you're doing long division you can do every single digit of the number that you're dividing it once effectively right if you're you can run those each process in parallel is that how to think about how big number arithmetic you know algorithms are applied in gpus yeah that's a that's a great question unfortunately gpus have have a lot more cores in that so like the devices that we were working on in the zprize have literally thousands of cores trying to think on top of my head

exactly how many it is but something like 4 000 cores so um it turns out that there are too many cores to really do this sort of fine-grained processing and what we do in our our implementation for the Z prize is we actually sign a problem instance to each core on the GPU and that gives you better uh better performance you don't have any inter-thread communication that slows things down so this work in your PhD this led you to actually work in Hardware tell us a little bit about you know what what were you doing what kind of projects have you been working on so I was fortunate when I was doing my PhD work I managed to get two internships with Nvidia so when I finished my PhD I actually joined Nvidia as a full-time

employee so I did that for like four and a half years until the Z prize came along and um I took a leave of absence okay this this I'm really curious about though like you having worked on this is sort of going back to my earlier question weren't you working at Nvidia like you're not designing Hardware you said you were working on the software side so like what what job do you have there or had there so I I actually was building the big number libraries for NVIDIA so during my first year I think there we we wrote some libraries um something called cgbn which is out on on GitHub and then I kind of got drafted

into other projects that did India working on uh something called tensor RT which is the Deep learning Frameworks for inference interesting and like these are libraries that the company's creating for people who have this Hardware to actually use it right is this an open source project that the company's just doing so that people can actually engage with it you're not selling it it's not like software it's an open source thing and it doesn't take a huge amount of imagination to uh to guess who might be interested in doing cryptographic stuff really quickly and and I guess like it's common like hackers you know people who are trying to crack passwords like gpus are you can use gpus because they're paralyzable

they can pack they can go you know basically try a million passwords at once that's oversimplification but that's like that's the reason why there's a very specific way when you're storing passwords on a database is you have to do it because that that's the attack that like a hacker would use if they got your database of all the encrypted passwords if you don't do an additional step they can just like Brute Force effectively you know a single password I don't know maybe you know more about that but that's that's like roughly the idea no that's exactly right so this is awesome background and now I think you know maybe we can turn to the Z prize which you've referenced a couple of times you know and I guess I should say at this point who you are in in kind of context of the Z prize so you were a

competitor in uh two categories and you did quite well in other words you won uh both categories in which you competed um given all everything you've just said about gpus it's probably not surprising that that was one of the categories you want so you won the category of it for accelerating multi-scalar multiplication on the GPU as well as accelerating I think we called it accelerating elliptic curve operations in wasm which stands for webassembly so the idea with wasm is effectively like it's a run time in your browser and in the competition you you know we had you you know basically performing this multi-scaler multiplication to replicate effectively as if you were running a consumer laptop so um maybe just before we go any further

can you define for everyone what is a multi-scaler multiplication um so a multi-scalar multiplication is where you have a set of n points from elliptic curves and you have a set of n scalar values and it's a pretty simple computation what you do is for each point you multiply it by its Associated scalar value and then you sum all of those up now the thing that makes it challenging is that it's um that's a pretty expensive operation to do and and I guess that's partially where the security comes from because this is an expensive thing to compute it's really a good key primitive to put into proof systems in terms of proof systems I

think Alex probably knows far more about that than I do so let me pass it back to him for for maybe some input on that well I guess just quickly to comment you know these like MSM is a widely used in most zero knowledge proof systems with one big exception which is Stark's Starks overwhelmingly rely on something called Fast Fourier transforms but MSM is used you know in most flavors of ZK snarks in the process of of proving right so there's yeah maybe I'll just leave it at that okay so competing in these two categories MSM for gpus and MSM for wasm which I'm now pronouncing it because it's web assembly I've been

told um but yeah like so when you're competing in this was there already a starting point was there some Benchmark that you were up against or was this sort of like an Uncharted Territory where you know everyone's benchmarks are their first or their kind so I think fortunately in the competition there was some starting code for both uh competitions okay on the GPU side uh the starting code was was shockingly good I remember taking a look at it the the first time around and thinking oh wow this is this is way harder to beat than uh that I was thinking we spent a lot of time and we did a lot of optimizations and and over time we we got it to be quite a bit faster than the starting code that was

was given out on the wasm side there was also some starting code um that was based on Arc works but um fortunately there was there was quite a bit of of margin and it wasn't so difficult to meet the to beat the artworks implementation but on the GPU side was it hard yeah the the GPU side was was really hard um so I I guess I was a little bit overconfident when I started the competition um but I also know that uh the some of the other teams had people from Nvidia and when I when that really started to sink in that that oh my gosh this is going to be a lot harder than than I thought um I I did a lot of work and and found

some places to optimize it it's probably important to note here is that you were the winner of of the competition um but but it was a actually I think strictly speaking a tie for First with another team which which was for matter labs and you guys did something I thought really cool where you both took slightly different approaches to getting almost the exact same Benchmark or being you know beating this at The Benchmark by almost the exact same amount and then you combine the two into the Super version the super team yeah Voltron can you talk a little bit about uh that process and and your reaction to their implementation which was different than yours but did almost as well or as well

near the end of the competition uh the matter labs and I had both Frozen our code um so we we felt comfortable sort of talking about the ideas that we used and we realized that we each had some some great ideas that the other team hadn't picked up on and ideally we had wanted to just do a single submission for the prize but for some complicated reasons we decided it's better not to not to do that so we ended up with with two submissions and you know our prediction was that they were going to be pretty close in performance but we didn't know exactly how close and uh they turned out to be basically identical in performance but we realized that we could merge the two of them and

actually get something quite a bit better so after the competition ended we we did that merging process and I think we we got about a 10 performance Improvement um for the merge thing and that sort of represented the best ideas out of both teams that's so nice to hear that's actually kind of one of the questions was about the competition and if you learned anything from seeing what others had submitted I guess this is sort of the hope of the competition right that like everyone's competing yes there's winners but like there's definitely going to be like new techniques uncovered along the way I I learned a lot but I would say that um these implementations are are surprisingly subtle so I I wouldn't be surprised if um if people come after the prize and start

looking at these things and realizing that wow you know there's some stuff going on on in those implementations that are really complicated and and not so easy to understand you know maybe there's possibility in the future that we could do a bunch of of really deep dives into these things and and sort of describe how they really work in in more detail than than just the readme's give because I think there's some really interesting techniques that that come out of that in fact um I've been invited to Stanford to give a talk on the implementations and I think that's going to be sometime in April or May be like a put a link to it out cool this leads to uh to a question so there's some amazing work that you did in collaboration with matter labs in the

end right with the submission the final submission uh for the Z prize now looking forward what would you personally like to see if we run this competition again or something like this happens again like what direction would you like to see it go like what types of uh problems would you like to see it focus on yes so that's a great question um personally I'm I'm interested in in MSM um and sort of the low level algorithm so maybe maybe also NTT I think that there's still more performance to be squeezed out of all of these implementations both on the GPU side and on the fpga I think on the GPU there's some new cards that Nvidia has released that have a lot more level two cash and I think that can be leveraged to

accelerate MSM further than than what's already been done and I think on the fpga side although I've said I have no expertise and that space which I which I really don't my belief is that there's a bunch of performance that can be squeezed out but by keeping running competitions on on the fpga side um I think it's the same thing on the GPU you know the GPU submissions are building on years and years of of development of of big num and big num libraries but I don't think that's the case on the fpga side I think this is really the first time people have tried to do something like MSM in in terms of really high performance on the fpga side so I'd be I'd be really curious to hear what the other interviews say about fpga and how they view the same question and

what about the wasm side how do you kind of look at that so we talked a lot about GPU and fpga but what about wasm and kind of the other part of the competition which of course you also you know you won on you won that prize as well yes so there was a implementation had some restrictions um you weren't allowed to use any of the vector operations the vector instructions and you were only allowed to use a single thread so um if there was to be another wasm competition it might be interesting to remove those restrictions um so because I think that there that would open up a bunch of new techniques um if you could use the vectorized instructions cool well thank you so much now this is amazing thank you for your contributions to the zprize which of

course are now open source contributions to the community um in the show notes we have uh we'll have a link to the GitHub organization associated with the Z prize where folks can check out Niles implementation as well as the implementations of matter labs for the GPU and then you know was them and yeah all the submissions are there and we encourage everyone to check them out and as Niall mentioned you know there might be some further optimizations that can be that can be squeezed out there so it'll be interesting to see what the community does with that so but again Nyle really appreciate your time being with us here today and really appreciate all of your contributions to the Z price actually Alex I I want to give a heartfelt thanks to you um I think you did an amazing job putting this competition together and

I'm really happy that I was able to participate in it I appreciate that and as you know it was a journey and there was there were high times and there were low times and and we ended up in a good place and I'm very thankful for your understanding as we went through that together because as you know better than anybody I think I was I did not know nearly as much as I do now on uh on how to structure competition like this and it wouldn't have been possible without your patience like frankly and your advice so yeah thanks so much for uh coming on sharing with us your experience and uh also your work before my pleasure it's been really fun interviewing with you guys so that wraps up our Hardware episode through the lens of the Z prize I want to say a

big thank you to the ZK podcast team Henrik Rachel Tanya and Adam I want to say thank you Alex for being the co-host for these four interviews that was really fun to do together thanks for being on thank you and yeah to our listeners thanks for listening [Music]