ZKPodcast: Hardware for ZKPs & VDFs with Supranational

Transcript

[Music] welcome to xero knowledge i'm your host anna rose in this podcast we will be exploring the latest in zero knowledge research and the decentralized web as well as new paradigms that promise to change the way we interact and transact online this week tarou and i chat with kelly and simon from supernational we talk about all the types of hardware used for cryptographic computation this is a topic that is increasingly important in the context of zero knowledge proof computation we talk about the spectrum of hardware

from cpus to asics and how they compare as well as the work that the supernational team is doing on that front now before we start in i have two quick notes to share with you first off i don't know if you're aware of this but earlier this year we kicked off a zero knowledge blog the zero knowledge blog is publishing one post a week the topics range from zero knowledge deep dives posts about particular blockchain or d5 primitives as well as posts that elaborate on previous episodes that we've recorded this past week i published a blog post all about nfts based on an earlier episode we had done on them it's basically a thought exercise to help me understand if i might ever want to buy one

tldr i'm still on the fence yeah have a read check it out let us know what you think but also the way that we're finding writers for this blog is that we're sourcing them directly from the zero knowledge podcast community so if you are doing research on a topic or you've written a really great piece that you think we should have a look at if you have top-notch writing skills we do want the quality of the blog to be quite high do get in touch about potentially becoming a contributor you can email us at blog zeroknowledge.fm just send over some sort of sample and we'll get back to you second really quick note i want to share is a big thank you to this week's sponsor ave ave is an open source decentralized non-custodial liquidity protocol

on ethereum with ave users can participate as depositors meaning they provide liquidity to earn a passive income or they can act as borrowers to borrow in an over collateralized way or an under collateralized way think one block liquidity flash loans a topic we've covered quite often on the show a new feature that they've recently released is called credit delegation this is where users can delegate their credit to another person who can borrow against it ave has an ecosystem grants program for anyone building anything that contributes to the ave ecosystem do check out the ave developer portal to learn more i've added the link in the show notes as well as the blog post where they talk a little bit more about the grants program

visit ave.com in general to find out more about the project so thank you so much ave for being a sponsor of the zero knowledge podcast now here's our conversation all about hardware with the folks from [Music] supranational so today taron and i are chatting with kelly olsen and simon peffers both co-founders of supranational which is a hardware company today we will be doing a deep dive into hardware from cpus to asics and look at how these are used for cryptographic processing so welcome kelly and simon hi thanks for having us hi guys pleasure to be here tarun you're the guest host today and i think one of the reasons that

we talked about you joining this one is that you actually have a background in hardware yeah i used to work on uh this asic for doing uh physics simulations of proteins so i i i once upon a time actually knew something about hardware now i'm sure it's all dated actually i think there's the first episode you're ever on i think we did dig into that maybe i'll add the link so we can hear that that was actually when we met yeah cool so i think as a starting point i'd like to find out a little bit more about supranational what is it what is this company and maybe where did it start from yeah sure so i guess i can start um so simon and i have known uh each other for a number of years we

both worked at intel corporation uh me for a decade and i think simon a bit over a decade i think we we actually first met during some investigations into the the bitcoin and blockchain space while we were both at intel i was sitting on the business incubation side and simon was in the pathfinding department you know eventually that that effort never really went anywhere within within intel uh at least that initial effort that we looked at but it was always an interest that sort of stayed in in simon and i's minds and so about two years ago we decided to leave intel and start supernational and supernational is a company that focuses on acceleration of cryptography and other algorithms

we do that by writing software and also developing custom silicon so i'm simon peppers and as cali said i was at intel for about 20 years and i worked in the the server architecture team one of our charters our main charter was to look at workloads that would be coming up in the next say five years or more and anticipate what those would be and build you know support into the server chips to for those workloads and so the team i was on did a lot of work on cryptography um worked on the the shaw and aes instructions and some of the large integer arithmetic instructions so it was kind of from there as a natural jumping off point to work on the blockchain space because there's a lot of really new and interesting

cryptography coming out in this space it's computationally very intensive algorithmically it's complex and so when when it comes to accelerating those workloads which is you know desperately or really needed in the space to become practical it was kind of a natural place for us to jump in we look at anywhere from asics as you mentioned to cpu gpu fpga all the sort of hardware platforms and we we can look at the workloads you know the cryptography that needs to be done and understand how to map it onto those platforms you know what is the best platform we work at uh you know assembly c algorithmic down to transistors right so we try to cover the whole space and

you know really understand the best way to get this you know these algorithms going quickly and practical for people were you working on like xeon phi style chips server chips or when you're talking about having shaw instructions is this for like generic processors or is this for like particular style of thing because it's actually quite interesting especially for a cryptography audience maybe to understand how those implementations differ on different pieces of hardware yeah one of our colleagues is really an expert on the shaw instructions in particular but yeah the team i was on developed those instructions and they're used in a variety of processors anywhere from you know the mobile processors back when when intel had those to the client processors to the servers probably less on knights but more on the

xeon you know the mainstream xeons and they do vary some i i mean a lot of the core of it is is going to be similar but the number of functional units and things like that will vary yeah and i think maybe to add on top of that um you know cryptography instructions you know there's sort of a long history of those being introduced into processors um and sha is one of the relatively recent ones um so it has been on uh amd processors for i think maybe a couple years now and is starting to hit sort of your general laptop and desktop uh processors for the intel uh processors as well when was supranational actually founded like how old is this business because you know

you've talked about like being at intel but like what is the window of time that supernational has been around who is it who does it consist of is it a team like it's the two of you but are there other people that are also part of the org sure yeah maybe i can provide a better explanation there so supernational was founded in 2019 uh simon was the the first one uh actually to leave until and start the company he was soon joined thereafter by myself and another one of our co-founders sean gulley uh sean gulley worked in a similar division as simon working on cryptography and data center pathfinding we also have a number of other sort of part-time employees and contractors as well that we bring in on an as-needed basis

depending on exactly what the project is that we're working on i'm just curious to understand how you work with blockchain because you're not like you're looking at the hardware you don't have kind of a l1 of your own so how do you interact with that space in the blockchain space in particular i think you know to date we are predominantly a service company so we've done a number of engagements and also received some grants from various foundations like the ethereum foundation or protocol labs to work on hardware acceleration not just for their own ecosystems but also develop general purpose cryptography libraries that are useful broadly across the ecosystem uh moving forward we are looking into developing and offering some product offerings uh and

these would be in the forms of uh various sort of cloud proofs as a service as we call it offerings uh to develop basically an easy way for developers to generate zero knowledge proofs quickly and efficiently you know so just you can think about that as a sort of an api service uh and longer term we're also investigating building custom silicon to do more general purpose cryptography operations with the goal of increasing the performance of these things by one to two orders of magnitude wow cool are there any other organizations that you know that are doing something similar maybe even like existing companies that we can kind of associate with the kind of work you're doing yeah in the cryptography space uh i think less so i mean there certainly are many bitcoin mining companies yeah

and in some ways they do a similar uh sort of exercise as we do right but most of those are predominantly focused on just really high throughput parallel hashing so you know slightly different as they're not working on sort of general purpose uh cryptographic type operations simon you gave a presentation at like sbc i think last last time there was one and in that you talked very much about like rsa that was sort of your starting point and i actually i don't know if we've really talked that much about rsa on the show and i was thinking maybe we could talk a little bit about it here what is rsa exactly and why what does accelerating rsa have anything to do with this cryptography that we're

talking about today well rsa is one of the you know forms of cryptography out there there's sort of when you think about uh public key and private key cryptography the two main forms are rsa and elliptic curve and i think what you're getting at is most of the cryptography used in blockchain is in fact elliptic curve um a big part of the reason is that you know the keys are much smaller so when you have to store these things forever uh you know with many copies of them it makes sense to have small keys and rsa keys tend to be bigger but um rsa the fundamental sort of components of rsa and ecc are similar i guess i would say they're both they're all large integer arithmetic and um part of the reason we got involved in rsa is because

and kelly can probably describe this more but one of the first vdfs that we looked at uh was rsa based and so when we started off the company the first project was to look at the possibility of building an rsa based vdf and so that sort of setup set us off on this direction of looking at rsa but since then i mean we look at both rsa and ecc most of our work has been in elliptic curve cryptography thanks for that clarification because when i was doing some research for this i was i was curious about that actually let's give a little framing to the project so it was like the ef right the ef had contracted you to do this vdf rsa hardware yeah that that's exactly right uh so the ethereum foundation has been uh investigating sort of strong sources

of randomness for a number of years and how to use that for leader election and proof of stake protocols uh and so today the ethereum foundation uses something called randow which provides you know very very strong guarantees but there are ways to improve upon that and so one of the things in the middle of 2018 a paper came out by uh dan bonnet and some of the other phd students at stanford that described how to construct one of these rsa vdfs that would provide these stronger randomness properties and one of the biggest assumptions that you need to make with any of these rsa vdfs is is how fast can it go basically and so we were able to do some work with the ethereum foundation to make estimates about how fast this

thing could be sped up to sort of inform the security properties of the pdf so when implementing kind of this repeated squaring vdf what were some of the which is sort of the main the simplest vdf that i think on a previous episode with joe bono um this was sort of covered what were some of the challenges that you found in actually like implementing it was it was where were like the issues more things like you needed a lot of memory was it something where you actually end up having a ton of gates or some sort of like intermediate state that like what maybe yeah it'd be great to kind of talk through like how conventional rsa circuits might have to differ for this application

i mean first of all on the cpu right it's it's easy you can just call libgmp and it uses the instructions and it we were doing 2000 bit 2048 bit rsa so that's pretty easy on gpu right on cpu rather right you can just call these functions and they're they're reasonably fast with the vdf you want to make it go as low latency as possible so to repeat a squaring with low latency we initially built an fpga implementation so a cpu implementation would be about 1100 nanoseconds per squaring for fpga we got it down to about 65 nanoseconds per squaring so quite a bit faster and then you know part of the project was to look at an asic based design so building actual hardware

dedicated for this operation i think the biggest challenge is that if you want to do low latency you have to parallelize as much of the operation as possible and it's possible to do you know do that pretty well with squaring and reduction the problem is that you end up with a lot of gates and they end up taking a lot of area and it's not really even the case the the biggest problem but the wires so you have you have a lot of wires it's a kind of an n squared problem and finding enough routing tracks for the wires becomes a problem and then the delay the time delay through the wires becomes very significant and so that ended up being the biggest challenge for for cranking down the speed was getting the the wires in the routing to be efficient

and the tools the the hardware design tools struggled with that that scale of design yeah and maybe just to add on top of that whenever you're developing custom silicon you always have uh what are called ppa trade-offs power performance in area and in a traditional mainstream cpu that you know is going to be in every laptop and desktop area is really critical because you're going to have to print millions or maybe you know hundreds of millions of these chips and so generally cpu type implementations are really going to focus on you know low power small area because it's going to go into somebody's laptop in the case of the vdf and developing custom silicon for it you know the number one priority was to have the highest performance the lowest

latency on that and so you'll choose very different sort of architectures for doing these multiplications in the silicon and you'll sacrifice area and you'll be willing to accept very high power budgets so that you can get that that best performance and so that's sort of one of the the trade-offs that could be made when you're developing custom silicon right i guess just to follow on kelly for the vdf right it's a very unusual design target uh which is what he's kind of getting at normally you want to have a balanced design for vdf you can really trade off all the other parameters to get the lowest possible latency so it's pretty interesting in that regard that's that's not a target you see very often is the device you made in use by anyone or actually like what what kind

of ended up uh happening with it yeah i can talk about that briefly um so there are sort of three main components that i guess you think about with uh with a vdf the first is this evaluation this repeated squaring and what we were able to do is you know implement that on a cpu you know make it 10 times faster on an fpga and then ultimately design an asic that would be let's say another 10 times faster the second main component is generating the proofs and that's a highly parallel process so that we were able to develop code that would run on a graphics card and would generate those proofs very efficiently and then the third thing is well maybe there's four things the third thing is the verification

right and how do you make the verification of these proofs very efficient and then the fourth thing is what is required cryptographic assumptions to actually make the vdf secure and with the rsa vdf project you have to do a trusted setup and the biggest issue that we encountered with the rsa project was really a secure way to do a large multi-party computation to generate the underlying modulus for the rsa vdf and so that was really probably the biggest bottlenock that we we did hit in that project was to generate that modulus and that was one of the things that sort of stalled the project and prevented us from moving forward with manufacturing uh uh custom silicon so when we first started looking at this

rsa2k operation one of the things that uh we learned about was a competition at mit where ron rivest one of the inventors of rsa along with some others had created this cryptographic puzzle in was it 1999 or so i think kelly may know the exact date but it was supposed to be so expected to be solved in about 35 years at the 35 year anniversary of this department and we figured that by using the fpga and implementing a fast repeated squaring circuit we could actually solve it in about two months or three months and so when we first started the company that's the first thing we set out to do with support from from ef and and others and uh using this 65 second per squaring fpga we were actually able to solve it

in about two months and unlock the the secret that had been encoded and uh you know we had a nice uh ceremony at mit where rauner vest their time capsule and everything we were unfortunately not the first to unlock it two weeks before we unlocked it a fellow called bernard peppero had unlocked it by using a cpu which he had run for about two and a half years and barely beat us to the punch but we were all did okay but it was pretty funny but uh yeah it was a fun fun ceremony cool well we'll add actually i think there's a link that we have about that competition we'll add a link in the show notes to that so if people want to find out a bit more about it

maybe actually it might make sense to just have the terms fpga and asic defined and what synthesis is and what the tools you use are and how you think about like you know the ppa example was a really great example of trade-offs that engineers have to think about in hardware that software people and cryptographers don't have to think about and aren't used to thinking about and i i think maybe yeah maybe a good way to kind of before we jump into talking about zkp hardware is talking maybe a little bit about a the different hardware platforms be kind of the tools you use you know the languages the synthesis and then maybe a little bit about how you guys do fab presumably you're fabulous and you go to some type of

aggregator or something like that yeah so i i'll give the very very high level and then i'm you know i'll let simon uh really dig deep into those uh so the first thing is obviously a cpu and that's something i think everybody knows you know what that is uh it sits inside of your laptop it sits inside your cell phone and that's going to do sort of general purpose computation and to program the cpu you can write in a myriad of languages you know all the way down from assembly up to higher level languages like like python the next one that's probably most familiar with people is a gpu which is a graphics processing unit uh traditionally gpus have been used sort of solely just for graphics processing but about 10 years or so ago we started to

see a trend called gp gpu which is general purpose graphics processing units and that's really about doing general purpose computations on a graphics card are most graphics cards today that second type that gp gpu yeah that's right most of the graphics cards today can support more general purpose programs um the area where a graphics card really excels is with highly parallel operations um so you're maybe you know on a on a normal cpu you may have four cores or 12 col or cores or if you're lucky you got 32 or 64. a graphics card can have hundreds or thousands of cores and so if you have a lot of little jobs

a graphics card is a great thing to use interestingly enough it's actually suited quite well for cryptography and that's one of the things that perhaps we can talk about um but but it's a great tool for that clearly what you've started to see is people are using gpus for things like machine learning uh for you know biology simulations you know a number of different things and and what we're you're starting to see a trend in the industry where more and more people are using these graphics cards for general purpose computation the next definition i guess is an fpga that's a field programmable gate array and you can think about this as um almost like a chip that you can kind of program on the fly so it's not it's you can sort of take

this uh this programmable chip and sort of make your own special chip with it and you can get some performance benefits out of that but it doesn't require spending millions and millions of dollars to build a custom chip which is the last thing that we'll talk about which is an asic or an application specific integrated circuit and an asic is really a chip designed to do you know one or a small handful of things and the audience will probably be most familiar with asics that are built for bitcoin mining so what those are is basically a chip that can do one function it can do you know sha-2 hashing or maybe some other hashing algorithm but there will be hundreds or thousands or hundreds of thousands of these tiny hashing engines on the chip uh meant to just do

as many as possible but you can't you know process graphics on that chip you can't do any sort of simulations on that ship you can't write programs for that chip but you know simon i'm sure can give a lot more information not only on the overall uh asic process but also just on some of the relative trade-offs between these platforms yeah thanks kelly that was a good definition i would say so you know one way to think about it i guess is when we look at us we try to always look at the system level so at a high level when you think about running something you have to think about the compute capabilities of the platform the memory requirements and the i o needed and so if you walk down cpu gpu fpga a6 so cpu you know it has a lot of ios they have

as much memory as you kind of want to put in there they're easy to program you know they're the easiest platform to use but they have you know a limited number of cores you might get 32 or 64 at most so you limited parallelism but they're pretty fast course as you go down to gpu you tend because of the thousands of cores and it has a lot of memory bandwidth but you have to access memory in particular ways for it to be efficient it's harder to get to extract performance out of gpus and so you tend to have to restructure your algorithm to work well on them and as kelly said cryptography can work well but we usually have to go in and change it around sometimes we use different algorithms sometimes we we combine algorithms you know to to really suit the platform

but if you can really take advantage of the capabilities there the sort of extremely high memory bandwidth and all the cores you get great performance four tests that can be highly parallel and they are quite common i mean most people have gpus right and they're pretty easy to come by mm-hmm although although they do sell out sometimes do sell out which is an interesting development these days yeah yeah um fpga is you know increasingly specialized and you kind of wire everything up but they they have more primitive um functional units like adders and things like that and so it does use synthesis tools they tend to be harder again like an order of magnitude harder to use than gpus but for certain applications they're much faster and finally an asic is really just a blank sheet of paper right you can build

anything you want and so they're probably a thousand times harder to build and much much more expensive but you can really make it do the exact application really well so you'll probably get orders of magnitude better performance out of an asic than a general purpose platform tarou what did you work on when you worked in hardware which of those four uh asics i worked kind of on the biology basics that they were mentioning before okay i i i think a lot of people i feel like who worked in asics got into crypto by observing observing how dumb the minor mining hardware was and i'm trying to be like come on there's gotta be something better and more interesting to do at least that was kind of how all my old colleagues used to be

yeah so we've defined these different types of hardware platforms now touring i think to continue on your point or question what then are the techniques used to optimize these yeah i guess like maybe walk i think software developers who who are used to writing user space programs in linux or mac or are probably not really used to this idea of like what is verilog and how do you how do you kind of have to think about when you're copying things from memory and not memory and there's a ton of tools that hide a lot of those details for you i think i think i have at least you know i think that people don't realize and

maybe it would be cool to talk about like what tools you guys find most helpful when writing cryptography for fpgas and asics sure i mean in the end of the day when you're designing for fpgas or asics it's really a fundamentally different way to think about it so there's a couple of languages verilog and vhdl are the most popular verilog is probably the most popular and rtl by the way stands for register transfer level and what you're basically describing is the hardware you know the the registers which registers the store bits so that's sort of memory along with the logic that goes in between them you're describing all that in code that is a functional definition of the chip once you write that and everything is parallel right so when you write rtl unlike in software

in hardware like real hardware everything happens at the same time which is the way the world works and so this language is designed to model that behavior so you write this model and is pretty different from software because everything is parallel and then you do something called simulation there are tools that will compile this model much like you know gcc does and run it and it'll tell you you know what how it performs and how it behaves and you sort of run it and run tests on it and debug it and and fix it and make sure it all works in in the hardware world the time frame to go build a chip is let's say roughly two years 18 months two years or more depending on the complexity of the chip and the cost is millions of dollars right so unlike software where if you

get you have a mistake and a bug you can go fix it if you build a chip with a bug in it it's going to cost you another you know a couple years and millions of dollars right so you have to make sure you you spend a ton of effort making sure it's right so you go through a lot of simulation a lot of testing and then once you have the design you have to go through synthesis so you have to turn that functional definition into actual gates and ultimately uh you know transistors and those get sent as you said you know to the factory to a uh what's called a foundry who will actually build the you know the silicon for you one actual question regarding testing um compared to maybe testing that you uh have worked on previously how does how does writing tests for cryptography

primitives differ like is there kind of like more a different type of testing suite do you find that you have to have more like edge cases or you know yeah is there any difference because i could imagine that it's actually much harder to test cryptography primitives because you really care about the edge cases a lot more than you might in some other processing yeah that's actually a it's a great question it is different so in a normal application you tend to have a lot of what's called control logic there's two two parts of a chip to control logic and data path so the data path are the units that are actually doing the computing and coming up with you know it's adding two numbers or something like that and the control logic is what decides what data to send into the the address and where to put it afterward

is sort of orchestrating the work control logic tends to have a lot of branches and things like that different conditions is looking for so it tends to be difficult to test and validate in cryptography you tend to have pretty simple control logic unlike for a cpu for example that's good news right because you can test it relatively easily and then for the functional units like say you have a multiplier typically what you see with cryptography is it either works or it doesn't and you either get the answer the right answer or there's just total nonsense however the corner cases like if you miss a carry somewhere um that could be tricky the approach generally used you know in these scenarios is to do is formal verification is the best way to do it which is where you you build a model

a logical model of the design and you have to go formally prove that it implements a multiplier or something like that and that can be a difficult task but it's the way you you sort of ensure that it's correct and you haven't missed corner cases the other thing is in some cases of cryptography like bitcoin mining for example it's actually okay to get the wrong answer sometimes if you come up with a hash that that is wrong because you miscomputed it because there's a corner case well in the end you're going to check that on cpu anyway and just throw it away so it's interesting in mining where you can actually you can accommodate sort of incorrect results pretty easily a lot of applications can't do that one thing i guess you know i remember a lot of times

for chips i'd worked on the past we we would like split the design into like pieces that were sort of mission critical and that would we'd formally verify and then other pieces we'd do sort of like statistical verification we're almost like fuzzing where we'd send a bunch of inputs that we kind of knew input output pair expectations for through the different units like do you do you find that you partition kind of the designs or do you do formal verification over the whole uh design because i was just curious because that it does add a bunch of time to the engineering process and it'd be interesting to to know how how you guys think about that yeah i mean you would informally verify the whole design most likely like there are some parts that you can

validate with normal validation techniques and get good enough coverage and have confidence that'll work or you can have workarounds in the case where it doesn't it's really in areas uh the arithmetic is really the place where you mostly want to formally verify things and if you if you could tolerate mistakes like bitcoin in other applications are like that too then you don't have to do sort of any form of verification you can use sort of statistical answers and and do sort of avoid most testing right and say well if i'm right 99 of the time that's good enough cool and i guess one last question like sort of a background question i think for a lot of listeners it might be hard to kind of know the whole hardware stack of like how you go to a factory and how you you know make a mask and stuff like that

but maybe as a startup you have to rely on a lot of other vendors and and different counterparts who who help manufacture the hardware so what are kind of the advantages and disadvantages of of kind of being a startup building hardware versus sort of being at a big big place that maybe has their own foundry has their own fab things like that what's a fab sorry fab's like a fabrication oh uh people who who wow that's such a cool shortening of that word so i mean maybe i can give um a little bit of of information on that um i i mean the biggest the biggest problem with being a startup in the in this hardware and semiconductor manufacturing space is just that it's so expensive right it's an

incredibly expensive process to make a chip and when you look at the ecosystem at this point there's really only three or four companies that have the ability to make cutting edge chips so these are people like intel taiwan semiconductor or tsmc and samsung and so you know in addition to being expensive we're in a situation now where there's sort of a global semiconductor shortage and so you really only have two companies that make chips for other people and that is tsmc and samsung and as a startup you're going and competing against the likes of apple amd nvidia you know google amazon who are all trying to make their chips at one of these two factories so i would say that's that's one of the

most difficult things about being in this space but simon maybe you can talk about you know some of the pros and cons uh based off of your experience making trips at intel yeah i mean one of the i think one of the pros for us being in the space is we the big companies really aren't looking at uh crypto hardware at this point right so we get to work with some of the smartest and best people on these problems and to the extent that we can pick problems that are the right size i mean we're not building we're not trying to compete with those big companies right we're building more dedicated functionality we try to keep things very simple right and simplicity means that it takes less work to build it is less less expensive and because the space is pretty new there's actually an opportunity to

deploy you know fairly simple designs that have a real impact and so you know being a startup here we can we can look at those things and they're they're more the right size for for what we need and what what the industry needs currently right and so i think that's one of the advantages there are a lot of companies that will provide services so if you need for example pcie interfaces or ddr you know those are readily available in the tsmc ecosystem you know so you don't you don't need to be a big company to get all those so we try to take building blocks that already exist in the ecosystem and just reuse them and we tear to take even open source you know there's risk five right open source um cpu we try to reuse what's out there and really both benefit from and contribute

to the eco open source ecosystem in the hardware space that's really a a new kind of development in the past five or ten years but it's pretty exciting because you know it's like open source software it's a chance to move hardware into more of this open source domain and hopefully accelerate innovation in this space so you just mentioned kind of open source hardware do you think you'd actually see a resurgence of open source bitcoin asics or do you think that that you can only do this open source stuff for like future tech something coming up i think for bitcoin asx is probably pretty hard because those are on seventh generation or whatever they're on right of highly refined hardware it's a pretty mature

environment i think it would be hard to compete on an open source basis for for bitcoin as xp just because they're they're really very good right now but you know when you think about building new chips sort of quickly and cheaply i think open source does get pretty interesting you know and there's there's sort of a need i think for that that's really developing and you look around and some of the big companies are are interested in that too right google right is really kind of seems to be interested in this open source hardware they have uh kelly maybe you could talk about that platform that google has opened up i forget what it was called to build chips yeah i think the main thing here is that open i think open source hardware in many ways is sort of in its infancy right now i think you know the barriers

to entry are coming down in the space and so there is an opportunity now you know to use open source tools to develop open source hardware and to release that uh sort of the source code for the the hardware out into the world it is definitely picking up interest you know at some of the major manufacturers like google and others and then risk five is clearly another area where open source hardware is is starting to see some uh momentum do you foresee kind of the open source hardware movement having sort of a linux like evolution where there's just like this many long years of kind of grinding before like suddenly just everyone adopts it because of some kind of use case like in the case of linux it was

you know aws and data centers but or do you view it kind of as like always a forever niche part of the market yeah i think it's uh it's a great question i think it could certainly you know some of the efforts around things like risk five certainly give a give a path towards that linux type evolution for open source hardware and and some of the exciting things that we're starting to see in the ecosystem today is that things like data centers and enterprise infrastructure is starting to move towards what i would call like a multi-architectural paradigm so it used to just be that x86 and intel was the only thing in the data center

and we're starting to see everywhere from apple to amazon to google experimenting with at least arm architecture in the data center and so i could certainly foresee as risk five matures and gets more performant that could be an eventual outcome you know the big the biggest difficulty there will be can you get enough talent to sort of accrue um high quality open source hardware implementations out there in the ecosystem um with linux and software you know i think that's maybe an easier proposition than it is in in hardware and and actually i think just for our audience um risk five is a instruction set so it's like a set of think of like op codes in ethereum or op codes in a virtual machine

and it stands for restruct restricted instruction set computer is that right guys i think that's right and and it's it's sort of uh it's it's like you know kind of one of the most well-known instruction kind of open source instruction sets do we think we're ready for snarks i think so yeah sorry i i hope hopefully it wasn't too pedantic to go through a lot of hard work but i do think um for an audience of like cryptographers and software developers sometimes it's good to get just like get a vernacular kind of like everyone knows the same language absolutely now so we had talked so far we've talked about the project that you did on vdfs we've now defined a lot of the kind of

pieces of the hardware stack which i think is super useful let's move towards snarks what projects have you been doing with snarks what's your goals with this how exactly do you interface with the snark community or snark technology yeah so i can i can start and maybe talk about uh one of the recent projects that we worked on with the folks at protocol labs so for those that aren't familiar protocol labs is um the organization behind projects like ipfs lib p2p uh and most recently filecoin and filecoin is a sort of novel new blockchain that uses storage power as a way to sort of rather than you know traditional mining as a way to give relative users influence in the

protocol to sort of ensure that the accurate accounting of the storage power they use zero knowledge proofs and these are things like proof of space time and proof of replication of which i'm sure you know you've had folks talk about about these things on the podcast as part of this the filecoin network as far as i know is the network that processes more zero-knowledge proofs than any other blockchain protocol out there i think they process on the order of something about five million zero knowledge proofs a day oh more than z cash i guess yeah yeah way more than the yeah maybe more than z cash over its you know entire lifetime at this point right um so so it's a it's a massive you know obviously

massive new sort of paradigm shift in terms of the use of z uh zero knowledge proofs but that also entails high computational requirements right um so you know one of the unfortunate things about zero knowledge proofs and this advanced cryptography is that it takes a lot of compute power to perform today yeah and so one of the recent projects that we worked on them with uh was speeding up both the verification of zero knowledge proofs and also the performance of actually generating the zero knowledge proofs and you're talking specifically about snarks in this case yes that's correct so in the in the case of filecoin what they use is the traditional sort of gross 16 size uh style zero knowledge proof

and they use the bls 12 381 curve uh as well um so simon i guess maybe do you want to talk both about some of the work that went on both on the the cpu side for the proof verification as well as the gpu side for the proving yeah so as as kelly mentioned we looked at both of those um the you know filecoin had a concern that you know both verification and proving for the number of snarks of processing were becoming computational bottlenecks and so those are areas we dove into for for verification there's kind of they're very different sizes i would say so verification of snarks is a pretty fast operation measured in milliseconds whereas proving of snarks is measured in you know seconds even minutes so you know one of them is

is much much harder than the other so the core operation of snarks tends to be something called multi-exponentiation and fft multi-exponentiation is really the dominant computation that has to be done so when we looked at you know verification we looked at a few things one is um you know there's a certain function that has to be performed to show that the snark is correct this is done on cpu because it's pretty fast we do that on cpu one thing you have to consider when accelerating is that even sending work to like a gpu takes time you know milliseconds of time so a lot of times just for something that's relatively quick the overhead of going to an off you know cpu platform isn't worth it so in that case we looked at you know how

do we algorithmically improve that multi-exponentiation on cpu you know we make use of all the cores very efficiently we even uh when we have a bunch of work to send the cores we we try to balance the work so that all the cores finish at the same time so if you do it kind of naively you can have one core that you know takes a lot longer and everybody's waiting for it that's inefficient so we we go to the trouble of making sure that the work is as balanced as possible using all the resources we do pre-computations when we can and that can really speed up multiplication many times so in this case you know once we once we did it put all the work in and so refactored the algorithm and made it efficient right we got about a 10 to 12x speed up for verification

which was pretty big and it sort of met their needs as far as making the system continue to function smoothly and was that on the proving side or was that overall proving and verifying that's just verifying so those are pretty small the next thing was to look at proving and because it's so much of a bigger operation that's a whole you know another up you know optimization effort there we use gpus right of worms it's another can of worm so we use different algorithms it's still multi-exponentiation but we use different algorithms different platform and uh we would sort of refactor the whole thing that that we sped up by about four to five times and that isn't released yet so that's still a work in progress as far as the integration but that'll be coming in in the coming months you just said two different platforms

was it like do you mean like two different hardware platforms like we defined before one for verifying one for proving exactly right so for verifying because it's relatively small we just keep that on the cpu okay right and we use all the cores that are available for proving because it's much much bigger we can we can tolerate the overhead of going off off chip you know to the gpu and it was already using gpu impact gpu for proving is quite a bit faster than cpu already but we sort of optimized it further and got you know another say 4x out of the gpu for proving it's a combination of using all the cores and the gpu as efficiently as you can yeah i guess maybe i wanted to talk through maybe some of the the technical details so it sounds like you know in the case of verification

there was a lot a its memory footprint is smaller and it seems like fast serial computation is much more value than sort of massively parallel type of things but what aspects of proving end up being the most sort of paralyzable because it does sound like a lot of the architectural kind of improvements you you were able to make came from you know taking advantage of parallelism within proof generation so is it like the fft like you broke up the fft into different components or yeah maybe like going through that would be uh pretty great sure i mean in both cases the theme is parallelism right so for verification we use all the cores and and we do the multi-exponentiation in

parallel but because it's small so proving use or verification uses about 32 000 bases we can do pre-computation meaning you know we can we can pre-compute n times each base and we can save a lot of work that way when it comes to actual verification and get say an 8x speed up and because it's small 32 000 bases we have enough memory on the system to actually hold this precomputation does the lookup table fit fully in cache or do you actually have to go to main memory oh it goes to memory but but memory is pretty fast right and you can prefetch those if you want to because you know what the next base is going to be so what what you can do if you want to it turns out it doesn't really help very much because the the cores are pretty good at

prefetching anyway but you can you can look ahead to the next you know base you need the next uh lookup table element you need and tell the processor to go read it in advance so that it's ready but in fact there's enough work going on that that's mostly hidden for you so proving is a bit of a different story because it's huge so in the file coin case they're working with 100 million bases or more instead of 32 000 so you can't pre-compute anything because you can't you can hardly even store all the bases alone so in that case it's really about picking the best sort of multi-scale algorithm in this case everybody it's pretty much accepted practice and state of the art to use the bucketing algorithm a pip interbucketing algorithm which was what was used here too so you know for speeding up this work on

a gpu it is it is massively parallel a multiscaler is embarrassingly and easily parallel by its nature and that works to our advantage so we looked at you know speeding up the field arithmetic on a gpu and we used nvidia gpus in this case that's you know pretty common and you could write assembly code for or a form of assembly code for gpus you know so we worked on that to make you know that a little bit more efficient and then a lot of it or some of it was sort of making the transfer of data to the gpu overlap with the computation as efficiently as we can you want to pick the right size of work to send to the gpu so that it efficiently uses all the resources um things like that how hard was it actually to deal with

implementing the field operations on the gpu where like you have quite a bit of difference in terms of like pipelining for different operations and thinking about warps and how much memory you have and stuff like that like because it does actually seem like a lot of these big integer operations might be kind of annoying on a gpu in fact you can you can actually use the same code to use on cpu pretty well on the gpu for those and it's really about how you uh the size of work you give to each warp you want to make sure you know for example that you're the way you access memory you want to you want to try to pull in memory so that it can be all the memory pull in can be used by various warps

one easy mistake to make as you can read in memory use one piece of it for a warp and throw the rest away right and so how you how you sort of traverse memory becomes important uh how you use registers right you wanna you wanna make sure that uh or try to optimize in a way the number of registers each each little program is using that allows you to run more programs at once you can do that by you know tweaking how much uh data you saw on the stack for example tricks like that when you talk about like using gpu cpu you're optimizing it using assembly you said but like what are you actually producing like do you produce libraries or some code that sits between the gpu and something else i'm trying to figure out

like what the output is when you do this work because i guess you're not changing the gpu right you're not altering the hardware in this case yeah that's exactly right i mean at the end of the day the the output is just some open source code that gets put up on github right so in the case of a file coin you know we provided some optimized code that they integrated into their software uh and now it lives on github and and folks can use it it would be in their client software then i guess or snark proofing exactly exactly yeah so filecoin uses um a fork of bellman which is the original software that was developed by the the folks at the electric coin company and used in zcash they've made some modifications so that it can run on things like gpus or at

least some of the operations can run on gpus and so this code has moved into you know under file coins repository in their snark software essentially do you see you know proof generation moving to to custom silicon versus staying on cpu gpu kind of like more traditional platforms and what benefits and detriments do you see if if say there's a zk proving asic that can handle like certain types of circuits um assuming we're like you know let's look forward like 10 years where we're in a world where people actually want to generate zk proofs of sizable programs maybe like programs that people run on ethereum not you know still a ti-83 level of

computation but it's still much more than you know a z cash transfer yeah i think you know looking forward i think we see this running on a variety of different architectures you know i think from the standpoint of of privacy and decentralization you know a lot of people are going to want to run some of these transactions on their laptop or desktop right and most people you know don't have the need for anything more than that ultimately the question about whether this will move to an asic is is really a question of economics i think does the asic perform well enough that it's worth moving to the in developing this custom silicon and so i think it's yet to be seen as gpus continue to progress you know

how much better asics will be and and what that cost improvement is but i do imagine a future where you know zero knowledge proofs are happening sort of locally on your own laptop or desktop uh they could be happening in the cloud um so we're starting to see zk roll ups be talked about and so or zkzk roll ups so it may be the case that you compute a small proof on your laptop and you send it up to the cloud where they aggregate hundreds or thousands of transactions together and do a very very large zero knowledge proof uh that could happen on large servers that could happen on graphics processors in the cloud or or someday you know if there's enough demand uh happen on on an asic one actual question i guess in that regard is you know in

in people's phones and i guess i i i'm not sure if people know this but like most cell phones nowadays have reasonable secure enclaves and like not non-trivial percentage of you know silicon dedicated to either privacy or security in some some manner do you do you foresee a world where there's sort of a zkp specific enclave on on people's phones or like in devices it's a great question i i think that world is is very far off um you know having having worked at intel every tiny little bit of silicon gets you know somebody wants to add into a processor gets heavily scrutinized because that ends up getting printed

millions or billions of times so i think we're a very very far far away from from that happening you know that being said you know a lot of our the work that we do is is to try and make these sort of primitives as fast as possible on every platform so one of the projects that we work on is a project called blast which is a an open source cryptography library for bls signatures and we've specifically written code so that you can run this on a raspberry pi you can run this on your arm macintosh you can run this on your iphone so you know we're optimistic that we'll continue to get increasing performance out of these platforms and we specifically target those as as we write these applications when you write libraries like this do

you do you end up relying on sort of compiler optimizations to be able to to guarantee sort of performance across different platforms like llvmir type of uh tools or like how do you actually kind of guarantee this sort of platform independent performance because that i i historically that's always been quite quite difficult so i'm curious how you think about that with cryptography primitives sure yeah it's actually quite the opposite in terms of of uh relying on the compiler optimization as as most of the low-level crypto primitives are actually written in assembly that is bespoke for those platforms but maybe i can let simon talk about uh some of the work we've done on this bls signature library and the work that uh one of our colleagues andy

polyakov has done uh which is quite amazing work yeah i mean as kelly said the way you get the performance and really ensure that you're getting the behavior you want is to write assembly i mean anytime you go through the compiler when those change you know it'll change the code that's generated performance may change and and in fact sometimes in order to get to take advantage of the architecture you the only ways to write assemblies so on x86 there are instructions called add cx and auto x that let you have two carry chains and the compiler can't even generate those instructions so in order to get you know good performance for a large integer arithmetic you actually have to write assembly so you know as you said as kelly said most of the code the low level

is an assembly it gives us a lot of control over over what's running there andy polyakov has been working you know in the space for for a long time he he's worked on other libraries in the past like open ssl so you know for example a lot of the work the cryptography on the internet runs on some of his code i think part of what we do as a company is try to find you know exceptional people in these areas to engage and really build the best software or hardware that that we can and really get people who understand the the platform and the architecture and how to really get the performance out of them if we view moore's law as dead or basically almost dead depending on on your part of the hardware world uh do you view kind of like the adoption

of asics and specialized hardware is strictly increasing over time or do you see kind of custom architectures coming up for for cryptography related primitives or do you still think it'll end up being you know risk and arm in x86 yeah i'll get my take and i'll let uh simon answer as well you know i i do think one of the the inspirations for the company is folks like john hennessey and david patterson from uc berkeley who have this vision of a new golden age of computer architecture that's driven by domain specific hardware um and this is just one exit you know cryptography is just one example of generating domain-specific hardware specifically for you know cryptography right and i think we'll continue to see that for other

workloads as they reach you know sort of critical mass and size it's certainly something that we've seen with things like neural networks right um this used to be something that was done just on your on your cpu originally and then it moved to your graphics cards and then now ultimately you've got a number of startups uh generating or building custom silicon for high performance machine learning and high performance ai so i don't think it's gonna it's gonna be one or the other but i do think that we will see increasing amounts of domain specific hardware as we move into the future yeah just to add to what kelly said you know while moore's law is maybe not speeding things up on a single threaded basis every year the way it used to there is a lot of

parallelism available and that's what you know fpgas and gpus and ultimate asics can provide one of the questions i mean i think machine learning is a good analogy here one of the challenges with going to custom silicon is that the space has to be mature enough to support it the last thing you want is to build silicon and then find that two years later you know the applications change they're using different algorithms different data types and your silicon is no longer useful so you know eventually it'll probably make sense if one of these areas really takes off and becomes solidified you know to have custom hardware you have to be careful that you know in the cryptography space it seems like every six months there's new schemes new applications of things so getting to the right level of stability

and then on top of that you know these companies are doing a great job increasing performance of these commodity platforms every year right you can you can wait a year and buy a platform that's twice twice as many cores is twice as fast off the shelf and so those are the things as kelly said it becomes a sort of an economic problem of balancing out those competing forces and it has to just make sense in the end to spend that kind of money and lock in the functionality and get the performance around that you know particular design so i only have one last question and that's actually about the nvidia the new kind of news that came out i guess it was like last week or two weeks ago where nvidia is now going to be blocking ethereum mining so like in the gpus that

they're shipping they're gonna make them specific for gaming and then they're gonna release a new product that is more for this mining on gpu-like platforms i'm curious what you think about that i'd wonder if it means anything to the work that you're doing sure i mean i'll take a a swing at it first um i don't think it has a big impact on uh the work that we're doing it seems that this announcement was sort of um very targeted at the ethereum mining algorithm and in the code that we write is sort of general gp gpu type code that would be indistinguishable from from any other program that someone uh may write you know in terms of why why they may be doing it i think there's

a number of reasons i mean obviously first and foremost you know their core market is is folks like gamers or even enterprises that are doing things like machine learning or ai on gpus and they want to ensure that those folks can continue you know they can continue to serve those markets the other thing though is that a lot of the gpu you don't necessarily need to do ethereum mining right so one of the things this could do is provide sort of a new product offering for them where they can take off pieces of the silicon that aren't needed or that were that ended up becoming faulty in the manufacturing process uh and they can create this sort of new product offering that ideally you know they're able to better extract value out of that market segment so if ethereum mining is really hot they can price that you know that one

higher and if it's not doing so well maybe they can price it lower uh so i think it was ultimately a business decision more than anything so that's that's my two cents on it semantically if they take off everything but the thing you need to mine wouldn't that just be an asic isn't that kind of what it becomes yeah i mean you know at the end of the day a gpu is kind of an asic right uh it just is has quite a bit of functionality and it's got a programming language that that you could do with it but you know you can look at asics are kind of on a spectrum from fully programmable like a cpu all the way down to single function like a bitcoin miner and i think a gpu falls uh sort of in in between there okay and

if it if you remove functionality then it goes further along that path towards an asic i guess yeah i think that's that's one way to look at it i don't know i don't know if that makes sense but at least that's how i'm hearing it well listen thank you so much simon and kelly for coming on the show and helping us explore this pretty wild world of hardware something we hadn't touched on before on the podcast thanks a lot guys yeah thanks for having us it's our pleasure cool so if anyone would be interested in getting in touch or finding out more how can they reach you how can they participate in this yeah absolutely uh if you're interested in some of the projects you can always reach out at hello at supernational.net also if you're interested in joining the team

we have a lot of interesting projects related to zero knowledge proofs so whether you sit on the theory side or are interested in the hardware optimization or even software programming uh you know please feel free to reach out we have a number of open positions and and would love to have some of the listeners join us here fantastic cool so thanks again and to our listeners thanks for listening you