Episode 313 - Revisiting Hardware with Ingonyama

Transcript

[Music] welcome to zero knowledge I'm your host Anna Rose in this podcast we will be exploring the latest in zero knowledge research and the decentralized web as well as new paradigms that promise to change the way we interact and transact online this week I chat with om and yval from ingonyama we cover the foundation of the company how ingonyama aims to tackle the ZK Hardware space and the role ZK Hardware acceleration has in the larger ZK story we also discussed what the hardware product cycle looks like

how a ZK Asic is made the algorithmic component in ZK Hardware some of the research coming out of ingonyama and more quick disclosure I'm an investor in the project via zkv so very excited to catch up on their work now before we kick off I wanted to highlight the ZK jobs board for you there you can find find jobs from Top teams working in ZK so if you're looking for your next opportunity in the space be sure to check it out and if you're a team looking to find great talent be sure to add your job there as well we also have our upcoming ZK Summit event to look out for it's happening in Athens on April 10th the event is shaping up we've been adding speakers to the website and we'll share the schedule there soon as always this is invite only and space is limited

there is an application process and you need to apply to be eligible for a ticket if you've already received an invite to buy your ticket be sure to secure it we expect the event to sell out I've added the link in the show notes if you still want to get an application in hope to see you there now Tanya will share a little bit about this week's sponsor Alo is a new layer one blockchain that achieves the programmability of ethereum the privacy of zcash and the scalability of a rollup driven by a mission for a truly secure internet Alo has interwoven zero knowledge proofs into every facet of their stack resulting in a vertically integrated layer one blockchain that's unparalleled in its approach Alo is ZK by Design dive into their programming language Leo and see what permissionless

development looks like offering boundless opportunities for developers and innovators to build ZK apps this is an invitation to be part of a transformational ZK Journey dive deeper and discover more about Alo at al.org and now here's our [Music] episode today I'm here with om the CEO of ingonyama and yal Chief Architect welcome to the show both of you thank you H I'm excited to be here thanks Emma it's great to be here so today I think we're going to be diving back into the hardware topic but before we do that ingonyama the name ingonyama I feel like we should say what that is it's one of

the most unusual names in our space just tell me what does the name in gyama mean where does this come from yeah so there's the origin story and there's like what does it mean mean I think that by now the association is with the Lion King right I mean originally it's like coming from Zulu language it's like from South Africa means lion right uh therefore they are mention in in the circle of life and and throughout like The Lion King um we recently shared like we recently announced the company uh funding and then we kind of like checked that in Google search we kind of like dethroned like when you search for inyama it used to be the Landon King now it's like you know our website our GitHub our Twitter and so on it's like

all the way to like the the YouTube video is like number nine or something yeah no it took me a bit of time to to get used to the name but actually a funny thing that happened two weeks ago I went to The Lion King in London yeah and I actually noticed every time they said inama it's pretty weird wow wait is it in the song in gyama so I'm not going to sing not going to embarrass myself and uh uh lower the rate of your audience uh but but basically in the circle of life like the song which is known by like 99% of you know world population they mention it 30

something like 36 times when you try to try to think to yourself like you know at least the beginning gu so they kind of like say it repeatedly like over and over again so it's like wow it's like your subconscious in a way like everybody like heard heard it at some point in life wow that's so cool well thank you for sharing that with us I've been very curious for a long time okay let's kick off Omar you've been on the show before you were on a few years ago but you were talking about a completely different well maybe not completely different a different topic a different project you were on for Zeno at the time and we were talking about MPC and threshold cryptog

graphy I'm really curious like yeah what's what's happened since then and and what made you make that shift over to ZK and ZK Hardware specifically indeed uh it was uh three years ago also with the ZK God Frederick on the show and um my old co-host for anyone who doesn't know yeah and and uh I had a lot of fun talking about like all these crazy ideas with with social cryptography and um indeed it was uh it's part of my role at Zeno I was uh doing a lot of research a lot of innovation a lot of Open Source uh cryptography uh which is very rewarding and and again like also you get to work

with uh people uh all around uh that are really talented and dedicated to the goal of like decentralization and and and security and and privacy and all of that at some point I needed to uh be aware of the context which is a consumer company and I fed that I kind of like gave it everything that I can give and it's kind of like I mean in a way I'm like a a Founder for the early stages of of that company so I felt that like the ship is moving without me now and I can let go and and I guess optimize uh on what should they do next uh the move to Hardware nzero knowledge is actually very linear right so ZK and PC both are like part of privacy announcing

Technologies the pts pets ZK is being being used quite a lot extensively in in in in NPC and uh it's also a very fundamental kind of ID so it was always there for me it was no strangers to to ZK Tech it also had a very uh clear opportunity and uh Hardware is is you know for me working in open source software uh Hardware is just taking it one step uh further like have more like another dimension like more freedom uh to design and build systems the way that they should be built uh so it it made a lot of sense and and other than that basically the other other things remain the same right so we i cho the consumer part kind of like kept it at the

infrastructure focus on on what I'm passionate about and and and that was it right so that's the overall ID but um the way that things played out is that I needed some conviction at the beginning so it's not just like you come up with this ID it was not even my ID so Justin Drake was advising us uh I think was the the one that was in charge of the Inception but also many others uh and the more I thought about it the more it like made perfect sense it's like inevitable like any technology that disrupt and become like mainstream need to have this kind of like leg of of for Hardware that's interesting you mentioned Justin because I remember having him on years and years ago for

like vdfs and then there was this conversation about these vdf Hardware pieces and I know I want to bring this up later on Justin recently shared on Twitter that there's some sort of ZK Asic that exists that's cool though that like that conversation was happening back then what else was it though about ZK because like you said you know ZK and MPC are sort of under the same umbrella was there something specific about the ZK space that made you decide to take that track indeed there's a reason why we focused on ZK when you think about it what we do is basically the intersection walk at intersection of uh Hardware or Hardware software Cod

design and uh mathematics uh and and whatever we do can be applied to ZK fully morphic encryption FG and and even in the future maybe NPC like there are some parts of NPC that can be accelerated as well now when you start a company you also need to think where is the where's the market right where people are building what they're building and so on and it was very clear that ZK you know yeah ZK you know four decades into the making in terms of research uh let's say a decade or so uh more or less when it comes to engineering software engineering it was a very uh and you see this kind of like products coming out and and actually

making difference like you know on on on real world uh and users so it was more appealing to us to start from that technology and also so the blockchain web 3 Market that's where the action is basically yeah cool yal I want to hear a little bit about your story it's the first time you're on the show so maybe you can tell us a little bit about what you were doing before and what led you to be interested in this problem right try and just give you a real brief intro of my background feel free to dig in um so I'm an electrical engineer I have a PhD in information theory in the context of

communications been working in the industry for nearly 25 years which industry well I worked at two uh uh large semiconductor companies Texas Instruments and broadcom uh very many startups a stinct in the defense industry I did mostly modems physical layer basically like everything from research algorithms all the way to uh very log uh designing and a system and uh a little bit of time I spent uh on algorithmic trading high frequency and also not so high frequency so you've touched a lot of

different things that would contribute to you working in this industry now it sounds like I think so it's it gives me um I guess some advantages but also there they're also disadvantages right because I haven't really been a computer scientist uh and like specifically in this field uh for very long so I think getting into this field about two years ago uh Danny who's who's also uh co-founded ingonyama with om called me up now he knows me from way back at Texas Instruments and he said um do you want to join us and then I think maybe like 15 minutes later I I was already on

a call with om and he explained what CK was and I I think it was pretty immediate for me I mean like it was just very exciting to get into this new world nice so I want to talk a little bit about the story of ingonyama after founding so you had the idea Omar and you were saying you were kind of talking with Justin Drake you were sort of thinking of the next Direction you landed on something Hardware so I I've been saying Hardware but actually how does ingonyama Define itself at the moment like what exactly did you set out to build and has that changed over the last few years uh yeah that's a good question uh so it didn't change much uh but on the other hand they changed quite

a lot right so I mean I mean first thing you need to understand about Ado is that Cycles are um like you know they're very slow it's not like software um it takes time like you need to develop expertise you need to build some kind of Ip even what you know the the programming part like is is just longer uh in in in general so originally we thought about ingonyama is kind of like um a very traditional Semiconductor Company in the sense that you have an ASC as a Target uh and you just go and and build this ASC there are few steps you need to take along the way like you need to again develop your IP Calles you need to

emulate simulate run fpga uh that would kind of like show you how it's going to work build some software stack on top find some customers and and basically scale from from that point right so one ASC follows another Asic uh design improving with technology process um and also just the architecture uh perhaps I don't know you can think about other semiconductor companies like Nvidia today where this is kind of like you know the way that they uh push products like every let's say six months one year you have like a new ASC coming up now like the starting point was basically let's break down ZK into the like

components that need to be that we want to accelerate and then um let's figure out a way to accelerate those in Hardware uh you know stuff that we are you familiar with MSM NT you know all of these like massive building blocks uh back then I this like um talk that I gave what we call it like there's MSN and there's NTT and there's fluff right now one thing that uh we learned is is that's a bit misleading CU like the fluff is important and also the you know the being application specific is important being uh attuned to what like the community the ecosystem actually need is also important and and also

making Hardware available like now and accessible is is is also important so it means that for us um while we still work in the same trajectory like we have now much more mature understanding of ZK uh we have kind of like IP that is like sitting on the shelf and now we started not long ago just to build actual designs out of it that can lead to Asic designs that can solve like a real world problem we also figured out that the approach would be kind of like end to endend so you want to put everything into the hardware everything you can and preferably you even want the design to start by considering the

hardware like you know doing it backwards meaning you already have a system whatever some Halo 2 proving system now let's try to find ways to accelerate it is much less effective than trying to accelerate or trying to build like howo first howo friendly let's call it um ZK prover uh so we've we've taken this approach uh at some point we've realized that gpus provide a lot of advantages they are like easy to come by like commodity Hardware almost uh very cheap easy to program and and provide you a lot of like value meaning that you can get quite a lot of acceleration at a good price while uh like just using gpus like almost just by

Plug and Play So we had some Focus or we we've been focusing also on on on that right so I guess like we do have few branches of work yeah um definitely the road map leads to as6 that's the way that ZK should run but we still want to also address pain points and uh solve problems and and you know enable ZK today for different type of use cases and and different customers that's cool so you've just mentioned a few things that I I think are listeners may be familiar with but I do want to just do a throwback to an episode that we did last

year all about ZK hardware and Hardware acceleration because some of the terms like fpga GPU Asic the MSM the NTTS like just how these things relate to one another all of this was covered in that show so I'll definitely add the link in the show notes I really like what you were saying though about like it was almost like Asic is your North Stars like that's the aim but to get there and this is what I've sort of heard all the time is like since ZK proving systems change since there's different flavors for different purposes or Trends almost in some of the research and like you're seen things pop up you know at times monthly like a new proving system that's

super exciting when you start to think about an Asic which is so concrete and sort of Final in a way like it's not as malleable as say an fpga is it very difficult to do that in such an environment like if things are changing can you ever really be sure that you're building something for the right system you right with this Insight there's like or at least what we thought can be a way to overcome it is by having something which is like a programmable Asic right so like you're building a computer that um can do many things but um is more suitable for the type of arithmetic used in in ZK and cryptography like final

field arithmetic so that's the notion of zpu which we try to develop and then push and till today we use it as kind of like you know when we think about how to design a general purpose Asic again I know the name is is like kind of like as a contradiction in it but still that that's the model we've been trying to to push so that you can in a way handle different provs protocols and accelerate all successfully I think this goes back to the talk at zk10 the talk that you gave yeah that I gave yeah but but others helped me uh build it we we'll add the link in the show notes for folks to watch it I did watch that in prep for this so yeah sorry go ahead you yeah um

so yeah I I think that um there was an argument that I tried to make there that is um I I think it sort of gives context to to everything that that you were talking about just now like Hardware is like an evolution like you mentioned the North Star is is some Asic that is like like exactly what we need but to get there we sort of have to have these like sort of steps of like we produce new hardware and then uh we use that Hardware to experiment then we get new ideas then we get new algorithms then we need new hardware and it sort of and it just doesn't happen without that process I I think that's just our thought

process like we need to be able to experiment and I think that that what you're seeing like that mature that om mentioned before is that it's basically us going through this Evolution I guess with the rest of the industry and we're just adapting so like the details change but we're still going in the same direction how is a programmable Asic or the zpu different from an fpga because I always thought of the fpga as the programmable Asic like isn't it sort of like between a GPU and an Asic you have the fpga which is malleable but like I don't know you have to be a bit more specific whereas GPU or CPU are more and more general purpose

but less specified I I think first of all all Hardware right CPU GPU fpga zpu whatever it may be it's they're all A6 right because they're just different machines built into silicon so I think if if we try to sort of rank them maybe fpga and its likes that they're Al so other uh like older things like cplds and and other types of programmable really like programmable Hardware at the at the gate level maybe they're the easiest to to reprogram but there is a very high cost associated with it it's an extremely uh uh High powerered device

um if I compare it to a very targeted Asic if I Target a specific protocol right than than the actual silicon area that I will spend on a certain function because I know exactly how it needs to be so I don't need all these interconnect that I'm not going to use I'll just throw it out and and so the area goes down uh as a result the power goes down so I get a very specific a but that as can only do one thing right it cannot be an fpga I can't now take it and program it for something entirely different so I think that's that's the way to look at it and

actually in that context uh a GPU is from our perspective also programmable device maybe not as programmable as an fpga but it is a programmable device and we think that it's actually a little bit more suitable for the work type that that you know zkp and and and those Technologies are currently uh using do you then put cuz I always had the order of like CPU GPU fpga Asic but what you're sort of saying is that like gpus is programmable but less programmable than an fpga and then Asic is extra less programmable and would you put then the

zpu between GPU and Asic in terms of its programmability actually I have uh on one of the slides I'm actually looking at it now but I this from your talk yeah I'm sorry that can't show it but I I sort of put all of these devices on a two-dimensional grid of uh flexibility versus uh Power or or power per security per bit of security and so actually I bunched CPU and GPU and fpga on the high flexibility but actually high power and a very very customized asek that may be

only does a specific protocol would be very inflexible but very low power and I think that the the zpu tries to be somewhere somewhere in the middle okay so it tries to be low power but it also tries to provide you with sufficient functionality to cover maybe a range of protocols is this because like zkps even if you're using different systems proving systems there still is sort of something underneath that is becoming more clear and optimizable that does power all of these systems is that why one could create something like this um I would say it's it's a bit of a it's a

bit of a chicken and egg I mean I don't know if but I guess we haven't yet finalized um like how wide or how flexible the CPU will be but I think that whatever we decide to put in that the protocols will also if this is good enough then the protocols will also sort of Target that right like they'll change the prime field because they can actually get an added value and that's what they've got so I I think see supporting everything um would not be a good operating point you would want to support something that would be sufficient to give enough flexibility but but also not too

much how expensive is it or how much work is it to get that first iteration that you could actually use is it something that a small team can do in a month or two months you can try something out test it out people could use it or yeah what are the cycles like how expensive is it I'm just curious how you think about this the as process actually starts off with maybe an architecture stage where you you first of all need to know what you're about to design right and that's that's very fundamental so so actually we usually think of like the ASC process as starting from that point from which we know what we're about to build so lots

of Asic follow a standard like a Bluetooth Asic right so you know what you're going to build now you've got a few stages you've got the front end it's sometime sometimes called where you actually write the very log and and you do quite a lot of verification and actually the verification is is extremely extremely important in A6 it's where you spend most of the time uh and then that goes over to the backend process where things actually start being moved from code to something more physical and then you'll have uh other types of verification where you actually

verify gate level stuff uh you do power uh type of simulation s um their various stages and it takes probably in in like the most Advanced Technologies uh to get your initial test chips from the minute you you kick off the design would probably take about two years that that's like the wow yeah and then if you're lucky then your first tape out is already something that that works a lot of times uh you do need to go to a maybe not a second tape out there's sometime there's stuff that you can uh fix in in

What's called the metal fixes which is a little bit uh less expensive and a little bit quicker and uh getting I guess getting from there to like General availability is is is another process that I'm I'm not as you know um but that that 2-year period you've gone through this I guess you like know what that's like yeah first yeah about the two years so just as an example you know the chip that um like recently there was this uh announcement from from exil about ZK snar uh Asic so they actually started to work on on this chip before we started the company before we started in G it's a long process it's also an expensive

one I can tell you that when we now think about uh taping out Alo uh approver uh like an Asic we're talking here about something like 20 million to get it uh you know total cost so that that can be also very expensive now there are ways to mitigate all ways to take it you know step by step so as I was saying one thing is just to have the IP and and stop there so you don't do the full process uh you don't need to spend all this money someone needs to do that right but they would just take the IP from us the IP call from us they would I don't license it or something you know and that's a big part of the work by the way I mean you know the all of the design process and and and testing and such can can also be

done there and it can be done on fpgas right so for example with the zpu ID there is this kind of like processing element that is kind of like being replicated or across the Silicon so you take one of these and then you implement it at the very log and you implement it in fpga and then you can test it with different inputs and so on see that like it obeys the you know your design and and what you predicted that it would to like the performance and so on m is that a lot of the work in designing these like kind of you all what you had sort of described is you're coming up with an idea you then do these verifications and I'm guessing like lots and lots of tests so is it sort of like here's the idea

here's the implementation and then you just like try to battle test it so so I don't think that you're not battle testing the architecture right that's like a a pre-stage but the verification would test the actual silicon to see that you've uh actually implemented it correctly so it's a lot of corner cases um a lot of coverage the basically taken uh a lot of uh Kno hows from uh formal verification uh that was done for as to actually uh verify these protocols not not as in like CK verification but but make sure that the protocols are are secure that they're uh full proof most

of their verification that we're talking about is to make sure that you've implemented what you wanted to implement by the way there's also a validation stage that is post silicon where you're actually testing the Silicon to see that it also meets everything you screen the Silicon you know you'd have like uh certain devices that are close to the edge of of uh you know that the actual like physical material that function a little bit differently um that's interesting so what you're saying though is that like this verification that you mentioned is less like battle testing but more like kind of in the spirit of formal verification just like mapping it out making sure it

runs the way it's supposed to mhm that's so interesting and especially testing Corner cases that that's the important thing there I don't know that I've heard that part of the story the time frame the cost the sort of the process I want to go back Omar to what you were saying though about like licensing if you're a small company you create the IP you create the design would this almost be something that like Nvidia or AMD would like because they have I guess all of the supply chain right like they have the chip manufacturers and they like I'm assuming they can do really quick iteration or like faster physical Hardware iterations yeah exactly as you were saying I mean for us like what you

need to understand is is you know small company betting on a chip and failing means the end of the company yeah like it's it's a one shot thing now let's take a you know continue the example from from from Alo let's say we work with bitm right so bitm do mainly Bitcoin as6 and they are really good at the supply chain like they can take design so we save them all the trouble of understanding what ZK even means right hypothetically speaking but let's say they take the design and just like go through the this like tape out process um all the way to you know sales and distribution which is critical also right and for them it's like super

efficient they already have everything in place we for us it's going to be first time everything going to be slower more point two ARs and and so on so definitely there's like a good point for collaboration here uh between us and these type of companies that's interesting just recently Justin Drake who we just mentioned a few times previously kind of tweeted this picture of what seemed seems to be a ZK ASI I'm just curious like yeah what is the landscape like for other people trying to build that what do you make of that tweet I don't know if if there's much more to the announcement there but yeah so first I think it's big right again we're talking here about uh probably a

process that started a couple of years ago uh and cost a lot of money and it also can be like you know a game changer right so again using my example of of Al like prover if you put this like ZK snug chip uh into play it would uh basically be game over for gpus or any other thing any other Hardware in this network until you have competing as6 right um that's that's kind of like our assumption and it's also probably the first AIC that is like following this like design pattern of becoming of being programmable uh like the zpu concept uh it's not going

to be the last one right so we know that other companies are planning and probably will also launch like even this year uh 2024 uh this type of as6 it's kind of like now I guess the the standard uh since again like there are so many protocols and you don't really know what you are aiming for uh then then everybody's is like going for you know one kind of uh general purpose ZK processor uh but yeah I mean the fact that this can be used like ACR different protocols it's very interesting and one last thought I would actually say that the first ZK Asic was by uh was from supernational as part of this vdf that you mentioned before so they actually

done as vdf Asic that includes a Nova um or whatever you need for Nova so I would say that would be the that was the first although it's now kind of obsolete uh but yeah I think it's huge I think it's like also very surprising like one of the uh impressive things in in in in this like tweet is you know the speed right the fact that we are already at the age of of Z6 it's it's here that's very cool um in that case he talks about it like ending ethereum fragmentation um I actually am curious today we know about ZK rollups we know that there's proving happening there's computation being used to create Zer knowledge proofs how does

it work today and how would an ASI change that potentially like yeah maybe just paint the picture for like what a sequencer or or prover on a rollup is doing and who it is and how many machines and where it lives I'm leading you in a Direction so so first first uh I'm far from from an expert and and Justin Drake definitely is is you know much more qualified to talk about it and he is talking about like this future of of ethereum um but in the context of of Z approvers and specifically you know very fast low latency Z approvers which is something you can achieve with an Asic so what you want to achieve is that

these rollups would be able to communicate like between themselves but in in this Atomic uh way right so you want to enable this like me and all of this other things that we have today um but happening ACR rollups and the way to do it is by kind of like you know selectively taking the transactions that are relevant from different like l2s by the way it doesn't like matter if it's zy all up or optimistic all up just like taking this like transactions and and and like proving them and and and have them like in in in this like one poof and for that you need to have relatively extremely low latency uh so that this can happen like in in this manner uh so today it's indeed like each rollup can be thought about as as his own Island

right like with the own set of smart contracts uh where there are some I mean I me I know Justin and and others are also trying to have this idea of having like ethereum as the Basel allowing more of like joint State uh in a way and and obviously shared sequencers but but still like they are very like different and and independent having ZK is probably and and that's by the way I think it's it's might be the only place I know that ZK is like it's like the only way to do it like if you actually want to achieve this real time cross chain communication you do need like ZK that would run very fast I don't know you maybe you want to add something um what om said is is I think it's what we

call the bridges right where really you simply cannot do them without having low enough latency I would say that also just the rollups themselves um at some point and it really depends on on how how much we want to scale uh blockchain or how much we we want to scale ethereum because I I think that if we look at the pricing right um if we look at just L1 transactions are expensive now a rollup takes a whole lot of transactions and and rolls it up into a single transaction so now that cost is amortised across many rollup

transactions but all of a sudden we've got new costs so one of the costs is running these proofs um but perhaps a more significant cost at this point is then storing all of these transactions because they still need to be stored so so maybe they don't get stored on like the main storage they currently get stored if I'm correct into something called logs at least in ethereum which is a little bit less expensive but I think that that that is currently the bottleneck that is why at the current state ZK rollups don't scale more than than they actually do and they actually don't scale very much at at this point

um I think once that barrier goes down and I know it's it should go down in ethereum I think that's the whole T shading thing is meant to bring that cost down but then I think the next barrier if we want to scale even more would again be power right so we have to bring down like the power per proof otherwise we'll get stuck there I think one other piece of this puzzle like because I just did an interview with Celestia and also with a veil and with Anan like the da narrative I think takes some of that weight off it's almost like you're reducing all of the other latency and cost so that all of the sudden that zkp and the actual time it takes and

everything becomes the bottleneck but so far it's not that I guess right that's not our impression actually uh it is already had this our impression now I had a very interesting conversation with uh with someone from ingonyama with Orin um and uh we we think that currently the the bottleneck is is the storage cost okay um of the rollups and and actually not the the cost of proving but it will become the bottleneck yeah eventually I think that follows that that follows what I was saying that it's not currently but but there's these ways that they're going to be reducing other costs yeah and then

eventually I might even take it you know into a more controversial opinion is it's not just it's not now the battl neck might be that it's never going to be the battl neck with ziki rollups you know as like Standalone l2s that that was I was trying to say is that it might manifest itself like ZK A6 more in this like Atomic Bridges but with rollups you know they are so complex systems and even if you take just like zik proof you already you know we have a huge battle like just at the witness generation part like that that is also important to mention now you also add to that like the sequencer which is also a battle neck and just again like generally speaking most of the cost today in in a

wup goes to this like on chain fees um and when I say most it's like you know 90% And even out of this 10% cost that you have like for this like running this infrastructure provs like can be very small percentage maybe maybe one 2% out of this um like total cost of of operating a rollup so I'm not sure I mean at which point in time depends I mean how we didn't see rollups yet like you know become like fully decentralized at this point so it's how to predict how this will play out cuz there might be that Prov will have like a more important tole maybe some po networks would start to to emerge but at least

now when we have this like stage where there are like mostly centralized I don't think that like Cora operators think too much about like we know exactly how much they think about provs so I'm saying that I'm not sure that's like the battl neck and if that's like you know when they think about improving it of course it's impr it's important to improve that as well as other parts of the system just not saying that oh this is where A6 should come in handy right oh we have to have A6 otherwise the wall is not going to work right I mean you just touched on a bunch of topics that I think are really relevant today um the prover Marketplace the shared sequencer like the thing is those things need more adoption for us to see how the economics

play out if they work and yeah if at some point a chip or acceleration would would actually help in that way you just also mentioned though the centralization of the rollup I kind of want to dig in on that because when I talked about like where is the proofs happening what like where are most like zkps being created how does it work today yeah take like one company can be uh any of the leading zik rollups and they basically operate everything right it's it's not just the sequencer and the prover it's also you know the I guess the Explorer right like you need this kind of like features in

in in in the chain so that like it would be active and and uh and usable um so they basically run everything and you as a user usually you don't interact directly with the rollup right so you kind of like do like there's like I can build an application I don't know on top of skull ZK sync whatever and my application would interact with the rollup smart contracts yeah um it can be I don't for example with GK sync like the most popular app uh is like a swap thing I guess they found a way to make swaps like really fast and also cheap so I I can do do this kind of like swap thing uh with some third party with liquidity providers whatever like this

entire like unrelated to rollup eventually I'm going to settle using the rollup uh and they're going to collect all of these transactions right and and as you said so basically batch all of those into this like single transaction uh on on ethereum and this like on chain fees is actually expensive right that's kind of like uh as I said like a big part of the expenses so it depends right so some waps are using some kind of like cloud-based Services um today right can be AWS can be uh gcp to one provs um they can also do it on Prem so today we also have uh companies that are doing this like cups uh provs like just with their own servers and and and that's

fine uh allows you maybe a bit more flexibility with your setup more control and yeah I mean you said Prim on Prim on premise like like you know on own Hardware yeah oh on your own Hardware okay okay sorry I didn't yeah yeah just like your own Hardware versus to like uh use some instance in the cloud uh so we see both and um we also see both CPUs and gpus right so ZK sync for example they are heavily relying on gpus for the prover it's like already very optimized for gpus um others can be going in that direction uh eventually I believe everyone would use like specialized Hardware just because again it savs power it's like better most efficent more efficient way to cut costs so

that's kind of like how all of the infrastructure there is working but yeah it's centralized like operated by by the zik Operator zik Operator talking about the marketplaces that's definitely an attempt to decentralize the proving action would something that you folks build like maybe maybe you design it and someone else produces it but like is that where you could imagine the ingonyama hardware being used or the zpus been used it's a trend like uh I see a lot of companies now um trying to build in that direction and in in a Marketplace you need to have you have two sides and here you actually have three right so you have maybe four I

don't know let's let's count together so you have the the consumer like the one that actually request a proof like I need this thing to be like ZK PID there's the the hardw provider okay so today we don't have marketplaces that running in this like whatever like hypers scolars we don't have this um marketplaces running in in in with hypers scalars or or someone needs to provide Hardware on the other end and should be rewarded uh um for for you know letting people use their hardware and then you also need the algorithm right and that's where I guess inyama plays a role right and and I guess the fourth player is the want to actually build the marketplace right think about uh exchange right so someone needs to

build the software for the exchange so someone needs to build the software for the marketplace uh and and I guess you know when it's becoming decentralized it's becoming this network that someone needs to to design and and and test and and and then deploy and so on and uh it's it seems like a very promising ID I have to say although when you try to we just enumerated this like four different uh participants when you try to think who's falling into each slot it's not yet clear again like who's asking for proofs like who's going to use this type of like Marketplace like asking for proofs who's going to provide the hardware so definitely there like you know let's say I have gpus um so might be a good alternative but you need

to come up with the incentive mechanism and and so on like it's it's it's a bit of a challenge and and even even for us I mean maybe I can do some let's say that uh I'm providing some algorithm for this Marketplace like making it available for the marketplace but um I'm keeping another Al which is better um to myself yourself and you're also running this yourself probably and then I run it myself right so how do you make sure you need the incentives in place how do you make sure that I'm incentivized to provide or to compete with others on providing the best algorithm right so it's it's there are like a tons tons of questions around that the only thing

right now we have like abundance of is this like teams trying to build the software the network for the marketplace M which is I would say important but it's it cannot work alone um so I'm definitely thinking it's important and it might play out and there's definitely room for such a network I'm not sure if like too many such networks but it can make sense but it's still very early to say right we still need to kind of like try to see the full like flow happening how everything falls into place like I need this to happen and and like you know uh the other play needs that to happen and so on how everything like fits in you mentioned that ingonyama could build the algorithm or be the the

part of this or the sort of the participant that brings an algorithm is that also something that you do today and is that similar to the work you talked about in terms of Designing a chip right so I guess I was mostly referring to how this would look like using gpus which today are are available and you can have access to a lot of them right like you have a lot of data centers just dedicated to gpus and they are programmable right uh we do a lot of work on gpus so definitely we can fit in there with algorithms um if you also add as6 to this Marketplace equation so as6 can be very specialized as uh you all

pointed out right so it can be I don't know an Alo ASC kind of like using my example here uh which means that it's like coming with the software built in right like you don't need the algorithm it's already like you know it can do only one thing it can also be this like zpu type of an Asic which can run uh different algorithms and you know might be that we'll see that once you have enough as6 in the market people would start explore them coming up with algorithms for this as6 um and also with like software implementation for these algorithms uh so it kind of like going to become like a zpu is another type of GPU that you can program and put into the marketplace and make some kind of

like profit out of got it maybe just going back to ingon Yaman specifically are you working on those two fronts in parallel then are you on one side working on like the design side of a of a chip but then also on the algorithms that work best on gpus uh yes you can say that uh we do have a walk on GPU that can be used today by by anyone um and we have also this like long-term walk about on on on as6 right I'm not saying that the first one would be like the zpu itself but but we definitely do a lot of experimentations and and iterations as as we pointed out to make uh Hardware um to make it real like to actually take you know as6 and and and

bring them to to the world also I I I would say that the GPU work that we do also complements the hardware process because um we learn a lot by using uh gpus we learn what to do what not to do so so it's complimentary to to the Silicon process cool you've mentioned in talks and in TW on Twitter icicle can you explain what that is exactly is that on the algorithm side so icle it's it's first it's an open source library and uh it was meant to allow developers to tap into gpus okay okay so today it's it's Nvidia gpus so it's

strictly Cuda based we put emphasis on on the API so to make the the developer experience extremely smooth and and make developers happy uh so the IDE is that you know the msms an entity that uh we talked about like the most basic level of integration would be take icicle and then switch from CPU to GPU specifically for MSM and NTT doing that for your protocol as we've seen time and again already gives you something like like it it gets you to a point where you have like very quickly you have something which is already uh more performant than what you had before um but it's only the starting point so icle can also allow

you to do all of this like glue in between and all of the polinomial arithmetic that we have and it in theory should be uh not in theory also in practice should should allow like practitioners and also researchers to write like big chunks of their protocol in the GPU right now we we we still as an industry as an industry we're um mostly focused around CPU but if you look at AI for example no one is using CPUs so everything the conversation is is around gpus how do you optimize gpus how you do the engineering with gpus we try to do that but for ZK right so I is is just a way to get startu Ed and uh to experiment

and to get your system to a much better place than it is but but very very quickly I feel like when you talk about sort of the proving Marketplace but also this work when there's this competitive environment the Asic development proving in general like if you think of a lot of the consensus systems that we have they're economic games of Balan to be able to decentralize a network right and some of them work better than others you know some of them end up becoming quite centralized but in this case where you have this competition like say we have a proving setup proof like multiple people trying to prove or other Hardware companies trying to enter the space when I hear it it almost sounds like you

would keep a lot of the stuff that makes your prover super strong to yourself or like your algorithm to yourself do you know what I mean like you kind of mentioned that you'd release part of it you'd release something but you probably wouldn't release all of it and I just wonder like how how does a market like that work can you actually achieve that decentralized evenness if there's this hidden part yeah I I I think well I mean obviously if there is enough competition then then that can break that right because if you keep it to yourself and someone else offers it then no one's going to come to you but I don't know perhaps it can work economically but I

think there's to me there is another question if I get you to do my proving then like I I have to trust you to some extent right because I I I need to give you the source so to me that's that that's something that that needs to be answered maybe maybe that's where MPC comes in but it's another level of complication I I just wanted to emphasize that traditionally semiconductor industry is extremely competitive and also extremely secretive and and we try to break it we've been trying to like use a different model since we started the company now I cannot tell you at this point how to win the market but I know

that we will fail if we'll keep this Wars between us right so I know how to not win for sure um therefore I mean for us transparency was and and is critical right everything we do we try to communicate right we were the first one to communicate this idea of zpu although I'm sure others I know others are like thinking the same lines uh but it's important to bring this like uh conversation you know to the to the public icle that you mentioned it's MIT licensed open source uh I think that very early on also we've realized you know you cannot win by saying I have the best MSM I mean it it it's not going to it's not going to work like you just if

you have something it's it's we call it St table Stakes just like publish it let others like you know let the research Community explore it improve it let's let's go as a as a community uh so that that was like key for us as we grew and growing I'm not sure what's going to happen in the future once we start to see this like uh modes and how it will impact decentralization I think you are making a very very important Point here that we need to understand as as as an ecosystem I like the way you put that to the Moes and I and I almost wonder if it's because it's the hardware which is these longer time frames way more

investment kind of colliding with the open source decentralized nature of what we work on it's kind of interesting yeah I think so I also want to go back you all to what you just said about the privacy of what's being proven kind of like I think that's what you were hinting at where if you Outsource aing action to someone else you definitely can't do that if you're trying to keep that thing secret because they then would have the information and then prove on it and I think it works well when it's like a scaling when it's more about verification or validation of something but on the Privacy front I mean you can't share it unless like you mentioned you've already like wrapped it

in an FHA or something like that it has to already yeah some other cryptography must have happened on it yeah I I I think perhaps it's even more than privacy because I I can prove something entirely different to you know what you want me to prove right I mean you are right that it involves like some more advanced cryptography um but also just like when you what we see in other places you can also have something walking inside of a tea right like a trusted execution environment true and uh in fact Nvidia came up with this confidential Computing environment so in a way you can Outsource the computation while maintaining privacy of your input uh to

some kind of like uh Nvidia GPU that's is that's running in the cloud uh people are already using it although as we know from history every T um getok like yeah in a way so it's it's it's not bulletproof um but but it's another interesting direction to think if anyone is kind of like sitting there trying to come up with some kind of like proof of concept that would be very interesting to see do you see a future where it would be more client side where proving like maybe someone would use some algorithm or some library that you've created and it would be actually happening yeah on on a phone on the client like where the person with the

information could they be using more sophisticated provs you Vol myself we just were discussing like a few days ago about the use cases for client side proving and if we already like I think that we both agreed that um there's a future there like uh you mentioned privacy as as one reason there are other reasons right it's also um I can also give an example on how it solves scalability issue I mean there in gaming for example like there are good reasons to do that but are we ready uh do we need it like today we're not sure I mean we we were thinking uh perhaps you know let's head support for metal which is how um iOS or so like like Apple uh Mech has their own

GPU and so on so this is kind of like their way to program it um mostly for graphics but are we ready to also add support for that is it going to be useful I think we talked with like the jolt guys from a16z like that was like one interesting uh uh potential user for this Tech but but like the overall conclusion was we're not there yet we don't have like the Privacy thing or like in general a client at proving is is not appealing enough today but in the future it's it's definitely going to be part of like ZK landscape like no doubt about it do you think it would come from the outside or do you think it would be like apple using zkps themselves like a tested sensor signing we've been sort of talking about that on the show a little

bit I I I think that um there are two things that you usually need for client s so one of them is really you need to be extremely power efficient if you're going to run on a phone or or on a watch or on something like that and that's stuff that we've been discussing but I think for much of the client side applications um you also need some like sort of third party da database right like maybe you need to prove your identity but proving your identity uh is actually proving membership in some government database right so I I think that for many of the client side

applications you also need that infrastructure otherwise that there's nothing to prove well we were I mean in our case we're talking about more like the here it would be almost you know you take a photo you want to prove that it's your photo from your phone at a certain moment this is where you need a signature at the beginning of one of these like content Journeys we were talking about this a little bit last year with the attested sensor stuff but um that's where I think maybe a client side ZK proving something or other would be really cool cuz I know what you mean where it's like if you want like a passport or an identity you're going to still need to figure out like how that's created in the first place yeah we took it to uh last year we had this um

conversation about iot right so imagine this like sensors like camera sensors like uh doing some security stuff but again not just like for not necessarily atation I think was the the main use case here was mostly about uh saving storage right so instead of sending a lot of like these cameras are doing machine learning right at the edge they need to detect some kind of like anomalies but the way to do it they send a lot of like noise like to some storage like at the back office so instead of sending all of this like garbage you can just send a proof that over the last 10 hours there was no event right so you save a ton of storage uh making the

sensors like cheaper and so on we called it insights so you actually send insights rather than than the whole video all the info H yeah that's cool it's a type of like safe compression right because why would you believe me if I told you that yeah no one actually entered this gate in the last 10 hours right you believe a video typically because maybe that's hard to fake may maybe it's not so hard to fake anymore that's why you need the attested sensors and signatures this is what we're thinking yeah no that's fascinating though like so in your case though when you're even brainstorming about these client side use cases do you imagine the work you do today kind of ending up in

that realm is that sort of a goal as well like Beyond just the Asic goal is it like that people could actually use what you're building on that side um yeah I mean both on the Asic and on the GPU side I would say that like it's it's so much of a goal but um we need a very clear use case that was a that what I was hinting before right so I can imagine like a process let's say we work with a company I don't know Nintendo right so Nintendo they have like consoles um client side kind of like you know gaming devices and they have gpus and they have like full control over their stack right so you start with the GPU you take like what you have what we have with icle you kind of like Port it

to their uh software stack and and and and hardware and then it it works great and they need this kind of like price y disrupting technology to work like even like at the lower power so they say you know what let's do an Asic right so then it it's it's like this Evolution right this like uh just like what we described before is like taking step by step so then you move on and say okay let's do an Asic together and just like you know push it into millions of like Nintendo consoles that would be cool so I know we're getting close to the end of the episode but I know you also have been releasing some interesting research papers and work tell me a little bit about what yeah what's coming out right now what's the focus of the company today I'm going to I'm going to talk

about two recent papers I'm going to mention them and youal you can like jump in and and fill in the details so one paper we released very recently is kind of like applying graph Theory to this very large PS like the ZK evm PS that you see with Tao and skull and there what we witnessed is that the memory is is you know even before Compu you have this like a lot of memory that you need to uh to use and and that's becoming a button like you are stuck on just waiting for memory to be free so the naive idea was can I take this PO and just like separate it into two smaller size PS that can run independently in

parallel therefore cutting the memory requirement by half moving the problem for memory to compute and the way to do it was to kind of like saying let's look you know you need to go way to the beginning of the protocol kind of like see imagine it is after arithmetization it is kind of like table and kind of like trying to understand where you you put the knife right like where do you want to do the cut and and there's no natural place to do it but if you add some redundancy like if you pay a bit to kind of like increase the table size um then you do you are able to to kind of like cut it in in in kind of like equivalent to in the dependent

pieces or it can be more than two it's like Community detection so it it looks better on Tao than it is on skull and we try to understand also why so that's like one work and quickly the second one the second paper we released is about some checks so that's like first chapter of like a much broader bigger project that we have about SS there I have to say that you know I have to admit that when iper PL came out I was kind of like crushing to publish this blog post claiming I think the title the title was of like SS are not Hardware friendly hyper plun is not Hardware friendly something like this like you guys like you just like you know you used a bunch of cool cryptography and math but it's not going to be used in the real world and I was wrong so we now know many ways

in which in which this like SS uh and and that's like fascinating topic right in how these s checks can can be run in parallel can be run efficiently on GPU can be running like uh like a lot of algorithmic improvements to that and and there's going to be much more released by us and others much more work on that topic going to be released by us and others in the upcoming weeks and months you Val on to to add anything yes so maybe I'll I'll just add a little bit about Sam check so yeah so so the question is really how how to get Sam to be more parallel and and the the main problem is that some Che is really it's sort of like a funnel right you start off with a lot of computation that you

can do and then as you go through the rounds you have less and less computation but you cannot parallelize them because of um you actually have this pamir process um and you actually need to complete a round before you can you can generate the the pseudo Randomness for the next round so you can start off very parallel but then at some point you just get stuck so so actually sort of that's at the same time uh we've been working with Justin teller for a few months and uh and we actually took an idea from his small Fields paper that that I think he hasn't officially released it's on his website you know

some preliminary version of it which which we also have some work on but but we'll leave that um it'll come out at some point um but we took an idea where we actually uh separates when you look at a subject protocol you can actually separate the data from the challenges the challenges are those random uh is that fure mere Randomness that you're generating and um there is a cost Associated when you separate uh the data from the challenges you actually increase the amount of data that you actually need to store quite significantly uh and what we've seen is that if we use this hybrid sort of

system where we start off with the standard s protocol we can uh actually saturate the GPU but then at some point we actually move on to the the other uh uh implementation where we actually separate the data and the challenges we do increase the storage costs but now we can do a lot of computation on the data in parallel and then at the end we will have like this little tiny tail that sort of is is a little bit inevitable but but we do get like an an overall better average time so that's one work that has been partially released uh by uh ktic and suash so they

haven't um actually uh discussed the hybrid system yet and and that's coming uh up but but they have discussed the two algorithms and now we're actually starting to look at some folding schemes that uh I think are very exciting yeah where we try to actually trade off again instead of doing one big S Check we're actually looking at splitting it into a few Su checks and then we think we have a method where where we can actually uh uh fold uh everything so that the verifier you know just saes you're splitting all the beIN like keep something you're spilling all the beans

anyway it's idea I'm just kidding I'm just kidding everything is open source like even the research yeah there's a repository called Super SU Che uh the the you know the Halo 2 The Tao thing is is open source there's code it's like you can you can use you can you can you know also collaborating with with Academia on that and like uh yeah epfl is is in the loop and so on so epfl actually uh yeah I don't know we don't want to say too much because I know that they've got something cooking and uh it's actually something very interesting that that we haven't thought of um but yeah we we we'll let them uh uh push it out and uh I think we're going to somewhat be

involved with them on on maybe implementation of that so that's pretty exciting yeah I feel like when you talk about these techniques and then doing work on these techniques and then you put it in this Hardware context and actual speed and cost I had old vatana on what is it three months ago we were talking about the Bas work which they described as being created much more for like a hardware friendly environment would you put what you just described in that category as well like are you thinking about it to work better with what you know about how architecture will be working like is that what you're developing for yes but I think that actually our process is quite

fundamentally different okay um I think that we actually try to take existing protocols so to speak and then we say okay what is wrong with them right or not what is wrong with them but like if I just naively put some Che on a GPU it it just doesn't work well okay so then what do I need to do right so we sort of try to we try to evolve right we we we discussed it in the beginning so so we'll see the problem we'll try and find a solution for it and maybe eventually you know we'll get a new hyra for it and then we'll do we'll keep on doing these steps so I I don't think we don't try to

invent something entirely new but eventually yeah like the algorithmic work that we do uh attempt to make the the protocol more Hardware friendly to to like at the moment to this given Hardware to the GPU you can you can imagine how we're going to do the same for binus right so there's going to be like a software maybe even hardware for binus we've done it for Hardware like you know we've improved other Hardware implementation so we just going to wait for applications to come up and and understand how we can like evolve uh and and get the system to become like better performant doesn't matter for us like if it's like beus or other I don't know

polom commitment scheme Po and so on yeah cool I can give another example of the work that we're actually doing on small Fields so we've taken I Justin tell did did this uh incredible job on on small fields and small fields is is this idea of um of basically like I I want to work with with like a smaller characteristic but then how do I get the security so I actually I do the sum check on on a small characteristic field but then or or I I actually put my evaluations uh on a hyper Cube that is in a small field but then I sample somewhere very far out of that uh field

I I sample like the sun protocol samples in some extension field and that that's how I get the security and he's got this algorithm where he actually uh splits up the challenges or the data and the challenges and what we've noticed is that that split creates a lot of of data and a lot of data that needs to be stored uh for very long before you start creating the challenges and by using I'm actually not going to Spill the Beans this time by using these very uh uh basic arithmetic uh techniques very well-known old stuff uh we've been able to actually shrink this storage to very very small

like like uh something like maybe uh factors of like uh shrink it by like n over log n or something like that right go go from like n Square to like and log in it actually makes it much more usable for GPU for example cuz GPU is a great device but it is very very much storage limited very nice well I want to say thank you to both of you for coming on the show and sharing with us sort of the origin story of ingonyama uh a little bit about your backgrounds how it fits into this larger ecosystem and some of the trends and then yeah thanks for sharing a little bit about the work you've been doing

recently thank you for having us it's been a pleasure talking to Anna as usual cool definitely thank you thank you Anna and I want to say thank you to the podcast team Rachel Henrik and Tanya and to our listeners thanks for [Music] listening