Episode 398 - Verifiable Databases with Shyam & Emanuele from Provably

Transcript

Welcome to Zero Knowledge. I'm your host, Anna Rose. In this podcast, we will be exploring the latest in zero knowledge research and the decentralized web, as well as new paradigms that promise to change the way we interact and transact online. This week, Nico and I are joined by Sham and Emanuel, co-founders of Provably. We explore the concept of verifiable databases. That is the tools needed to prove something about an underlying database without exposing the underlying data. The conversation covers how it works under the hood, as well as

realworld use cases where proving the correctness of SQL databases unlocks new possibilities. We then look ahead at emerging applications such as in multi-agent AI systems and in onchain data markets. Before we kick off, three quick points. First, ZK Summit is coming back with its 14th edition. Applications to attend are still open. It will be on May 7th in Rome. Links are in the show notes and hope to see you there. Second, ZK Mesh Plus, our paid subscription tier is rolling. We've released a few bonus clips over there as well as some of our exclusive research reports. If you want to get access and support the show, please join the paid tier. Third, if

you're looking for other ways to support the show, we have donation accounts visible in the footer of our website. Support definitely goes a long way. So, if you decide to support us, thank you in advance. That's all for now. Links are in the show notes. Now, here is our episode on verifiable databases with Sham and Emanuel. Today, Nico and I are here with Sham and Emanuel from Provably. Welcome to the show. >> Thanks for having us. >> Hey, thank you for having us. Good to see you.

>> Hi, Nico. How are you doing? >> I'm good. How are you doing? >> Good. This is the first ever recording after your six-part series on lean Ethereum. >> It is. Yeah, we're back to regular scheduling. >> This this episode was a suggestion of yours. Uh the team at provably are building something called verifiable databases. Before we kick it off, I actually wanted to ask you something like have we seen this before in any of our interviews? >> Is we mentioned it in an episode with Patrouch. >> Ah >> and I think in that episode I said oh there's this team provably they're building also verifiable databases and I've been in touch with the provably guys for a while like I've been advising them for a bit. So this also acts as a disclaimer in a way. So that's why I thought it would be super nice to have

them on the show as well. >> Very cool. So yeah, welcome Sham and Emanuela. Let's learn a little bit about both of you and potentially the founding or yeah, sort of how you got to work on Provably. Uh Sham, let's start with you. >> Yeah, my story is I was at IBM for 15 years in in the UK building non-crypto stuff just working on physical products from Formula 1 cars, submarines, airplanes, etc. Bringing physical things to life basically. At the time, nothing was connected. They were just simple standalone entities. But in the last five years of my time there, I started to work on data ecosystems and particularly I was chief architect of a big electric vehicle program that was also becoming semi-autonomous. That's

when I met Emanuel. Also the interesting thing that was happening there was as these devices were becoming connected, there was this whole question of what type of data ecosystem would they exist in in the future and future specifically like going out to 2030. Now if you think about data ecosystems and if you take a normal brand they would put themselves at the center right so they will write I don't know Pepsi Coca-Cola Tesla we're in the center and everyone is all around us and that that's how it was that we would be in the middle and all our systems that we make all our IoT devices that we'd make will be able to talk to each other but what they realized on the research side and then also then

communicated to myself on the IBM side was Actually the future is going to be that these unknown autonomous systems are going to coexist with other autonomous systems and they don't know anything about each other. You know, we can start off in a world where the brands know each other, but over time you actually want something where you have like distributed integrity. And that was kind of like the background of how can we prepare for a world like that. And I reached out to Manuel who was at IBM research and then we've uh been collaborating ever since and provably is more recent, but that was the kind of the backstory of how we met.

>> Nice. Did you leave the company or are you still connected to it? No, I I left IBM. We both did work for a couple of other startups in in in between between IBM and and provably it wasn't like the immediate idea, but what became apparent was that it reached a point where we had to go out and build it uh build this thing of verifiable data, verifiable data exchange, verifiable data collaboration and yeah, we did a few other things and then we thought, okay, this is now time to put it all together and that's what we do.

>> Cool. Emanuel, tell us a little bit about you. >> Well, I I am a much more a technologist as a background. So, I did the usual path, meaning went to university, did a PhD, work for a couple of industrial R&Ds. >> Did you do it in CS or in math or something? >> No, maths. I'm a ma mathematician by training. But uh curiously enough my math background the beginning was uh what is called the functional analysis and the partial differential equations. >> Oh >> and that's why I got into IBM research actually because basically we were putting together supercomputers with big system of equations to predict the

weather or you know very complex phenomena. It came in 2016 and I just stumbled upon Bitcoin and the first Satoshi paper and cryptography was always there as a side hobby etc. And so I got into it. I got permission to start a group metam. We started to talk about the ecosystem. At the time we were doing Hyperledger Fabric. I don't know if you guys >> Yeah. So I was part of that team. >> Is this era 2017 2018?

>> That is correct. Yes, it was very hot at the time. >> I remember. >> So, I met Shyam. We collaborated a bit. We had clients together, etc. We start to build uh I think I built one of the first uh peer-to-peer trading systems using blockchain for IBM at the time, not not in the not not globally. >> We actually called it Agora at the time. >> Okay. And then Shy and myself, we got offers from another startup, moved there, then tried something else together as well. Some of those experience were a bit traumatic.

>> Oh, startup life. >> Startup life. Yes. And then uh it was a sort of coincidence basically in around 2023. Basically we always wanted to do verifiable databases etc. And at the time we stumbled upon a blog by Mato Campanelli. It was called tabloid >> and basically he started to describe how to basically build the verification and the proving around the tabular structures and certain type of protocols actually had become much more robust as well and much more performant and so we thought it's the time it's the

right time and we launched probably and we were not alone so we we were we work together with Alberto Simon and others uh that have been our co-founders since and we basically matched together different set of skills around the databases cryptography vision with sham etc etc. So we came to exist around April 2024. >> Cool. >> So in um Paris blockchain week Anna you were there.

>> Uh you mean like ECC? >> ECC. Yeah. Yeah. So, so >> there is a Paris blockchain week or there was I don't know if it's still around but >> and then it was the modular summit. It was modular summit. Yeah. Yeah. So, I I went there. That was when this whole new chapter was beginning. I was there. I saw Nico and then I just kind of mentioned to him we didn't know each other but it was like just we just have to bump into each other and I said we're thinking about this thing and the first version of what we worked on for the for one year uh till the end of 2024 was verifiable analytics under uh and how do you do you know queries on private data and prove that they performed correctly and completely. Now we've gone on and we can go on to talk about what we're doing

now. But that was like the first version of provably and exploring if there was an interest out there and should we start building it and I think at the end of that conference I met a lot of people who said this is a good idea you should do it and and then yeah we started. Yeah. >> This was the modular summit 2023. >> Yes. In in Paris. Yeah. >> Cool. Okay. So I want to obviously dig into the verifiable provable databases but what happened to the ZK for analytics? Why did you move away from that? We didn't really move away from that. The first iteration of the protocol that we call verifiable analytic used bulletproof.

>> Okay. >> For example, and uh there were clear limitations like we had issues with proof sizes, we had issues with performance also. It has to be noticed when when we talk about you know this first phase that we call verifiable analytics, it was also let's call it a commercial use case phase. So we were looking uh let's say for uh I mean very specific type of web two customers all centered around uh basically business intelligence and etc etc and it was a bit hard it was also a bit hard because the performances of the system at the time were not really matching what

clients wanted. >> So the system was working and etc. But uh like any first system it was a bit clunky, not properly performant etc. And there were also parts that had not been let's call it created yet. Okay. So for example when you deal with SQL databases and you have to perform linear operations it was a little bit tough to do that with the bullet proof and and those kind of proof. So in the meantime again compani published the linear vector commitment maps and that was like a kind of a moment because again that improved performance security and etc.

And in fact Mateo is he's not officially a part of probably etc. He has been with us I mean you you can see him in the in the e prints the papers and etc. He's uh he has he's he has his fingers in the protocol etc. So basically we call that a verifiable analytics phase because of a combination of use cases and the protocol that we built and uh we moved away from there and we got much nearer to the SQL algebra if you want to call it like that. See >> from a computer science point of view basically we got as a protocol very near to what they are called the data

structures that you use to represent data and SQL queries. >> One of my first questions on this topic is just like what are verifiable databases because I sort of when I heard it I thought like does this replace something like SQL? Does it take the same role in like building an application or are you talking about like database at a different part of the stack? These are the things that were on my mind. So maybe just kind of going from what you just described define what a verifiable database is for us.

>> The answer is no. We don't replace SQL. >> Okay. >> Okay. SQL as an engine for a database remains the same. So what is a verifiable database? A verifiable database is a database whereas the data owner has the feature of keeping his data confidential. So he does not exposed the data to a user that needs to do SQL queries. >> Mhm. >> And a user that does SQL queries basically gets back the result of the query with a proof. a proof that says the query was done on the data inside that DB that uh were supposed to be used

as input for that query. So there is no cheating on that side and also the result of the query is correct. >> So the function this select aggregation filter etc is has been computed correctly. So you're not rebuilding the concept of databases and you're not like hosting them and you're not like like I guess when I heard verifiable database I sort of thought of like a full kind of you would replace just any database tooling and and now it would be verifiable but instead what you're saying is this is like an add-on to existing >> it's a middleware >> or middleware. Okay.

>> Exactly. So the database let's say you have an existing database you can use provably as a middleware to connect it to that database commit that data and then all the queries that you run on it from there on you can prove its correctness and completeness and the big advantage here compared to let's say previous attempts is that there's no data migration >> if you have a lot of data already or you have some existing solution it's just APIs and then in minutes you're you're good to go >> and then on the SQL side the users just or the app on the other side is just running SQL queries, plain SQL queries.

So there's nothing for them to know other than the result will now come with a proof which proves the integrity of that data. >> You mentioned sort of in your history that you were kind of working on blockchain like this verifiable part is there some immutable part of this or can this be used with like all types of databases like basically does this still live in the blockchain world or not? >> No. So we need a rel relational databases. Okay. So you need a row columns schema. Okay. So typically the transactional nature of the of a blockchain is not set up in that way. So it's transactional transaction blocks etc. So we don't apply ourself directly to a to a blockchain for example. But we

work really well for example with indexes. when a blockchain is indexed and you index that in a DB where you can also put other metadata etc etc and you make it relational that is a very good application for us in fact we have been discussing that with a couple of blockchain projects and the wallets as well those are some of our potential customers now when a blockchain becomes important as tool in our stock is that because a data owner or probably itself the system eventually there is the need again depending on the use case to post these proofs the commitment of the SQL

query because you want to verify them and you want to verify them also at different times not only immediately etc. So a blockchain becomes a natural storage place where the pro value proofs are posted. In fact on this side for example we have been working a lot with midnight the privacy blockchain out of Cardano. We are one in their ecosystem partner and uh the DAP that we are building for for them for us that is going to be released when the mainet comes out actually we'll have this feature basically connects probably to midnight and the proofs then are posted on the midnight ledger.

>> I guess you were saying also you get a proof that my query is run on the database. So I guess I need some kind of notion of a commitment to the database to exist somewhere and I guess a blockchain is also a good place to go and put that or update that maybe. >> Absolutely. The way in which we do now for say non blockchain clients is that basically we use for example transparent logs databases or a database where the data owner access the proofs and then distributes them around. But eventually yes as an immutable place is actually the desirable space where where you want to post these artifacts.

>> So in our like blockchainy world we're used to storing data and committing to it in Merkel trees and then when someone wants access to that data we say like hey here's the opening here's the proof that it was in my Merkel tree you know you're happy with that. I think some teams have been also doing some processing on top of these things and that's you know the co-processors of a few years ago. Is that sort of the same thing or completely different? >> No, it's it's different in in this sense now. The notion of verifiable databases we did not invent it. Okay. It already existed. There were already few prototypes products around. One of them actually uses this approach that you mentioned. So the Merkel tree approach.

>> Mhm. >> And it works. I mean nothing and and it's abducted. The issue becomes when you try to put a lot of data in the Merkel tree and that's how we were born. So as as verifiable DBs we want to enable big data, a lot of data, a lot of rows, columns etc etc. Now the way in which a Merkel database tree or basically a verifiable DB based on Merkel tree works is that every time that you run a query the Miracle tree needs to verify all these data paths etc etc and this is extremely costly when when the data grow exponentially we

don't do that we go through polomial commitments that's that is what I was mentioning when I was saying we are much more near the algebra of the data. M >> we are much more near the algebra of the query and that is basically SQL basic SQL are algebraic queries >> and they are very prone to be represented by polomials and vector commitments and so we do that and in this way our dependencies are on the complexity on the query the data themselves being 1 mgabyte 1 terabyte one pabyte for example that dimension

does not really impact probably >> it's the complexity of the quer that does so with Merkel DBS you know you are dependent on the database size with us you are not >> it's more about the complexity of the query is there any sort of size blow up for you guys in that case >> not a size blows blow up but more like for example it's very hard for us in fairness to represent what they are called recursive queries. >> Ah, >> so there is a point in which SQL from becoming basic relational SQL can become recursive and it becomes Turing complete. So let's take for example an

exponential. For us, it's hard to process an exponential because it becomes an SQL query that it's extremely complex, recursive and etc. for a zero knowledge VM instead. Mhm. >> You know, because how they are done representing on exponential is somehow easier. So from a certain point of view, we are a very specialized machine that performs really well on basic relational SQL. >> Mhm.

>> And why did we do that? Because basically that's say 80% of the workload that exist in verifiable operations and use cases. >> Okay. So in that in that example, are ZKVMs using basic relational databases or Yeah, maybe I'm mixing it up. >> No. So a ZKVM is a generalistic approach. >> Okay. >> It's a computing machine that you use to do zero knowledge to verify some type of computation and they are generalistic.

So they can compute upon performance costs any type of function. we focus on SQL. So simple algebra. Okay. So when it comes to simple algebra and simple SQL relationships, a Z AKVM is an overkill. >> Okay, I see. So the ZKVM is overkill in this case. You and and so but are you kind of then doing a similar activity to what a ZKVM would be doing? >> No, because we are very specialistic. >> But when you say it's overkill, it almost sounds like is it just kind of you're saying it's more generalistic?

Are you doing sort of a narrow version of what a ZKVM could do >> somehow? >> Okay. >> Meaning we we focus again on linear algebra while a ZKVM can do nonlinear stuff. >> Okay. But when you try to do the linear algebra, our performance and our simplicity is much better. >> Okay, >> maybe we should talk about performance part because there there are limitations on how. So if you take ZKVM the size of the data and queries matter in terms of there's a threshold on how big the date in the queries and the complexity of the query can be. So an example could be if you ran an SQL query on a million rows in a database in a

ZKVM, there's a certain user experience, you got to write that query, you got to turn it into a circuit or you got to compile it and and then when you run that query, there's a certain performance. Whilst if you do it using this approach on a verifiable databases, there's nothing to compile. You just write your SQL query and it's it's ready to go and compute the proof and give you the answer >> straight away. So it's great for like ad hoc queries because it's already there and you can produce proofs faster on on bigger data. >> Since we are comparing to ZKVMs, the other end of the spectrum is a circuit.

You know that's how people would go and write specialized applications. They go and they write a circuit. Is that what you guys are doing? Like you have a special SQL circuit or completely different? >> No, no, no, no. >> Because we use knows the answer here. >> It's an interesting answer. That's why I want to pull it out from them. >> No, no, no. Because otherwise you would have to write a circuit for every data >> a circuit for every query etc etc etc. We didn't want to do that. So we wrote a protocol that is non-generalist. It's not a general framework. We don't we do not use circuits. We use a kagg that is let's call it one of the Swiss knives of uh you know polyonomial commitment

cryptography vector commitments and linear vector commitment maps and zero testing for proving linear relationships on the data. >> Yeah for our resident cryptographers we have kg but we don't have a snark for all circuits like you'd have with plunk. We this is actually like a polinomial IOP specifically for SQL. Is that correct? >> That is correct. Yes. >> I was wondering if you you could share a bit more on how this would actually be used in practice and a bit of a user flow like you talked about it being used to prove things about a database but like who is doing that when are they doing that? Why are they doing that?

>> So very generally speaking let's say and maybe I can start off with the blockchain world and then talk about something that we're doing already outside of a blockchain. But typical example in a blockchain world is let's say there's some private transactions and that data is primarily onchain and then the user knows about it because it's in their wallet. Now it's very hard for that user to prove facts about that data to someone else externally. So let's say I'm uh I'm an institution. I've done a lot of transactions on chain in some private database. Now I need to prove that yesterday I earned x amount of income and these were my expenses and this is my profit. A simple thing like that, if you had a verifiable database, if you had that all your private transactions verifiably indexed into

your local state into your wallet, now you just write one line of SQL and you can prove that this was my business's total turnover yesterday, total income, total expenses, all of this type of stuff. >> So that's an example where I think the way we are seeing the world is we are creating an awful lot of verifiable data when those transactions happen. It's just that afterwards we don't use it because we usually have to index the data out and then you create your dashboards and you prove performance and all these things when actually what a verifiable database allows you to do is kind of like extend that chain of trust.

>> Now everything else that you want to prove about your onchain data for example >> you can just do that using SQL queries. So that's like one one example in in in in the blockchain world and offchain there is beginning to be a need in in let's say you can call it web two or emerging autonomous systems where traditional databases want to prove that the data that they have is authentic and anything that they say about their data is correct and complete. So this is the other space that we're focused on and very excited about uh because this is like everything else.

>> Yeah. Yeah. I mean, in this case, I mean, I'm getting slightly echoes of like the ZKID state where it's like you have the ability to prove certain things without revealing everything. If you're dealing with like a private database, would you be using these kind of proofs of querying in order to sort of protect other parts of the database like you could keep parts of it private because you just need to prove these little bits of it or do you think of this as you would do you would actually verify everything for the dashboard or for like for another use case? the the umbrella is kind of is distributed systems integrity, meaning even if the data was public, you could prove that I'm answering to you correctly. Um, so let's say there's an awful lot of

transactions. There's like a million rows. Okay. >> Now, the user journey of someone independently downloading that million rows and then computing it versus me just giving you a proof of this is the sum of all of those things on a million rows. That's an example where just giving a proof is faster than somebody else downloading it and and doing it. So there are those use cases as well where it's about public data. Uh an example that we've had a lot of discussions about is and it's usually people that are operating blockchain nodes is how can they know as fast as possible the state of a blockchain node.

>> But weren't like the indexing projects supposed to help that? >> Yeah. Well, they they're kind of so if I having listened to the podcasts of the past okay >> these things have come up. I think Lrange for example is one >> or longer longer ago like the graph and stuff was doing indexing. >> Yes. So I think there's a lot a lot of examples where there's been data integrity but not necessarily query integrity. >> Okay. >> So you can prove that you've indexed everything over but now let's say I need to report on that index. How do how do I prove to you I didn't leave certain data out?

>> All right. And how do I prove to you that I did the maths right? That here's my sum or here's my count or here's my average, here's the minimum, maximum, all of these things. That's where so I think a there's a lot of work in the past and around data integrity, but the query integrity is what the user wants in the end. So if I'm reporting to you my business's total earnings last month out of a given wallet for example, then that's like, you know, showing all my income minus my expenses, for example. >> Yeah. Yeah. >> Right. So we could have like a block explorer that also tells me I ran your query correctly and this is the answer.

>> Exactly. So that would be a great bolt on to a public index. And then you could also imagine doing this on a now that you know 2026 is the year of privacy you can imagine that >> a user may want to prove facts about their private state and then for them if you've got a verifiable index now you can >> hide certain things you you may not want to show who your customers are >> your source of income but you can prove the state of that that private database and this really so examples where we're hearing the need like the other side the demand side is I think we're starting to see it around let's say in vaults where there is a we're slowly going towards like onchain credit cards.

>> Now, if you want to do lending on chain and you don't want the user to overcolateralize and bring their assets, now you need to know something about them, >> right? So, they're not going to bring assets. Now, you're going to give them a loan based on some information you have about them. So, they need to prove to you their average income, their savings, right? And then this t takes us towards basically you can do credit scoring >> and all of this credit scoring is based on immutable data that they're slowly doing more and more via stable coins on chain. >> It does have to be on chain though for this to work it. It's not like you're going sort of to like a a bank statement online or something.

>> You can I think the opportunity like from a goto market perspective is in the trady side this infrastructure exists with all its problems. that kind of exists. But as stable coin transactions kind of enter, there's this convergence of you're kind of building verifiable data and verifiable history. >> Wow. >> That's the bit that is open to be exploited. And if you listen to more for example, they they talk about we don't yet have private va private data oracles. >> So if I I'm a single user and I want to borrow on chain, how do I prove that to you, you know, to get better rates? And that's an example of a private oracle where this this would fit really really well. I think we're still like a year early from that type of use case because

now the vaults are just focused on onboarding and growing and that side. But I think if you fast forward out, they will have the money and then you'll have all these onchain transactions that are in some private wallet >> and then you can see the two kind of converging. By the way, linking to the Tradfi thing, there is a world where banks will sign our bank statements or we can use like TLS attestations to prove statements about a bank bank statements. We build up a private index at home and then you can go and issue proofs about your private index of like okay I've uh statements for the past 5 years with no interruption. They're all in my name. All these good events happened. None of these bad events happened. Like you can do these kinds of fruits which would be quite hard without an index. like without an index to go

and prove that you did not I don't know spend more than x amount of dollars on a single thing is hard to do like how do I know that you use all your data to prove that to me so in in the episode you had with Shan Bowie >> he spoke about this layer in between the chain and and the wallet >> right because at the moment Zcash is tied to specific wallets >> right on on the rich client side and we don't know what the answer is there but they're working on something and I'm guessing that there's this intermediate layer where there is a verifiable index on which different wallets can tap into and from which Zcash transactions can be used in DeFi in this way where I can prove things about private state and the other thing that's very interesting that this is like just like loose talk almost

play talk you you can call it is we are kind of observing that in the last 6 months you've seen with open claw and stuff that it seems that we are all going to get this one extra device it's going to be always on it's going to have GPUs It's it's going to be always connected where you could run your proofs which in the case of open claw is like MacBook mini uh Mac. >> Yeah. So that's we haven't done anything there but it's just interesting that that's like this new layer that is not on cloud. It's actually with the user >> local.

>> Yeah. It's local and it's it's always on always connected where you can run workloads. That's like very interesting from a ZK perspective because all the constraints of client side proving kind of is less of a pressure now because you have a sovereign device that has your verifiable index data that you can pro prove things. >> I mean there's a lot of different use cases that you just shared here. I'm kind of still trying to figure out like maybe with one of them. Can you tell me like where are the proofs generated and like who's generating them and where do they go? Would it be like an application would offer this like would a wallet offer this to its users that they could back it up or create this verified version or is it automatically happening

and where is it living and who who's doing the proofs? So at at the moment with one of our clients for example the proofs are done by us. Basically you have a proab you have a proab server that uh runs the prover and then stores the proofs in a transparent logs database. >> Okay. And you're seeing it too. So there's no privacy there I guess if you're doing it. >> Yeah. Yeah. because you know we we manage the system basically that that is enough for for our current client and for uh the requirements.

>> Mhm. Eventually you want the data owner itself to run a prover or probably server middleware >> and uh then ship or post the proofs somewhere that is easily accessible by the the user and where the user then can just invoke the verifier and gets a tick the box. >> Yeah. Get some like certification that X has been proof. at at the moment basically our first product that we call probably does all of this. So you you log in with a user account, you can set up your DB, you share uh the schema of the DB, you

either put your data in there or you attach the the API of your database, etc. etc. >> But that is our product. I mean that's probably then eventually we can offer the middleware as a service or in a C2 etc etc. Hm. If this was to go then like onto the client's device, onto the the user's device, then I'm assuming this has to be really efficient. >> Yeah. What kind of proof size, what kind of machinery would you even need to do this?

>> So, our proof sizes are very small. >> Mhm. >> We said that before. So, they are independent on the data size. >> Okay. >> So, you can have a 1 TBTE database or a 10 kilobyte one etc etc. It doesn't matter. What matter is the complexity of the query. >> Okay. >> And typically for basic relational SQL, our proof size never goes higher than 1 kilobyte, >> 1.5. And that's that's that's really small.

>> Yeah. >> Also, our verifier time and verifier complexity and requirement for running the verify, it's it's extremely small. So you could run it on a mobile phone or whatever even on an IoT device for example and we are really talking to verify a proof just one two three max 10 milliseconds >> the prover is uh a little bit bigger and uh again so because the prover actually has to check data etc etc so that's better that it's run for example by the data owner in let's call it a more complex machine but we are not talking again about GPU farms or computer

cluster etc. A laptop is enough. So for example you know we we get really amazing times just by running on some small virtual machines that we have on Google cloud. >> So we we we never really had the need to optimize with the GPUs or computer clusters to now. >> Just to be clear does the prover depend on the table size? I'm guessing the pro at least needs to run the query. >> The answer is no directly. Okay. It doesn't because it depends on the data that are fetched from the query.

>> Mhm. >> So it's not the entire data set per se that influences the prover. It's the data that are included inside the query for computing the the query. So again it's like the proof size. So there is a dependency linear on the complexity of the query. So if you call the complexity of the query Q. So the prover complexity analysis it's an O of Q. Okay. While remember before we talked about a Merkel DBs for a Merkel DB it's an O of N. So the number of uh data that are in the DB.

>> Okay. So Q is smaller than N right? like >> again yes unless you know you want to compute I don't know one of the >> something on the whole database >> well not only that I mean maybe you are crazy and you decide to compute with SQL I don't know fluid dynamic equation on the entire Pacific I I I don't know that would put us in trouble yes >> okay so in a way that's a bit reminiscent to lookup arguments where we got to a point where the prover does a one-time sort of setup >> that depends on the time of the table and then lookups only dependent on the size of the thing I'm looking up.

>> Yes, >> that is correct. In fact, the lookup is actually one of the technologies that we look into eventually, how do you say having further improvements in the protocol. There has been in the last two years some very nice results on at the intersection between lookup and zks etc. But at the moment we are pretty good. So this is really call it research directions rather than production directions. >> Mhm. So we've mentioned lookups. I'm curious to pick this thing apart a bit.

We've already mentioned like okay you are doing a succinct proof but not for all circuits. You're doing a succinct proof specifically for queries. What does that look like? Do we have the same insights as we'd have in plunk for example? No. because uh we commit the structures of the rows and the columns via polomial and vector commitments and then we run the predicates aggregators with linear vector commitment maps and especially using what is called inner product.

>> Mhm. So you know we we basically can express the SQL predicate queries etc by using what is called inner product arguments >> uh so we don't need the algebraic machinery that is uh that is in plunk uh etc etc. >> Is it a totally custom system then because you said before you had tried bulletproofs I'm assuming you moved away from that. Is it based on anything that we're familiar with?

>> Well, it's based on KZG. It's it has been around for a long time. Linear vector commitment maps were published as well by company a couple of years ago. They also been around as open-source codebase uh for uh for some time. So, we did not invent new primitives. >> Okay. We have just composed well-known stuff that is pretty secure. It's also one of the advantages that we have or features is that our protocol is not overly complicated.

>> It's free primitives composed in different ways. So it's also I know Nikico you are a ZK security auditor. So we are quite easy to be audited from this point of view. our code base is not uh you know the code base of Jolt for example. Okay. >> But is it like cuz I'm trying to figure out is this like a ZK system? Are you just using parts and pieces and techniques that are often used in ZK for something else? >> No, this is quite interesting. Okay. Technically, if you ask you know an hardcore ZK cryptographer, we are not a ZK.

>> Okay. >> Okay. We're not a ZK simply because when you do a query and you know the query even if it is verifiable on private data etc you can build some inference some side channeling etc etc. So we are not we are not a pure ZK. >> So you're saying you're not ZK in the privacy sense. >> Correct. >> You're not zero knowledge. Maybe the question I'm actually asking is like is it a snark? >> Oh that's a great question actually. >> No >> it's not a snark either. Okay.

>> It's not a snap. A great question because in a way you are doing succinct proofs, right? It is succinct. >> They are succinct. Yes. The proof. >> It is non interactive. >> It is non interactive. Yes. >> And it is an argument, right? >> Oh no. >> It's of a snark but not the way we've defined them. Oh my god. >> But no, it's the problem is we've been saying the word snark specifically for like proofs for circuits. >> Okay. >> And in that sense what you guys are doing is not that kind of snark. >> It's a snark for a custom thing. And right now we're we've forgotten that snarks could be for something other than circuits.

>> And so we don't have a word for it yet. >> Wild. >> It is. Yeah. Like I think it would be very interesting like how do we call these systems that are like custom snarks? >> That's correct. Well, you know, we don't typically care too much about the labeling put in this way. >> Oh, but you should. The labels matter. It's it's the marketing of this scene, I think, is is the naming of the protocol in a way. >> So, actually, the e-print that you guys have put out is called QEDB. Is that the specific construction? Is that a a set of things? We've been talking about verifiable databases. I see you guys abbreviate it as VDB. Like, what are all these things? And are these potential names for what this thing is?

>> We haven't named the baby and whether this is a specific direction for the future, >> right? So this is kind of actually what we're all exploring that >> is this a new direction for for ZK and and it seems that on on one side you have ZKVMs that can prove the integrity of or prove that an application ran correctly and properly right and it's creating transactions and the and we are coming on the other side to say now you can do queries on all those transactions and you can prove the correctness on that. So in a in a in a very simple sense you have computer that generates data and now or what we're doing is the other side that all this data that you have created you can ask questions on it and you can prove it correctly >> we should take the time to name this

baby and say we are solving a big useful part because if you think about you know leaving ZK for a second if you look inside the EVM world doing data queries on chain is very problematic like you have to write a lot of code you have to go fetch the code you have put it all into a certain structure and then you have to compute it. >> You have to run it. >> Yeah. And then you have to run it. And this is like there's a lot of complexity and overhead there and performance issues. We've just like taken all of that away >> and therefore I think it is kind of a big deal and we we should take the time to name it >> uh to to name it. And and I think the the future requirements why this isn't like a one-off is we're just going to

get more and more data. This just like that's this is obviously an inevitability more and more verifiable data because just more and more things are going to be on chain and the need for going from just doing payments and transactions to actually making sense of the data will only grow. That's never it's not like going to die away. It's actually just starting. You know the last decade was we can do private payments, private transactions and now people's entire financial lives are in there and they >> Yeah. So like as Nico gave the example that if I've got 5 years of some transactions, how do I prove that?

>> Wow. Yeah. I mean this this does bring up the concept of ZK. I mean you you actually haven't said ZK databases. You've said provably verified like you are using the succinct words more the succinctness words the fact that you're proving something about it instead of the private part. Is there going to be a private part though? doesn't make sense for what you're doing to have like you're running things on privacy on private databases potentially but you yourselves don't need to be private do you I mean if you're client side it's with the client but it's not like private per se >> actually maybe Anna I can jump in quickly in that there is privacy if I understand correctly on the original commitment like when you commit to your database that is private the commitment

doesn't reveal anything now when you allow people to do queries on your database you're already opening yourself up to like okay come and learn things about my data >> if and I think you were talking about that earlier Emanuel when you said we're not ZK per se is that correct okay do you have control on what people can ask about your data and what they cannot ask >> absolutely this is a you know this is a normal uh database management feature >> because we did not create a new database >> but uh you as a user come in with your DB your DB structure your DB management systems. You are able as a data owner to continue using all the query control

systems that you always had. So basically you are able to say you know I don't want you to for example do dummy brute force queries in which you just do select each element for every row and column etc. Exactly. So you it's up to you it's up to the data owner. the data owner decides the let's call them privacy policies >> dependent on the queries. >> So you could I guess mix this in with differential privacy and make sure that the queries that are made don't reveal anything about the underlying data if you really wanted full on privacy. Okay.

>> Absolutely. A very interesting system that we built as well is we have associated this with what is called a privacy budget >> and we we have deployed this for a client. So the client basically has a set of databases distributed and is trying to trade the data or trade information inferences on the data. And so the privacy budget basically computes how much value is left >> in the database or in the data source after it has been queried few times or etc etc. And that influences then the price

>> of the data set or the queries that are in the exchange because we have basically built a data pricing exchange for this client that sits on the verifiable database infrastructure that we have built. So you can imagine that the databases start off with something like 100 initially because the no queries have been done and that's when >> potentially the price of querying that is at the highest. >> Okay. >> And as it gets closer and closer to this threshold when you cutting off access then the price of the query drops and then there's this race to go and mine as much data as you can uh by by the time you get there >> so that you can be the first to acquire

all of that data and kind of lock that data away. So yeah, this is an example of where with another client much more on the web two side, they're going to have users with a lot of data specifically, let's say for the energy market and to understand the state of the what is the available battery capacity in and around for vehicles in a given area. They could actually go run these queries. Uh they could also run queries to figure out when are those assets going to be charged, when are they going to drive and therefore that capacity won't be available. So this is an example of these distributed IoT systems where verifiable data directly from that IoT device could be super useful with with privacy.

>> I have to ask one big question which might you know start a fight but it has to I think it has to be addressed because it is one of the elephants in the room is how do I know as someone making queries to your database that the data in there is not junk. In this case what we will do and we're going towards a production system in the next 18 months. This is kind of like the big exciting news that the last 18 months was proving to this web to major OEM that verifiable databases are performant enough them uh for them to think about it as something that could go into production. So we've done a lot of trials with them and now we're going as as we as we head towards the direction of production in the next 18 months which is amazing to have refiable

databases out in the wild. Then we would look at this question of is the device that is generating the data actually going to sign it and then you create this whole chain. At the moment it's it's obviously in trials is that the signing of the data isn't the complicated part. It's much more are you performance enough and what can I do with it? >> But as you get to the next phase where you have thousands or millions of these assets then yeah we we would just sign directly from from that asset that is generating the data. >> Okay. So I'd get like some kind of provenence guarantee on top of the guarantee I get for my queries.

>> Exactly. >> That's very cool. That's a beautiful world if we get there. >> You know, with a lot of the ZK systems and this type of cryptography, you know, security is a big issue. You've designed it sounds like, you know, something using lots of different techniques, not exactly in a wheelhouse of like a proven system. So how are you thinking about security stability like making sure that this is working but also making sure that it's you know not going to be broken and actually maybe even a a follow-up to that is what happens if it is broken like what is the worst that can happen >> well nobody has broken k it's based on that so you are using something proven fair >> yeah so I'm kind of confident that given

the algebra of kg >> but wait weren't there these like kg like vulnerabilities or was it like in an implementation? >> No, there's implementations and there's the quantum threats but KCG otherwise is sound as as it gets the the quantum because it's based on pairings etc. But we are going to worry about the quantum threats when quantum computers come and I'm I have my own opinions about that. So I don't >> By the way, Nico, we so have to do some episodes on the on the skepticism because I feel like we've mostly talked like >> in a very like quantum is coming, we better get ready way. But I, you know, there are voices that don't agree with this. So

>> yeah, and there was a very interesting post in LinkedIn by the CTO of a very well-known quantum computing company recently that was very revealing >> on that. But we I assume we don't name names. Okay. You're on the podcast, right? But yeah, so coming back to your question, we are KG is at the core of what we do. So nobody has broken it yet. The algebra is also simple. Algebraic attacks on those type of commitments.

The the pairings protocol that is used behind is also very well known. So we are not too worried. Of course, you know, we wrote our own code, etc., etc. So there is a point in which we should go for an audit of our wouldn't that be nice of >> of our code and so that's that's probably going to come sooner I think. >> All right so we're almost at time. What can folks look out for going forward with provably? >> I think in 2024 when we first started out we didn't do that much with AI and provably is called provably.ai on on the domain name site.

>> I think that time Yeah. Yeah. So it felt inevitable that we will stumble into that world. And this year is very interesting because couple of years ago it was very much about what can one agent do. Now we're going into much more of a multi- aent framework and multi- aent setup everywhere. Whether you're in an enterprise or everywhere, it's agents talking to agents >> and here the performance there are a lot of data integrity issues and this is something that is very much on our radar and the stuff that we're doing around data markets is revealed a lot of you know green field things for us to do there and therefore I think we will we'll do stuff this year around solving data integrity between agents like handing off data from one agent to

another and allowing verifiability there at the moment verifiability in that space is able to only detect about one in three errors. So there's like a big space for >> can you increase that up that that's like a very mainstream problem. So that's like one example. >> Cool. >> From my point of view, we are doing a lot of exciting things there. So we have implemented also the first system that uses some very complex game theory techniques. It's called the minfield game theory and we have deployed it with 10,000 agents running. So we have we have seen actually some pretty amazing results that we were not expecting on how the the stability of a verifiable

system is affected by the agents etc and by this game theory engine that we are building. >> Do you have an article out on that? No, but uh I have to basically deploy it into the proof of concept for the client by the end of March and I'm working on it something like probably 10 hours per day, seven days per week. >> Oh, okay. >> Uh so once once I'm done with this, yes, Sham and myself, we have a long string of blogs and articles to write about that.

>> Well, maybe some of these will be out by the time this airs. If not, I guess folks should keep an eye out on your channels to see if they're coming out. >> Cool. Thank you, Emmanuela and Sham for joining us on the show to share with us all the work you've been doing with provably the idea of verifiable databases, kind of how they interact with existing databases, what kind of use cases they open up, and yeah, thank you so much for digging in with us. >> Thanks for having us. >> Thank you for having us. Thanks, D. It was really, really fun getting to talk more about Provably on air rather than in private as we usually do and talk about the fun things like how do we name this thing?

>> Nice. All right, I want to say thank you to the podcast team, Rachel, Henrik, Tanya, and Hector. And to our listeners, thanks for listening.