[trustless://eil] Kaan Uzdogan - Why Contract Verification Needs to Open Up

Transcript

Why contract verification needs to open up by Khan Usdogen. Okay, we can do it again. Hands together clapping. Nice job, guys. So, can you hear me? Yeah. Hello everyone. Um, my name is Khan. I am a developer at sourcifi source code verification project I'll be talking about and also a member of the argo collective a recent itum foundation spin out and I'll be talking about that also but my main focus will be about why contract verification needs to open up why the current state is suboptimal and what are we doing to change that so the other day I was looking into the ERC

1155 standard and I was I was in this open zeppelin forum And the this guy was explaining what the ERC does and thanks to thanks to them um very nice of them they have written an example contract and they deployed it on the girly test net and I wanted to see what the contract does. So I click on the link I open it and then I come across this. I mean it's funny that we I'm talking about this today in a massive CloudFlare outage but this is permanent. Uh today's outage is a temporary outage but this is permanent and the I I wasn't able to read the contract. So you might say okay Khan like this is just a test net why do you care so much? But I think this kind

of shows what could happen and I couldn't help myself wonder looking at it and tell where are these contracts like where did they go? Are they lost for good? Are they in the graveyard? And what happens when this maybe happens in the main nets and what do we do next time? >> Okay, >> but don't get me wrong, Ethereum actually has really good verification culture. So we want >> contracts to be verified. We don't interact with the contracts that are not verified. Hopefully, I hope you do that. Um, and I think in general we are doing a good job, but we can do better. Um and again like it's pretty good and we can say the Ethereum contracts are open source in general but these contracts are open source through closed source

verifiers. >> How how much does does it make sense and they're also closed they're in closed data sets. So of course you can see the individual contracts and that's the whole point right you can see the contract one two contracts but the whole data set is siloed and it's really difficult to get all of them and do anything meaningful like data analysis or any anything that requires multiple things multiple contracts but before I dive into the problems I want to give an overview of the source code verification since I assume there might be people who are not um really familiar with the process and the verification term gets tossed around a lot and it can mean different things. So in our case in this

context the source code verification means looking at the source and matching it to a byte code and the reason this exists is that Ethereum contracts are stored as bite codes on chain. So this is what a contract looks like when you look at the Ethereum chain and by looking at it you actually don't know what it does. You have no idea what this contract does. In fact, this is actually the beacon chain deposit contracts bite code. But as I said, by looking at the bite code, you have no idea what it does. So because humans can't read and write byte code, that's what machines does. We write programs in higher level programming languages such as solidity

wiper in EVM context or other languages in general. >> Okay. Then let's look at the look at a source code. So this is a source code. This is part of the source code of the beacon chain contract. And I can show you this source code, right? I say, "Hey, I wrote this contract in Solidity. There you go." But how do you know I'm not tricking you? How do you know this is actually the contract? And this is where the source code verification comes in. Source code verification is actually checking a source code to bite code matching. So let's go over an example. So let's say we have these files for our contract that we deployed, right? And when verifying, we take these files again. you come to us or another verifier, you provide us these files

and then you give us some compilation settings and we feed these into the compiler and we want to reproduce the compilation. The compilation gives us a bite code again. In the next step, we go to the chain. We say okay like this is the address of the contract that we want to verify. We call get code and then we get another bite code from from the chain. Then we check if these actually match. If we say if we say these are matching then we say okay like these source files are the source codes of this contract. Of course I'm skipping some details here but in general this is the idea. But what are the problems there? So the main one of the main problems is of course open

source as I said right now the contract verification is done on mostly closed source verifiers and that means the verification process is a black box like it's you can't really easily reproduce the verification and I don't think I need to explain more that in such a such a setting that we have that we want to open source the contracts the verification itself is kind of closed source And the second thing is the data is closed. So the reason I care about this that much is the difference between the EVM data and the contract data. The EVM data that is on chain, what's happening in the transactions, what's happening in the blocks, those are public and those

are replicated across thousands of computers in the world, right? So anyone can build a block explorer. That's no issue. But the contracts are offchain and they're siloed. they are inside a database in a prop right now in propriety databases and to see the source code you always depend on those who has this data so I see this as a problem and it's also centralized so when you think about it if the source codes of contracts are under one provider this obviously fails the walk away test so when they are gone it's gone and again I'll bring up this test net issue test net that are deprecated these links are all dead What happens if these links are dead one

day for the mainet? And the last point is the tooling is stagnating and the developer experience is sub-optimal. So the APIs behind source code verification hasn't changed for a while. I would say many people who try to verify a contract had an issue once in a while. They don't know why it failed. You sometimes need API key. You sometimes need API key for certain providers. And also I think in general the APIs can provide a lot more rich compilation data and also better vis visualizations for users and developers.

So we can provide source maps, bite codes, storage layouts, immutables and the complete standard JSON input and output. So I would say this is not implemented in all verifiers and we can do better. The nontechnical problems I would say in the ecosystem the higher level problems are that if we don't do anything the current players are not incentivized to open up by themselves. So it's not going to happen automatically that they will give up this data and open up to everyone. They're of course incentivized to keep it keep it to themselves monetize it and use it in other ways.

So this in general requires active pushing and raising awareness so that over time the community realizes this this this is a problem. this might be an existential risk for Ethereum and that everyone demands this contract to these data to be opened up and shared with everyone and in general it's not a sexy thing to build like it's there are a lot of a lot more frontiers and in general contract verification is taken for granted even though it's not in the best shape it can be improved someone really has to sit down and get their hands on this this problem to work on it so that's why we that's what we are working towards at sourcify and the verify allian So sourcify our project is has started as a solidity side project within the EF

and within EF it spun out into a separate team and recently we also spun out from the EF into a new organization called Argo Collective. Uh if you haven't heard about it, Solidity recently led the largest uh Ethereum spinout and with Solidity Sourceify FA language and formal verification around 25 people, we spawn out into a new new organization called Argo Collective in a new entity and we were seated with a um three to four years grant from from the Ethereum Foundation. If you want to learn more, you can check out uh Leia's talk at the solidity summit today. It's it was this morning and recordings will be live or also visit argot.org.

But yeah, in general, all of the projects including us uh are nonprofits and we don't generate any revenue at the moment. And we are three threeperson team uh with Manuel, Marco and I. And the verify alliance is an verify alliance as the name suggests consisting of blog scout source rout. So around one and a half years ago we joined forces to improve the verification ecosystem and open up this data. Um thanks to Sam if you recognize him for coordinating us and we started sharing data amongst ourselves. Our goal is of course to provide easy unified and open access to the source code of the Ethereum EVM

smart contracts. In essence it's just a at the moment at least is a shared database of verified contracts. So what happens is when you come to us >> give us give us a contract we verify it then we say okay like we verify this and we write this on verify alliance as well saying this contract was written by sourcify. Similarly, when blocks scout verifies a contract, it they write also on on the database and that means we can also s as sourcify we can also look at the data database listen to it and then whenever someone else verifies a contract we pull it and we also verify it ourselves and maybe the most important thing is that we export this data daily in parket

format and we basically want everyone to use this data make use of it. I'll speak I'll speak about this a bit more in detail in the following slides and maybe again to mention right now it consists of sourceify route scan and block scout and you can check out more in our documentation what we are up to so how are we solving these what are the solutions to these problems so first of all open source open source everything we think like the whole stack of the verification should be open source and it should no longer longer be a black box.

For instance, the verification logic of sourcify is open source under the mpm library. And of course, all all all of the stack is there, but we mainly open source the verification stack. And this is a reusable library to verify yourself. And the nice thing about this library is that it's browser friendly and it is possible to run the verification on the browser. And I think this really aligns with what we are talk talking about in this in this summit about trustlessness. And in fact we have a running example on on the autoscan block block explorer. If you haven't

heard about it autoscan is a local first block explorer that runs with argon nodes. So what you do is you run an argon archive node. it syncs and then you run a simple autoscan instance and then autoscan can directly use this argon node to give you a local local first block explorer. Um and they already use the data provided by sourcify. So normally autoscan already goes to sourceify and asks if the contract is verified on sourcify and displays you. But what we do now with with autoscan is you can verify locally on your browser. So what you do is

autoscan runs you we have a contract you click verify locally and it fetches the sources from sourcify and it compiles a contract on the browser on your machine. It fetches the bite code and it verifies the contract if actually you can reproduce the verification. So this allows us this allows you to do it yourself on your machine without trusting actually sourcifier. Then we also have the sourceify server which is our main component. Um it's live at this API.

We have the repository UI to we have contracts but also across all other all other sourc components are um open source. We also recently took over the open chain signature database. If you know openchain.xyz XY Z it has been one of the most used Ethereum 4 byte signature databases and we just took over the database then we also implemented the exact same API on in our machines and now we also took over the domain and for now we're just redirecting the request to our instances from open chain XY Z but over time we will trying to push uh the source domain

and to use to make everyone use source yet >> addresses like sourceify URLs. But for now, it still works on open chain and as I said we took the took the open chain database. Then we also looked at other for byte uh databases such as byte directory and we also used the verified contracts to seed to to fill this by signature database. That means whenever your contracts get verified on source, we take the API and we also push the signatures in this database. So they will be available on these APIs immediately. So we also have a UI for that if you want to play around. Right now as you can see we have over 7.5

million signatures. In the middle you see like more than 4 million public submissions. This means like these are not coming from a verified contracts. And on the left hand side you can see the ones coming from a verified contract. So if we have the API and if we verify the contract we can say >> where these are coming from. So if it's a function signature, if it's an event signature or or if it's an error signature, we can say that. But public public signatures, we can't really tell if these are errors or um functions. I recently ran a nice data analysis on collusions and I found around 1,000 >> signatures that are indistinguishable.

So by looking at the signatures, you can't really make sure if one is a spam or not. So like in case for example, you can have a deposit function and then a full bar that you know and many numbers that you know like one is spam and one is legit. But I kind of figured out around 1,000 contracts are by nature colluding. There's there's a collusion and you can't really tell which one is which. So you for that you can check out the blog post in uh the source website and yeah that's the feature of having a limited space of four bytes uh for all the signatures >> and we also have a lot of source by instances. So like that's actually what we want. We want people to run their own

sourcify instances and these are the instances as of now that we are aware of that are public. Uh there are six instances. The first one is used by hashcan. Hashcan hash scan is the block explorer used by hedra chain. Second we have the monet test net that is using source for the verification. Monot testnet blocks block explorer. Then ronin is another one. Parseek finance is running. Vchain is running one and also the walnut uh walnut is also running a sourceify instance. Walnut is a recent uh EVM open source EVM debugging tool and they are using to >> using source to verify contracts both privately and publicly.

>> The the second thing is to verify everywhere. So we think this is key to open up the data. So I really don't I don't think it makes sense to verify a contract in a single place. If you have a contract, you want to have it verified. Ideally, you should verify in all verifiers or all explorers like why should it be in one place, right? And you can also see if it's actually verified everywhere. If it's not verified on everywhere, then you might think, okay, there's a problem either with you or with the verification. >> So for that, what we do ourselves as sourcify is whenever you give us a contract, we separately send it to all other verifiers. So let's say you sent a contract to sourceify, we

verified it. Then what we do is we also send the contracts to all these verifiers that we are aware of and you can also see the status in our UI of the verification. So we show first okay like the contract is verified and then we show what is the status in others how how and if this contract is verified we send it to ether scan route scan block scout. You may notice an overlap between this and verify alliance here. So this is separate than the verify alliance database. We push ourselves regardless. Uh but the verify alliance members route scan blocks cloud can already pull them themselves. But we do it anyways. So we want to share this data with everyone.

The only problem there is scan has a limitation of maximum 250 verifications per day regardless of the tire. So even if you pay the highest the most expensive API key, you only get 250 contracts per day. So we can't share everything with them. Second thing is a verifier alliance. As I said, we have this shared database between different verifiers and we want everyone to pull and push data to share all of it all of the data among all of the members. The second thing is the other thing is the remix plugin. So previously the remix had remix had separate plugins for different verifiers. There was an ether scan plugin, there was a source plugin and we

said okay like why don't we have a single unified plug-in and this really showcases what we want in general in verification and by default in the plug-in all the verifiers are checked in. So whenever you want to verify a contract sourceify it scan blocks got routcan they're all by default on and when you click verify it goes to everyone by default. Of course you forcan you need to provide an API key but if you provide an API key it will also be uh since they turned on v2 the API keys for all chain so it will be turned on as well.

Also recently hard hat added support for sourcify and now it supports parallel verification and foundry is also going to support uh verifying on multiple verifiers at once. So we want this to run really smoothly without creating a burden on burden on developers and the verification to go all places and the other thing is the open data. So as I said in the verify alliance slide we share the whole data set in parket format both for the source data set and for the verify alliance data set. You can click on these links and see a manifest file and it will tell you where to find the data, how to how to download the data and we share it in a in a data format called parquet which is a modern

data format used by use that data analysis very um very friendly format and we want this data we want this data to be used and right now we have more than 10 million contracts on sourcify this number is also similar on verify alliance and in sourcify at least we have above 10 million contracts around for around like 180 chains. So any EVM chain we can support easily. We also mirror the data in BigQuery and in Google cloud. And the reason we do this is first anyone using BigQuery for their for their data analysis to be able to just refer to this and use it. And the other nice thing is we want this

data to be available for Jupyter notebooks, Python notebooks. I think in Google it's called a collab notebook. So what you can do is you can just refer you can say from the sourcified database use this table and combine it mult with different data sources and you can do any kind of data analysis you want to do. We also built a AI playground to showcase what's what's possible with this data and how what questions you can ask to to say to this data set. So if you go to sourceify.dev dev our website you will see this data set playground what it so you can ask questions such as give me a contract with solid version X or you can say which contract is most popular contract by contract name and

then behind the scene this will send a prompt using the open router um models uh we use the free models by default so they might be not be as good but you can use anything you want and this will generate the SQL query for you and then this query for example, looks for the most popular contract and then you can see okay like the most popular contract on sourcify is actually a safe proxy with more than like 2.5 million um instances and the possibilities are actually endless with the open data. So I think this is the most exciting part and what I really want to see in the future is going to happen.

If I can go on examples, it's possible to use this data to do bulk testing for EVM tooling. Right now, Nomic is using this for example to bulk test their uh compiler slang. They they take bunch of like I think couple thousands of contract maybe 100 thousands and they feed this into compiler and then they see if there's any errors, if it's expected, if it's behaving as expected. You can run any data analysis around contracts. You can feed it to an AI model. You can train an AI model with this.

Another exciting um project I want to see happen that I thought about recently is maybe building a better decompiler. So I think the the decompilers are not as good as we want to be right now. And since we have this data set of millions of contracts which essentially mean we have a mapping of source code to bite code. I think we can build a better someone can build a better comp decompiler with this data. If you have any ideas or if you have any expertise in this please find me and talk to me and we can find vulnerability patterns as well. For example, let's say we figure out a solidity or wiper pattern that causes vulnerabilities. We

can just easily search the whole data set and see which contracts are affected in a minute. And yeah, this is in general a call to action for everyone to use this data set and move the EVM tooling together forward. And I hope by the end of today when you see a contract you really ask the question where are these contracts and we hopefully won't see this um empty pages anymore. So these are the links to sourcify and verify alliance Twitter accounts and you can always reach out to me through matrix and telegram. So thank you.

Are there questions? >> I'm gonna >> All right, we've got two questions from the audience. The first is how do we ensure alliance passes the walkways test? >> Good question. I think the main thing there is the verify alliance consist of multiple members and the data is replicated across them and also the data is shared publicly regularly. So every day we um we dump the data share share the whole data set with the community.

Someone can take backups of that. I would assume like some people have can be already doing this. So I think the how it's structured and what we do is a good guarantee to to pass the walk away test at least much better than the current situation. >> All right. And a followup how do we contribute to the verifier alliance? >> Um if you go to verify alliance website there is a link on of our telegram chat. you can join us the telegram chat and of course start using the data give us feedback and just reach out >> and actually Eli there's one more when ether scan >> when ether scan um um I would want to know as well hopefully someday soon in

the future. >> Awesome. Thank you so much. Big round of applause for our last speaker of the day.