Exploring AI's Frontier in DeFi: From Vulnerability Simulations to Secure Protocol Design

Transcript

Yeah, thank you Jack. I think we met before so I might know you. Uh I'm Arttomy. I work at Sherlock. Um essentially my talk is about the you know there is a lot of fancy words here but essentially it's about AI and I'm trying to explain what we did with our Sherlock AI and how the teams use that. So essentially I think on this you know conference and the presentations a bunch of the team said a lot about the AI why it's useful essentially the main thing that we believe is that it could be really useful in the full cycle security because during the each development stage and not only in the pre- audit stage but also when you're doing the

fixes after the audit when you're doing the whole thing it can save you a lot of time um after um you know you don't need to spend spend a lot of time on the audits because you some can already find some of the issues that and the security researchers can actually focus on the um finding the other vulnerabilities but that's the main thing uh essentially we we released uh the v2 version of our uh several weeks ago essentially two weeks ago and my main thing that I want to talk about is like how this is very high level uh how we approach um multi- aent system in the in the sherlock AI So essentially our idea was that we want we want to try to put some of the best

minds of the security researchers that we have at Trello like Xerox 52 and some others and trying to implement their frameworks how they are auditing code bases into AI. So we start like as every auditor we start with the global preparation and the planning we just trying to AI's agent is trying to understand what this codebase is about and uh what the codebase is doing and etc. Second thing is actually where the brain of AI uh so we have multiple steps there. So the first thing is a graph builder. So essentially we are trying to put the whole codebase into different graphs and the main thing we're trying to understand what talks to what. So what part of the codebase to talks to

you know the different part of the codebase. Second thing is the aspect modeler. So we put the whole codebase into different aspects and into different parts and um for example it could be like Oracle the meth part the liquidation logic and we also come up with some specs for each part of the codebase and then threat modeler is more a general thing. So we're just trying to come up with a threat model for every part of the codebase and every aspect. Um and of course we're doing some context like for example scout uh context. So essentially we we are looking for first is the previous audits the uh the docs and uh yeah similar code business maybe and some other stuff that

could be helpful to come up with a threat model and then to to doing an actual uh vulnerability search. So in the vulnerability search we using essentially what we already done with the graph. So we're trying to um match and see if every part of the codebase is it matching the actual um like the threat model you know that we've built and if it's really you know matching the real code bases or not if it's working as it should be or uh the frag model the the attack is actually could happen and a big part is actually in the validation and the judging so every EI is you know you have this

problem with a lot of false positives. Uh if if you ever tried implementing the codebase um and trying to run couple AI runs uh you have a bunch of false positives. So we're doing it in two steps. First is a kind of validation and trying to come up with a PC for that. Um it's kind of the first step. So let's say after the full run you have like 500 issues uh in there and for each issue we're trying to validate doing this first layer of the validation and come up with the PC and the second thing is actually when we judge and triage all this all of these issues. So we have a separate agent. We've been running this judging agent on our um

essentially all the competitions that Sherlock is doing for the past few years. And um this judging agent that work in the competition and we're validating like five hundreds of issues uh it's also judging the whole issues that AI has has has found and then this is the fi stage five is where you as a user actually face the interface and uh reporting and see the all the issues that AI come up with. So we have this kind of feedback loop where you have like let's say six issues found. You have two highs, two mediums, two lows and then you can also mark it as valid or invalid and then come back to the AI and maybe do another one based on what you alreadyed as a person or as a security researcher. Uh I have 5 seconds

left. So this is for example how some of our teams uh using AI and we believe that it could be not just the pre- audit thing but um let's say you want to do a run before the you start the actual audit you find some some issues then you're doing the human audit then you do one more run on the fix on the fixes then you do one more run before the let's say audit competition on the second audit then you do again run on the fixes and uh yeah we can just doing with every development uh every component that you have. Um yeah, that's actually how this is a real example of one of our teams.

Um thank you. Uh if you want to try it out, I'm the right person. Uh thank you very much. [applause]