Scaling Smart Contract Security with Automation and AI by Shashank

Transcript

Hello everyone. Uh I'm Shashank. I'm the CEO and co-founder of Pred Shields which is an uh cyber security web3 cyber security company and we are more towards the automation side of things. Uh you know today we will talk about uh you know like scaling um automation security in web 3 using AI automation and uh what we have done and what we have achieved uh in in this format of the game. So a bit about myself. I have been into security for almost 13 years starting at the age of 14 when I found a bug in Google and uh my name is listed in their security hall of fame page. I also found bugs in Microsoft uh Apple uh 30 days in Apple iOS 10.3 and many many more. Uh my web3 journey started with Avalanche

where I joined Avalanche team as a security engineer and and then later on I started my own venture. So why smart contract security automation? So if you talk about the the numbers uh you know like uh numbers of hacks which has happened across uh let's take the numbers 2023 1.5 billion were lost in you know like smart contract related hacks and out of which 55% that is roughly around 700 million was due to bugs in smart contracts. The numbers for 2024 is quite the same you know um somewhere around 1.44 4 billion and you know same 55 to 60% of vulnerabilities uh um in the in the smart contract you know it's like logic or the code

itself. Uh these are the graphs which you can see you know it's like 1.58 billion hacked across 420 you know like different hacks uh in 2023. Now let's talk about the problem. The problem is you know it's like it's pretty evident that we lose money you know it's like we lose faith in blockchain as we as we get hacked and uh the solution currently right now is you know it's like manual audits uh you know it's like uh multiple audits bug bounty programs uh so uh there is one sort of lacking is you know it's like if we expect blockchain to grow we also expect security incidents to grow and that's what we need to reduce and this is

something we can learn from you know the the web to security industry where the talent has catched up, people have two links, you know, it's like people have better form of security and you know it's like risk management etc. So if you see the current you know like web two web two security ecosystem uh it comprises of you know it's like a shiplift approach where they implement you know CI/CD uh security pipeline tools uh to remove like 60 70% of the vulnerability and then they do a private audit uh uh you know it's like or maybe an in-house audit team which will take care of you know it's like maybe uh 99% of security and then for.1% or you like 1% of the masterclass bugs like a very edge case that's where

you launch a bug bounty program and my belief is the same you know the web3 goes into same ways as well we have immini uh to you know uh to solve uh you know bug bounties but again if you directly go ahead with a bug bounty program you will end up paying a lot of money which is uh not good for a web3 company for financial you know like purposes and then you know like web two audit sorry web three uh audits like you you know uh competitive audits or something like that that also you know it's like requires a lot of money as well. So how do you cut down cost you know it's like you implement automation tools then you go for a private audit and then you launch a bug bounty program that makes your entire you know like financial cycle of spending in security

lower and it also increases you know it's like uh uh your security because there's nothing such as like 100% security. So when we talk and and in this you know like uh in this talk I'm focusing more on you know uh scaling security which is you know the the first 70%. So there has always been a problem with you know like automation security is that you know with static analysis you know writing detectors you can reach 60 70% of security but it always fails when it comes to business logic vulnerabilities you know it's like uh uh bugs which can differ from application to application and that's where you know it's like uh the the power of AI come into

picture and we'll we'll do this entire we'll we'll understand this entire thing uh through you know a test case uh you know like for an example case scenario. So in 2022 uh there was a company called Shadowfi uh which got hacked and you know it's like the exploited amount was 316 uh you know like $1,000. Now let's understand the hack behind that. It was a very pretty simple attack which you know like which could have been found using automation as well. So if you see the first transaction uh the hacker tried to gain some you know it's like wrapped BNB then the hacker you know it's like I think made a test transaction to try and buy you know it's like.5 shadow fight tokens

and then the hacker bought you know it's like full amount of nine shadowy tokens then we can see something happened and 10 million of the shadowy tokens were burned. This is where actually the vulnerability lies but we'll come to the point later on. And then rest we can see the rinse and repeat where you know the hacker tries to sell.5 shadow fight tokens and then sells whole lot of you know like shadow fight tokens and then converts it into you know like BNB taking a profit of $316,000. Now when when when we looked into the hack you know it's like uh we were obviously curious about the fourth one where you know like 10 million you know like tokens are getting burned. So uh our first guess would be you know

it's like the hacker found a way to uh to you know like burn the tokens and which was actually the case. So if we see uh see the contract code line number 962 you will see you know the burn function was public and this is this is a very very simple bug uh which could have been found again when when we come to these things like you know why this was not identified because there was no audit done. Why there was no audit done? Maybe they did not have enough money to you know like fund the audit or they did not want to do it. But these things you know it's like when we scale security uh we bring the cost down through automation and that's where you know it's like these things could have

helped. So when we saw this we realized that okay you know we need to automate uh you know access control vulnerabilities. If you see uh last year data as well out of the 1.4 4 billion 93 uh uh 900k dollars were lost due to access control vulnerabilities and it still remains access control vulnerabilities still remain the top vulnerability for 2024 but automating auto you know it's like automating access control vulnerabilities is not that easy uh let's see the problem we tried to automate that uh on our you know solidity scan platform and we did actually find the bug through automation you can see uh bug bug ID 232. It says

that okay you know it's like the burn burn you know like function is public but there is a catch here you know it's like uh since access control vulnerabilities are not remedated in you know like a singular fashion people can have you know like open zeppelins ownable module they can have a require statement they can have a custom arbback uh there are many many other possibilities right so the problem here was you know it's like uh we were detecting access control vulnerabilities but we were also also detecting 24 others in same vulnerability in different different functions and those are false positive alerts. you know the the scanner is not correct with uh you know it's like uh other you know it's

like 21 22 of them and this was a big problem for us you know it's like trying to find access control vulnerabilities because the false positives were much much higher so how did we solve that that's where AI comes into picture you know it's like uh when you write general algorithms you know it's like when you write general algorithms the problem with that is you know Um you you can write some cases but then you cannot obviously write all the edge cases all you know like require statements if statements find out arbbacks which is in place this is something an algorithm cannot do because it will vary from contract to contract.

Now, but with an AI, if we you know, like send all the functions and the modifiers uh to uh to uh to a you know like to an AI or or we create a rack based system with a vector DB, we can actually guide the algorithm to find the correct access control vulnerabilities. And this is what we did. you know it's like by providing context to an LLM model we were able to reduce the false positives uh by you know like you can see you know it's like in the next picture when we implement all these robotic figures with a small cross are actually invalidated by the AI stating that there are certain you know like uh excess controls in place uh and then you know it's like the AI also confirms that bug number 392 is a valid vulnerability

and rest are not so you can see you know like in the next slide you can see It says that okay there is this is an incorrect vulnerability because certain you know like access controls are in place and the result was uh you know like the result was something like this without AI uh access control vulnerabilities which were detected were 24 valid detection was one and false positives were 23 but with AI context you know it's like uh the implementation of rag and a guided you know tour of the algorithm to find access control vulnerabilities We uh we removed 19 false positives.

That's a you know like that's a reduction of 82 percentage. So uh so uh I I uh that's that's something you know like uh we are actually working on and uh the next what is next for the future of AI you know it's like in scaling you know like smart contract security is you know uh we find more and more context and how we do that is like we read each and every modifiers and then the the the modifier function is actually sent to the AI to understand the context if there is any internal function or any you know like external function call we we send that to the AI to understand the context to validate if you know it's like uh there is certain

access control is in place or it is not. So uh thank you everyone for hearing uh that was my talk and if you if you want to discuss security or you know like if you ever want to you know talk about security I'm available on Twitter or LinkedIn I'm happy to respond. Thank you so much. [Applause] have any questions from the audience? If you want to have a question, just raise your raise your hand. Yeah, would love to answer any questions if you have.

All right, I'm on. Hey uh good talk man. Thank you. So is this only for smart contracts on Ethereum or do you also audit uh like Rust and other smart contracts? So we are EVM compatible. So if you are anything on solidity and EVM chain uh this works. We haven't moved across rust uh because of the demand. uh we did create something called ruscan.com uh with hardly four or five detectors maybe in the future but currently we are

focused on solidity itself and and it works across all EVMs so uh be it you know it's like binance be it polygon avalanche anything we have support for almost 4050 plus uh EV around 40 EVM chains as we are partners with block scout okay nice thank you any other questions Yes. All right. Oh, sorry. I didn't see you. Sorry for that. Hey, thanks for the presentation. Thank you. Did I misunderstand that you were intuitively or implicitly saying that you think like bug bounty security spend is inefficient and and should be uh allocated away from that ideally? Of

course in an ideal world that will be the case but no no no no I didn't say that. uh so my my understanding and belief what I understand from you know like uh from the web two perspective because I spent 10 years there is it eventually reduces your cost you know for example if you have uh you know kept a bounty of $100,000 for critical bugs you know it's like my point here is you know like that should remain the same is just that you ensure that you don't pay up you know it's like five five times of critical vulnerabilities because you have patched them in the very early stage of your development So that that that was my point.

Thank you. Any last questions? And and just to add you know like additional point to your question uh that you know like what also happens is that when you use automation tools in your software development life cycle uh when you reduce like 60 70% of vulnerabilities and when you go for a private audit it increases the efficiency of that audit as well because the auditors needs to be focused on you know like uh on deeper vulnerabilities rather than you know something which is quite common. It it has been the same thing with web two vulnerabilities as well. There was a time when cross-sight sifting or SQL injections were everywhere. Now it's not is because you know most of the like almost all the

companies have you know these SDLC tools chip left approach to automate those vulnerabilities and fix them during their like development life cycle itself. Yeah the low hanging fruit gets picked out first. Yeah. Awesome. Uh round of applause for Shashank. Thank you guys. Thank you so much.