Zero to Smart Contract Auditor in 1 Month

Transcript

Uh today we're going to talk about how to become a smart contract auditor in one month. It's quite a tight timeline, but uh you guys look pretty smart, so I think you'll be able to do it. Uh I'm Jack Sanford. I'm the CEO and co-founder of Sherlock. Sherlock's one of the leading security companies in the web 3 industry. Uh we do auditing, bug bounties. We're one of the pioneers of audit contests, one of the pioneers of smart contract insurance, um and a bunch of other areas that we help customers with. So yeah, let's get right into it.

All right, so a couple assumptions in order to become a smart contract auditor in one month. It's a very ambitious timeline, but we're going to assume that you're basically an intermediate level smart contract developer. Um, so that means you know Solidity, you're familiar with protocols. Um, and basically if you're not familiar with protocols or not an intermediate uh, smart contract developer, the first steps are learning basic blockchain and EVM concepts, uh, studying solidity, you know, reading the docs, reading protocols, um, and learning foundry. Learning foundry is a really important aspect of obviously being a developer, but even more so, um,

when trying to find exploits in protocols, which we'll talk about a little bit later. Um, so who to take advice from? I'm the CEO and co-founder of Sherlock. I am not a top level auditor, but I have been talking to interfacing with top level auditors for about four years now. I've read thousands of issues and um, yeah, luckily I've interviewed almost all the top level auditors at Sherlock in our Blackthornne program about how they would go about this process and I've collected all that data here for you guys today.

so you all can quickly get up to speed and start finding issues in a month from now, maybe two months from now, three months from now. Um, getting paid in public audit contests, getting solo audits, um, and everything like that. So, these are some of the researchers. Um, this is Sherlock's Blackthornne program. These are the elite of the elite. Uh, as you can see, there's a very select group of them. All of them have won a lot of audit contests. So they've performed extremely well in public environments, outperforming hundreds of researchers in the past on different types of code bases. I'll point you to this 132 contest won. So as you can see, there's about 10 to 12

Blackthorn researchers. And so they've each won about 13 contests on average. And winning an audit contest is kind of like winning uh a game of Fortnite or, you know, winning the Hunger Games. It's you against about a hundred or in a lot of cases 500 a thousand people. So winning one audit contest is an extremely impressive feat. And so winning 13 audit contests, these are the guys who you probably want to be taking some advice from and learning from if you want to become a smart contract auditor uh as quickly as possible and as efficiently as possible.

So, we're going to talk about five steps to becoming a top level auditor. The first one is the one-month timeline. Okay, maybe that's a little bit clickbaity. I think maybe some of you in here could do it. Uh, but you know, if you don't think one month is possible, two, three months um, you know, is is reasonable for getting up to speed. And of course, to become a really top level auditor, you know, that can take a year, that can take two years. It really depends on how dedicated you are. Are you full-time in it? Are you um specializing in a certain area of crypto, whether that's the EVM or Salana, learning Rust. Um so the timeline really varies here, but one

month may not be realistic to become a top level security researcher. Just want to throw that out there. Um the biggest problem for solidity developers is around the mindset. Uh and so that's one of the areas we're going to talk about in these five steps. The mindset is a really big shift from being a developer to being a smart contract researcher. Um and we're also going to cover on the last step a four-step flywheel so that you can progress as quickly as possible and you know outperform your peers in audit contests.

So step one understand code bases deeply. Basically, one way to go about this, again, if you're a smart contract developer, which is kind of one of the assumptions here, then you probably understand at least the codebase that you've written deeply. If not, you would want to look at ideally really well-known protocols like unis swap if you're interested in AMMs or a if you're interested in lending protocols. And one key is to really pick the protocols you're interested in. you're not going to survive long if you don't like what you're doing, if you're not interested in it, if you're not curious about the codebase and about the type of um protocol that you're looking at. So, ideally, you know, these protocols,

maybe a fork of the protocol is going to go through an audit contest soon or it has a live bug bounty. So, you know, if you do end up finding something in your first month, you can actually submit it and uh make some money. So one thing you want to do is you want to go extremely extremely deeply on the codebase. This is what a lot of the researchers I've talked to have really emphasized. This is kind of the first step. Um basically you can develop you can implement protocols from scratch. So literally saying hey I like the idea of a lending protocol. Let me see if I can make one. Um, you can look at code bases that exist or you can kind of do it just off the top of your head and kind of add the functionality that you think should

be there, then compare it to one of the top protocols. Um, and you actually want to get to the point where you can visualize a codebase in your head. So, this is not even looking at the code. You know, you're going for a walk and you can still think about issues. A lot of researchers, they will think about issues in the shower. they'll think about issues when they're, you know, walking, uh, going on a hike, whatever it is, and you want to get to that point. Going to sleep is a hard one. You know, getting to sleep, thinking about the issues. Um, so yeah, and that can help you with the last step, which is basically putting in the hours. Um, this step is pretty timeconuming if you're not an intermediate or expert level Solidity developer because you really want to have that deep deep

understanding of a codebase um, in order to to start out as an auditor. So once you do that, you're actually pretty far. Um, if you fully understand a codebase, you can actually find issues in that codebase. And this is extremely common. We talked to a lot of developers who they just build a a derivatives protocol and then they do an audit contest looking at one of their competitors derivatives protocols and they find real issues and they make thousands of dollars and you know that's auditing. So just by deeply understanding a protocol um you're actually auditing and you're actually able to find issues.

Another pro tip is getting good at Foundry. So again, if you're a smart contract developer, you're probably pretty familiar with Foundry. Most teams use it at this point. Um and this is really going to help you when it comes to writing tests and writing PC's. And writing tests is actually super important because a lot of the teams that you will see in audits, they don't actually have 100% test coverage. Um, or they haven't tested every case that maybe you could think of or if you're understanding the codebase, um, you would want to test. So, you can actually kind of look at their code coverage and write tests to give them 100% coverage.

And there's a decent likelihood you will find issues just by doing that. you know, you're kind of acting as a developer almost for them. Um, because as you can tell, they haven't actually tested those areas of the codebase. So, there's a reasonable possibility that there would be issues there. Okay. Um, yeah, so I'm going to add some quotes of, you know, top researchers here as well. Um, Panrog number one on the Sherlock leaderboard. you know a lot of people come to him for advice and you know half the people are still on step one or still working on step one where it's like hey they can't understand the codebase fully. So again this is a really timeconsuming step but it is probably the most important step because

if you can't understand the codebase fully you're not going to be able to do anything else in terms of you know the next steps or in terms of being a great auditor. Um yeah so read issues and build your checklist. uh we're assuming right now maybe you've studied those protocols either as a developer you've developed your own protocol in a certain area or you've picked a protocol like unis swap or a and you've read it deeply enough so that you understand the codebase and what you're going to want to do next is figure out all the places where issues have been reported in that codebase. Um this is important to essentially give you an idea of what the attacks are because you don't want to be coming up

with every attack from first principles uh on every single codebase because there are a lot of patterns, there are a lot of things that repeat. So looking at audit contests, audit reports from traditional um audit firms, looking at bug bounty writeups um and again understanding the issues really deeply. Um so you want to understand the code base deeply. That's kind of step one. Step two, understand the past issues really deeply. Uh yeah, so next step here, build the checklist. Um there's a lot of checklists out there depending on what protocol you're looking at. Um but you really want to build your own checklist so that you can have an understanding of the issues. Like don't put an issue on your checklist if you don't understand

it really deeply. Um yeah, that's already on there. So, and then you want to start thinking about patterns of issues. So, categorizing issues into um certain vulnerability types so that you can have like a a running knowledge base of different vulnerabilities and apply that to code bases that you're going to look at in the future. So, having this checklist is is really important. We'll maybe talk a little bit about checklists at the end. Um but this is also a super timeconuming step. As you can tell, you know, doing this in a month is going to be difficult. Maybe it's possible. Um, but you know, it's not something that anybody who's a developer wouldn't be able to do. You'll be able to understand

the issues if you look into them deeply, and you'll see lots of issues that are talking about the same type of vulnerability. So, you should have no problems there. Okay. Yeah. Some other quotes. So, wow. Now, you can find 80 to 90% of bugs in an audit contest just by having a good checklist. Um, and what Panrog would tell you is focus a little bit on one specific aspect. If you really are interested in lending protocols, focus on that. Um, sorry, uh, five minutes left.

Okay. Um, yeah. So, if you're looking at lending protocols, uh, you really are able to find a lot of those issues if you have like a really great checklist and you understand a lot of the common vulnerability types, um, in those protocols. Yeah, you need to know what to look for. And again, you don't want to have a checklist that you're trying to memorize or you're trying to take from somebody else. You're trying to have a checklist that you've created that you really understand. even if it doesn't have all the issue types or whatever, you really want to focus on the ones that you understand.

Okay, number three. This is the one that really trips up developers like smart contract developers. Um, a lot of teams. So, yeah, here's chief security officer of a top 20 protocol. Basically, if you're coming right into web 3, especially as a developer, you aren't trained in the classical attackers mindset. Um, and this is a really important thing that a lot of the researchers brought up in terms of having the right mentality to find issues in a codebase. Um, so as you can see, if you're a developer, it's all about making it work, making the functionality work. It's a totally different mindset. You're building something and now with security, you're kind of breaking it down. You're trying

to find ways that it breaks. You're trying to make sure it doesn't work in some ways. Um, so you really have to change your mindset. And this is probably a big reason why the audit industry in web 3 exists in the first place is because it's really difficult for the developers to switch into a totally different mindset and start attacking the code that they just became familiar with over the last six to nine months uh building their protocol. Okay. um how to develop the attacker's mindset. Even basic CTFs, I think Simhal was saying this, basic CTFs like um Ethernet or um damn vulnerable DeFi, they can help you get into that mentality because you're basically just breaking code uh when you're doing

those, even if it's really simple vulnerabilities. So, that's one thing you can do if you're worried about this step. Um, and then reading exploit reports, whether it's hack reports of, you know, major hacks, $20 million hacks, that kind of thing. Understanding the post-mortems there, that can help you get into the attackers's mindset. And then bug bounty reports for criticals, audit contest findings for criticals, traditional audit findings for critical bugs. These are the ones where, you know, you're stealing funds, you're doing denial of service attacks, you're doing things that really break a protocol, really cause a protocol to lose its functionality. Um, and so that's part of getting in the mindset.

Again, it's about finding loopholes. It's kind of, you know, you're like, uh, uh, the the juvenile delinquent mindset. You want to be kind of breaking the rules. You want to be thinking, how can I cheat something? You want to be thinking about finding the loopholes in it. Um, and you definitely don't want to assume that the developers did anything right. Um, you always want to make sure that you're not making any assumptions in the codebase. And that's a hard thing for developer. That's why developers have a hard time auditing their own codebase is because they've spent six or nine months having a bunch of assumptions about the code and the functionality and what it should do. So your advantage as an auditor is you come in with a kind of blank slate on the entire codebase.

Okay. Uh yeah. So Sammy focus on CTFs exploit reports. Um yeah and then Vinica Boyd just kind of under uh emphasizing that this is one of the really key mentality differences especially from being a developer or even from you know coming from a non-cyberc background or non-security background like this this mentality and mindset is super important. Okay, step four get hands-on experience. Um, so one of the things that is dangerous about this presentation is the last three steps, none of them were actually doing auditing and ideally you want to get to the auditing part as fast as you possibly can. So assuming that

you understand a codebase, assuming that you, you know, have a decent grasp of solidity and maybe you have a checklist, you want to be getting into the the real auditing um, as soon as you can. That's why audit contests are great. So, starting with short midsize contests. This is a little bit controversial. I heard some researchers say you want to start with something that's a little bit bigger um and really go deep into it because there's probably going to be more issues, but most people think starting with short and midsize contests and really kind of getting the iterations, getting the practice in um is important. You can stop when you believe you can't find anything else in the codebase. So maybe as a beginner, you're not going to be able to find the

really deep exploits. That's okay. You don't have to beat your head against the wall for 3 weeks if the contest is that long. You can move on after a week, a week and a half and just get to the next contest and start the process over and just get in that habit. Um, so a priority list, this is what researchers have said are the best ways. Five minutes. Okay, cool. Uh, this is what researchers have said are the best ways to actually get hands-on experience. And again, you want to be getting that hands-on experience as fast as you can. Audit contests are the best because you get the feedback quickly. You figure out what issues you missed. You are often competing against some of the best researchers in the world. So, you're going to actually see the issues you missed. But the most important part

about it is that you actually are in a real work environment. So, audit contests are real audit. You are doing the same thing that the top level researchers are doing and you can make a lot of money if you do a good job at it. So, that's really cool. You can actually make, you know, $2,000, $5,000, $200, whatever it is if you're doing one of these audit contests, and that helps give like this extra bit of motivation, I think. Um, so shadow audits, these are really great. This is what a lot of traditional audit firms do to get their um junior researchers up to speed. You work with a more senior researcher. Um, and not every not everybody's going to have access to this, so it's a little bit more difficult, but if you're able to get on Zoom or you're able to audit

with somebody in person, just sit next to them and see their process. Um, that can be a really great way to get up to speed on vulnerabilities that we would miss or, you know, using the right mindset. However, it's dangerous because you aren't as incentivized, like you're not in a real vulnerability finding um, engagement. you're not being paid or you're not having the opportunity to make money. So, it can be a little bit um you know, difficult to motivate in those scenarios. Bug bounties are great. They're a little bit controversial. Um most researchers have said that bug bounties are a little bit dangerous for beginners because they're super difficult. Um like bug bounties are usually live code. It's code that a lot

of really top auditors have already looked at. Maybe it's been through an audit contest already. Um, and so it can be a little bit demoralizing if you're coming in as a beginner because it's going to be really hard to find anything material. Uh, but the good news is if you do find something material, you could get paid a lot of money for it. You know, bug bounties, if you find a bug bounty, that's like the highest price someone is going to pay you for a bug usually. So that can be really lucrative and it's somewhat of a real work environment. Okay. Um, pick contests that are similar to protocols you already understand. use your checklist and think about how each item could apply to the contest codebase. Yep, I think that's all pretty clear. Um, yeah. So, Dead Roses, one of the best bug bounty hunters in the

world. He keeps it very simple. He says, "Look, just do a lot of contests, speedrun, figure out what you did wrong, and start again." Um, Simhow, yeah, bug bounties are a little bit dangerous. Contests can be a better fit for beginners. All right, step five. I think I got two minutes here, so this is the last step. This is really really key. So the fast feedback loop, this is probably true for anything whether it's a sport or you know something you're doing in academia. You want to really know um what you're doing wrong and be able to fix it really quickly. Uh and the the good thing is again you can do contests like Dead Rose said, speedrun contests um see what you missed, repeat. This is kind of the

magic formula. I'm going to talk about it a little bit more in the next slide. Um but you're going to fail a lot. You want to be comfortable learning from your mistakes. Uh, and if you're if you get comfortable with this process, you're going to succeed really really fast. So, Vinegy's four-step process to becoming a top level auditor. These are the questions that he asks himself uh whenever he's doing auditing. So, this assumes he's already done a contest. You ask yourself, okay, I did the contest. I submitted three issues. There were 10 other issues that were submitted that were valid that I missed.

You go through each of those issues you missed. And you basically do this sort of checklist or this process. You say, why did I miss the issue? What question could I have asked myself to find the issue? You know, what pattern can I see from this issue so I can add it to the checklist. And you don't want to take it too literally in terms of like that specific issue, you want to pull out the pattern, pull out the heristic so that when you see a totally new codebase, you know, in the same area. Um, hopefully you can apply that to finding a bug there. Uh, and then how can this issue be mitigated? That's basically how do you fix the issue. This helps you think about um what proper code looks like uh what proper development practices look like.

So um yeah for creating a fast feedback loop Seric top ZK researcher nine contest wins happens to be in the room with us today. Amazing researcher. Um don't optimize for money in contests optimize for learning. Really really important you know um don't think about money in your first six months. it's not going to be material even if you are earning some money. You really want to just optimize for learning and optimize for those big payouts getting to private audits and that kind of thing later on. Um and then yeah, this is basically Panrog's process. So he asked himself you know why he missed the issue and um you know what caused it basically. So, um, yeah, I guess we're out of time, but

these are the five steps to basically becoming a smart contract auditor as fast as you possibly can according to some of the best auditors in the world. Um, yeah, I wish you all luck in your journey if you do decide to take on this mission. Um, and thank you to the researchers who helped me put this together. [Applause]