Paradigm Shift: Building Invariant-focused codebases

Transcript

all right hey everyone I'm Natt um just a little bit about me I'm an independent security researcher um I spent four years at trailer bits um I joined as a junior and then um I did a bunch of security code review I did a lot of invariant testing and um I did a lot of uh kind of like fuzzing um before that I was doing blockchain development um as well as teaching uh for blockchain so the focus of today is primarily going to be uh kind of discussing the practices of secure code review um and this is kind of the I think there's a lot of overlap between code review and

invariant development which I want to touch on here so a while ago um I was talking to one of my friends about um just the process of doing a secure code review and we sort of settled on the fact that um trying to figure out what developers intended to do um tends to take up the most amount of time within a secure code review and the quicker you can understand what the developer intended um the more fun the code review and usually the more complex bugs you'll find I think that's also similar for invariant development because the more the faster you understand the code base

um the better in variance you come up with with and the better um or the higher quality your fuzz tests are so I want to start off with um kind of a disclaimer that this isn't going to be like 10 minutes of like here are all the things you need to know about unit tests fuzzing and like formal verification I don't think that's effective um and think given infinite amount of time ideally you would Implement everything um there's trade-offs to everything and the purpose of this not this talk is not to debate which one is better um so to start off um I've been

getting a lot of questions on just what an invariant is and at the end of the day it's very broad um it's very high level um it's essentially just a property that needs to hold true on a code base so again very broad it can be Access Control related it can be how functions um are expected to work and not work together um it can be data structure related and so um it's kind of hard to provide very strategic guidance on how exactly to find invariants um and so this talk is mainly going to be kind of tips and tricks on um building invariant driven code bases such that

from developers and for security researchers we have an easier time and one last disclaimer is your invariants are never going to be 100% complete I've been on engagements that have been very long um up to 12 weeks and even then the list of invariance that I came up with um or the process still wasn't complete in any sense of the word um this talk will not be either and I could probably go on for hours is on end related to um how to figure out invariants how to find them um so this is just kind of an in initial starting

point so from a developer side I think um one of the most timec consuming pieces or practices as security researcher is looking at a codebase reading a function and then realizing oh that function doesn't necessar doesn't do at all what the function intended to do it said one thing and then you actually look at the what the function does and it's completely different if you from a develop perspective if you do create accurate function names it helps in variant development and like the process to make that way easier um and especially to commun adequately communicate what the

function is doing um the other thing thing is if you can adequately name your functions then you can more easily determine what the relationship between different functions are um and then you can also have an easier time um to essentially figure out how functions are related um generally if you have like one function that uses multiple helpers um having those at least close together visually helps a lot this is kind of repetitive from my talk two days ago um but documenting everything in terms of assumptions because any wrong assumption can result

in un unidentified or unexpected Behavior which is very likely going to result in a bug um from an invariant development perspective though this ends up being way more useful than I think people realize because you can essentially read the documentation and any that is different between the documentation and the code is something to look at and something to report the next thing is just to comment um all the expected data validation of the system um ironically I've actually found some bugs while looking at a code base um pseudo coding the data validation that's ex that's that the

system is currently implementing it and then sharing it with the developer and then they're like oh no that's that Clause is missing missing some specific access control or some specific check um and I think it goes a long way to commenting um all expected data validation because then you can more easily determine what the function is expected to do and then my last one is um kind of the relationship between um developers and security researchers so I think there's um each party brings slight something slightly different to

the table devs I think bring um how users are expected to use the system um what is expected uh kind of the um functionally um how users are supposed to interact with it security researchers bring a different understanding of security risks protocols um different things that maybe other protocols got wrong um and I think there's it be invariant development becomes way easier when you take advantage of the fact that developers know or should know how their system works um so that can be async that can be um through calls but making

the time to talk with developers on what is expected um can also go a long way to understanding the system um I'm going to keep this kind of shorter um helpful practices for developers it's really easy to identify when a code base is built with um invariance in mind because it's much easier to know like when State changes it's really easy to know um how to get tests up because you can just look at the code base and you already know um it from reading it um if you do end up practicing tests while

or writing tests while coding um you're kind of forced to think about invariance by default when you're coding um obviously I think and and probably something that's been talked about for the last couple days is maximizing test coverage if it works um gives you an idea of what might be missing coverage what might have more coverage and give you an idea of what to focus on um and explore automated tools the tooling is going to help security researchers because then you can leave the tools to find all the low hanging fruit and leave the security researchers to find all the fun stuff and hopefully the automated tools as well um for security researchers I think thinking in an invariant focused way is

a lot harder than we realize because we're not used to taking a step back from the code base um we're too used to going into the weeds and seeing how the deposit function Works how the withdraw function works and not necessarily um the indepth of or the the higher level view of what is expected um I found it useful to just pick a code base try to find some invariance on it try to bounce ideas and see whether um you can you may not find the same invariance um and your invariance are likely very not are not going to be correct in the beginning um but it's a very useful practice to start think about invariance in this way because you don't need to set up a

fuzzer or you don't need to set up aover in order to test those invariants you can start um identifying them and then doing your code review with invariance in mind and I will emphasize um to just take a step back and make sure that you get a bird's eye view all right so that was pretty quick um I left some time for questions so I guess we have a few minutes hello hey so a lot of good stuff in that talk um one thing you said that I

particularly like is putting code you know functions near each other when they're related how would you advise developers to balance that against common coding practices that would be applied to large code bases like oh we put all the public functions up here all the internal internal functions down here and there can be benefits that kind of uniformity as well so you know how how would you advise people to find balance yeah honestly it's really challenging because I would say personally I found if like the depth if if it's a relatively um if there's not that many helper calls then having I find it just easier to read in the same function um but obviously if there's like many different helper functions

like kind of putting that into like different different contracts um honestly I would say um the thing that I found that helps is just cloning the repo um and then having like say like a split screen and Bs code either um side by side or like I guess up to four quadrants and seeing how hard is it to swap to like jump between um the main function and its helper functions so that you can kind of see visually um which one I guess is is the clearest in terms of its links okay any more

questions we are meanwhile preparing for a panel if no more questions for the crowd we will thank not for presenting and uh