Smart Contracts Fuzzing: Current Problems and Proposed Solutions

Transcript

Andy Mley, founder and CEO of Memory, and he's going to talk about the current problem and solution of fuzzing. Okay. Thank you. >> Yeah. Yeah. Thank you. Thank you. >> Yep. >> Oh, I don't I don't know. Yeah. Yeah. Okay. Yeah. Cheers. Cheers. Yeah. Um so um so basically like um it's really bad title. I know it sounds like quite boring. So the first part basically I'm going to talk about like um the problem of fussing like we talk about a lot of like problem fussing. Then I would use a

lot of like real world example for instance like the balancer hack strategy and that like um to dive in why like we cannot cover those kind of uh those kind of exploit and then later on I would also like um using quite a lot of code snippet to actually like show what my proposal solution. Yeah to give a quick intro. So um back in time I was teaching at and uh teaching and researching in in economics actually and then I was doing like a microeconomics optimization base and then later on I moved to like machine learning research like back in like uh 2019 something around that and then doing like those kind of random forest and optimization again and then uh I worked for a desk for a short while. I'm thinking about like u what kind of things stopping us to actually come into crypto and then it's actually

here why we are here security is the major robot for the crypto mass adoption so I just like start the company and then to actually like trying to contribute um the the security space and then so this is like basically what I built um memorial to actually for value extraction so first of all like we need to understand like what is value extraction my my my feel on the nature of exploit so in the end is actually two things the main two things one is like uh two step of optimization. The first step is actually when we plug in a smart contract what kind of state changing function we should actually test and then we can actually form a sequence for instance like we have add collateral liquidate and remove collateral and then given this sequence what kind of parameter of input so that's the second

step of the continuous optimization then in the end actually like combining these two functions and parameter then that will be calling value expression so our muv board is is the same thing intensoling is also the same thing economic exploit is actually something the critical one that we should really soft it. So um the two things that I I'll later on like talk about the jargon and those kind of things as well. Um yep and then we basically like all these function in the end we are actually changing the state. I don't care about like uh we run 100 functions together as a sequence or we run 100 function one by one one by one change the state and then we add a function after to actually trigger the exploits. So in the end it's actually changing the state and change

change in value. So um I'll start giving like some motivation example that um why um it's actually feasible to actually using fussing to find exploits. First of all like uh like actually back to last uh last month like we have the mean spell uh exploits and then 1.7 million exploits and then the core uh the core the core root cost is actually um when you run a weird action action ID. So it's like one of the function that you call is called um I think it's called cook. Yep. This is like I think like really a lot of people should visit the D5 headlabs like uh reproduction and then this is the code snippet from that and then when you form the uh the code and the code function in a actions the action is a you're in array then if you

put it the first action as the borrow and then the second action is as sort of like empty zero so it's like a sort of invalid op code then we can actually bypass the in in solvent track so this is like the core critical root cause but you can see the reproduction That's it actually like the actual exploits they have like uh generate the full cost and then they also like swap the function back uh swap the swap the token back and then return the fresh loan those kind of things in fussing is actually something I call like dimensional reduction we can actually reduce those kind of things. So uh we talk about like approval and then like profit vacination swap and then and then the loop based exploits is also like uh something that we can actually reduce. So for the next ph is like I

give another example for uh I also found a a a another explore like uh in back in April that talk about live protocol that they leverage a sort of price manipulation by using the loop. So they do like for loop for like uh 50 53 time like buy and sell and then when you buy we push the price and when you sell they actually forgot to update the price and then they can they can actually uh gain the profit. But those kind of loop when we actually think about fussing we don't need to run 50 something times we can actually run or maybe or five time or even one or two times we push the price we get like 0.01 01 then actually we can sort of detect the exploits. So this is like making more feasible to actually think about the fussing. The next things

is actually like the the the the major news about like the balancing exploits. Um the first part I was talking about the sequence that uh is reducible. The second part is actually uh we need to understand more about like the input parameter mutation. The second part of the optimization though. So uh what's happened in the balancer is that like uh the core details is actually like they have like a upscale function that uh actually causing the rounding error. The entry function is actually the best swap and then the bash swap they go in several internal core and then like the internal core they the core things is actually like uh from the best swap one of the full contract they have the internal core internal function call and then one of the internal function call that will be talk about the external function call on swap and then on swap

they go for several internal function call in the bar in the pool contract and then they will trigger the upscale and then to trigger the rounding error. So um later on like when I talk about my framework uh well my solution is actually uh proposing a solution that how do we even we know upscale is the problem how do we get into this branch yeah and then I was talking about oh why uh why balance batch swap is really difficult so I'm talking about the difficulty so first of all like we actually understand a bit like about the input parameter so a lot of time we think about like the input parameter is actually continuous but uh a lot of complex type for instance like dynamic tupole and then dynamic array those kind of bite type and then we also need to know uh what kind of input parameter we

in this batch swap function is used for the external call later so it's like for instance like the uh I screenshot here like the the swap.pool pool id that particular bite 32 type is actually used in on swap later. Yeah. And then also like talk about like the external call dependency and then uh how do we actually know that the linkage between the caller function parameter to the callie function parameter because we cannot mutate the call function right so we have to mutate the the the entry function. So basically that's is like the motivation example. Then uh when I when we talk about fussing we need to actually understand several jargon that I will keep using. So first of all like it's actually think about uh the common framework will be when we start a smart uh first a smart contract we think about

oh this function we need to start then what kind of parameter of input so the argument initialization and then like we also need to think about how do we generate the sequence what kind of test case that we need to do and then if given this given this parameter of input we how how do we mutate for the next one so this is like academic paper that I I I also read I find it like really good for learning fussing and then the sit mutation that will be and then the feedback mechanism is thinking about like for instance like code coverage or like coverage based or distant metrics though kind what kind of metrics that we can use to guide for the next mutation. So uh this time I deposit $10 next time do I deposit $9 or $11 which direction I should go. So this is like basically the

the the seat mutation and the fifth band mechanism and then of course like uh we talk about the oracle a lot of people confuse about like the the oric price oracle in in the space but it's called like the butt oracle is like what kind of things we classify as the butt and then the scheduling those kind of things. Today I will basically focus on the on the sequence generation and then mutation part. So basically the two-step optimization I talk about. Yep. So uh I was talking about like oh what kind of problem. So it's we're still in the first path and then um so those kind of common problem that people would expect and then we think about like a state exposion. So it's like we have so many like for for a smart contract we have so many different multiple function and we

can actually combine this and then we uh combine this together to actually concern the state exposion as well and then that could be like also looping between like uh between the functions as well. So this is like the state exposion and then we also having like within cont within protocol multicontract uh dependency. So you can think of like each smart contract address is a system. Then the major problem of the traditional software engine uh software testing is actually we targeting one single system for instance the gaming but like in smart contract we actually have like multiple address for one single protocol. Then how do we actually cover the external core and multi system and then also like the proxy contract that we have the s storage in the proxy contract the the logic is in the

implementation those kind of things is actually also like quite increase the diff difficulty then of course like the dynamic external core that those kind of price oracle manipulation then how do we know that this protocol is actually integrate with other for instance like the uni4 actually having the hook you can actually link to so many different protocol then this is another like difficulty then current approach for actually solving doing this um sequence generation. How do we actually form a sequence? The first one will be thinking about the Rick and Y relationship. Then you can think of like Rick and Y is like deposit like and withdraw. How like we deposit and then that will be changing uh as store the storage slot one of the storage slot. Then how do we uh then we can know that the next function we actually use is actually load that

particular storage slot. Then we can actually combine this fun uh function together as a sequence. And then the next thing is actually we talk about like customarium. can actually leverage the leverage the auditor knowledge to actually like really specify the test case that we want to do and then the next thing is actually uh think about like those kind of state based snapshot which is like the ATF and then uh smartsh shot they also do it in in this way then the next challenge I I'll go yeah sorry I yeah I need to go faster as well and then the next challenge will be thinking about like uh you see the back swap is really difficult and then how do we actually do the parameter generation what kind difficulty that we actually have. So the first thing is actually API type uh we use AI type uh mutation a lot

and then really unique address string those kind of things is something that mutation the transitional mutation we cannot do and then the next things is actually dynamic array. So when we actually think of dynamic array uh batch swap they have like dynamic array dynamic tupole dynamic address then those kind of things is actually increasing the possibility of the argument as well uh argument uh argument mutation that will be really increased a lot and then the format is actually like uh to put the dynamic array data that is quite complex as well and then also like the dynamic array and then we also have having those kind of core data as an input. So uh this is one of the one of the example moon hacker actually last year. Yeah. Go ahead. And then by like 300 around 300k and then you can see

that like what they do they actually use a core data to as an input and then later on is actually decoded and to the exploit basically like when the uh attacker put the address after decode as the malicious address then you can actually bypass some validation and then you can take the money out. So the the difficulty is actually craft a call data that actually bypass it and later on I will talk about the solution into it as well and then compress data type call data that will be uh used it for the later on uh collie function on swap as well. Yep. And then current approach for the Yep. Um so we can actually think about like what kind of current approach and the classic one is something called like uh Harvard mutator with the lip AFL uh library. a lot of like um fer

actually using it and then they have like so many uh different mutator bit those kind of like flip zero to one and then one to zero and then those kind of things by ABI and then the next ph will be thinking about distance metrics you can think of like I require the balance greater than 10 and then now my my balance is one then I know that I need to increase my balance to like 10 in order to bypass some uh bypass a branch and then later on that also like leveraging some optimization algorithm to actually mutate for the input parameter. Yep. I think I'm going to the second part. Yep. So, um my my proposal solution that will be like thinking about a three- layer approach. The first approach is actually leveraging some LM into guiding the fusser and then that

will be having four part of it and then later on I will talk about like uh uh the the state based fusser that um that that that that I I'm building and then like the the the EVM execution actually we can actually parallelly uh ex uh execute on the on the GPU. So the first part for the LM we uh we actually have four part the T analysis external core trace and then compressed data generation and then dynamic runtime information those kind of mechanism. The first thing is actually you can think of um a lot of people actually doing it as a like uh a you can think of static analyzer version of the LM. So replace a sliffer. You can think it this way and how do we actually having uh having having uh like we can actually have different technique to do it and then

the core things to actually link the static analyzer to the fussing is actually what I think about if the LM identified line 177 is the vulnerability we need to get into the upscale then how do we know that is actually uh how do we actually get into the upscale and then one of the mini trick I try is actually that we can on the source code we can actually uh add the assembly code or something like ox7777 F7 and then we know that on the control graph here is the particular basic block we need to get in. So we can actually identif identify what basic block we actually get in to trigger the potential vulnerability. So the next phase is actually think about um the for the LM to guide for the first thing is actually uh I was talking about the best W and

then how do we know that what kind of best war parameter I can mutate on the on the later on inner on swap function. So on on swap function they are having requests but how do we know that we cannot change requests how do we know that which parameter I should change in the upper function to change the request. So this is like the LM understand the source code can actually identify uh which one is it. So uh the approach we do is actually you can think of like given a smart contract we find or all the all the caller function having external call. The second stage is that give uh we know that uh the caller function. So for instance like uh for this is a unis swap uh pool manager contract uh we have the swap function in the pool manager contract what kind of

external call function they are having the callie function of the swap function and then we have like before swap and then the third part is that like what kind of parameter of input that we can actually trigger the address to for the before swap. So that's the third part and then the fourth part will be thinking about given the before swap uh par parameter of input then what is the suspected uh um valid input input for the call function and then we can actually ask the LM to generate it and then we the next step is actually map the collie and caller argument together to generate a valid input. So this is something that I think we can do. And then the next part is actually thinking about complex data uh generation is uh thinking about the I was talking about

the moon hacker having like a core data as an argument. The approach that I do is actually think about you can think of this core data is decoded into three things. We can actually think about another logic of oh that means I can use these three things to encode it to a call data and put it in the function. So this is like one of the things that we can actually generate ask the LM to generate a script to for a function that having for instance like operation m token and then a mind to supply or redeem to com to to generate call data to yeah to actually like start the this kind of compressed data type. The last thing is actually like if you think about um the the the revert message like human read it when we test it we know

there's like some reverse message for instance like this case is like we have like require some uh verify this is like one of the exploit that I I I was checking and then like uh oh it's not eligible then how do we for human we can actually know oh it's not eligible but in the fing process like because it's so fast we can actually when when we read the rever message in the runtime and then we can actually ask the LM to guide it uh what kind of requirement we need and then the proposed argument that of input. So this is something that we can do. So the second layer that we'll be thinking about the state base the three-phase uh CFG guide uh first thing and then the state pool and then some sort of like uh state diversity concept.

This is like quite uh new in the space. So feel free to uh later on challenge me on all this idea as well as always. And so so the first part is actually think about the question we are asking is how do we get into the particular branch we know of scale this particular branch but how do we get into it. So the first thing is actually that we need to understand for the CFG which basic block is in for that particular branch. And then the next things is that when we actually bypass the path we uh into that basic block we have different like jump I condition or like comparison op code and in the comparison of cook you can see that they are taking storage. What kind of storage they are taking? Then we know that by this comparison of cook and then jump I we know that we need to

mutate the storage and or argument. And then in this case this is the storage we need to mutate and then yeah and then the when we observe the argument of storage we can actually see the the distance metrics that I I I I talk about. Then the major challenge of this problem is that a lot of things like uh for the CFG generation like uh any CFG expert also like feel free to to discuss with me as well. And then so that will be like uh infinite loop problem when we generate CFG one function access together then they keep accessing together that will be causing CFG problem and then for loop that will be like a while loop how do we know that like CFG generation keep generating and then the external call that will be difficult because like in the external call we see there will be some in here

that will be a call up call and then the external call we don't know uh inside that uh like what kind of further logic that we have and then the memory usage as well some sort of jump they would use memory to to to help for the uh to help for the comparison to bypass the condition. But we can actually do memory tracing yet memory tracing to actually find find out like what kind of argument or storage that we do and then dynamic storage operation those kind of things are uh using KitKat as the as the memory to find the uh me use the key cap for a memory to find the storage location and use the storage to find the storage value. So those kind of things is something difficulty that we encounter. The next part is actually think about we know that uh we need to

mutate the state but what kind of state we need to fetch. So uh you can think of like uh first of all we store the interesting state similar like ETF first also doing it and then um like some other fer they also do it it and if there's like state change and then that in that sense we need to add to something called state pool a a server that actually storing all the state and the next things is actually like we fed uh we see the slo op code to know that what kind of state we need to take the another is something that state compounding is thinking about like if I want to get into Sorry. Uh if I want to get into a particular branch, we know that the whole path then we know that a set of storage that we need to get in then but

we also have like different state that we generate with the with the core chain. Then we we can actually combine this different state together by re-executing the the the the core chain in order to uh generate more of uh more more diverse state. Yeah. And then the next things is actually think about uh diversity concept. So is that um a lot of things we need to manipulate the state even I get into upscale how do we trigger the particular vulnerability. You can think of this is like two two things constraint optimization or like uh and then like constraint generation when we get into the path we can actually generate different constraint some other uh some other f they actually do it but like they assume the linearity problem. So we we can only like keep uh doing like basically fussing and then

when we bypass it we can actually record the value and those kind of things can be used it to help for the constraint. The next thing is actually think about um the given this constraint how do we actually extend the state base and then pretty much like it's running an optimization problem that uh we can actually do. So the third layer is actually thinking about like the yeah the the GPU uh so GPU execution the high level idea. So basically you can think of like when we plug in a smart contract and then that will be like a a bite we transform it into a basically GPU paralyzer code and then execute on the GPU. You can see like uh one of the MU first like one of the academic paper they do. Yeah. So this is basically my solution and there was some further

discussion then I mean I time this is up and then feel free to come to me and then there will be like a lot of different question that we actually uh can can can talk about like and yeah I mean time is up and then reference list and just come and talk to me. Yeah sorry for speaking that fast. [laughter] Thank you. [applause] >> Yeah, the the time is good. Yeah. Yeah. >> Hello. Thank you for the talk. It was pretty good. I wanted to ask you when

you are thinking about fasting and you know sometimes based on resources because I read we we still do not have quantum fing and we have like infinite paths to what are some key points or some key ideas you use to guide the fast test? you know maybe bound certain values that you that you think or path that would lead to certain bugs. What are some how do you say key techniques you know that you think about like okay I think like if I take the zero value like it's probably you

have set to zero set to the upper limit or you know try to get it to the edges probably what are some techniques or >> I think the core technique I will think about I was talking about the distance metrics when we look at the compression opt code. So you the op code they will have like greater than less than or storage greater than something and in the source code we write it balance greater than something then we know that that compression of code guiding us that if we even we start from a million then the compressor off code is actually telling us we just greater than one and then we know that the distance is actually from million to one then we know we can actually either increase or decrease we decrease it and see the distance whether it's closer then if we

is it closer then we keep going this ression. So this is like some core technique that I think that guiding guiding the fusser to to do it. Yeah, it's like sorry if I keep talking. It's like something used in finance that it's like the less amount of tries like that you you you see okay I have free tries to determine something like and you go like I don't know in a 100 store floor that you would test the nine the 29 and something like that. It's yeah I get the idea. Sorry then I will ask you. No worries.

>> Yeah. Yeah. Yeah. I think that's good. any any other question feel free. Yep. Thank you very much. I mean feel free to actually talk to me and yeah I mean uh like the whole presentation is really for facilitating discussion and that's the purpose of why we actually hold DSS maybe. [laughter] Yeah. Thank you. [applause] >> Thanks again. And now we have a lunch break. So a lunch break. So if you want to talk to the speaker, it's a perfect