Foundry, a portable, fast and modular toolkit for Ethereum application

Transcript

thank you all for coming so today we're going to talk about foundry which is a toolkit we've been building at paradigm with internally and external people with the goal of making ethereum application development easier how this presentation is going to look like is basically some motivation the high level solution some metrics about what we have so far and then i'm gonna just dump a bunch of features that we've built over the last six months just to start with the problem um writing ethereum smart contracts is done in solidity programming language or in viper or in something else but the status quo has been that people

write tests in javascript in typescript in python or in other languages but that is a bit complex it requires learning a lot of apis different libraries it also requires context switching so you start on solidity and then you switch back to typescript and then you switch back to solidity and connect switching costs both in computers but also most importantly in terms of one's focus when you write tests in javascript or typescript or python there's a lot of overhead because when you talk to your application it goes over the wire every time and that ends up being very slow and anytime things take more than 20 seconds they end up taking two or three

or five or ten minutes because i'll tap the twitter and start doom scrolling and then finally the most fundamental thing is that we're building programmable money and turns out security can be quite expensive when you get hacked so we need to test very very well and testing very well requires not only you the developer to be very good at testing but also the tool making it easy for you to test well and in non-smart groudon or non-blockchain context people are like i can test like i don't care because i will patch it or whatever in reality here when the money is lost people cry and so we don't want that this is like a bit of an inside joke

like for the people that know but like js.big number is a big like has been a big headache for many and i'd rather not see it ever again in my life so what's the solution it's basically everything inverted like we change the paradigm of testing into always writing tests and solidity we give nice utilities for writing property based tests which i'll get to in a second we allow hooking on arbitrary vm operations this makes it easier for somebody to explore more code paths in their testing maybe you want to mock an oracle maybe you want to mock like a time lock operation anything you want can do

and you have a crazy optimized pipeline for both compiling with very aggressive caching with very aggressive pruning of the dependency graph but also with a very very fast rusty vm the author of which i think might be in the crowd and we also have like a nice to have which is that we have smaller binaries which are cross platform which makes developing and like just launching new machines and installing the software really easy and this is foundry which is the purpose of today's talk and just to you know show the classic vc hockey stick graph foundry is now the most starred framework of the ones that people use so i don't have truffle in this but i think

it's kind of falling out of trend lately daptools was a predecessor to foundry we're very thankful to the adaptils team for making all this possible and hardhat and brownie most people should know people use this now it's adopted in optimism in synapse in a bunch of other companies like this guy super cycled he made the few million doing meth trading recently or over the last year now he's using foundry metal loans also uses it maker dow uses it 0x i can keep going we have a lot of people using it and the more and more and more like people are adopting it which is very fulfilling we have about 150 open source contributors to foundry most like i

think beyond me and the top three to five contributors like all are just open source contributors just giving back to the tool and using it we have a bunch of repositories a book a standard library templates docs everything you need to get started very easily and uh this is a nice quote from my collaborator t11s where he installed foundry in 15 seconds on the first gcp instance whereas in anything else you know you need to install python download this download that we don't do that here just download you run foundry app and you're done less than 15 seconds on a good internet connection okay now i'll get to some benchmarks because we'll get to the features etc in

a second but just to paint the picture we take two of the most popular repositories that exist today open zeppelin contracts which most developers have used in union swap v3 core the infamous unswap v3 and we can see that on the same code base compiling it with forge foundries testing toolkit we get 1 to 10 or 11x improvements the same thing even more so happens in uni swap v3 core this is for a lot of reasons again because as you have a better caching system you can save compilation times but also just from native overheads that we avoid because we use rust and not javascript

or typescript testing is ridiculously fast blazing fast people say compared to dab tools we are on certain benchmarks 20x other times x which is pretty good on unisof v3 periphery which is another set of popular smart contracts we ported the tests from javascript to solidity we also converted them to fuzzing tests while we were it and we found that 16x improvement in speed so obviously you know i can keep giving you big numbers but the good thing about that is that instead of you spending time waiting for the code to run you're actually doing your job so it saves time from the companies it saves you time it saves money for everyone it's good and on other benchmarks that we run we

found also quite big speed improvements especially against when running fork tests so fork tests are very common when you want to simulate your smart contract state against things that are already deployed on mainnet and we also see pretty crazy speed improvements what's the catch there's no catch it's just very good so now let me get to the meat of how this works you have a smart contract and this is obviously a developer talk so anyone that's not a dev and you got what you needed from it um so we start with just writing our testing solidity the first philosophical difference you write your smart contract and the solidity test looks how you

would expect it you define a function you prefix it with the word test and you add a bunch of assertions so let's say the top right example and you maybe define a setup function because you want to run some initial setup logic you can see the test double function very easily it checks initially that the value is 1 which it should be on the left it doubles it it checks that it's two simple expected it should pass and obviously two times one is two not four so the bottom test should fail and so we use a pattern which was a legacy pattern that we used from daptools so you write test fail

in a way that it will show in a bit you can also do it in a different way so this is a very simple unit test unit test i take a scenario i write it i write some assertion done property based testing is this technique where instead of testing a thing with a certain input so let's say by configuring the thing with the value 2 i can make it run over a broad set of inputs and you know it might be one input might be 100 by 100 000. you configure it it keeps running it gets smarter as it runs now why do you want to do that obviously this is this is a trivial case imagine another trivial case dividing by zero

and maybe you don't have a division by zero test and then you have some unexpected condition we have found many bugs using this and matter of fact auditing firms are like trail of bits have their own internal and external fuzzing tools that people use and the whole principle is that instead of writing a specific scenario you write a property and the property is supposed to cover a bunch of inputs if your property is correct you're good you pass on if not it spits out an output which says this test failed for this input you have a bug which is great now we have you can take that a step further and i realize this this is too small to show but basically um

here you need to write the sequence of calls that will potentially break the property but you cannot always do that maybe your contract is super complex maybe you forgot something maybe you don't understand the system as well as you thought that you do a lot of things can go like wrong frankly so in variant testing or multi-step fuzzing basically what it says is that you say you define an environment you say this should always hold an example would be the uni swap environment x y equals k and you write an environ called x y equals k and then you say find me something that breaks it you see the test has nothing it doesn't have any logic on how to find the call it just says find me something

that breaks it if it cannot find something that breaks it great your contract is secure in this case the foundry fuzzer is smart enough that you can first call set 0 then it can call set one with the right inputs and figure out a way to break this environment which means that you can use that when you're doing security analysis you write your environment if you understand what you're doing and then it just finds you inputs that break it on the right we can see that it's even faster even smarter because our contract we have our environment which you know we want to check that we haven't lost money and what it proceeds to do is that it will generate a set of calls and calls that are not

even in the contract matter of fact what it will do is that it will call hello and you will see that the goal is mal.world and world is not defined we don't know what world is but foundry is smart enough to inject a custom call because you're making a fundamentally unsafe external call it will inject a custom call to backdoor and so it will do hello backdoor closed door so normally if you were writing this you would not be able to find this but it's very smart this is a feature contributed by one of our contributors joshie by the way so shout out now we have another thing called state overrides or cheat codes and cheat codes are what you'd expect you are in god mode you can do whatever the hell you

want so just to give some trivial examples you start with a blog number which is whatever and then you roll roll 10 blocks in the future this is similar to the common hard hat i forget what the api call is and then you modify the block number you can also do other things you can overwrite a nonce you can have a ride a storage slot you can also get the norms you can also sign things so imagine that because you're writing solid tests solidity natively does not have a way for you to sign so we have the cheat code that says sign and what it does is that it makes a native call to rust and it signs and it gives back the result and the solidity keeps interpreting it

so a use case for the last one would be if you're testing ap 712 style signatures we have something which people love which is called call traces so a call trace you have a test that says ken gm gm um and then it proceeds to show you all the calls and matter of fact in this test it also shows in blue calls to the role cheat code shows you the gas that's used shows you the return it shows you if it's static called it shows everything and for invalid tests where they should fail it shows even calls are gonna revert with red not that the bottom test fails because we handle the revert but that's not relevant here

and you know if you want like just just to flex a bit on the speed see how much time the tests take to run inside the system it's like pretty fast not even a millisecond we have interactive debugger so you write forge debug you have a contract and you can step through this step through the code execution you can view the stack you can view the memory you can view every op code being executed you can also view the stack the call traces while it's being executed this is a very powerful feature for security researchers for gas optimizers looking to understand how the system works it's it's insanely good if you're doing reverse engineering if you're doing on-chain analytics if you're trying to understand what's going on under the hood this is the tool that you use to

debug and people have wanted the interactive debugger for years daptools has had the debugger this is a basically one-to-one port of the debugger so again thank you tools big shout out we have gas reports this is self-explanatory any test that you run it gathers the gas metrics and it pretty prints it in a nice way so that you know what you're doing when you're gas optimizing and gas optimizing is important because you save people money and so on and you become well known as a good developer so people have been asking for code coverage like we see this team put up a 20k bitcoin bounty this guy has been harassing us this guy built the coverage feature so now we also have code coverage

so what code coverage does is that it does what code coverage does like it runs all your tests it gathers all the paths that were executed and gives you some nice report so for the people that we're asking for coverage we now have coverage it works like 80 90 of the way there are some edge cases i'll get to that in a bit but on the left you can see how it pretty prints a nice table just for you to get guided and it also prints the so-called alcove report where if you look at the second command being executed over there you do forge coverage report elcov and then we'll give you a nice cover report on your browser which you can then explore and see green red you know code paths that were taken or not why do you use this to check if you have

any untested code because the moment any time i have left code and tested i got i got screwed so not good moving on a feature that did not exist before foundry which i think is non-standard but we're very high conviction that it enables this whole new field which we like to call chain ops this is a term that the tenderly founder said to me first time we use a cheat code which we call broadcast or start broadcast broadcast for one call start broadcast for many and what it proceeds to do in your script so you write forge script arguments blah blah blah it takes the run function and it runs it it runs it locally it simulates it so top right you can see

how it generates it generates a trace this trace the call trace is an example of what would happen had this transaction been executed on chain so on the top it shows the local simulation on the bottom it shows the on-chain simulation and then what it proceeds to do is that for every call in the solidity contract that would get affected by the broadcast chip code it proceeds to submit it on chain um and it prints you hash if it's a contract it brings the contract it prints at which block it was mine the noms the fees everything it has rebroadcasting logic it has yeah it just has reproducing logic it's pretty good the best thing about

this is that you still remain in solidityland while writing your deploy scripts which means that for the first time i think people can finally write tests for their deployments why do people want to write tests for their deployments because how much money has been lost due to an initialized proxies because people didn't write the damn test for their founder for their deploy script so now please write tests for your deploy scripts like no excuses um we have another cheat code called create fork creates a left fork there's a whole family of cheat codes basically instead of doing this on one chain now you can do it on many chains which you know i'm going to deploy my things on many places maybe i want to like run an arb

between two networks i can even do that in a foundry script and how it looks it's like very easy you define the id of the fork you do select fork or create fork and then you select fork and we can see in this trivial test below i could give you a more complex one but not worth the time you see you give it select fork maintenance there's a typo there i did not run it you get the block and the hash you switch to optimism you get the block in the hash and you compare and they're different so something's going on here there was a tweet that went viral a couple of weeks ago from sam from megadow which showed that basically maker dao is now doing all their multi-chain deployments and scripting using this

feature basically using this and this together formatting so many people have used prettier with a prettier config with a what do you call the solidity printer plugin all that stuff like you know when you see a javascript repo it has like 10 dot files below it we don't like that we want one config file which does everything for you it's opinionated we know good devs like we trust the good devs like you know earlier on their career devs can use like these things and they can get like very very good results so forged fmt what it does is very expected you run forge fmt and it formats your code the code formatting is instant contrary to prettier which takes four or five seconds ish on bigger repos it takes

longer the great thing about this is that it does like a little under the hood kind of um opinionated changes like for example it automatically checks some any addresses that exist which is a very annoying solid compiler for people that are used to copy pasting addresses here and there um forge fmt runs instantly which means that you can finally have a formatting save experience so you can configure your editor to always on each save it automatically formats your solidity you never need to deal with trailing spaces again you never need to deal with like misformatted stuff it's it's a it's a hand opener so people know ganache cli people know harder node this is

this is a copy of them but in rust faster easier to run et cetera we have a heartfelt plugin which you can basically use in your repositories maybe i have it here oh yeah exactly i have it here so you do two things yarn add at foundry rs slash hard hat you add one line as you see it here and immediately the hard hat compilation pipeline is replaced with the foundry compilation pipeline which as we saw earlier it can be one to ten times faster pretty good for for free um and instead of running hard that node it runs anvil anvil is shown to be two to

three x faster in processing transactions we benchmarked it on the uni swap and the optimism test suite when we found like two to three x improvements on how fast your tests run drop in replacement no change like no brainer big shout out to the founder contributors the other founder contributors on the book where we have like and like these two gigantic chat rooms which are like very active every day uh oh yeah and next few months basically what we're gonna be doing is that we're mostly done with the features that we wanted like the product works great we don't have anything else to add maybe flame graphs maybe kind of on the margin stuff all the end-to-end flows that we

want people to be using are done in the next couple of weeks we're going to have a v 0.3 release which basically solidifies all the things i said in this presentation because invariants are still not merged in master and released and because multi-chain forking still has some bug fixes in the next two months we're trying to have a stable 1.0 release which will kind of put a line and say okay this software is production ready everybody can and should use it it's fantastic like you should use it um more performance and stability improvements so right now there's a very low hunting fruit optimization we can do in the evm that we're using which will 2x the improvement of all of all

fuzzing tests um for a reason that we can discuss if anybody has a question um and basically it's just going to make it even faster so for 2.0 we don't know yet what we want if you have feedback please share maybe we'll do foundry for start net maybe we'll add viper support maybe i don't know honestly like come find me and ask me um the dogs are there if you haven't used it i urge you to try it you can install it in like maybe with this wi-fi maybe not not so fast but like try it out at home it's going to be 10 seconds run the book try the templates it's super easy here are the links and i have two and a half minutes if there's any

questions [Applause] [Music] [Applause] or do we not do questions hey um so i was wondering um i'm working on a project and i have a lot of tests from a smart contract and i'm doing it in typescript because i need to call an api to generate a transaction parameter more concretely i'm working with

exchange aggregator like one inch or para swap is it possible in forge to call a web api to generate parameters for my transaction yeah so i don't have in the slides but there is another cheat code that's called ffi ffi stands for foreign function interface what it does is that when you call it you give it a shell command which can be anything and it proceeds to make a system called the shell command and feed it back the input so somebody i think uh i forget the name like devanon and another guy they built a library called searle which is curl but serial solidity curl uh which what it does is that it has a nice api for doing get post put

delete whatever request you want so yes uh there is a library you can find it um maybe in the chat rooms or on twitter i i'm gonna like flag it but yes you can okay thanks a lot awesome project um i wanted to ask about dependencies so if you're using like open zeppelin or whatever it comes from npn normally but you guys use get sub modules right um i found that to be

probably the worst part of the developer experience is like translating your project because if you want your existing tests to still work you basically need to dual like both npn and these sub modules so if you've got any like future plans to make that a bit better and like support both so we don't have to throw away our existing test suite for big mono repos if you go to the foundry book there's a page called integrating with hardhat which shows exactly it's a like two-line config change on the foundry tunnel where you say get all my dependencies from the known modules folder and you can configure it that way but also foundry is smart enough that it even if you don't have a sub module installed dependency you can figure out

that the dependency that you're looking for is under node modules so either it should work out of the box or if it doesn't check the foundry book and also we definitely don't want to add node style package management in the system although for 2.0 we might explore a proper package manager inspired by crates and cargo cargo ad but right now like we like the sub modules because they're easy you commit you pin it to a commit it requires some people some basic git knowledge and we've seen people do like really crazy git setups that you know you're pulling your hair but in general you should know git right so

if you know git gets a modular not that big of a pain but i acknowledge that it can be sometimes a concern so yeah check the book thank you very much [Applause]