Reth: A new Rust Ethereum Client

Transcript

[Music] Mic check. Thank you all for coming. This has been a beautiful week. EthC is a very special event for me. In the first ETCC, I was still a student. I had zero money and ECC gave me a €250 euro student ticket to come to attend and that was the beginning of it all. So big thank you to the organizers. [Applause] Last year I presented Foundry. Foundry is a developer tool that many people in

this room maybe use, maybe not use. We don't call names or families. We love everyone. Today I'm here to talk about Reth, a new Ras Ethereum node that we've been building at Paradigm for the last since November October. Over here in the screen, before I get into any exciting slides, you're seeing a few things. Earther scan on the left and a node running on the right. On the top Timox panel you see a lighthouse node. On the sorry on the bottom left panel you're seeing a lighthouse node.

On the top one you're seeing a ref archive node singing at the tip. On the bottom right the ticker that's going up is not some countdown. It's not some bomb. It is the block number going up. And on the left we're seeing Etherscan. So let's start refreshing Ether Scan and see what happens. one breath gale goes eventually to two and so on. So we're trying the tip. It works. Holy Okay, let's talk about it now. Wanted to give this as a sign that we're going to talk about real engineering live in

production that we put a lot of hard work into. Paradigm is a building a new Rust Ethereum node. Ref is built to be modular, contributor friendly, and of course blazing fast. It's built in the Rust programming language, which is a language that empowers developers to build fast, efficient, and reliable code. I don't consider myself a great programmer, but when I discovered Rust, my life changed, and I hope that many others will too. You will find some links there, our chat rooms, my Twitter, find me after for stickers etc.

And the person that stole it from the cafe, we will find them. I don't know what the right pronunciation is. We make decisions by not consensus necessarily, but by or by committee. We have different opinions. I call it ref. Anybody is free to call it whatever they want. Again, we won't hold anyone to any accountability here. In this talk, we're going to talk about why, what, when, what on top, and what's next.

And I hope that this is going to be an informative talk and one that will hopefully pave the new wave of Ethereum infrastructure. First and foremost, we are all Ethereum developers and what is happening is that the world is changing. The world is evolving and so is Ethereum. Here we are seeing Vital Buterine's very famous or infamous road map for Ethereum. The Ethereum road map involves more and more and more and more complex and simple. Yet time is finite and things are happening and we don't have infinite resources.

We also like to have client diversity. Why? Because the Ethereum protocol is built in a way that if 66% of the validator set which is now running an execution node suffers from a critical bug or yet worse is compromised, the network will finalize a bad checkpoint and that is catastrophic. So it is critical for the Ethereum network's health that there is no node that has over 66%. There was a very noble work by a prison where they literally encouraged people to run competitive nodes. That was beautiful. Um the goal of Reth is not to

overtake GU or anything else. Peter's team is the team that brought us all here. Is the team that gave us jobs, you know. So Peter's team deserves all the support it can get. We're not going to be number one client. We'll be happy with number two, number three, whatever. We want to provide a new wave of Ethereum infrastructure. Here's like the thing that actually bothers me though. This is how it should be. There should be a great code base with great abstractions in the middle and everything plugging onto it.

Instead, we have this. A lot of people maintain get forks, some better, some Frankenstein, some you don't want to see them ever again in your life. We want to get out of that business. We want to get to a world where instead of having endless forks, we have some perhaps because it's going to be hard to agree of course on one thing, but some varying implementations with clear abstractions, clear interfaces, clear tests, clear benchmarks on every one of these components that can be mixed and matched to build new EVME ccentric infrastructure.

The thing that bothers me actually the most though is that people think that this is hard and that this is something special that only talented people do. That's not right. That's not how it should be. However, um this has been a system that has been developed for a long time and uh as the protocol has been evolving like we cannot always do the best engineering from the beginning and we cannot always figure out the best abstractions. We have deadlines to meet and you know and we have families to meet and you know not everyone can be over the node 247 to make sure that the person like me like 5 years later will be complaining. So you know no shame yet we want to see can we do better than

this performance is something that I deeply care about. Archive nodes are a critical piece of infrastructure for the ecosystem and most importantly they allow everything to work. Every piece of historical infrastructure indexers etherscan anything, anything you can imagine literally that has some notion of historical data needs an archive node. Archive nodes do not need to be expensive and slow to sync.

Archive nodes are a piece of technology that is critical for the future of the industry. Secondly, the rollups road map demands more performance on on the edges. Current nodes can support more performance because contrary to what one might say, it's not the CPU or the whatever that is bottlenecking you. It is a state growth and the state growth is the big enemy here. However, one could say that let's try to do better in this vertical also. And I will not talk about the third and the third fourth bullet point. It's kind of obvious why these are useful.

Comes that question then. So that is all the motivation so far. So we spent like eight minutes talking about the motivation. Let's talk a bit about the principle around how we approach the problem. We thought we went to the drawing board and we said what do we build? I thought let's do a node and on the node like I'm not a node developer. I don't know anything about nodes. I didn't know anything about know anything about nodes until we started it. I know dev tools though. So we're like okay let's give it a a fresh look from the dev toolcentric way. So we built rest to become very modular very easy to consume a thing that can become the backbone of the new wave of ethereum infrastructure. We also want to give

back to Ethereum. Again, Ethereum has given me everything and I want to give back to it and this is like hopefully one small contribution. So to the meat of the conversation, Reth is an Apache MIT licensed Ethereum execution layer written in Rust. It is driven by the open source team that shipped Foundry and is going to be shipping a lot of many great things. We have some people from the team in the audience, Roman, maybe Alexi as well. And I want to spend a minute to talk about our culture. The culture is open source first. It's inclusive mentoring and fully async.

What does this mean? That even though we have a core team of very very very strong engineers, the most talented engineers I've ever worked with. We have no ego. We talk to everyone in the community and we mentor them handtoand. There was one guy like that said day one journey of learning Rust and uh we jokingly said on day three red contributor by day 10. He was the contributor by day eight or nine. Thrill the rust meme turns out it's happening. Beautiful. The async first component is something that I learned from working at paradigm on the investing side which was something about having very very low tolerance for noise.

The synchronous communication is like a mutx on your calendar while everything should be flowing like a channel like an async stream. So our culture is like heavily built on async written communication, very clearly communicated, low ego, close every feedback loop. And that is what truly enables us to build great software. We're not geniuses. It exists. I show it at the beginning of the presentation. This is running on a big box. Feel free to hammer it right now. Open an issue if it breaks.

That is how we operate. We share our code. We're not afraid to share experimental code. We invite people to test our experimental code. Foundry infamously was on nightly, still is, and we screwed up the V1 release. We'll talk about that another time. We have a Graphana board here which has all the Prometheus metrics exported by the rest node um that you're seeing above it. And the current tag is alpha 4. It got released on Monday. I'm going to talk about the new releases in the future. So what right um so eight people stood up a new node in eight or nine months we have a 100 plus open source contributors of various size obviously

it's very front-loaded to us but like what we want to do is that we want to make create the next 200 300 400 500 core devs you know not application not smart contrade devs core devs people that will make the ecosystem antifragile and real for the long The node has a very attractive performance characteristic which I'll talk about in a second. It has basically full Etheum JSON RPC. Here I have a link to a tracking issue that my colleague Matt sites created which basically is a checklist of all the RPCs supported and has support for both gets debug tracing rewritten from scratch in Rust including the JavaScript tracer which some people love some people hate I hear and the

parity trace module following the spec well no spec but like following the initial implementation like as much as we could actually and the last bullet like I'll talk about in a second. 359 pull requests have been merged the last 30 days. 135 contributors, 2,000 stars, four releases. The first eight devs in this picture are named Matt Sites, Roman Kasuk, Oliver Norberg, Dan Klein, Dragon Arit, Alexi Shakirin, and our ankon Joshi. They're the best RS team I've ever worked with and big shout out to them for dealing with me on a day-to-day basis.

[Applause] The feedback loops couldn't get tighter. Maybe not. We want more feedback. Please give it to us. Be on that list. People are starting to get hired out of our code bases. People come into our chat rooms and they snoop. Competitors hire out of our chat rooms. We give it for free. We have first look on the on the best ones hopefully. I will not name node two in this uh slide. There's no point in creating uh any conflict. And the upper numbers are a bit easier to measure. The lower numbers there's like some nuance and I don't want to be too strongly opinionated.

The numbers what you see up is the at block 17.4 million. That might be a month ago. You get it? It doesn't scale that much like in a month. Wrath achieves best-in-class singing speed. We tested on latitude on bare metal servers on Samsung EVO devices with very very high IOPS. That's the bottleneck when you're singing a node. So, we chose to use the best hardware to get the best output. And I want to caveat that for less good hardware, the gap might be less big, but we're going to work on keeping it like to be as good as it is now.

the database size. I think honestly the difference with the air going to be it doesn't really matter like if you're like a that much like it matters like to be on the order of magnitude you know two to four 4 to 8 8 to 16 and so on yet another never mind they have the try structure built in to the database which means that they incur significant overheads for historical nodes caveing that this comes at trade-offs for like how well you can operate at the tip we're trying to do both TBD if we'll be able to do The initial goal for us was to build an archive node and we're talking about archive nodes today. Just want to be clear on the slides below these three things and I will explain how these charts were

generated because we take a scientific feedback loop driven approach to all our charts. Um you see a throughput chart which means that input output one request in one request out. Breath scales linearly. Success rate means that you keep hammering the node and it keeps responding successfully. Notice how it is not any small number. It's 16,000 requests a second. It's a lot. Latency matters also for any customer. So it also performs very well there. Um for the people that will ask later, the spikes we don't know yet what they are.

Like obviously latency should be going up as you're like um increasing the load. might have to do with caching. It might have to do with other things. We're still investigating how to get make this charts better. Any data analyst or data scientist or somebody that's very experienced in load testing, please reach out. This is made possible by a tool called flood that my colleague Storm built for ref. We were like, we need to get numbers for this. So, how are we going to do it? It's also hilarious. Storm builds flood like like that's his real name. Um so flood is a tool for load testing nodes at the Jcurve inflection point when latency and throughput start to come into contrast with each other

and that's the only place where you need to measure. The only thing that we need that we know how to do it like these open source projects is measure measured measure. Anybody that doesn't test at the Jcurve inflection point of the thing is giving you false numbers. And that's why we build a new tool. And that's what I'm trying to also get to here that like we didn't invent any new algorithm. We didn't do anything crazy. We had good abstractions and um we just applied good software engineering principles from scratch. You know that's how profiling and benchmarking the feedback loop. Big shout out to Aragon for the stage sync architecture. It really opened everyone's eyes to what can happen and to the flat DB design which

had deviated at this point but still a big part of it like has overlap and like they're like the the best people that like could create this. It's built on Rev which is the library that Dragon Rakita like our colleague built which is also the AVM that parts Foundry and I think all of ME infrastructure or anyone that's serious about it um today and the Ras programming language we apply a defense in depth approach uh I'm realizing I have seven minutes so I'm not going to go through all of these but switches model of security you go through everything you apply everything that you can standard security practices you If you thought solidity auditing was like something, do this.

The action item from like this group, this audience is help us make the node great. Currently, their nodes tracks 20 nodes. I don't know if that's the case. If it is the case, we got to do better. Um, but we don't have telemetry and we have no way to find out more about it. We're happy to handhold people. We work with Latitude who will be creating a one-click deploy solution and we also give discounts to people to run on bare metal. Literally if you start seeing a node on latitude right now call me in 50 hours and we have a node and we can talk about it. Crossplatform priority very very important. We also want to support ARM devices. We won't support any device. I studied Lego engineering. I care about my Raspberry Pi you know and

the hardware. The only thing that matters is the disk. less cores on the CPU because we're doing a single threaded process which is the AVM. Not fundamental. We'll talk about it in a second. Memory usage 16 GB. We're going to make it lower. It's not fundamental either. It's just more tuning. Yeah, I think that's it. Okay. Now, let's open up a bit the design space. We've talked about the node, but we've said it's modular. What does modular even mean? The node is the first application. We're building EVMcentric infra and we're creating great abstractions for it.

This is what you can do with RET. And not just that, this is like the entire crate. Reth has like 30 crates in it. Every crate is carefully designed, careful interfaces, careful tests, careful benchmarks for everything. It's built to be hacked on. And I will do a community showcase just to show how powerful this is and that I'm not like selling you anything. So why are we extracting data over JSON RPC when we have a database available to us locally when performance is important? Why are we not talking to the file? Like it's like it's like spinning up a JSON RPC for SQLite. Are you kidding me? So these guys um Van

Beethoven and his colleague who I forget his name big shout out they went and built this Ethers. So ethers RS is a library that I wrote three four years ago for like Ethereum in Rust of course and basically the one key takeaway to have here is that like on local host HTTP IPC which itself you would expect that IPC forking would be like gigafast then you do it directly to the database you get another 2x two to 3x that's not trivial you're like 2 to 3x over something that's already super optimized like people are going to make money off of this Python restb bindings. So, you know, we're rust maxes, sure, but like Python is amazing. And like the same thing that like happens in numpy, for example,

where numpy calls out to C code under the hood, we want to do here. We want to have Python bindings for the entire Ref ecosystem. Any concept that exists in Ref should be that ideally has to do with data science because that's like one core segment that we're after should be using some Python bindings. This is like very PC, but we want to like have something that's good. Um, I some people like thought that this statement was a bit strong, but basically we have a benchmark. So there was Joe Stevens who works at a heck on a weekend without any input from us, a rest indexer that talks to the database

directly and achieved incredible results over what u many people in the industry use. And again, don't want to be causing any conflicts, but you know, numbers talk and I hope that this is something exciting and that we can all like start using more of it to make all our infrastructure better. This is something used in me. We don't need to talk too much about it, but basically we have high quality inspectors that are used for how this to operate the node. And it also does this. Um, again, MEV, I don't know if we have any builders here. We have an amazing trade abstraction. Um, and no access to builder code because that's proprietary.

Like I don't even have access to it. But basically, if you're considering building a builder, we have a good abstraction that's let that lets you like build it just by thinking about the business logic. I have a buffer of transactions. I need to order them in some way. I need to pick the best bundle of all like not a again not rocket science. This is like pretty nice. Um, P2P entries without having to run an entire node. Pretty powerful. You can expand your P2P coverage. I'll speed up a bit because I'm running out of time. Red RPC is also reused. Same thing. And now let's talk a bit about what's next.

Stability and growing the the contributor set. Um self-explanatory. We want to create the next 100 to 100 400 500 core devs. The next feature that we're going to be shipping is uh the RPC calls uh sorry is the pruning and full node. So right now red is an archive node. We're going to become also a full node. Why validators? We want validators to be able to run it without needing like a three terabyte database because two terabytes plus like Lighthouse and R200. EVM parallelization and I optimizations contrary to what people might tell you are very possible. We have solid ideas.

We're going to work on them. Um Cancun we're starting to work on it August 1st. We actually have a offsite this week where like some of these things will be scoped and we also started working with the base team and the optimism monop and we also hope to have something on this by the end of August. I don't know what product means. So I went on Twitter as any lazy investor would do um and ask how what does it even mean? You know Peter said the right answer. G is production ready. get is the crown jewel here. So we have a high bar to meet. How will we mitate?

Performance improvements by making sure that the database doesn't break by making sure that we can keep tracking the tip reliably. The list keeps going. If you have ideas on how we can call it pro ready and we can have datadriven, not narrative driven, datadriven things to call it, please reach out. Thank you. [Applause] One one question.

Who is the confident one? Uh hey uh thank you for your talk. Uh I just wanted to ask uh when you were implementing RPC for ref so you said that you try to be as close to the spec as possible but how uh can you share how do you like what spec do you use because like currently if you use Ethereum API spec it's uh like lacks a lot of methods and uh from my experience personally if you want to understand what RPC currently have what capabilities RPC

seek on the you only way is to go to you know arroggon source code or get source curse do you have some kind of your internal specification that you try to maintain no it's a lot of work we want to contribute to execution APIs and specs because as you're saying it's a problem we indeed um have to like get is the spec sometimes you know um that's fine bitcoin core is bitcoin spec so I don't really mind uh it does make the job a little harder the good thing about the Ras programming language I will stop after this is uh that um if we define the types well we know we find the spec deviations very easily when we peer with others so like in ethers for example like when somebody when we integrate

with like five different RPC services everybody would go off spec they wouldn't even put like JSON RPC ID in them you know so like stuff like that like we catch but yeah acknowledging it is a problem no we don't have a great internal process it's a lot of work we also use hive for crossclient testing. So of course like we try to if we all pass what they pass probably good. Thank you. Done. Thank you all. [Music]

[Music]