Ream: A new consensus client's journey to Fusaka mainnet

Transcript

[music] presentation. So please about dream uh take it take it away. >> Okay. So I will start now. Good afternoon everyone. So hope everyone here doing great. So I'm Daniel and uh this is Hans and together with Tosin but uh he cannot make it here due to the visa visa and we are EPF cohort uh seven fellows uh who working on dream project and uh our project that we present today will be our journey to make rim become a new beacon consensus client that be able to join the ether mainet and uh which

supports uh fully fullu specification uh for the faka road map. So I will start with the starting point. Okay. So uh at the start when we start to join the project uh rim is just lin client and it not fully uh not fully electra uh client. So because of no fulu support so we cannot test it out our rim client uh to join the ATM mainet because the peers will res uh because different of the fork digest and uh we can only test it on local devnet where we can run

multiple rim clients. So to make that work we have divided into five way and here is the first three phase. The first ph will be close full spec gap where we fill up all the missing function uh compared to the consensus specs and the next ph will be fully custody pad when we try to implement the rim client to be a super node that support pas uh which is a concept that that I will tell you uh later on and alongside with that we implement the ring because uh rim is really lack of a lot of functionality.

And on the phase three when we finish rinsing and full custody padas we start to do the ctoet interop when we run with multiple client like prism or lighthouse and during that part we also do the back fusing and uh on the two last ph uh which is phase four we do the custody mode and the last play we will try to run our node with e internet and on the hoodie test net.

Okay. So let's go to the path that we make re fu compatibility. So in fulu uh we have three eibs. The main one will be pas that now a nodes will not store the whole blob data. Instead, it will store the columns and when it receive half of the column, it can use an algorithm called uh eraser coding to reconstruct the whole data. So let's look at it uh more detail.

So here is the picture. Uh so before fulu uh we can see that uh layer 2's will post their data with include the transactions on layer 2 uh through uh a transaction named blob transaction and those transaction will be included in the block ethereum and at that time uh every node will store the whole blobs data. Here you can see that every validator will store a block that include a lot of blobs here and it uh create a highly data application and uh for the pas you can see that the

block going through another process called eraser code to create extended path. uh that it will be the redundant part that I will talk to you later on and those datas will be divided into columns and validator will store it. So right now we have two tie a valid validator uh sorry I mean note uh the the first one is super node who store the whole 128 columns and the other one will be custody note who store only is duty columns uh which is only eight columns.

Okay. So let's go to the eraser code part. So you can see that the block is divided into uh columns and now apply eraser code to create more redundancy path. So now you can see that we have eight columns and even two of eight column is missing and we can still reconstruct the missing part and get the original data. So to make it more beginner friendly. So let's take a look is as a function uh y = 2 * x + 1. So during that line we have two point codes three uh which mean with

x = 1 e = 3 and the next one will be 25 and we can uh find another two point which is 10 21 and 12 25. So even now if we lose 1 three or lose 25 we can do with 1 3 and 10 21 and do the equation sol solving and get the function back. So that is the main idea of the eraser coding in simple term.

Okay. So the next EIP will be BO4. So that is just small EIP. So when minet apply the busaka uh specification it will need some time uh based on the epoch to increase the amount of blobs that can be included inside one block in eum. So you can see on the figure in Mayet is start to grow from 9 to 15 in December 2025 to uh 21 in January uh 2026. [snorts] So the last last one will be proposal

look ahead. So uh because uh the balance change and uh uh before we have this EIP the balance uh I mean effective balance is still changing. So we cannot premp compute the pro proposer for neck epoch. So that uh that uh EIP will help us to do that. Uh so in one epoch is will calculate the premputed uh neck proposer.

Okay. So let's go to our phase. Uh our first phase will be close the full speed gap where we implement the missing function. Let's take one example here is get data column side from block. So from below uh we can start to get the uh columns uh and have the proposal to create block with the columns of for the last one uh is the matrix reconstruction. So now uh as the example uh when we have only six columns we can reconstruct back to get eight columns as example.

Okay. So let's go to the phase uh 2.1. So we can see that uh what dream was missing. So the first part will be proposer. So when we when proposer uh build a block and is brass cut the block into the peers but [snorts] uh it will not um also actually it also build the column cycle but it never bras cut those column into the uh network layer. And uh the next part uh rem is missing uh is the block uh process flow. So when a block arrive uh it should check the

parent of the plot to see that whether that uh parent exit. If not, it should wait for the parents to arrive and the cube block will be pending. And when it has the parent, it will wait for the column that will be the data column from PT. Uh so which is the data that I mentioned that is a data layer to upload into layer one. So it will wait for the all the column come and once it get them it import into the fork choice but uh in reality at the time we work uh remainn't do that one so if the parent

block is unknown is just drop the block and even if column is not available is also drop the block and column columns ejected from the database. So only one uh it implement the wrong logic. So only one column is valid is consider that block is also valid and this is wrong one. So so to fill those gaps uh we implement the new model called data availability checker. So when the block arrive uh it will check whether the columns have

arrived or not. If not it will wait for the column and also support for the request the missing column and when the block come you can see the uh third one on the unknown parent lookup. So when a block arrive if its parent is not exist uh is also help us to fetch the parent block. Okay. So our next plan will be the wrenching and will be uh presented by Hans. Okay. Thank you. Morning. Uh good afternoon guys. Uh I will continue it.

Uh before we received the project uh we used to plan uh implement all the EIPs on the full uh specification as mentioned there are trees but when we trying to running it with SEO I mean running singing data with SEOA notes we realized that uh there is some problem with the singing flow on dream Um yeah so basically there's um four uh errors uh lead uh let rim node cannot synchronize with other peers on the sepoleia

uh network. The first is the checkpoints uh sync uh it cannot fetch the blobs uh through the full uh preflow APIs. I mean the codebase of rim is support only preflow APIs. Now uh before we implement uh the next is uh beacon client cannot trying to uh handshake with other peers um due to uh the wrong h and ignore the uh second EIP the block parameter only uh EIP. Uh and the

third is uh beacon client on rim cannot uh trying to forward uh singing uh towards head um block uh due to it only stop uh on the finalized uh blocks and uh so as a result uh dream beacon client stop the singing flow with zero block and stuck at the finalized um checkpoint.

So basically what we implement here is fulfill all the singing flow needed uh for a beacon client uh based on what we compare the implementation between uh lighthouse prism and others clients nowadays. Um and um we realized that that uh there are four main flow for singing in a rim note. Uh it rings things back field things just back field and the block lookup. Um the passive one is the block lookup. Uh and the main uh the remain trees singing flow is the

active. Um basically the ring sync will cover the singing flow from checkpoint to the head [snorts] and the back field will uh fulfill the missing block uh to the genesis. [snorts] Uh and the custody backfield will trying to back field all the columns needed for the beacon client uh to custody in 18 days. So uh what we implement in the rings flow uh rim al rim used to implement this flow but uh not uh enough uh because um on the civilia we realize

that there is zero blocks downloaded uh when we run the ring syncs and because the uh the at uh block is fetched uh is fetched based on other peers instead of uh its own clocks. So is um store wrong stat for the beacon client and uh uh other than that we have uh uh below uh those error like uh the mark head slot or of peers uh and we ban all the peers and um uh we receive no because of they

respond no data and we don't have the data column fetched after full specification and uh the missing parents uh after range uh the beacon client drop immediately. Uh so we fix all of them and um making rinsings uh become um uh available on dream client and um so after this phase uh dream beacon client can singing forward uh head to against other peers on the sepoleia and um beside that we also implement uh the ktosis integration to make sure that

all we implement on rim is correct. Uh be um because the integration test is the the most um reliable test when we trying to run against uh in a network. Uh so basically when we trying to uh run the kasis we also met some errors uh from uh uh beacon client when it connect to the e or the vc also and this the flow uh while fetching data columns and some hardcode param in the codebase and uh the last uh step is oh sorry um

And that is the result of uh ktosis uh run against uh lighthouse and prism node uh and all the test plus uh rim can reach to the head uh block in um minimum 20 minutes uh against other peer. And uh the last step we implement in the singing flow is the bike fusing. uh uh this flow we will uh rim didn't have uh back filing flow before we implemented it and >> [snorts] >> uh we fulfill this singing flow to making rim can rim node can cover any requests of other peers when they want

to request uh the previous block data. Uh so this is uh some result and some error roadway fixes and uh the custody mode uh turning back to Daniel. >> Okay. So on the phase four we implement the custody note mode. So we can have the node uh that store only is UT UT column which is columns instead of 128 columns. So we have to implement some feature like from the node id we can compute the compute is duty uh columns uh that we based on the

consensus specification and uh for the we can need to f we we also need to find and pick peers uh by columns uh so we don't don't keep the connection with all peers so and we can still uh have enough beer to get the data columns. So you can see the result it go down from 900 to 20 connected pier on mainet and um and uh also we make the custody uh as the default mode when we run the rim client. So I will go a bit quick uh after this phase. Uh it will be our

phase on running on uh uh on real environment. So this is the first ph we run with SEO minute and we also realize that rim has another problem uh which is it imp imports 1 minute block uh which is longer than 12 uh 12 uh 12 second which is the block time of so for sure that even after the ranging the note still fall behind the head and cannot catch up with the main head. So to make it work uh I have to do a bunch of uh we have to do a bunch of uh uh fix uh the first one will be uh for example in

dream uh once it receive one attitation uh and it will do the uh signature check uh for 27 keys on every block. So every time it check it decompress the key uh validator and check the key. So it is really slow and it is uh contain a lot of duplicated work. So we need to catch them. So from that one we can go from 1 second to 4 millisecond.

So the next one will be the committee lookup for attitation. So on handling it attachation it can all the 2 million and four uh two.4 million validators and suffer again but in reality it only need to suffer once per epoch and other fix. So we can get the state root from 3 second to 0.04 04 second and [snorts] state transition from 0.54 second to 0.06 seconds and here is also other fake and I will skip it also the fake for the column

workers so we can speed up uh this one. So the next part uh so here is overall what I have to fix and the performance that we have achieved. So actually uh you guys can go to the slide and uh uh take take more closer look at at it later on. So here is the phone result when we run with sum minute uh the the starting point will be uh 18 uh.5 seconds before the fix and today uh when we reach we have the block permit uh in median time will be one 1.98 second. So that is the v video demo. Oh

okay. So we have build the dashboard and run it with the assume minute. So here you can see that we received the new block and it handle immediately and that our work on the performance. So [snorts] let's take a look a bit. So you can see from the line here uh even we have the commit time will be uh more than 3 second I believe here uh 3.9 seconds but the block land at uh 3.69 second uh which mean uh after 3.69 second we just receive uh the block that we need to import uh on our fork choice.

So it is the latency related to our server. Uh and uh when it received the block, it handle uh it process really pretty quick in in uh I think 200 millisecond and was at the head at 3.9 seconds. So this is our dashboard that I do the overall about one day uh what is the anatic data of our rim client when running it with mainet. So overall uh we handle block at uh uh 215

millisecond to import the block into the fork choice and we also don't need to wait for the stall up right now and a lot of uh uh optimization in the performance. So yeah I will go through this part and this is our part when we run rim on hood test net because we want to test also. Okay right now our client can run uh with the full node mode on mainet. So whether it can support for the validator client. So we run it on the hood test net and h will finish.

Okay. Um on the hoodie test net we done we run the phone validator. Uh we use uh validator clients of uh lighthouse uh run against uh reimon client reimon node uh and alongside we run the validator clients for uh I think it's about 9 days uh before eicon >> [snorts] >> uh we fix um a lot of small errors related to um request and respond from validator clients uh like testing aggregating signatures. Uh so that's uh

and uh here is the demo one vote on uh through rim uh beacon node. Uh I use uh also use the dashboard to uh uh represent all the step between re beacon node and the lighthouse validator clients. Uh and uh also some demo from uh uh the ring sync flow and the back fus sync flow when uh the rim node uh crash or goes down and coming back to the network and uh this is speed up um video when the rim node stopped and it

uh needs some time to start it. some second to finding pierce and start rinsing flow until uh uh it following the head check and uh also uh the back fields is successfully um in the background uh to uh flow like a block back field uh to an concurrent checkpoint and uh the uh custody back field uh fulfill all the 18 days columns data needed uh for be client to custody.

So uh the last uh last but not least uh what we going to do next uh we intend to uh trying to make rim client [snorts] uh catch up with Glamsterdam and Higgot upgrade also um lean in the future. Uh but uh for after EPF we only have three member in this project. >> [snorts] >> Um it would be great if uh you guys can join us and contribute with us and uh making uh REI become true uh Ethereum clients on the mainet in the future. Um uh and last thank you Mario.

[music]