Light Client Backfill in Prysm

Transcript

We should have the stream going here. Very nice. Very nice. Okay, welcome everyone. Welcome to the EPF day. Um, it's been uh it's been a while and we finally meeting here. Defcon Mumbai EPF day. It's an honor to have all of you here. Thank you so much for coming. We made it guys. Mumbai, come on. Let's just cheer. Yeah. Awesome. Nice. Nice. So, um, yeah, thank you so much. I mean, it's been five incredible months we've been here together online and, um, it's just, yeah, incredible to meet you all in person. Uh, just to put the faces to the GitHub handles and profile pictures and, uh, today we are here to celebrate

you, your work, to just enjoy what we did after 5 months of hard work. Uh basically we've been waiting for Libra Office to download because this is Libra Office presentation. So online it renders a bit small but I hope you all can see it all the way in the back there. Um it's not so many people it's just here. Um so just to just to give you some overview some statistics because it was a it was a lively cohort. It was uh it was a fun cohort and uh yeah it's been five months uh and as you know there have been many people uh many people are excited to join the cohort.

uh many people from all around the world come but uh we get almost 50 people 49 people who actually registered at least one update in the first table who actually came to these early calls. We get like even more we get like 70 people on the first call maybe. So the last couple of 20 of you here are those who actually made it towards the end. Um or I mean at least 38 made it to the second phase there. Uh but you guys are here uh after all of that. Uh I'm very sorry about the folks. I'm sending regards to home for those who couldn't make it. Uh we have a couple fellows who really made great projects and couldn't arrive here.

They presented online but today you will present uh rest of the projects um uh here. So we got 31 projects that came uh over 20 will be presented today. Uh but of course there u there was even more of them and these these uh projects have been documented in five more than 500 are still coming sometimes. So, more than 500 updates, guys. I read all of them. I was I mean, I tried at least like sometimes I skim through it to be honest when I see it's good like bro's doing well. I'm tired. But I did look through all of them. Uh even if I did not read every sentence, but I did it.

Yeah. And it was it was great. I I always enjoyed every week just to go through uh just see everything you guys are doing the progress and uh try to just keep up with every project because it's been a lot of work a lot of progress uh we did 15 office hours calls which were mostly AMAs we get the project presentations in the end we got bunch of great guests I hope they inspired you to work on the core protocol every week so for like the 18 weeks we had a standup and uh we did we did our updates. Uh we updated each other and um and yeah, it's been a it's been an incredible cohort. Uh now after 5 months meeting here, uh the question

is what's next? What now? And now after you finish your cohort, I want to remind you submit your freaking final report, please. Some of them Yeah, some of you still did not do it. Now it's on me. I did not read through through the PRs yet. I've been busy before coming to Defcon. So, uh there is a lot of open PRs with the final reports. I'm looking forward to read and uh for those who did not submit the PR, you still have time. Uh do it. Please do it for yourself. do it something you know your own assessment to just think about what you did through the cohort and it's something that you can show the people who will work with you in the future that this is everything I did in in this program and they can see the way you work you think and I think it's a great

great thing to uh have ready uh we have the presentation live as we are streaming right now so please if you have family friends if you have colleagues people from your project from your team who couldn't make it to Mumbai you can share them the stream and they can watch you live right now. They can even ask questions if they type it in comments. So maybe we will get some more online questions and uh please um uh uh meet more people here at Defcon. We are bringing you here just not to meet each other but to meet the whole Ethereum ecosystem and the core protocol ecosystem. There is so many people here from all the different client teams from all the different uh communities. uh if you're looking for opportunities something that uh where you can apply

the knowledge that you gained during the EPF you can meet the people right here I see couple people from client teams in the audience as well and there's there are more of them around the venue so please go bother them ask them crazy questions technical stuff just impress them with your work and your knowledge I'm sure they will love it and um yeah we will have an alumni group I'm going to add all of you to EPF alumni group where you can meet all the previous fellows and that's work further or where we're going to share some updates if anything is happening. I will invite you to future EPF events same as there are some former EPF people right here and uh yeah please continue contributing like this is not the end this is just the start of your core protocol journey it's been an intense start but you can

continue with your project you can follow up on everything uh you did and uh yeah you can you can uh uh you can uh uh stay active just so people see you and uh they will keep noticing your work and uh and Yeah, enjoy Mumbai. Um, I hope you love the city. I hope you love the Defcon here. It's again honor that I could bring so many of you here. And that's the last thing I wanted to say actually because this year um the EPF we've been doing it uh for started by Piper MCAB, you know the history. Uh but after after two cohorts, it was taken over by me and Josh that we worked on this for last three four years and um yeah, four years actually. Yeah. Uh now

it's been exactly four years we started because we started the first one at Defcon Bata and uh uh we started this one together and it was a tough one because for 2026 our EPF budget got cut by 70%. Uh so EF been going through uh uh just cuts and we had to figure this experience for you as best as we could on the limited budget and I mean we've managed we are all here and you did guys incredible work so it worked out and I'm happy about that. Uh but we lost Josh along the way. George had to leave uh like June July this year. So the last 3 months or what was it four months he's been without him and uh the pro the court wouldn't start without him uh because the the court been beginning to

interview dozens of people and review 500 application is the maybe toughest part of organizing this and we did all of this together and we started the cohort we prepared everything and then he had to leave. So um yeah and it's been such an honor to work work with him and it's been tough to organize this just for me. I mean, that's why we had to skip some calls and we had to just like that's why some of you couldn't get the came to the hotel and you found out you have no room because I I kind of up the reservation because with Josh we were like pilot and cross pilot and and first officer always cross-checking each other. So um yeah please give it up for Josh and uh yeah enjoy Mumbai guys. Yeah um yeah too bad he couldn't be here. uh but now it's um it's the time to

actually move towards the towards the presentation. So sorry we are running late and we already are past the first presentation. I mean it was the the the laptop and the you know free software always running us late. So yeah uh we got light client backfield Imprism Jeff as we know him here uh colloquially. Thank you so much for being here. The stage is yours man. >> Hi thank you. Thank you Mario. All right. So, I'm Jeff and my presentation will be like sharing about the progress the the learning of my project and the like climb back field in prison and special thanks to my mentors Ethan and Bassin. They helped me a lot with the

project. Yeah. So first of all I would like to talk about like why we need this like the original idea was like uh we want to achieve the centralized checkpoint sync because currently we are having a kind of centralized checkpoint sync process. We get from the trusted uh checkpoint and we just verify the state and the route if they are the same or not. And this is kind kind of weird because you don't just trust you have to verify right and this is how the process work for the light claim protocol and so you can see it's uh you start from the

trusted block routt and you get the bootstrap and you update with the uh you verify with the light client update for each period and then uh you keep doing that until you get the recent final analyze header and then you can start from the beacon stage from like any node and you just check if it is uh equal to the stage root. Yeah. And the idea now uh is that so why we need the back field for like client is because uh so like client update actually needs your post states right uh we can do the block back

because it just need to verify some from hash tree roots but you cannot uh verify loads without any state and with the like the full node or the node running on the P2P you don't have the post state. So that is the reason we want to have a protocol to start from the checkpoint and we we back fill our LLAN updates with this protocol and so the backfield will be one of the parts not all of the part but just one of part to get the data availability for the lite clan updates.

All right. So, so Ethan actually write the spec and it iterates several times and then finally we got this kind of spec here. So we just the idea is just to verify the cananical chain. So we have the block data on the right side and uh in each and also we have the EA data from the left side and the EPA data will include the block data like it starts from the uh E minus one + one to the the first lot of nest epoch. So uh

as you can see here it is just you can start from the trusted roots and then you get the EPA data and it gives you like 32 blocks data and then you can start from the from this one and then for this field you are able to like construct the the root and then you can construct the block route and then you can verify with the parent block header with it with its root and you just verify if it is the same from the this block. Yeah.

And it could actually form a chain a trust chain from here to here and in the end you are able to uh verify if your route in the beginning is correct or not. So this project is kind of uh we have to achieve with different rows here and so the first one will be the supplier and the second one will be the receiver. The supplier is mostly about the data availability for the for the EPA data because we have the new data type and the block data and the EPA data. So uh in our node currently we don't have

those data and we have to kind of start from genesis and get those data collect into your node so that it is possible to serve with the endpoint like P2P or REST API to uh give the other nodes the data and on the right side the receiver is just I would say it's more like a verifier and run in the it's best update algorithm to get the best update for each period and then just check everything and verify everything and then it could uh verify that the data is correct or not.

And for my project actually for I actually just just finished the supplier and because the timeline is kind kind of tight. Yeah. And for receiver I have some some progress but I haven't have any PR or any any branch on it. I just tested in my local but yeah so here I'm talk going to talk about the the things or the PRs what I did in the supplier site and I'll just introduce those uh like one by one. The first one will be the data type and this one is

quite funny because I didn't know that uh there are so many forks and we have to uh kind of rebuild every every data type each time for each fork. So it's very cool that uh I'm able to do those of the data type implementation. And one tricky thing is that uh prison actually because I think it's uh kind of like the tradition or like it's how they do in in go is that they use a proto prototype and it's called proto message. So you define the proto message and in prison there's a a

script or something you can just convert those message to the SSC code. So with the encoding and decoding thing so it's very convenient and one thing I have to uh know I had noticed is that uh those data might be have different data size for example like the gloas you have I'll talk about later but uh it has the progressive container and it will have different uh when I implementing the proof it is different from the uh from out air to fulu. Yeah.

And the second step is that um actually this one I was working like I was waiting for Bastin's PR because he implemented the the glow us proof and I just kind of inherent his proof and just implement the s aggregate proof and for this uh you can just basically see those field these field are actually from the block data here and with this field you are able to like construct to the uh construct a body root from this is uh this is just a container from out air to flu and you

can use those field to construct the root and then you put it in the block header and you just check if the block uh block route is equal to the nest headers parent root so like this chain here. So you just prove there's uh like the parent header here and then the second one and then form the trust chain. Yeah. Yeah. So for glow us there's something like interesting they introduced the progressive container in beacon block body and beacon states. So the field and the size of the Merkel tree will be different from that. So I

think I can show yeah this is a very good picture made by cloud. And so you can see here like the first uh left tree will be just one field and then the second will be one to four and then third will be 16. So four to the something sub tree will be grow from this left side and for the right side is just zero and for the right sub tree it it will be always be the active fields. So this is what I talk about for the glowas stuff.

It is just um it is just different different you have different data type and you have to you have to define the different way to build the proof. Yeah. And the third one will be the storage like we uh because uh prison using are using bolt db which is a key value uh key value db and we have to define like the the mapping from epoch to this epo data and the period to the sync committee and for this I separate these two is because uh you have the same sync commity for each period. So you don't

have to store in every epoch. So you have to you you just have to store in each period. Yeah. And this this set was quite simple. Yeah. And yeah, then we have to like kind of extract the data from somewhere to to our DB, right? So I built a function that would get some input and then extract those data from uh from those inputs and then store into the light client uh store to the prisons DB and yeah I think this is just a simple extraction function

but there's some H case like if there's empty slots if the uh if the block uh is empty at somewhere or if there's um some epoch edge like I think mostly like the fork age but yeah I I I haven't like honestly I haven't checked that much about the correctness of this one because um I think maybe I it is it is like we can only check after the receiver is like working.

Yeah. And yeah, the the fifth one will be the data collector is that I wrote a script and I started from Oh yeah. So I asked Ethan like uh how do I get the the history data from like the other node? Is it archive node or something? So I think the best way to get those data is like from the error files and Nimbus actually have a have a endpoint for this. So I just go to their endpoint and just download a bunch of data from it and then I use the uh function here to

extract the data and also we have to replay the state in memory and then just because we we're going to get the sync committee or things like that those field so we are able to we are able to like extract those data from from it and then store it in in the DB And actually, oh yeah, um, so this is what what it looks like in the era file. So the first, the blocks and for example like like here we have the stock index and it's like just in little Indian so I have to switch the bit. So

that's just very detailed thing and in the end I wrote a script to like kind of extract the data from those error files and I kind of in the in the beginning the performance was very bad. I I was the ETA was like 10 days and it was not able to like do it for I don't know maybe a I'm I'm using my my home lab like the server of my home lab and then you just have to do a lot of time. So I kind of I just call I just call my AI to like iterate it until

you got the best ETA and then it just did it. I think it's very good for for for AI to do this and in the end I have to serve the EPA data right so you got to set up the API you got to set up the P2P like the endpoint of that and yeah so in the end I was able to like test it over the rest and just with my like I'm using some curl command to test Yeah.

And for the status of the nest uh and the nest step it is like uh I already done those uh supplier stuff and for this receiver stuff it is not working yet. So for the next step I think I will be working on more receiver side or the verifier side. Yeah. And yeah thank you. This is my sharing note about the >> Awesome. Thank you so much, Jeff. Great job. Um, appreciate you working on this uh in the EPF7. Uh, folks, any questions for Jeff? Oh, we got actually a mic there.