How Snap Sync Works on Ethereum and Starknet | Han, Nethermind | ETHTaipei 2025

Transcript

[Music] Yeah. Um, so just a quick warm up because I believe this talk and the next talk will be core protocol development things. So it might be boring to some of you or it might be very interesting if you work on corporal like myself. Um I'm just curious like if anyone runs a node on Ethereum whether it's test net or mainet. Anyone? Okay, I see one there. Only one. Okay, I expected more but it's fine. Um I know it's hard to run a node but anyways like um I personally run a node on Ethereum mainet

and start um and and test net and definitely also starknet and um I remember my first experience running a node um two years ago was when I had to gather a bunch of hardware making sure that my network setup is right and when I finally got everything right I pressed the start button so I expected it to sync for a day but to my surprise it actually took 30 case and actually even more. So, so, so then I wondered like is there a better way of syncing like no definitely I was very surprised that I mean I was very happy that my my note is sync after 30 days but to me at that time it just feels very slow. So I did some digging to my um setup and I

realized I was running on an configuration called full sync not snaps sync. So after that I think after a few weeks I bought another pair of machine I had the same exact setup and then I ran it on Snapsync and I managed to sync the note in just under a week. So that's how powerful Snapsync is as compared to the regular full syncing um experience. I'll explain what those things means later on but um I at the same time I do not want to trick you to think that Snapsync is the endgame solution. steps sync is the best way of syncing a node on any blockchain or anything else. Uh because in fact it's not and there are much

better syncing algorithms out there and at the end of today's talk um I'm also going to share with you how the future of syncing looks like but without further ado um let's dive a little bit deeper into how snapsync works on Ethereum. So first let's think from first principles like why do you even have to sync a node? like when you run a node. So typically on a blockchain um system you would have two roles, two actors in general or two important ones. You have the block producer and and then you also have the block validator. So basically as the name implies block producer propose a block or produces a block and then the block validator basically would um validate

validate a block. So just assuming that the system is this diagram, you only have these two actors without anything else. It's basically a trust me system because you're just um the block validator will be just trusting whatever block that it receives. But um of course it's a blockchain so we don't we don't want to have this kind of trust me both system. This is where we get the I don't trust you that kind of thing. Um that's where you have to verify if uh the block is correct or not. So in order to verify if a block is correct, you basically have to introduce something called a state right. So both the block producer and the block validator have have to have the right state um and also the latest one. So in order I mean for

example for block producer they need the latest state in order to produce the right results and for bot valitor they need the latest state in order to validate that this is correct. So now again from first principles what exactly is a state on Ethereum the fundamental data objects are basically these two things accounts and slot values. So an account two types EOAS and contract account. So EOAS are externally own account basically a regular account with just the balance and also nons and then you have contract which is your contract and so for each contract they will it will also have its own slot values. So two fundamental data objects account and slot value. And how are these um objects being represented is by using a

cryptographic structure called try or mer try or to be even more specific hex Merkel padisha try. This diagram is a binary one to make it simple but in actual cases is actually a hexary maja try or hex MPT. So you start off by having the state try or other people may call it the account try. So in the leaves of the state try you would get all of the accounts. So account contain balance nons root hash and code hash. And then for each of the contract account you will get the storage try right and the leaves of this storage try will basically contain all of the slot values 32 bytes raw bytes right. So this

is how a state is being represented. So when people mention yeah we need a state we need a state we need a state to sync it's basically what they mean is basically we need the latest state try to sync right so this is the kind of the general picture of what we need now let's talk about full sync or let's talk about one of the most naive approach when it comes to syncing a node which is um we call it full sync so um how full sync work is again very naive you start from zero you start from scratch you start from the genesis you take the genesis block, you execute it, you get a state and then you take the next block which is block number one. You execute it, you get the next state and then you keep doing it as uh the chain progresses

and until you have reach uh you have reached the latest state or latest block. Right? So as I say this is naive and the problem with this is it's way too slow. If you attempt to do this on uh for example someone at low-end machine or a Raspberry Pi that kind of setup doing a full sync will probably take you from my experience nowadays I think probably about two months two to three months so it's way too slow um but if you have to run an archive node then you have no choice um um using go ethereum or nethermine if you are using Aragon or ref or yeah these two clients then they have their own syncing mechanism which is much faster but um okay I'm I'm not

going to talk more about them because I don't know much about how how they sync but anyways yeah it's too slow so going back to the goal of syncing right the ultimate goal of syncing is just we want the state try and to be more specific we want the latest state try so how can we do that without executing block by block and the answer is steps sync So how does steps sync work? So first step is that you will select a pivot block. So a pivot block is basically you want to sync up until a certain point in time.

Time means um in terms of in in measured in block height. So this is usually the last finalized block not the latest block. Why is it not the latest block? Very simple. Because if you choose the latest block, if you want um if you choose the latest block, this latest block that you have chosen could be rearched. So when it reox that means if you have downloaded the state, you have to revert the state. So and you revert the state and you then and then you choose a second I mean yeah you choose another latest block and there's a chance that it might get real again.

Then you have to do the reverting process again. So this back and forth rewarding kind of thing is isn't ideal when it comes to engineering. It's it's not efficient. It's not effective. That that's why you choose you usually would choose the last finalized block because it's less likely to be real. But if in the worst case it is real then the process the stepsync process will become very expensive in terms of the reverting process. Yeah. So anyways choose a pivot block. Next you would have to download the leaf ranges. So what do you mean by leaf ranges? A leaf here means again the fundamental data object account or slot.

So leaf ranges means it's either account range or a slot range. So a range is basically a range of accounts from one address to another address. So for example the requesttor will request for um the accounts for um for example address um one to address three right. So that is a range of one to three. So the request will send back the actual account objects back to the requesttor from account one address one to address three. So that is a range. So assuming that the current Ethereum state try contains account from address one to address 10,000. If the requesttor has the account from 1 to

10,000, he or she can reconstruct the entire state try, right? because that's how the the objects are being represented. But there's one caveat here. What if the requesty is malicious or what if it sends back the a false um value, right? So there must be some way to prove that this range is correct and that's where we need a proving mechanism. And here the proof that we use is called range proof. So um yeah so the range proof will prove that um this range is correct to dive deeper into how does the range proof look like let's give an example. So assuming that on the left side um you have the try um yeah

and then the account range that is being requested is from um address one to address 4. So first step on generating the range proof is you're going to proof the left side meaning to proof the first item or the first account in the range. So here you will generate a Merkel proof for the left side and then you're going to generate a Merkel proof for the right side. So here in this example is account four which is the last item in the range and then a range proof is basically merging this account um local proofs together.

So this is how a range proof will look like. So it's kind of like a partial view of the try. So this is a range proof. In terms of the verification part, now you have two things. You have the range, you have the range proof. So in order to validate if this range is correct, you simply insert this values into the range proof. Right? So again now this becomes a slightly even more complete view of the entire um state try. So to validate if this is correct you basically check the root hash. So if it's a state try you check if the root hash of the of the of this partial try that you have just reconstructed matches

the one in the block or if it's a contract try then you will check if it matches the one in the account. Right? So what happens if one of the values um is is false? So you basically get a mismatch root hash. So what will happen is you will reject this uh range. You would reject this response and you probably would um ban this pier because it sends you a a false range. So that is a very simplified version of range proof. So after verification if it's correct the requester will simply persist the range into the database and this process would repeat until all of the range all of the ranges has been has been

downloaded. So yeah that's kind of it for snapsync. So it's actually much simpler than you might have thought or is it not really because there are some a few more issues that we have to address. The first issue is pivot block constantly changes. So the pivot block so last finalized block it means that latest block minus 64 something like that usually. So um it's practically or almost impossible that you have downloaded all of the available state meaning accounts or slots within this period in this pivot block. It's practically impossible. So the pivot block would would change would definitely change and

when it change that means that the state you have downloaded before they could be outdated and there could be some missing gaps in this in your current available downloaded state. So that's one of the issue that we have to address. The second issue that we have to address is that um we want the latest state right so because again we we are only requesting up until pivot block because of Rio issues but um we still this is still our final goal. So we have to introduce an additional u phase called healing. So the healing process is that you have already downloaded all of the range. You have reconstructed the try up until a certain point. Some of them are

outdated and then you compare it with the latest state try, right? So you will see some um you would you would get some missing gaps or outdated values. So the solution to this is actually very simple. You simply request for the missing trinodes, right? So the requesters would request for the missing trinos and the requesty will send back the trinos and then you would um you would replace the outdated ones or fill up the missing ones. So one very important thing when it comes to snapsync um during this phase is that you have to have a good network setup and also a good hardware setup such that you are able to heal much faster than the chain progresses because if you

can't heal that fast you will be forever be in the healing phase forever stuck in the loop and that's why um when you go into like um go ethereum discord or um go into discord even check their hardware specification they always emphasize a few things. One is that uh you need to have a certain amount of uh network uh bandwidth. The second thing is that you must use SSD. I don't think anyone can sync a node in using an HDD anymore. Maybe you can be um two years before or even three years before, but nowadays it's practically impossible. So yeah, to recap, step sync, five steps.

Select a pivot block, download all of the ranges plus the range proof up until that pivot block, verify the range, persist the range in the database, and then once everything is done, perform state healing, make sure it's fast enough such that you can reconstruct the state to catch up with the latest chain head. And from there once you have caught up with the latest chain head you can then do the regular syncing by executing block by block. So yeah this is steps sync. Next steps sync on starknet. So or any other layer 2 uh that that uses a similar try structure. So a quick introduction of what starknet is. It is a validity roadup or some people may call it zk rollup and it uses stark

proofs as the name implies. So how would it work on stucknet or how would it work in the in altos? First let's understand the differences between um ethereum state and also stark net state. So this is how a actual state try on the ethereum will look like it's a hex MPT as I mentioned and on stnet is use it actually uses a binary um MPT. So but there are two things here I mean uh two tries one try is called contract sh. So a contract try is an ex is the exact same thing as the state try on Ethereum.

There's a second try called class try which I won't dive deeper into it but just remember that there are two tries. So how the snaps sync process will work is generally works the same. Yeah I don't have an interesting answer for you because it's try structure snapsync is to download try is a protocol to download try. So um since we are use still using a MPT structure then it should generally work the same. A few things that are different are hash functions state structure and try structure and one more additional fun thing that L2s can usually do is to leverage on the state divs posted on L1's I mean L1. So u for example um

since we post the state divs to via call data or um blobs. So what L2's L2 nodes can do is to extract this state diffs um decode them and get the actual state diff and those would basically be a partial range right you can use that as as a range and persist the database as well so that's one of the things that that can be done but other things in terms of a general protocol flow it should work the same now the fun part or the final part which is the future of syncing how would syncing look like for those who are running nodes. Will we have a better syncing algorithm? And my answer to this

question is that ultimately we will go into stateless sync. Okay, let the name sink into you a little bit. It's called stateless sync. And throughout the past 10 minutes, I've been talking about syncing to get the latest state. But now I'm talking about getting stateless sync to work in the future. This is the end goal. So how do we make sense of this? So um in the f in the future very near future maybe two years or three years or even this year we might have some demo. Um how stateless clients will work is that you anyone can basically take a block

without any prerequisites. you can execute the block and progresses uh with the chain. Why is that so? Is because in the future on Ethereum if it if it act if you actually go with the road map um the blocks will contain proofs or what we call as witnesses to prove that the state changes given in this block is valid. So that means again um a block is a single execution unit. So we don't need the state anymore. That hence is this is why it's called stateless. Why it's good for us is because if you want to do block validation, you can simply run a node without any state. But if you want to produce a block, you still have

to have predefined state. So this complements very well with proposals like EPBS proposal builder separation where if you are only interested in validating the chain uh with staking you can just run a block validator and the your node will have a much much lower hardware setup um because you don't need the state right so that will also mean that it may be possible that we can run an Ethereum node just on our phone because um the hardware requirement is just so much lower. So this is a road map that I'm very excited for um for on Ethereum. If you want to know more about Stalus clients or stateless road map, go

check out Stalis.fyi. This is um an official um documentation given by the Ethereum foundation specifically the Stalless consensus team. So yeah, that's all and I'm an engineer working in Nethermine. I particularly work on um stark net client called Juno. So if you want to talk to me about clients, protocol development or just blockchain in general um do let me know and thank you all for listening. Thank you. All right. Uh we have quite some time for questions. Uh any questions?

Yes sir. Hi. Thank you. Great talk. So um as a user is is there any difference uh when making an RPC call? RPC call RPC call uh to a snap sync server versus the full server. No um the exact the all the RPCs should work the same. Yeah, there shouldn't be any differences. Uh but um the difference of the RPCs would differ depending on the type of the node. So there's few types light node full node archive node or some um maybe stateless node as well. So if you we we focus on

full node and archive node. So for full node uh so for archive node you can answer all of the RPC calls but for full node um it might not be able to answer some of the question for example what is my um what is my account balance at block 100. So block 100 is like many many years ago. So is it we definitely wouldn't be able to answer that and maybe some of one of the calls are could be um gap proof. So if gap proof so for full node on gaff and maybe also on mine um is only it can only give you the proof up until um 128 blocks before. So anything beyond that you cannot get a

proof for it. So yeah. Okay. Got So there's no performance difference. Um performance difference? No. Okay. Uh another question. Uh I'm curious about uh what is the hardware requirement for running a snap sync? No. Hardware requirement for running a snapsync node. Uh well I can't think of it. I mean I can't recall right now but because they recently they they came out with a not really is it EIP like a official documentation to upgrade the the hardware uh hardware requirement. I believe it's 32 gig RAM, 4 terabytes SSD, uh 50 megabytes per second network, um and yeah, some and I think also some CPU

recommendations or um yeah, so somewhere along along that line, there's some documentation online, but I I'm still using the old one, which is 16 gig gigabytes of RAM and two terabytes of SSD. I might have to upgrade my hardware soon, but yeah. Okay, got it. So, it's much easier than running a phone. Now, um if you value syncing time, yes, if you want to sync your note much faster, please use snaps sync and don't torture yourself using full sync. Okay. Thank you. Any other questions? Cool. Um I have some questions actually. So kind of like as

if you're going to do like a TLDDR like snapsync fast, right? Uh what's is there a downside to it? The downside is that um it's usually very CPU intensive. Um it's going very resource intensive. So um for go Ethereum node is usually more stable. For Nethermine node is usually less stable because I mean based on my experience um few two years ago but nowadays I think it's much more stable. Um because last time when I when I ran Steps sync on Nethermind on a slightly low-end setup um it actually crashes because due to memory issues but I think it will be um it's it's more stable nowadays but um nonetheless it's still

going to be is it's definitely going to take up all of your RAM all of your CPU because it wants it wants to be fast. There are a lot of things going on behind the scenes when it come when you are snap syncing. So yeah I see I see. Um then perhaps on the question from another perspective which is the users right so for the users who want to sync nodes typically there are develop developers or like miners right for miners you mentioned in the future we might be just get using the block list uh sorry stateless uh ones but who would be using the like just kind of like from your perspective who would be using snapsyncs later and other kind of syncs.

So I think um we the road map is that we will go with PBS proposal builder separation. So that means there will be a clear separation between proposal and builder on the protocol level. So for for someone like me who doesn't care about me, I don't want to extract me when I want run a validated node. Then I just want I just want to validate if the chain is correct, then I would run a stateless client to just validate the chain and it's going to be much easier for me. um the the setup the hardware setup is going to be much lighter. The commands or the instructions is going to be much better. That means the user experience is also going to be much better. Um so that will be one of the user profile and then for block

producers who are typically institutions who you know quans who want to do some MVV MV stuff or even RPC providers then they will want to run the regular um snap sync process rather than um stateless sync. So yeah I think that will be quite interesting to see. Awesome. Uh just want to make sure there's no other questions. any other all right one last from me actually I have a lot of questions it's it's great great talk and so I'm very interested uh one question that I have essentially is also coming from the user side so as a user like my so I'm sort of ignorant on the node side of things I'm more on the

solility and just like the smart contract side so it would be good to have your input my understanding is that wallets at the end of the day we hope to have like light node, but is that practical nowadays? And do you see like you know like ideally it should be able to run on my mobile phone, right? But does it like is it practical nowadays? And then you know like how how is it looking in now? Definitely no with the current syncing mechanism or whatever protocol um rules that we have. But as I said stateless would start to enable this road or this opportunity. But I do see um like my my goal is also that um such that I want to be able to validate

on my phone. I think um stainless clients will be able to enable that in terms of the hardware side but in terms of the networking side it would have some issue um because mobile devices for my for my knowledge uh mobile devices are bad when it comes to P2P networks. So that might be another hurdle that we have to overcome but at least on the hardware side on as uh at least on the execution side um it's kind of fixed but on the networking side we might have to solve it later on but it's definitely feasible once we have stateless clients hopefully in two years optimistically.

Yeah. Which brings me to interesting follow-up question. I promise that this is the last one. I keep saying the last but it's still interesting. So um you mentioned that like on this direction basically the stateless one will be enabling the mobile one but on the light client my understanding is that it's still needed some state for that or no completely don't um light site I'm not sure if light client is even even needed after we have um sus um yeah I I can't I can't think of why maybe there are some users who want that but I'm Not sure why because like client a light client ultimately relies on um on a full node

to to to do validation correctly. Um so if you have already have a stateless client then why would you run a like node? So yeah, I'm not maybe there are some things that I I've never thought of deeply, but personally I don't think people will be will run a like node anymore after that. But yeah, interesting. That's very great insight. Essentially after the state list is ready then like client is basically replaced by that should be like per current understanding and current understanding. Yes, per current understanding which means that could be an interesting topic to dive into later and then you know talk about in the next East Taipei or regular meetups. Um so let's thank thanks Han for the wonderful

presentation. [Music] [Music]