Verkle Implementers Call #3

Transcript

okay okay everyone welcome to Vertical implementers call numbers three this is issue 829 in the PM repo where you can find the agenda we have a few things on the agenda today starting with an update from Guillaume on the overlay transition followed by a discussion on relaunching the Palestinian testnet

then a few things on vertical snap sync and lastly State expiry as a potential alternative actually to the overlay transition and then if time remains we may try to touch on pre-image distribution as well uh okay Guillaume do you want to kick things off with the overlay update uh yeah sure um well it should be fairly short actually uh let me just share my screen um where is the

too many too many times it's always the same problem um no the one um sorry yeah uh is that the one um okay well then if I can't find uh if I can't find I won't share my screen but anyway so we have been um we have been trying to optimize a bit more and make make more more performance measurements for the for the transition uh so everything is shared in the document like the the document I'm going

to copy paste right now um geez I can't find anything today oh here it is um so uh the the results is well for arm basically so we we started running on an let's say average machine and AMD ryzen 7 and also on the Rock 5B which is a which is a Raspberry Pi on steroids um So currently um at least our test code doesn't have any of the ism assembly like um any optimization for the crypto so the performance for uh rock5b is pretty uh bad

um so with uh basically with if we were to transfer 1 000 Keys per block uh the results uh would be uh roughly half half a second on average and in the worst case one one second extra compare like for for a given block production um and as a reminder if we only translate 1 000 blocks per sorry one selling leaves per block the transition the overlay transition would take five months uh which is you know still acceptable but apart from ideo um and if we take uh if we decide to transfer ten ten thousand leaves per

block the transition would take uh 15 days so um yeah I would say a month two months might be acceptable for for this overlay method but um by the look of it and we still have some Avenues to explore uh but if they don't if they don't deliver any any significant Improvement we have to choose between uh dropping the like low lower power devices like the rock 5B um and or the alternative being uh having a six months transition um so yeah like no no definitive no

conclusive answer yet because there's still some things to try but by the look of it I I mean my my preference would be to yeah to you to give up on the Rockstar be at least during the transition make sure it's still um it still is possible to to run vocal Tree on those machines but we will tell those owners those validators to to just uh wait uh like one month and sync again um yeah like I said it's not great but it's uh it's definitely acceptable and uh yeah I think that's pretty much it oh some people raise their hands uh

yeah Josh uh maybe it's to the question but just to clarify uh when we say we would have to choose between dropping those lower powered devices we just mean they would not be able to participate in the actual transition is that correct yeah I mean they would probably fail doing like they would they would probably uh not be able to follow the the chain and or more exactly they would not be able to produce blocks on time um and so they would they would have to yeah they would just fall off the fall of the edge and they can probably join later but um but during the transition it might be extremely hard for them to follow the chain uh fast enough to be able to produce blocks

yeah so one question you mentioned that uh well if we process from 1000 leagues per block that we end up with a five month transition period whereas uh if we do 10K per block we we get 15 days which is preferable uh could you elaborate a bit on why a longer transition period is problematic so my understanding is that thinking if you try to sync up a new load when the transition period is in is being done then that tends problematic but are there other like downsides or is that an actual downside uh there's yeah I mean there's uh several downsides for um they they all

come up with uh for example the the sink um if you know the question is do you want to to create a specific method a specific sync method and there's there's another item on the agenda about this thing so I guess we can we can plug uh back uh like we can connect to this uh when we get to that point but the the problem would be if if your transition is only 15 days uh I would say do you really want to care about designing a syncing method for just 15 days when you could just snap sync to just before the transition and then full sync for 15 days it's you know once again it's not great but you don't you don't need to create a syncing

technique specifically for this transition um so yeah if it's it's a lot of things like this like there's a lot of problems you could say well you know we have a good enough method that we can um we can deal with uh for a period of 15 days maybe a month um but if it's six months you definitely need to be able like you can't tell people okay next time you're next to a window your next opportunity to sync is in six months that's that's not realistic um and there's another reason that I was thinking of but just so I can't remember now um yeah sorry it will get back to me

yeah cool thanks so it's mostly about the inability to think during that transition period for new notes but yeah cool yeah um all right so the second reason was uh you know what if uh I mean this might not be the best time to to make some uh releases right uh new releases so like add new features and things like this of course if there's an emergency uh like books or someone discovers a an exploited will have to be released but um theoretically and we would like to avoid making releases and 15 days without a release is definitely manageable maybe two months is manageable six months is not like people

will want to roll out features and um and things like this so that was the second the second thing oh yeah yeah that makes sense thanks oh sure and Enrico yeah I just raised the the hand when you were talking about rocks B rocks 5B and uh something pops in my mind like so they will probably affect also the uh at the station occurrency if I if I get it right because if they have block import very very slow they probably have more chance to start voting importing blocks late and start

voting for a wrong head so I see that the impact is probably also have less Effectiveness as a validator am I right here uh yes absolutely I mean this whole extra work could could be done in parallel right because uh all the attestation voting would be done in the co where the the El could be able to prepare this transition for the next block um all right so you say the block import should be fast similar to to the normal block import you're saying this uh it will be well it should be um

currently it's not because I do everything sequentially or we do everything sequentially in that prototype but in theory assuming you've got enough processing power and everything yes it should be pretty much the same okay so you could do that in advance and actually have these two seconds just after the last block import so you have time to well not sure okay uh only true for linear blocking personally like switching to different branch can be very very costly right so the late block reorgs could be a nightmare

yeah that's true there's there's also that yeah um yeah so I guess uh we need to get more performance basically for that to learn uh I mean yes if you ever if you have like if you have a late uh a late reorg actually uh some people are gonna miss their slots that's for sure uh this being said sorry I'm just thinking all of a sudden that would not be necessarily entirely true because you

could prepare uh the block to so the problem was very cool is the update the tree update uh next the commitment calculation update and the problem with this is that uh one it's not a problem the advantage of this is Computing one commitment update is extremely expensive but there's a good there's a batching technique that's uh that's extremely efficient so if you if we would be able to if we have to reorder basically we could reinsert all the missing blocks directly in one go and still uh get a good speed up um yeah okay I've just uh just bring dumping here there might be a good way to to do this and that might not actually be a problem but it needs to be

investigated I'm not sure if that's possible because um you if you have a rework you will have to compute all the intermediate State routes as well yeah that's true and for so for that you need to do the individual dialogue updates but I mean I'm not I mean is that really that much oh yeah you do get more efficiency if you have more updates to batch yeah I think you'll still have to do it block by block unless you get a trusted route from somewhere that you're sure it's the correct one yeah it's true uh but this is going to be a problem yes because if we have 10 10K uh 10K leaves it takes about a second

uh per block of 10K so sorry a second a half a second um like okay worst case is a second an average case more like 300 milliseconds so um assuming the worst case you you have to Rio 50 blocks that's 50 seconds that's um it's not great but in any case okay it has to be tested um yeah that was pretty much it for my update so we can go on to the next uh the next topic if you unless there's more questions of course okay next topic is the test net relaunch

game do you have thoughts there as well you want to start with sure um yeah so we have Cal student which is a test net that provide provides proofs and blocks it's uh well it's been working fine um but it's a bit outdated um so one thing for example is uh that we'll be discussing um is that there we've we've been talking about uh putting uh putting um the post State So currently on a cal student only contains proofs for the pre-state [Music] um of each block we would like to add the post date as well see how how it behaves um so that was one option uh there were

currently Cal student is also pre-shanghai so no there's no withdrawals there's nothing like this um so these were the two features I wanted to try for Cal student uh well for a potential relaunch knowing that guest hasn't been rebased yet that's that's my current task um and uh so yeah it would probably take a month or two to to get it relaunched but while we're relaunching I wanted to know if there were extra features that we should add and I know tanishq had some ideas I think about this so yeah I'm basically collecting input what what should be different for the new version

of kelston um remind me so uh G11 your uh oh tanishq is off this week um and um yeah I'm so but um what's your name sorry again G11 tank hi I'm gogender oh yeah that's it um so um which uh because I know you're part of two teams which one are you talking about like are we basing on uh the shangri-basing are you talking about load store or uh I think you're also

ethereum.js right yeah both have a lot of code merged in the master for dinner so I mean just to figure out how to not mix it with the app would be difficult maybe it would be a little easier for you MJS because the activation is is based upon the EIP activation but on load star uh you know we have a pretty deterministic flow of how the hard folks are scheduled so um of course capella you know of all our code repositories basically make sure that Dana is the next one so it will be a bit difficult to figure it

out so right if I get okay if I understand the prompts uh you won't be able to activate yeah okay you won't be able to activate presumably Electra if that's uh if that's still the schedule we we're playing to if we make that schedule um you will have a hard time activating a lecturer before dinner basically and therefore it would have to also contain uh block transactions and things like this for load store to join yeah and I mean if we if the test net is to be launched in two to three weeks I mean it's quite feasible that maybe we can leave us on done up because then at this spec is now quite stable and the

definitely expect most likely we can use to rebase everything on yeah but guests won't be ready for that um yeah I don't think so I mean yes it's I mean clearly we do have an implementation of uh of Cancun in guess but I don't trust it I don't trust it enough at this stage that that we should attempt Cancun as well I mean ideally it would be good to have withdrawals uh sort of part of the next test night because you know when if the block is empty withdrawals do change the state

so I mean it's a very good thing to have withdrawals as part of uh the next devnet but uh yeah it would be a bit tricky how to basically include uh capella for load stock but maybe we can figure it out I mean I mean ultimately uh it's mostly a it's mostly an El Fork at least at this point we we have this API like requirements uh on the on the El CL side but uh if one client cannot make it one client can make it for this relauncher I don't think it's too bad

um the question is can Lighthouse do that I mean I assume is anybody from Lighthouse here I guess not it's Australian um and I'm trying to remember Enrico are you from I am assuming that you know it with that difficulty level would be same for all of the CL times because most of the CL times are not really you know configured in that way that you can basically schedule or unschedule of work so I mean but I think yeah maybe it's doable we can just patch and and figure it out okay

um okay well then uh yeah I'll get back to you uh offline uh to see if we can organize that um I'm pretty sure I'm pretty sure the lighthouse should be able to do it because this is where we've been using uh Perry do you remember were we using techu as well uh no we're just using Lighthouse okay and for tikku if anybody from tiku is here um but we haven't started actually uh updating our Very Old Branch around integration with work on 3 so we need to pick up the the job again and refresh our branch on top of all the uh things that has been done so far so

yeah not started yet okay that's fine my question was more like theoretical uh would you be able to rebase your old branch on top of like Shanghai but not having uh Cancun oh that's uh I don't think so as as a my initial gut feeling yeah we have pretty straightforward and sequential uh spark applications so I don't think so to be honest okay gotcha uh I see there's a zahari

from uh from nimbus yeah I'm here but I'm from the embassy but I'm not implementing Berkle at the moment I think Sahara is also here uh yes sorry for I offering you to myself but basically we are just starting uh implementing things we'll be focused on that and he started just a week ago so we are selling from the most lowest we are building blocks and now we'll go from there okay cool but in theory do you think you could be able you would be able to in enable Prague or Electra without

enabling Cancun um it's a bit we have this General problem how to kind of enable certainly ABS Etc and how to relate them to the sharp works and we have a solution that works relatively well with compile time Flags uh some teams have asked us testing teams after to provide runtime tax and this has been a bit more difficult right um I don't see anybody but maybe um I don't know every name here is it is there anybody from prison

I guess not um yeah so I guess for uh on the CL client at well I'm going to assume it will only be a lighthouse thing again um I guess we could also restart Beverly Hills in which case we can use every single client that's no longer a problem because Beverly Hills is the same thing except the proofs are not provided uh that's true we can also join clients postgenesis yeah that's true I mean yeah absolutely yeah uh okay so I guess that's uh that's it for now and I will write uh tanishq to ask him if he's got some extra requests for uh for this relaunch but

otherwise I assume it will only be uh um the sorry losing my words again um it will be the post date and uh there was something else I forgot anyway uh those two things um yeah I think that's it for for this topic okay so next we have vocal snap sync uh right so I didn't realize okay

time she's not here uh I don't have too much on vertical snap sync but I have one message from him I got last week that he was testing um stateless execution on Cal student so we can start the node and sync in the background while uh having execution and he was able to process um new payloads without any without the state being synced yet but he had failures on some block with different state route and he was looking to the issue and he um wasn't sure but um thought that maybe the witness is not

correct in the blocks but he wasn't sure yet so this is also a good test if the witness is correct right um so either either it's uh processes wrongly or the Witnesses are a problem but I don't have any follow-up from that I think it will be next week okay that's nice yes it's a great test um I used to check the witness I haven't done that in a long time so they used to be working I think Gabrielle also I know right if you're not checking the proof right so giveaway in your yeah not yet okay so it could be that the proof is wrong but at least the witness should be correct

um oh right there was another thing that was missing and now that you mention it the the coinbase was missing or the the reward so that's also something that needs to be added to kelston and maybe that's what he meant um yeah okay but otherwise uh so okay so what's uh that thing is God would be basically snap sync right except it uses a different proof system yeah so uh I think we we the idea I'm not entirely sure was the state but the idea is to have something and they also have uh stateless execution in in the meantime right during snap things so the idea is to be able to execute blocks as

soon as you run never mind eventually maybe even like provide a flag not to do state sink at all but we'll see cool okay well I guess that's it for the conversation but that's nice that's nice to know um was there anything else on the agenda yes the other thing on the agenda is State expiry versus the overlay method I think you want to touch on that briefly oh yeah sorry it's let's just be talking um but uh yeah I I also I wanted to discuss

this so uh you know we we have this overlay method where we trans where we translate um n leaves for per block um and it had been discussed uh like the idea there was another proposal initially uh which was to never do the translation so keep the overlay tree right to the overlay tree but never uh never do like never voluntarily get leaves from the MPT and move them into the vertical tree so you would have this two three system for a long uh period so definitely more than one month definitely more than six months um and then with the help of uh still expiry

someday this tree would be deleted like once State expiry is properly defined and and implemented uh this base tree like the MPT would be forgotten uh translated to Vertical in the background and uh by by some by some nodes and then when someone needs to access the data that was never translated never that was in the MPT but never got overlaid or never reached the overlay three uh this those people would use the state expiry scheme to resurrect that data and bring it into the the Virgo tree um yeah so there's a bit of a

you know there's a few problems with this approach one of them is that it would make Virgo tree dependent on the on the design of the steel expiry or uh that would be a Hail Mary we would just uh we would just go for it and hope that someday some someone will Implement that uh but yeah I was just wondering if people were like if people had an opinion about this I mean especially uh other El clients um like for example woodcash would you be completely against this idea uh no uh so the question is do we want to have state expiry when do we want to have it do we have a backup for the

state right those are the questions here and yeah so if we if we would agree on the state expiry I in general doing the transitional State expiry makes sense okay so yeah basically we we need we would need to find something uh is there anybody else from an El that would oh yeah for example uh atm.js so people do you have any idea about this yeah I don't see the downside to doing State experience at the same time as the transition it does I mean the only downside I can see is well obviously that's uh meat to the

to the actual uh transition period because we're doing more upgrades at once but uh I think it gets us closer to the end vision of Virgo implementation which I mean it's often been presented in pair with Aesthetics Parry anyway and it does have the downside of having two trees for longer period of time but yeah uh personally initially when we began talking about Virgo I thought these two would kind of come and pair with each other so it kind of reconcile that Vision at all on personal insoever so um well technically you don't really need to have both to have both trees you can all you need to do is to have the leaves for the mpg after a while once the once the fork has passed the

finalization point you don't need all the internal nodes because you won't be updating them right oh that's true yeah yeah but yes uh admittedly it means you have two data structure types of data structures that you keep forever like potentially forever um so yeah it's uh for code maintenance it's not great especially if we want to have clients that don't care about the you know the long uh the past that is long gone um that doesn't help with the simplification basically but um yeah it wouldn't be as bad um yeah and there was another question for this which actually is the same as

the transition what would be the gas cost for this because if you hit the okay if if your transition is two is to 15 days uh you don't really need to care about uh being too correct with the gas you can just say the cost of the gas is just reading from the overlay tree and then you add a constant cost for in in case you have to go into the key value store because that represents the the bottom layer of the of the MPT um and then you could just say well okay it's going to be a bit more expensive people who end up in the overlay tree in the vertical tree are going to pay more than they should but it's only for 15

days if you do this forever it's going to be a bit more expensive I mean it's going to be more expensive and inadequate for uh potentially forever so you mean that the operation that will be more costly is when one piece of data isn't yet available and uh like overlaid Virgo tree what has to be retrieved from the American trying and written down so because you're doing like two reads in one right instead of just one read right exactly but since it's greatly complicates the yes model I would just try to do something I mean I don't know if it's realistic yet but just say you have the cost of reading into the vocal

tree and then on top of that you have to cost a fixed cost for reading into a key value database which corresponds to the MPT basically it's the lazy approach of counting the the costs twice or checking that if it doesn't hit the first three then you have to pay yes for the second for reading into the secondary yeah yeah would that would that approach be idiosyncratic to that like transition period is that something that's ever been done like yeah specifying a gas cost that's mapped to the underlying like data structure yeah no it hasn't been done and that's

what I'm trying to avoid precisely um so that's why I wanted to offer like to suggest a fixed cost for reading I yeah I guess I've written the ipu but this to get some proper feedback um but yeah it would be just assuming you don't you no longer read in the tree right in the MPT because uh you don't need to update all the internal nodes so you only keep the bottom key Value Store and then uh you provide a fixed cost for a fixed gas cost for accessing a key Value Store really uh Karim I saw you were here at some

point you also have an opinion about uh uh there was a condom the state expiry method I don't think we can hear you Kareem if you're speaking okay uh maybe there's a microphone problem um I'll sync up with him

okay well okay I guess that's it for the for this topic and so Karen can but yeah I'll sync up with him oh hey I have a quick question on the the gas cost during the transition period so I understand the concern about you know reading the read and write operations are more costly therefore have to be compensated by an additional gas cost to compensate for you know client load but the there's also the transition word right which obviously isn't priced in because it's just work that the clients are doing on their side uh is it worth thinking about how clients

I mean there's going to be a 15 days period in which the clients have much more work to do than they typically have right yeah and there's no compensation for that are there is there has there been like economic analysis as to what the incentives are going to be during that period and what that might look like because it feels to me like we're going to increase our load like 5x and just they're gonna they're not gonna have any additional income for that right uh so there's the research uh the Russia the robust incentive group in EF that started looking into this but they started I don't actually know if they even started Josh do you know if the video started working on the topic

uh just very initially hasn't gone deep yet yeah so yes there is a going to be research but it's still um yeah it's not very Advanced so far um yeah sorry what's your name in veghan yes oh hi thanks I'll just I was just wondering like as the entire network state grows like well how many more years do you think that uh can the entire ETO Network sustained stably without Solutions like stay aspiring to tackle the state boat problem just just

wondering and wanted to get your opinion on this yeah um yeah I mean that's a good question we don't really know but the idea is precisely that if you don't have uh like still expiry I mean stateless is precisely the uh trying to offer a middle uh a middle point or I don't know a an intermediate solution so that the state growth is not affecting everybody on the network only the only the people who have to produce blocks um so how long can this go um yeah I don't know where you know even even with the MPT we uh like there was a

presentation by uh Peter at East Prague last year we still have a few years ahead of us um now if you want to if you want to deadline where everything collapses I don't have that but I think I think it's not like State expiry is not super super important at the moment we still have we still have some time to to go I mean actually vocal trees um if anything will enable us to postpone State expiry further because we will be able to uh allow people to use stateless plans instead and so the the

it's okay if like the people the cost of running a node that has the full State goes up a bit oh I just wanted to follow a vlog on that um because from my understanding um voculture itself is tied to stateless but it doesn't mean that I mean it might be wrong because I'm still new to this place so correct me if I see anything wrong but for my understanding stateless might not occur even though uh Circle 3 is uh live on the main net so and I also saw some analysis that um by inventing will go through because we are storing more data in the database each client's notes the storage might

increase as well so I just wanted to clarify that was that's incorrect where did you see that you were storing more data with circle tree um I'm not sure maybe I'll get the resource but I saw it somewhere but I could be wrong I just want to clarify on this like oh that does local tree increase the um storage capacity okay no that clarifies all right correct the transition might depending on which transition is used but the ultimate stage um shouldn't increase it okay got it and uh so vocal enables statelessness um but statelessness itself is um is just a separate thing so like uh

basically as soon as we have worker trees it will be easy to for anyone to construct a stateless client and it can actually be done independent like there's no uh there's no requirement for consensus updates or anything I mean theoretically you could have a status client now the only problem with it is that you might encounter witness sizes of hundreds of megabytes which just isn't practical yep so like I said never mind is already implementing a stateless client kind of uh did you mean a MPT clients because I think that's what the Encore meant yeah yeah no no he yeah yeah

the vertical one okay yeah cool thank you for the questions um anything else Kian on this topic uh nope I think but I don't know if uh karim's uh microphone started working well I'm coming back to try to get it fixed okay

all right because match you're also you're also yes from our perspective this is the same question right about uh coinciding the vertical transition with uh statelessness right or with a stateless transition we're going back to the previous topic yeah from the basic perspective it's not I mean it's just complexity with since we're already kind of adapting our Bonsai format it should be relatively straightforward to do them at the same time just based on the work on the way that stateless could potentially look I deferred to Kareem on the specifics but you know I'm weekly in favor of doing them at the same time just because it's going to be already complexity but at the same time we can

it doesn't really matter because we'll have to adapt our pattern anyway but I think Kareem's mic is is back on it seems origami yes sorry could you repeat what you say much I was disconnected yeah so our we we went back to the question about um doing the transition to Vertical at the same time as potentially adopting statelessness and I think from our perspective since we're adopting a new format for Bonsai anyway um reworking it for stateless at the same time would not be too much of an extra lift especially if we're putting in the same Fork but that's kind of my

personal opinion based on how it's working I wanted your opinion there too yeah yeah thinking is the same because as you said we are trying to to modify Bonsai to work with vertical tree so if we can do all of the notifications at the same time and not to do several rework the rewriting of this part to it later right but uh the question was uh okay at least I didn't see the connection um in theory would you be okay with still expiry or do you have some strong uh oppositions to this to this method I don't see any problem yeah to do that okay so that could be a nice backup all

right uh yep thanks that was all for me on this topic thank you okay great uh so we only have a few minutes left do you want to save this for the next call we we did have something on the agenda for uh to discuss free image distribution uh General discussion and potential next steps there but maybe that the best for the next call yeah I'd be I'd be okay with that finishing already sure okay yeah no need to use the full time we can end early there unless anyone has anything else

okay oh uh one more from uh weihan yes yeah oh sorry just wanted to ask uh because I'm new to this space and I've just recently been doing research on this and I recall that um vocal tree is set to be launched after the Cancun upgrade which is in prep without mistaken which I'm assuming is going to be next year so from from your side is that a specific timeline that when Local 3 is going to be on the main net perhaps like during the first half of next year or second half of next year just just wanted to know about this yeah well I mean one of the questions is when is Cancun delivered um because we need to make sure that uh this well yeah this first uh thing is

delivered so okay the short answer for your question no there's no timeline um we were also dependent on Cancun and then there's uh yeah once Cancun has been delivered people with other clients will will start diverting to to vertical and uh yeah there's currently I mean I'm still I'm still hopeful we can get it by the end of next year um but there's definitely no promise on that yeah I mean it's likely that Cancun will be delivered this year um I would expect it before Defcon at least I'm not sure if vocal trees by then will be ready to uh be the next hard Fork I think like we need a very convincing

story on the transition in order to be able to push that and otherwise um there's probably going to be e of uh which really want to get their updates in and have been pushed back several times thank you okay great uh well that's it thank you everybody for joining see you next time thanks bye thanks everybody thank you bye-bye