Verkle Implementers Call #22

Transcript

all right uh welcome to veral implementers call 22 PM issue 1119 in the uh PM repo we have a pretty short agenda today we have the usual updates from client teams then up next SSC measurements some updates from Gom there then Cal and updates and uh last up it's naio wanted to go through his new EIP for the state conversion so just those four things on the agenda right now um does anyone have any any client teams have any updates start

with um I guess I can start we are still working on several things at the same time one of them is implementing The Fill cost so we made uh we made some progress in this um yeah we also worked on the naota C so there was a a gigantic uh issue that we had that I wasted a couple days on but Perry helped me fixed it last week so we're going to resume that effort um overall yeah we're not going to be able to do the we're not going to be ready for for test net relaunched by the end of the month obviously but we're we're getting there so hopefully by the end of uh of the summer we'll we'll be uh we'll be

ready um and uh what else yeah we we did the series of measurements but we'll we'll be sharing this later so that's it uh ignasio you got something to add uh no not much else I will share the a updates later so it's fine yep I think I can go next so in NE mind we have implemented all the changes that we discussed for devet 7 so we are almost ready for defit 7 we still need to pass the hype test I'm not sure if we have like state test I know we have transition test that's one question like do we have uh like test apart from trans

transition test as well for workl and second thing is I'll be off like later this week and the next week but if you launch the test net next week I'll probably be up for that so yeah that's it uh just to add yes I'm also off starting Thursday for two weeks so I don't think we're launching anything uh I mean unless ignasio wants to to do that by himself cool any other

updates on the question of which test we currently have available uh I don't know Spencer if you're free to maybe answer that question for Tan yeah sounds good so is currently just the transition test um the last call we we discussed creating a new release with um the updated gu branches that were used to fill the tests um but those branches produce the same fixtures for the transition tests so um the existing release for the transition tests is um sufficient for now with regards to State tests my assumption is that that those tests are starting

at uh starting at the verticle work um we don't have tests for that currently um I do have a branch to fill starting from veral on our existing tests um but they're not yet working on the the the test cases that um ignatio has produced um however if it if it's worth having them working before def 7 we can we can make a push for that uh yes if it's possible to get them

working before de 7 that'll be great because it'll be a good like kind of a test bet to test all the implementation against it so that we don't start the test and like within like 100 blocks we find the issue a consens issue so yeah yeah that would save a lot of time if we could have that uh yeah ready before we launch oh sounds good uh I also had a question so uh we discussed this morning about uh yeah because I have a lot of branches to mer so thanks for for that like I was able to to check against what was filled but if I want to fill again like I'm still a

bit not sure what I need to do if I want to fill again like are you still blocked by a bug or could we use the F test to to refill new test so that I can check that my fing is correct compared to what I replay yeah we can we can we can still do that um I can I can send you information to refill refill the tests I think a it's uh the way we're currently doing it is it's there's like a lot of steps and it's using a different different tooling so it can be kind of confusing so um ideally I want to make it as easy as possible for everyone to just run a command and just be able to test

everything um but um I can send you the information to refill and then reexecute we can also use the tagging feature of the of the repo um I don't know if that have you have you tried that in the in a repo for Burl because the what we're doing for eof right now is that um we push new tests and if there's a ification in evm one we tag any branch uh in a repo and automatically fetches the latest evm one compiles and then refills so we could do something similar here so basically we tag something for Burl it fetches latest uh Gams branches uh builds and refills

automatically uh it's veryy easy to do we can we can set it set that up to make it easier uh but wait because if it's the is the one branch what if I want to use a different branch right because what I would like to do is uh currently I have a branch that's not merged into the main branch and I would before I do I don't want to break anything so I would like to test that Branch before I merg merge it into the the main one would that be possible to somehow specify that of course so how long do you think uh it's going to take until you merge that other Branch into the main because if it's going to be like a long time we can simply just push a PR right now to point directly to that branch that other

branch that you're working on and this tagging me method should automatically fill using that other Branch um it's a super simple change uh we can we can do pretty quick that branch is just waiting for this to be merged I mean unless I found a bug of course uh so that Branch would just be merged immediately then we have another one which which is ignasio branch that we are trying to use for uh test Genesis we would like to merge this one too uh so we need to test uh to test it as well I see I see in that case so what we can do then is that we can create uh a branch in execution spe test that points

to whatever Branch you need to fill and then just T make the TXS that we make uh for ver for ver releases can just point to this alternative branch and the only difference between Main and that this alternative branch is either vertical changes but top on top of that the the the evm building and fetching all that stuff will point to this other Branch um I can create a a PR this alternative Branch day if I can um just just give us the um the branches that you need to point to uh okay yeah that would be that would be awesome thanks cool okay next up on the agenda Gom you

had some updates on the SSC measurements that you were doing all right uh let me share my window yeah this one so this is a document I'm currently working on um so it's not quite uh it's not quite ready so you'll see a lot of to-dos um but as a we were trying two different ways to encode the witnesses and now we have yet another one uh oh the description is missing one of them not not good okay um oh no sorry that's the so there's the one that's specified in the consensus spec which I

guess I can if the link works yeah um where is it right so there there's the traditional one where you have the St State diff as a stem and then the list of suffix State diffs so for each suffix you have an optional Uh current value and an optional new value what we discovered is that because optional is not really supported by most ssz libraries yeah including the go ones it tends to create very very large Witnesses so we were experimenting with other ideas to try to reduce the amount of uh well first to remove the use of

optionals and on top of that try to reduce the amount of uh of um NS so that so that they they are encoded properly Well turns out ns are quite difficult to encode in in the go library and presumably in other libraries so we wanted to try to avoid this uh so we tried this first method where um we put all the suffixes together then all the current values together and then all the new values together so the index for example suffix like index I into suffixes will have will also have an index in current values and in new values um so the value can be null so

it's not perfect but um it was saving some uh some space we tried a second approach where we try to remove any instance of null so we would different shate all the use the use cases um so for example there was the case where the value existed before and was overwritten there was the we call that an update there was the case where the value did not exist before and was written too so we call that an insert and uh there was the case where the value existed before and was not modified we call that a read and then there simply what we call missing which is a value that did not exist before and is still not uh insert Ed so you just

need the list of suffixes ignasio came up with a third method which is an improvement on on the second one which uh in which you you got you get rid of the idea of the suffix State diff and you put everything in the same state state div directly so you have the stem and then you have a list of all the updates and all the reads and all the inserts and all the missing um and what you get out of this is that you remove uh because for each each one of those uh of those uh lists you have a counter so if you repeat that structure multiple times you have uh sorry not

counter but a header so you repeat that header so what we wanted to do excuse me is uh we wanted to get rid of this rep repetition of the of the list header so we put the list itself uh inside the St the stem State diff and every type of update has is a fixed container so it doesn't have any header so we try to see if it would save some space and as it stands yes it does so you can see that the spec on average I have the the numbers here um the median uh the median witness size for the spec uh so that includes

witnesses that were produced between uh during the transition and after so it's like a global view uh but the median the median uh witness size is 700 kilobytes which is quite large and we go almost to uh 200 sorry two megabytes when with type one so as a reminder type one is just grouping the the suffixes and the current values and the new values in in lists uh we do get a better approach so it's already much uh much smaller the the median is almost is about 372 kilobytes and the maximum block size we could find was it's been

done by replaying by the way I should have added um it's 1.3 megabytes about so it's it's much smaller um then what we did was uh try the the the second method so as a reminder the second method is you uh you try to get rid of all the nulls because the nulls were poorly encoded so we expected it to be smaller and indeed it is smaller uh the median is now 322 kilobytes so we shave 50k on average and the maximum block size is less than one megabyte um and the final method the one we propos where where uh all the like

the the structure is is removed to to re to try to shave off the the list headers uh it does work because the the median the median size for the witness is less than 300 kilobytes we still have the maximum block size so it doesn't really have an impact on the maximum block size but uh that's because this block is most likely very pathological so these are preliminary results for example this is a bit surprising that they have exactly the same maximum block size they're so yeah we need to dig into this we also need to dig into how much of that space goes to the previous the current value and

how much of that space goes to the new value because even though we said well it should be fine to also have the new value maybe you know if it takes more than 50% of the of the block size maybe we want to get rid of it um so yeah that's the first uh that's the first graph I wanted to show um and I'm skipping ahead to the end where uh like this this diagram was conflating both the sizes during the conversion and after the conversion here uh we ignore the conversion because it's not really representative it's only values that were not uh existent and got copied from the from the MPT

so we see that once the conversion is over the the tree is a bit bigger so the median size in case of type two is uh almost 4 200 kilobytes and the type three still shaves some um some space but the worst case is still the same uh the type one is not as uh not as bad I'm trying to remember what the value was yeah it's it's a bit worse than uh than the than the overall View but it's still uh it's still less than half a megabyte so the conclusion at this point even though the document is not completely written is to say well uh maybe for the for the new the new test net we should

be using that new type that new transaction uh sorry that new transaction this new witness type uh that is much smaller and um the problem with this approach is that it requires some input from the CLS um but yeah I just wanted to know what people thought do we want to change the witness format at this point we can do it later too just trying to to gauge if it's worth the effort at the moment uh I think if we have this big of differences we should try and change the size but I'm not very sure if it should

do do it in the next test net like it also depends on when we are starting the next testet but what I'm thinking is that we are changing a lot of things at once yeah and there's a a lot of there's a there's a lot of chances to like some of the things break and some doesn't and that kind of creates it hard to debug as well so if you can like kind of uh only do the changes we discussed in inop or basically other things that we added to that in DEET 7 and then we can move this change to DEET 8 that will be good but I'm okay with doing this in De 7 as well yeah I'm also quite on the fence about this I I like the size gains um but we still are going through some

testing so we might find an even better approach um yeah I guess uh I guess we like tan is is right we should uh we should not do this um I'm still open for other people's opinion but since yeah there's more work to do maybe maybe we should wait actually on the C side I can probably make the changes that is not a big deal depending upon whether we want to do this change for the next or next or next d PM you mind sharing a link to that uh

note uh not yet because it's uh work in progress and I need to a lot of the text is nonsensical I need to keep editing it so if you don't mind I'll share it uh yeah Midsummer when uh when it's a bit more thought through and uh and ready yeah sounds good cool then next up on the agenda uh any updates on cin um yeah I had two questions

actually one of them the answer is probably going to be the same as the as the previous one uh in Kenya was pushing to replace the the proof format from this ssz stuff we've got so just to be clear there's the witness part the state diffs right and there's the actual cryptographic proof all the IPA vectors the final evaluation etc etc um Kev was pushing to move this back from ssz Json into an opaque binary format um I I mean we agreed we should do that in the long run the question is when so my question was roughly do we

want to do it for K and seven I expect the answer is going to be the same let's focus on the what exactly Tanish um I agree with that but yeah I just wanted to put it out there so that people have a have a say one small Point uh I was involved with Kev when that that was discussed and one of the reasons was because one of the underlying Ross cryptography wasm that we use notably in ejs uh require the OPEC format rather than the explicit format but since then we've modified the underlying library to be able to consume the explicit format that is that we have

in C six so I don't think there's any urgency to do that in gos 7 cool well then that settles it because at the same time we don't want to have two like to change the CL twice so might as well wait for for when we do the ssz update as well uh my second question about Kel and 7 was initially we said we should do Kel s including the cost sorry the cost we agreed on in Kenya and also the field cost um on my side I've been uh working on trying to integrate more stuff into G Master to simplify the

rebase um the rebase on top of Cancun which hopefully we can do on devet 8 and as it turns out there's a lot of changes currently going into master that will make imple the implementation of the field cost much easier so I wanted to ask if everybody would be fine with changing the spec for death Net 7 which would be to have all the the Kenyan costs except for field cost because uh yeah for us that would inquire that we're incur a rebase and we're probably going to need another two months to do this so yeah um if that's what we need

if there's some push back we'll we'll uh we'll make it happen but it's going to take two months just uh yeah let me know what you think I think if if if it's going to take that long then we can probably skip it with this test net any opinion from E CHS I personally don't have a video or objection on this cool oh Gary's here as well um Gary did you guys implment fi

cost already no we did not Implement fill costs okay so for you that would be actually uh also a good thing not to not to have that for the next test net yep exactly excellent uh yeah I don't know maybe subnet or Oliver you have uh a comment on this for for your own implementations oh Roman is here too nothing from me okay cool then uh thank you I guess

we're uh we're doing this and uh yeah so that we can uh have a new testament sorry go ahead yeah I was going to say I arrived a bit late do we have link to this the criteria apart from the Kenya costs that are going to be devet 7 is it is it mostly going to be the the Kenya costs it's uh so I think we haven't merged the pr yet I need to check but it's uh it's yeah it's the Kenyan cost except except The Fill cost so um if the EIP has not been updated we will merge it um yeah we should merge it today get rinder if we if we haven't done so and uh uh yeah it's just the EIP so all all

you have to do is check the EIP and set all the FI cost to zero great Okay cool so then last up on the agenda ignasio do you want to go through your new EIP for the state transition sure I will share my screen uh hold a [Music]

second um yeah so before um talking about that one I also made a lot um other two PRS that I shared in The Matrix Channel but just think in case I'm mentioning them here one of them is some uh quick fixes on 7612 nothing really that interesting here but I will share this link in the chat some minutes the other one is an update in 6800 and here it's basically a naming fix but also I clarify how how we uh encode the

um the fields in the new basic data uh Lea uh this is part of the change that were decided in Kenya so previously the pr kind of assume that we encoded the code size noun and balance in little Indian but that was problematic because we had the intention okay so if we remember something important is that we have four Reserve bytes between the version and the code size and one of the ideas was um that choosing this

ordering um where we have these four bytes before the Cod size was was interesting because if in the future we want you to bump the size of the Cod size um like things will be properly aligned so we can use one of those four bytes for extending the gold size but the problem with that was that if we were saving this in lendan like this was wrong because um like we we need this Reserve bite to be the the new most significant bite of the Cod size not the least significant bite so what I proposed and this got merge a bit fast because Auto

merge did it uh was um just changing everything to Big Indian so basically that means that code size will be big Indian and then we can kind of extend this in the future using this extra bite that is on the left um so that was like the main intention so just be aware of this because um this is part of a change that uh usual account for the next devet it should be pretty easy just like change the the ining from um little Indian to Big Indian and should be fine um I will share the link also in

the chat in some minutes and then I created um a new AP which um yeah has number 7748 and basically a formalization of all this state conversion algorithm that we have discussed like in the last months uh or even more so I will switch to a better view here so I won't go through all this in detail because uh might be quite long but the style of the CIP

is like quite related to the execution specs codebase so it's mostly described as code which I hope will make things even uh easier to understand because I know maybe explaining all this in text would be a bit confusing so yeah uh if you are used to the execution spec test execution spec um in Python this will make hopefully a lot of sense but if not that's fine it should be pretty

intuitive um so basically in um this first like four or five go sections I just describ some helper functions or some cod additions in existing files and basically the last Cod chunk is the actual main um Logic for the conversion uh where you basically walk through the MPT and for each thing you have to migrate you have like all the logic so um yeah I added a lot of comments so just to make things as clear as

possible so hopefully this should feel simple I mean there shouldn't be any kind of weird thing um and I also in the rational section added like multiple sections with expected questions um that I guess most people reading the might have um like why we chose um uh to put the conversion in the proposed place in the execution pipeline like why um we choose

the conversion stri number um yeah Mis slots and other things something also that I want to say is that I included in this proposal also this idea of um uh like removing all these remaining accounts that are related to AP1 158 um because we have still some of them and we have talked about many times that during the conversion we can finish up this cleaning um so that is included too um so yeah so feel free to Simply take a

look and make any comment all the code here is very like high level nothing around this really depends directly on the work tree itself so we have to do another migration in the future probably will be pretty similar um so yeah I think it's like useful um to uh mature this and because we have to eventually do a conversion so um I think it's it's useful um to make progress on this and really solidify if this is the

right strategy or we have to include something else um that's cool anything else oh there's question from Gary in the chat um just confirming that there's no other changes just the gas witness cost I think that's right gon maybe you can confirm uh yeah sorry I need to remember I mean it's mostly that in book fixes uh we have a list in our repo that uh we tagged every PR that is uh connected uh yeah it's a um it's a

fix so the only major difference are the costs indeed we add the parent State Route in the execution witness uh so that stateless clients don't need to download two blocks to ex to run to execute the second block and uh we also used the 2935 that was agreed on in pectra so the the difference is that instead of at block activation inserting everything into the into the the contract uh we just insert the last block and that's it uh

we it should make a difference for that test net because we start at uh a Genesis but the behavior is a bit different so just be aware of that and yeah that's pretty much it oh right so there's it's a bug fix but I I can share the issue in the in the chat um the the major difference is where we charge the the witness gas cost because there are two cases where 164 of the gas is subtracted uh one of them is in create and the other one is in call and they don't happen at the same time meaning uh in I'm trying to remember but it's

explained in the issue I just shared uh for call I think uh they are charged before uh executing the the contract whereas for uh for contract creation they are uh charged as part of contract execution there's there's a bit of a finicky thing like this I don't quite remember uh but yeah if it's not clear from the issue I just shared we we will uh yeah we'll discuss about it and uh we we'll make sure we all agree okay cool if nothing else we can end the

call there thanks everybody talk to you soon thanks when do we want to launch 11 7 uh not before end of August okay Spencer did you have something yeah just a small thing while I would it be okay for us to change the fork name from prag to a different different name um maybe maybe we want to do this after deit 7 but um just from our side and there's some work arounds because we have the prag test as well within our framework uh yeah it's a bit of a pain I agree um yeah we we can change them we

can change them it's not it's not a big deal yeah let's do that what so um in fact we should do that in curtosis too currently everything is activated on prag right when you set a um when when you set a uh an active a fork activation time you say override PRACK time because when this branch that we use was created uh verle was supposed to be in prag now now it's Osaka um yeah now we want uh I mean in gu Master it's already known as veral

and not Prague but in that Branch it remained as Prague I think everybody was I mean okay I don't know I I get the impression that every other client was doing the same but it should not be known as uh as prag it shouldn't be known as veral time all right and I just want to point out uh that Perry was the one insisting that it should be called prag and now he's the one insisting it should be called veral yeah

yeah all right enough uh yeah any other question cool all right that's it then see you all take care all right bye thanks bye bye