eth_multicall Meeting, Sep 9, 2024

Transcript

hello everybody so Luc will probably not come today um and so let's begin yeah so on our side with help of kar and Lucas I'm investigating the uh problem with self-destruct logs uh s also helped uh potentially it will solve this maybe we'll also solve the ISS with uh the PES potentially but we'll see uh and we are currently working on it uh still fixing it so hopefully we'll fix it this week created the first PCH

but it is currently failing on rlp so we'll fix it and uh yeah that definitely was an issue uh so that's it with us how are you guys I remember there was some talks during the week on the chat board so if you would be able to recite that for audience that is watching on demand that would be perfect thank you hey um I can go next uh yeah as for me I um we had a bu in one of our tests that made it fail every once in a while which was

super weird because I don't really have any currency in my multic call code so it was very strange and uh kilari helped me track it down so it turned out to be a bug just in the test itself not in the code and it was really yeah strange because I was generating a address randomly and every once in a while it was having zeros in the beginning which caused a different guas price anyway um yeah and apart from that uh we had the team Meetup last week and during this Meetup my team kind of looked at the multioud pr as well and they merged it so um that was a big milestone for me

yay congratulations yeah thanks uh right there was also some chat in the Discord Channel um but maybe we can talk about that after other updates I have been doing some testing with the time stamp stuff and then but nothing else really and there's currently differences between the time stamps with the net NE mind is not returning the correct time stamp all of the cases I think that's still the biggest issue that's is with those

hashes and I I wouldn't think that the self destructing would be fixing some hashes well of course fixes hashes in the cases where the self DCT is used but it's not used in the most of the test mhm yeah but while we were fixing that we found out that maybe we were updating State on the same block and not on the previous block that c us so maybe this would be one of the big differences but yeah definitely time stamps would be the main culprit of any potential P changes that can appear oh yeah that would probably explain it I think it's somewhat confusing that when you're are specifying the block then you're

actually specifying the parent block and not the block that you are creating then you're just creating one block the current logic of ours is basically we we when we are given the state overrides we apply them in the current version that is online and public uh we are applying it to the same block onto which we then apply the transactions and in my understanding CA does this on the previous block so now we'll update it to be also on the previous book and this uh will also potentially change the hashing I find this a bit confusing I don't really understand what you mean by

apply the state overr right on the current blog so uh basically we can commit a tree on this current block and start working from there at least it was allowed in nethermind architecture so we were using that but now we will uh go to the and apply it to previous Block in the E call and it simul when you are specifying that block number then you're always specifying the parent block number and not the current block number and then when you're creating blocks then you are specifying the current block that is being created is is that what is what you mean by previous B yeah kind of

okay no so the the reason why I ask is because um the end result should be mostly the same I I feel like there might be some Corner cases that I'm missing but the state should be computed like the final State should be the same is it because there is um the commit make some difference so like in at least it is possible HH wise to which block that commit was allocated but we'll see oh oh yeah I see I see I see okay okay yeah ah okay that's a interesting

question we after oh then wait okay if that's what you mean then uh those changes uh will be applied as part of of the current block so I on that we agree I kind of misunderstood your question uh at first so all right so you're applying it on the beginning of the current block uh yes mhm yeah all right yes yes then time stamps are the like the biggest

yeah if we have different in state has then it's probably something else yep all right so you've discussed on the chart something about uh is simulate V2 can you talk a little bit about that oh yes um that's that's an idea to add basically in V2 return also the whole Merle proof that is needed for a client to do the simulation themselves and I think this is important

for improving the wallet ecosystem um so that basically when the user wants to check their token balance they are they also get a proof from the provider that they can check themselves uh and not only trust that their Balan is 4 the2 have you tried making a version out of it I'm just wondering that the proof is probably going to be really huge and I don't know if it's going to be so huge that it's going to be useful or not I think um well the proof will depend on the complexity of the

simulation yeah so we probably will have to limit like um I mean the the gas limit that we have kind of prevents it from going out of place or um yeah by setting a low gas limit uh it should be possible to contain the the size but I think yes it would be interesting I can I can work on a branch and then we do some tests with common use cases like just looking up balances and things that are most common to have ideas what would be the worst case scenario what could the user do to generate as big proof as possible

I'm really familiar how the proof works so I don't know what's the worst case what you need to do to quering worst case would be quering lots of different um pieces of state that are all over the Merkel tree and Merle tree is generally fairly well distributed so that's actually pretty easy to do Bas do a bunch of State reads and you'll get a bunch of like getting Banes yeah like every state read uh particularly so every state read you do is going to be problematic especially if you do them from different accounts because you won't even be able to get um like any sort of reuse in your in your

proof I mean this this is this is an idea that people have been wanting to do for very long time I think there's among the cord devs I think there's a lot of desire for it and the problem has always historically been the witness sizes for Merkel trees are just horrendously huge like many megabytes for a good witness and uh this is I think basically the the number one reason people want to move to verle trees is to address this issue so we can have witnesses that are not crazy big I can get witness size down to kilobytes instead of megabytes now for for a for a single RPC call I don't know if it's going to be in the megabytes range Witnesses are usually talked about in the in the context of blocks and so for a full block I believe their Merkel a Merkel

witness is megabytes a veral witness is like hundreds of kilobytes or something I'm guessing for an e call or an e simulate or whatever you're going to look at some amount smaller than that but it's still probably going to be huge that being said I'm also a big fan of uh proof this makes like clients possible look into you might want to look into Helios Helios was built by somebody I forget who and it's a light client that basically depends on this kind of proof Generation Um and it was a proof of concept and I think you can actually use it but they all R into the same problem these the proofs are just or the Witnesses are just too massive to be pragmatic but I'm I'm saying all this just to warn you ahead of time that this

is what you're run into but I'm a big fan of it yeah and especially in the case of helas I think a problem that they are facing is that uh to verify e call basically they need a lot of round trips so they they start executing and whenever they face a state that they're missing they have to query the provider and so on so that's why if we provide the proof up front that that's going to save them a lot of pain um and yes so to your point yes so it's for the blocks it is I think around like four megabytes or

so um but yeah given that we were working on uh V1 for almost a year um I think by the time we sh V2 um there's a good chance the verle will be around the corner so and and what happens if somebody simulates block that is is preal and or even preos yeah I mean that's that is a question if if you implement this after veral trees do you uh allow this function to work pre veral trees and just give Merkel Witnesses or not um I'm guessing my my recommendation would be

to have a client setting that says like maximum proof size or maximum witness size and then people running the RP RPC nodes can just configure it and say hey you're not allowed to request things bigger than like 200 kilobytes or whatever Y and I think if this is in place then we can already ship it with Merkel and it will already be good for many simple use cases which are very common yes luk please you a bit cut cut out uh I

canly is still in audible all right let's continue this one probably a sync and we'll be able to clear it up uh tomorrow or the next week uh I mean I doubt we'll clear it up next week I think it's gonna be an ongoing thing I would love to see um you do like a prototype implementation scena and see how painful the Witnesses are for simple things like balance hups and token transfers and swaps and things like that that' be very useful information y I had to question so as we are currently more or less released on

two clients uh when the others will jump a b that is an excellent question um so we should probably start talking about what our strategy is for merging getting this merged into getting this PR merged and so are the are the clients both passing all the tests right now or do we still in a state where we're not we have hash differences that make basically every test not equal but as far as Behavior goes they're mostly the same for the most right like this is just a bug hunt not a disagreement on how we should do things is that

correct there's the has differences and then time step differences and then there's that uh this D structor on how it logs asend I think that's the those are all the difference at the moment okay that's yes maybe we should and then the difference is on some error code so but I haven't been bothered too much about them in a way that just mostly about if the other client erors and other Cent doesn't eror then I think it's a bigger bug but if the error codes are written differently then I haven't been changing those too much okay so I think we need to figure out what our plan is for getting this

merged it does feel like we should get the two clients um passing all the tests before we merge but that sounds like we're pretty close to that maybe a week or two optimistically question optimistic I think okay so let let's say we hit the optimistic it doesn't really matter but the if we we get the two clients passing all or the majority of tests and then do we need to do anything else before we emerge you think or like should we go do another approach to all coord or do we just go ahead and merge it or talk talk to the people maintaining that repo and get them to merge it um and then go to all four devs and say hey new spec everybody please

Implement they have been also discussing about if this would be e or not and I'm at least bit confused on like since I have been thinking that this is not not shouldn't be e but then some people feel that it should be e so then I don't really understand how it should be so the the people that are in charge of Eeps have decided that it's not an e so by definition that means it's not an e there are a number of people who dislike the apis not being part of the EIP process and so that's what the discussion in alward's Discord is about as people disagreeing on whether or not execution API should be moved back into the E process or not in the meantime until that's resolved they are not e was e call and its derivatives in e

EIP process it was not it was in fact very little of the Json RPC spec went through the EIP process I think there's like one or two ends there's some yeah there in there I think more than one or two like 10 or something it's possible there is as many as 10 and then there those wallet methods that's under e as well like e get accounts yeah so there there was a period of time so so we started out if you go back in time to the beginning There Was You Know gu and parody basically and um guess I guess guess first should go back far enough and that

had an API a Jason RPC API and then the other clients copied that and then at some point after that people started wanting to make changes to those apis and there's a question okay where should we discuss this and the EIP process was used and then that was done for a while and then that was kicked out of the IP process to the execution API is repository instead and so that's where we're at now is my understanding correct that if we will like go the EIP road then the E simulate method would be enabled only on certain like spec versions and it will be like all right for it to be not enabled in like pre veral for example or

preul so this is this is exactly what or this is one of the reasons why this was moved out of the EIP process into execution specs because um from a core dev's perspective the core eips are all consensus things and execution API is not a consensus issue like you you don't need to the clients can disagree and nothing will break the blockchain won't break it'll just some Downstream apps will break and so it was moved out of the IP process for for that reason because they don't they also because they don't lock to versions at all and so even when this was part of the a process it wasn't version locked to any hard Fork because it's not part of consensus only consensus changes get locked to hard Fork versions MH thank

you uh so there is there exists a SE category within the E called the interfaces eips um and historically there has been some things there as was mentioned for example like the whole graphical end point that was speced out as an um as part of the interfaces C I think if there's ever a method that that needs AEP it would be assimilated up till now because um we've had to like just having the spec wasn't really enough for us we had to write down this whole uh notes document that is kind of weird very

weirdly placed uh lost and hidden in some uh doc folder that nobody checks because there's nothing else in there um so I think there there can be a case to be made for this but I know that yes the the maintainers like the a editors probably are not on board um but I'm not sure how to solve this either because I think like the how it is now the the spec itself is kind of lacking for our use case I would say that if simulate can be a case for an EIP as it's validate on

mode will sync up the understanding of client implementations on reactions on calls on basically every call so you would basically get guarantee that every transaction and every block can be executed in the same manner no matter the client so this can be went through this route like synchronization between clients so all of these arguments are arguments that other people have made and continue to make in favor of putting execution specs into the IP process the it is certainly not unreasonable to join that battle um but just be aware that people have been fighting this battle

for years now and so if we wanted to go that route it means this is very unlikely to get merged anytime soon so if we're okay with just sitting kind of this in this weird intermediate state for an undefined amount of time while we wait for that battle to be fought and this ultimately decided again and this just be separated like you merge the ethereum spec like say right now no matter how no matter when and then in like in some time uh we create an EIP and discuss it there so basically you get your like unstandardized version of it and then you synchronize it up

through an AAP you certainly can you're going if you want to fight the Battle of to put this stuff into eips or not I think merging it into the execution specs is going to weaken your argument significantly so the what the county argument you're going to get is like see the execution specs repo worked fine you guys got your thing in everybody's happy so yeah just um to point out so to get this PC as it is merged we would need a thumbs up from the other clients because so the def facto is that when a method is added to the east name space uh of the execution API then specs

report then it means that every client must implement it um and right now we don't have really an approval or uh veto from any CL from any other client apart from G and and uh NE mind uh so the least we should do is go to the all C devs again and say like hey guys we really need like a reaction whether you're okay with um implementing this in future or not like we we would need a decision from them yeah I would be totally fine with that just present we can present it one

more time just tell people remind people what it is and say is already in gu another mind is it is Basu and reth and who's the other one we missing aren't there five clients these days or is there only four now anyways get thumbs up from all the clients eron thank you if we can just get a thumbs up from the client saying yes we are not opposed to this method existing we do not have it implemented yet but at some point in the future we will then we can merge the the spec but I agree that we should get at least a thumbs up that it's a good idea do you know people from to if you can reach them

out not personally um at least not the people that would be making the sort of decision all cour does so we have reached out to them previously and no one said this is a bad idea they all said said um that's interesting it's low on our road low on our priority list so we're probably not going to do it now but we're fine with it so I got kind of a tentative thumbs up but nothing official this is like back channels discussion yeah I have a question for Cena uh when the like RPC is merged into potential rpcs that are under e uh does it mean like that everybody would see its hi tests as part of e hi

tests and thus would need to make sure that like first you implement it and only then you you pass the hi test so is it this yes potential risk mhm yeah get approv anytime soon yeah basically yeah it's if if this is merged and we merg the test then all the other clients will get an extra 100 something failing test because they haven't implemented it will will get high on their priority list definitely I'm assuming there's some way for them to just say we haven't implemented this and ignore the tests or is that not possible it's not possible as of now and I

think um I'm hesitant about that it opens doors to like a segregated e Nam space yeah I guess I guess you wouldn't want to ignore them but maybe like red green versus yellow where yellow is you know red means failing green means passing yellow means we haven't implemented this yet and so you still get the visibility that there's unimplemented code but you don't it's easy to differentiate between your clients's broken oh this actually means that either they so it is a good argument for inclusion of this into EAP proposal like set mindset because it would definitely allow them to like plan

ahead like by this time of this release we definitely shall include the ecap and uh until then we can like not think about it or its test but on this like EAP merger we definitely shall it be being included yeah I doubt that even if it uh becomes an EAP then there would be like an activation block where everybody starts turning on it simulat I think there's I don't see why that that should be necessary like we have it already implemented and be a to use it for Merle tree would

be like in megabytes range I think some people may be inclined to like await for vle oh if you're talking about V2 then maybe I misunderstood yeah that's uh a discussion we had yeah but if three clients haven't even yet started implementing V1 then maybe they'll be implementing V2 straight away so well there's a lot of things that I wanted in the V2 so I think it's still kind of research thing or something that we need to discuss what what we want to be included there MH yeah so the HP test situation does

make this ch Jane you're you're right I didn't realize that was going to you don't have a good solution to that I think I think the next step okay so how about this let's get the two clients passing all the tests we have then let's approach all core devs and say we have this we have Hive tests for it even but we're hesitant to enable The Hive tests in the main Hive test Suite until most or all the clients have implemented it and then maybe that maybe we can get some feedback from them on ideas for how we can deal with this so we can get the hip test updated without them breaking like failing all the tests or get some commitment from other clients to actually get this implemented

but I feel like get these two the two clients passing then go to ACD is probably the next two steps does anyone disagree with that is most three or four out of five is most what what is most clients three or four five oh I mean that's that that would be part of the question in all cordev like what at what point is it okay to implement to add Hive tests for Json RPC end points does it need all five clients or is four enough is three enough is two enough is one enough like where where's the threshold is it just as soon as you get a spec merged you go and add the hive test and if your client is now red

permanently that's your problem go implement this back I think that's a quick question for AC and they can make a call yep sounds ni okay so step one let's get the The Test passing yep we do all right do we have anything else for today I don't know if Lucas can now speak I don't know what Lucas had in mind can you hear me y yeah H yeah it was just talking that witness size is proportional to gas right so it's even simple eth call with 100 gas 100 million

gas can have a huge Witnesses and proof so that's not there's no connection to that it's it's a block or it's a single call or whatever and there was something I wanted to say but I don't remember now be honest ah that Vero might not happen so there is a big question mark there currently on ver trees well there probably something will happen with a bigger tree fun out but it doesn't have to be vertical Tre or it might be something else I see so they still want to solve the general problem of giant Witnesses it just might be something besides veral yeah yeah yeah

yes is there way of doing e call and getting the witness of e call that then we can at least know how big it can get not currently that's basically what Cena was proposing to add just do it any instead of E Yeah but I wonder if if you have e call already and then we want to do it in simulat no as far as I know no one's imped this yet like with 100 million gas like what is the worst gas size you can get it's going to be yeah is gigabytes or is it gigabytes or 10 megabytes or what is I you can you can do some back

of the napkin calculations each basically if you have a well distributed Merkel tree which we do then each storage read the worst case scenario has a certain amount of nodes in the tree that you're going to need and you can assume that you're going to need that every time because you have no overlap in your tree and so um you divide 10 million gas by the 5,000 gas per read that gives you a number of reads you multiply that out by basically the Merkel size per you may be minus 32 or 64 for the first two layers in the tree because those on will duplicate um but by and large you're going to end up with around that order of magnitude um like I said I think if Cena

in has in implementing I think the easiest solution here is just Cena implements it and we get a definitive answer yeah I guess s very practical then unless there's useful calls that you can make in the but in most cases you don't create that big witness yeah and that's my that's where I'm pretty sure that for really simple simulations it is feasible it will be around 100 kilobytes um well the the contract code is a question mark um I'm assuming that especially if clients like wallet

can hard code or cash contract code and we don't need to emit them in the witness that's going to give us a huge uh saving amount and I'm pretty sure most of the usual calls will be easily um easily done is possible but yeah it's um yeah I would like to prototype this and then we can test and have a more informed conversation yeah if you wanted to leave out contract code you want probably want to have some way to make it so the caller can say I already have the code for this and I expect it to be used please don't give it to me again like you say I already have the code at address X so don't include that in the

witness that way the caller can say you know here's maybe an array of things I already have or here's some things I suspect this call is going to include in the witness and I already have that data so please don't include it and that can definitely get your witness size down but it's going to increase complexity in the you know the API and so I think it's is a trade-off we'll have to make and I suspect if we go with Merle trees we're almost certainly going to have to make that trade-off and even with verle trees just code size is can get big though it is limited you know 27 25 kilobyte is the biggest code well the point with uh veral trees is that the code will be chunked so that you don't need to send the whole bite code and only yeah yeah I think it would be reasonable

to assume that the client knows the contract code because they they can be cured separately and then cashed well well so the problem is is the wallet is what you would want to do the proof verification not the app most likely and the app has context of what's being called but the wallet does not and so you need a wa for the app to tell the wallet hey here's the here's the code at X is probably going to be in the part of this call and that needs to then tell the the client the same thing yeah but you could make the witness so that just says that this code in this address was used and then you

first run it and then you get all those addresses and then you run get code for all of those and then you run it again or you don't need to run it again but then you can kind of have all the data after running the get codes and then you can cash those for later calls yeah I mean that's that's kind of what Helios is doing um like ca mentioned helos is running into problems with that though they doing that not only for code they're doing for everything so basically the client says you know here's um here's what was execute here's what we're executing or what we're exec the Helios client basically just makes calls to get proof for every single individual lookup and so we can definitely do better and limit that to just calls to lookup code um and maybe

that's a good tradeoff um I'm I'm a little hesitant like an ideal witness is one where the witness is everything you need all at once so you're not making a bunch of round trips but you're right the code can likely be cached very heavily by side unlike storage data yeah these are all good questions I think yeah gassing all that stuff sounds bit more difficult but I guess still useful to do that as well but then you need somehow need to indicate them what you have already can it be made sure that the the each simulate will just give all those

places that are touched and then they can be CED separately so that you need to make multiple calls instead of one what do you mean I mean that instead of providing the whole widness it will just say which which datas are need to be needed and then it kind of gives instruction what what you need to cury to get all witness or get all the data to have the witness I know how much you can shrink down down down witness but not kind of sending anything that can be curate separately it's it's certainly possible um you'd save some round trips but you

might end up with the client having to run it might work it would be better than the current situation with Helios I guess there's these are all things we could do as stepwise improvements with W full Witnesses on the one extreme and then Helios On The Other Extreme does the Helios work at the moment it does what it is programmed to do um it's not really usable yeah I think for a lot of stuff it is just not e call and estimate gas because like even if we do e stimulate with with uh proofs then es gas is another question um that one I have no idea how

we want to approach but certainly interesting question um anyhow guys I I have to drop off now I'm happy to discuss this further uh asnc or next week yeah let's dis continue discussion next week yep but from yeah what it sounds like is that there is some interest in this group of like potentially seeing uh if it's worth researching yeah I think so yeah yep all right um yeah then thank you and talk to you

later see you see you we have anything else I think not not really I don't think so yeah let's continue I think all next week all right thank you see