EOF Implementers Call #36

Transcript

all right um e implementers call on 6 March 2024 um guess the first thing we start off with is client updates um for Basu uh we have exchange and create for too um I've been focusing on making sure it passes the the uh reference tests that been published before I move on to many any more features um the big reference test we got done was um stack validation that one took a bit of work because I had to rewrite the entire stack validation section but it passed all the ones in the pr that I saw at the time um still need to put together a full um collection of all the other um get them all merged so we can make sure

that um we can the basic can run them get a a master Omnibus set um I also added a pretty printing function to our evm tool um I don't know if we want to once we get uh a spec nail down if we want to standardize a standard pretty printing format um that can be consumed um it's similar to what's seen in the some of the reference tests I can make it conform to that but if as an implementation guide if you get into something you don't understand pretty print the uh um the eof and you'll be able to see what's really going on rather than having to hand decode it so that's that's something it took took a couple hours for me to write but it's paid off in dividends and speeding up

some of the fixes um next on my list is Dragon anything from RE to report or RM to report uh yeah I am in full sing in implementation I did like first of the simpler up codes static jumps functions jump F data section Swap and basically those things uh the the thing that I Curr working on is youf create DX create I probably finish that in a few days and after that uh I still need to do calls and validation I still didn't like start

validation but yeah progressing oh good to hear um Hugo's next so I guess that means evm one is next whoever wants to speak for evm one uh yeah yeah no major updates from M we're still working on the calls implementation I think and X code Logic for legacy and adding more tests for TX create uh we yeah we've renamed grade three and Grate four in in implementation and in the tests

and um yeah mostly yeah working on on eding mod test and experting them for others to consume too um minor change in the TX create implementation was moving the validation um point where validation of unit call happens a bit later after we charge for uh for hen before before it was between charge and for H and uh IP 3860 charge now we moved it after all uh gas checks now we do the validation like it seemed this way that

it's like slightly fewer chance of Dos like scenarios this way so first we do all the checks then do this linear pass to validation yeah and like this there's here's the link to corresponding Chang in Spec which is like two line two line change um yeah I think that's that's all cool um three things I'll bring up after the go around great renaming the call renaming and the call return I think there was some discussions on various discords over the week on that just want to finalize on that um but let's table that um Charles is next on

my list anything from Viper uh no I uh think there might be some spec things that are waiting on me but I'm I'm not sure so if anybody is waiting on me for something just P me again okay um unrelated to eof I'd be interested to hear just how much of an impact T store would have on re-entrancy but we can table that to the end quite a lot we could enable it for all contracts um especially with the pricing changes that I've been proposing okay um Cyrus are you representing a team no I'm just here out of curiosity

cool and Peter's with evm one so that's everyone on my screen okay so um I guess we have the open discussion on spec issues uh we have create rename call rename call return on the list anything else people want to bring up Charles you said you had some spec items uh I said things uh might have fallen off my plate in terms of spec so I'm actually depending on other people to cool Peter has a hand up uh yeah I'd like to discuss the call St that check in create call and the new calls which

is going know we ping pong the spec back and forth so I think we should we should stabilize it and also the EIP 663 which is the exchange one which is waiting on approval from Alex and Alex had some concerns so we either clarify the concerns maybe and or if no concerns then we should approve this PR and and finalize this as well okay um exchange seems the most important from this list so let's start there let's just do this list in reverse order um what was Alex's uh concerns yeah I think Alex can you can you speak to that was that that there's three Ops in one Ty to that if your

voice can't handle it uh you typed will go through this today keep forgetting but are there concerns or do you just want to go through it and approve okay okay so it doesn't appear like there's any changes that need to be done to exchange so hopefully that'll get merged

soon um call stack depth uh you wanted to discuss that I think that's another discussion I forgot about us having you want to outline it for us yeah so I I don't have full background but I think there was intention to remove this check and have it um have the 63 rule uh we effectively limit the stack death in the new calls and also I think we were not fully clear enough fully clear on whether this C dep would remain in create sorry in eof create or KX create and then I think at one point we did a change to bring the C depth back

in the new calls because create you have create and takes create headed at that point and now we we're kind of making the move in the opposite direction and so I think the the arguments to remove it are that the the 63 rule which is now obligatory limits the Co stock co co yeah Co stock depth effectively to um I mean to it depends on the gas limit but and did this napkin map I guess to and that was 22 billion gas so that's quite a

lot and my calculation is 785 billion gas if it's 65 64s raised to 1024 times 100 which I think is the call number for warm so it's still a large number but still wisely reasonable yeah that's so so the the this number not being insanely large is one argument to keep the C dep the other one that I think I I had I don't know if it's valid it's just to touch Legacy Behavior as little as possible and since calls new and old calls can can be intertwined then it in

some in some way you know this tangentially touches the or crosses the Legacy behavior and also what I think youan also brought up is that I mean yeah the cost that is just a hard limit that we don't need to think about too much to say that we don't let the cost death become very large due to some weird gaming of the gas behavior so it's just just just a hard cup which is easy to reason about and it's maybe good to have as a fuse fuse

box mechanism yeah um I'm in favor of keeping the call depth for I think for the one of the reasons you mentioned is BAS does use a lot of the same code for call and for axt call for eof um and that would make it easier because all we're changing is what we pull off the stack and what we return in our situations when we fail um so we're trying to reuse as much data as possible um and another thing I mentioned is layer twos could do crazy things with their gas schedules and they could you know make um instead of 100 they could make calls one but even if they don't make a 100 um 785 billion is still reasonable because it

fits in a 4 byte signed integer um so that is something that I can see some people wanting to use because they have a space for um so yeah it's it's it's a it's unreasonable by mainnet standards but not unreasonable by implementation and I think gets uh burnt pixels um actually use that much gas for some of its generation for their Benchmark so I I think there's still value we can't rely completely on the 63 64s to provide a cap that won't be reached by 1024 and I think we still need a cap I don't know another example are tooling that do some fuzzing something like that they

like have very very high gas limit amount to f a lot and having like gas depth limit matters for them for example okay any other discussion about keeping or losing the limit yeah to me also feels safer I guess to keep it for both calls and crates uh but an extra question then is whether we want an special status quote for calls uh which failed because of this limit this is what suggested in the

pr and because this failure is different from others because like execution didn't start and gas was not consumed for execution so it's like a light failure of the call right uh I tend to think that we don't need this extra status code but I'm not sure which one would you use instead regular failer because it's not a revert but just a failure however for a regular regular failure there would be all gas consumed meaning I mean the Coler could reasonably decide whether to continue or or not based on that piece of information because if if that's a hard

failure then the color only has what 5,000 or the one 164 of gas left and when that was the light failure then they have pretty much all the gas that they they Des send there initially so it's there might be this decision point to be made to be to be to be kept or or I don't know exposed in the in this in this API also just to make it complete there's also the second light failure scenario which is not having enough balance for the value send so it's not only about the cold up cold depth stack cold stack

depth what do you think yeah I guess I'm trying to paper to see what call and create do on call limit reached oh that's a create the new creates have different stack return because they return the address on success and zero on failure so they kind of different different uh case yeah whereas the new calls will return zero on success and one on revert two on the heart

failure and you know the life failure either goes to well it can also go to one because the the gas implications are the same at least that would um that would dispel one concerns but but but one means that the the frame I mean the execution frame began which is not the case with the light failure um for what it's worth um I don't have a strong opinion on this but from the compiler perspective uh we wouldn't generally care unless people started asking for it and I think it's a fairly

Niche case to check whether the I mean an exceptional halt is an exceptional halt I mean right now we don't even distinguish between revert and exceptional halt so um yeah it's okay way yeah from what I read from the yellow paper it's treated the same as if there's insufficient funds passed in so that would be a light failure I think hold on about the Viper story there is no distinction right now between the heart and right revert and failure because it's just always zero yeah I'm saying we wouldn't really you wouldn't if you had the

opportunity okay and yeah I mean we might care about revert versus exceptional failure but distinguishing further is not really um I mean usually you just bubble it up or try to bail out with yeah like gutfield which one would you put the light failures in which category revert doesn't compile doesn't take all gas whereas exceptional halts do so I think light failures would go in the same categories reverts and and like the distinction between the colie frame have begun or not is not really that

important I don't know to me it seems to very different sit situation one like the C code reverted somehow and so signals to you that there was some assertion failure and when it didn't start it at all it's like different for a caller yeah actually that's a good point um we don't really care actually how much gas the contract use but whether it's a system failure or user failure user generated failure right but the light failure one is is

fuzzy on that because the insufficient balance can be can be the user F that's a user failure yeah what they call the call St death failure is more system failure so it's a mixed flag like new tree yeah we I mean we're discussing whether or not to add it would matter a lot but if we have distinction between the failure that could happen we should that proper like number that's going to flag that so if we put a case for this in a

spec we're going to have to write a reference test for it too so keep that in mind by the way I don't really understand why um we have failure um light failure for the call stack depth being 1024 that seems like an exceptional failure it always has been like that I think someone correct me so consume all gas so that would be a change from from Legacy calls might make sense but what's do Legacy calls do exceptional failures

right let look this to be extra sure so all the Legacy calls so they return zero but they do not hold on they do not consume all gas uh in I in the balance case and also in the PO de case they are light fers at least in dm1 implementation I don't know that's reference no

course and create oh create light print what why um by which part what what why don't we treat call depth limit as an exceptional why different it doesn't spend it doesn't spend all gas that's the difference from

River or in this case ex exception why is it treated differently from like a SC from like a stack overl underflow yeah I don't maybe it's implementation detail well the Coler can continue after this unlike data stack overflow the color can

continue oh I see the color's fault not the color is fault I see they can continue but they have barely any gas left but gas is not consumed for Andre you saying that it's the caller's fault right for these two cases for for for operan St underflow no for for call depth limit right okay okay okay sorry I'm so that's that's how it's different from St overflow so you don't even enter the call in context okay yeah no I mean no we

don't probably that's the reason you know the reference test on like static call depth isn't very helpful but they do set the gas limit to an absurdly high a ridiculous but not unreasonable 892 billion gas so this is testable yeah I mean I don't even actually know why I mean if it's the caller's fault then you shouldn't even um get a return code you should just exceptionally halt that's my opinion okay but now I given that create the old

create old calls have the light fa failure do we want to change this for the new instructions um I guess I can see the value how the extra return code because it should basically not happen right like if you're trying to make the call and um you're at the end of the call stack and you can't make the call then all you can do is clean up all you would want to do is clean up sorry wait you would you would like to

have the new return code but consume o gas for C no no no I'm I'm agreeing with you just have the light failure I mean the only reason you would want to do this I think is if you're trying to introspect um the evm state which I don't know if that's really a good thing but you can imagine like a script which is like do something different if you're at the end of the stack uh some transactions where like maybe some of these like uh pathological transactions where you're trying to like consume as much gas as possible you can do something different depending on if you're at the end of the call act or not but like for most user applications

regular applications you wouldn't really care right h and um kind of the same thing with the send value thing like normally if you try to send value and you don't actually

have the value to send it's like an error I don't know why I can't really think of a case where you would want to catch that you know I don't know I mean just it it always works like this I think it's more important that it's well defined than it makes sense because it's such an Edge case we also want to make sure that we don't create an exploit but I'm not seeing how we could do that unless we deliberately cause problems I don't

know I mean the conservative thing is always just like preserve semantics right um and the most like the I guess the most in line with the old ones would be to use the reverse return code and not consume all gas but I guess it's the least Divergence yeah and and we just do not uh like using the language you used do not let the color introspect the state of the evm that led to the return code one

so it might be either a revert in the in in the frame in the color frame in the C frame or a insufficient balance or a cold spe dep case and this is equivalent to the old call returning zero so that's like the if if that if we try kind of apply this rule then I guess this would be the the the outcome okay unless we find problems with it

let's just go ahead with that rule then okay I can I can spell it out on the on the channel so we can read it and and uh see how it's how it sounds in writing okay next on the list um we kind of touched on this call returns and I just want to make sure from a uh user experience perspective that this will be a problem for the EXT calls we're flipping where you know Zer failure one of success zero is success and the higher numbers are different forms of failure um in solidity there is a low-level call delegate call and static call function um and returns a t one of them is I uh

my concern is if we flip the E switch do we want to either fail or have solidity mask it and if so would they provide um an EXT call that would preserve the value as an En and since we got Charles on the call and be worth asking does Viper have a similar low-level call function would you map it or does it impact users you know how would that work um that's interesting actually I didn't think about the so we have Rockall and interestingly actually um we're adding um a keyword to

Viper um which syntactically differentiates between external and internal calls okay and the name of the keyword is confusingly actually EXT call which I think was a coincidence but maybe um on one of these calls uh Peter said it said it or I said it and then it buried in everybody's subconscious and then it it manifested in in in both implementations but anyways um so I'm wondering first of all if that might be a point of confusion but yes yes that's next on the agenda that's the point I want to make so let's focus on the values then we'll focus on

the name next um from the the point of view of compiler generated call instructions uh we prefer zero is Success because it reduces an instruction right so if you saw you were running an eof mode and they had the low-level Callum struction would you be able to um flip it for them turn it into a Boolean not the result we would probably change the API okay so you make them change it go ahead yeah if we're running in eof mode uh well I need to think through this

actually but um one thing is like if we're running in eof mode you like reverse the argument of the return values so that they're forced to uh rewrite the code but yeah it could be an issue I can also check how often people people actually check the return value people usually don't do it unless they um except for these erc20 transfer things where um a bunch of tokens are like non-compliant and they don't return true or else they um express it in the error condition I have to think about it

Alex yeah Sol it would be it would be fine because the uh the variables in the Tuple are typed so if it's currently a Boolean this would change into like an enum or a number um so users would need to to change the the the use cases um I think at this point users directly don't really use the lowlevel calls but all the all the bigger Li libraries do um so something like um soulmate that's this other one so lady yeah I think these heavily rely on the on the X because they don't want to have anything

done by the the compiler but in many cases they also just use inline assembly directly um yeah this is this is a good point basically these libraries would need an EF version which which may be a problem so sorry I think even if the um signatures are changed it's still very error prone because people are going to have to go through and look for these low-level call things and then update it and it's usually called like success comma result equals raw call or something right and if you don't rename that variable to is uh failure comma so you have to like

rename the variable and then change the check to um and I think that seems kind of error prone but doable but yeah I think maybe the more important the more important um outcome of this is that maybe solidity should have um some kind of a a meta variable to show if you're in UF mode and there were some ideas to even expose like the evm version um to the contracts during compilation but maybe e would would make sense the the other option is if the lowlevel call is just called differently like X

call um then it wouldn't compile but yeah I think we probably have to have like a a discussion with Daniel proper discussion with Daniel about this yeah my thought is if you're doing assembly YOLO you're on your own um but what solidity and Viper could do is they could keep the original call and depending upon whether you're in eof or normal mode if you're in normal mode you keep doing what you're doing if you're in eof mode you just do a KN on the value and you still get your Boolean there but we introduce also an X call um that will preserve the Eno as a number and then that will fail in non eoff mode so you still have portable code that

behaves like you might accept expect it and if you really care about what the return value was you would use xcode but maybe that's a bad idea I see Alex's hand went right up so yeah my my concern is really the the libraries so there it has been painful with open Zeppelin so one one example I mean it's not a problem of open Zeppelin per se but um this has has been a problem for like two years back like three four years ago that a lot of the the projects at the time dependent on the open Zeppelin libraries and open Zeppelin was was not uh not in a position or not willing to update to solidity 0.5 for a longer per period of

time so they maintained the libraries on 0.4 and this meant that basically a lot of the ecosystem was still stuck with 0.4 when the solid team was working uh like a year on 0.5 and the same problem could occur you know with the the new calls because basically the all the libraries would would need to be change to be e compatible and and that's just a question you how to handle it so maybe the right way to do this is to invite maybe soulmate so or open Zeppelin or all of them to the call and and go through some of the possible solutions um I agree with um Dano I

think the most sane thing to do here is just I mean Rock call is not actually that low level so the most thing to do is just to flip the error condition if you're an e if you if we're emitting call 2 or ex call now and then there can but the raw call is configurable so you can add a flag to the signature which is like use euf enum and if you do that then then you get the raw error code or the raw return code yeah because python can do vartic return types based on inputs I'm not sure solidity can uh we don't not uh VI in Viper you can't do

that in general but for built-ins we can um bypass the type system so we can do whatever we want and also the question is what what was what is the the main reason to use the the lowlevel calls if that is for example to have control over the gas parameter or the output buffer that's not that's not present in uh in the anymore it used to be used uh sorry was that a question Peter like I thought if if by removing the output buffer and gas parameters we

are removing the use case or the main use case for lowlevel call maybe there's problem so people to some extent um people use the the lowlevel call but I think more frequently they use inline assembly and the reason is they do this in um certain highly optimized cases for like erc20 transfers and you know e transfer um even the signature verification uh calling the the precompile people tend to do this um in inline assembly not relying on solidity itself because they the measurements

show that it's much more efficient if they do it in assembly um whatever the reason may be the case is that most of these popular libraries use inline assembly uh for the calls and which means that they will need to have an UF version uh which may be a barrier for adoption I think that's a key point to maybe discuss and uh bring some of these people on the call yeah I me because jumps if if your if your assembly does a jump guess what you got to rewrite it uh something they still structured ual so they don't have jumps or anything like

that but they do have calls calls and static calls car you saying yeah something that I just thought of which is related to um um which is not the return code but the return data buffer uh which is exactly if we don't expose if we don't fill in the output buffer then users need to load from Return data and that uh like right now Viper doesn't expose the return data buffer to users but we might want um in in in EXT call

world and it might be kind of user unfriendly with the exceptional whole te for out of bounds which I don't actually know what the current status of that is okay um so not much can be made on this till we get solidity on the call let's see if we can try and get them in two weeks so lady um next to last on my list actually two things is the renames um I don't think the create renames are too controversial I already did the changes in in uh

Basu uh but the call ones especially with the external keyword um that does make me wonder if we should open the door again on the callary name and come up with a different like maybe just the letter X instead of the XT I don't I don't know uh from my side I would like to have s and D at the start or at the end of the name because when it's in middly it's very hard to basically to read that other that other than that change seems better than what was

previously yeah maybe calls out ecstatic call S EXT call yeah I thought we wanted s and D first e call Se call D call I think the reading issue is not as big if it's a yeah I like that yeah because they're all part of the same Series E call S call and Ed but do you want to spell delegate out every time can can you uh elaborate on the

issue of the EXT prefix because it has precedent with ext code so for one there is the external modifier and solidity and if you're I mean if you're doing an EXT call it's necessarily an external call um even if it is to yourself even it's an internal it's a mixed it's a call but said he was adding the external um keyword maybe we stick with external and and we just it's not too bad I don't know just that's one thing that crossed my mind as Charles is mentioning what Viper commands were but even if you target yourself it becomes an external call so it creates a new frame right if it does an actual frame

call and it doesn't just inline it as a jump you mean in solidity in solidity like an internal call won't always be external even though it looks like a function in the call you could just jump to it but if you're using EXT call it will always be an external call so it it lines up there it still lines up I mean maybe we should just stick with ext I like what Charles

proposed e e s and Ed yeah but I'm I don't have any strong opinion that honest yeah but what where do you have EXT call in solidity for external we don't write it go ahead you meant external like in full right not EXT but the external was a key word but EXT stands for external so but It ultimately has the same semantics so it's not too much of a problem

there I don't have anything against the ones that char posted just want to um looks like Charles is using the word await as well so I don't think there'll be the name Collision no uh there was a lot of bench uh bike shutting over the name and now we've settled on EX call um I'm okay I'm okay with the um short names because it's assembly and ponics are usually short I'm personally not a fan of having delegate call and static call but you know it is what it is um what I care more about is being

consistent and even if you look at the exchange instruction that wasn't shorten so I think generally the evm doesn't really use short hands um with the exception of like the the store and loads right so it's not really fully consistent but I mean I think that's why I'm leaning towards the the non extremely shorten versions EXT call exts call extd call I mean I would rather just ex St delegate yeah that's my favorite version too I

mean it's more mental load you know with the short hand okay EXT for EXT static EXT delegate and EXT okay that's enough bike shedding I guess for the day then um yeah anything else people want to discuss we got 10 minutes left we don't have to go the full distance who the the version that Andro posted is is what we take out of this yes okay um I can make a PR to the E

measp and advertise it onm Channel again and okay Charles Charles uh if we're discussing other things I want to see if we can revisit the return data copy uh return dat copy okay what's what's your concern go ahead and start oh it's just uh what we're discussing earlier which is um if the users are accessing the return data buffer directly it's going to be kind of confusing because it behaves uh differently and it's not very user

friendly um we can work around it which is uh if somebody does raw call then we we do the copy from the return data buffer for them um but yeah for from a ux perspective if we want to give access to the return data buffer correctly or directly then not yeah it's just not very user friendly like the compiler can deal with it but I worry about users more I don't have any strong opinions on

it ifon team have any comments is there a proposed for change or it's just a notation that it's user unfriendly sufficient um well the change would be to make it consistent with the other copy instructions and does anybody know why uh the current return dat a

copy uh has the out of- bounds Behavior or was it just like maybe kind of an oversight that the spec diverged from the other copy instructions you mean the the current one the the proposed one or the live one the live one the live one was deliberate the the idea was that the the previous Behavior was not not good and people wanted to move towards this direction but it never happened because it's every time I watch a solidity traps presentation that's always one of the things they presented

there yeah I think again it's the the question of consistency wins you know the less mental load is there if uh less exceptional cases we have in the in Isa and return data copy is like a big exception so I sense disagreement on it but I

don't sense any s very perm div Vision on it so I don't know if there's what consensus we should change it or not so it's it's kind of a sticky question on that so I think Sol was really in favor of um uh making it non-terminating because right now they they have to have like extra code um to check the length yeah I mean I was kind of uh in the don't really care Camp uh but like yeah if we want to expose

return data buffer to users it's much better to be consistent yeah I mean the the the risk have but I don't think it's an actual risk because if the code still checks for the length uh that code will still work it just won't be optimal um but people using return data copy and uh calls in assembly they will need to update assembly for the calls anyway um at the same time they could simplify the return data copy pieces but if they don't simplify it it it should still work um with the you know if it's not terminating yeah I think like in general

what happens is um you end up needing to do the check so um I think from from a ux perspective it's better to be fail gracefully because um yeah because I'm not sure that there's any savings and then people have to add this check and then the code gets uh more bloed I guess

um but that's all I really wanted to see on it and uh to bring it up for discussion again if it's not too late to change it uh we can reconsider um feels a bit like it's probably too late if faciliity is on it but we can discuss it again next week with solidity um the friction is is you not necessarily the best but the gas are actually in agreement solity is also in favor of the non the zero pading version okay so what's the outcome I was

reading specs we're going to change uh I think the only uh when we all spoke uh on about return data copy in the previous discussions and when has been long call uh basically the conclusion has been the zero padding version is preferable but to be conservative just uh but the current option was chosen because it's conservative and doesn't change any semantics soid Viper with the zero paded version I would say so okay I don't think there's any tests

written for this yet so this is probably the last chance we have to change it I'll post the proposal to change it to zero padded on evm along with the other decisions and not decisions we made um okay all right that's all uh see everyone next week thanks everybody see just one last thing if I may say so do we want to keep the 1600 time because in that case I might have to change the

protocol calendar 1,500 the current time 1600 the current time yes oh because in the United States yeah DST yeah non-metric daylight savings time is going cause problems um which means it's going to be an hour later earlier for United States it is an hour later yeah I just remember you are right um let's keep the UTC time I think that's what all cevs does and let's stick with UTC time okay in that case I will change it for uh people in United

States I mean like protocol calendar and we'll keep it at, 1600 UTC yeah sounds good perfect thank you right thanks bye thank you bye right