eth_multicall Meeting, Jan 13, 2025

Transcript

but yeah usually I prefer that we have all the implementors here who can actually actually do stuff I I don't have any updates uh for me nothing as well uh I mean there are some updates uh I removed total difficulty and uh ped the value of difficulty to zero and I was looking through the block rlps uh right now for somehow from neither mind I am getting duplicates blocks uh duplicates uh dummy blocks so I am fixing that so that's many we the duplicate your child and

Cent are the same uh sorry can I repeat it please uh what do you mean by duplicate blocks do do we mean that it tries to recreate some blocks that already exist in the chain yes yes yes so if I compare the outputs I can see that for block number zero sorry not zero let me see I can see block should start from 13 and should go 14 so there are blocks for 13 but there are other blocks which are which which has numbers like one 2 three that should not be there no yeah

have you been testing I guess the the test shoot if there if you have been getting more test to pass or is it D gate block now kind of causing all of them to fail again uh right now yeah duplicates block causing most of the test fail but before that I was I have reduced the failing test significantly like 20 or 21 from 83 to it was 60 sorry 72 yeah yeah and for rlp I need some help uh do you know how do I get exact rlp I mean uh hold decoded rlp for a

block while like debugging or shall I just compare the logs or get it from the logs I don't believe well there there should be tools out there I I don't have any links handy where you can give it a transaction and get back in rlp um I would assume though that within nether mind there's probably some built-in mechanism for for doing this that you might want to find out lcas we have we have we have we have four transactions get draw transactions in point but you don't have one for blocks for Block logs I need to see so you're

talking about like an RPC end point yeah yeah but yeah so I don't think an RPC endpoint would help what you need is you need something that will take a because the E simulate throws away all of its results at the end so any RPC wouldn't give you what you want um because the the block disappears before the end of it wa what you want is something where you can take the Json of a transaction I don't know this doesn't help yeah I think Lucas is the right person to ask I'm sure there's a way but probably some debug method or something I don't think there's many

tools for that because the blocks are more rare that people would use but for the transaction I would imagine there would be tools yeah but we don't actually want third party tools we specifically want to know what the nethermind client the rlp it's generating and same thing for GU we want gu to tell us the actual rlp all right you don't want to test against something else that third party one we need to know if the hashes are wrong it means the rlps are different and so we need the rlp that gu generated and the rlp that nethermind generated but because all of the blocks are transient they only exist for the life

of the E simulate RPC call you need something internal to get it like log line for example or some hidden parameter that lets you tell it to give me the rlp back uh it's it's also very difficult while executing a h test I mean yeah it's difficult to get stuff out from the highve when it's running like where it fails I can see the logs different differences in logs but for RL don't I I feel like the right answer

here um not just for for for testing in our case for Hive tests but for other people as well troubleshooting the right answer feels like is to have an optional parameter on each simulate that says give me rlp back instead of Json like an encoding block encoding or something like that would we add this to the spec or would be this just something I I don't need undocumented feature for V1 spec maybe undocumented feature and for v2 spec maybe documented feature is well I don't know how useful that feature is after the things are [Music]

working most clients would not Ben well I don't know the thing is is you you can generate one from the other so from an i you can generate the transaction Json from the transaction Json gener to rlp so there's not really any critical need to return one or the other other than for troubleshooting but some cons is would be troubleshooting is a thing that people have to do often and it does feel like it would be nice to be able to get this data easily we at least need to get that to the get as well yeah if we're using simulate to troubleshoot a consensus failure for

example it'd be great if you could just say okay from block X which was the block that we failed on um simulate block or its child and its child and give me the rlps for the blocks and the transactions or something yeah somewhat rare case but yeah yeah you're right it is it is rare it's not something people are going to use regularly not in production it's more of a Dev case I'm fine with it being an undocumented feature where we just have an optional parameter that you can pass that we'll say return R rlp not Json or maybe in addition to the

Json no probably probably separate uh that in the spec the tests are not up to date but in the hive we have the test where they are up to date but even then when we are comparing we don't have many that kind of absolute test they are most comparing the clients to each other so it might be in some case that that the gate is wrong for example and then we need to if that that is found then we need to kind of figure out what to do yeah so to expand on that a little bit Gabriel The Hive test Suite is what we're using the hive tests and and those are updated and modified regularly but if you get a different result from the

test let us know in telegram or in this meeting so we can evaluate whether your client's wrong or gests wrong or the tests are wrong because the test right now the the result of a test is literally just whatever gu returns which may not be right we found several situations where gu was wrong and we had to change the tests so um you can use a test if you see a difference you know do a quick check over your code to see if it's obviously the bug is yours but if you feel uncertain like you feel like your code is right and the tests are wrong um that's very reasonable so just share it with us let us know and uh we can investigate and decide who's right okay are you uh Gabriel are you in the telegram group yeah no sorry uh could

you could you please add me or send me the telegram group CH yep and and for the hive test is that uh particular Branch or in in the main I don't know CL do you know where the hyp test livees is it a it's a branch I think it's can share it as well thank you very much we just adding uh to bzel as well so that will definitely help thank you anything else CA is not around I don't

know what CA is C has CA been able to contact those RPC providers about the know gas estimation we have anything else then I don't have anything else do you have any questions Gabriel that you've run into in your implementation yet uh we have uh an initial implementation um but it's very basic yes very very basic scenarios okay how we adding more features now awesome very glad to hear Bas was adding e simulate CA said that is not coming I'll probably have more questions

as as soon as I get deeper into the implementation y feel free to ask and telegram or we're here thank you thank you out of curiosity why do we have a telegram Channel instead of just using a channel in the E R&D Discord server anyone know I think initially net might wanted to it to be in the in the tegram that's what we have is sticking to I see I don't know if nether might use Telegram in general or not I used to use uh we use tegram we can use something

different if needed it's not that we don't if we had a a lot of dialogue in the channel then I would probably Lobby to move to a better chat client that doesn't suck but um and we don't actually talk about that much in chat it's usually just little issues here or there so not not worth moving just curious uh there was I was mentioned on some discussion should I but I joined late so not sure if I can help with something so got something persistent or something about rlp yeah so we the you might know the answer to this so when you have two clients that have a consensus difference one of the troubleshooting steps is to share block rlps with each other right and

transaction r with each other etc etc what mechanism do you guys use to actually get those Roop encoded things out of your client uh so the bug get B blocks is one way so those are saved at least temporarily uh but I think it's currently even to dis for us the bu get BL but blocks we also do some um uh we do some Auto dump when we have this kind of potential consensus issue we do auto dump of uh rlp receipts by default I think and if the no is configured you can also Auto dump traces um this is

those are dumped to like a log file or something on dis no a temporary file on disk like we create new temporary file we log that we log that the file we dumped to so it's a temp file because um yeah because we don't want to clutter Logs with it so we just log that we them to the St file and uh yeah and generally uh if it's not an rlp problem or something we generally use traces to uh to uh compare what's going on in consensus isue uh but that often needs archive noes are really useful for for getting the traces right so um yeah it's good to

have an access to an archive note uh sometimes for example because the traces for the whole block can be huge uh we can uh already found find the transaction that is the problem through receipts so that's why we also dump receipts uh because it's easy to compare receipts and for example it has different gas usage in the uh in some transaction and then we can easily identify this transaction and only compare traces for this transaction which makes it a bit simpler so for eth simulate eth simulate is much similar more similar to eth call in that it's all transient which means I'm guessing it won't make it to get bad blocks um any dumps probably won't get triggered do you have thoughts on what

the best way to get rlp for simulate calls so that this is for we could we could either add the parameter something that we return rlps right uh that's for the rlps in general and the second one would be to develop uh eth uh simulate Trace right or something the bug simulate Trace so not only simulate but also trace the transactions yeah I think I'm leaning more towards the the first thing you suggested where we just have I I don't think they are exclusive

steps uh sure I think for for the immediate problem of we need the test to Line pass for both I feel like just having it some hidden parameter or secret parameter that you can pass to simulate V1 and if that parameter is present then the client will return rlp instead of Json for or the blocks or something like that would be the simplest maybe addition or it can return additional rlp in dayon somewhere yeah maybe just in the block perhaps we could have an extra field which is the rlp encoding of that block yeah I think Rohit could also pick

one of the tests which kind of fails and then dumps everything about it and then share that with the Cena and then Cena could maybe look where the difference arises probably just you know in a debugging window even or bugging terminal you can get the rlp um it would just be nice if we had a smoother way to do it yeah but the problem is uh for case uh I mean for me the de ter is kind of Cy where I cannot if if it's running three tests tests then first for first test I cannot debug for other two

consecutive tests I can debug something like it's always not very uh predictable no you have some failures that are non-deterministic or heisenbug I think I remember seeing that in the net mind that sometimes some test pass and sometimes they didn't pass interesting we definitely get to the bottom of that because that shouldn't happen are the tests run in parallel in any case if they are there might uh the test might not be th safe I think they R par and there might be race condition as well maybe or maybe

there's a race condition in the client itself well there shouldn't be a race condition but there definitely could be um like there incorrect incorrect isolation between the tests yeah exactly like a Shar state or something between the test breaks it or between the calls even right in the Cent or two e simulates are which would definitely be something you want the test to catch because you know in production you're going to have many people calling eat simulate at the same time and you definitely don't want them sharing steak exactly so that's something to to double check if we are not showing State I think we shouldn't um

but yeah I'm pretty sure we we aren't unless we broke something recently but I'm pretty sure we aren't because that that's one of the like easiest mistake and we are looking yeah well can Rohit produce some output of the immediate output and then give that to ca with that we could kind of maybe figure it out where where the problems are for those cases where where you can kind of deterministic defy that this goes wrong okay yeah but the question is how do I do how do I get the output during

the execution of a high test can you get it to the not to print out stuff I don't know how it would be easiest to do oh you could just modify the P returns that stuff or you recreate that P test somewhere else as well that's also one case I think the short answer is there's no cross client uniform way to do what you're wanting um you'll probably have to ask someone on the nethermind team who has troubleshooted similar problems before with Hive and figure out how people do it in nether mind specifically and I guess for the traces there's no

standardized method to kind of show those either but I guess you can just PST the block R then we could right away see which has is different I guess I think for traces we can we can I can add something in the configuration file for nether mind right Lucas sorry what was the question to to enable tracing uh I just modifi I just need to modify the configuration file that execut what kind of tracing what kind of tracing tracing the transaction in in the evm OR tracing in terms of uh logs TR Trace value so yeah logs it's only configuration but if you want

tracing transaction you need to attach a tracer to to execution so there are G and parity like traces right CU it's it's about something different so depends what you for E simulate we don't have um any sort of tracing ability at least yeah but we can like add it programmatically temporarily right say yeah yeah copy the kind of how it works in the trace calls got it it's fairly simple to do okay okay sure uh sure I'll find a way to do

that and share the outputs anything else with the gas estimation stuff I would wait back to hear from Cena he has any comments yeah I agree so when trade simulate well CA has been wanting that and said it should be easy to do but then we haven't done anything about it yeah so similarly it should be fair fairly easy just attach a tracer to execution I think it's similar in G and you getes for for transaction or

block if it's super easy then in theory we could add it to e simulate B2 if you're working on for the gas estimation stuff yeah but this is very heavy so it has to be under some parameters so you want it not in the E name space so it can be it could be it could be in this name space if needed the bug name space would be better probably uh the bug and or parity name space because depending on the uh Trace format right because it would be consistent with with the other stuff yeah well I think it could be like it Trace simulate or something yeah it could even be share all the almost all the code it would just be invoked with

the parameter right that's pretty much it well if you could make an implementation of it then we could kind start from there I don't know how different those can we get traces to match in the get and Ne mind or is that something that they even want want to be doing or I think they already match or almost match so it should be doable tracing you with tracing you do you don't get the RPI I guess but I think it would be still useful so well that would be one other way to debug this that we just add new method that is almost the same yeah but rlp is also useful so both are

useful and we specifically don't want it to be e simulate V2 or simulate V3 because uh operators like just has a parameter on those because no operators want the ability to disable specific methods is that right might be yeah it might be yeah because it's it will be a lot heavier so maybe I much it's a lot heavier and it's not gased properly no no no it's memory memory mostly and just a sheer number of data being transferred because tra can be very big yeah I would also think it should be a different

method clear is it different mhm the back Trace simulate is fine I guess it is it the the tracing stuff is in the deach name space so I think it would fit there it's in the debug uh for the G like tracing and in Trace in for uh for parity like uh which uh parity like uh is I think support by all clients except G not sure about B but I think too I know that Aron and ref definitely support parity like

tracing is there any preference which one it should be uh well we definitely want to go with debug and let's say the for the G like tracing and the parity like tracing let's say they are more optional um but uh there are some advantages to them because they are generally lighter than G traces like they the data is smaller like they can be order of magnitude smaller in terms of data I think we would prefer the heavier options here if you just used for debugging and it's used for users well don't know I think it's so for example both of those are used by

block explorers right for example it's a use case to get all the data about transactions so they are using it and they're caching then later in their own databases the data uh that they um extract from the traces uh do you see there should be utility for the explorers to use it simulate I don't know for it simulate so but I'm I'm giving you a bit more context where this potentially is being used so it's it's called in debug right but it's sometimes not only debug right so it's it's also like it's like more more deep inspection of what's

being done so it's all so it makes sense for I think it might make sense for potential simulate right so like which code Buffs were taken or which whatever right in in the actual simulation and you can inspect that very deeply in in that can you trace block right away or or do you have to trace each transaction separately at the moment uh I think both are generally available I'm not entirely sure which one is but generally yeah and you can you can trace an existing transaction that was already on blockchain right if you have the state and you can trace um new transactions new virtual like eth call but just doing

a trace on the on the current state for example so you can trace both existing ones that were included or you can trace new ones that haven't been included yet right I guess with the block expess they could be utied when you have pending transaction it would maybe simulate what the pending transaction does yeah yeah yes yes yes I think that's used quite a lot in terms of like simulating uh future transactions and inspecting what they are doing maybe from some some security analyzers do that too maybe yeah I think Cena also has a lot of opinions because he also likes likes tracers

stuff well they are useful like I don't know maybe all the use cases that's maybe a b problem but they are definitely useful and needed yeah is it difficult to add either of those or or both of both of them and test them if that for us it's for us it's only creating uh different uh Tracer objects and passing them correctly and then converting them back to Json so it's fairly simple I guess there's some of the it's symbol kind of stuff that is being done is something that doesn't exist on the

tracers like if you're creating new blocks then does that app there and if you're doing if you're doing overrides do we want to show those in the tracers yeah yeah yeah so uh the tracers would uh be agnostic if the block was created or not like I think it wouldn't matter I'm not sure if there will be any Trace if there were any transactions in the block uh not sure uh don't remember enough and in terms of override so the trace would be invoked on the already block with all the override so it's like the override also would be like transparent to it yeah well I'm also find that they would be displayed of the

traces H yeah yeah it would be it would be like it would be like what executed right it's like inspecting the actual execution and if the overrides override something yeah that would be in the traces because that was what being what was being executed I think the only thing I would change is I would along with tracing the transactions I'd also like to trace the Block in particular getting the state route between each transaction um for the nice that is possible uh that is possible um not sure if there would be any potential negatives for example from

performance reasons well it will be it would be even harder on performance M yeah you have to recalculate the hash the state route and receipts route each trans I think right now you guys only recalculate the end of the right yes so there's additional work that that would end up with right but it's not something that is not doable so e simulate V2 and debug simulate Trace V1 well I don't think we need V2 for this I think it's separate eat simulate V2 for the gas estimation stuff and then debug Trace

simulate V1 for the tracing yeah well there we have to decide what kind of tracing we want or do we need both kind of tracing like the get or party like I would say you know the traces that the client supports right so G doesn't support parity like so it definitely won't have parity like but if your client supports parity tracing it should be should support this just just like a parameter that you can pass saying I would like this type of Trace back and then if it's different name space again on the on the we can fix that like you would you would actually break it to some extent

so I would leave it as is wait what because the traces even in the first place they have their own parameters and they are different for for GE like and parity like traces so you can so the traces itself have parameters how uh how detailed Trace you want so yeah uh yeah so it's complicated so you have to have to those parameters are completely different yeah and doing um discriminated unions and no no no no no no no no no no no no no no let's not do that and then you will have in G the back trays and someone wants the parity like trays and G doesn't support it it does make sense it it's better to have

separate method or both okay so um does g only only Implement G like traces or they imp yes yes G doesn't Implement anything that's not g that's G is rest of the word agnostic right so we don't care what everybody else is working on we're do our thing exactly you guys want to copy it go for it so so do you mean we should have two of these tracing ones we should have one in the debug and one in the trace and then they behave differently uh I think they should not share they should both be in the same name space but we should have different methods so debug gu Trace simul no no no no leave them in the

separate name spaces because those are the name spaces for the other Trace methods so it would be consistent with at some point we should fix that and this feels like a perfect time to fix the no we shouldn't because those are two features one is geik tracing one is par tasing so they are in different modules it makes sense so I would be okay if if the Nam space was gu underscore or parody underscore but it is very non-obvious that the debug name space is gu likee and the trace name space is py like okay I can there there well is the all the methods in under debug and tracing as well yeah exactly and some some of them are shared between clients some of them are not okay I can agree with that statement but you don't think it's worth fixing

you we just kind of run with it for the rest of time to be honest I'm don't care could we then at the get name space and party name space or how would you fix it or just one method in the tracing and then some no I would I would leave it as this to be honest I have so many more important things on my mind I'm mean in the perfect world in the perfect world uh in the perfect world I think we would have only one tracing but well I do I do think so if I I agree with the statement that there are different ways to trace that are both all useful and the different ways to trace may have different parameterization and discriminated

unions are a pain in the butt in some languages and so doing discriminated unions as the parameters is a bad idea in the perfect world okay I got it I got it now sorry guys uh I would have parity trace and G Trace are separate modules because the in the debug module there are also other things than uh G Trace method so uh and leave everything as this just uh just rename Trace module to parity trace and have a g Trace module with only G tracing methods and that's it so it's very close to what we have I would just name it differently and split up the Deb module into gift trace and other debug methods and that's it we

also have a um nethermind and B namespace and rest do we have the namespace I don't think so basically would this is the idea that you'd have some name space where clients can Implement whatever they want their client specific and then other clients can then choose to follow them but your namespace is your own you have total control over it you don't have to coordinate with anybody for stuff in your name space yeah sense makes sense that would be interesting and then if nethermind wants to implement something in that gu name space they don't really get to say you just copy whatever gu did because that's their name space it's not something where you have to get consensus teams well there's a point where you want to

promote something to a general name space right that might have started yeah i' say at that point then you move it out of the the name the client name space and into like debug or something or is okay yeah yeah but generally like separation of trace and the bug for traces right is good it's just that the bug is a bit too uh too wide and it also has some other things that I don't like so that's that's the only thing I don't like right if we had and potentially naming that we have that Trace because Trace modules suggest that all the traces are there so it has to be like parity trace or whatever and G or you can whatever the naming right but that

that's the only thing I don't like about the current thing but the rest is in my opinion optimal I don't have a super strong opinion this isn't something that I would probably Lobby heavily for I I do I like your proposal where you have a GU name space and the party name space and another my name space if you guys ever want to have your own stuff um if someone wants to Champion that or if no one argues against it if we do add a trace simulate we could do it that way how always push this forward do we need specs for this eventually well so technically if it goes if we do with the the GU name

space and the par name space we don't need whatever they want I'm not talking about that I'm talking about that we just to get the tracing method any tracing any tracing it simulate no but in practice those are uh shared name space right and fairly well specified let's say so I would keep it as this for now don't do yeah I mean that they would add those methods I would I also agree with you we can keep the debug and race ones but then to add them to we make a specs for that uh yeah probably do you guys or do you guys just add it and then we say to Cena that we should add this as well or we which whatever they want to add of those yeah we can add it and derive derive specs from the from the

implementations does pz have any comments on this I don't know if best way supporting both of this I don't know if okay I don't know if there's a good spec for the traces in the first place so it's hard to uh it's hard to uh get spec for this because this would be the same just uh like wrapped in the th simulate uh block structure let say and that's it that would be the only difference I guess there's no specs for this this at all in the way they have copied so if you try to make a Specs then we probably quite difficult going to find how they yeah and traces are extremely complicated inside so trying

to spec out the so we can spec out the outside structure that we are adding but specking out inside of the traces I wouldn't even try to do that I guess this worked in a way that they get and parate their own traces and then everyone just copied those looking for the code yeah yes that's exactly how it went should we follow the same process or do something else this time maybe might them we can specify the outside uh object so the one so we have blocks right and block traces are a black box for us but we can specif how

we return those block traces if you follow me yeah okay okay that's so that's the only thing we could specify I wouldn't go into the details of the traces because that's yeah we won't finish this year are there any input fields that we also need to add or is it the um the input Fields would I would take the input Fields uh as same as we do on the aage tracing methods so there are there are something called G options and parity Trace options I think or something that uh that are um that are options yeah do you guys there is

one yeah do you guys want to make an implementation of either of those and then we could go from there yeah we can sounds good then then I guess we need to I would imagine that it's hard to get those both of those methods to match on all the other clients as well so but maybe we should not sure prior getting the it simulate to match and then see what we do with those other methods we did have some issues on some formatting Etc but we are trying to align with especially G uh outputs but also maybe it would be good to align on other outputs so yeah well the party doesn't exist anymore so

there's no one who controls what that looks like yeah yeah yeah it's kind of guessing work what it what it should be then I guess you don't have the El no not me okay talk to you next week or in the chat yeah bye y by you later