All Core Devs - Consensus #89

Transcript

[Music] [Music] [Music]

thank you [Music] [Music] [Music] [Applause]

[Music]

Danny Ryan: hello

welcome to consensus layer call formerly known as the eth2 call number 89. this is issue 549 on the pm repo here you go here you go we will talk about anything and everything merge related and then do our general updates and discussion points if anybody has anything um on the agenda i did i put robston question mark we did go over a bit of like the postmortem um all core devs one week ago

i wanted to if there's any further development or anything that um warranted discussion amongst this group

now's the time um the eatstaker guys as well as sam from the ef have been doing some sync tests and i can just post a quick rundown of uh what yorick sent about it sam is still confirming all of these um but we should have like a proper report by next all codefs

Danny Ryan: okay great thank you for that anything else on

robson okay great um main net shadow fork is it seven is happening next week is there anything that we need to know about that perry or other people involved

Parithosh Jayanthi: um yeah the conflicts are out and everything's in the same place it usually is uh there should be an announcement message on the topic in in the interrupt channel and we do a ttd override closer to the date the main idea is to try and see if we

can replicate any of the issues we found in drops hopefully none of them are replicatable by now

Danny Ryan: gotcha and are we going do we have the ability to perform some of those sync tests while the transition is happening or we're not doing that

Parithosh Jayanthi: yeah we're definitely going to do that as well we've done it i think every alternate or every third shadow

Danny Ryan: okay cool great um anything on the coming shadow fork any questions for perry or thoughts here excellent

um alex wanted to bring up sepolia there's probably two things here one is the genesis the other is ttd um sapolya will be a permissioned beacon chain which gives us kind of similar to clique properties and that we can decide who is in that set um and can actually kind of give them a weight as well which we don't get to do in clique um and i believe perry that these configs are ready to go and that everyone kind of has the information on genesis

Parithosh Jayanthi: exactly all the genesis while data should have received the mnemonic from me and the configs are in the eath client slash merge test that's repo

in case you wanted to be a validator a genesis validator and haven't received the mnemonic please do reach out there have been some people who reached out after the genesis date was created we just do token deposits with them and have them on board later

perfect thank you any questions about sepoliogenesis not yet gtd uh there's one other um oddity

Parithosh Jayanthi: that where we opted for in sepoya while creating the genesis state we set the balance to be about a million per validator and essentially

using this chance to inflate the sepoya heat supply such that whenever withdrawals our uh withdrawals are enabled we can just withdraw a bunch of validators and inflate separate supply um there's so there's like three different ways we can inflate the supply on sepoya now and we just use whichever manner um based on how critical it is there's a bigger write-up in the in the github issue

Danny Ryan: great and i think props to light client for realizing this is a great time to generate eath and not have to manage an irregular state transition on the

execution layer okay um now for ttd based off of all core devs last week based off of some of the conversations i and others have had i don't believe today is the right day to do set the ttd and instead probably one week from tomorrow after the uh after at that point sapolia beacon chain will have launched and will have run through the mainnet shadow fork seven to make sure that any of the robson issues are no longer standing um and

then at that point can make a call um i think so we don't need as much community lead time because this is more of a controlled test i think that it's very worth talking about is that plus one week or plus you know is it is it plus five days from that point at which we're comfortable doing a ttd or or a week and a half is there any technical or coordination reason for one or the other from this group so pretty much tomorrow's friday plus one week probably

decide ttd if um may not shatter four seven went well and then is something like four or five days lead time sufficient so that we'd have the fork in the middle of that following week or uh is it does it need to be more something like 10 days so that's the middle of the following week after that

Tim Beiko: so the robson ttd override we published five days or so before the fork um so it feels like maybe that's also sufficient and um this is maybe a weird question but um is it if somebody really wants to like

get ready for this is it possible for them to run a cl version which has the high sepolia tte um and then like a robston el version and simply do a ttd override on the robson or like whatever latest yale client version that they have i feel like that would work you might still hit some of the bugs on roxton like if obviously there were bugs in your clients on the el and you didn't upgrade but like

yeah

Danny Ryan: so in terms of releases not like that you think drops

in got you

Tim Beiko: yeah exactly it's like i think because the podia takes like less than an hour

to sink the eo um so yeah it might be

Danny Ryan: there's no there's been no breaking changes with respect to the spec so you know you could probably even use a client from four weeks ago um yeah so i think client releases today should work but you know there's rapid iteration on fixing

but then to

Tim Beiko: be care that's the el client releases the cl's you'll need to download a specific version which has the ttd which has the bellatrix forks activated

i

Danny Ryan: correct but that can probably also be handled in a config

Tim Beiko: right okay

Parithosh Jayanthi: i think some people who are setting up their nodes have just opted to use the

custom network stuff so you just pass the testnet dir and pass the genesis state at each other um and in their scenario they would just change the ttd override

flag and they're good to go

Tim Beiko: okay so i feel like you know if we do do this just five day delay um i don't know if someone has the bandwidth to write the quick hack md of like reusing your rotten clients for the sapolia fork um so that in practice it's like somebody could like set it up today um [Music] and and and you know effectively get like an extra week of lead time

Parithosh Jayanthi: yeah i think we have a ttd override

drill document that marios had made a long time ago um i guess just repurpose that that should be

okay

Danny Ryan: and our sepolia beacon chain releases are those out are those still like coming monday tuesday like those cut proper into clients

Parithosh Jayanthi: i'm not sure if all the time themes have been released but i know some of them don't know um tristan will read this in the next few hours so yes we'll read this today thank you tomorrow

i think

Danny Ryan: right so tim to your point even by tomorrow they might not even have to do anything except that ttd override and not doing any of the custom config stuff

Tim Beiko: yeah so we can probably like put together a hack md tomorrow and like share it either tomorrow or monday and that gives people who are really interested uh like an extra four or five days

Danny Ryan: yeah i think it's a good call um okay so it looks like no technical or coordination reason to do more to to do a longer delay than five days from awkward devs next week from this group i guess it's probably end up be if

there's any sort of technical reason to block that but i think that timeline makes sense okay um anything on poliogenesis supply ctd um before we move on to talking about muv builder network testing great um okay alex you have an issue up in the builder specs and want to talk about testing um whether it be on sepolia girly shadow forks or otherwise can you give us a quick on that

Alex Stokes: yeah definitely so yeah i was talking with some of the flashbots team and they're obviously looking to like test you know their software uh as we do the different merge test nets so i guess the first thing is yeah immediately i was wondering how people feel about doing this on sepolia i know there's like different amounts of you know different teams are in different places in terms of implementing the required parts of you know the specification so i'm wondering if anyone has any thoughts on that separately there's this issue because we'll want to actually agree on basically delaying through the transition and just having consensus around how long we do that

Danny Ryan: so you have a particular question um is the question if clients are ready to do so or right i can give an update on the prison side of things

so um we

Terence Tsao: so we have been working on it but the progress has been slightly slower just due to like the coming bugs with the current um local ee and stuff like that so yeah so our latest progress is that we are able to use the robson relay and we're able to simulate like an end to end like workflow with registration to to with the register with the registration to get payload and

and to the same block so yeah so um i think we're still like one to two weeks away from like putting this into the put putting this into a radius because we do want to test it as thoroughly as possible before like we just said to the public

Alex Stokes: yeah that makes sense so might be hard to ship for sepolia is what it sounds like

yeah that's

Terence Tsao: true for prison but i'm not sure if that holds for other clients

it

Ben Edgington: sounds like we're in a very similar place with uh taku uh enrico can

probably um give a

Enrico Del Fante: yeah we

uh we are we are not yet testing robstone but uh the for the implementation point of view is done and is already working for the kill network um so um so we are still in testing but the entire the flow has been has been tested uh already and it's supposed to work

are there relays deployed on kiln yeah so i believe flashbacks runs really gotcha so the builder

dapplion: being an external

dependency would it be even possible to test that with something like kurtosis so that all clients can benefit from that testing infrastructure there's um there's a like open source implementation of a builder and a relay that clients can run locally as well

so i mean it

Danny Ryan: sounds like in terms of polio we're not going to be running this on the bulk of machines but that maybe there's some experimental or not quite merged branches that maybe we can run on a very small number of nodes if there's interest

Alex Stokes: yeah that sounds like the way

persepolia

Stephane Gosselin: yeah i'd say any of the client teams that have something that they want to test i think there's huge benefit of trying it out during supolia and having a couple validators running it even if it's not necessarily deployed to or like merge on their master branch or anything like that um i i see like from the flashbacks perspective there's two things that we're worried about one of them is making sure validators get the chance to um and like particularly the validators are going to be running the mainnet merge i get a chance to set this up and run it at least once on like a live test net merge simulation and then the second which is i think what we can achieve for sympolia is just confidence for the

um the consensus client teams that that they're able to run it um so you know ideally would be sepolio consensus client teams running my boost and connecting to relay and sort of getting a chance to test their infrastructure there and then for gourley then we have um like actual third parties running this

okay so

Alex Stokes: that brings me to my next point uh which is the issue that danny dropped in the chat so we had talked about this some i think

over death connect with some of you different parties and basically the idea is that you know medford does add this like extra sort of uh operational surface to what we're doing and so in terms of keeping the complexity of the merge down like the actual you know running your software through the transition we decided to delay running mev boost like it would essentially be hard-coded into clients to not use that software sometime after the transition was finalized so i kind of have the very beginnings of pr in this issue where i just said you know we'll delay for 16 epochs we can decide what that number is i suppose i want to know if anyone thinks we should not do this and then if we do do this you know is

this a good number should it be longer i think 16 is like around an hour so it's a starting point

Dankrad Feist: who enforces this like um what if i what if someone manually disables that well they yeah that's fine yeah i mean we can't can't stop that right yeah yeah sure um i'm just wondering in order not to create this race to the bottom if it would be possible to just tell flashbots not to serve any blocks during that time

Danny Ryan: by the local software not um necessarily respecting it you introdu

you introduce the live testing of an alternative case where the relay looks down rather than it uh just going locally so i think it's good for the clients to also respect just kind of bypassing the flow even if a relay is not serving

Dankrad Feist: sure i don't disagree with that i'm just saying that uh like i'm worried that many people might be tempted to disable this um because they think oh like that's probably when there will be i mean you can literally hit the jackpot during that time right um so if you could someone do we have currently any other real layers other

than flashpots in there so whatever

Stephane Gosselin: yeah i mean answer a couple questions here and maybe provide some comments um i think it's definitely possible for flashbacks to just like turn off relay for um an hour after the merge but there are going to be implications right like whatever mev that is not going to be extracted through like by boost around the merge will be extracted through pga and i've already talked a few funds are inquiring about like how long this delay will be to like measure how much they want to prepare and invest into like things that can network through through pga um so there are

delaying preferred guess auctions

Dankrad Feist: i see yeah i i mean like basically but i mean my my thinking is that for one hour like for the vast majority of people or even staking pools or whatever it won't be worth like implementing their own uh special mav solution right so if we get some sort of a voluntary moratorium of relayers during that time to say we don't serve any blocks then i think that would be a fairly stable equilibrium for people to just not worry too much about that time and yes like maybe you get that

yeah i don't know

Stephane Gosselin: yeah i agree i don't think i don't expect any of the validators to try to run custom software but i do expect operators to prepare for the merge um through the transaction

Dankrad Feist: that's fine right because that is that is out of consensus layer

Paul Hauner: so that's in a way much less risky for us okay would be my feeling paul from lighthouse here i'm generally in support of um of just trying to disable this while we get through the merge i think we've got

a lot on our hands and just removing variables from it could pay dividends

Alex Stokes: okay i can turn uh i can make like a formal pr to the builder spec and yeah we can go from there

Danny Ryan: thank you um and on the sepolia people running this on a small number of validators if prepared is the the ball in the court of client teams to carve off a few validators and do a

sequestered node to do so or um is flashbots or somebody else want to help support this any any feeling one way or the other

Paul Hauner: so what was the question again denny

Danny Ryan: um testing during the sapolia transition um [Music] i i don't i don't know if anyone's gonna be running this in the vocal validators but there was some desire to run maybe experimental branches on a very small number of validators would you handle that yourself uh paul or were we looking for somebody else to handle that

Paul Hauner: yeah we'd probably just run it on the is the way allocated i would say okay

Danny Ryan: okay anything else on sepolia builder testing great okay um anything else on the merge in general okay for anyone listening uh the

execution layer the work chain is having a hard fork very soon to delay the bomb if you are a staker your eth1 endpoint needs to be updated to this uh so keep an eye out for releases coming i believe uh at the beginning of this coming week

Tim Beiko: and no they're uh yeah so they're coming out sooner everyone except basu already has a release and basis release will be out today um so three out of the four els have a release and we'll have a blog post live uh as soon as as the basic releases up they're having some ci issues but um yeah today or tomorrow at the latest okay

Danny Ryan: okay so upgrade your proof of work node if you are a staker or a non-sticker upgrade your proof-of-work node period okay great moving on any client updates would like to share cool

and yes i did say eth1 because it's literally called that in terms of cli parameters um great no other client updates research spec other technical points of discussion today

Mikhail Kalinin: i have a small update on deposits processing post merge yep yeah so there is a document uh the proposal that we have been recently looking into and working on with uh danny um i can briefly share my screen and go through it if i have the time for it now great

have a desire okay um sharing my screen can you see my screen

Danny Ryan: yes

Mikhail Kalinin: okay cool um deposit process and post merge so there is a lot of things there is a lot of um um a lot of space to improve on it after the merge because we don't need to have this bridge between the two blockchains anymore um and yeah just going briefly through the key things of this proposal um sorry

um on the execution layer side uh the idea is to surface deposit data into the execution layer block so basically the execution there client will go through the receipts that it got from the block execution filter them out by the um by the address by the contract address and then uh filter out deposits out of this key of receipts and uh add them as deposit operation to the uh block body um additionally there are a couple of uh validation rules

um one is to like basically the consistency between deposits and deposits root in the block header the other one is to verify that the actual deposit the actual list of deposits matches the one that was grabbed from the block execution and the address of deposit contract in this setup moves to the el side so it becomes the network integration parameter on the outside on the consensus layer basically we have the deposits now it's called deposit receipt to avoid confusion between deposits structure that we already have and this new one so

but we may call it deposit by the end of the day um there is a uh deposit queue in the beacon state uh yeah first of all there are deposits in the execution payload that we get from the execution layer there is the queue in the beacon state that cues these deposits from a block and attempts to process them in the block after so yeah here is the processbend and deposit method it's basically similar to what we have currently we don't have to verify the miracle proof anymore because we don't need at all to prove anything the validation happens on the el side

um yeah and yeah that's basically it yeah there is the process block we can still rate limits uh if we want to rate limit uh the computational complexity per block we still can rate limit the process in this queue by max deposits but i don't think it's necessary because the computational complexity is already reduced by removing the miracle proof verification so this like one of the things that can be removed um the the queue

Danny Ryan: even if you do process them all is still valuable so that you delay them by one slot so that execution and consensus layer can be run in parallel

for a given slot correct or is that

yeah yes

Mikhail Kalinin: yeah exactly and we could process the um them in the same block right but you we can process them in advance but uh like uh optimistically processed deposits uh the problem here is right it requires bls verification which is heavy and it may affect the uh proposal flow right so we get deposits we get the execution payload first from the um transaction there then we have deposit we have to process deposits before we compute in the state route uh the state root of

after applying a beacon block and here we have this bof bls verification which is in heavy operation that's like seems to be the only reason to have this you know um uh to have this delay between processing and induce adding them to the um to to the begin state the other uh yeah so that's basically the mechanics the machinery after after the merch after uh we have this of this new mechanism but we also have to uh hand over from the eth1 data deadpool to the new one and here we have this like a kind of transition period

uh the idea of this transition period is to stop doing with one data call when we have induced all when we have processed all the deposits from um from from the previous uh uh from the history uh before this new machinery started to work so there will be a time the period of time where we have both things happening but then once the eth one data poll starts to overlap the new blocks featured with the deposits on the execution player side so it will just stop here is the logic

that may work for this transition um and then we may like after this transition is done we may deprecate all the logic of one data poll and methods and supports so like there is a question of data complexity here so it's obviously uh deposits on el side i will increase the the execution block size and will agree increase the history and will and will increase the max amount of data to gossip as a part of the block gossip uh so if yeah if we take of uh of overall size of deposits that we got

until today it's like 75 megabytes so in terms of like historical date it's like really marginal increase uh but what we would like to look at is the uh how many how many data in addition we could have to hit on the network uh if we use this machinery instead of the previous one so it's like uh yeah i um i use like 30 million gas as a limit if we like have a full block of deposits it's going to be like 95 kilobytes of data in addition uh uh on on the l side so but uh we have this you know currently we have like uh

have to gossip the um the amount of deposit data we have to go to like is 90 kilobytes at max so the increases can be uh computed here uh easily but i don't think like it's a big issue uh because we have we can have like temporal increase and somebody may try to um abuse this and attack the network with like throwing a lot of deposits and making like a a block full of deposits just to delay a block but it will just be one block or probably two um one thing where's uh mentioned here is that we have one eath as a minimal

deposit in the mound which is big limits big restriction to this kind of attacks and also yeah worth mentioning here that since we don't have this miracle proof anymore in a block uh the over uh like the complex the data complexity per deposit is reduced by one kilobyte so this kind of things what thinking on deposits uh on like data complexity side alternatively we could have a deposit queue uh on inside of el state but it will need probably uh more work and

would add more implementation complexity we haven't done anything like that yet uh with the yell state so that's kind of it uh yeah but overall uh what advantages this proposal has is there is no deposit cash and it's one data and voting and other things that are really um implementation complex and uh require sophisticated implementation of deposit to a queue and managing this queue and the green on the block and so forth also uh which is nice deposits are processed like literally in the next block so it requires special transition logic

and the data complexity but i don't think the data complexity is like a big issue um um from uh from from the perspective of this uh proposal yeah it will require transition logic but that's where it's doing it and some work on the outside so that's kind of it yeah and uh we're not uh the other way to like improve on the deposit processing flow is to reuse the existing a1 data machinery um but if we reduce the um follow distance a lot like say to one block or to like 10

blocks whatever to one epoch then clients will have to manage real works and managing reworks in a deposit cache it's kind of like pretty comp sounds pretty complicated from implementation standpoint it could be a source of like new bugs and we're not considering

it

Danny Ryan: yeah i i was i kind of liked the idea of making ether one data just a validity condition that was and you just do it every block and you can just update things every block but then mikhail pointed out that then you'd have to handle we assume no reorgs in that deposit cash implementation that deposit cash implementation has been buggy in sorts of issues in the past and so having to

re-engineer that to assume reorgs would be probably not the preferred path

Mikhail Kalinin: yeah that's kind of it i'm happy to answer any questions

if

dapplion: anybody does have them to extend to what what that is was saying i think athlete love star and other implementations do the assumption that index number n would always and always

correspond to the same public key the validator index yes because so the the cache for keys and uncompressed keys is handled separately and is a global instance uh attached to all the states so not having that assumption yeah it would require definitely a huge engineering effort and definitely higher memory costs

Danny Ryan: does this change that assumption i'm sorry

dapplion: i think so why so if i understand if you press if your process is

the deposit on the next block it doesn't mean that that can get reorg and then another deposit could take that specific index in

in the processing order oh let's see what it means so yeah yeah i get it so there is the cash yeah that's that's a good point it's like pretty similar to deposits cash so um like there is a cache of whatever public keys um yeah right i see yeah that's a very good point

Danny Ryan: and either of these any method that does

quicker uh processing would have the ability to invalidate that cache because you could have read orgs on the execution layer within you know even a few block steps that could reorder deposits

dapplion: yeah and also there's another optimization regarding exactly this that we are going to implement that members have that they consider the so the section of the state of the validators that only includes the pub key that we draw credentials is strictly append only

lists

Danny Ryan: yeah good to know

um i suppose there could you could instead um have the dequeueing of the deposit queue based upon finality if that greatly changed the engineering requirements here in a good way something like that could be explored

Mikhail Kalinin: yeah but you have to look for public key not in other pub cases so that's the main point of this optimization i guess uh the main usage of this cache so you have to like i don't know have to update it and keep it up to

date

even personality

Danny Ryan: right right but if you don't if you only dq deposits that have been finalized in the execution layer then you're only adding appending keys that are not going to be reorg which i think is the point

oh yeah i see so it's like easy easier to um uh the size of well there is that are like of deposits the which process in is not yet finalized is much less than the size of all the yes all right thanks mikhail um does

Danny Ryan: anybody

have any questions or thoughts mikhail can you is did you share the link okay okay um if anyone has a chance to review it uh mikhail myself would be happy to chat about it in the discord and if there are further questions about the proposal as it evolves in the next couple weeks we can chat about it again in two weeks okay any other research spec or other

technical discussions for um today

have a question maybe somebody can recommend some some tooling to to simulate deep forks after the merge so preferably with different clients is there something like that available

Danny Ryan: i mean to set up the network that that forks and then you know observe it sorry these are proof of work forks right before the

merge or i missed it sorry

Saulius Grigaitis: i mean [Music] i'm looking for some tooling that would allow to set up a network a post post marriage network that that would allow to simulate deep forks after the match and observe the behavior of the clients i mean some orchestration for cooling so maybe somebody already done that and there is available setups

and

the easy way is to just you know

generate a network with a bunch of validators run two nodes and shut one of them down which has you know more than one third of the validators then the other guy will not finalize

yeah i'm i'm more more asking what maybe there's already some scripts or or some some configurations with some within some pooling right

Danny Ryan: so you know there's the ability to do to do these types of maybe partitions and then resolving partitions in hive but that's more of manually writing these tests and then

can kurtosis be configured to have a partition for some amount of time and then be resolved because that would that would also be allowed to have a deep

fork

Parithosh Jayanthi: i think you can't do it right now but it's on there on their agenda to add networking stuff to it right now you could start a kurtosis run and just do a docker stop so essentially you have half the network offline um but that's the best you can do

Danny Ryan: yeah so that's not but that doesn't end up with a fork that ends up with a large resolution yeah okay thanks

okay other technical discussions for today

Tim Beiko: we have a 4844 breakout room tomorrow if anyone here is interested um i'll post a link in the chat

thank you

Danny Ryan: okay any other discussion points on anything at all great

um maintenance shadow fork seven i believe next week and then um towards the end of the week we'll be discussing potential ttd number and have to update clients with the an override targeting the following week assuming things are going well thank you everyone talk to you soon thanks

everyone thanks everybody [Music] [Music]

[Music] [Music] [Music] [Applause]

[Music] [Music]

so [Music] you

Speaker names adapted from ethereum/pm, CC BY-SA 3.0.