All Core Devs - Consensus #81

Transcript

[Music] [Music] [Music] in the stream

people of the

Danny Ryan: youtube please let me know if you can hear us and this is consensus layer call 81 we have issue 475 and the pm repo shared there the first portion will be dedicated to kiln aka the current merge sprint um and then we'll go into any other updates and research discussions that we want to um on my end i do have carl shawway and vitalik so multiple people might come out of this audio

Vitalik Buterin: gm

Danny Ryan: great um

let's go ahead and get started kiln i want to um i think most of you probably all of you have seen the kiln v1 uh thank you mikhail for putting that together uh there are two issues that we've been discussing over the past couple weeks that were known to come into a kiln v2 you can think of these as kind of extensions and additive you might get some warnings if you don't use the exchange transition configuration but that would probably be just at the beginning before the merge happened and you'd still be able to merge and the authentication api written by martin has been written in a way that

authentication is on a new port so if you're not ready to speak authentication you can use the old port although we should uh move in that direction as soon as possible so these v2 extensions which will be added to a quote official v2 very soon you can yeah they won't they won't interrupt they just add a little bit that you need to work on those issues are linked in the those pr's are linked in the issue are there any questions comments or discussion on these points cool

um yeah not nothing really unknown there we've been talking about these for you know a month or more okay uh next up we're going to talk about some kiln milestones uh tim has put this together tim you want to take it over

Tim Beiko: yes if i can find a new button um i just shared this in uh on the agenda um i can share my screen as well let's make it here uh so these are basically pretty similar to the stuff we've had for konsuki and amphora still building one on the same format um high level uh just a tracker for uh

execution and consensus air teams um milestone zero is implementing a starting implementation also one is implementing kill v1 um and then we'll probably change this to kiln v2 sometime in the next week maybe we can add a separate milestone for v2 actually um yeah i think it's probably cleaner to just add the second milestone um then basically if you can try to uh interrupt what other clients some kind of an ad hoc basis that's m2 m3 uh like like uh then he said uh we'll be starting some definite builds us

i guess the one thing we want to make sure that we get right in these definite builds uh additionally to uh like everything else is uh running optimistic sync on cl clients uh that was one of the the things people commented on about the consumi milestones where we were kind of done but hadn't actually gotten optimistic fully nailed down yet um so we want to make sure we get that and then similarly at the last time after we've run a couple death nets that things are working as expected uh we'll we'll set up kind of a persistent version of kiln uh i still have kenzugi here so any need to fix that as well um we'll set up a a persistent version of kiln and then um once it's up and it's stable we'll

release it to the public and basically advertise kiln as the last kind of uh place to test things before we actually run through uh run through the merge on on existing test nets if everything goes as expected um and then this last part is just like previously saying we'll focus uh the first half the the core devs and that's their calls on uh on this stuff

Danny Ryan: sweet thanks tim uh any questions yeah i think it goes without saying um

in parallel to the kind of production engineering effort that is highlighted by this lots and lots of ongoing testing so like uh we'll likely do regular shadow forking uh to kind of more continuously test the builds uh there's a lot of work coming out in um hive and some other simulation frameworks and uh maybe some new consensus vectors along the way actually i just kind of mentioned testing let's do that right now um does anybody have any any testing updates or anything to share in that domain

Lightclient: i just merged the pr to merge block and

add support for kiln so if anybody is interested in using that that should be available on master if you have any problems feel free to message me about it

things like that

Parithosh Jayanthi: i shared a message on discord about kurtosis it's this tool you can use to spin up local multi-client test nets if anyone wants to try that out for example with your feature branch versus stable from other clients do let me know i'll share the documentation and happy to help out with that

Danny Ryan: kurtosis using kinsuki builds are they i is there going to be much work to swap it over

Parithosh Jayanthi: so you just specify it as a flag so you can just change that on the fly

Danny Ryan: great

Parithosh Jayanthi: and the at least the idea is to have that tested locally before you decide to join any devnets um that being said i probably start working on a cabinet tomorrow most likely with all the changes the devnet won't work but if it does then i'll let people know about it

Danny Ryan: great that can lead us into the next component um i know kiln's been up for a relatively short time but it's also not the deepest of change um for perry's sake and and for initial interop's sake where do where do people stand are there

any kiln v1 implantations on cl or el

Adrian Sutton: has a kiln engine api implementation we don't have authentication yet um we haven't done much testing on it but we could be able to match that up with an el that has it and actually check it works well

Terence Tsao: prism has the implementation we've just passed the vetter test uh i passed the link of architect result there

and we still need to do the authentication

Danny Ryan: cool

[Music]

dapplion: load star i'm going to merge now implementation and we are running in ci against gath and nethermine successfully no authentication though

your lighthouse is still progressing with the spec implementation

Dustin Brody: so if you're running against um nimbus likewise doesn't um have authentication uh there's a [Music]

branch which is passing tests thus far um but it's been essentially local tests or you know local test net type

type things yeah and for grandina we are still working on that

Danny Ryan: got it and um on the execution layer dapline you mentioned y'all are building against geth another mindset or geth another mind at least uh expect compliant i know there's a lot of productionizing to happen beyond spec compliance

dapplion: sorry could you repeat

Danny Ryan: pretty much are you're saying that geth

and nethermind are kiln compliant

dapplion: yeah we're running so this war has been done by jinder but we are running some commit of some branch

Danny Ryan: got it okay so we're in the we're in the realm of being able to do some initial

interrupt

dapplion: i think so

Danny Ryan: sounds good

Adrian Sutton: um one thing i should have noted is there is a flag for

uh take you to switch from consumer to kill mode will support both so it's in the testing channel tbenner posted it just recently

Danny Ryan: thanks okay um great well those are the primary items that we had in here um the additional stuff coming in v2 which are these extensions kill milestones uh thanks for giving the update on y'all's kiln status and any testing discussions um are there other

merge kiln or other uh related items for discussion today

Paul Hauner: i have a pr open to the beacon apis repo it's number 190 190 it's to add an execution optimistic flag to responses to indicate when we're returning information about optimistic data um it's pretty tricky to figure out what we want to do in terms of the api i've written if you go to that issue you see over the pr i've written some history about why we've chosen to do that i think it's getting pretty close to the point where we need to decide on how to standardize this api so i am tempted to go forward with 190 unless we're

going to get um some opposition from people it seems there's not a whole lot at this stage but um yeah i just kind of want to call last rounds on that one before we commit to it

Danny Ryan: thanks paul yeah put your eyes on this thing um i do i agree um waiting beyond monday doesn't make much sense i think people given their input

Dustin Brody: i mean i don't know this is better discussed in the issue or not but i'll just say i last i saw it was this the same one there was an open question of um kind of changing the kind of the idea of an optional field or

kind of a little bit of um chaos that might be caused by changing the presence or absence of a of a field in the return

Danny Ryan: yeah there's there's two conversations one is the the use of this additional pool and the other is how do you upgrade um if and how do you upgrade the the spec versioning are the the endpoint versioning paul what's the state of

the

Paul Hauner: uh i'm not sure that there's a clear decision on whether we need to bump the tags or not on the version things or not but either way i think we need to go ahead with this change the api

if if anyone feels strongly either way about adding that field they should jump into that issue and start but that pr and start commenting about it it seems there's a precedence for adding things to that um like adding values there without bumping the version so um it's tempting to move ahead with that yeah game to get get some opinions

yeah

Adrian Sutton: i very much like not to have to bump the version on everything i think we've shown we can add metadata in the past um because otherwise we just wind up breaking every fine um because we've got a change of a

number in the url um there's a a slightly more complex one where that field wound up you know in the body for one of the requests and we currently i think the pr bumps that to b2 personally i'd prefer to leave that as v1 because it's still just an additional field um i'm not strong on it though

whatever cool

Danny Ryan: um please take discussions to 190 and yeah i agree we should wrap this one up pretty soon other merge related items today

yeah

Mikhail Kalinin: i'd like to rise the question of execution payload the duplication here so here is the pr um yeah for the context um the execution payloads will be in current design they will be stored on both the l and cl side and there is an idea to just offload uh execution payloads from cl and request them from the l side at hawk this pr is just one of the approaches to do this and it's been an unattended for for a while um and uh

that current like default option is not include this into the scope of the merge and not implement it at the merge not implement this functionality unless there is the other opinions and somebody thinks that it will be critical for to have it at the merge also um in addition uh note that if we want to introduce this kind of change after the merge we don't need to do hard work for that so it may be implemented out of the hard work timeline which is good and this problem of duplicated data will start to annoy us

sometimes after some period after the merge um so that's two things to consider

uh by making this decision can i say yeah yep

Danny Ryan: the consensus layer um is expected not to prune at least to a depth of i believe four months so what this looks like in practice is four months of duplicated execution payloads or eth1 blocks you know essentially the size of what is on that network today four months of it you get additional storage on a local node um

that is did we run the numbers i can't remember where the numbers are it's probably not horrible but then if you start looking into things like uh any of like 4488 or any of these these items where you're adding to the um the core blocks size without doing any sort of sophisticated sharding um you know you get that that multiplier on what that four months looks like so it's probably something that's very nice to do in terms of timing i would put it not super critical on the merge but uh probably pretty important to get in miguel do you have an estimate on four

months of execution payloads without

yeah it's like 80 gigabytes or some kind with it

Danny Ryan: i see

Mikhail Kalinin: um if we take into account the average size that is like this average size of blocks that are on the mainnet currently

Terence Tsao: um are other clients implementing indexing using the payload hash whether it's to look up the payload itself or is to look for the bitcoin block that contains the payload because i saw there's some

discussion on discord but i wasn't sure whether there is um um some country uh conclusion that then came up with it

for

Paul Hauner: lighthouse we're trying to avoid creating an index for it um if it's pre-finalization it's kind of lost for us at the moment if it's after finalization uh isn't unfinalized then we can just search the proto array for it which is not um 01 but it's still fast enough generally

this

Dustin Brody: this is a potential point of concern um for nimbus because nimbus is switching to depending more on slot numbers than

hashes as paul was alluding to for

pre-finalization

Terence Tsao: specifically so but i guess my concern is more on the api side so consensus client will not serve this type of payload loophole right so this will be outsourced to the execution layer client

Danny Ryan: well the idea is that there's network protocols where you would be expected to serve it but if you had de-duplicated it you'd need to be able to dynamically get it from your execution layer to serve those those requests on the on that four-month bound at least and so there's probably two ways one is

to do this hash look up where i just gave you a bunch give the extra layer a bunch of hashes to retrieve them or you could do a more you know more like by range request um and there was a bit of debate probably six or eight weeks ago on that and there's the the hash lookup is currently what's in that pr but you could probably make appellate argument to be doing the buy range request as well but again that's an implication not really for the execution layer i guess you need to be able to get those payload hashes for requests but it's more of an implication on the execution layer i'd say

Terence Tsao: got it

Danny Ryan: okay um i'm not i'm not like super eager to like try to release this and kiln um i think that this is something we should have discussion with the execution layer on i know mikhail you have done some initial discussion with geth especially on those hash lookups but it might be worth opening that up once more before we push this pr one way or the other

Mikhail Kalinin: right and i would say that if like cl client and rumors think that it's critical just um comment on this um write this question in chat and discord so we can proceed with that um otherwise as i've said the default

option is just a little bit there and as it is now

Tim Beiko: yeah i'm not sure how many uh yell client dance or i guess if all yale commanders will watch this so my viewers just having an explicit opinion on discord

right i

Mikhail Kalinin: i think this is critical for cl uh more than for yale like having this feature so and yeah definitely at first engage yellow client and players into discussion when we will be finding the right way of doing of implementing and designing this functionality

yeah fair i

Danny Ryan: suppose the the question of criticalness is also is a software implication for the cl but it's also just what are the full node requirements because if you don't add this you know it is plus 60 to 80 or so um gigabytes of requirement which kind of impacts the unification of cl in some sense okay um please chime in there uh and we will continue the conversation um anything else in this one before we move on

from my side

Danny Ryan: okay

other merge and kiln related items for today

Paul Hauner: i did just want to raise um that something we're looking at now is trying to collect the fee recipient from all existing phase zero validators before the merge so um you're kind of putting interactive props and warnings and stuff around the vc and trying to keep keep track of people who haven't provided their fee recipient i just want to kind of raise it because it's going to be something that faces all clients um it's probably going to require updating to all of the become a validator guides it's going to be a bit of a weird thing where we have to collect one piece of information extra information from all the validators so i just wanted to kind

of mention that to bring it to implanters minds and also it's something that um people who are doing kind of education uh and documentation are gonna have to deal with as well

Terence Tsao: this is a great question and do you know if there's any like documentations around that already just i've i thought tag who has implemented this already so i wonder if we should have some sort of standard standard alignment across this

Paul Hauner: yeah i don't know of any documentation about it i wasn't aware that um that can tech would already done it would be keen to yeah i agree be nice if we could make it uniform

Danny Ryan: so there's two things here one is you know where are you providing this information you know is it per valid or on the vc is there a backup on the bn like that kind of stuff and then the other is because we're going to need to gather the information uh you know how does the client go about getting that from the user um and both of those need to be answered in a sane way um i believe the answers to the former or there's a bit there's there's beginning to be conformance on um maybe bn fallback but the vc by default drives on a per validator basis is that the case

Paul Hauner: yeah we're we've got so with the we're kind of like the graffiti in a lot of

ways it's a similar deal where it's like the value of the client you know you can supply one that applies to all of them you can go and like edit a file where you can manually specify it for each validator and there's also an option to read it from a flag so you can from a file so you can change it programmatically and if none of that works in the vc then the bn can have a backup one to deal with it um i'm not sure we necessarily need to standardize all of that just because it leaves room for innovation um but yeah the the other one about figuring out how and when to collect it and um yeah is is something we could probably standardize on i think what we might influence for the time being is you can just edit we have this validated

definitions file so you can just edit this yaml file and add a key for all your validators if if you want um and alternatively the really super easy way is that you can have a prompt when you start like when you start your vc you get a prompt if you don't have a fee recipient for any validator and you need to fill it in to continue i will have a flag to bypass that for people who are dealing with automated setups it's been raised once before

Danny Ryan: got it and i was gonna just say i was muted and talking to myself for a second um the trent we should add that to kind of our our outreach list i think in general we should do a round to make sure that we get in touch with all the ecosystem participants that maintain guides and there's probably a handful of things we want to nudge them on and this is this is one of them

sounds good

Danny Ryan: anything else on that okay other items for today on the merge

Tim Beiko: oh sorry one quick shout out there is a

merged community call tomorrow uh basically the same time as this call uh so we're gonna get uh application developers infrastructure providers and whatnot uh to come and ask your questions uh it's always helpful if the head full of client devs show up uh when there's more technical questions or if you just to show your clients to people to the ecosystem um so there's a there's a issue on the pm repo but it basically starts at the same time as this call tomorrow

Danny Ryan: cool uh y'all should get a couple of clients to raise their hand saying that they'll deploy their client end-to-end on kiln you know full interfaces and everything

Tim Beiko: you mean applications right

Danny Ryan: yeah sorry application developers

Tim Beiko: yes that's it um and if people i if people here have ideas of applications that are tracing heavy i think that's one thing that we haven't been able to like uh that we i i don't have like a great candidate off the top of my mind so um if people have have ideas of something that's like tracing heavy that we could deploy on kiln to make sure all of that works as expected that would be really valuable

Danny Ryan: who are the big tracers i know exchanges always say so i would assume maybe the

um yeah

Tim Beiko: block explorers i've asked some a lot of them are pretty busy right now so they're not

they haven't said they won't do it but they haven't committed abby exchanges and black explorers are the two that i i had off the top of my head

Danny Ryan: thanks tim other merge items cool thank you to some of our execution later dubs that joined um sweet

moving on are there any other client related updates that people would like to share today okay excellent easy um research spec r d items i know there's a lot of uh stuff floating around on some dank sharding data availability sampling and various things any updates or comments here sweet

um i guess i'll say in preparation for what is on the execution layer called shanghai um we should be honing in on on a feature set for that associated fork um execution layer execution layer eip for withdrawals is in progress to complement the work in progress prs on the consensus layer to put them together for withdrawals for executive validators and partial withdrawals for proposers um assuming that that goes to shanghai uh we'll get those specs iron out on the consensus layer uh i think the other additional item

we've been talking about is yasik's cleanup of historic roots and i'm not sure if there's anything else other than maybe kind of these four choice updates like that being a very concrete place in which if they had not been upgraded they become upgraded are there other things that i'm missing for the discussion for that future fork

Tim Beiko: the one thing i think this is unlikely but uh wasn't so vitalik had these two proposals for blob data transactions i think right the more complex one might have

some work on the beacon chain

and right yeah i don't think it i don't think it's like anyways

Tim Beiko: i think it's more likely we do the simpler one but if we were to do the more complex so

Vitalik Buterin: the discussion on this on this side has been increasingly going in the direction of the more complex one the argument is that even though the total implementation complexity of the complex one is uh higher the simple one puts all of the effort on the execution clients and the total and the more complex ones a bit less effort on execution clients and more on beacon clients and execution clients are already expected to be very overworked for shanghai

Tim Beiko: oh i like that actually

[Laughter] yeah i mean and

Danny Ryan: and that more complex

one does get to something that looks much more like a final shorting proposal and the complexity on the consensus side ends up being primarily the updating of data structures and the management of data um not a lot of what i call consensus complexity

Vitalik Buterin: right it's like the the comp the consensus complexity is like not very high in the consensus side not very high in the execution side by far the highest complexity of this of that eip is on the block creation side um actually enough um because if of how it introduces these transactions where the transaction has a piece that goes on one side and a piece that goes on the other side but like in

some ways that's the in if we're willing to make some some sacrifices that kind of complexity is fairly manageable like basically we look we only really need one implementation that can make uh these uh blocks right and if we have one good one then it would like as long as everyone can do the verification and still it would still be fine

Danny Ryan: so just a heads up denver comes up next week and there's a crew that is attempting to prototype um a version of this end to end so hopefully have some more insight through that exploration yeah that's definitely uh

the other item that's being explored and potentially in what might be called shanghai or a subsequent upgrade um we had discussed in another channel to do a um dank sharding educational workshop for the consensus layer and i guess anyone that wants to join um we tentatively discussed early this coming week i'm going to circle back with donkara he's on this call actually um and some

others to see if if we want to target a date for this coming week i think it'd be valuable to get in before the east denver sprint um but we'll let you know very soon cool other things other items things to discuss okay

the silence implies we were reaching the end of the meeting are there any other items to discuss any announcements anything today i will say uh the dev connect dates are public that's the 18th to 25th there's going to be lots of fun stuff going on that initial calendar is out we do plan on doing some uh l1 r d workshop stuff right at the beginning so if you're going to make some portion of the week and not the entire one i'd say show up right at the beginning more details for that soon

cool let's close it out thank you uh easy meeting and see y'all on all these uh fun killing milestones talk soon

bye guys soon thank you thank you [Music] [Music] [Music]

[Music] [Music] you

Speaker names adapted from ethereum/pm, CC BY-SA 3.0.