All Core Devs - Consensus #85

Transcript

[Music] and we are good

Mikhail Kalinin: it's so cool let us get started um yeah it's an honor for me to run the call okay so welcome to the consensus layer call number 85 um yeah thanks for the agenda team um so

um let's start with the first item which is the kiln office hours and uh i will start from uh testing updates um destin and shadow forks uh perry they wanted to uh give an update on that

Parithosh Jayanthi: sure um so the last shadow fork we had was on monday that's girly shadow fork three uh oh actually since the last week we had girly shadow fork 2 and shadow fog 3. shadow fork 2 just had an equal climb split and we didn't notice any major issues i think nethervine was able to figure out a few issues and bezou as well and maybe one or two other clients but in

general things went okay um girly shadow fork three was on monday and we replicated a minute client split since then i think the get team has been debugging a specific issue but uh it's just affecting a subset of nodes the network is still finalizing and people can um do people can try also all sorts of sync tests against early shadow for three and just as a general announcement uh shadow fork one and two will be deprecated later today so please migrate to shadow fork three as soon as possible

what about the main edge shadow fork

Parithosh Jayanthi: yeah so i um sent the configs for mainnet shadowfuck yesterday on chat the shadowfuck is planned for monday i'm currently syncing the nodes and the corresponding beacon chain will be launched tomorrow conflicts for everything is on already on github there are boot nodes and genesis configs etc

Mikhail Kalinin: cool and is anybody is welcome to to take part in this shadow fork

yep

Parithosh Jayanthi: feel free to join um because it's limited to genesis validators i'm running all the validators but the main purpose of these tests are to um

well test sync so anyone can sync up nodes and join in

Mikhail Kalinin: great and uh yeah just to remind uh that this is the main net so the um disk space requirements are much higher than on girly right

Parithosh Jayanthi: yeah definitely you have to sync up a complete mainnet node um so i do suggest if anyone wants to join you start now so that you have a couple days to sync

Terence Tsao: do you know how big of a disk space is required for a

manual these days i think about five six hundred gigs is enough i've provisioned one terabyte

machines just to be safe

Terence Tsao: got it

thanks

Micah Zoltu: i mean that's that depends on what node and what settings you're running like nether mind fully pruned i believe is under 100 gigabytes maybe i mean it's kind of rude for the network to run that way but if you just need to run something and you don't have much space that can work

a great

Mikhail Kalinin: uh very excited to see uh the first mainnet shadow fork um is there any comments on the girlish shadow forks that clawland developers want to make or

anybody else

Marius van der Wijden: yeah so we are still investigating an issue uh on gas that happened on the last shadow fork and yeah but it only happens on a small subset of notes so if you see a bad block happening then you might be that that might be because of the issue

Mikhail Kalinin: um

yeah is it like the issues about the bad block production or um any other relations

too no it's um a valid block will be seen as bad and okay yeah probably tricky too um like to debug okay got it um yeah any other testing updates

Parithosh Jayanthi: i just had one more tiny note about the mainnet shadow fork um just be wary by

using the mainnet chain id so if anyone's trying weird transactions etc um they might gossip to the actual mainnet and you would be wasting mainnet ether so please be careful by default there will be no transaction fuzzer running on this um i don't think anyone's reimbursing me for that

Mikhail Kalinin: yeah this is a really good name thanks barry so be aware of that the basically shadow fork is the is the main shares like a state with the main ads so the transaction around there may also be um included in a block on the mainnet

so and made according changes to what it's what this runtime of this transaction will be okay cool um i have a small test and update um i have been working on this test coverage documents and yeah just drop the link into the chat there is a transition section that i have been mostly focused on recent days and yeah this is like a kind of the shape that this document will look like this is just an example this transition sections have like raw checks parsed from the engine api spec and like more

more appropriate testing scenarios that we might want to implement in by our testing tools so with respect to transition checklist the work will still need to be done on other specs so go into parts uh other stacks and also include the information from them uh into these scenarios yeah but you might look also thanks a lot mario for the input on that i've made some check boxes marked already um and with the corresponding links to hive tests that covers them yeah and for just for the

background who haven't noticed this um it's been announced on the previous acd call i mean this effort this is literally go uh going through all the stacks and just parsing all the statements and putting all the checks that are need to be done to test the merge software um yeah and then make it in a good shape and then go and cover these check boxes with what already implemented while testing tooling and yeah work on what

has have been missed

Terence Tsao: i guess i can also um give a quick

update i am also working on certain set of uh manual test cases in our um github tracking issues so these are like the issues i thought of they are like harder to trigger in a way it's not unit testable it's not really end-to-end testable we don't have that today so yeah i'm looking for feedbacks to see if i'm missing anything here is anything like i should be adding and yeah please take a look and give us feedback thank you

Mikhail Kalinin: yeah great thanks terence um do you want to share a site or whatever else you have on testing

Terence Tsao: yeah so besides that we have an

end-to-end test case that we are actually implementing a proxy that's sitting behind the city in the middle between the b beacon node and the ee and the proxy is able to manipulate the payload um as it gets passed through so it's kind of similar to the test case that you mentioned but this allows us to easily um get the pending status back or sorry seeking status back and forth so that we can test optimism more smoothly for in our influence so we have that i'm happy to share the progress on that a little bit later once we finish it but yeah really yeah i am really excited for all this testing

stuff

Mikhail Kalinin: cool okay and any other testing related items um okay so i guess we can start with the client updates who wants to be first

i can

Paul Hauner: i can go from the lighthouse break the silence i kind of got out of the habit of preparing these updates because danny kind of stopped but um we're

working on the merge obviously um we've got full-time person working on testing now and we're kind of dealing with mostly kind of little tidying up issues and just making sure that everything is locked down we can't produce attestations when things are optimistic and also trying to make sure that we um that our logs kind of make sense um after the merge because we're used to kind of complaining when the execution node f1 node is not synced um but we need to do that less now because it's kind of our job to sync it so yeah just um i guess probably in the tidying stage um and also testing we've also cut a release a couple of days ago that

enables proposer boost on the on mainnet so our lighthouse users are starting to do propose a boost on mainnet in production i think the other clients are well aware of this and looking at also running it as well that's about it for me

Mikhail Kalinin: and this uh if one client simkin or yeah this complaining thing is just logs right it's not like um anything is like deadlocked or whatever i mean this thing yeah process

Paul Hauner: yeah it's just logs it's one of those things where it's just logs but it turns out to be very difficult to solve we have to do kind of a bit of a refactor uh but yeah it's just logs

okay see yeah thanks for that great and uh about the professor boost i mean that

works now on the um on the main net yeah it seems it

Paul Hauner: seems to be going there um yeah i guess we'll see probably as more clients um start to use it we'll get we might start to see it taking effect on mainnet be interesting

definitely great um okay next i have uh taku um yep hi so

Enrico Del Fante: um we also have a new hire which is uh stefan bartonov which is you can see now it's connected to the call welcome

um we are working on uh mev boost integration during this last week weeks and um in terms of merger we may be worth mentioning that we did um we implemented another uh check in the terminal block difficulty detection uh terminal block detection actually because we were not taking in account the the block timestamp coming from from from the proof of work uh

and in some cases um running some testnets we we we got a terminal block uh but with the future timestamp and uh also due to uh a similar bug in in bezel we in our acceptance tests integrated with our our our ci we were hitting a bug where we are actually considering that that terminal block uh to be to be good and we were then building a next block in improve of stake with with the timestamp which was before

the timestamp of the terminal block so we actually added a check we we act this we discard the potential the mailbox that have time stamp in the future um there are other things yeah

let's see and

Mikhail Kalinin: what is the decision on the blocks that are having time stamp in the future

Enrico Del Fante: we simply don't don't care about that and wait maybe another block that has a timestamp which is in the past in with regard with

our consensus client time

let's see yeah that's interesting case yeah maybe yeah

Enrico Del Fante: worth to mention maybe in your document as a transition test even if it's kind of weird but we hit that comes in that case

yeah

Mikhail Kalinin: i see okay let's let's have a more look into this um thanks a lot enrique welcome stefan okay next one is

lotstar

dapplion: hey everyone devlin here so we continue our merch work all going well no substantial issues

we have also also merged substantial updates to our gossip and ssc libraries and hope to have them in a release soon and we have also keep iterating on like lion proof of concepts and we got a technical demo where we closed the loop on the kiln test net going from consensus inaudible all the way to showing execution data and we showed that in each dubai so fun stuff that's it thank you

a

Mikhail Kalinin: great congratulations on that treatment um thank you next is prismatic

Terence Tsao: oh hey guys so we also have a release

coming next week and that should never propose a boost really excited to see that happening on the main net and then um regarding the merge we have a dissolve our kiln branch so everything lives in our master now so that's very nice and then we are mainly tightening up and adding more test cases we are fixing the last few things of the music syncing such as uh pruning uh invalid nodes deleting them from the db and um also just making sure when the ee goes offline or when e times out uh our beacon node handles uh gracefully

and um we also are working on web designer and that's that that's a pretty big one it's taking out some of our time and um yeah that's it thank you

great thanks terence um okay next one is nimbus

Zahary Karadjov: uh we are also in the process of integrating our merge code to the mainline branches we're looking into integrating evp boost in nimbus we are also doing uh light client experiments uh nimbus currently implements a custom peer-to-peer protocol for obtaining like grant updates and we are testing flight client syncing

in the pratter test net our server is actually compatible with load star so you can also try this setup if you wish we are also working on support for threshold signatures in the remote signer setup such that you can configure the image to operate with multiple remote signers where each of the signer is configured to operate with a partial key and something quite interesting starting from this week the nimbus execution layer is successfully being able to participate in the

cube intestines

Mikhail Kalinin: oh

congratulations on that um thanks harry and grandin

[Music] he

Saulius Grigaitis: worked mainly on small fixes for the merge and they also worked on the new feature which we didn't have for a long time

this remote the three signer so that's the main things thanks so much

Mikhail Kalinin: um great okay so uh we also have el client developers on the

call and i want to give them a word to spread the updates

so

Marius van der Wijden: i can start um we've been debugging the issue that i already mentioned and uh not much more than that that has been very

handsome

Mikhail Kalinin: okay any anything that can help you with the debugging and probably enabling some yeah some tracing or whatever

not not really no we we're still trying to figure out how this can actually happen and uh

we don't know how so yeah but uh we're we're still looking into it working on it yes it's blocked with that

Mikhail Kalinin: yeah the worst yeah the worst is the worst category of boxes this kind of box which are not deterministic um right so yeah luck with that okay um any other client updates probably i have missed some anyone okay

so since we're dealt with client updates and uh yeah um as we're still on the merge um uh topic um tim could you please um give us uh like a quick brief walk through the decision making process uh we have touched it on the acd but i think it also makes sense to share it on this call as well

the decision process yeah i'm in the um for uh where we are at the fork and test nets and so forth okay right right yeah yeah so

Tim Beiko: um roughly speaking uh like there's a difficulty bomb on magnet

uh which is set to start being felt sometime in may um and at some point we want to decide whether or not we think the merge can happen before the difficulty bomb becomes to uh to pronounce on my net and just to give some background in case um in case it's not clear to everybody on the call how the difficulty bomb works exactly but it it it's like an exponential increase which kicks in every 100 000 blocks basically the amount of difficulty on the proof of work network uh gets gets like there's a fake difficulty that's added by the difficulty bomb making it harder to mine

so basically every hundred thousand blocks or two weeks like that amount gets an extra chunk added to it and and that grows exponentially um it's very hard to predict exactly when it kicks in or sorry once it's kicked in like once we go to a spot where the difficulty bomb adds like goes from negligible non-negligible addition to proof of work it's quite hard to predict how long it takes um just because you then need to make estimates about like how much the hash rate grows or shrink in that period and in the case of the merge it might be the the most complicated one because um we we might expect that like the hash rate would would drop as as we get closer to the merge that said we can do like some rough pop-up calculations and

i the last i checked which was about a week ago um it's likely we hit roughly 15 15 and a half second block times sometime in in july and that by the end of july we would be up to like about 17 second block times um if if we did nothing and personally and i guess having talking with a bunch of of different people it seems like 15 second is like the maximum that's like that we can tolerate before doing something if we go to 17 then the one after is 20 and then i think it's 25 so it starts going pretty quickly and and that gets no less a lot on the network um

so uh if if we're hitting like you know 17 by the end of july uh the one little buffer that we have is we might be able to bundle the execution client releases with a small push back to the difficulty bomb so if we needed like a couple extra weeks for the merge you could you could have like a mini fork that happens on my net which just pushes back the difficulty bomb before we hit ttd so if we want to be like in that that world where we either don't push back the bomb or uh maybe push it back only slightly but in the same release as the merge and don't need like a whole separate network upgrade to do that we need to be in a spot where um by the the last alcor devs in april which is like in three weeks from now

we're deciding about forking testnets like we don't need to like have the exact the exact like test net blocks and and everything on on that call three weeks from now but we would want to be in a spot where like we're comfortable that in the next two weeks we're gonna sort that out and that uh you know around mid-may we start we start forking test nets this assumes probably something like we fork test nets every two weeks and so the first one gets like six to eight weeks of of like being forked and then the last one gets more on the order of two to four um and there was some kind of contentious on awkward ev's about if if that's enough so that's something we also need to check in parallel um but

yeah long story short i think like late april is is when we need to make a call about do we think we're ready to now move to test nets and if not i think it's it's probably the better approach at that point to just have a small upgrade on the execution layer side which kicks back the bomb a few months and we can can debate how much we want to kick it back for

cool

Mikhail Kalinin: thanks tim and um do i understand correctly that uh if we want to work test nets in may then we probably want to have production ready clients in the beginning of may

Tim Beiko: right yeah so that's why like basically

the the call three weeks from now what you would hope clients are like ready for for test nets and that what we still need to figure out is basically you know pulling out the release that supports the test net and maybe some small you know say there's like a small tweak in json rpc or something like that that's not the end of the world but like we shouldn't be changing consensus related code at that point yeah

Mikhail Kalinin: see any comments on that okay the silence

yeah just curious what does it mean okay so um yeah and uh we are getting back to this in three weeks right i mean to this to the decision on on what we do next and when

Tim Beiko: well i mean hopefully yeah like with the with the main edge i think for me personally the mainnet shadow fork will be a really strong in the in indicator like if we have the mainnet shadow fork and things uh go roughly the same as they went on gordy that's good if they uncover a

whole host of new issues then i think it's it's unlikely that we're in a in a good spot to uh to be ready but yeah so i don't think i guess what i'm saying is like i don't want to just say we're gonna like table this for the next three weeks like i think we're gonna have a lot of extra information that comes in the next three weeks and and like on the call next week and then in dev connect two weeks from now and what not like um we can we can evaluate how how we feel about

dates um correctly yeah mike i understand correctly you're

Micah Zoltu: basically saying that in three weeks all clients should be prepared to answer the question

are you uh feature complete

Tim Beiko: yeah yeah and i think if we're not in that space like and you know if we're like 95 feature complete then there's like maybe some like wiggle room that like do you want to do the first test net at 95 percent feature complete something if you're like at 85 80 feature complete i think in three weeks and that's the case across like several clients then i think it's it's it makes more sense to delay the bomb and obviously it's not it's not just my decision but like i think that's assuming we want like a sane test net deployment schedule and we don't want to rush them for the merge um yeah

yep

Mikhail Kalinin: makes sense um

on the mainnet shadow cork and so we are expecting to have one for uh uh pretty soon right and uh we are expecting to have a another one like on demand or uh like a week on a weekly basis or whatever probably perry can answer that barry what do you think on the shadow

of working main ads well we have uh another one next week

Parithosh Jayanthi: yeah the plan is to have one every second week so we would have one on

monday and the one after that would be i think the friday on dev connect like the dev connect week friday so we'd have at least two minutes shadow forks and we can still talk about another girly shadow fork if we wanted i think doing three minute shadow falls like at some point it's a like a point of diminishing returns um like doing it two times a week might not bring

as much yeah see yeah especially when the internet notes take so long to sing um it's just yeah right okay

Mikhail Kalinin: okay um anything else on the um like merge release timeline and related

comments what have been just discussed and shared okay let's move on

there is a pr by danny on the timeouts on the engine api calls so it's basically set the timeout uh uh on the calls with like a big uh um with a pretty big enough amount of seconds uh for the calls that have an execution in it execution semantics which is the

new payroll the book just updated the other two calls has much less timeout and the requirement is that the consensus layer must break must just expire the call by 10 out yeah so that's that's it and there is a um like reasoning about it in the comment to the pr so please take a look and we're expecting inputs on that do you want to discuss this engine api time mouse right now okay so let's just uh down and move to

the um one thing yeah i yeah just

Tim Beiko: one thing on the timeout is um does anyone feel like 120 seconds is too s too quick um and maybe this is more a question like for the el side um but like the idea of like a 10x difference in execution between like the high-end hardware and the low-end hardware is kind of what the 120-second is predicated on and so i think that's the bit of feedback and i that that would be helpful is like is there a case for like actually this shouldn't be 10x but it should be maybe 20x or maybe it should be you know 15x or something like

um that's and it's it's something that's kind of easy to change as well but yeah i think if people have that type of feedback it would be really good to to get it now

Mikhail Kalinin: yeah sure thanks tim um yeah also to add from my end um on the timeout stuff um yeah as then stated in the um comment to the pr uh there is a point when uh waiting for uh this block waiting for a block to be executed by executioner client is not valuable after some point in time so and probably it makes sense to have like

timeouts as huge as the number of seconds in the e-book because in this way the chain will be able to still make progress uh the chain i mean in terms of consensus of course block supply will be significantly lower if there will be only one block per ebook but it's still theoretically possible for the chain to make progress so we can take a look on the timeouts from this standpoint as well also another thought that i had on them on the timeouts is that there are two um yeah there are two uh operation modes that the l client has it's basically a sim kin and block step in

uh when like a node is staying in sync with the network and just receiving the block via gossip and sends it to the payload to the el and probably in these two different cases timeouts should the match like in different ways so that's things to consider um on this topic so please take a look it's very important because uh recently these timeouts appear like many times in this court and in conversations um so it's kind of going to be an issue if you're not being able to specify them correctly

okay

the next thing to discuss and to remind the people about is the um latest valid hash stuff so um we have touched this on acb and uh yeah this uh this thing is is going to be like resolved sooner than later because yeah we need to write tests we need to be sure that we are on the same page with respect to this um requirement in the spec so just yeah just the quick context there is the latest valid hash that the el client have to respond with uh when it has been like sinking for example and found that some payload in the chain

that it has been catching up with is invalid so it should respond with uh the latest valid ancestor hash um on the next uh cl call so uh el will have to store some information about this uh chain uh until it receives then a subsequent call from cl uh to to be able to share this information this is important for consensus fair clients so it can invalidate not only the most recent payload on this chain but also like all the invalid payloads which is uh yeah which starts after the latest valid hash that is provided

um also uh yeah the problem here is that the execution layer clients doesn't store this kind of information today and it needs to be added and kept in memory and returned back to cl um so yeah what i have just shared is the just my thoughts my initial thoughts on how it could be implemented and the main question is whether we want to support this um or we would like to explore uh like um yeah we would like to to reason about to reason about whether it's important

for um yell client to respond with this information uh while it's been sinking or it could be kind of you know optional stuff yeah because it's it's going to be there the semantics will stay there and the request format will not be changed because uh there are clients like aragon which may execute payloads uh multiple payloads and functions updated and they need this um semantics to respond accordingly if this uh if the if the rework is happening and multiple payloads are being executed and one of these payloads is invalid so this client may uh immediately respond with

the latest spell attach correctly without storing any information and this is valuable and important for cl clients in this case yeah but for the sinking case it's like there is a question on that um topic so please uh reach reach us out and discord reach me out and discord comment on these documents um so we are about to to make a decision like early next week on that any questions or comments with in that regard okay let's move on

so um uh we have uh yeah as long as we are on the merge topic still and the mav boost has been mentioned here uh by a few several developers uh like client we want to uh make an update on on your recent work on the spec

Lightclient: yeah thank you so for context for people who have not been following much with the builder api or mev boost this is the interface for cls to get blocks from external builders and it was changed from a middleware design sometime in the past few months where

mev boost would sit between the cl and the el and route for choice updated and get payload calls to the builder network to a separate service that can provide blocks to the co and as far as i know this was originally proposed by paul and so a lot of the latest pieces of the spec have been derived from his ideas the new approach does require some more integration on behalf of the cl teams so i'm curious to know like what how far people are on integrating those things but the things that appear to need to be implemented with the newer spec is is that you know of course you need to implement the builder api which right now has three methods

the builder get header builder get payload and builder set fee recipients and then there's two types of signing that the seals will need to support one blind block signing so they'll need to be able to sign over blocks that don't expose the full list of transactions it's not i'm not clear on exactly how that is communicated to validator clients if the entire data object is sent over or just the signing route is sent over but that's something that new would need to be supported and then they'll need to be able to support a fee recipient announcement and we tried to make it so that fee recipient is not something on the critical path for building blocks for a few reasons

one of the most important is trying to move to a world where this is more of a gossip based approach rather than a request response approach over rpc so the validator client needs to be able to support signing a fee recipient message just a testing that this is the fee recipient that they wish to accept funds at and that would be on a different domain than the normal block production and attestation um signatures and then of course there needs to be some modifications in cl block production codes to try and get a block from the external builder if the builder

functionality the external builder functionality is enabled so the latest changes to the api are in a pr to meta boost and i'll post the link here i would really appreciate any feedback on this we'll try and move it to the execution api's repository sometime in the near future but right now we're continuing to develop it there so that's the main update if anybody has some feedback on that immediately that would be great

and it feedback a little bit of that um just due to internet dropout but i think

Paul Hauner: the idea of changing the names to not

conflict with the engine methods is a really good idea

Lightclient: great yeah we've also tried to simplify before there was a concept of like builder methods and relay methods and we're trying to just simplify it more so that med boost is a piece of software that exists out there but the design isn't really built around it it's just built more around the concept of retrieving external blocks and methboost is you know obviously a candidate for utilizing that functionality but you could directly connect to a builder that supports the api other people can implement software it's less bound to a certain you know existing implementation

Paul Hauner: yeah nice i saw there was um

just a little bit of chat and wanted to merge discords about um other um mev people getting involved um i haven't really had i think blocks reached out to me um but i haven't had many other people mev teams reach out to me about it um something we might want to consider is moving those api away from flashbots and into a more perhaps neutral repository right that's immediately a problem but just maybe a long-term goal to consider

Lightclient: no that's that's a short-term goal for me i think you know in the next few weeks we'll try and move it to like an ethereum repository and i think this is just a different way we need to be approaching this i feel like the block building like api spec is

something that's just sort of lived external to a lot of the discussions around the merge and to me it feels like this is something that should very much be an important part of these discussions because in all likelihood you know most validators are going to be using this software and so you know we as people working on the consensus and execution layer should also be spending time making sure that that is as robust as the other pieces of software that we're working on

Jacek Sieka: i mean this this raises an interesting question right which is that it's basically becomes part of the attack surface against

ethereum because if the networks are not able to produce blocks and uh clients rely exclusively on them as as sources uh we have a problem right so ranting would be to almost require that clients implementing this feature would um would also implement the fallback where where they make sure to have a block ready from uh from from our traditional sources as well and this is important both for censorship reasons then and yeah just network down reasons right

Lightclient: i agree

Paul Hauner: yeah i think it's it's critical that the two things that are critical is that if the builder gives you a payload you use your ee to verify it not you don't just assume that it's valid or use it to verify it so that's kind of a bit of a correctness um guarantee and then like you said yes the one that if they don't give us anything or it's invalid we do have another block to pick up i think those are probably the two key safeguards we can we can use here it wouldn't stop the builders from causing havoc but hopefully it would keep the chain correct and live

well but we can't do that right with the blinded block because we don't have the transactions can't do what sorry

Jacek Sieka: we can't verify a blinded block

like that the transactions therein um are valid

right so with the best we design the builder is is a is a separate entity so you'd have or muted make it disconnected yeah i also agree on that at it and it probably makes sense to have an accompanied document for ceo client employers on how to integrate and maybe boost in this software with

default backs and other stuff uh what all has been mentioned in the yasuk also uh yeah if there is a builder who produces bad blocks that are invalid i guess there are going to be a reputation system to quickly sort this out and uh just mv boost won't be connected to this kind of builders am i right well i think it

Jacek Sieka: i don't think a reputation system necessarily works reputation system lots of problems because you know really you you're you're reputable until you're not

um and i think well attack

it yeah

Lightclient: i was gonna say it only works if you can attribute the fault to a builder like if i can create some sort of proof that shows that they told a validator that for some slot they would produce a block based on you know this input and then they produce a block based on a different input then you know we could attribute to the builder and then we could say like this proof would cause other build other validators to disconnect from them removing them from that and that could protect from some instances but i think the worst case is if the builder simply withholds the block that they tell a validator they're going to release and there's no way of attributing that

fault to the builder

Jacek Sieka: yeah indeed there are a number of these like liveness issues right where i don't know a couple of blocks invalid blocks get produced in a row with no way for the um signer like the validator to really uh validate whether they're valid or not before signing

and so on right i think from from our perspective the important question should really just be like where do we draw the line between responsibility for like the med systems like dashboards or whatever systems and and the western right because there's like small issues i think we should just not be concerned

with like it's up to the user to decide whether to run the client if so then like how well that protects against i don't know griefing or whatnot like that's all i think not not really our concern but of course it becomes our concern and like if if there's like a big failure where we really want to be able to notice that and switch over manually maybe even just disregard the the the let me be boost or something and so the question is just like basically where where where should that kind of this for this emergency disregard maybe uh before they kick in i think yeah because otherwise it shouldn't be our concern

Lightclient: right i i think like a reasonable expectation is one for cl clients to implement a fallback mechanism where

they do continually continue requesting their el their local el to build a block and that will protect from whenever the fault is attributable and you can see that there isn't you know an issue with the block that you're trying to propose and you could just propose the local one that you build and then the fallback mechanism for these other types of liveness issues could just be if we haven't seen a beacon block come with an execution payload and in number of blocks to just disregard builder blocks until some threshold is met again just so that we avoid these like really bad situations where we're going you know epochs without any kind of actual block being produced

Ansgar Dietrichs: right but even that is hard right because you don't actually if you just look at the beacon chain like you don't you don't know whether a payload you're looking at actually came from from the med network or not so if you just have the threshold of saying like if there hasn't been a valid payload in 10 blocks then sure after 10 blocks of just no payload or the involved bailout you'd have one valid payload but that manual manually created maybe but that would then basically trigger the next 10 blocks to just try to use mev again which wouldn't work because it would always oscillate right because it wouldn't actually know whether the things were coming from the valid mbv network or just from some people manually using their clients so it's not easy to

Lightclient: right i mean yeah i think these are all hacks to sort of try

and maintain liveness and not an optimal system for dealing with the fact that we have centralized block producers and that we need to continue working towards having an end protocol mechanism for dealing with these situations

sure that as long as we have these tags sorry we we should try and make these the hex work as as well as possible at least right is there any way to at least say have some sort of right where at least whenever we see payload see whether that was produced by an external builder or by the validation himself or something i mean hopefully we'll

Lightclient: be able to get

more information whenever we switch to more of a gossip approach to external builders rather than right now directly connecting to them that way you as a member of that gossip network could potentially see that these payloads were coming from a builder but again this doesn't protect from if the builder is withholding

yeah definitely there are triggers we just requested that all all clients all clients have requested

Micah Zoltu: all clients um put something in the uh graffiti that if they're using a builder versus local i mean it's obviously not enforceable but defaults are very powerful and we probably get you know 99 of people to go along with it it's just default

Lightclient: sure i don't know how deep we want to go into this discussion right here but i do feel like it's something that we might want to consider talking about more and having being like more conscious about this situation

i also find like it's there's no representative there's no one from flashback here i feel like this will be maybe better i think the better place for this type of discussion is probably by bi-weekly having a sync up with a flashback team and then just to go over the spec what

Tim Beiko: what do you think is the best format for that this is like a call that's hosted by flashbots

right so last week we had one on the

flashbar walking group discord and the deal from the flashback hosted it so i'm not sure what the next one is but i think they're planning to do this quite frank i think i plan to do this quite frequently

Micah Zoltu: i think if our goal is to make this less flashback centric and more open to flashback competitors we should uh perhaps start by just having someone else host the meetings like the theorem foundation or

whatever

Tim Beiko: right and i guess the reason i'm i'm asking is like is this a flashbots meeting or is this like a general like block builder meeting because i think one of the reasons flashbots doesn't send people to say alcor devs and stuff aside from you know maybe having better things to

do is there is like this this fear like they don't necessarily want to like be i don't know taking up the time and like space in the core protocol stuff for their like private stuff but it does overlap and i think you know they've been mindful of like how they how to like manage this interaction but yeah we yeah so so if if it makes more sense to have it as like a ef thing we can probably do that the question is also then like how do you make sure you don't just overload people with meetings because there's like awkward devs consensus layer flashbots call or mvd call

yeah yeah

Lightclient: i don't know if it makes sense to have something similar to what we've been doing with acd is having those breakup call break breakout room calls for more specific topics yeah because you know this is something that's important to all cls but there's usually like one person from a ceo who's representing and working on that area

Tim Beiko: yeah i think that makes that makes a lot of sense and like especially having at least it doesn't have to be like a recurring thing forever as well

right like it could be just like we have a couple and and we see how uh you know if we needed if we could get more uh yeah more frequent yeah yeah

and

Lightclient: i have been working very closely with flashbots on this the past few weeks and like i have also generally been pushing to avoid enshrining anything related to flashbots um into these concepts you know including removing the term mev from any of these like apis like i don't think that any of this any of these things should be is like that opinionated we're just working with an external block building protocol not um you know flashbots itself and that should be done in an open arena

i'm i'm

Tim Beiko: mindful there is like def connects two weeks from now but do we want to schedule like

either next week at or an mvp breakout or uh maybe wait to see like i know there is like an mbv event that devconnect i know the flashbacks people will be there most of the client teams will be there as well so do we want to like wait and see if there's just like organic interactions that happen there and then and then kind of follow up after

death connect um yeah

Mikhail Kalinin: probably people will be busy next week with traveling about some other stuff

Tim Beiko: yeah but yeah it's definitely okay i can i can make sure that like we we we touch on it in person there and then um and then that like after death connect

we organize something uh that's that's open where where we can discuss this

Mikhail Kalinin: great um yeah before yeah before the call on the dev connect please go to this pr take a look it's a real simplification and this design is like um that's a pretty optimal separation of concerns things like time for the work on it and yeah this is very important that it's been mentioned as the component of the system so the more eyes we have on it um the better or the end goal for the end design so go take a look comment out anything on that topic before we move on okay anything else merch

related cool next step is other client updates uh we had a bunch of updates already but probably somebody wants to share anything else not related to the match okay the next one is research spec and other things um there is and from what i know there is an ongoing work and partial withdrawals and there was roles uh in general

um alex do you want to give any update on that

uh sorry you're asking about partial withdrawals yep i mean there was roles in general uh yeah

Alex Stokes: yeah not too much other than the prs danny's been looking at this from the ciel perspective so he'd have the most context here um yeah i mean i guess just anyone on this call if you're curious take a look the changes are i think not too too wild and yeah as far as i know the

latest there is just um tuning some of these parameters around like how frequently we cycle over the value set to either do uh sort of the full jaws or i guess in this case the partial withdrawals so yeah i just take a look but i think there's not anything there that's you know there aren't there shouldn't really be any like big open questions or anything like that

oh

Mikhail Kalinin: thanks alex and uh yeah when do you think it's when it is expected to get merged

Alex Stokes: um i don't know it's definitely not you know the top top

priority because we're all focused on the merge uh but maybe in like two weeks i mean i'm just making up a number um yeah

okay sure

Mikhail Kalinin: so it's gonna be soon then crad do you wanna say something

okay

Dankrad Feist: oh no sorry just rejoined

Mikhail Kalinin: okay cool um yeah anything on withdrawals questions comments okay cool take a look at the pr uh some of uh like the initial vr has been

already merged into the dev branch so specs are there also worth checking and the ips as well nice great so um yeah anything else on research and spec topics

okay i want to remind that we have a couple of prs in the networking to the networking spec and we are expecting more input from engineering side

um yeah please take a look at these pr's it shouldn't take like much of your time i know that everyone is busy with emerging other stuff so but these are reasonable changes to the networking spec and yeah please take a look comment out approve and so forth um uh one one of them is the deprecation of the stepper enter in beacon blocks by range and the other one is ignoring subset aggregates um your comments on these two

okay cool uh so it seems like we're done with the research and back stuff um the next one is open discussion closing remarks um does anyone have any other updates announcements wanna share here okay so [Music] um we are going to cancel the next consensus layer call because we will be making person on the dev connect so the next one is

it's not gonna happen we'll meet each other in this room like um four weeks after okay i think we are done yeah one

Tim Beiko: one quick thing though uh so the next cl call is four weeks from now but in three weeks so the week aft oh sorry well next week there is in all core devs and then two weeks after that there will also be so the week before and after devconnect and we'll obviously um cover all the merged stuff uh there too so consensus light layer folks who are working on the merge please show up at those if you can

Mikhail Kalinin: thanks tim

and yeah from like client we use block production channel on discord to discuss the mvp boost and other builder api stuff okay okay thanks everyone see you soon

thanks guys thank thank you [Music] [Music]

[Music] so [Music] [Music] [Music]

[Applause] [Music] so

[Music] [Music] [Music] [Music]

[Music] [Music] [Music] [Music] [Music]

so [Music] [Music] [Music]

[Music] [Music]

[Applause] [Music] so

[Music] [Music] [Music] so

[Music] [Music] [Music]

[Music] [Music] [Music] [Music] so [Music] [Music]

[Music] so [Music] [Music]

[Applause] [Music] [Music]

[Music] [Music] [Music]

[Music] [Music] [Music] [Music] [Music]

[Music] this [Music] so [Music] [Music] [Music]

[Applause] [Music] so

so [Music] [Music] [Music] you

Speaker names adapted from ethereum/pm, CC BY-SA 3.0.