All Core Devs - Execution #90

Transcript

[Music] [Music] [Music]

[Music] [Music] [Music] [Applause]

[Music] [Music]

[Music] [Music] you

[Music] hello everyone and welcome to episode 90 of the etherium core developer meeting I'm your host Hudson and we're gonna get started today with the first item on the agenda it's a topic that's not purely technical but we think it's important enough to bring up and have a conversation about so we're allotting a good amount of time for this it's something that has been talked about on Twitter lately a little bit and also in the etherion magician's forms and others mainly from alexey and its features

versus maintenance how do we balance creating features versus keeping the network healthy avoiding burnout from implementers and promoting implementation diversity and other things like that so Alexi if you want to go over that and if Regenesis is involved in that give a definition and stuff like that Thanks go ahead I think if a person so I wanted I know that there are a lot of conversations happening lots of places but I think it's and I think they should still happen it's not only the place where to do it but I wanted to bring it here because otherwise if we don't then it would seem like okay so these conversations happen on the fringes and

they're not really important and they should it should basically be that way but I don't think so I think we have to this is whether the place where we should start you know just for liberating those conversations because they seem important at the moment and of course I don't want to all these issues that have been touched them upon and then I can remind people that in some case it was that recently it was triggered by the increase of gas limit but you know if this issue existed for a long time and the things that I could think about is this Hudson mentioned are basically well the issue number one I think we need to look at is the burnout

issue of the the the implementers developers and somewhat connected to that ESG we I think is related is the how highly we value the the new features that we add to up on somebody's request versus the changes we would like to make to make the do to run it even easier to make it actually less critical because I think I mean the sentiment I'm getting is there there is or we some people actually operating in like permanently critical mode and I don't think it's very healthy I think people need to have ability to

relax sometimes and just step away without worrying that basically the the sky is going to fall down because they're not propping up from propping it up and then again as a separate thing which is also related the implementation diversity I would like to people to talk about how did they feel about the past when we had I would argue like a stronger diversity when there was a when the parity had a larger share of the installations and everybody remembers probably the episodes from 2016 when implementation diversity did save the network from extended downtime but I remember at the same time although we

had this diversity but we also had much more of the kind of the competition and rivalry where everybody wanted to say yes our implementation is the fastest is the best and whatever so I I guess you know how did you feel about that as well so I would like to understand like do we want to bring up you know if we if we if our goal is to bring back with diversity do we want to keep the competition or do we want to be more kind of cooperative and what else yes so we could start with that so I I don't want to dominate this so I would like to just people to speak

up and say what they think about all these three problems essentially the burnout the the features verse versus kind of the maintenance end to the point of the implementation diversity you I can go so a couple of points there so I totally agree that back in 2016 the diversity we had there was awesome and it was great and the guest team has never strived to be the biggest super

majority client and that's not what we want we want there to be diversity so that if we [ __ ] up and there is some denial of service then they their entire network does not go down and we can sleep better at night and we don't have to feel this constant pressure that the gaff is holding aetherium and network up so in an ideal world yes we have multiple clients again and get it's like well one of many clients regarding all that it's a kind of hostility race racing between clients yes that I guess was an issue but we've in the guest team

explicitly had this for a couple of years now has an explicit agreement not to I mean we can post about things but never more I've never liked we do this and the other someone else does this we have this much this speeds the other as this one we have this much data and the other one else you know we never compare so I mean I think boasting a bit or highlighting features whatever you want to call it it's totally fine but if in general it's nicer not to drag someone else into yeah regarding features versus [Music]

long-term stability or what you may call focusing on like core issues I think there was very good initiative of kind of started couple years ago in Berlin call crackin that we need the progress on state and and man how does how do we manage the state and how do we get it under control and it took a while but led to this initiative of the 1x group and then researching all these things and unfortunately from my perspective the Ebonics research camp has not really taken part in all the

court of calls to the extent that I would have hope to help focus the old quorum calls a bit more in the direction of how do we solve the base layer and [Music] yeah and and kind of prevent the feature creep from add-on features that do not solve the inherent problems that we try to combat I don't really mean to criticize the 1x I mean I think it's a lot of good things are happening but I think we need to try to feed that back into the day to day or week to week all court of discussions so if I hope this

no no it's actually this is exactly exactly these things I wanted to hear and I just I would very short comment on the very last thing that you said about the h1x researchers not turning up to the today over their calls and from my point of view I mean I can't speak for everybody but for myself I recently was more and more frustrated with the or coke or all chord F calls it's because it does turn into this kind of very bureaucratic project management calls where we just go keep going over these new features and I want a question why do we need all these things why do we need them now are we you know are we

really ready for this yet why do we keep pushing this way that's why I was trying to question why do we need this BLS right now why do we that the whatever the subroutines right now do we actually want all these things now because we do actually have lots of other problems we're solving and it's simply like I just lost this desire to go into this cause because it was such a chore and I I did not want to speak because it always kind of made me stressed and I said okay I'm gonna just keep myself from from stress say yeah anyway so that was thank you marching anyway just to add a few more thoughts to what

Marty already said and Alexi also hinted that substantially regarding the client diversity yes currently I think it is the kind of poll a supermajority client so if there's an issue in gas then there's a big problem in the network even if the network can travel long and and I we just like to highlight that although I'm super happy that people are using GUI theorem it does have its downsides namely a very concrete example in that we've built an acceleration structure forget that should make a lot of things a lot faster but it is this this all starts at rest

I've tweeted about it a few times but it is also super consensus critical so it's it's as critical as it gets and I think that code we've been sitting on it for over half a year now it's actually being merged and released in gap for months now but we still haven't enabled it although I think if it was more than half a year ago since we had the last actual issue with it but the thing is that there's always that risk that we're always afraid of just pushing the button pushing it out because what if there's some weird thing that we haven't thought about and we take the whole network down with it and then I think that's a that's a super stressful thing to constantly

have to think about that too much depends on on your optimization so from this perspective I think it would be super nice to bring back the whole tie in diversity even if I'm well aware that that means taking a slice out of gas by I think that's completely reasonable and completely beneficial for the entire network so here actually I want to ask her I mean to be to become a bit more concrete because we we did like I mean obviously we haven't heard from everybody but I'm suspecting that everybody would agree that this is a good

but we could keep talking about but also we could also try to make it a goal and I don't know exactly how we're going to measure it but I think if we do make it as a goal and try to go to it instead of just holding that it's going to happen automatically which it might but it might not and so I mentioned in the in the core in a chat and some kind of bargain I know that a lot of people don't like these kind of bargains but essentially it's the it's the way for us to say that we do have this goal of either diversity or performance or both and until we get to this goal we're not going to try to implement big new

features because we have to sort this stuff out before we go further otherwise we all gonna snap and not all of us but some of us will snap so it's a bit pre-clear issue I guess then just simply agreeing but I've also later on asked what to see what the opinion is okay so I mean it's nice to say that hey we should really push the words client diversity but I mean apart from gasps just going on vacation for a year and waiting for other clients to become more popular I mean from I'm okay I'm speaking from my perspective I don't really feel that there's much I can do I

mean I I won't stop improving ungass the team won't stop improving hunger so it's not for you to do it I think if this goal is not for you because you already I mean it's actually for everybody except you including people who are not core developers including the people who actually are users who are deaf developers because the easiest thing they can do right now is to just simply go to guess and then everything kind of works right and I know that people even started to you know after all this like a difficult conversation about parody and open aetherium I saw a lot of people making sort of political stunts and say no I'm not gonna stop running parody or something like this so yeah this is

actually contributes to the lack of diversity I would rather like two people outside of the core developers actually feel that they are also involved into this they're not simply they if they want our us to achieve this goal quicker they should actually try to use different clients because otherwise you know if they if they don't use it I mean I know they might be a little bit slower and they might not be so familiar with with the new clients she should use them otherwise we're gonna be keeping you know keeping this mode I cannot just make a clarification so this is what I understood being your proposal it's not like the clients should preach a priest or anything but it's that until we reach

a degree of trying to versification that we're happy with we should not do hard works or add more pictures yes yes that's what why it might kind of bargain bargain proposals and that is not to just ask but to everybody say look you've got to help us with that I think I'm on the fence about actually having a complete freeze however I do think there are some concrete things we can do to help client diversity and it's gonna be a thing that's outside of the core developers you're absolutely right and it's gonna be on a few different levels one of them is a concerted effort at a communications and marketing level to

make sure that everyone knows to run different clients that there's been some behind-the-scenes stuff including months ago with myself and others going to the major miners and saying what would it take for you to run nether mind or what would it take for you to run Basu or what would it take for you to run parody or open aetherium more and I know that there was some collaboration there that I don't really know how it panned out because I passed it along to some of the core devs but some other ideas that you know we can kind of brainstorm from with in coordination with core devs is like maybe some coordinated blog posts where it has like a run your own core devs and it has a like old war posters where it's

like getting in the cause or something something stupid like that like kind of cheesy but something that brings up the issue and people are aware of not something that seems more like a threat of you know we're going we we refuse to do this you know new stuff until until we get this divert client diversity I I think that we shouldn't say it that way necessarily it might be the reality but the way we mark no I've already I've already said it yeah it's already on the table but that's the way it's the narrative and the way we market it that's going to make a difference so I think pushing more from that perspective is gonna be healthier you know you know the I would have agreed with you some time ago but I don't

anymore and the reason I'm not gonna explain why is because we are although you liked or some other people like to call this ecosystem a community but I think it's grown much larger than just a community so you cannot own any more say please please and they everybody will do that so what we're dealing with is not simply with this sort of nice community but we also deal with rational actors with the businesses with game theory essentially and you have to play by the game of you know you know yes you have to do both you have to be nice and say please please please but you also have to understand that a lot of people simply exhibit rational behavior and they know what other people are likely to do and

therefore they will coalesce on certain action which basically means that minimum effort maximum reward so from that point of view unless you unless you buying certain behaviors together you cannot get the result on a scale that you want you changed my mind so that's what I'm putting on the record I think you're right we need to go in the direction you're talking about and then Tim passing it off to you yeah maybe to make just like an excise point a bit more concrete right like what's the game theory looks likely you know we are them and our minority client and basically say every you know you talk with a company that's building on the theorem whether they're in exchange a black Explorer you know whatever you want if

they're using only a single client whether it's guess whether it's open aetherium their entire infrastructure is then kind of you know dependent on that client and and there is a cost for them to make it like client agnostic right um so it's I I think again it's easy for like individuals running nodes that are just watching the network or sending transactions whatever to run base you instead of get there open aetherium or not to mind but if you're talking to say in exchange and they're all built on get you're asking them quite a significant architectural like lift to get to a spot where they can run say one of each client um so yeah just to make this maybe a bit more concrete

than like what what are the implications of an ask like that of businesses that are built on etherium rather than just individual users if I might have worked for not too popular it'd be I would say like I don't accept find diversity will ever happen just because as mentioned many times before people will run whatever it's easy and for example I would never run a sever base crime because it requires 3 mm teller my weapon for example and then there is second problems even with all the teachers we saw already implemented before since problems but they proved to be less of each server provided various

orientations and other extent it responses how much science happens that have a problem with performance which was never taught before and no one would ever have it I think in the long term if we didn't start to page Marty I don't think it's just a sustainable water to keep four five six crimes more or less the fastest one go read anyway maybe two of em but I don't think proposing and trying with all the intentions to push the client diversity well payout at the end of the day we already

sees that there was not possible to push for purple and with the same part a lot of people from all other bodies already investors of us in maybe their modifications yes or any other client I think a less visible ever speech if the heap they wouldn't switch for kind of direct benefit which would be potentially icy features which is hard because they have to be implemented everywhere or for performance which is much more difficult for you just amazed but I mean I don't like maybe it's a personal thing but I don't see the diversity thank you for your opinion actually it's very

interesting so I'm just gonna answered too quickly so I would not say this is completely wrong because I don't know obviously we're speculating here but I would also add that it's not this particular kind of bargain that I'm suggesting it's not only about planned diversity it has a secondary effect because as you say that it takes so it will take a while for the for the client diversity to materialize if people or we actually it's gonna take us a while to understand that this is not going to happen right but I think if it even if it does not happen and it takes us one year to realize this I bet that in within this one year we will actually

have more performant client coming into the scene so it might even happen so at least what would it really would have achieved we would have given people more chance to catch up so one of the issues I think is that it's really difficult to catch up with I mean try to implement something which and it works but at the same time to keep adding all these things it's not actually easy so it's like trying to catch up with a train that goes away all the time ok but thank you for your advanced heavy like having time to build more clients would say

performance recall implemental differences everywhere is great but it's only possible because the factor is hume is a monopoly if there would be ever added watches would be competitive as much faster in traditional features and blah blah blah this would never come like no one would ever care about it defiance and through our protocol and everything they wouldn't like from my opinions they would be rushed and pushed to implement more features to stay Empire with some other competitors which is absolute on the market at the moment which makes this problem and if there would be a institution on the larger scale this

problem all right look maybe not even in the first place but some would be natural is off as a result association between racially yeah I don't agree because I think the I mean it's easy to spin up a new blockchain and add bells and whistles and everything but couple years in you get to the heart problem so actually managing state that's where we're at it's how could you move away from the

birds a bit I'm having a hard time understanding you with all the birds that's a problem that just outside we left yeah I would see I feel like maybe we should return to like the conversation of solving the burnout of client developers and the pressure that's on clients and the tension for adding these people want features but they don't realize what they're doing by having the ecosystem be like this is to inhibit Cortez from being comfortable with putting in new features and I expect to be comfortable here you

know with tea now I feel totally fine with having in the future so just put so much much burden on us it feels like it's much harder to catch up if you give time to all the teams to settle because then the teams that are already established will just keep improving what they already have without any features it's it's not any easier you know if you have to compete against stability and performance because that's that's what all the teams will do they have time to focus on their things that just start improving the crowd it's natural for any development and that's what we do that's what we race and I'm totally fine with this new future so I lay the EIP s they are coming they're just two or three weeks of work a year it's not that much they probably those calls are taking more time implementing

their speakers in yet and this they're also important we also so fun to to be on and trying to be artificial and preventing this natural course of action where we just have to implement the things that we're expected to implement because we are competing in a in a reasonable way why are we competing with also respect for each other sometimes together client teams that's why I think they're doing an amazing place it's hard worried but in a way I think why we do that I mean it's not like we cannot move away I think we can do we do that because we think that this is something important this is something big this is something that gives you a good feeling to keep doing something great right

like it's a it's a theorem client it's a cute alternative huge task on the side of technology or are they requirements to understand what you're doing just amazing I'm just to add to this so I would say like for example some of us unquote have been working on account abstraction and it's gotten us pretty deep into the goth code and now we've been on boarded to that and so I think from our perspective something that's totally valid is for example you know we get an EIP out but

realize before that moves forward it's time to like double down and help the clients to reach better efficiency so I I don't necessarily think always the answer is you know don't you know discuss new features don't you know work on these new VIPs because if you let people do that they're probably going to become more I guess more involved in understanding of the code base as well and able to contribute in these in these other manners Peter I mean one thing Oh afternoon let's have Peter go because he raised his hand oh yeah I was just gonna add one thing I guess basically what Thomas just said and we'll it's true I think also from the basic perspective

implementing most eeap's is not like a whole lot of work like it's I'd say probably ten percent or less than what we spend our time on um I think one challenge though is there are a lot of bigger more complicated features coming down the pipeline so we'll you mentioned account abstraction our team has been working on 1559 Alexa you're obviously meeting a lot of work around witnesses and whatnot so there's like these these huge changes that seem to be like much bigger than you know your average keep that are coming down and I think they're given that it's a risk that we'll probably never you know reach zero and even with

the BLS CIP you know the get team is saying that that's like a pretty risky one um I think that's it's a spot where like having client diversity is kind of a hedge against the complexity of those features so even though I agree that like you know most of the feature works so far in terms of eeap's is not like the bulk of the work and if we do have these way more complex each that come in that will obviously have a higher risk of a split it like maybe just having more clients there is a way we can we can protect against against that let's see the tutorial to has the same situation a lot of clients implementers on you you still will end up with a some

clients having majority of the network because Indian some of the clients will be will be better in some use cases the other ones will be mentoring some other use cases and some of them will be just better than most of the use cases one thing that we can potentially improve if if we can ask for something is maybe a slightly stronger separation of research and the client development like what we have a material to you have the researchers that are providing very detailed specs of how the things are done and then the client implementation teams are not burdened with doing so much of the experimentation and research and also there is not an implementation of the huge in the futures in particular code base but whether we want to really

change it and want to complain about it and not sure so at the moment you have the gap theme and gifting is doing amazing work on the Reach's research side adding the actual implementations of advanced new thinking conference right we were doing this kind of beam sinking into adaptation and that was coming from Trinity and researching and and this main starts it's really hard to catch up in those cases but again maybe this is the way to go with this board of competition of being in Nova tape this is this is actually the good outcome of the diversity the different teams will be coming with some big things those big things will be competing with each other and then will be implementing so maybe it's all about sharing this this

outcomes as soon as possible writing with respect but can you imagine and actually enforcing great expect some people already so much strained with a with what they're implementing like one you know what Peter as if he said myself probably we're doing fine a lot of my coding work long hours and if you want to add on this just sharing the Torico Berger clients won't be hard let's go Peter than Martin than James based on who raised their hand first I'm just gonna say let's go by hey grant hands raised so we can keep maybe yeah a little more like an order go ahead Peter thank you so I want so so Tomas I

mentioned a bit back couple minutes ago that essentially adding writing the EITS takes relatively small amount of time and and I think his suggestion was that we shouldn't really suspended because it's not really that big of a burden and I would like to voice my disagreement with that because although it can debate whether how much of a burner of it is implemented or not but I think most of the people have taken for granted that they implement their code and then somebody else to find the bugs in it and just to highlight it Martin and Mario took literally took them weeks of fuzzing the various BLS implementations

and I know that we found I don't know an order of magnitude seven eight bugs in the go code somewhere in assembly somewhere and go the last one which actually is a consensus issue in Yolo we fixed yesterday or two days ago and we actually also managed to crash Basu I think I'm not sure whether it was never mind the rule I know there was some what we there was a surprise in which we found an issue and the thing is that each of these cases would have been a catastrophic failure for me not thought but a very serious problem for me not and and when people say that well yeah the yeah be to just one week to

interpret yeah but there was there some dis rendom dude somewhere else which take to whom it takes months to test it and who finds the issues and it's easy to delegate that to somebody else but so the reason why currently the gaff client is I mean the guy can really pick this testing up is because since we are the majority of the network for us it is - very important that we are correct because we cannot afford that to be correct because if we are wrong and for example neither mind is correct then it doesn't matter that that the minds code was correct and ours was wrong because the network went off on the wrong change so for us it is super incident just an extra dump of burden just to verify it

but yes it's code or for our code so that I just wanted to highlight that it's not really fair to say that any IP is uh is little work but because it said it's a lot of testing work which is completely hidden from everybody currently so that was one thing and the other thing regarding features and different clients working on different features I think that's completely - for example there was this Trinity is being seen horses gets a snap saying I think these are completely healthy I want to call them competitive things I mean these are experiments that the client are doing there are nice things about these experiments are that individual clients can try them out and can roll them out and if they worked and

other clients can pick it up if they don't work then it goes to a trash can and we've thrown out at least six pruning algorithms so it's only the first time something some of our codes ends up in a trash can so so from these kinds of features I think we can definitely work on an improved but any anything that requires global consensus across every team that's I think that's very very tough task okay Martin that Artem yeah I guess Peter said a lot of what I was going to say just wanted to add also that the the whole passing of Yolo I was a bit disappointed because I was hoping that

since everything I've merged into the clients we could have frost testing based on state tests but it appeared no other clients actually implemented it the status we actually had to do the passing on the test network I just took a lot of time it was very inefficient but despite being so inefficient and us testing or actually not me I should is Marius I should take credit for this he we still managed to find a couple of issues so I mean I think they hold these big features harbour dangers and the big work is not on the implementation side okay Artem yeah so basically I just wanted to voice my agreement with

general agreement with Aegis assessment I mean I cannot say if the total feature freeze whether it is a good idea or not like an absolute feature freeze but I would just like to highlight them the fact that what seems here is basically the contradiction between goals of their minority client and the majority client simply puts the year or minority client can experiment because if something happens and if there's the consensus is here and the minority client gets kicked off the network then the network will chug along but if a client with large market share gets kicked out then takes

the whole network down with them so basically maybe on the perfect goal would be to have a lot of minority clients basically eventually and once we attain that goal then our experimentation future experimentation will become a lot easier and it will move much faster so basically it is the groundwork for future experimentation Peter oh oh I I don't know if you raised your hand or not sure you and then Thomas one thing that I think so one so

I think one it's been highlighted quite a few times here that one thing that is really hard for for smaller clients is actually this whole playing catch-up is because Roddy theorem is a moving target and you have to do optimizations and that King and whatnot and it's everything is moving forward so one thing that perhaps could be of help is if we twist somehow so from gas perspective it doesn't make sense for us to put in work that is not beneficial to get only for some other clients but nonetheless for example Trinity had this proposal of extending the East protocol with some request IDs

which guess it doesn't need at all because we spend an eternity implementing our synchronization algorithms to work without it but if we were to have those request IDs and it would be super easy for anyone else to implement simpler synchronization algorithms and I think this is one possible direction that we should explore namely what for example how can get help simplify certain things for example how can we roll out a tiny modifications to the networking protocols where it doesn't for effort so for us supporting it would not be too much of a hassle but it could really cut

a lot of burden off from from clients who don't want to implement the same horan this complexity that we we did the last three years and I think this is one possible direction which could help on board I and into the network easier but I'm sure other clients have other ideas where they think it's things are too complicated and maybe maybe we can somehow work together to solve them where this where that work isn't necessarily beneficial to the end-user nor is it beneficial to gas per se rather beneficial to client diversity superbee a to two things for you I want

to respond to the first one when we were discussing whether the eyepiece or a big finger small so I totally agree with marking that it's a lot a lot of work on testing but I think it shows this a blurred line between various teams self-maintained by the theorem foundation that are having theoretically different roles that are actually within on same team so we have like testing high testing security of the network and experimentation research and the guest implementation these are practically the same same people right working on all those things cell and this is great and it probably helps to deliver good stuff but at the same time this is what makes it a bit more single client focused because these

people feel more emotionally attached again right so they'll be implementing things in get first improving things in their first speeding up things and get first test and click at first and and this obviously makes it harder but again for the network I think we should focus on what's best for the network and that's for the network is to have at least one super stable super efficient fine ally we do all our best you have to but obviously one thing that's most important is to have care this puts a lot of burden on the gift team so then if we want to simplify forget time actually is looking for something that makes it simpler but maybe we

should consider splitting those roles and having this security and testing to look a bit more client agnostic and okay it's like always picking out free clients in a flow basis and implementing the improvements in the same ways whether this is desired I'm not sure but this is for position they the second thing that just Peter mentioned was actually example request idea I think there's one example of the things that I'm slightly feared about because this are improvements to the things that are already there and deserved particularly things that are hard to for us used to gain a bit more of the usage of

the point right because they are quite invisible they are equally accessible as Peter said it would be relatively simple to implement in gas it would be actually coming back to the synchronisation improvements again in undermine a lot of work and I know it can be helpful in the long term but this is actually one of the biggest burden things if it was introduced like the things that are not in consensus not he repeatedly and changes with me on our request IDs symmetric optimisation this are very hard to implement and never visible so they practically decrease base diversity of the client system let's do Alex

blasts off Tim and then Alexi yeah I will be good I think Thomas kind of already explained what I wanted to emphasize it even a feature priest happens and all the clients become people it's difficult after this point to send see the difference and actually kind of rationalize why just switch to another one if they all become equal and if you use guess before you will just continue to use it after but Thomas already mentioned it okay Tim yeah I guess related to this testing and security infrastructure question this is something that we at base you have tried to take more of a role in but it's it's kind of hard from the outside sometimes

to understand what's the most valuable thing we can do and and how to actually contribute value so for example last year we test that was kind of the big thing we implemented on at that and it seems like it's it's maybe not as critical in terms of infrastructure now we've tried to add some reference tests to the east when we when we implement them but there's no it always feels like we're kindness tumblings likely in the dark and and you know trying to do our best but there's no there's no way to know in advance oh these are actually the three most important things with regard to test a nyan and security that we could do and and if we wanted to have the highest impact this is how we should do it um so I guess yeah I did that palaces

point there would be if it was less dependent I think on just the EF generally and and or at least you know I don't know if it's like a question of documentation or or having a more explicit process but yeah we'd be happy to contribute to those things but I think for us we want to figure out what's the best way to spend our time when contributing to that okay Aleksey yeah so I wanted to address the kind of the couple of things about the implementation diversity again so first thing is that this point of view that we should just let the what they call market decide just let the things to do

to go as they go and somehow you know if if the equilibrium of the situation we have now is it's basically like one majority client then then we consider this inevitable but I disagree with that I think we do need to try an action to change the equilibrium and we the way we do it is to to shift the incentives and that's that's why I started to think about things that might actually shift incentives of different groups of people into the right direction otherwise we you know it's we just throw our hands and say oh yeah oh no can't do anything about it and the second the this idea about when we do get if we do get to the

implementation diversity so what does it actually bias actually I if you think about the theorem client is a at least at the moment it's a quite specialized thing and it does require a lot of expertise to operate and so if there are let's say ten different clients let say ideal situation and if you are a big business that wants to you know once the proper support wants to you know utilize it once to have like a direct line to the developers or something like this you can't get it with Goya theorem because there's like there's only three people there sitting there who could actually answer your phone calls or they won't even talk to you because they're so busy so if you had like 10 P 10

different teams you can choose from you just figure out which one you kind of like or maybe approach the ones that you know they they currently available or something like this and then it's it it will grow the the the entire ecosystem bit bit more because you spread things out we decentralize and that's what I think we should try to do that at least okay and then I think we had Peter Alex Greg and James I think so let's definitely keep things not super long everyone's been really good about that so far but just as a reminder stop personally man complained that

essentially the problem he sees is that hi this run by gas to get theme and testing is run by the gas team not really but yeah and he fears that a lot of the problems stem from the gas team doing all of these things and I wanted to react to this that we don't do this because we enjoy doing it so actually we really really hate doing the concerns testing and we really hate doing all the other stuff but the point is that since we currently have a huge majority of the network it we kind of feel it as a responsibility that we don't Bork the network so it doesn't matter what other

people are doing as long as we have majority we will keep on doing it because that's part of maintaining a client making sure that it works so and you cannot really ask Martin to start doing the test another mind or or etc because simply we develop in death so I'm going through and that's where we feel at home and that's where we are going to continue pushing and we for example with hi the goal was to make it as cross client as possible and to share any testing effort across all and so we do try to to help other clients too but at the end of the day we are getting t Belapur so it's fine if

you bring up if you can form another thing that does this testing or that testing although there I kind of still have the feeling that people are in are deflecting so instead of making sure that their clients work correctly we're just hey you just hire a testing team that will automatically solve the nasty work and I think that's every client who wants to become really a majority kind they kind of have to realize that most of the maintenance is dirty work and it's it's relatively rare that you get all the glory of hey you shipped a new feature anyway I don't remember what Alexa said so I can answer anyone yeah all right I think next was Alex

blast-off um I already oh okay okay you click the lower hand button to lower it oh there's a lower hand button I didn't know that either um Greg I think you were next if you I think you're muted Greg okay I found the button I do think to some extent death having a big majority is inevitable it just is in in this sort of market it's not a game

theory issue it's a market issue someone mentioned the integration issues once you have it integrated into a system unless we have an actual plug in pull a standard you can't just take gas out put something else in if there's a problem and given that it's just not acceptable to have three people trying to develop and maintain and test this client and I don't I don't know how to solve that problem but freezing features on the whole system I don't see that that that really helps a whole lot

we don't put that many new features in we it's our job to be the gate on that but it's I just see the gap team is ridiculously understaffed for the purpose of managing and network with so much value riding on it and it's so important to the job that it does and I don't know how to fix that but if that isn't fixed freezing features anything like that is it's just not going to happen and the minority clients are not likely to compete on that level they're

gonna compete by fitting into places in the market where death is is not going to fit like people who need actual commercial arrangements there are companies that simply won't use open source software they want somebody just to when things go wrong and they want 24-hour customers to put things it gave or provide so it's not for us to try to make the market do what it's going to do that it's just a market issue and I won't take much more time

Thanks okay James then Tim then Martin the just the distribution I don't think will change like the the equilibrium as it is won't change unless we do something I think that's a fair assessment the other part I wanted to make sure from Alexei that we've talked a lot about client diversity and I want to make sure that that is Haiti if there's other parts of this that we should be talking about I want to make sure that those get highlighted as well for the talking about client diversity the this might be radical but this would totally work if we said for 1559 or some change that the community wanted that we

but we required a certain set of diversity and quantified that before we're comfortable deployment then we would get an army of people who are who are trying to get client diversity to happen so there and just I'm thinking about next steps and to end concrete action items for approaching this I've been spending a lot of I get previously have been helping with eath Linux and kind of getting that the last couple weeks I felt like what I wanted to do with that I sort of happened I and that perhaps I could change that time and effort on with the thwa next stuff and then move it to being towards client diversity and it fits in the well in the

realm of heart for coordination if we talk about it in the that if we set the standard of the more complex that changed and the more we require client diversity to happen to implement it so that it's safer for everyone and better for the client developers okay Tim if you had something I saw your hand was down so I wasn't sure if you still had a comment okay Martin yes situation also give a hot tip to Kareem for a good work on from the besa team but actually so testing it's one of those areas where

if you don't know anything about it you kind of imagine that testing is easy and it's a good like entry level task and and in fact that's quite the opposite of how it is because if you're going to write the test or validate test or make sure that it's a meaningful test in ethereum consensus you need extreme knowledge about exactly what ppm is supposed to do and should do and what might be interesting corner cases and similarly for like the dev peer-to-peer test of having hive it requires a fairly low down knowledge about how the peer-to-peer networking works and it's it's actually something that you

probably I mean you probably know client implementers or at the bounce who have this knowledge to write these tests so it's kind of a tricky situation and it's hard to add new people because these new people will almost by definition either not know sufficient sufficiently for the task at hand or they will also have their other day job of being no developer on one of the client maybe we can try to do to include more people from the other teams into

the the test team that the foundation has we also have a weekly phone calls which so far it's mostly it's me and a few more one one guy from two dotto and we talked a bit about passing it I will be totally open to opening that up a bit more so other clients can also partake and understand the kind of challenges and what what there they can do in their clients to make crossing easier and yeah make that effort to go more smoothly about a year ago I mean long time ago we had the physical we're also paired folks

were involved but I think he left and I guess that's problem people leave after a while yeah so what I meant to say was that let's try to to do something together with the testing and I'll try to reach out to some more teams and see if you guys can volunteer one would like have one person who can join some calls now and then assist with testing and what some way shape or form that's it okay so open a theorem and base you both jumped on the chat and said they'd be happy to contribute Alex blast-off said that they're curious as well we'll go to Thomas alex blasts

off with an Alexi to bring it back in interestingly some of the suggested solutions here are actually definitely going in development different direction and promoting diversity first of all when we say that 15 is understaffed of course it is and it will always be even if it have 20 or 30 people just will be developing faster better more stable solutions not saying that they are not Austin to do it now but it's any project where you Ori on the marketing competing against a single other client will always make you feel understand if you add more people to the team and get

which are not scientists shouldn't then you make it harder for our clients to complete because we need bigger budgets to match it which means that I need to ensure that commercial if we bring you more clients more customers to be able to continuously maintain the maintenance solution on the same level as gap because that's what they have to do you have to ensure that the main that is supported in their mind equal or better or slightly worse than death and then we can only insure with our customers that they'll feel confident to another one but it means that it requires more funding and the funding sources are different across different clients try and some of them

will come with Co money some of them will come free their funding from Foundation some of them open as a funding from the commercial customers class maybe grants or a mixture of all of those and if we if we talk about the adversity office it's really hard to avoid for fixes of money the funding will be shaping somehow and how with clients operate from my perspective I'm I agreed to participate this marketed with your clients on these on these rules based on those rules that were known to us right and I want to be protesting against any changes it will make it even stronger not obviously if

you want to set the goal of increasing diversity then suggesting and appointing is understaffed then you should be centrally funded more it's it's increasing the diversity and the other thing is like testing and somehow we actually contributed by they the one person is working on testing full-time now but it's just not easy testing now that's the invitation to participation from other clients again there's an additional burden right serve while I fully understand its huge amount of work effort from gifting this is actually what helps us events like they get accepting that because they are bigger

there are some functions that are slowing them down the things a bit more responsibility taking a bit more of the burden off of testing even if others are a bit sloppy here because it was easier for us to just catch up always join slightly later at the testing phase because that helps us to catch up engine okay Alex laughs off yeah my it's more or less like it may be naive reaction to what Thomas said before that you have to to perform certain testing you need to have a person who actually knows the client from and from supporting maybe it

would be kind of maybe not a perfect solution but maybe it's better to just take people from every team mix it all together and say now there is no client team as a whole there is a team could ask implementation three different languages and features whatever they want for network player there is a team who does it for IBM and it's a lot of computation PDM right now there is a team who does well let's say whatever storage model and then you have all those people who actually know what they're doing in this whole scientific system but they work separately and even while you can apply the problem

different languages here they still would be able to synergize better you see for MSM joint problem which for which in every project there is not enough people to test maybe it would be better it would be hard to standardized interfaces between but it's a technical problem but maybe you can start by a social sweater okay Alexei hey thank you so a couple of things so first of all I would like to remind everybody that you know we seem to a lot of the suggestions seem to fall back into this comfortable territory of okay there's something that the core

developers has to do but I want to remind what I said before that which I convinced that any solution that actually works has to include people outside of the Chordettes I don't think we like as a little group can actually solve this we need to have help from outside so anything which kind of only like oh let's do this let's do this cool or that let's do this bit more testing here I don't think it's going to work because it doesn't have enough power and momentum to push back on there this has to have like some sort of corporation another point is that Thomas said that the rules were known from the start and therefore we can't complain I disagree with that because we all you know we all make mistakes we change our you know we

change our mind we we turn we find themselves in a very different circumstances from what we were in like say three years ago it doesn't mean that okay if you got there you have to stick with it it's like saying to the gate team at the moment because you're there you know you can't leave you have to stick there it's inevitable you know just you know deal with it I don't think it's fair to be honest to do that and then another one from a comment to what Gregg said about the market again this invertibility I just simply don't believe in it first of all I would say that there isn't actually like a lot of market around the core developer development the reason really I mean I don't know if managed somebody probably did manage to make it commercially

viable but at the moment is simply run on some sort of subsidies so I would that's why I prefer to talk about game theory rather than the market because the market isn't doesn't exist to my point for my point of view but the game theory does exist we know that certain people act in certain ways because we are basically providing them with something and the question is that how are they using this how do they do we do do we want to be kind of to be crushed by this by this desire to satisfy other people or do we not and to the Martins point about fuzzing I want to emphasize that at some point I

was I started to kind of fall in love with fuzzing unfortunately I can't spend any sort of serious time on this right now but I would love to and I think this is very often under overlooked area which is like it's a blooming area of computer science and I think this definitely has to be people definitely need to grow this this kind of skill that's it James you're muted if you're talking I was I was muted and not talking the I wonder what other kind of like I I would

like for I think this discussion is important that we're having I think it's identifying some some underlying things that have caused friction among the cortes and the community so I want to make sure this turns into things we do things about and perhaps I can have a discussion about what that actually entails and who would who was a part of that yeah so building on top of that just a little bit I want to remind everybody that this will continue and the getter chat and probably in a theory of magicians thread is would be my recommendation since that you can only talk about so much for so long and a

call and the organization is a long form like a magician's form can be so just that and something I was kind of wondering either Alex Parrish's E or Vitalik or someone else is there's something from eath to that they're doing that like or they're doing with the community or getting some kind of outside coordination from people to do like client diversity is there like a plan for that or anything we can learn or obtain from that as so the one thing that we did do at the beginning which kind of learned from each one's experience is just not accidentally create a situation where

one client has a big head start I think in 2050 and the big problem is basically that guess is like the only clients at the beginning and that created a bit of a kind of self reinforcing equilibrium and but that does seem like something that's harder to like it's it's advice that you could give to a new project but like it's it's hard to see how to turn that into a kind of an actionable recommendation for an existing project I mean unless of course like something happens like when each one actually and ends up merging with each to then agree deliberately try to I kind of get different he's one of my implementations

talking to the different these two implementations or something like that but you do need a pretty kind of serious shot for that and then in terms of so the main thing that we're kind of doing there is just supporting multiple implementations and kind of committing not to launch the phase zero until it's already with with multiple clients which it seems like it is what about with about for about two connects a altona I don't think there's that much other kind of special stuff yet like because everything is still a kind of in

construction and testing phase like we're not really at the points where you can have conversations about like who is going to develop who is going to focus on speed and who's gonna focus on enterprise and who is going to focus on like getting some fancy new syncing protocol out to people who wanted who wants a beta tested faster and so forth I see a few hands raised I don't know if they were from before or if someone wants to talk again I think Martin just did one I would just like to something also concrete in saying being which Alexei proposed and that would be to take a hiatus from Berlin planning for like July and not basically

show that that could be helpful who else has an opinion on that specific course of action that kind of concrete plan I would agree and I would I would say that the reasoning is is the BLS pre-compiled is complex enough that it requires an amount of either client adversity or things to really consider safely to point okay something else I want to wrap this up in about six minutes so that we have ten minutes left for Matt to go over there a IP that was just an EFI that they're gonna be introducing

because they did attend the meeting today to do that and I want to definitely give them time if we have it otherwise everything else can be pushed back to the next meeting or not even discussed at this point as far as Berlin II IPS go and we can continue a lot of this features versus maintenance stuff offline and then bring some actionable items to the next meeting is how I want to proceed but I'm definitely open to people thinking there's a better approach or a different venue or other than a theory of magicians in the chatroom if people have ideas that would be nice as well what other comments do we have on this or do we want to wrap this up I I from the catalyst perspective wanted

to say something over here from the discussion it it appears to me that there are some kind of coordination may be required among the clients that we have like resources are available tasks are available but the missing link is the coordination maybe and that is not Anya and you anyone not on any individual client that what they are the need to report it to somebody because it's open and it's transparent we at github but sometimes it may be an unimportant information that is slipped so I fire I feel like I have been here for a while and I feel like that his

tedium cat herder group is some kind of underutilized and they have been doing some of the coordination jobs and this is one of the area there where they can you know can't provide a lot of help to all these clients kind of syncing up together and as a as Marty mentioned that there are a lot of jobs for testing and kept team is only doing so if we can get this information and share it and reach out to different clients trying to get people for that particular activity could be very helpful cattle herders can play a major role over here but I really

do not want to impose everything on that I just wanted to throw this idea so if people agree to it we would like to do that and we can plan and discuss something coordinate in coordinating different meetings that may be required or something yeah a coordination task especially once the require meetings we can the cat herders and people like myself and James can route to the cat herders or coordinate ourselves even though it sounds like a small thing a lot of it this miscommunication of not understanding the other teams resources and what they want to do an approach I think is an issue that we definitely can

help solve a little bit so I'm gonna let this finish out by going back to Alexi if you have anything final you wanted to say since this is your topic if you don't have anything that's fine - yeah I was just gonna say thank you for everybody to join the discussion and I hope that you felt as I do that it was useful to do that and we it was definitely right to start this conversation here although we sacrificed a lot of the like pretty much the entire agenda but I hope it was the right thing to do so thank you everybody thanks Lexi yeah yeah definitely the right thing yeah I agree okay let's go ahead to the last one and then we'll go over the disk previous

decisions made and new action items so the next thing would be e IP 27:18 type transaction envelope there's a link and item number three that you can click on to view that and that would be I think Matt you got you came here for that one right I did go right ahead thanks a lot some two quick things before I go and said I just wanted I was in a really great conversation was glad to hear that talks about all Cortez I don't want it to seem especially after that but I'm like trying to push more change into the protocol I'm just here to try and champion that's the IP so at some point in the future whenever it we're adding changes again if if we do a freezer and odd that this could

potentially be included and then this is also I think the first time the IP has been made into an all-court dev so I'm not exactly sure from the efi he IP if that means that I need to come back in a later meeting to determine if it can actually become eligible for inclusion so you can just let me know yeah you'd be coming back at a later time to determine if it is this is if this is the first time it's being discussed then it would be more of an introduction to the e IP and then if it's efi or not would be determined in the future so it's actually we put it on the agenda incorrectly that's our fault B as it's not EFI this is more it does say in a5 review but yeah that's just to clear it up okay that's what I was thinking

yes let me just introduce it real quick if you guys haven't had a chance to read it the EITS 27:18 pipes transaction envelope this is really similar to something vittala propose and EIT 232 back in early 2017 the problem is trying to address is we're trying to introduce new transaction variants or spend a lot of the IPS over the years that have wanted to introduce new transaction formats and right now the best way to kind of do that is identifying them via their field counts with the RLP encoding and and/or you you know with the IP one city thought or one five five we did some bit talking to get the chain ID into the vlm another signature so these have kind of caused problems about how do we introduce new

changes to trance actually how do we have different kinds of transactions and so this VIP and like yet P 232 we want to create some sort of like our LT tuple where we have an identifier as the first element that says what this transaction is going to be and how to finish decoding the second element of the tuple which would just be some our LT bytes and so this way we can introduce like new types of transactions and then pool can understand differently and we can also try and make it a little more vote for both like in the sense of the chain ID so this isn't we're not doing the sort of bit packing I think this is what let's some complexity and the client implementations so that's the like really sure it's a description of the

type transaction envelope just two quick points on the backward compatibility for legacy transactions I don't think this is going to make a huge difference in the near term for how cooling interacts with the theorem Network because what we can do is we can create a special case for the first transaction pipe and we can say that the way that you hash this transaction type is equivalent to the way that we're currently hashing them and so this way we can just have the client implementations kind of transform these legacy transactions that might come over RPC and transform them into the new transaction format and the hashing the signing will be equivalent so the thought won't be don't be really a problem okay I think that that's

actually the main backwards compatibility answers note on a couple proposed transaction types that have I've seen over the last years to wrap this up is the IP 1559 is going to have a new transaction pipe and over time this transaction type will eventually overtake the legacy format and there's been a little bit of desire to break 15:59 up into a little bit more manageable piece because it's already such a major change as we can put some of this into 27:18 where we're confident that the transaction pipe system is working and i think that will make 1559 a little bit more digest cool there's also 2711 which is sponsored transactions which actually is the thing that preempted this type

transaction envelope and this is going to let people with no ether potentially how their trans actually included on chain by being paid for a sender 27:33 and other ones for this transaction type of transaction packages this will allow some sort of batching format so that we can view kind of like EFC 20 approving transfer treatment transfer transactions atomically account abstraction yet 286 transaction post dated 22 42 temporal replay protection 1681 and then obviously yet be 155 simple replay attack protection so these are all things that i think that transaction pipe format can help make a lot more verbose and easy easier for

clients to implement and more digestible for people to for people to understand other network so that's why summary I don't know if anyone has any feedback already from that yeah any questions or feedback anybody you know I like 2718 okay Martin yeah sorry so I just want to eat so if you wrap some wrap transaction but don't grease I knit you have wrapped it and the data that you wrap is designed data what's to prevent someone else from wrapping it differently from someone wrapping the transaction differently yes could you

elaborate like what what are you going people are wrapping you know maybe I should just comment on on something studs and that's the questions there thanks so short on time okay sounds good anybody else and there is a theory of magicians link in the EIP and the link that we have in the agenda or rather click the link in the agenda and there will be an Egyptians link in the EIP page okay we have a couple minutes left or actually more like six minutes let's go over last times notes I think we have them this 89 yeah let me get to those

real quick I forgot to put the link so this would be meeting number 89 s minutes so two decisions made from last time eighty nine point one is for the testing static call client James will follow up with Martin and different teams over the next week and that can be discussed in the next court have caller on getter and then maybe a decision to include it in yellow will be made and this is for the testing static call and client okay this was the one that was in yellow but we hadn't decided if it's going in Berlin is that correct James yeah and I don't know what progress has been I I was on vacation last week and don't have any update okay we can move

that one we can move that one either to next call or beyond July since we are taking a break from Berlin's stuff potentially actually we should decide that before we go forward really we can still leave it on the next call okay you'll just follow up on that that sounds good also just before we go forward with the decisions made section what do people think about taking July off while we kind of figure things out for BER like to take Burton July off to not do anything in Berlin related what do people think I think it's important all our work can open the theorem is

pretty much done okay same for base you I mean we don't feel that strongly either way yep and it sounds like during this Martin is gonna do a little bit of coordinating to get more people involved in testing or fuzzing or things of that nature right Martin and so this would be a good time for that as well that's that's probably a good idea okay well we can put that on hold too if you want it's up to you know the people doing it I think the good reason to do it is as part of signaling that it's an important

issue it's been so that as part of hard for coordination or network upgrade role it's really difficult it's difficult to be in the in between the core devs and the community on some of these things but I feel really strongly that it's important that we give a buffer and a space for the client devs to do the work they need to do and I I think a lot of the community would care more if they were more aware but it's hard to even just know all of these things so it's it's I don't think it's a something that is done intentionally to not pay

attention to these kind of things like the natural order was for this for the system to go into this one client dominant II which is pretty natural yeah so if I think you know in order to get this message across obviously it's not enough to just talk about it once what we'll have to do unfortunately or maybe I don't know if this has to be discussed many many times with the different sets of people I think this is how you're gonna spread this around just talking about it one somewhere most people not gonna see it this issue needs to be carried along and discussed in all sorts of places and all sorts of ways with different sets of people so that people understand it a bit more great then let's take July off

from Berlin's stuff regardless of if people are done or not that can't include testing since Martin's going on vacation at some point and everyone's kind of really tired of that stuff cuz we've been going really hard at it and we'll focus on the stuff that Alexi brought up in the next meeting as well and a few other like maybe the 1559 call can be brought up in the next meeting too but otherwise we won't make the next call to be fee from like forth technical stuff we'll kind of talk about the next steps for solving some of the things Alexi brought up yeah and I'll focus my efforts on client diversity okay there

were hard for coordination or network upgrade role having that be that short term focus okay sounds good EW our this is a decision 89 point to a I P 2565 to be moved to EFI officially that happened I don't know if the EIP is updated will have James if you could look at that from eighty nine point two Alex s and team to look into another security audit for the eath 2.0 deposit proxy contract that utilizes the IP 2537 we haven't looked into oh wait this is I thought this was the bounty security audit wise I need to catch up with the people doing

that deposit proxy contract like all of them and see if there's an audit unless does anyone know about there being an audit for that contract yet a security audit planned okay yeah that was mine so I'll pick that one up for sure and that actions required Kelly will share the test vectors with Thomas and others on getter I believe I saw Kelly do that am I correct we couldn't find them I don't know okay we can work with Kelly to get that going then so that's last time for next time we don't really have any as far as the

note-taker for decisions made we're going to take July off from Berlin and as much time as people need to take off from the testing role and the you know testing actions I suppose for Berlin using the yellow tests net another decision is to continue the discussion of item one from the agenda for for meeting number ninety and we'll continue that at at a theory of magicians forum and also at a getter chat and any other venue that we feel appropriate so I'd say those are the only two decisions

made or action items or decisions made depending on how you want to view it so does there anything else that I'm leaving off if people are upset about it they can tag me just throwing yourself out there James okay that sounds good let's end this then we'll meet again in two weeks on July 10th at 1400 UTC thanks so much for the conversation everyone I think this was really good and have a good day thank you guys [Music]

[Music] [Music] [Music] [Music] [Music]

[Applause] [Music] [Music]

[Music] [Music]