All Core Devs - Testing #96 (CL Breakout)

Transcript

Justin Traglia: Hello again. Just one minute, and then we'll get started. Alright, hello, everyone. The first item on the agenda right now is The progressive container deserialization implementation. Whatever. I'm curious to hear what all clients are doing, and if you plan on changing anything, and any concerns you might have. Who would like to get started on this? Or, like, start talking.

Enrico Del Fante: Again, if you want, then…

Justin Traglia: Yep, go for it.

Enrico Del Fante: Everyone can. So my… my current, ideas around that is… I feel like we are moving back to having hard limits closer to the SSD layer. Which means that, to me, sounds like those fields that now really wants to have a limit, that on the SSC type potentially could have been unbounded, but… so the limit becomes kind of an optional thing, but when you decided that a limit must be there, then at SSV level… at SSD level, we want the data structure to be invalid, so we will be back to the pre-progressive list situation. There are few, few exceptions to that.

In some cases, we keep the optionality, but now… most of the time, we still want to have a limit. So what will be the difference is always That when we want to change that limit in the future fork, that will not be… breaking any, any, any prover, and no big changes on the SSV. Layer. layout. So, to me, from… from… if we want to interpret this… the situation this way, it means that on the reference test side, we need to go backward. And, even potentially remove some of these checks from the state transition function, because those… those… those limits are… no more visible from the satisfaction function, there will be just SSD validation thing.

So, it means that all the SSV tests need to take into account this. Otherwise, if we keep those, those checks in the function, we need to have a reference test that breaks the definition of the SSD, puts something which is invalid from the SSV perspective in the definition, so everything becomes optional, and then you feed those, the structure into the function, and you get to the the… the check… So, everything will become a little bit messy, so… I believe that should be the direction, but I don't know how our client feels about this.

Justin Traglia: I can say one or two things about this. So we did alter the SSC static tests. to remove, tests, which had more elements than the soft limit for progressive containers. But we did leave the state transition function tests Which had, like, 1 plus the limit, or 1 over the limit. Oh. It's… it is a bit awkward to handle, like you're saying. Like,

Enrico Del Fante: Yeah, for a tech, it would be very awkward, because it means that we can't use what is supposed to be used in the schema, the definition of the structure, we need to have There's a structures that are not in production. Data structure when we run state transition function for… to match those tests.

Justin Traglia: I think you just have to say if… If deserialization fails, and the post state for the test was null. Treat the test as a success. Or something like that. For the tests that have, list over the limit.

Enrico Del Fante: Yeah, it means that for tech, the test won't even run, because whenever we try to get the SSV from the test, this… Even if the materialization of that data structure fails, so we don't even run the, say, Transaction function at all.

Justin Traglia: Yeah. I mean… Yeah. How are other clients planning on handling this, or have handled this? I know, Nico and Casey have worked on this some.

Nico Flaig: Yeah, on the spec test side, we have a separate, or we added a handler, which basically throws an error, so this case is handled. So we fail the invalid tests that test the limit.

Justin Traglia: Could you clarify what you mean, like, they're currently failing those tests? And you have to, like.

Nico Flaig: Yes, they are expected to fail, right? It's like the invalid request limit tests.

Justin Traglia: Yep.

Nico Flaig: So, basically, we just updated our test harness. It was not a lot of changes.

Justin Traglia: Okay, cool.

Nico Flaig: But basically… Yeah, so it just failed… Sorry, I trust failed…

Enrico Del Fante: Failing, yeah. Yeah, yeah.

Nico Flaig: Sorry, I mean failed them correctly, I guess. Got it. Right, right, Tom.

Justin Traglia: Enrico, is that something that Taiku could do, or is it just something you don't really want to do?

Enrico Del Fante: Yeah, I mean, we can do that, Yeah, yeah, I believe we can, we can introduce, I believe, generally, those things could become a little bit, less… Trustworthy. Hmm. Because in those cases, whatever… if you have something different, a problem that is completely different, you're not running the say Transistorian function at all, and you can just have problems in supporting that data structure for some reason, and the failure will be just… We failed in loading the SSD. from… from… Perform the test, and…

Justin Traglia: is…

Enrico Del Fante: Like, it's good, but… It's… It's less,

Justin Traglia: Is Takers, SSE library is sophisticated enough to print out why exactly it failed to serialization? Could it say that, like. The number of withdrawals was over the limit. And then look for that specific error condition.

Enrico Del Fante: Yeah, but then we need to have a specific, checks on the… On the test. So we have a general test executor there, and then for each particular test, you need to understand how this test is supposed to fail, and yeah, I have to check… I have to check, but I believe it could become a little bit bad-looking.

Justin Traglia: Yeah. I agree, it's a bit…

Enrico Del Fante: Kind of a hard-coding test name, saying, I'm su… I suppose this to fail.

Justin Traglia: Yeah, it's not great. I mean, if you want a general rule, you can say, like I said before, like. If SSE deserialization fails, and the post state is null. treated as a… I'm sorry, as a success.

Enrico Del Fante: I mean, the post state from the test, because you're expecting the Transaction function to not run, and you've got no… Yeah, yeah, it could be a general rule, for sure. It could catch, maybe it's good enough. Not ideal.

Justin Traglia: Yeah, it's not ideal. And I was thinking about whether or not the specs could… Define the limits and have the SSV deserialization. Bail and des… sorry. Have the soft limits be defined. Like, in the types… And, fail when deserializing as well, but I think that might not be great either. So I'm unsure if the specs can do the same thing that clients are wanting to do.

Enrico Del Fante: If we want… to be honest, if we want to go in that direction, I feel like this optionality should be introduced in the in the, progressive list library, Python library as well. So, you have this optional limit there, and also in the reference test code, you have this… optional limit going there, so you don't even consider them into gossip rules anywhere. We are back into non-progressive era, essentially.

Justin Traglia: Well, it's not non-progressive error. I mean, the generalized indices would still remain constant.

Enrico Del Fante: I mean, but from the limit perspective. It becomes something that is handled by the SSD. And we don't care about it in those cases, in the gossip rules, and even in state transition, I believe.

Justin Traglia: Yeah. I'm not super… sorry, I'm not opposed to going that route. I would just need to think about it more. Is there anyone that's…

Enrico Del Fante: And Eitan probably should be in this call.

Justin Traglia: Yeah, I would want him to be included in the decision as well. I would… I would like to hear from other clients on if they've made any changes regarding These limit checks, or if they plan on making changes, or anything. Okay, I know Lodestar has already done this, I know… Prism has a, a PR open, Do you know what the state of that PR is? Anyone? Otis or James?

Potuz: It seems very simple. I do agree with Enrico that this is just a bad thing to have limits back. I mean, why in the hell are we going with progressive if we're gonna hard-code those limits anyways on our machine library? But that seems… it's a simple PR to… it seems simple to… to merge.

james: 8. I think, casey said that he needed to update some more spec tests or something.

Potuz: Yeah, there are some other structures. I think Justin left that in the review. There were some more structures that we needed to add the limits, but these limits are hard-coded there. It's simple. It feels like a bug waiting to happen whenever we change those limits. And it's going to be a consensus bug just because of a hardcore number that we're going to forget to update.

Justin Traglia: I mean, it would be a consensus bug. If you forgot to update the limit, and a gossip checker in the state transition.

Potuz: No, it's in the unmarshalling, right? So this is not… so Casey's not touching the stateless function, we're just… Enforcing those limits at unmarshalling. So, the problem is that if we ever forget about these limits, and the limits change on the state transition function, and… which is don't change on the R-Marshalling this hard-coded number that is there, then pressing suddenly is going to be treated as invalid some blocks that were valid.

Enrico Del Fante: But I hope that SSD reference test at this point should.

Potuz: Yeah, I understand, but again, I mean, all of these things point towards the failure, which is we wanted to have this, as I said, without the limits, and now we're adding them back by hand.

Justin Traglia: I… I mean, I get what you're saying. I thought the primary motivation was, like, Not breaking contracts every upgrade. Like, the generalized indices thing.

Enrico Del Fante: I feel like if we introduce those limits back into the SSD, I believe we should be in the… should be getting the good thing from both worlds, which I believe it should be the right design. It probably is just… it's a little bit late. That's my feeling. I mean, I feel okay to have SSV libraries that accept optional limits with the progressive list structure, which is the right thing.

Justin Traglia: I also feel the same way.

Potuz: Yeah, it sounds to me to also be the right solution, and then we can just… Pass those optionals in the spec. And then our library takes them, instead of we hardcoding them.

Enrico Del Fante: Okay. So, there is just one thing, They come up later, yeah, this morning, which is the deposit in the Beacon block for gloss should be zeroed Now, zero land, and this is only enforced at gossip. rules plus the transition function, so I believe this needs to be updated as well with a zero Live. Correct?

Justin Traglia: Possibly. I know what you're talking about. I can look into it.

Enrico Del Fante: Yeah, it's been a discussion happening at the very last. moment in the same Discord thread. I believe it's, it's another… Ugly situation in which you could start deserializing and considering good, block containing… glass, Beacon block containing deposits, and then… Too late, consider it invalid because it has deposits. And since it's a progressive list, you can stuff things in there.

Justin Traglia: Nico, to clarify, you say that you can't deserialize a block of zero deposits? Or do you mean, like, no?

Nico Flaig: That is the legacy deposit.

Enrico Del Fante: Yeah, it means that currently we… there is no, soft limit currently applied to that field in the gloss Beacon block. We are addressing all the fields, except… That one So I believe it need… it needs to be included in the… A set of fields we are focusing on, with a zero limit.

Justin Traglia: Yeah, I see. I mean, I'm just… init or a clarification? Are you saying you can't deserialize with zero? I mean, isn't it expected to.

Enrico Del Fante: We can, because there is no soft limit defined in that field currently. So, it's a full progressive list with an unbounded list. So we can decode it now. Even with your latest change addressing the… the limits for the other 12 fields, I believe.

Justin Traglia: Yeah, sorry, I feel like I'm not getting my point across. It's alright. Any other… comments on this, Mike? the whole SSD thing.

Nico Flaig: Yeah, how do we deal with the deposit requests? Like, the problem is you can limit all of these fields, but in the case of an attack or a DOS, the deposit request keeps it basically unbounded.

Enrico Del Fante: How are we back to the 10MB limit there.

Nico Flaig: Yeah, that's the problem. And it's mostly… maybe it's an issue how our queue works and the network processor, but Yeah, I feel like these limits don't buy us anything in case of an attack.

Enrico Del Fante: We had a bite budget in some of other pending blocks. But the point is, we haven't, Added this byte budget everywhere. like… Few… Future blocks, like with upcoming SLOT, are not covered by that. But, blocks… with a known… parent, root… Pass that… Mmm, but… Yeah. But is this anyway a problem for…

Payload… transactions in payload, in which we don't have Anyway, a limit there, so… We cover The request, but transaction… Still relying on… 10MB limit.

Nico Flaig: Yeah, I think it's already a problem today, right? Especially brick laws.

Saulius Grigaitis: the one thing for this 10MB limit, if I remember correctly. the discussions we had, I think it was a few years ago, The problem is that… This 10MB limit works, because… No one intentionally I mean it. the blocks are not going to be that large. I mean, the key problem is that everyone has a different understanding what is the exact limit of this 10MB. So, some, some clients, take, like. the whole size, including various metadata or headers, or whatever there is, and some measures without the headers or metadata, so… so we cannot rely on this 10MB limit that we have.

Because it seems it's possible to craft, Around the 10MB blocker. which some clients will accept, because it will be within the limit, and some clients will say that it's outside of the limit. So, this was not fully sold back then when we had this discussion. maybe somebody who remembers can… can extend what I said.

Daniel Knopik: I don't remember the discussion, I think this was before my time, but I'm pretty sure the 10MB is just… the P2P message, like… And, like, with snappy compression, or something? I think… it is news to me that is apparently ambiguous, because… Yeah, so I'm also curious if there's anyone who has a different interpretation from that.

Saulius Grigaitis: So, I remember that there were multiple clients that had different interpretation of what exactly is this megabytes, and I mean. The interpretation is not just discussion, but actual, the code, measures differently in different clients.

Pawan Dhananjay: I think we clarified it to be the maximum uncompressed snap, uncompressed size. Before, like, there was some confusion if it was, like, the compressed snappy limit or the uncompressed one, and I think we clarified it at that time to be the uncompressed. tailored size.

Enrico Del Fante: Yeah, I believe, there has been effort to unify most of the client on the… to be on the same page, to avoid… network splits, I remember that.

Saulius Grigaitis: But was it actually checked and, you know, like… Are we 100% sure that all the clients are aligned?

Pawan Dhananjay: I think it was a cut.

Enrico Del Fante: consensus behind. Yeah, sorry, but I believe all the clients did… did some… spent some time to… to fix that.

Pawan Dhananjay: Yeah, I think we fixed that as well, because it was, like, a consensus-backed PR as well at that point.

Saulius Grigaitis: Okay.

Justin Traglia: Yep. Okay, anything else on… SOC, deserialization, or, gossip… stuff. Otherwise, we'll move on to POTUS's topic of… Enquivocations and fork choice.

Enrico Del Fante: Just one final wrap-up. So, we decide to cover everything with a limit, with the expect of the execution requests and transactions. Is this… the overall… Situation, and we rely on gas limits and… And 10 megabytes. We're limiting.

Justin Traglia: I think so. would someone else be willing to open a PR for this? Because it's not exactly clear to me what will be changed.

Enrico Del Fante: Yeah, I believe we need to, anyway, have a discussion with Eitan, because the underlining library should… Handle that, so… Maybe we can have a subsequent discussion with him as well?

Justin Traglia: Yep, I can do that. And I'm sorry, I was thinking of the… The other thing that you said about, like, the deposit.

Enrico Del Fante: Checking the deposit.

Justin Traglia: it's…

Enrico Del Fante: Yeah, yeah, okay, maybe I can… I can open a BR for… for that one.

Justin Traglia: Okay, thank you. Yes, regarding SSC deserialization, I'll… I'll chat with Eitan, and tama, C. Who's, working on the SSC Specs library. Okay, let's move on to the next agenda item. Otis, I see that you're here. You gonna leave this one?

Potuz: It's a very simple observation. We count locations informed choice, when we need to apply or not proposer boost. And… there is a problem with fork choice itself, in that fork choice is aware of blocks only when you call on block, which only happens after the block has passed validation. So this was already present when we were worried about DHX, because we won't count for, GA, for example, so Fortress needs to be aware of certain… of GA, votes for C, for example. And, but we want to make sure We need to act on this information that… where the block itself is not in for choice, because clients do not add the block to fork choice until it's passed by Validation.

And equivocations is a nice one, because we don't put equivocating blocks in port choice. We only put them when we're forced to, like, when we do an RPC request, and then we add… we are forced to import a block, because we're syncing a branch that has equated. But over gossip, we would never do that. So, we need to make a decision as to whether, act or not with proposer boost, and this is a decision over the gossip, not over the RPC path. And we never import blocks that are cubicated. So… The problem is that it's… it's an awkward position on the Python spec, but… and this is the issue of being aware of How people make decisions of a purchase, and then how they actually

specified in the specs. Prism doesn't really follow the spec in those things that we know that… what the… what the specification was meant to be. And when Francisco coded this, I didn't really pay attention, because we… I didn't care, right? I was going to implement the thing that I know was right, and that was… not to care about if the equocating block was valid or not. As soon as we see a signature, and we know that there is an evocation, we're just signaling for choice that there is an evocation. And we even keep track of the timeliness of these equations, because that is actually important, to keep track of, and the order of equations that we get is also important if we want to protect the builder.

Anyways, so the long story short, Prism counts non-valid equations in fork choice, and other clients only count valid equations in fork choice because that's what the spec does. And I think… I don't care who's gonna rewrite the spec, I know that this is an awkward situation, and I… I am happy to have the things that are off-chain, things that are not in the Beacon state, that are not part of the state transition function, particular things that are in Fortress, I'm happy to, like, have them written down somewhere, and not in a Python executable spec, because I think these things leak into clients, as this scenario. So I'm happy to just write a comment. But… and I don't know how this is going to be solved in the spec, but I think we should agree that we should count invalid applications as implications for chips.

So I don't want to modify Prism on this.

Enrico Del Fante: I also believe it will be the easiest way for… Teku? Currently we don't. have… Revocating detection in that situation. even in fork choice, we don't have the code looking into fork choice, searching for equivocation. We don't even have that signaling from gossip that there is an equivocation, so we are a little bit off on that. Oh, and… And the timeliness of that will be even another thing.

Potuz: Wait, but aren't you… I mean, I think there are four-choice spec tests over… for this.

Enrico Del Fante: Yay.

Potuz: I didn't even want to add this, because I thought it was just something.

Enrico Del Fante: Compliance? Compliance?

Potuz: No, no, no. Actually, just the usual spec test. We are not passing compliant tests yet. I think just glow aspect has covered already this. It should apply proposer-boost tests.

Enrico Del Fante: Yeah, I need to check, because this happens… Multiple places, but…

Potuz: I think I was forced to add this just to pass spectests. Because I didn't want to add this either.

Enrico Del Fante: Yum. Actually, I need to refresh my mind over what we have. But I believe there are some additional things happening for compliance tests, I remember. Forces us… forces us to consider additional checks. Wow, yeah.

Justin Traglia: Potus, would you be willing to open a PR for this? And I can help you, like, if there's any… Test changes, or, the little details that you don't really care about. I think it would be good for you to be the one to open this, at least.

Potuz: I'll give it a thought. I'm not really sure how… I mean, probably I'll… I'd be kind of radical on these things, but yeah, I can do that. I thought Francesco might want to take it, but he's not here.

Justin Traglia: Okay.

Potuz: Yeah, I can take a look at it.

Justin Traglia: Okay, and if… You don't want to, or whatever, maybe, Nico, you could. I'm just hesitant to do it myself, because I'm really not an expert with fork choice.

Potuz: I can do it, but the thing is, most probably my preference would be to have it underspecified. And have an external thing that just counts the publications. Instead of having this actually Python executable. I think we're over-specifying things, both on P2P and on poor choice. Things that are not core state transition functions, my personal opinion is we should underspecify and leave clients to do whatever they want.

Justin Traglia: Okay.

Nico Flaig: Related to that, does PRISM look at more than the two first proposals you observe? Are you muted, Paul?

Potuz: I'm muted. Yeah, so I think we validate the signature of any block that comes in. Probably this is a no-wop on… because we only count timely, aggregations, so if we already have one, then another one that comes also is going to be. What I don't remember is if we keep the map of… of BlockFruits, in fork choice that I co-located. I… this I don't actually remember. Because for should… should apply proposer boost requires you to… to know But if you only have two, I think that's good enough, because, Because no matter which order you get, you're guaranteed to have, one of… I mean, if there is a timely one, you're guaranteed to have one of the bad blocks already in a timely

In the timely, evocation list. So I think what Lowestar does, if you have 3 blocks, I think you're guaranteed to be right. Like, one valid, and then the other two. But, but I have to think about… I have to look at… I have to look it up. What I do know is that Prism would just validate the signature of any provocation that comes.

Nico Flaig: Yeah, that was my concern, that you could DOS the node by forcing it to verify signatures.

Potuz: Well, but the thing is that you drop… so… yeah, so… Actually, it might be the case, although for verifying signatures. Yeah, so I actually don't know what, if P2P would just drop them early. I don't know, actually, but if a proposal prepares a thousand blocks and signs them differently, and it sends it to us. It might even doss us, I'm not sure.

Pawan Dhananjay: It would, because I think the message IDF would be different, but, like.

Potuz: Yeah, yeah, so… so overly P2P, the hash would be different, but I don't remember if… and I think we… I think we verified the signature. I think we do not drop… we drop them over gossip, perhaps? if we've seen… no, we don't. We… We don't drop blocks over gossip if they come from the right professor and the right slot, because we prepared the slashing event. So we never drop them. I think we verify the signature, if they're valid, we just broadcast the slashing. I can add code to… to stop verifying after the third block.

Justin Traglia: Okay, anything else we want to discuss? I've decided not to talk about deprecating NPLEX. I don't think that… I don't think the people that want to discuss it are here. Casey, Paul, Jasik. If you guys want to, we can talk about it if not.

Potuz: Is there anyone that doesn't want to deprecate it?

jxs: Yeah, I was curious about the same.

Justin Traglia: I don't… I mean, I think most people are supportive of it. I… there was a comment on the PR recently saying that, like. I forget what it said, I'll have to find it. That's saying that we don't… we shouldn't rush into this.

Pawan Dhananjay: I think last time we mentioned, Deku didn't have, like, Yamix or Quick, one of those, but now it seems like they do have it, so…

Enrico Del Fante: We have Quake, but… Yeah, it's… it's working in production, we have a couple of things to fix. There as well, because… Orientation is relatively young. But I believe we are on the good track to be… fully… Was it working?

Pawan Dhananjay: So, like, in your connection establishment, you still, prioritize TCP and Plex over WIC?

Enrico Del Fante: No, I don't believe so. In terms of prioritization, I don't remember when booths have available, if we'd go quick first, maybe?

Pawan Dhananjay: The prioritization is… like, quick, then, like, probably MPlex is not being used at all at this.

Enrico Del Fante: Yeah, well, it's not currently a node that's… a default-configured node. It's getting, like, 70%, 75% quick. Connections, and, like, 20%-ish, the address in TCP currently in Daegu. So it's getting more quick than TCP.

Justin Traglia: Enrico Luca was the one that left a comment on the PR.

Enrico Del Fante: Yeah, yeah, he said that.

Justin Traglia: But some nodes were having issues with QUIC, and were forced to disable it, or something.

Enrico Del Fante: Yeah, we haven't… we have had the issues, and some… some memory leaks on the quick side. We are fixing things, currently on the week, so I would, wait a little bit to… Have the implementation fully tested and with no issues for a while, to say… to signal less… to signal, let's depregate it. That's my current… View.

Justin Traglia: Is this something that we'd be interested in doing before the… Amsterdam may not upgrade. Or after. To be clear, it is deprecating Inplex. And that was a question for all clients.

Enrico Del Fante: Yeah, I mean, I mean, I believe we are on the… on the… On the right path to… Together that way.

Justin Traglia: There's also some, I don't know how to phrase it, but you don't want to be the last one. Which depends on, MPlex, right? Like, if you have… if there are some nodes, which user clients, which depend on NPLEX, they might… Be unable to speak without everyone else. Praising it right. Does it make sense to do this out of hard fork? Deprecating impacts? Or, I mean, maybe it just doesn't really matter.

jxs: I can come in, but I'm surprised. We've been this… Again. It's been years, and I think it's been months, Since… This situation… we tackle… appeared, cause… We are exposed. MPLEX is being deprecated as a protocol. the Rust version is being unmaintained? Hmm… as Polytech vectors. we've been… Trying to deprecate it for so long, and it keeps dragging and dragging and dragging. Like, I was surprised by this, I thought Teko was already supporting it, and… we could finally merge this PR. We've been… we've merged this PR before, we reverted it, and now we… we merged it again. I don't think it makes sense to revert again. And keep… keep doing this. We don't want to speak NPLEX.

And Teku, I've told this many times, Teku… enable Yamooks, would… We speak Yamooks.

Enrico Del Fante: We don't.

jxs: No, I mean, Lighthouse speaks Yamooks. Teku has support via JVM with P2P of Yamooks. You went through the harder route to try to enable.

Enrico Del Fante: It's not production-ready, Yamos.

jxs: Yeah, that's what I've been hearing for 3 years.

Justin Traglia: Yeah. Enrico, do you think that TyQu has bandwidth to work on this over the next, like, month or two at all?

Enrico Del Fante: It's actively under implementation, and we are refining our implementation to get it better. So, I don't see, currently, a blocker to say no to deprecate buying. gloss mainnet. I believe it's… it's a good timing. But if you ask currently, we need to do some additional refinement. even if the current implementation is fully working, I disabled TCP on my personal node.

Justin Traglia: Okay. so we're currently at time. Any last comments on this? Or anything else. Right. Thank you, everyone. Talk to you guys on Thursday.

Potuz: Thank you, guys.

Daniel Knopik: Thank you, bye.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.