P2P Networking Call #5

Transcript

Csaba Kiraly: Hi, Kamil.

Kamil Salakhiev: Yeah, all good. Just keep ping everyone.

Csaba Kiraly: Hi, Anton.

Anton Nashatyrev: Hey guys.

Csaba Kiraly: Long time, host.

Anton Nashatyrev: Yeah. Hey.

Kamil Salakhiev: Hey Raul, hey, Marco.

Marco Munizaga: Hello.

Kamil Salakhiev: Yeah, hey, hey. Yeah, maybe let's wait another minute… Okay. Yeah, I think let's just get started, and I think more people will join. Yeah, Today, it's, yeah, it's a pretty packed meeting. Yeah, I think we have, like, four…

presentation topics to discuss. Yeah, let's start maybe with client updates, then we have a quick presentation from Marco about topic stream extension. Then Sukun also wanted to bring up, attestation bundling, and then Anton will present JVM. simulator, and finally Shah will present chain score. So, yeah, I will try to manage not to make it too long. Anyway, yeah, let's start with client updates, maybe. Daniel or Joa, anything on the Lighthouse side?

Daniel Knopik: Yeah, so… We, deployed Gloas support for partial data columns to DevNet 7, yeah. I haven't looked into it too much, but it worked, like, it's running, it's… Definitely still online. So… Optimistic that it's, Working well, but I haven't looked into the details yet, so… Yeah.

Kamil Salakhiev: Okay, nice. I think later today, Cabo actually will be able to show, like, percentage of notes supporting, Partial messages extension, so, yeah, that's great. Just one quick thing about, yeah, Rust-P2P. I had a PR opened to the repo, it's a really small one, if someone could take a look, it would be great. It just enables, hybrid key exchange for the… Tls… whenever it's either, I don't know, TCP, basic connection, quick transport. Yeah, basically just enables post-quantum key exchange. Go-libp2p already supports that, so it would be great to kind of have the same behavior there. Cool. If nothing much, from… at Houseide. Yeah, maybe with Prysm, Aarsh, or anyone,

Aarsh Shah: Yeah, sure, I can go. So, we have finished implementing support Because for Globus. And we've finished an internal review. The next step is to, yeah, do some rebasing on the Glamsterdam DevNet 7 work, and then we'll deploy it on the Glamsterdam DevNet 7 branch as well. And the other update is that We're starting a new workstream to revamp peer scoring in Prysm. Mainly for the purposes of instrumentation and observability, so it's easy to understand why we are disconnecting from certain peers, or why we are connecting to certain peers. how long we remain connected to certain peers, what are the major cause of disconnections? Basically, yeah, like, because right now, it's a bit hard to instrument and observe that. And, you know, based on that feedback, we can make some good changes to our peer scoring algorithm if…

If the instrumentation turns, you know, turns up some surprises. So yeah, just, I'll share more. I'm gonna write a design doc on the changes we plan to make, and how it currently works, etc. I'll just share it on the Telegram channel when I'm done with it. Just wanted to tell you all about it.

Kamil Salakhiev: Good, nice. Yeah, let's proceed maybe with GEF. Yeah, Bosu, would you like to bring something up about sparse blog posts?

Bosul Mun: Aye. So, from gas side, after Spurs Book Pool is merged, I… I was doing some… Curtis is testing with other mind. We found a… I found a bug on gas side, so I fixed it, and also fixing some minor bugs on the implementation side. And… on the ACDT. on this Monday, I pres… I suggested to make the engine APIs as mandatory on GlamstraGlam 10.8. There were some… Opinions about… Piggybacking the custody field to purchase update from Daniel and Lighthouse team. So, it would be nice if you can talk about this topic more, if we have time after all the presentation.

Cardana, and yes, and also I'm working on the… Caic listener integration on… P2P package. I will link the spec here. This is still dropped. Might be interesting. Something.

Kamil Salakhiev: Nice, nice.

Bosul Mun: Thank you.

Kamil Salakhiev: Yeah, Caba, would you like to add anything?

Csaba Kiraly: Yeah, maybe I cannot sit on the, from the guest side, so… Yeah, Get Specific, the… detected pillars, so… detecting good quality pills, where we have different, quality metrics, and, the working mode is that Along each metric, we are protecting a few pills. That's… the first phase is merged. So gas is now, not dropping the best pills, along two metrics. One is PL Switch. Send the most… included transactions, and the other one is… is the builder-sweets and most finalized transactions. What these counters are… these are not… so these are, Fading away, so exponentially averaged over the window. counters, So… So yeah, anyway.

Pills, which are… which are good in… along these two metrics, are kept… we keep, at the moment, these percentages low, so we only keep a few pills, and we are keeping the churning, which we had, previously for the last… that's… this emerged in the… in the… Lastly, for the next one, depending on what is prepared, is another protection, which is… which goes in another dimension, which is peels, which are, responding, transaction requests fast and reliable. A few peels of those will be… will be protected. So that's… that's a latency-based optimization. Again, it would be a 10%. Cabbed. At the beginning. So we are proceeding with… with this. And, yeah, you can look at it as the PBS calling, the P2P version of PBS calling, or DAS version of PBS calling, actually.

So… It's… it's, I guess, internal policy. The other thing, to mention is this V5 migration, so… Nothing changed in gas, but I was doing a Kurtosis test, with, all the EL clients. Trying to force them into this V5-only mode, that… Two clients didn't want that, so they still, go on this before as well. But that's an easy change, and then, yeah, I had to fix a few things in Kurtosis. Anyway, the… the essence is that, apart from this, everyone is working, so… so we can… kind of start thinking of turning off this V4, and yeah, that's it.

Kamil Salakhiev: Cool, nice. Thank you, Caba. Yeah, maybe proceed with NetherMind, FOCILL. Would you like to share any updates?

FOCILL (Nethermind): hello. So, in previous release, we had only V4, active, on mainnet. And we plan to activate both on, mainnet and just V5 on testnets, and both will be default settings for discovery. Yeah, we are working a bit on topics. I mean, topic discovery, And, working on Bot Node, that would allow to test, Discovery, topic-based discovery, and maybe even text pool in some isolated environment. with, bigger count of nulls. Yeah.

Kamil Salakhiev: Cool, nice, nice. Okay, yeah, any other clients that I missed? Okay, guess not. So yeah, let's proceed maybe with, Some presentations, so Marco, you can go first.

Marco Munizaga: Alright, you guys see my screen okay?

Kamil Salakhiev: Yes.

Marco Munizaga: Okay, cool. So, let me start my timer. Cool, so this is an extension to Gossip Sub that helps remove the head-of-line blocking issues. I'll explain a bit more about that, and avoids… lets us avoid the topic name overhead. So… In our gossip that we send to peers, we have a very different set of messages that we send. We have big messages and small messages. We have the big block messages that have, like, the whole payload, and then we have, like, very small attestation and aggregate information. And all of these are put into a single stream. Even if the transport… all our transports support multiplexing, we don't have… we don't take advantage of that. Everything is just serialized on that one stream.

So what that means is if you have a small message, and it's stuck behind a big message, you have to wait for that big message to be sent out before that small message can be read by the peer. So… The ordering is useful, but really it's only useful within a topic. And we don't want… we don't really care about ordering across topics. And the control messages are very nice to have ordered. It just makes implementation simpler. And we don't want this to be, like, a massive change that everyone has to organize. And, synchronize to deploy. And while we're at it, we can also avoid sending the topic name on every message. And this is a little subtle if you're not thinking about the wire.

Details, but here's a… let me… break it down a bit. If you have the attestation, it's a very small message. But the topic name is, like, this really big thing that has, like, ETH2, whatever, whatever, subnet, whatever, and then the encoding. And this thing is a sizable part of that whole message, like, if the message is, like. I don't know, 500 bytes, and this thing is… just the topic name is about, you know, 10 bytes, so… Yeah, a big part of it. And we can, we can avoid that. So… That's what this extension does. It's about splitting messages into… the topic messages into separate streams. So each topic gets its own long-lived stream. And at the very beginning of that stream, we send a header that says, this is the topic name for this stream.

And then messages Topic messages just go on that stream. It allows interleaving of big messages and small messages. This works, obviously, with Quick. But it also works with our existing muxers on TCP. So yamux will interleave frames for multiple streams. I'm pretty sure MPLEX does this, too. Yeah, so even with all our transports, we get this benefit. This is the rough shape. We… both peers advertised this extension. And then they can open the… Topic stream protocol, which is GSTS beta. They specify what the topic name is, and then they publish the RPC messages on that topic stream. They omit the topic name in those RPC messages because they don't need it. Yeah, so again, this gives you the…

idea of what's happening on the wire. So, before, you would see on the left, we would send this topic name every time along with the data. On the right, now we could just send that topic name once, and the data is really compact. So, yeah, this solves the head-of-line blocking, reduces the overhead for topic names. There's a couple other issues it doesn't solve, right? If it's… You're gonna have the same… Messages within a topic be serialized, so we're not… Avoiding that, but that's actually kind of the intent, because it might… it's simpler to reason about a model where topics come serially. And we don't have any stream priorities here, although you could imagine adding stream priorities would go really well with this, because you might set

At the stations have really high priority, and then the payload body have a lower priority. Yeah, I think we can skip through this. So… I will post this link… I'll also post the whole this whole slide. But… Spec 729 has the topic streams working draft, the detailed specification of this change, of this gossip sub-extension. We have an implementation already in Go, and I have an interop test. that will let people make sure that their implementation works. It's built on Shadow, so it's agnostic to the actual, Implementation. And the way it works is… It's wolves. Concurrently publish… well, it will publish a 1MB message on one topic, and then a small message on another top. And it'll look at the timestamps in the log, and it fails if the small message gets blocked behind the big message.

Yeah, so this… this test passes on the Go implementation and then fails on without top extremes. So… like, taking a step back, basically, what we're trying to do is everything right now has to go through a single queue. Topic streams, let's… Everyone get their own cue. there's some future work that could be interesting, so we've talked a little bit about this in the past at Gossip Sub. A message header that tells you what the message ID is that you're about to receive. This would allow the receiver to do something about it, instead of just having to wait to receive the whole message. a way for the sender to reset that topic stream to drop a message mid-flight, so you can imagine if I'm giving you, like, a really big message, and you tell me, hey, I already have that message, there's no point in me finish… finishing sending that message. I should just…

Drop that stream, and then open a new one. There's also, like, some interesting work we could do here with the receiver setting flow control to the throttle peers, right? Because, like, we might say, for this subset of my mesh peers, I want to throttle them down so they can't send me a ton of data right away, but for these other peers, you know, I… Want them to be able to send me data as soon as they get it. That's it. That's my super quick presentation on topic streams. Any questions? Sorry, I haven't looked at the chat.

Kamil Salakhiev: Yeah, let's maybe address some questions from the chat.

Marco Munizaga: Let's see… So, yeah, I guess Raul says ordering doesn't matter within a topic either. I guess it… that is an application? Dependent, so it really does depend on, like. yeah, what the application is doing there. I think it's…

Raúl Kripalani: Yeah, I am…

Marco Munizaga: Specifically.

Raúl Kripalani: about Ethereum. I don't think… I can't recall any case in which it matters, but maybe there is one. Off the top of my head, I can't recall a case.

Marco Munizaga: Yeah, I don't… I don't think so. But I don't think there's a case where, like, we have too much contention within a topic. Right? Like, the topics where you get a lot of messages are… Relatively small, from the same tiers.

Raúl Kripalani: don't click. Yeah, right, the only… the only one, would be Agile Stations, but that's a different… it's a very different pattern anyway.

Marco Munizaga: Yeah.

Raúl Kripalani: Others tend to be singleton objects within a particular slot.

Marco Munizaga: Exactly. So, Sukun asks if this requires yamux-MPLEX interleaving. It does, yeah. So, without this, without the yamux MPLEX interleaving, the test would fail. But yamux and MPLEX already… well, I didn't test MPLEX, but yamux does interleaf. And then, I guess this answers Raul's question, which is… yes. So yamux has, like, a… a message, like, a max frame size that it'll use. So, like, if you send a 1MB message, it gets chunked up into I forget the exact number, but let's just say, like, 64 kilobyte chunks, and then those are written. at… In those chunks, and so if you have two streams, you'll get, like, the 64 kilobytes from the first stream, and then 64 kilobytes from the next stream.

Hopefully.

Raúl Kripalani: I think when there is congestion, that happens, but only as a result of backlogging, not as a result of, like, share, like, any… share, like, fair share, like, policy of the link or anything like that, right? I think that…

Marco Munizaga: Yeah, dude.

Raúl Kripalani: Yeah, makes sense.

Marco Munizaga: Due to the scheduling of the yamux implementation. You can imagine a yamux implementation that only takes the… from the first, is like, only takes from the first stream that has data. But you would probably want… it's better if you're yamux implementation round robins. It's the streams that can be flushed.

Raúl Kripalani: Yeah, and it doesn't… and it does… yeah, and it doesn't behave that way unless the con… unless the stream is congested. That's what I'm trying to say, right?

Marco Munizaga: Unless the stream.

Raúl Kripalani: I agree.

Marco Munizaga: tested.

Raúl Kripalani: Yeah, the window, the window for the stream is congested… congested in yamux, Samplex doesn't support it.

Marco Munizaga: I… I don't think so, and the reason I say that is because the shadow test passes my scenario, and I don't think the links are congested there.

Raúl Kripalani: Right, yeah.

Sukun Tarachandani: Correct. It depends on, basically, the code being pushed to a channel. So, both the streams are basically blocked on the push to the channel. Huh.

Marco Munizaga: So… Let's see, so which layer should potentially be aware of topic priority? I think you kind of have to plumb this through to the application, right? Because the application is the one that knows What topics should have priority over other topics. There's… Alexi, does that answer your question?

FOCILL (Nethermind): Yep, yep, kind of. I mean, we cannot inform yamux about a potential topic, like, stream priority, right? So… What could we do with the prioritized?

Marco Munizaga: Hmm. Yes, we could actually prioritize. yamux, but that would be an implementation change to yamux. There's nothing in the protocol that would prohibit that. You can imagine a priority queue of frames that need to be written out. But… I kind of feel like the effort would be better spent on having the quick implementation You support stream priorities.

Csaba Kiraly: how is the Steam polities in Quick? Is it an absolute parity, or is it some kind of…

Marco Munizaga: That's an implementation.

Csaba Kiraly: Decisional. It's not standardized, right?

Marco Munizaga: No. There's some, work that, like, outlines how you would do HTTP stream priorities, But those are just, like… As far as I know, there's no standardized way to do priorities. But it's pretty simple. You can imagine, like, a very simple version, and you just have, like. Party number, and you've… They just read in a… read the frames from a priority queue.

Csaba Kiraly: Hmm?

Kamil Salakhiev: Hmm. I guess we had one question from Casey. Just throw it in the chat.

Marco Munizaga: Could topics be as granular as a block hash within a logical topic? This would be interesting from a priority perspective, columns for the block group. I already believe will be canonical. I think I need to think about this a bit. But I think… The tricky thing will be… How you build your mesh. For a topic, if you just have, like, a super granular topic. But you could imagine, actually, with partial messages, you get this granularity. So you say. you know, I'm subscribed to this, like, data column, but I only want the data columns for this specific block route. When peers give you their group ID, you can just ignore the things that aren't the ones for that block route.

And I didn't mention this in the presentation, but this does integrate with partial messages in kind of the way you would expect. So, like, every partial messages For a topic goes into a separate stream.

Kamil Salakhiev: Okay, Raul?

Raúl Kripalani: Yeah, so… Here, so this is actually relatively easy to build today. If you extract yourself from the gossip sub-world, which is extremely rigid in the way that it does topic membership, and topic lifecycle, and subscriptions, it assumes a finite set of topics, because you need to find members, and so you're kind of, like. the model just does not, like, it's not well prepared, it's not well-suited for dynamic topic names, so… but in reality, the protocol already communicates this, so right now, we have the status message, and we have, block announcements that peers do on the consensus layer, right? So, right now there's nothing, that… there's no signal that a peer emits when they are speculatively

decide… like, when they have speculatively made a block canonical, awaiting some other data to be confirmed, right? Like, for example, in the case of columns, I believe this block has is good, but I'm just missing a few columns. As far as I remember, like, at that point, the status message would not change, right? Until you fully commit the block, because you've been able to fully validate it. So, the notion here would be, if you have a pre-status, or pre-announcement, or, like, some other small signal. that you broadcast to your peers, telling them, hey, this is the block that I am working towards, then they already have that information. So, literally, it's just, like, adding a single message just unlocks

a whole data-flow. You don't really have to think about how we're gonna, like, make the square fit into a circle, right? By making, like, you know, by bending gossips up to work this way. So it's just kind of like… would be… what I'm trying to say is, like, the… this… the data-flow is already there, it would be potentially about sending a single message, And that's it. The value of this, I can test. I don't think there are that many competing blocks, at a particular… at a particular height. So maybe it's not worth it, but yeah. Just the pattern of 1.2 blueprint.

Marco Munizaga: Cool. I don't want to use up too much time, so, feel free to ping me if you have any more questions. The draft is there. I'll post the presentation today in a second, and, yeah, the implementation's ready, so someone please make another implementation so we could test Intero.

Kamil Salakhiev: Nice, nice. Thank you, Marco. Yeah, let's proceed with Tom's presentation on JVM simulations. Quick, and bum…

Anton Nashatyrev: Yeah, yeah, I, let me share, Okay, do you see my screen?

Kamil Salakhiev: Yes.

Anton Nashatyrev: Okay, great. on. Yeah, so hey, I'm Anton, I'm a researcher at Consensus, so yeah, I'm going to talk about… my, JVM QuickBase Simulator. So… Why, why did we want, another simulator? So, yeah, my goal was, to make, mainnet-scale, simulations. So, a lot of PBS, a lot of messages. Yeah, so, Shadow is great, but, It's good… it gives us good precision, but, it cannot run this many notes and messages, for… for the scenario, like, attestation, dissemination. Beep. So, I actually have had a gossip simulator, it was scaled well, but, like, the TCP behavior was too hard to emulate correctly, so, we have,

So, a lot of issues with, like, matching real behavior with simulated behavior. Yeah, like, I, I loved, marco simulator approach, and, it inspired me to do the similar, with JVM lipid. Yeah, so, it's pretty, pretty, well-known scenario, so we have, nodes, each node connected to peer random peers, so, And, like, all the notes published M attestation-sized graphic messages, so, like, about 150 bytes. And the run completes when every node receives, every messages. Yeah, before diving into details and jump, jump straight to the final results. These are practical scale numbers from my local runs.

So, yeah, the maximum I could, I could do is having, having 8,000 nodes, which, every of which, publish, publish, Two, messages, so in total, we have 16, thousand, Messages. Yeah, it took, it took, me, it took the simulator, 20 minutes to, to complete the simulation, and it took, Like, around, 5… 500 gigabytes, of, RAM on my machine. So, yeah. I validated my simulator against Shadow, so I made several scenarios, Among, among them is, gossip dissemination with smaller, network, and… I also have a kind of large message, just simple transfer from one node to another node.

And, like, several… Various scenarios, and, it looks pretty close, so, the cumulative errors, in this, in these scenarios was around 1%. Yeah, so, a little bit… about implementation, so, we, Simulator have, network, UDP network specification, very close to what What Shadow, does, so, you specify nodes, you specify routers. You specify links between them, the links are, like, double-sided, but usually you just, specify. Specify, latency and bandwidth for both sides. Yeah, so this is implementation details. It's, like, interesting to check the… with core interfaces, like, it's, two interfaces. First, it's controllable, it's, like, any, any, any, controllable means the component can advance its own simulated time.

And the packet processor is the controllable, which means it can receive packets and produce packets. Yeah, actually, the same idea works for nodes, links, routers, and even for the full network. For example, packet processors could be a node, a direct link, and, like, the whole UDP network. Oh, like… Controllables could be, like, internal, node scheduler, and even the host simulation. Okay, so the, the… Actually, the interesting site, the interesting, Thing is how… how this was, parallelized, because, And our initial simulation needs, like, needed one global lockstep for every small event. And so, here, if you have, Like, for example, two nodes connected to a router, they have,

All of them have their own, Mmm. like, local time, local simulated time. So here, for example, router has its local simulated time 20 milliseconds, and nodes are on 0. time. And they have both, they both have, latencies in their links. So, here you may, advance, you may still advance a router. advance it 10 milliseconds forward. You may still… you may advance node, A, 60 milliseconds, and you may advance node B, 50 milliseconds. So, you may do it in parallel, actually. So, here, here, you may, parallelize the whole… the whole thing. Yeah, so this is some, integration details, so, I need… so…

under the cover, it uses, KatyQuick, Which in turn uses Quiche, quick implementation, it's Cloudflare, quick implementation. Yeah, and I had… initially had a problem with, cliche, it's, like, good from the point… it's less… it's, like, it's a good state machine, but it has… it had one problem, like, it's, It took, it took current time, internally, like, Just, like, instantiate, I don't remember, like, what's the… I think it's instant. Yeah. So, I, I needed to… I needed to modify to pass, to pass the current time, for every call, yeah. So, yeah, actually I've made changes, like, made these changes, and there was a lot of changes, and I don't think they… They're going to be accepted, upstream.

At some point. Yeah, so probably, probably, if… If we want to develop this simulator further, then we need to consider Consider, another, another quick implementation. Probably, like, pure, pure, and VM. So yeah, what I would love to do next, like, for first, make it easy to use for everyone. Second, the simulator is still non-deterministic, so, Because, Under the cover WorkShare uses some library for TLS Handshake, and it, instantiates random internally, so, you got, Different keys, and you even finally got, different, packet sizes, so… It's not, deterministic, so… but, again, if we may modify internals of Quichelle, we can use… we can reach determinism.

So, next, if the, for example, if Teku could implement this controllable, controllable interface. Then we may, run, run the simulation of, Ethereum network, with DecoNotes. We may also use a remote API, for… to make… to make, ledger sim… ledger simulations, so just to make a cluster of simulator… we also may, include, like, other, other impl… other, other B2P implementations, if they implement this API. So, that's it. Let me… thanks. Listen.

Kamil Salakhiev: That's right.

Marco Munizaga: Couple questions.

Anton Nashatyrev: Yeah.

Marco Munizaga: So… Could you explain a bit more on the latency yields parallel execution? I didn't… I don't think I quite followed. Like, where did the 60 milliseconds come from?

Anton Nashatyrev: Let me check…

Marco Munizaga: I mean, I think I understand the base… the general idea, and this is something Shadow does, but I'm just curious what your approach was.

Anton Nashatyrev: Oh, let's conceal, yeah, so, basically, so… If you have, if you have a link with a latency, right, you may… And there are, there are some messages pending in this, in this link, right?

Marco Munizaga: Link?

Anton Nashatyrev: For example, it's, latency for 40 milliseconds,

Marco Munizaga: Yeah.

Anton Nashatyrev: You may, you may advance, the receiver. this 40 milliseconds, in the future, so, it will, it will just drain, the, all, all the, all the messages. And, but the sender, sender just, may, may sit on, in the same, at the same, at the same time. I mean, simulation time… And, like. Before, so, it, it, it cannot, hell, hell.

Marco Munizaga: Maybe… wait, let me see if I can repeat this back to you. So Node A can advance… can do its own work up to 40 milliseconds into the future. In parallel with whatever else is going on, because at 40 milliseconds, it might receive data from the router.

Anton Nashatyrev: Yes, right.

Marco Munizaga: Okay, I think that makes sense. Then… oh, did you profile the many nodes tests? And if so, do you know what the bottleneck is?

Anton Nashatyrev: Oh, yes, but You mean, could we… could we even more increase the speed, or, you mean, profile with respect to memory or, performance?

Marco Munizaga: Both. I mean, I think memory is probably going to be mostly on the data on the link. So, probably more on the performance. Unless the memory's not bad.

Anton Nashatyrev: No, yeah, I remember you said me that you've got the most memory consumption, like, for the packets in the Like, in the flight.

Marco Munizaga: Yeah.

Anton Nashatyrev: Yeah, it's not the case. I think it's about, like, 20% of the whole memory consumption. The remaining is, gossip, router implementation, So, yeah… Oh, just like…

Marco Munizaga: buffers in there? Yeah. Assuming…

Anton Nashatyrev: Yeah, buffers, yeah, I did a lot of optimization, optimized it, like, more… almost twice, with respect to memory consumption, but there are still a lot of, you know, like, message, CC in both, like, message woofer, because they… it needs, the, we have, we have almost simultaneous, we have simultaneous publishing, so, it needs to, it needs to keep all the messages in, in its, in its synth buffer, in its, mcache, I'm gonna solve. like, consumes a lot of… a lot of, memory.

Marco Munizaga: Well, this sounds like a possible optimization we could do in the future. Not… I don't think it actually matters, but, like, you could imagine that if we have this data duplicated in our local… like, the ideal implementation should just be able to offload that data straight to the network as soon as possible.

Anton Nashatyrev: Yeah, to be honest, yeah, I did this optimization, so, we have almost no duplicated data now, so all the messages, you know, like, there are a lot of supporting structures, so it's not just messages, like, pure messages, they, it doesn't consume that much, but if you, sum up all the, You know, all the structures, in a gossip router. So, yeah, it's actually not ideal implementation of gossip router. It could be, yeah, it could be… Optimize with respect to memory, but anyway, it's not… it's not that interesting from the practical point of view, I think, because basically, basically, it eats just, like, 5 megabytes at maximum, it's not…

Marco Munizaga: Busy.

Anton Nashatyrev: It's not some… it's not something you… you want to optimize for. Got it.

Marco Munizaga: So what is… so, is this multi-threaded?

Anton Nashatyrev: Yes, yes, so…

Marco Munizaga: Is the scheduling of threads deterministic?

Anton Nashatyrev: Yeah, yeah. I mean, no, it's, scheduling, scheduling is not deterministic, but, the result is, deterministic.

Marco Munizaga: Hmm. But if two threads have to do this work at the same time, then… And there's only one slot. Then, depending on which thread gets scheduled, you'll have different results.

Anton Nashatyrev: Mmm, no, no, let me get back to… Yeah, so here, for example, you may, you may, in parallel, you may, advance both node A and node B. The result will be the same, so it doesn't, it doesn't matter, which of them, Advanced first, right?

Marco Munizaga: Maybe. I think it does, but we… yeah, we can talk about it later.

Anton Nashatyrev: Yeah, yeah, yeah, it's, it's actually true, yeah, but, it, yeah, actually, this, this is, like, fancy, fancy stuff with parallelizing. This, simulation, yeah, so it's… it's not that, intuitively, Clear.

Marco Munizaga: Yeah, the other thought I had was, so, you might be able to just run this whole experiment within Shadow and just get the determinism. Free. And then Raul says you can also do some instrumentation on the bytecode to also get the deterministic randomness.

Anton Nashatyrev: Yeah, this is a good idea about instrumentation. no, no, not bytecode, because we have, native implementation of QUIC, so… No instrumentation. S, yeah, shadow, yeah, shadow is deterministic, because they, yeah, because they intercept, Randao call Yeah, RANDOMC calls, and you've got… Always the same random number. Yeah, that's great, but yeah, as I said, Shadow wouldn't be able to simulate 8,000 nodes, especially JVM nodes, so…

Marco Munizaga: Well, what I'm saying is, you have your simulator with 8,000 nodes in this one process, and you run that single process within Shadow. So you're not using any of Shadow's networking stack.

Anton Nashatyrev: Oh, I see what it means.

Marco Munizaga: You're just using its determinism for syscalls.

Anton Nashatyrev: Yeah, but I don't think it's, I don't think it would be, like… Any meaningful time to execute.

Marco Munizaga: No, no, no, it's… oh, I don't think you'll be that much slower. I don't think you'll be slower at all.

Anton Nashatyrev: Oh, really?

Marco Munizaga: The only overhead shadow it's giving you is the interception of the very few syscalls you should be doing. Like, almost all of your execution time should not be sys calls.

Anton Nashatyrev: Oh… I'm… Right, it's just like…

Marco Munizaga: Intercepting the randomness, the randomness.

Anton Nashatyrev: I'm not sure, because… yeah, yeah, yeah, I see what you mean. I'm not sure, because GVAM, like, does a lot of syscalls under the cover, but I don't know, actually. Maybe, maybe you're right, yeah.

Marco Munizaga: Thanks, Anton, this was… this was really fun.

Anton Nashatyrev: Yeah, thank you.

Kamil Salakhiev: Yes, sir. Great, thanks a lot, Tom. Just trying to be mindful with time, yeah, let's… proceed with the next presentation. Caba… Finally, you have a chance to present Chainscope.

Csaba Kiraly: Hold on, Max.

Kamil Salakhiev: It's been a long time cooking.

Csaba Kiraly: Yeah, so… ChainScope, which I wanted to present.

Kamil Salakhiev: Oh, yeah. I don't know.

Csaba Kiraly: I think it's 6 weeks ago.

Kamil Salakhiev: Anton stops sharing the screen,

Csaba Kiraly: Yeah, I will share the…

Anton Nashatyrev: I think I stopped, right?

Kamil Salakhiev: I don't think so.

Anton Nashatyrev: Oh, really? Oh, that's true.

Kamil Salakhiev: Maybe I can… I can stop it for you.

Anton Nashatyrev: Oh, yeah, great, yeah. Thank you.

Csaba Kiraly: Yeah, let me just share mine. Oh, no, Zoom is asking for… Oh, yes.

Kamil Salakhiev: Okay.

Csaba Kiraly: Yeah, let me just find the light window and… And I'm ready to share. I think I'm sharing the wrong thing. Do you see something shared?

Kamil Salakhiev: Yes, we can see change calls.

Csaba Kiraly: No, you can't. Yeah. I just had different windows open onto it. And this should be it. Okay. Oh, let me also minimize this. Yeah, so chain scope. So… I was having, yeah, so I had a, a claud, which I wrote. 2024, I think, which was the fast etum caller. Which did, fast cloning, so it was… it was clawing the whole network. In less than 60 seconds, if you were showing a bit of bandwidth on it also less than 30 seconds, it was kind of… Mapping Autodesk V5 network, but it was only this V5. And recently, I extended that in a few directions. So the directions are…

not just this free 5, but also this freefall, not just this free fall, but also adding, like, P2P and FP2P support. So, I… after finding notes, I can also connect to them. If they let me, we have a date on that. And then, then get… data from them, just identify them, should have P2P only P2P. That was one dimension. Second dimension was putting, Making it a service, basically, so it's continuously running, and it can do scans of the… I'm now running scans at the 6th house. And the third one is putting a web UI on top of it. So that's… that makes up Chainscope, and it has a few neat features. the… the public URL is not yet ready, so I'm… I'm showing my… my local copy now. Actually, it's not local copy, you can access it if you just ping me for the URL, and then I will do also a public copy in the next days, as soon as I set it up. Okay, so what we see?

We have a bunch of screens, I will show a few of them, and then we dig into some network details. This kind of overview is showing you kind of the extent of things that we are doing, so, we are… This is just the last 24 hours statistics. And we are looking at, more than 200,000, node IDs, but node IDs are just node IDs. And they are on 77,000 distinct IPs. More interesting is this one, because these are nodes which actually we managed to connect to at the discovery level. So we had a D3.5 or D34 message exchange, they do support the protocol, and that's, 80,000 pills, client notes something running, from about 30,000 IPs. And that's really cool. And then, we also identify them, so weP2P connect, and then, again, we have, nice numbers there. And obviously, identification then accumulates, so if I cannot connect to them today, maybe I managed to connect yesterday, or if not, tomorrow, and if not, then they might also…

Passive… so we also have a passive palette in which they are just coming to us, and they will connect to us. Overall, this whole thing is mapping more than 120 chains. So the nice thing about the Discovery Network is that, it's a global network, and it's many chains, many EVM chains, So we have the mainnet, and then we have a bunch of L2s. Not all L2s are running clients, which are on the Discovery, but many of them. And then we have many other chains, testnet, DevNets. Yeah, we will go into that. Clients. So, clients here means the client's code, so guess is 1, Del, and we have, like, 173. client software. This is just based on what they claim, but different families of… of clients, folks, of whatever. So this is much more than the 5 or 6 EL and 5 or 6 EL clients. We have much more stuff going on.

And then this is just, things that we do. We identify chains, and alpholics. We identify clients and quantify Cool. How they share the, the number of nodes. for EL for CL, for other stuff, We measure the performance, we check history coverage, based on what they claim, so how much of ethereum history is ethereum-specific, how much of ethereum history notes are keeping, geographic diversity, protocol diversity, and protocol details, we will dig into this one. Yeah, we get lots of contacts, many nodes which are on inbound, so we never manage to connect to them, but they are coming to us. We do scans, yeah. That's it. Let's go into details. So, chains is a nice clean, I like.

Because… This is the one I like. So, this is the number of nodes per chain, and actually, Polygon is very nice here. So this is… there's a bit of naming here, to clarify, and I'm updating it. Here I'm… taking the maximum number of ERs and CRs, so,

Kamil Salakhiev: Caba, I'm not sure if it's just me, but I see a frozen screen, it says chains, and nothing happens. Oh, huh.

Csaba Kiraly: Let me… Lee. Load it. But you see chains.

Kamil Salakhiev: Yeah, I just see change, but it's loading, but not loaded, like, only…

Csaba Kiraly: Is it loaded?

Kamil Salakhiev: Yeah.

Csaba Kiraly: Do you see me scrolling?

Kamil Salakhiev: No.

Csaba Kiraly: Okay, then let's stop sharing and just sharing it again.

Kamil Salakhiev: Yeah.

Csaba Kiraly: the hub. Zoom is not always off, and let's see, share again… James. What about now?

Kamil Salakhiev: Yes, now it works.

Csaba Kiraly: Okay, so Chance. So this is a view which is showing… the different strains that we see, and I was going down to this one, which is… Different chains, polygomery, the mainnet, etc, and the number of nodes. So the nice part of this is that it's a low rhythmic scale, so polygon and ethereum has 10,000, notes… And then, those which has 1,000, and then those which are around 100 notes, and… and, those which are having 10 nodes only, so you can see the diversity, but you can also see that the whole Ethereum ecosystem itself, at least the public part of it, is already huge, so much bigger than just what we have on the mainnet. So, on the mainnet, we have this

more or less 10,000 nodes. Overall, we have this more or less 40,000 nodes, in the ecosystem. There are those which are more distributed into the mainnet, bunch of counties, bunch of nodes, and those which are less distributed. Take what you want from it, and then for each chain, we have details, obviously, so we can look at the chain and its forks. Who is learning what? Yeah, see how well we can look at the chain and get the chain details. It's again loading, I'm not sure you are seeing it, let's see. But you can, you can get, dig into details about the chain, The current fault mix, client mix. leachability of nodes, so this is something we are looking at now. Who is leachable on this V5, who is leachable on this V4, how that's changing with time.

And, yeah. just lots of, lots of detail, which I'm not going through. We also have these kind of things. So, which clients are learning which fork? For example. So you can see. BPR2, alkaline fork, how many clients are running those, and then other folics, and, what client is running on those. Okay, let's, let's dig into something else. Clients. So this is the usual client distribution, with some tweaks. Client distribution is, we like those figures, obviously. You can select EL only as EL only, let's say EL only. And then, the tweak that we have in this tool is that we have those which answered and we identified them. Okay, let's look only at mainnet.

So, you can obviously do a bunch of selections, deduplicate based on IP, and do lots of filters, but if we just look at it in the mainnet, and ELs. There are those notes which identify the EL. But we also have those which we didn't manage to identify. But we do have an influence module, which is… trying to guess based on a bunch of signals, who's who. So even for those which… which we never managed to identify, we are guessing. And, that gives us numbers, we can debate about those numbers. Well, we can just hide them so that we don't see them. But I wish she was clean about, about the influence. So there's this influence module. And, obviously we can…

we can do filtering. So, so different clients have different policies of changing, you know, depending on what you run, you might not rotate your IP, because you're running from home, you might notate your node ID, and all these things matter, so we have different filters, to count things this way or that way. Okay. Then you can go and look into a specific client. And then you get client-specific details. So… Yeah, I'm moving it into the fastest node at one point. So, this is gas details, and to look at some interesting things, yeah, which force it's running on. This is now just mainnet guess. And then which versions are learning. So… This is the last one.

And we have… these numbers running on mainnet, and then you can also, do… They're down based on the version. Which counts as a learning git, ESNs, forks, etc. One thing I haven't… I wasn't showing on the chain's page… is… these numbers, which we like. So this is, yeah, take it as a gain of salt, it's… It's based on… public data, based on what counts as data center, what is residential. But we have, for each chain, we have this residential versus data center percentages. off. who's running these notes. So, ideally that it's quite good on this metric. Okay. Let's go to more fun. This is the notes screen. So, where are the notes? We can geocate them based on public databases.

And then you can just look at either the mainnet, any forks, any clients, And… dig into. How many nodes are learning well? Based on what they themselves publish about, themselves. So, whether they identify as lighthouse Nimbus, etc, you can… you can see, and… and you can do really nice statistics based on this about, who's running what. Not going into more detail. Historic coverage is very specific to, historic spirali, so I was adding this to… to understand what's happening with… Is Siliac Spiley? with the new versions of their P2P notes are publishing, how much they are going back in time… was ahead of, and how much they are going back in time. And that's what you see there as you are going back in time in the block range, how many notes still have it. Basically, 60%, 60+, almost 70% of the notes have order history, and then here you can find notes which are keeping less history. We expect this to change this time, but…

We hope… There will be stuff here. Then we develop, other… techniques. You can also look at things like, guest notes, how many Of them are working as a block archive, how many of them are, having, etc. You can find the definitions, so just, just… Based on just this, declaration, how we are keeping history in the live network. Let's go more into the network research part. So it is a peer-to-peer network, you have churn. Quantifying churn is awfully complex. That's what you see embodied here in this filter part. But basically what you see here is kind of how many… Since we are running everyday multiple sweeps of the network. how many nodes we are seeing, how many we are seeing burn and die, and then, we have some extra tweaks here. So, for example.

we see notes dying, that means just that we haven't seen them today. But maybe we will see them tomorrow. So, we can count this in, and we are estimating this, so this bar that you see here is kind of a collection based on that. So, we have historical figures on how many nodes we haven't seen, and we've seen later, and based on that, we can collect the flash data. So we have seen this many dying, but we kind of know that this pollution is just temporalizing, and they will come back. So this is the real figure. And here we have to… filtered a lot. I'm not going into the details of these, but Yeah, did application, IP filter, minimum spam that they have to be, there, because cluers, for example, generate these IDs, which are, which are just very short notes, so we require the minimum spam to count, and so on and so forth.

Journal, really nice to quantify. We can quantify it for… Into the mainnet. And then it will update and show that. But we are not waiting for that. Now, I was also adding this network performance. This was part of my original call, that it was measuring latency, and also trying to measure bandwidth. The bandwidth measurement here is a passive measurement. Estimating bandwidths based on packet pairs, so let's focus on the downtime now. The difficulty of long-tip time is that I'm measuring from one location, I'm measuring from Italy, so if you… if I would just show that. What you would see is, Nodes farther away have higher latency. not too interesting data. So what we do here is we don't do that. We are taking the local average latency, and we…

measure nodes compared to the local average. So if you zoom into somewhere… you see that… this node, has a much higher latency, 6 times more latency, than nodes around it. So, the nodes are good nodes. blue is kind of average-ish, and lead notes have some issues. They are, responding very slow. And then wherever you go, you find these slow notes, and then if they know they are slow, they can… Check their own, notes and fill out why they are slow. We have bandwidths also, it's a bit… Less reliable data. And then, of course, you can also do histograms, bandwidths and latency. And analyze all this data. Even more networking stuff. Is the network protocol details.

So first, you find an explanation of basically what we are doing here. So, we are discovering nodes, we try to contact them, through the… These are just node IDs, which someone generated, maybe it was live at one point in time, who knows what. These are the ones which actually matter, so these are the ones that we can, UDP connect, And they respond, and then some of them, in the last day, We managed to… to contact with that P2P Hello, or with the P2P Identify, And then, They are going through phases of, of identification, and then, you have this funnel, basically, of, of getting more and more data from nodes. We also have this nice view of this, so how we are getting to know someone, who they are.

They'll need to do the mainnet, and then what they are, are they… an EL, I'd be inferring that they are EL, and so on and so forth. So, this is just how this kind of form machinery is working, and we learn about the network. This is a new one I was adding, because we are looking into EL renewal in DHT, so… I was looking at how notes are renewing the analysis. There are notes which are just doing it once and then keeping it, others are renewing it on average every day, once a day, every few days, etc. A few notes which are doing frequent iterations. Many things on this V5, this V4, this V5, because we are doing the migration, so how many nodes we find on this V4, how many nodes we find at this V5 only, and how many on both. I expect this to change this time, and I will plot also the time history of this.

Yeah, let's go into something… Interesting. So, protocol evolution. This is DF P2PDTO. So… We are changing versions, new versions are coming, and this time… the version… versions are getting deployed into the network. This is a slow process, as you can see. So, we have slightly more… East. 71 going into the network, and then we will see things dying out. But it's a slow process, we just need more data to see this. But this is something we can check around forks and, and see. how things evolve. Maybe even cooler is this one. So this is, the version support matrix. Whenever you are contacting to an EL node, it is declaring a minimum and a maximum version, basically. It's not, it gives you a list of supported protocol versions.

But it's always a continuous list from something to something, and what we show here is… is kind of what is this something to something. So the… this is the max version, this is the min version. And then… these notes… The max is 71, the min is 68. They can speak to, who they can speak to? They can speak to… these guys. Yeah, you can figure out which is the triangle they can speak to. It should be clear. So they can… they… these can speak to… to anyone who is speaking, those versions, so yeah, I think they can speak to these guys, and they can speak to these guys. So you can, you can just see this kind of who can speak to who.

Kamil Salakhiev: Sorry, Chapo, just trying to be mindful with time.

Csaba Kiraly: Yeah. Going straight.

Kamil Salakhiev: Can you show this data, I mean, about, yeah, gossip-sub-partial message support? Yeah.

Csaba Kiraly: It inserted its one. Because that was the P2P, this is EIP2P, so, yeah, this is stats on, on, basically. How many nodes are declaring what kind of protocol support on the P2P? And, it's also colored by, kind of, what they are required for. And the interesting side part you wanted to look at is this one, because it's a partial messages extension, so now this is at, this level. on… on mainnet. I don't have the time visualization, but we can also plot it, like, time-wise, how it's… It's, evolving in time.

Kamil Salakhiev: It's interesting, we already have execution payload envelopes.

Csaba Kiraly: Yeah. So, things are good. I will, I will do a time plot, actually, for this. It's just… more fun to have also that one. So, yeah. But you can see, kind of, what's needed for a light client, what's needed for, etc. This is the new one. Nice. Yeah, I'm not switching now, because then we have to load, but two testnets and another thing, just to see what they show. Yeah, not going into details, just to say that we have an exploration view. If it's loading… Whenever you have these figures, there's… I'd like to have it for most of the links. You can open it in this view, and then you can just, change dimensions and explore the data. So you can put it on the… on a node map.

And then, if it's loading, and then you can just visualize it, oh, this is 70,000 node. So, you can add a bunch of filters, you can change it to… Show by client family. And then you have, like, the current distribution, maybe it's better than a consens map, and then you see, like, yeah. This is just numbers, but you can do a nice pivot table about who's running what, and and this is just a genetic data explanation. I can also show a few more details, but I guess we're out of time, so… other DAS I prepared are… One is, just the influence part, but you can believe me that there is that. Another one is… is… is co-location, so how…

CL and EL that's… it is a minute specific, how CL and EL pairs are showing up, just understanding how… CS in the RPC alone in the network, and There's a fun screen about callers, so, one showing the other callers running in the network, and identifying them. That's it.

Kamil Salakhiev: Me, it's really nice. Yeah, let's maybe just answer a single question, like, how can people access it, or when you're going to publish that?

Csaba Kiraly: So, it's called Chainscope, it will have a URL which is called changescope.something. I will publish the right one soon. Or just ping me, and I send you a link where you can get it from my home, so I will… I will have another one, which is not standing from my home. But it's lightweight, so… The whole thing is relatively lightweight.

Kamil Salakhiev: Okay, cool, nice, nice. So yeah, the answer is ping Caba to get access.

Csaba Kiraly: I wanted to do it for today, but that didn't happen.

Kamil Salakhiev: Yeah. Okay, cool, cool. Yeah, we're running a bit late, but I think we had just two discussion points to bring up. First, I think Sukun wanted to bring up about the session bundling, Sukun here.

Sukun Tarachandani: Hello. Yeah, so I also have a presentation, but yeah, it'll be quite quick. So, yeah, I just want to, like, give a brief overview of the thing so that we can have some, basically, background for discussion. So, I have an EIP for…

Marco Munizaga: Sukun.

Sukun Tarachandani: attestation?

Marco Munizaga: Can I interrupt you for a second? Can you choose a different microphone? It sounds like you're coming from a tin can. I think…

Sukun Tarachandani: I mean, try.

Marco Munizaga: I'm not here. Very awesome. Unless that's just me.

Csaba Kiraly: No, no, it wasn't for me, but I think…

Sukun Tarachandani: Is this one better?

Marco Munizaga: Oh, yeah.

Sukun Tarachandani: Okay, cool, yeah. Thank you. So, you can see my screen, I hope. I have the EIP, and the EIP is fairly simple. The idea is that currently we send Each attestation with its attestation data separate, and so it does, like, the dumbest thing possible, and it waits for a bit to batch a bunch of attestations. So that we can deduplicate this attestation bit in current day single attestation, and it sends it as an attestation bit with a list of signatures. So… and this is, like, a… just the improvement on the message sizes, you also get improvements for the gossip with this, which I have not included in this presentation, but you can see the EIP for the details. And, like, yeah, as soon as you bundle 4, you get almost 45% of savings here. And in my simulations, we bundle up to 10 most of the times.

So, I just want to, like, give a brief background as to why I have proposed it this way. So, the thing is that if you do it with partial messages, and just do this change, and not do any other change. What we get is this nice deployment strategy that we don't need any fork, and each client can ship it independently, because partial messages are completely backwards compatible, and we don't have to, like, have this sort of coordination for, like, deploying this. In terms of results, yeah, like, almost 50% bandwidth saving, I see 30% in, like, the tail latency improvement. But, like, yeah, I'll just give a brief, also, rationale for… why I haven't gone with the custom protocol. So, with the custom protocol, I was able to see better… 20% better tail latency.

Now, all that protocol had different from partial messages is, like, this… Tight control over the queue of outbound messages. which maybe we can also include with gossip sub, but the reason, like, I dropped that line of thinking was that, my primary reason for proposing this EIP is that This gives us very comfortable latency profile for I-Star, where we want decoupled Consensus, and, like, an 8-slot finality. And then post-I-Star, we anyway need PQ signatures, which would, like, require something very different, because the aggregations in the PQ world are not as cheap. Which is why, basically, I've proposed the simplest state possible. But we can actually do something much more complex, which is, like, we can allow in-network aggregation. It's not much more complex, it's, like, slightly more complex, it's just that the complexity is around how we deploy the thing, because once you introduce aggregations within the network, you will have to…

have this coordination between, and we'll have to deploy data for boundary. And also, I'd like to highlight that if you do a patched attestation at source, thing, which is like… okay, sorry for the poor naming here, that is pre-aggregated attestations at source, which is Raul's EIP here, 8243. we can basically add it to the same bundle, and it should be… it should work in conjunction with this proposal. So, for the aggregation bits, Daniel, do you want to talk about it?

Daniel Knopik: Yeah, sure.

Sukun Tarachandani: So… Yep.

Daniel Knopik: Aggregation, like, just, just sending, Any… or allowing, people to send any aggregates as the problem, of these overlapping aggregates, which… which are useless. So in the past few, few weeks, I don't have a proper presentation right now, but I've… Investigated more. About, like, limiting the valid set of… of, attestations. In, like, a tree structure, by… basically, we have the… we have the… shuffling of the committee, and by basically just allowing neighbors to be, aggregated, and then in this next In this, like, new layer of… of, level 1 aggregates, we, again, allow the neighbors to aggregate, etc. We can limit the, the, the faction, the… the… the set of allowed,

aggregates. And this is nice, because then we don't have the problem of encountering these overlapping aggregates. And… I was able to get some simulations with, like, up to approximately 70% bandwidth savings. So, yeah, the question is now if this is something we want to integrate with this? And also… How exa… like, if this should coexist with patching at source, because… Patching, it also has the nice property of not needing, like, it… for, like, a tree structure, as I described, we only benefit if if signatures are present for neighboring, nodes, and this is quite unlikely, like, if you have

10 validators in a committee that, according to the shuffling, they will be neighbors, so we would actually not benefit a lot from this in this, like. Early, very sparsely populated tree. S-so… So yeah, I guess… I don't have, like, a complete solution for now, but I would love to find something that would allow us to basically used this idea by Sukun, while also yeah, ideally benefiting from… Some kind of batching at source, and… Even better would be if we could also avoid, like, the new… Like, the… the required ceiling of, valid, you know, of the… of the batcher, just to avoid more…

Yeah, signature types floating around in the protocol, which are quite cumbersome, for example, for… Dvts. Yeah, kasey?

Kasey Kirkham: I spoke to Sukun before about doing something similar, but based on, you know, a modulo. I'm wondering if the neighboring concept is important for something I missed. Before, like, is there kind of a tree structure this is aggregating up into that, where the ordering of the indices is important?

Daniel Knopik: So, the… Tree is basic… like, the idea behind the tree is to… So, in principle, it does not need to be a tree, right? We would just need some kind of way to, Avoid overlapping overlapping aggregates floating around, which then, in the end, become unagregatable. And a tree which cleanly separates, this was the… like, most intuitive choice there for me. I didn't quite catch what you said about something modular?

Kasey Kirkham: Yeah, just, just, moduloing the, The Validator index of the attester to decide if it was something that you should batch or not.

Daniel Knopik: Okay. And I forget.

Kasey Kirkham: The advantage of that is that it could be done with, you know, less, less… Not needing to actually compute, or have that data local to the process, deciding to batch, I guess? Which maybe you do anyways, since you're validating, but sometimes you kind of stick these things in a pool, and… and deal with them later. And yeah, I guess I was thinking that it's cheaper to check their modula than it is to check if their neighbor's in the sorted list.

Daniel Knopik: Yeah, I… I think… The… Problem with that might be, again. That we only have, like, we… okay, we have our committee. And a certain subset is modulo 1, another is modulo 2, and we might only have some of the modulo 1 signatures, so again, there might be overlapping aggregates in Modeler 1. Unless I understood… Your idea wrong.

Kasey Kirkham: Well, I guess it's kind of the same as the tree idea, I think, in that, Like, is it…

Daniel Knopik: Sorry, you cut off there, I think?

Kasey Kirkham: So I think in that way, it's kind of the same.

Daniel Knopik: Okay.

Marco Munizaga: Can you repeat? You cut off in the last minute.

Kasey Kirkham: Oh, I was saying that I think in the tree idea, the concept is that aggregators are kind of claiming different leaves in the tree. And, in order, in order for it to be non-overlapping, and so I think I'm saying that modulo, seems the same, to me. Yeah.

Daniel Knopik: It's not really different aggregators claiming different parts of the tree, it's more like every node Maintains a tree of… of… of aggregates, and if a node… if a peer is… is missing some… some part. We just sent them the parts of the tree with, like, the… the, highest, like, the most aggregated, or, or, so the, yeah, so the highest node in the tree.

Kasey Kirkham: Yeah, seriously.

Daniel Knopik: Allow them to… sorry?

Kasey Kirkham: So you could… so you wouldn't have to send the individual attestations if you had… Right. Yeah, I get where that's going. Okay, I like that.

Kamil Salakhiev: Sorry, I might be missing some… Yeah, Daniel, how do we organize PBS into a tree structure without breaking?

Daniel Knopik: We're not organizing peers, we're organizing validators, and this is… this comes naturally from their shuffling in the, Committee. I'm sorry that I didn't prepare any slides, like, this is a bit ad hoc. I… have… Yeah, Raul?

Raúl Kripalani: So, yeah, I think we discussed a little bit of… about this in Svalbard, and it's great to, like, have… to have the idea mature, and I think there is definitely some convergence with batching attestations at source. It was also highlighted in, that All Core Devs call where I briefly introduced the idea. That, you know, if you take it a little bit further, then at one point, you can, like… if you assume that you're gonna be having nodes in the network, that are already pre-aggregated attestations, for the nodes that they're responsible for the identities that they're responsible for. then you could take it in a little bit further and say, well, really, are they forming an aggregate that can directly flow into the aggregation stage, right? So then, kind of like, these two fuse a little bit, and it's just about, like, figuring out, I think, the partitioning

policy, or the assignment policy, and making sure that that is, like, that is reliable and safe, and that there is fallbacks, right? Because… With, if we still have, like, this two-phase approach that we have today, which is the testing and then aggregating, then you could still have a node that has been that is, whose attestation didn't make it, then, like, have a salvage pathway there. But if you don't, because everything converges into one, then, you know, I think you need to have that redundancy, in the protocol somewhere, so that maybe a node can designate, like, two aggregators, or, like, two other nodes, or something. I don't know how… That could be, and maybe the tree structure is trying to solve for that.

But I would be very keen to, like, join forces and figure… figure out, like, even if it's not… even if we do batching attestation at source first, and this other idea continues maturing, like, I think it would be great that the first version, if the first version Like, if the ultimate version wasn't breaking the first version or the other way around.

Daniel Knopik: Yep. You could still do the same thing with their IDs, for example, organize them into a binary by the Validator index. Okay, but… Yeah, so… I think your point is, instead of… Using the shuffling in the committee as a basis. For the tree? like, the ordering of beliefs, we could just order them by Validator index. And this actually is interesting, because Like, a node is more likely to have validators with adjacent indices. and this could… Automatically be some batching at source, then. Cool.

Raúl Kripalani: Yeah, I think that's a little bit trickier, because making assumptions on validators and Beacon nodes that are responsible for those validators is a slippery slope. So that's why the seal mechanism exists, right? Because it removes any other protocol wants to move around those… those two worlds, which are… Technically disjoint.

Daniel Knopik: Yep.

Raúl Kripalani: And, the other advantage of modeling it that way is that then you could… You could… right now, even if this is used in a trusted domain first, which is, you know, multi-validator operators are matching their own attestation-source, by modeling it in a way that does not make any assumptions, you can then generalize the solution for a pri… to build up towards a privacy layer, right? So then you have the standardized solution that… with which any validator can choose any other validator that is part of a committee to channel their attestations through them. By using the SEAL mechanism. So that's another motivation for the SLO mechanism.

Daniel Knopik: Okay, I must admit I didn't quite get that, but maybe we can talk about this some other.

Raúl Kripalani: Yeah, no, I'm saying that because you don't make any assumptions about the layout, of, like, any structure about which things need to be aggregated. So, no adjacency, no tree, there's nothing like that, right? It just, the way that the association is established is by the seal, where one validator says, hey, I… Empower you, another validator that is co- attesting on the same slot, on the same committee. to channel… to aggregate my… to pre-aggregate my validation. So I'm not going to release it directly to the network. I am going to channel it through you, and you're gonna be responsible for sending the batch to the network. This, in the first version that has been proposed, is used inside a private domain, right? So within.

Daniel Knopik: Okay.

Raúl Kripalani: Validator within a deployment, that… where you have a single operator that's responsible for multiple validators, right? So then you do all the… all the sealing and everything, it might seem redundant. But the… or, like, you know, we might figure out ways to get rid of it, but then, the other advantage of this is that it allows you to apply the same solution outside of that trust domain. So now, say you have, for example, I don't know, Lido or some other Participants decide to offer a service for validated privacy? where they have sufficient Validator density to be part of literally every committee at every slot, right? This is potentially Lido is there. So then you're… you're guaranteed to have one co-located Validator.

Right? Or one co-testing Validator for every stop for any Validator in the world. Right? Because… just because of, like, you happen to have that density in the network, right? That quantity that leads to that density in the network. So then you can offer a service and say, look, if you're a validator and you don't want to release. your attestation directly to the network, even if you're not a Lido operator, right? You can channel it through me, and we have a standard to do that. You just have to produce a seal. And if you see that I don't release it. right, that I don't include it in my batch, you can still send your attestation… you can still send it directly to the network as a fallback.

Daniel Knopik: No.

Raúl Kripalani: So it gets interesting to reconcile this with aggregation, because at one point you realize, well, if you have these entities in the network that are going to be pre-aggregating, in such volumes, then, like, if the moment that they pre-aggregate… 5… Like, I don't know, like, 300… At testers, for example, in a particular slot. They… they are definitely below a committee, but then if you… or the volume of a committee today. But, if you sum all of their attest- all of their attesters across all committees. within that slot, that they… that they would have… that they're… that they're aggregating for, pre-aggregating for, you actually are forming, in terms of quantity, like, maybe 5, 6 committees. Right?

Like, if you have 3,000 validators at testing. Right? And they all happen to be, like, thinly spread across committees, so then you only get, like, I don't know, like, maybe 100 validators, I'm not doing the math, but 100 Validators per committee. Literally, what you could do, instead, if the network was not arranged in subnets, you could literally release all those 3,000 attestations, or divide them as much as you want, right, into a single channel. that happens to be an aggregate, like, you know? It's already pre-aggregated. And, taking into account that an aggregate today represents, like, the… if it's fully 100% filled, it represents around 450 attestations, I think? then aren't you anyway doing, like, aggregation? Why not release it directly on the aggregation channel? That was the point of, like, you know, the ACD conversation. Like, at one point of view, like, push it far enough, you can just rearrange. It's just like we're drawing artificial lines around the subnets, right, and the way that we're doing shuffling.

That maybe don't need to exist at all. So if we remove those lines, and we add the mechanisms for pre-aggregation, or aggregation directly at source, then maybe you can release those aggregates directly onto the aggregate channel until stop.

Daniel Knopik: Now, these are all… Great ideas. I guess, in practice, And of course, we can't, like, this is… Maybe beyond the scope for today, we are already over time, but at some point, we'll have, like, a different attestation scheme anyway, because of post-quantum, and I guess the question we have to ask ourselves is. what… Approaches, or what, you know. Things can we do now which are not too high effort? Yeah. Okay, yeah, thanks for listening to my idea, though. give me some food up for thoughts. If, like, I'm… I'm still going to look into this further with your input a bit.

But yeah, I hope we can… Find the ideal… Solution… With regards to effort and, complexity and, usefulness. Because also, and… sorry, closing note, I get that… Sukun's EIP and your EIP, like, batching at source, and, what Sukun presented. surf… Different goals, of course. But I'm still curious in, let's say, simulations where, like, One of those, EIPs versus both of those, just in order to gorge, How much benefit they bring combined, because there is some… some,

Raúl Kripalani: Yeah. Yeah. Yeah, look, totally right, because at one point, we're just, like, you know, shaving off maybe 1 or 2 kilobytes, of unique data. That's it, right? It's, it's… at one point, it's diminishing returns. So yeah, these are… even though they're different, entire different categories of solution, right? Because one is… you can think of it as, like, normalizing data. I, I, like, Sukun, the proposal that Sukun… Sukun presented, that he and I had been working on as well, like, in my head, it's like normalizing data. That's basically what it is, right? Because, it is a normalization app that works in transit, right? And it's a good compression technique, essentially, right? The other… the other one is kind of, like, changes, like, inter… yeah, changes the… the architecture, of… of how attestations are… are sent, or…

Yeah, channeled in the network in more fundamental ways. If we do batching attestations at source, then maybe the efficiency gain that we can get from, from the, the normalization from, from the bundling is, I don't know if… is gonna be fractional. And, the other one… way around, you're gonna get the more… the same ingest traffic, so… like, origin traffic, right? Unless you do… Unless the origin nodes themselves also have a notion of matching. Right? At source, but without the cryptographic element here. So it's just a mechanical optimization. But you're still not sending an aggregate, right? As far as I know, if I remember correctly, I haven't looked at the latest version of the proposal. But you're still not sending an aggregate, you're sending a linearized

A linear collection, right, of the attestations. And just denormalizing, like, normalizing the common data. Yeah, it's different efficiency gains. I think… I think we can definitely run theoretical simulations. We don't need network simulations for this.

Daniel Knopik: Yeah. Bye. I think… I think, though, if, like, the bundling gets us… like, I think Sukun said, 50% bandwidth saved. And… batching at source gets us… Let's say… say, 60% saved, and both combined gets us 65% saved. Yeah, Sukun?

Sukun Tarachandani: There is no guarantee that validators will batch at source. On the other hand, this is a network-level thing which we can implement to bundle them. Together. And also, like, the single validators will not batch anything, so I think bundling will still make sense at a network level.

Raúl Kripalani: Yeah, I think they're both mechanisms that make sense, and bundling is very easy to implement, honestly.

Sukun Tarachandani: Yeah, exactly. I mean, that's my whole point in there.

Raúl Kripalani: Yeah, exactly. It's the easiest thing to implement. So I think it's mechanical optimization that makes sense, and you can keep it as a transport optimization, and it has zero impact on the protocol, right? On the other hand, I… my hunch, and this is just a hunch, is that for… PQ, having some form of batching at source is gonna be desirable. I don't know if… the problem is that we don't have enough information to know whether this architecture for batching… I'm not talking about the cryptographic parameters, because they're all gonna change, but I mean, like, the interaction… You know, and the choreography between the participants here, so the batcher, the tester, and so on. If it's gonna…

remain the same. So if the overall shape of the solution is the same. And whether this solution can be applied in… in… I guess a fractal way, right? So, we're thinking… the reason I introduced it was… Just for… For the multi-validator operators, but you could also envision applying this solution… in a… practical way to breeze, right? So, if I have… if I have 3 layers, then the first layer is as it is today, but then the first… that's… the next layer can then nominate the next layers, and kind of like, you know, you form a transactional, propagation tree or aggregation tree. One of the disadvantages of all the solutions that have been proposed so far is the static nature of those solutions, and maybe this is the way

To… to actually create a dynamic more secure version. I don't know. Just thinking out loud, honestly. I had prepared zero, and I know that we're over… over time, so… And none of this is actually on the agenda, but… Good snow, man.

Kamil Salakhiev: It's great discussions, but yeah. with PQ stuff, you know, we kind of keep in mind, like, the size of data that we're working with, like, if we… Let's say we have a signature size of 1 MTU, but, like, the size of aggregation is, like, between 128 kilobytes to 512, like, with recent discussions around FOCIL, that's how large the aggregation might be. So if, like, if we're aggregating less than 128 kilobytes divided by 1.5… kyobte is, like, I don't know, like, around 80, whatever, signatures, then it doesn't make sense, so yeah, we kinda need to keep in mind, like, the sizes, and there are many unknowns, I mean, we don't know, like, what's gonna be within VM proof size, like, how long the aggregation's gonna take,

But yeah, some kind of boundaries would be great to keep in mind. Anyway, great discussions. Cool. Anything else we want to bring up or discuss here? Yeah, Sukun?

Sukun Tarachandani: Yeah, so just to, point out, Daniel, Casey, I'm happy to include, in-network aggregations in the EIP, just to leave a comment there on the EIP, or… we can collaborate on a hack MD as well, and then just change the EIP. Same for now, like, if you want to add the batch attestation to this one, like, the data structure itself, you would need to make it compatible.

Raúl Kripalani: You mean adding sections that describe the compatibility with adjacent proposals, or…

Sukun Tarachandani: That I have added, but if you want to explicitly highlight how both of them can work together, we'll basically need two fields, basically the bad signatures with the seals, and, like, the… set of validators there, Bachel, yeah.

Kamil Salakhiev: Nice, Unless there is anything else? Yeah, I think… let's drop off here, so yeah. It's been a great call, yeah. Thanks, everyone, for joining. I guess see you next time in two weeks. Have a day. Bye.

Marco Munizaga: Thanks, everyone. Lots of good presentations.

Kamil Salakhiev: Yeah.

Marco Munizaga: That's so scared.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.