Ethereum in numbers: Where TPS meets physics / Péter Szilágyi

Transcript

foreign so ethereum and transactions per second usually that's a combination you don't really want to read on the Internet it's really usually the sentences sound bad and uh what I wanted to kind of talk about today is that usually when you read about transactions per second in ethereum is how ethereum sucks and your

favorite theorem killer just rules all and I was I wanted to talk about it a bit how much truth there is to it there's always the truth is always somewhere in between but let's try to dig a bit deeper into it now before we do a little bit of a detour I don't know if you probably most of you heard about the boring company it's one of Elon musk's company the the goal of the company is to dig tunnels underneath large U.S cities like LA and somehow try to speed up transportation system somehow try to avoid the traffic and uh well a lot of people would like to do the things that he wants to do but the challenge is that tunnels are prohibitively expensive so everybody

says it cannot be done and well musk had this really nice idea really nice questions rather that okay when we say that the tunnel is prohibitably expensive is it because physically it's impossible to construct it faster or cheaper or is it because we socially accepted that it's prohibitively expensive and the answer generally is that while a tunnel has a certain diameter because it needs to contain a ventilation system emergency exits etc etc but if you take a look at it a bit closer the only thing it truly needs to fit is one car the ventilation system is necessary because most cars are combustion engine coils and they need oxygen so without

oxygen the cars just stopped but if you just switch everything over to electric vehicles boom no more ventilation system or at least a much smaller one emergency exits are obviously needed unless you take the control out of people's hands and you just automate everything you just let the AI Drive in theory no accidents happen again the tunnel can be reduced and plus there are a few more things that you can explore the interesting thing is that the observation here is that physics kind of limits you to certain aspects and then we construct a lot of extras on top but the the True Value in this certain case the particular case the true cost of a tunnel is kind of limited

by physics the rest we just pile on top now to go back to ethereum what does this mean for us everybody on the internet always says that ethereum does not scale and that's a very very hard thing to unpack it's very ambiguous but what people kind of say or want to say with if ethereum does not scale is that a theorem is too expensive for them this is again a bit ambiguous what does it mean that ethereum is too expensive is a Tesla car too expensive I mean it depends on your salary and depends on what you want to use it for so saying that ethereum is too expensive that's too subjective but we can agree that ethereum is expensive now expense is kind of driven by capacity supply and demand so if you would like ethereum to

be cheaper we cannot we can never make it cheap because cheap unless it's free it's always too expensive for somebody but if you'd like to make it cheaper then the two things we can do is either raise capacity or lower demand and the EF is actually doing both of them lowering demand we're trying to drive people towards l2s and raising capacity well let's see can we do that and uh well capacity is kind of people kind of Define capacity as transactions per second which is again super ambiguous because I mean is a plain transaction and each sound a deck Swap and an FD auction these are all transactions and these are absolutely not comparable so what ethereum actually does under the hood is that it uses gas to

abstract out the disproportionality of transactions now ethereum currently runs at 1.1 Mega gas per second on average and we kind of feel that this is a pleasant number however there are other networks like Avalanche and the binance marching who are really pushing the limit and then the question is that well they are essentially internal gas at least the evm they have the same state so how can they push it faster is there something they not telling us and there's a really nice hint there that kind of points to what they're not telling us binance March in one year ago was running at 25 Mega gas per million gas per second and currently it's running at seven so we can say that the

binance marching guys are super smart and they optimize gas but how did they unoptimize it one year later so what happened there and the answer is that gas is a fairly bad approximation of how much it costs to run a transaction because usually we have four aspects to a system load you have CPU load memory disk and network now running a transaction last year and this year takes the same amount of CPU same amount of memory same amount of Network traffic however disk is a bit interesting typically I won't go into the detail but this is how ethereum stores its state it's kind of all the accounts are laid out flat and we have a cryptographic tree structure on top to just try to

prove that everything is correct and well since this is a logarithmic structure we have every note has 16 children every mathematician will say that this is super efficient we don't talk about it anymore yeah the problem there is that as time passes and the state becomes bigger this logarithm starts to appear and starts to cause troubles specifically currently ethereum has about 170 something million accounts which translate into a state try of depth 8. that's nice that's a perfectly acceptable try that so when we try to do a transfer let's say Alice tries to transfer one is to Bob then we need to create 18 sorry 15 new trinos we just need to create eight

for the new account of Alice and eight for the new account above we have a shared root okay 15 per transaction still not bad yeah the problem is that we store the state in level DB which is another try which means more levels and that will be specifically stores data in Seven Levels which amplifies my 15 rights into 105 rights and in order to actually calculate the numerical path I need the old one too so that doubles my input output operation so we're up at 200 plus input output operations per transaction and of course binding in ethereum it's not the Run ethereum isn't run by a

single node so we have a miner and then the block is mined and then propagated mind propagated service transaction is kind of ran twice so if we do that essentially what we arrive at is that running a single transaction will require 420 input output operations on the network that doesn't sound bad except when you start laying out the exact numbers of different Technologies hdds are kind of capped by their six times you have the various ssds with various transports serial at the nvme and PCI Express 4 and I mean I won't go into the numbers but the idea is that every hard drive technology has its limits and if you just take those limits and divide them at 410 we got some super

hard numbers on what is possible and ethereum currently tries to aim for PCI Express 3n vme hard drive so it means that technically 857 give or take transactions is the absolute maximum that the ethereum network can do the truth is we are actually doing more than that now and the way we are doing more is uh these limits are only relevant if every single transaction pushes everything to disk so kind of like an archive node in our case what we try to do is to try to keep as much data as possible in memory because then all of a sudden we don't have to um screw around too much with discs so what

does it mean to keep stuff in memory there are actually two [Music] two ways in which we can't keep data in memory one of one is actually done by the operating system for free for us if you have a 64 gig machine the operating system will actually use the whole state I'm sorry the whole Ram everything to give to Cache disk so whenever you try to access all the data most of the time it will access it from memory the other thing that gath does is that we try to do state pruning and straight State pruning kind of means that if Alice sends one ether to Bob then one ether to clear and then one ether to Dave then these all these intermediate states that alliance's balance was decreased by one two three ether you don't really need to save that it's fine

to just save the last state and these are nice optimizations that allow us to reduce the number of input output operations and thus to actually raise the transaction per second capacity capacity of the ethereum network however this omits a very interesting detail and that is that system memory is limited so if the operating system is really nice at caching stuff in Ram if I have enough RAM to fit the entire State into it or we can use a very nice in-memory pruning in gas if we have enough RAM to fit it into it now I'm not entirely sure what the average gas ethereum node user has how much RAM I don't know 30 to 64 gigs

that's probably on the high end but ethereum state is currently over over that way over and then what happens is that the more you overflow it the more often the operating system produces a cache miss or page miss Page fault and that results in actually reaching down to disk and the same happens with pruning I can prune I can stuff a certain amount of data into our pruning algorithm but as the blocks grow as the state grows I either need to keep raising that limit or I keep leaking more and more and more info data down to disk and this is actually a self referencing cycle because the more the state grows the Lesser Ram I have to operate in it so the faster the state grows the less Ram

I have up to operate in it and what happens is that I am eventually reverting back to my original hot original transaction per second cap produced by the hard drive and the question is essentially based on this slide we kind of see that a big state is bad but how big is that I mean it's very hard to put the numbers to how big is bad what we can try to see is okay how fast does it grow because the faster it grows the worse it gets so the faster it gets worse and last week on a Sunday so Sunday is considered a pretty idle time for ethereum pretty much nothing happens on a network

the ethereum state grew by approximately I don't know 6.64 account per second eight storage slots that doesn't say absolutely anything to anyone not even to me it's just a random number if we look at how much actually it weighs bite size it's approximately I know 600 or something bytes per second which is 55 megabytes per day 20 gigs per year okay those numbers start to make a bit more sense but this is actually just the raw data just the useful data that okay Alice has one ether robot Bob has two ether it doesn't contain any of these marker proofs if we actually just dump all of those numbers in there I won't read

through the numbers we get kind of an extra double the size so all of a sudden the Merkel tree weighs I don't know 40 gigs per year that's the addition but this was interpolated from a Sunday during the week it might be two three times more so it kind of fluctuates it's kind of hard to just say that okay this is the growth but let's say I would say maybe 50 gigs 100 gigs per year that is the growth of the ethereum state I'm not talking about the blocks I'm not talking about receipts I'm not talking about anything just the raw State that's needed for contract execution the question is is this a lot or is this acceptable

and I don't know I mean if I have 64 gigs of RAM then piling 100 Gig every year on top of it kind of exhausted pretty fast and what does that kind of mean what that means to be a bit uh apocalyptic is that ethereum and all its Forks is kind of like on a potential death trajectory currently meaning that if you take any of the ethereum networks by ethereum networks I mean mainnet or the Avalanche contract chain or the binance smart chain or pretty much anything that is a fork of ethereum and if you just stick a constant transaction throughput on it the state

will grow maybe it's slower maybe it's faster in ethereum it grows at 100 gigs per second but binance marching at some point was um sorry 100 gigs per year but it's fine a binance marginal at some point running at 25 times our capacity so that would be 2.5 terabytes per year that's a nice number and the problem is that the more that number grows the higher Ram requirements we have and the higher requirement Ram requirements we have the more and more data we flush the disk the more iops we generate and essentially we're hitting that brick wall that thingy I mentioned those numbers they are approximations we can debate

whether it's 2x larger 0.5 x smaller it's the order of magnets to discount but those are the numbers that the hard drives impose on us so the moment we revert to running on hard drives the moment that are those are sorry the the hard limits so but that still begs the question our mainnet is relatively slow compared to everybody else's main nut so does that mean that we just suck at are others good at it or is the truth somewhere in between and my answer is that I may not can actually do significantly more transactions per

second that'd be sorry it can do significantly more transaction per second than we allow I appreciate them oh very nice so Armenia can be significantly more transactions per second than we allow it however the more we allow it to I mean the faster we allow it to grow the faster we bring that brick wall closer and it's kind of a Race Against Time can we keep that brick wall far away so that it doesn't actually impact us in a very very detrimental way or is that brick wall something that just arrives and ethereum shuts down and game over

and usually there are some very nice proposals do we have the eip444 from map I think hmm which essentially aims to get rid of historical chain segments unfortunately that doesn't really help the state growth we have short block transaction proposals which try to move as much data as possible to l2s Plus delete them afterwards at L1 okay I'm sorry I haven't talked in a while so it uh yeah

so um the sharp blob transactions try to move as much data as possible as on top of l2s the and that little data that actually ends up on l once it's defined in a really elegant way to be able to be pruned off but unfortunately neither of them really touch on on raising the capacity or moving this brick wall forward it's okay so as essentially the we are kind of running out of time and the only interesting Solutions we have is State Range which was shut down we have exponential costs which is kind of the current situation

or we have probably possibly stateless clients and that's kind of the hopeful direction that we are going to towards this does mean that there is no solution existing currently and ethereum has time however if you look at other chains which are pushing the limits super hard the danger there is that there will actually beat out ethereum on on chain growth and they will actually beat us to the brick wall and that's not a problem for us because we can see what happens when people reach the brick wall and you can actually see it on a

balance March and it's really interesting to dig through their issue tracker but uh it's a dangerous place to be at to arrive at the Bricco without a solution So currently my takeaway is that ethereum's transaction per second is deliberately low we could push it significantly harder but that would actually make it significantly harder for us or give us significantly shorter time span to to fix things it's definitely the smoke but I guess the good news is that we are trying and we have Pioneers who are pushing it harder than us so we can learn from them

and thank you [Applause] you made it wow barely but you still made it you didn't hit any brick wall do we have anyone in the audience who has any questions for PETA about or who who's first okay great oh yeah wait we've got a sorry one behind you with a microphone um yeah thanks just quick question so none of the things you talked about were like latency or internet related no so what if computers just get better fast enough

that would be nice but they don't so the problem is that currently so computers get better in certain areas but uh current limitations are kind of disc latency and currently the best technology is PCI Express Express and vme hard drives which have approximately 1 million i o operators per second which translates to 2500 800 transactions per second and there's nothing really coming that would make it significantly faster so you could say that okay there are always some super fancy technologies that you could deploy in a data center or in a controlled

environment or in if you are willing to spend some crazy amount of money but and I guess that's what kind of binance go stores do that they have a limited number of nodes limited number of validators and then you can just push them super hard and have super high Hardware requirements that's an option ethereum kind of tends to take the more public approach where anyone can run a node I mean does have the costs but if you want anyone who's willing to run a node be able to run a node then you kind of need to keep the hardware requirements within reasonable levels and I don't think there's anything coming for average users that can actually make it 10x better

long story short l2s are the way to go and L1 needs to be kept in a reasonable shape okay there was a second question here at the front uh hopefully not a bad question I run some notes for pocket Network and I run Aragon as the archive node mainly because it's so much smaller this footprint than running Geth is there are they doing some totally different or is that applicable here or is this some new technology thanks I'm not in stylish or I caught the question but let me make an assumption and then you can correct me so I guess

the question was somewhat around the lines that uh if Aragon is kind of smaller then wouldn't Aragon be able to run more transactions per second compare less and that's somewhat true the problem is that that's why I highlighted how much raw data we add to the network it doesn't matter whether it's gether Aragon the raw data is kind of that's the amount of data that gets added and the Marco Patricia 3 those operations per second input output operations per second those are the raw operations required to calculate the proofs now in the case of Aragon they do all those operations in memory but the catch is that it has to fit in

memory and once it doesn't fit Aragon also tries or also starts to leak stuff out to disk so again it's kind of like this trade of your in order to do the market potential calculations you you must do those operations how much you can still store in disk depends on on your algorithms but if this sorry how much you can store in memory but if memory runs out then you're falling back to disk and you're falling back to the brick wall but yes you have you might have a different Runway with a different data layout but but can we move some more questions would that be right I'm sorry thank you thanks

what was the reception of binance chains reduction of the capacity of the TPS and would that be potentially an option to vary the DPS um so the reason buying as much and as far as I know reduce the uh gas limit they are pushing it super hard the State Bar is going super fast and the thing that hit they had very very strong machines so they didn't hit the limitation that they couldn't process transactions what they actually hit is that they couldn't synchronize so if a new node was joining the network it just could never catch up and they realized that they just have to stop have to slow down

because they don't have the algorithms necessary to handle that throughput now again the question is binance margin is using a bit different Hardware they are really pushing the hardware I don't know so they reduced it from 25 to 7. whether this seven is because that's the maximum they feel comfortable with or I don't know why exactly seven but it definitely was that they were people had so many problems with running their nose that they just had to do something okay we have another question over here so I have a couple of questions so one

of them is you said at level DB so we all know that level DB is a sub-optimal of um data structure basically to store uh ethereum state so I have you looked into other Solutions or are you actively looking into other Solutions as an interim optimization a lot of people try to create an alternative database but it's not really easy to create a new database from scratch and the problem is that level DB also makes some trade-offs level DB actually is a very very good system because it it is very Compact and is highly performant you could make something faster but that

could entail just the disk usage explosion exploding and the question is okay are you willing to be 1.5 sorry 1.5 times faster but use 10x more disk space now certain people are certain projects are experimenting with creating their own database I mean I think parity at some point wanted to create their own database Ava Labs wanted to create their own database I haven't seen anything working yet it's definitely interesting thing but it is a very very hard thing to do and other databases that are out in the market essentially they are all uh all rely on the same fact the same try structure okay

level DB stores the keys and values together Rock CB stores them separately now we have Pebble which is kind of somewhere in between there are variations but as long as you have a tree structure with levels you kind of somewhere introduce this amplification back okay thank you and the second question is I heard at the time that there was somebody within EF researching uh decentralized storage solution like a bespoke solution to to do archiving is there anything for decentralized State Storage I guess as if you want to store the historical State that's very nice I mean you can definitely store it but there you don't have the requirement to have

instant access to it because I mean if you I want to I want to dig up what my account balance was last year I can wait half a second or one second or two seconds however if I'm a full node that one that needs to run this block now or I'm a miner and I have 100 milliseconds to create a new block then I cannot rely on um on some remote notes to give me the necessary data so it just the network latency will be too high for me to gather all the state data to be able to run that block okay I think we're up for time you survived it thank you very much Peter