Advancing Parallel EVM: Parallel I/O via Programmable Access List and Intelligent I/O Preloading

Transcript

[Music] the intro too early but yeah we're talking about paralyzation anyone knows about paralyzation oh only one oh God okay so there's going to be a quiz for you at the end of the talk about paralyzation if you win the quiz we will I don't know who's the conference host can we give them something if they win the quiz a voucher sckers a sticker no no maybe food or like I don't know beer or something anyway go ahead okay hello everyone um so it's my great

pleasure here to uh explore some our latest ideas on top of the paralyzation uh evm um so oops let's have to move oh Point here okay great uh a quick intro of myself has been working in this area oops for for about 6 years previously working Primary in um some big companies in parallelization high performance system SC PhD and so that's why I'm particularly interested in for examp eum data availability and luse all those scaling Solutions and got a couple

supports from Ean foundation so what's going on oh I guess I pressed I used the wrong direction that's why uh oh I guess this is the wrong link okay okay great uh yeah so why we want to particularly interested in parallel evm um so it basically would like to answer some ultimate questions what is the ultimate performance of evm uh if you don't have any overheads of the fix Amo consensus uh which happens like for example right now for ler all those sequences can just produce the Block in

almost instantly and so there will be a lot of question about in these scenarios what it will how will evm can achieve what maximum PPS uh in this setup um and also another question is that are we able to build the evm that even faster than like for example sonana and there's a lot of this question flying around and so and people say hey um why not use some like web to experience to accelerate the the execution especially uh right now we are heavily relying on for examp paration to in order to speed up those

transactions so in the parallel evm there's generally two categories one slice in called parallel execution tion or sometimes called optimistic execution which basically to execute a sequence transactions a future transactions for suppose we have AB BC then the normal way is I execute one by one that's a called sequential execution that's have a b c but is slow then the parallel execution say hey why not we are able to running a at the same time we also execute B and C using the the the state that the a actually is executing and once after executing A and B then you just double check if there's a conflict

of re and right set and if not then it can just easily merge the execution result of B with the A's result so that we can get a corresponding uh results and that is able to better to utilize the current paralyzation concurrency feature of the of the node and another Direction is called parel IO uh which aims to reduce the latency and boost the performance you uh using um some technologies so why we want to uh like study or accelerate IO of the EV of the ethereum so here is a really nice chart

from uh Meb uh that is Flash BS that shows what what are the the major normalized runtime that would be used in the uh basically in the ebn execution and this is using the main net data so you can see the corresponding broad numbers and so they list a couple of the major op codes and the last one here is called ops low which is basically reading 32 bites of the data from ethereum get by given a key for example I read a I use balance account or read the ownership of the for examp mft images and this all this access code

actually accounts for approximately about 70% of the evm execution which means that even though we may be able to accelerate the for example other executions like for example by using uh a better way to interpretate the op codes or other ways we still have a large amount of overhead in this ASO operation why the reason is that the EVN IO patterns is very unfriendly for the iOS first of all a lot of the io are random access patterns which is hard to predict for example when I read a balance of Unis swap balance of some for example the pool uh balance then the

solidity actually will generate a random position using K and the corresponding key to randomly okay a position of the data in the in the database and such kind of a random access is really really unfriendly for the underlying database uh uh storage and second the Patricia mle tree that if now uses further worsen the case of Random Access because the random tree structure is also highly randomized and so in order to solve this problem um there are a couple of the pioneer ideas that aims to say hey whether we are able to um figure out way

to reduce the latency that R read the random positions one thing is to hey rather than just have these random positions or locations during the ebn execution why not we are able to using token in kind of incentive to let the sender to specify called access list which can tell the EVN that hey there are some data that I'm going to access ahead of the execution and now the evm can parallel loading this data from this access list so that when reading this data in in the later is already in the cache So reading the data will be

generally in the in the cache with generally uh takes uh zero latency and hence we are able to significantly reduce this IO speed and so in our test is can uh enhance the evm execution suppose we have full access list we can increase the performance about about twice uh with all those a very simple strategy and this is really nice idea and this is also proposed by VC uh about I think two or three years ago but unfortunately there's a recently there's a paper that is written by eth zich is a university it's not ethereum suborganization so it's there's a really

good paper saying that they do a lot of study on the mainnet optional access list and they found even though there's a great feature to accelerate the the basically the the transaction execution is highly underutilized on the main net with only about 1.45% of the transaction utilize this feature and there are a lot of reasons one reason is that because the current incentive of incorporating this OB access list is very very minimum because it increase the core data price because

have more data then we the E needs to broadcast all these access patterns or access list to network and this have extra cost and and so a lot of users found oh the if the corresponding cost of broadcasting this up access list is greater than the cost that a gas savings that that this EIP brings then definitely nobody would like to use this feature and also there's a couple of things because it's a lot of client execution client return some wrong option optional access list and so and then it may incur like even worse result

so if if you are interested in this topic especially like parallel execution I think this is really excellent paper to take a look and a couple of ideas which actually highly aligns with what we are going I'm going to talk about so in order to adust this these problems especially under utilization of This Ss list feature um there are a couple of things that we are working on uh one is to propose a EIP that 7650 which is a really good name easy to remember the idea is basically say hey rather than have the user to encode the access list in the transaction itself why not we allow the program the

contract to tell that hey I'm going to read this random locations of data but at the very beginning and so that the contract because the contract has the full knowledge what the access future access part pattern of the execution will be then we are able to harvest the parallel loading and reduce latency and further the sustained benefits because every time it execute this contract it will load using this parallel access lead programmable access lead to load the correspondent data so the benefit will be sustained another thing is inspired by reason

Ai and we also found that we are able to using some way to find a common access pattern so that we can be able to even without like program this data but we are still able to learn what are the common random locations when a execution like smart contract call will uh will read and then we are able to read this learn this Spin and then load this in the future uh execution so I will explain all this to you in details so first of all before we dive into details of how how we are able to implement this uh parallel iOS one thing I would like to uh highlight is about what are the

underlying storage how many iPad or iOS the underlying storage can offer so here I just give a a commercial uh commodity uh 4 GB M SSD which is about $300 I believe every know can can offer and so I run some parallel IO program uh which is the famous fio and so this will tell us hey how many iOS that I parallel issue to the underline storage and what is corresponding latency that for each Random Access so that we can tell that how much pariz we are able to achieve in terms of IO using the commodity like dis

and we can clearly see that if even we just increase number of random iOS to like example 12 more than 10 the number of the latency increase is just very very uh minimum and for some numbers like four is almost basically the same which means that if I can issue I suppose I sequentially read this random data or I run this data parallel in four iOS they actually use the same latency when reading all this data which shows the current capability of the MMS that allows to high number of the random uh parallel IO and random

access and so definitely you can also using the famous fio tools to reproduce these numbers and but this tells us what is Ultimate optimization goal that we can achieve using a simple commodity PC and definitely we can using even higher Advan say using rate zero or using more advanced ssds but this is some number that we can easily get so the program for Access list is allowed a smart contract to basically to specify which part of the data you will be accessed ahead of the actual execution so let's take a Unis swap swap as an example the Unis swap each time it will read five uh five

uh that is the storage slots including the the pair of the address pairs of the tokens and Reserve which is basically a compa version of their balances that is the the token that has deposit into the contract and price Z price one accumula last Des is using to calculate some uh think statistics and some numbers and so that they are I think used to distribute some fee all those things to the uh basically LP and also uh yeah the the lp I believe so these five storage slots will be constantly

read when calling a Unis swap swap so suppose now we have a parallel IO loading in a smart contract and also corresponding OB Cod that is supported so now we are able to start when we start this call the Meer we can say hey I'm going to read all this data let's just prefresh all this data and ask em to say I I want to read this data in the future please read this data ahead my execution and this can also okay prefresh these two contracts to tell us hey I'm going to call these two contracts for example I want to check

the balance of the call token zero and balance of token one of this Unis swap contract then please load this contract ahead before I call this and so we can list as much as we could uh it can be combined with example some parameter that is input um so that we can also load to this is you can see that this is as fixed locations every time is fixed but definitely we can combine with some input parameters so that we can know that what is the future access by the by the uh by the smart contract and every time when they call the smart contract will automatically load all the St data par in parallel so how we are able to uh with this now basically reduce five random

access into one random access suppose we are using some devices with this high performance with this latency because we we can see that there's no latency increase with about four or five iOS and which means that we are able to reduce this atos uh for example suppose is 70% of the execution then we can reduce it maybe half or even less because we are using the underlying uh parallel IO capabilities of dis and also we assume the gas is is the cost of the basically preloading is the same par preloading is as same as a one sequential loading then we can enjoy a constantly gas reduction

uh for example right now one random access of the storage store it takes about two ,100 gas price gas then we are able to save to single single 2 uh 100 suppose we assume this gas price is the same but definitely there's a rooms to tune but suppose we are have this safe assumption then we can easily save um thousands or tens of thousands transa uh gas per transaction um so this is the basic idea of parallel uh I is you can imagine it's kind of like traditional programming with kind of parallel like threading all these ideas in in program by itself and

at the same time uh we also observe that all these random locations they are constant for each unit call and even without for example I tell that whether this location um they do not depends on any input parameters so the idea is that would be possible that we can have the evm to learn those common read patterns associated with a method in this cases is the swap method which can be easily figured out by using the method ID according to the API encoding so that we tell that for this this method this is common access pattern for this method

maybe for mint this is another access common access pattern so that once we are able to learn this and once this is become stable then we can easily using this idea to for example every time a color method find the access list call another method call method next time it will basically filter out this find find out the common list and then for the future they can use this final uh this list to basically call all the smart contract automatically without having any for example introducing new eips just purely EVN optimization and that's basically idea what we call it IO preloading as we have observe a

lot of patterns in allot of smart Contra which has this constant common access patterns in the uh in the smart contract call so um here uh here's my concluding remarks we in this talk we explore to Innovation Innovative strategies which aims to uh Bas uh to reduce the io latencies by enhanced access list um which post proposed by V charic the first idea is that we are able to allow the programmer to specify what is the future data data locations I would like to read and please read this data uh ahead uh par in parallel so that we can greatly reduce the uh the the latency

when we read really read this data because these data are already in cash and the second idea is that given the method ID and also looks contra then we are able to learn to identify what are the common access pattern of these contracts and then just can automatically uh got the benefits of this parallel uh iOS and so we hope that all these Technologies uh once we become mature in the future evm we are able to significant accelerate um the evm execution that definitely together with other features for example that is the like just in time comp that Mega is that is developing so that you can reduce The

Interpreter time to uh interpret the I IO codes and then as a kill so there are a lot of interesting idea that's flying around and that's all of my talk today and thank you very much I'm I'm happy [Applause] to anyone has questions yeah questions please yeah I want to ask so paralyzed EVMS has recently become more of a narrative in the context of of other chains so I was curious given that you're kind of closely familiar with this if you have seen some uh chain or um implementation of this that you think could be long-term viable in terms of

connecting for example Solana and uh ethereum because there has been neon eclipse and different kind of uh attempts at achieving this so if what do you deem the best effort um at this right now um so the question is about what is the best effort to make uh parallel EV in in fix in production right or success um so first of all I have like a one is according to my experience previous experience trying to fix a lot of par paration bugs uh in general takes a lot of time to make concurrency works correctly so first of all it's really hard to debug and also it's it's really hard to um make sure that the code runs

correctly and also fix them WR writing the test um so I think right now is is we are at a little bit early stage one example I can tell you is that um so polygon has Implement their version of the parallel evm uh in terms of go with go based on top of gas and so we take a look at the the code and study how it works it's basically idea as block STM uh solve the transaction memory uh using the optimistic execution um one but two things we observed we found a couple of bugs and fixed them and second oh looks like this

are some easy bugs and I say why not it's happening in the production and then later f actually is still in experimental Stage IT haven't be used in the the D for example main net and so it it helps to answer some questions what is the ultimate performance game using all these Technologies but still in order to run in production then um it will um I think it will take some time to uh especially for some like critical applications I won't imagine if the L one will use it because right now the botonet is consensus it's not about the iOS but as more and more

um like U layer tws are coming and definitely they are able to harvest benefits and also there's the idea of call diversifi client right suppose I have a client that runs parallel EVN and no other client can catch up that I have no idea when it enters some something that's wrong I I cannot tell because nobody can tell me oh sorry the transfer balance is should be supposed to transfer 100 East but because some par EV scheduling it can just transfer 10 e but with the diversity Cline so we have two implementation of client but each of them can catch up with each other and then it will be much easier also to

figure out the problem this is the same issue applies to ZK proving because like ZK proving is also using the diverse diversity uh Diversified ZK Pro so that if some something is wrong then we can use a multiple Pro to make votes rather than maybe people just using the multi nature so there are couple of things to to adjust this problem but as a infrastructure I think it's still in a very early stage we need a lot of excellent Engineers excellent uh researchers to help to solve solve this problem thank you all right great uh from Smart contract development point of view do you foresee any downsides of using this uh pret like uh

are there any reasons why every new smart contract will not just like pre prefetch everything that's a great question so um I have developed a lot a lot of smart contract one's concerned that as as as a smart contract developer is the gas because I really want to have other applications other uses when cors the gas can be save in a really really uh um uh optimized way that's why for example uh open C when they try to optimize their Dax on chain they use it assembl C rather than

solidity it's that because they have money to do this extreme optimization and for us I think if there's a proper way to reduce the gas using this explicit gas gas metering to save this gas and especially those parel I ization execution takes a lot of like execution run time a war time and and and then and then um so that like from the evm perspective and also developers perspective there's a inch alignment so that we want everybody to implement this feature so that smart contract developer can save gas at the same time EVN can

can process more transaction so I think this is the ultimate goal that win wi goal that both developer and EVN protocol Dev velers can be Harvest from this approach but how to determine the gas for example how many par loading um that we are able to accept whether it's four or five or maybe 10 that still leave a lot a lot of room to discuss but I don't think that should be um um impossible because we see that like 300 mm dis is already be able to support a large amount of P yeah another one from validator

perspective uh do you think it will increase the load on validator Hardware will uh will validators have to upgrade somehow or like what what is running now it will it will suffice so the question is like whether was the validator maybe have created concern that increase their their nose right is that yeah I mean uh the load on their Hardware if they will need like more I don't know more storage or or more memory or like or yeah basically yeah um so that's is really good questions for example recently Arbitron they are having a issue that they because they process too many transactions so a lot of other notes are

not able to catch up with their latest transaction even they just running all those transaction 100% because the sequencer or abiton sequencer has much higher like a uh how require devices compared to others and so that is really good questions and uh actually we run a lot of Ean notes and to do a lot of experiments and I think I found there are some Cloud providers that offering some ether friendly devices with pretty much uh I would say really high performance like 128 GB minutes with 16 cores and about 7 terab rate zero SSD

can offer 7 GB read and right that takes about 7 70 EUR per month and I think that should be acceptable and or something that we are able to design for for this spec from our protocol designer perspective that's my answer yeah uh okay uh I more I was asking more about From perspective of uh home validators who run like uh on who who have Hardware from the on the lower side like of performance so like rock pies and and thing like this or arms uh yeah so you mean the whether it

increase the cost of home validator yeah exactly will it increase the cost of running uh of running home validator yeah basically yeah who who don't have like big big servers or who don't pay like to cloud services uh depends on what kind of home validat like like for me I kind of like a geek for for computer I assembly those things by myself I don't I don't think it is hard to buy some cheap Comm like commodity devices like for example uh I I I I I have set up like four SSD using n Samsung 990 that which can easily offer 10 10 GB read and right um with

maybe less than 10,000 uh 1,000 um so um but definitely actually is really great questions whether I suppose I I would like to run it in the cell phone or or some rock p is very common sorry Rock pie arm Arm based yeah Rock Box yeah I I I do think right now is using the modularity layer Solutions so that for Layer Two they can run as fast as possible but for layer one like for example uh we can have work trees denness list like Milos is also working on that so that we are able to allow uh example maybe I watch or some devices to vote at that time um the PN actually is not the major issue like astion is not a

major issue all right thank you so much she did we learn about paralyzation after that session okay great thanks I learned something too all right um next speaker is toia yes