All the things I wish (native) Solidity would let me do - Hadrien Croubois

Transcript

Can you hear me well? Yeah. Uh, so the title is a bit of a lie because we will not have time to cover everything, but we will cover a few things. So, I'm Adrian. You may know me as one of the developer of Open Zeppelin Solidity Contract. Yeah. Uh, we just released our version 5.5. Uh, I hope you like it. I hope you can can try it. I put a lot of effort into it and a lot of of our heart and our our energy in doing new interesting thing. Oh. Uh so the opening contract is something that is built and designed by solidity developers. Everybody in the team has some experience in doing solidity development. We love going to hackathons and so on. And it's developed for solidity developers for for you guys.

But most importantly, it's developed and implemented in solidity. That's one of the key features of our library that maybe separates it from from others is that we take a huge importance in the fact that what you are building is actually built in solidity so that it can easily be audited, reviewed so that you can understand and so you can use it as a tutorial on on what to do and what to not do. But are we really writing it in solidity? Last time I checked, not really. Like this is an example of a proxy contract like our base proxy and you'll see this big chunk in the middle.

That's not really solidity. That's that's assembly. That's fuel. And I think that's fine to have some of it. But over the library, we have 400s of these blocks. Most of them are procedurally generated because we don't have generics. So it's not that bad. But it shows something. It shows that there are things that the EVM is able to express. There are things that the EVM is powerful enough to do that the language doesn't give us access to. And we have to bypass the language. you have to bypass the solidity to direct to go lower to go closer to the machine not directly into bite code even though sometimes we do that but at least going one step below and so I want to go over like like why do we use assembly where are the places where the language is not

good enough and would we be able to improve the language so that we don't need to resort to assembly and maybe solidity core is part of the answer I hope so but but let's go over that the most important part I think is for low-level calls because in in native solidity you can either do a basic call and if there is a revert the revert reason is bubbled and that's fine for possibly 95 to 99% of the use cases and when you don't want to do that you can do a try catch and the try catch is quite nice but the try catch will fail and not catch like actually fail if the target is not a contract and so if you want to use that on arbitrary addresses you have to check that the contract is not an address first implement some kind of fallback and then solidity will

recheck the same thing and and it's and it's a nightmare and we also have the low-level calls which are solidity is like solution to go low level but even those are are not great first mean you have to encode the inputs and encoding means allocating memory and we'll see that that's an issue a bit later and also like the result we get are are in memory and that means we expand the memory even when we want to discard the results and overall Like those are nice in many cases but when we want to do something like super optimized when we are chasing every single bit of gas because this is going to be used by possibly everybody in this space uh then our best option is to is to go low level

and and as much as I would love this safe transfer function to be written in in in in solidity native solidity we've tried our best and there is just no way around it like writing it in ule is just more effective and it's way way less readable. So we have to to to to do some some some some choices here and our choice was to go against readability and into performance and I think that's that's a bad consequences of of the design of the language. So this is something I think should be done better and I don't have any proposition on how to do it better.

I think it's very very hard to do. It's just yeah this is somewhere where the language is not ideal and then we go into memory. Memory is nice. You've seen it can be used to go around stack to deep error but memory even though like a single M store or M load are like basically cheap or free like three or five gas basically insignificant what's expensive is is allocating memory and this is a piece of code that we have in one of our governor what I found while while preparing these slides where we actually have a for loop and inside the for loop we compute a hash doing an ABI encode yeah here and this ABI encode we will will allocate memory possibly a lot of memory and and just in order to hash it. So after it's been hashed, we don't

care about it. But the thing is that unless we have this special call to our memory library, solidity by default, we just keep the free memory pointer moving and we'll do that over and over over the for loop. And even if like actually doing one encode might not be that expensive, expanding the memory over time, this is this is really bad. this is going to hurt your gas cost and solidity doesn't know whether this value is going to be reused or not. So it's assumed for the worst case it assume it's going to be required. So it's going to reserve the memory and move the free memory pointer and the current there is currently no way in the language to say oh this buffer that I'm allocating I want to deallocate it and when you want

to deallocate it you want to deallocate it fast because if you don't do it fast then other stuff is going to be put after that position in the memory at the free memory pointer and it would make you won't be able to deallocate it anymore because the memory in solidity is just linear I mean not in solidity in EVM is linear so so that's bad so yeah ideally we would have a language that would support better memory management. Maybe using like I don't know maybe modifiers or or or or or annexation like like a status from the type keywords on the type saying like please deallocate that and deallocate it fast. Um there is a bubbling revert with solidity. You are able to revert with custom errors. This is great. Thank you Solidity for adding that. Uh you can

also revert with nothing or with a string but sometimes you just want to bubble the return that you got before. So let's see we do that by default when you just do a simple call but if you do a try catch and you want to look what is in the catch and in some cases you want to process it. In other case you just want to bubble it. Solidity doesn't provide you a way to bubble a bite like you could if you start reverting with a bytes object. Solidity will encapsulate that inside a narrower bite which is basically the the first customer that exists. The customer that is used for everything that is not um a customer actually maybe it's a string. We have to check. So we do that in in assembly. We do that quite often. Honestly like adding that to native solidity would be

a no-brainer. I think like if we were doing that in solidity core it would probably be a two or three line function but but we don't have access to that by default. Maybe it's something open zeppelin could provide. Let's think about it. But in that case, it's a bit particular because we are returning something that that we are we haven't even allocated properly. We just put that on the memory and this is not memory safe. Uh decode call data. Okay, sometimes we want to AI decode stuff. Uh so lets us AI decode objects that are in memory. If you provide a cool data object, it will be put in memory first. And if the AI decoding fails, you get no way to catch that. That's one of the things that we think is really bad. We would like to be able to to catch it. Maybe in 3D core we

will have a generic type like sorry an algebraic type for that and that will be great. Uh but sometimes we also know that the input will be in code data and we just want the output in code data because we don't want to allocate memory because memory is so expensive to allocate and so what we want is we want to take the object like go deep into like how it's encoded and this is always painful to do. So if could do it natively it would be better and even if we do it ourselves in the end we still have to create the the the cool data object like data object is basically a pair of integer which is an an offset and a length a position like a slice in the cold data but like even if we wanted to do that in full like native solidity

we we have no way to do that last block without going to assembly. So yeah, I mean the the best option would be to have an AID called call data natively, but we don't have that uh em panic code. So that's going back to the the revert things like there is not just revert like a custom and so on. There are panic codes and panic codes are great because panic codes are explicit about what is failing. If you try to pop an array that is empty, it will tell you it's not possible. There is no item to pop. If you start like doing like division by zero, there is a penny code for that.

when you when you do checked arithmetics and you check for overflows, those are the panic code that are emitted and we want to use them. We want to use them when we sorry when we build our own data structures that are not native solidity. Maybe at some point they will be generic in solidity core. I'm really hyped by that. But that sometimes like we we're trying to pop something from a queue and the queue is empty. Well, we want to panic and and so the panics are native solidity to solidity, but they are reserved to like the compiler level stuff as user. We cannot emit them unless we do it manually in assembly. So that's what we do. But ideally we I mean maybe we maybe it's best that open provides that for everybody but I don't

know maybe the language should allow us to do that. Uh unchecked access. Okay, this is something uh because you know the unchecked keyword unchecked keyword is for arithmetics. It tries to remove the checking for like multiplication and addition and subtraction overflow. Divisions by zero are still checked. Like even in an unchecked block, if you try to divide by zero, EVM will return zero. But if you do it in ule, you'll get zero. But solidity will give you a panic anyway. And what solidity will also do is that when you do a lookup on something that is an array like a by32 storage array like that if you start accessing it in solidity natively solidity would first load the length of the array check whether your index is

within the bounds of the length and if it's okay it will return the value otherwise it will panic. The thing is that sometimes you want to do a for loop and you want to go over the entire array or even if not the entire array a large section and so every time you go to a you you look the next value it will redo the check even though like we know that the lens is okay but but solidity doesn't know that because solidity doesn't have the high level logic of this loop and so they will check every time and every time they check they have to load the value from storage which even if it's a hold value has already been accessed 100 gas plus do a if to compare it that's maybe 30 gas with a panic code that hopefully won't be

emitted. Uh and when we know that by design that the loop is the bounds are correct well we want to skip this checking and unchecked doesn't do it. So we have our own implementation that does that and and yeah it's a bit of a pain to do this unchecked access ourself uh because we have to compute like the position in the same like it's it's a bit of a pain but but we do that anyway and and we are saving a lot of gas with that. So again going back to solidity core maybe we can have a container algebraic data type with like a for each function that will feed you all the values one by one and it knows that it's inbound so there will be no checks but we don't have that today. So today we we

have to to go do our way around it and and it's not ideal because you know here like it's not actually fully memory safe when we loot from load for memory because like in most cases if the input of that functions are okay then it's memory safe but with that function alo if alone if you don't have the call context you cannot know whether it's memory safe or not. uh chunks of bytes. Yeah, if you have a bytes buffer like this, this is a huge like basically a huge buffer of of data and solidity lets you access it one by one like one bite by one bite. So one value between 0 and 255 one by one but memory word in in EVM are 32 bytes. So you could go 32 times faster by loading them as loading them

as chunks. So lets you do that with cold data buffers but not with memory buffers. So if you want to go fast and have this loop go like 32 by 32 instead of one by one which goes much faster then you have to to read the buffers yourself and again this is a bad example because previously I said that this was not memory safe and this should for the same reason not be memory safe but here in that case the function is private and so we control when it's being called and that's how we make it memory safe but that means this function is probably declared three or four time in our library because we don't want to make it available And so we have to put it privately wherever we use it.

Uh transient structures. Okay. So I mean solidity is moving and added transient value types which is already good. But as far as I know most of the usage of transient values like in um in unis swap v4 or more complex objects than just that like they use mapping they use I for I want I want to do enumerate set in transent storage. I want to do dq in transent storage. we would actually have use cases for that and right now we cannot. So in some cases you can like use our libraries we have der derive mapping function we have the tto library when you combine all of that we're able to mimic mappings and things like that with with transient in the transient

space. Uh so we have a way to go around it but it's not ideal and in particular like all the structure that we would want to build like big innumerable mappings and things like that in transient storage we just we cannot build them right now I mean not it it would be a nightmare to build them and I think the language should hopefully supports that uh I'll have to see how solidity core uh handles transient object maybe the standard library will have to deal to deal with that anyway um so I mean I I don't want to say that things are trivial or hard and it really depends on how you want to do it but there are really low ending fruits here maybe the transent structure is I mean I'm saying it's a low ending fruit I tried modifying the compiler it was a

pain but uh honestly it's it should be a lowering fruit because there is no more logic to invent I think the main reason why transent structure were not implemented early on by solicity team maybe solid team will contradict me or the ar collective will contradict me but I think the reason why that when you use normal storage you try to pack stuff a lot to save on slots and in the transent phase, maybe it's a waste to pack too much. Maybe you want to expand a bit more because the storage slots are not that expensive and the packing has a cost. But honestly, like I'd rather have have something that is packed for no reason than have nothing. Also, since it's transient, if you change the logic later, you don't have to like keep the same structure because it's erased at the end of every transaction anyway. And

there are things that I think would be easy. Maybe there are not. Maybe so has a good reason to do it. But I think it it should be decently easy to do. And there is stuff that is hard like low-level calls. I have no idea how to fix that because the expressivity of the language would have to be very very complex for all the use cases we have. And the deallocation of memory, it can be quite easy if the user lets the compiler knows when to deallocate and the compiler just blankly trace the user. And if the user messes up, the code is messed up. But if the compiler wants to do checks or does do it automatically, this would I think be very hard. So yeah.

So can 3D core help? I've mentioned 3D core why in this talk and with all the information from this morning I want to mention it even more. I really hope that it does. Uh the thing is that everything I mentioned is stuff that we do having access to u. So if we can do it today having access to u there is no reason why a standard library in in core could not also do it. And I think with algebraic data types, we are able to to have a control over the flow of the execution and and with also with lambdas that have been mentioned, we can do stuff that I think would would be really useful. So I'm really really hyped by Solidity Core and I hope it gives us an opportunity to contribute to the standard library or to I mean I would even be ready to take care of the

standard library if that if that helps the community and really like do things right and and have a language that is more more readable and more efficient in terms of gas. That's everything I've got for you today's I'm open to any questions. >> [applause] >> Thank you so much Adrian. Um as I read out some of the questions already trickling in more questions you can post using the session QR code. Um when open Zeppelin standard library >> uh so if it was up to me it would have I

would have started four years ago when you announced in Amsterdam. No, but more seriously, uh I would love to work on that. Uh but as uh as was mentioned earlier today, this should be a community thing. Uh and I don't know who how it will be governed. I think the governance of the standard library will be a major thing we'll have to fix in the in the upcoming months and years. But I personally want to commit all my energy and possibly open Zeppelin's energy to to being part of that effort. It's really something we want to do. But we don't want to exclude I mean we should not be able to exclude anyone anyway. So obviously everybody will have to be welcome and we'll have to deal with how decisions are being made.

>> Any more questions for Hrien? Any from the crowd? >> Okay. Well, thank you so much. That was a wonderful talk. Please give it up for hatred. [applause]