Glamsterdam Hard Fork: Out of gas, still running

Transcript

[music] [music] Hello. Um, I'm Yanis. Uh, I'm a blockchain security engineer at Chen Security. And today I'll talk about Glamsterdam hard fork and how it's possible to believe that your contracts or your execution run out of gas but actually it's possible to still be running. So actually this finding is what led uh the Ethereum Foundation decide to uh postpone the deployment of clamster on hoodie. But before I dive into the details of uh our findings, uh let me

give you a little bit of a background about chain security. So uh chain security is one of the OG's in the smart contract security review space since 2017 and we had the chance to audit some of the most important protocols out there. And as a matter of fact, it's not the first time we delay a hard fork. Uh the same thing happened uh in 2019 when another bug uh caused a delay of uh the constant upgrade.

So um it was already known that the glam drum upgrade is going to break some smart contracts and actually the folks from the AF uh were already aware of and the assumption was that only honest work non-malicious transactions uh will fail to to work. But uh my colleagues uh flipped the question and they asked is it possible for malicious activity that was not possible before Glamsterdam to currently be possible post Glammsterdam and it turns out that uh the answer is uh quite interesting and this led to a

big investigation collaboration with the Ethereum Foundation uh where we found a lot of instances of uh issues that uh are caused by Glamsterdam and we are currently in the process of mitigating and responsibly disclosing uh to all affected protocols. So before I dive into the details of uh Glamsterdam and the EIP that caused this issues, uh let me give you a little bit of an intro so you're able to to follow better. So many smart contracts in the wild have the the need to continue executing after

a failure has happened. So usually a contract makes an external call to another contract and then this call might fail and then we don't know why it failed. Did it fail because of some logic error or did it run out of gas? And in order to be able to to follow all the issues, I think we should introduce an attacker model. And in this particular case, it should be a permissionless function than that can be front run with lower gas by an attacker. And this we want because only when we have when we run low in gas is it possible to to trigger such issues. And this attacker model might

seem a a little bit arbitrary and maybe hard to to find but actually it turns out that a lot of bridges and metat transaction and protocols that enable metat transactions are susceptible to such issues. Um so to understand uh the issues better because we are talking about external calls and gas maybe we need to remind ourselves a little bit of EIP 150. Here I'm oversimplifying but the idea is that whenever we make an external call we forward some gas and the default is to forward uh the 63 over 64 of the initial gas and when the call the external call terminates we

are guaranteed that we have uh at least one over 64 of the gas left. And then naively we can think about it as whenever a call terminates uh because of out of gas then we at least have one over 64 of the initial gas. So to the actual EIP uh the AP makes introduces a lot of changes. The first and the biggest one is the so-called CPSB the cost per state bite. So for example for uh cold storage to a fresh slot and by fresh I mean a slot uh in storage that is zero it was zero before the transaction begins uh a

user under Osaka would only pay uh 22,000 but in Amsterdam it will pay more than 110,000 and this is a big change and to explain a little bit how I decompose here the 110,000 there is a part uh uh that I will touch on later and is really defined on the EIP838 uh and the 97,000 comes from the multiplication of 64 bytes which is a slot in storage uh in on Ethereum times this 15 uh30.

So and this brings me already to the first issue. So transactions now require more gas to run, right? Um and many contracts already break and these are the contracts that the EF was already aware of because many contracts have a a fixed budget. So they check against this fixed budget and if they have enough gas then they run uh the external call. Of course if the fixed budget is determined pre pregamsterdam then such calls will not work.

But the most interesting finding is if we think again of the reverse case. So if contracts if external calls need more gas to run this means that they can run out of gas uh with when higher gases is supplied and this means that we can still run out of um gas and have more gas left to execute operations. So if I if I take a very naive example let's say I call I make an external call and I assume that I I have a lot of gas but still uh the call fails then premster pre-lamsterdam it wouldn't be possible

to make any uh additional operations and successfully execute the transaction but in the postclap case I can for example emit a success event. Of course, this this is a very simplified example and someone could argue that one should check uh the success of the call with uh the variables that are returned from the from the low-level call itself. To to understand uh the the other issues, we need to dive a little bit deeper. So, the EIP837 introduces the reservoir model. What is this

transactions? Now the gas applied to the transactions can exceed the transaction max gas limit and any additional gas on top of this uh cap of 16 million gas is accounted for the reservoir. This means that we essentially split the supply of gas into two different parts. Right? The gas left, this is the known part. we already uh have before Amsterdam and the reserveware which is the newly introduced uh feature and external calls as far as the gas left part are is concerned work exactly the

same. The difference comes uh with the reservoir. So when we make external calls only uh we supply the 63 over 64 of gas left but we still forward the full reservoir. And what is more interesting is that the gasop code only returns uh the gas left part. It completely disregards the reservoir. So why what do we need the reservoir for? Um so each operation now consumes two types of gas if I'm allowed to say there is execution gas that is consumed

out of the gas left and there is state creation gas and the state creation gas we first consume it from the reservoir. If we don't hold enough reservoir, then we consume it from gas left. And this is the so-called spillover and we increase a state gas from gas left variable. So we remember that we spent uh gas left to create uh storage. So to give you a quick toy example, let's consider only two operations.

Let's consider the addition. This doesn't create any storage. So it only consumes execution gas. So it will only consume from gas left. And the stor on a fresh storage. This consumes both state gas, let's say 100k, and execution gas 10k. So if we were to walk through uh this very simple example, h the addition will only consume gas gas left. So we will only uh reduce gas left. But then when we get to the sore the first time because our reservoir is enough, we will consume uh gas from the reservoir to create the uh the storage and we will also consume some gas left.

But in the second the second time we try to to write on storage, we don't have enough reservoir anymore. So we consume the maximum allowed from the reservoir and then the the leftover we're going to consume uh from gas left. And of course because we consume gas left we need to remember to increase the state gas from gas left. And this brings me to the second issue. So let's consider this particular example. We we want to execute an operation of a specific ID as long as the operation is not settled and then based [clears throat] on the result we either want to set it done or we want to set it failed.

Now with the introduction of the reservoir, the reservoir can be quite big and the external call might not consume big part of the reservoir but it still can run out of gas because the out of gas is only uh concerned with the gas left part. This means that when we come back from a call that ran out of gas, we might still have a lot of gas to create new storage. Before Amsterdam, this was not possible because all the gas was consumed by gas left. Now however one even though that the call failed and call it failed

because of gas can still write uh on the else part and essentially an attacker can now grief an operation and make it uh seem like it failed while in reality it just run out of gas. And I want to clarify something here. It might look a little bit uh contradicting uh when you think about the first issue here. I want to emphasize that the execution requirements are lower.

So the execution cost of an operation is lower but the total cost of an operation is still very high. But to run out of gas, we just need to consume the operation part of the the execution part of the transaction. to to understand the next issue we need to understand refills first. So because the EIP makes an effort to be very efficient with storage. Anytime we reset a fresh slot, a slot that was initially zero, then was set to a value that was not zero and then it's set again to zero in the same transaction, we have a refill. A refill first is given to gas left and then to

the reservoir. So if I'm to follow up on my example, um let's say we want to reset uh the slot A. We notice that uh we have some state gas from gas left. This means that we consumed some gas left to create some uh storage before. So we increase gas left by by 50. And because we in total paid 100k, we need to increase the reservoir too. Uh here I disregard uh what we need to consume to to execute uh the store this 100 gas.

Then it it gets easier because now we don't owe to gas left. So in the second reset that happens, we only increase the reservoir by 100k. Um, and I really want to emphasize this red part over here. If you notice here, gas left increases. And it's the first time this happens. Uh, before that, before Amsterdam, uh, gas could only monotonically decrease. But here you can see it increases.

And this breaks this monotonicity assumption. So if I were to expand a little bit on my previous uh example and introduce some storage variables that represent uh some locking. Now once the target call uh finishes because we are resetting the storage slot suddenly gas the gas left increases and suddenly we are able to persuade the contract to convince the contract that we didn't run out of gas while in reality we did run out of gas.

So this breaks a lot of logic that is already on chain. And uh for my last issue, we need to think a little bit about failures. So failures in the context of uh EIP8 uh 37 are essentially storage rights that were never committed. Right? So because h during a revert the state change is not committed this means that we experience the same re refill uh operations that we discussed before. So we reset the gas left from state gas

from gas left and we set the reservoir to the baseline exactly be uh to the value before the external call. And this brings me to my hardest and more complex slide. Let's consider three contracts. We have our own contract that calls an intermediary and this contract creates uh some state. Then this intermediary calls another contract that fails with out of gas. Then the intermediary catches the the failure and reverts. This means that the state which was created at the beginning now uh is never going to be committed. So we

experience a refill. So when the con when the flow comes back to the contract to the original caller even though our intermediary never failed out of gas but simply reverted we end up in a situation where we have more gas than uh we would originally expect. And this is an issue that we find very often. Uh we have found a lot of instances of it in our research. So to summarize, the EIP 8037 and 8038, but mostly 8037 make malicious actions possible that were not possible before because it increases the gas requirements.

However, reprices the operations. So some operations might uh consume less execution gas breaks the monotonicity of gas left. So suddenly gas left can increase uh in the middle of a transaction and causes uh increases in gas when reverts happen. So if you want to check your code, you should scan uh this barcode. This includes a skill made from the Ethereum foundation. Uh if you want to reason about your code and whether you are safe, you need to revalidate your fixed

uh budgets. You need to revisit what assumptions you made about the gas when an external call has failed and you start you need to start thinking about refills and how you're handling failure. So if you want to read more about it, feel free to visit our blog and if you have questions, you can mail the Ethereum Foundation directly because they really want to support the community and make this transition as uh seamless as possible. So thank you very much and if you have any questions have to take them.

>> Thank you very much for the talk. So uh we know now what can go wrong with the contracts with gas left. Uh let me check if we have any questions so far. Yeah, first landed which is good. So, uh let's take the first question uh for answering. So, uh why is the if issue number two, why is the problem if the call failed OG or reverted for another reason? If failed in both cases, shouldn't it be handled the same way? >> No, it shouldn't be handled the same way. I mean this is up to the protocol to decide but some protocols want to figure out whether a transaction is worth making or not essentially whether an external call is worth making

or not. So if we run out of gas it means that we can supply more gas and the transaction the external call can still succeed. However, if it fail for another reason then it means that it's probably never going to succeed and we want to completely ignore it and disregard it. Right, thank you. We have one last question which was uh submitted by me but uh uh anything else uh go wrong with the upgrade? >> Uh we have looked through the whole spec change and I think this is uh the most important uh way you can get impacted by Glamsterdam.

>> All right then. So thank you very much. >> [music]