When One Dependency Breaks Everything: Securing the Web3 Toolchain

Transcript

Perfect. We can begin. Hi everybody. Uh so as I mentioned, my name is Ray. I'm one of the team leads and a security researcher in Stora. Uh today we're going to talk about when one dependency breaks everything. Basically I'll show some examples, real world examples, things that we found in the wild, things that we saw other researchers find where your system, no matter how well audited, how well secured, you invested a lot of resources to make sure that it's perfect. But because you're relying on some assumptions and relying on some other code that you haven't developed and audited yourself, you introduce weak points to your system.

Uh it seems like you have the either the wrong link or you're showing the hidden slides. This is the previous version. I mean it's fine, but if you can >> it's a it's a shared link, so it should have updated. We could do this, but it just has bad graphics. Could you see one minute if you can fix it while I So, we're going to talk while they see if they can uh figure it out. Uh we're going to show uh a few different examples uh which show how supply chain attacks come in various forms. Uh one of the forms is going to be where you're using uh uh where you're using a package. You include some functionality that has been well

audited, but the way that you're using it isn't trivial. And it's a bit more robust to what you need, which introduces bugs. I think guys, if you can't figure it out, we'll just continue with this. Okay. So, some of the graphics might be a bit messed up. I'm sorry in advance, but it'll be fine. Uh, so this is me. So, supply chain in the context of web 3 is when you rely on other packages uh to develop your code. when you rely on extensions for your VS code, things like that. Anything that helps you to develop your final protocol is what we call supply chain. Uh so the first example

we're going to talk about is a pretty old bug that was uncovered which which is in a package called timela controller. [snorts] Uh the core purpose of that package is pretty simple. You want to have a functionality which allows you to uh delay do some sort of time lock on some operations in your code. So you want to schedule some sort of uh call or function or configuration change and you want to give some some users of the system the ability to react. So the bug that was there uh was that the way it's supposed to work is you're supposed to schedule your the transactions that you want to do give

some sort of allocate some sort of delay schedule delay for other actors to respond to challenge things like that and then once that time has passed you can you can execute an executor can execute the vulnerability itself was that in the batch execute the actual check that the call is scheduled was after the call itself right so you might be saying okay sure but what does that change anything this contract was self-governing so the the the initial role of this contract was admin role so what an attacker did was he did a batch of multiple different transaction the first thing he did was

he set the the schedule delay to zero right so now he can schedule for the same block as the execution. The second thing he did was basically make the attacker the admin of this core protocol allowing them admin privileges to your code to your protocol. And the third thing he did is he re-entered he scheduled after executing the first two he actually added it to the scheduled transaction. So when the check at the bottom actually check whether or not this was a valid thing to execute, it was after it's already been inserted.

And because he can change the delay to zero, uh it it's valid, right? It's the same block that was scheduled. That's fine. This is frightening, right? This is a few years ago. Um I think uh the researcher, the white hat who showed it was paid like $25,000. Uh I think probably should have gotten more, but uh yeah. So, let's go to the >> update. >> Yeah. Yeah. I was wondering what happened. >> Perfect. Yeah. This is the nicer graphics.

Thank you. Uh, okay. So, this is this is a case where you truly just have to be paranoid, right? This is code that is well documented, right? It has been audited. You can argue that it's not your fault, right? It's uh if you can't trust anybody who what you can do I argue is always be paranoid right always assume that if that the bug can be hidden anywhere possible if you're getting audited don't throw the scent of your auditors one of the worst thing you can do is you can tell them oh oh yeah don't don't look at this right it's completely out of scope obviously you want to direct them into the areas that are interesting but if you're telling a researcher that something is out of

scope he is blanking on Whereas if you don't say anything, he might investigate. And a bug is a bug. It doesn't matter where it is, right? Okay, let's go a different route, right? Another package uh which was used very reasonably. This time the package doesn't have the bug in it, but the way it was used. So this is a ROS package. Uh I think this is also from a a major like developer, but again this is not like a thing against them specifically. This is just examples that we saw in the wild, right? So the way it's supposed to be used is you grant some sort of privilege roles to some actors, some addresses, some smart contracts, things like uh allocators, uh admins, some sort of executor, pauser, some sort of roles

which gives you extra benefits. And then whenever you have a privileged action, you can check if that role or subset of role is actually active and usually do it through a modifier, but obviously you can do it through other things. Seems really reasonable, right? I'm sure your protocols are protocols that you've audited. You saw a pattern like this or even this exactly. So, uh again, right, it's harmless. It's well audited. Uh it's out of scope. But we had a scenario where we saw that uh a developer already had this functionality and they thought, okay, we have it. Let's expand on it. Let's use it for uh the band role, right? An actor who's done something that is against the protocol. So, they're supposed to be banned and they lose the funds because

they can't access them anymore. So, this might seem reasonable, right? You might say, "Uh, okay. Yeah, sure. What What are you going to say? What how is this bad?" Now, if we go up a little bit, right? There's there's a a small note that I put here, which is it's a bit documented, but not super well. This is supposed to be privilege role. So, the idea when this library was developed is you only do it for privilege roles. And as you can imagine an admin or in this case maybe not uh well some versions don't allow for admins themselves but you can always re revoke your privilege.

So the banned user just revoke being banned which is obviously not ideal. Now this if you're getting audited this is like three five inheritance down the line right your auditors are going to have to work hard to find this. So right the the best way to to get around that is to basically if you're incorporating code that you haven't wrote invest the time to research what what it's doing right uh and document it somewhere for yourself not for any not to help any other researcher not to help any other auditors just for you to know what you're doing so you don't have those mistakes if your developers changed things like that

right the the next one is relatively new. I had something older uh and then a month ago uh Glassworm uh was uncovered. So I'm not sure how how well you guys know that. So Glass Glassworm is a VS Code uh extension worm. So what it does is it's a malicious extension for your VS code. So you can imagine something like maybe it does like a pretty font for some things, right? Maybe it gives you some AI capability, right? You can imagine how you might have on your computer right now some extensions which you're not sure why you download them.

Maybe you got like uh Ethereum and you saw like six different ones and you just said I I'll get all of them, right? I'm not sure exactly which language package gives me the best functionality. Right? So one of them had the malicious code in it and they did a few unique things there. uh one of them is the malicious part of the extension was developed in unprintable uh co well uh uni code right so your vs code browser your notepad your terminal whatever you try to if you go the extra mile to see what what it's doing it will show you just characters that are look like spaces it will be nothing you won't be able to work with it if you have some sort of um client that's supposed to trigger if there's

something malicious it won't know how to handle Right. Uh so this is this was in innovative but not not that much like we've seen that before maybe not in web 3 extension context but it's pretty well known that if you're doing uh payloads like that you can hide them. It takes a little bit more work but usually that's not the hard part. The the the big thing the thing that they actually did which was unique and it's the first thing we saw the the command and control center was onchain. So this is a bit of lingo of like uh well hacking in general, but command and control is how you tell a computer that you already have malicious code running on it. How you tell them to download a a different payload, usually a bigger payload with more capabilities or uh this one I think stole like 50

different uh wallets. They have integration for their keys. Uh, and another thing that they had, not not on the original small payload in the extension, but on the bigger one that you download later, is it actually checks if you're developing on a GitHub account extensions yourself, and it inserts the original payload into the extensions that you're, uh, developing. So, uh, this was uncovered like a month ago. I I I think I got the first researcher who reported on it. I'll show you a link in a moment. But all this how to download the bigger payload, how to control your computer was actually is done onchain. And you can look at those transactions where they say, "Okay, now download this payload. Now do this scary stuff." Uh you can you can scan this uh

as far as I can tell, he was the first guy at least that I saw that reported on it. Uh maybe somebody else did it first. Uh he is a better article than what I'm summarizing here. Uh I'll give a moment to anybody who wants to take a photo. You can also search glassworm on Google. Uh very interesting. Okay. So we saw an example of just code that had bug in it. Really hard. You have to be paranoid. Uh we saw an example of uh a code that is okay but you used it incorrectly. And now we see that even your IDE could be malicious.

Right. So all is lost, right? Yeah. Uh so there is no security and and what can we do right the uh there's so much that we can do maybe we'll work uh through uh pen and paper in a in an isolated cave uh it will be more secure but let's think about actual things that we can do in order to make it uh a bit more secure. I'll be honest, it was extremely hard to find four words that start with the same letter. And so I did three. Um, so these are actual things that you can do and you can recommend if you're doing audits and you can implement that would definitely solve the things that we mentioned until now, but will generally make your supply

chain, your packages that you're using, the de debugging of your code, maintaining it, all that stuff faster. And hopefully I managed to show you that it won't take a lot of time because we don't want to do stuff that's wasting our time. So we have document deflate delve and document again looks good right uh so document right uh if you're using a piece of code that you're haven't developed yourself obviously I recommend it for code that you're developing yourself but this is not the point of this talk specifically why you're using it which version are you using right what sort of constraint caused you to actually uh work with it how are you using it do you do something that hasn't

been strictly the use case of it. Do you expand? Do you add functionality? Uh what is your cost if you stop using it? Can you develop your own code? Can you change the design to avoid it? Right? This is not just for packages that you include in your in your protocol. This can be also for VS Code extension, maybe operating system. Right? If you're using some weird random build of Linux that has a lot of like cool agentic AI stuff, right? Uh would it really cost you to move to something with less blow? Maybe.

I don't know. So, next up, deflate. This is how we trim we trim the fat. Okay. Uh it's so easy to try new things and we all are exploratory. You want to make sure our process is as good as possible. I understand. But specifically when you're developing protocol that's going to hold LTV, this is core things. Minimal is best. Okay? Treat it like a classified army top secret project.

Okay? Do take the minimum things that you need and try to treat everything as potentially malicious. Right? If something can be removed, obvious extension that you don't know what it's doing. Remove all the extension. Start from fresh. Download just the one that you need. Maybe don't use VS Code. Use Notepad++. That's a joke. Uh if you can't remove something, reduce its functionality. What do I mean? Uh don't develop the wheel from scratch. But if you need a tool that does a 100 million thing, maybe it's a parser that can do, this is not strictly web three, right? but it's a parser that parses images and videos and every format in the world and you only need it for PDFs,

just remove everything that's not PDF. Reduce the attack surface. Uh if you can't reduce, you might want to develop yourself. Obviously, there's cost in developing. Uh ask yourself, do can I if I can't augment if I can't change what the thing that I want to use is doing, can I develop myself something more lean, more minimal, uh that does the only thing I'm doing and nothing else? uh if you can't develop, be suspicious, right? Uh I had uh I had a a session with a with a client and we basically had a week where we investigated all the services that are running on their development uh VM machine and we went a bit overboard. I'm not saying go that

far, although you might want to, but be suspicious of anything that you don't know why it's there, how it's used. uh if you're auditor uh you or if you have if you're research inclined get wire sharkark whatever see if anything is calling out just be suspicious delve um if you're using a piece of code that you want to add right uh or an extension put some time aside to delve into it right internal research figure out what it's doing right internal audit see if that's something that that's needed if there's other things on the market that is better, right? External audits, right?

Uh if you're if it's super critical, uh have somebody else look at it. Somebody whose profession is to make sure that you're not losing money. If you can't understand something, don't use it. Don't use it. If something is so complicated that you can't figure out, this is not like developing like Android and there's so many subsystems and so many things and oh no, what can we do? No, this is this is web 3. So far it might all not always be this way but a lot of the things are simpler a lot of the ecosystems are simpler you can conceivably understand your own protocol and if you can't try to see if there's any version of it which you can understand right okay right so why did they say

document twice uh I don't want like the worst not the worst but I don't want you guys to get out of the this lecture, go to your teams, your your firms, whatever, and just take say remove everything, delete everything. Uninstall VS Code, uninstall, remove all the includes, not because it's not secure. But before you do that, write down what has been deployed. Write down what has already was already used in your system. Before you remove some sort of service from your deployment machine, write down what it was. what version it was. Trust me, when somebody is going to post on Twitter that version

4.3 of some random package has has been exploited, right? It will be so much easier for you to look at that document and see, oh, we used 4.3 when we deployed this protocol that is live for the last three years, right? Before you go ham and you tell your your product managers or whatever, we heard Ray, he gave the best presentation in the event, we have to remove everything. Write it down somewhere, right? You I recommend in some sort of markdown file on on a GitHub, something that is text. Don't don't use like a like some sort of proprietary sort of Slack Notion thing that in three years might be removed and

nobody's going to know what the document is. uh take a while to ask yourself what I'm doing, why I'm doing it, which version am I using, so that if there is some sort of worst case scenario, you have a quick lookup that you can check and see, have I been wrecked? So, uh we talked about document delve, uh deflate document. Conclusion, right? Supply chain in most ecosystems is one of the hardest things to audit and to maintain. Right? If you think building a car, right, they have to take into consideration where every component was built, right? I I had I I I audited Intel code for a few years and we had to

come up with solution that if you have a factory in China that produces a specific chip uh how do you secure that the bits on that chip are encrypted so that the workers in the factory can't leak that information out. Thankfully we're not there, right? It's not impossible. That's something that you can do. That's something that can elevate your your auditing. It's something that can elevate your code so that you won't be wrecked. so you can help secure DeFi and so that we all have just generally a better time. Not everything is lost. Web3 is a great place to be in. Make sure that your tool chain is good. Thank you very much.

[applause]