How One Crucial Flaw in AI Could Destroy The Entire Sector - Phala CTO

Transcript

But the problem is uh there are still a lot of things cannot be done on Ethereum or blockchain like very heavy computation AI you cannot really run any AI model on blockchain. >> Why did you guys opt for an onchain governance uh component to the stack? Maybe you can explain more about what this piece does. What why did you choose onchain governance? Give us the real side of the story. What's actually happening here? Do we have to be worried about this? And what are the ramifications of this with AI? >> Welcome back to the rule. This is your home to learn from the biggest guests, the brightest minds at the tip of innovation.

>> Our goal is to help spread the collective knowledge of everyone in our community. We are breaking crypto out of its echo chamber and taking it to the real world. Welcome back to AI Supercycle Hang. Welcome to the show. host our premier show presented by Near, powered by Near, pushing forward the ideals of privacy, open-source, and collective intelligence in the era of AI. We're going up against the big technocs in their closed box, blackbox AI systems, and we're trying to bring the power, the data, the organic models back to the people. Hang Yin, welcome to the show.

>> Nice to be here. Really excited. >> Sweet, man. Well, it's a good time that you're here because we've got some news to talk about in the state of AI. Hang, you are the co-founder of FA amongst other uh research roles in AI. And you know, this week we had quite a bit of of news come out about TEES, which is something that is very important for confidential computing. And um I had been watching confidential confidential computing summit podcast and they were talking a lot about privacy and this type of of kind of confidentiality in AI and it was actually centered a lot around TEEES.

However, a new exploit has become known that makes tees fully exploitable. Basically, TEEES promise people that you can build hardware that can process information without anyone, not even the operator, seeing it to gain privacy and security. In reality, it's failed a bit in the past, and with today's brutal exploit of Intel, um, basically, it's reached its peak of failure. Billions of dollars were poured into Intel for its secure enclave program late last year. However, given physical access, about just about anyone can break into an Intel SGX, Intel TDX, and AMD SEV SMP cards. Even a hobbyist level attacker can extract

attestation keys and enclave secrets. Tees in crypto and AI are done. Hang, this has been the headline. This has been the news. Give us the real side of the story. What's actually happening here? Do we have to be worried about this and what are the ramifications of this with AI? Let's get this conversation started. >> Of course, I'm happy to talk about that. So, um you know, most of the people can get the catchy word like Intel is done, SGX done. What about the reality? The reality is that uh it was attack targeting the legacy hardware. The latest level of the hardware is still good and there's still no way to attack the latest hardware. So um people when

when people think about the vulnerabilities in any software or hardware system, people think that okay software is able to be fixed but hardware cannot be fixed. Uh but it turns out this is not true. Hardware can also can also be fixed by applying the firmware upgrade and this is what has been done in the past few years. So you know this year SGX is done. If you check last year SGX was done and if you check like a five years in a row then every year SG SGX is done but you know we're still good and all the system running on the production is not really affected.

So this is a really interesting mindset. Not only the software can be patched the hardware can also be patched and whenever we find a bug what we do we release a new version patch it and then we move on. So this is how you the ecosystem and the industry move forward. >> You're not too worried about this latest exploit then it's just another kind of uh flash in the pan type of event. >> Uh not really. Uh so to give some little more detail about this attack, this attack is called uh wire tap. So what does it mean? It's yeah you can imagine how how it work. So basically they attach some wire into some hardware and so that it can read the content from the

hardware like the memory but actually the thing stored in the memory is actually protected. So you cannot see the plan test. It's not mean like you attach into uh the CPU or the GPU or uh you know the memory then you can see everything running in this so-called TE but actually TE has the has the design against all kinds of this kind of attack. The real problem lies in the detail. So one of the problem people cause is uh is called um set channel attack. So what is set channel? Side channel means that there's no way you can directly get what you want from this attack. But you have to come up with

some very very clear way. Uh it can leak some data supposed to be protected but with the very clear tweak. You can still get the data from you know protect protected area. And in this case the uh wire type they try to get the encrypted data and they try to inference the plan text from the encrypted data. And because of uh some uh bad design made by Intel in the past it make it possible to attract some really important keys and it turns out those keys can affect the the the the impact of those keys is really broad. they can even pretend uh some non TTE to be a TE for all those

affected um you know uh uh uh hardwares and in the newer hardwares this is not the case because so far there's still no way to attack those hardware and usually uh you know all this kind of attack it's a combination of multiple vulnerabilities together. So if there's only one vulnerability it's not good but we can still fix that. When you combine multiple vulnerabilities together in a way that can really make a exploit then this is where you know we can see the uh the uh zero days attack or any hack really happen. So u even before that people will try to prevent all those uh problems uh as much as

possible. So uh in this case uh it's so good that uh it's still targeting the old generation of the hardware and the new generation of the hardware is still good. Uh at the same time we're still you know waiting for like u Intel the CPU manufacturers to apply some fixes so that we can patch this kind of problem uh once uh and for all. >> Habachi provable trade per earn rewards visit hibachi.xyz XYZ trading on fire. The future runs on intelligence creativity engineering health, money, productivity. AI promised utopia, but delivered only hype.

Blackbox models and zero accountability. With tons of tools claiming to be the best, how do you know what's real? Introducing Recallfind. Top models ranked by transparent reputation from researchers, users, and developers. Test AI skills. curate favorites and compete to see what works. Don't guess what's good, recall it. Yeah. I mean, I I know that you've been critical of of traditional encryption in the past, and it's good to hear your, you know, your take on it, seeing like the difference between like a zero day attack versus kind of what's actually happened here. Um, you know, when I was listening to the confidential computing podcast, you know, they actually spoke

about uh like basically um a form of like malicious prompt engineering from like uh the outside. Maybe you could just lay the landscape here for the kind of encryption uh stack within real world AI systems like from from inference to models to data to user, right? Like how does the encryption stack work in traditional AI systems today in the real world? and then you know perhaps we can start to talk about what follow is doing and just generally how you see the future of encryption when it comes to AI.

>> Of course this is a great question. So uh to me actually you know what's the encryption people already have now today is basically nothing. So like you talk with open AAI it looks like everything is encrypted everything is secured you send your prompt to open AAI and the next day some automan can just pick your prompt and if there does anything bad for you they can even send that to the like US government government and they can just use that as the evidence to against you. So this is what open AI said in the past. Um but you know a little bit uh Jo joke here uh actually in the industry people have three kinds

of encryption. So one is encryption in trans uh transition. So that means when you send something to someone else like you send a message to me even during this uh podcast all the encryption is encrypted. So that means like uh if someone uh bat they get the uh physical access to the wires they cannot see anything in between our conversation. Second one is uh the data in rest. So imagine there's a huge database all the prompt you send to open AI or some other uh uh uh you know services they store it somewhere and usually those storage is still encrypted because if it's not encrypted anyone can easily you know

like uh steal the hard desk from some uh some server and then everything is there. So of course this is uh the risk that nobody want to uh take but the most critical part is actually the third one it's called uh encryption in use and this is really really really hard because you know whenever you want to use some data you still need to decrypt it and then calculate like you run the uh inference in the GPU GPU has to read the plant text data and process it but this is the trick if Someone has the control over you know anything in this process like they have the control of

the GPU or they have the control over uh the hardware then uh or even the software you know you're in trouble because uh the software is still designed by the developer and if there's a tiny uh uh back door in the software they can literally see everything no matter how you protect the data in transition or how you protected the data in at rest. So this is why you know confidential computing come here to fix because for security it's either zero or one or 100 like you either have no security or you have full security there's no anything middle ground. So you we have to make sure the data in use

is also fully protected and using confidential computing. The way to do that is that now we have a super secure you know box and the box have the CPU have the GPU you can still do whatever you can you want in this environment but at the same time it make sure that nobody can see anything happening in this black box. So, you know, with confidential computing, you can use the data, but without exposing the data to anyone. >> Well, I I want to just kind of like re make sure that I understand kind of the point that you're making. It it's really the full stack that has to be confidential. And if there's really any back doors at any point along the stack, then that compromises the entire security. And so, you've deployed hundreds of AI agents. Maybe this is a

good time to to talk more about what Fala is doing and how you guys are making sure that you have every single piece of the stack locked down. Maybe you could explain, you know, what is Fail and how are you guys making sure that you have every component secure and encrypted so that there's no point of entry and no attack vectors. >> Of course, I'd love to uh I'd love to talk about that. So, um actually we started this company six years ago. is pretty old company and uh everything we focus on is just uh to solve the trust problem. Uh it was really fascinating.

We uh the first day we learned Bitcoin and Ethereum you know they created a blockchain and you can do the things you can never do in the past. You can now have something that is a software and it's not controlled by any single entity. It's just a run. So this is how like people built their DeFi ecosystem. But the problem is uh there are still a lot of things cannot be done on Ethereum or blockchain like very heavy computation AI you cannot really run any AI model on blockchain. Second one is privacy everything you threw to to the blockchain got exposed to the public. So we want to make sure there are there are still something uh some uh services that

can be privacy preserving and no uh no not to mention like software uh running on blockchain can hardly communicate between the other services like you cannot access Google from uh Ethereum. So this is the reason why we started uh thinking about uh finding some solution to fix this problem. And uh the the the the conclusion is uh the technology confidential computing and uh in the very early days you know we try to design everything just right for the blockchain and on the in the in the web street uh uh space everyone is like a paranoid. you assume every player in this uh ecosystem is malicious. You

don't assume them are honest. So we have to make sure the entire stack is fully protected otherwise even if there's a single loophole people are going to use it and exploit then uh everything will be gone. Yeah. >> So this is the man like a mindset from the web three world and now uh you know we suddenly realize okay there's a AI industry they really want to solve the privacy problem and the trust problem and they never got the same mindset as web 3 before. So I think this is the perfect uh time to you know bring the web three um mindset and use that to help AI industry to solve their problem.

So yeah, this is you know all the reason behind why we're doing all of this kind of stuff and in practice uh last year we have uh u moved the focus to the AI and uh then we have designed a completely new solution and it turns out to be an open source project called the Dstack. So uh we got the initial you know we bring all the experience from uh the days when we building the um blockchain solutions and uh apply that to the AI scenario. So we want to make sure there is a software that is fully end to end encrypted not only you know the data in use but also we make sure even if you expose say a so a

service to the user the user can easily verify everything behind without changing any of their codes or any of their existing software stack like you can use your Chrome browser to connect to the service running in this environment without any So, >> yeah. Yeah. >> I'm just curious what what would that enable? Like if you used a Chrome plugin and you said, "Hey, I want to run my AI stack and I want to use this Chrome plugin to plug into DStack and basically turn it confidential." What would be happening behind the scenes to enable the confidentiality for the existing AI stack?

>> This is going to be very interesting. So, initially maybe people think, okay, that's pretty simple. There is a TE. Let's just enable T and then run the software and that's done. But uh actually uh as we mentioned like uh software is very complex need to make sure everything is secure. So it turns out this process is really hard. It turns out we have to make a very um you know uh uh careful design to make make sure everything is secure. So let me give you some example. If I run some software in this uh T hardware then one thing uh first thing come uh come uh problem come up is that how you're going to encrypt everything you need to

encrypt the disk you need to encrypt the uh you know the network connection and when you are dealing with encryption how can how do you get the key for that encryption and usually the answer is I get the key from the hardware so this is the TE key but then you tied everything to this hardware. So it make a big problem. If the hardware is gone like to tomorrow the CPU is burnt then all the data will be loose right. So uh this is a typical problem. So we have to make sure the key you use in this system is not just a tight with a single uh physical hardware. We don't want single point of failure. So this is the first

uh uh principle when we design this system. We want to decouple the key management from the uh you know the application. So key manage key management system and ideally we want it to be decentralized. >> Yeah. >> And second one if you think about now I run some software software it's good I create some proof and the user verifies the proof but the next day I found some bugs in this software. I want to fix it. How do you fix that? If I can update the code running in this T at any point, it also means as a developer I have the power to change the logic, change the code. It means I can introduce the back I can introduce a back door at any

point. So pretty similar to smart contract, right? You have a smart contract and you want to make sure the smart contract is either immutable or it's mutable but it's governed by some onren governance. At least you want a multi- signature. So it's the same thing also apply to the soft running in the TE. So this is the second point. We want to make sure everything in this environment is governed not by a single person but by some onra governance. So there are still a lot of similar examples but basically you know all put to put all of them together. This is why we have a open source solution called DSAC to solve all this problem so that

the users only need to you know like give a docker container and everything can run. We take care of every uh you know detail and the hardcore security problems. >> Yeah. And so you've got key management, you've got this kind of onchain governance kind of distributed set of ownership. What you get in now by DStack becoming a Linux foundation project as well. Um >> you kind of get this default framework for running AI privately kind of like what Docker or Kubernetes did for Dev Ops. You've got now a new framework to build private AI systems. Um the Near team has been working on a a really

really sick private chat product um as well and that you know they will that'll be something that they are are going to be you know eventually rolling out and whatnot. I'm not going to give too much information, but I'm just curious from your perspective, is DStack kind of the default framework? Is that the right way to think about it? Is it the default framework for building AI pro uh private AI products? Or what would you need in addition to DStack to truly build a endtoend real world AI system that has the confidentiality components that we spoke about today?

>> Yeah. So uh dstack is the infrastructure level uh software. Uh just imagine it's kind of like uh just like you mentioned it's very similar to docker or kubernetes but it's the secure version. So you can still use docker you can still use kubernetes that's that's in the future. Uh but we to enjoy all the security benefits. Um and uh but when you're still uh when you're uh uh uh deploying AI workloads, there's still a lot of nuance you need to take care. So for example, you want to make sure everything put you put in this environment is secure is transparent. So the way to fix is fix that is uh you want to make sure everything is open

source so that uh everyone can check the code. Okay. Now, now I have read all the code and there's no single place you steal my data or log the data to the place it shouldn't uh go to. So this is one point and second one I think uh usually um uh you know underestimated part is to make everything explainable because uh you use a TE or not doesn't directly affect how people feel about that.

Then how can we convince people now you're using a super private AI nobody can even see your input or output how can they feel that I think this is really important >> so uh in order to achieve this uh from the dstack part or from the you know te part uh all we can do is to provide enough evidence the evidence include okay what is the software behind what is the hardware behind what is the operating system behind >> those kind of evidences are really technical so user cannot understand that but when presenting that to the user I think it's really important to have something very similar to uh now the big enterprise has if you go to anorico

website there is a page called trust center uh and their trust center is just a you know a list of all the evidences and the reports and explanation so that use user can uh uh try to understand close stuff. But here with TE, we can have a trust center. It's even better because now everything is transparent. You can see the overview of the reports. It can just easily tell people hardware, software, operating system, network connection, PMS, everything is secure. Uh but at the same time, it can also give the auditors or the secure security researchers all the details. So they can check every part one by one. follow just to follow the evidences to check them by

any third party. So I think this kind of um you know visibility is very important and another I think very interesting case is uh HTTPS or TRS. So when you connect to some websites you will see a a small like a lock uh on your browser and that's basically all for normal users and I would like to see for all the confidential AI project to have also have like a small I don't know maybe a small lock or a small check mark so that they know okay this is really behind uh really the confidential computing behind this product >> right yeah that makes a ton of sense because you expose the visibility of the

underlying so you don't have to go and look into this and make choices about where you're selecting your infrastructure. It's almost like look here's an option if you want to explore more you can look at it but here's the visibility and and oftentimes in crypto you know we we try to push our ideologies on people sometimes governance and and uh some you know sometimes you're being forced to run a node I can remember you know back Ethereum you almost back in when I first got into Ethereum you had to run a node in order to have a wallet and I think pushing some of these things is not helpful but if you expose it and you say hey this is what we're doing you can choose choose to do more or not. You can give a wider gradient of preferences to the end user. And you mentioned

governance early on as part of some of the stack and you guys have this portion of onchain governance. I'm just more curious about that because you know the critique about onchain governance has been that it's slow. It's more difficult to make decisions. Why did you guys opt for an onchain governance uh component to the stack? Maybe you can explain more about what this piece does. What why did you choose onchain governance and how it enables and and helps the overall infrastructure. >> Oh, of course. So, onchain governance is of course something not trivial. There are there there's like a it's a basically a rabbit hole. Uh but at the same time, uh we want we use onchain

governance for reasons, right? We want to make sure something um is secure by using onchain governance. So I think the basic uh uh philosophy behind this we want to avoid single point of failure. We don't want to rely on you know either one hardware or one developer or one company. We want to make sure whenever we have to trust something we want to make the trust as distributed as possible. But uh a gang governance doesn't really only mean voting. It also means a lot of other interesting uh uh uh you know like a mechanism. Let let me give you some example. If say I run uh you know just like a private AI a

private open AI in the TE and uh at day one I deployed everything and you use it everything is so far so good but next day okay now I have deepse v3 v4. Okay, we have D3 v4. I want to upgrade D3 v4 in this TE enclave. Of course, uh everyone wants to do that. So I can do the update. But it also means for the developer, they can even change the code during this update. >> They can even insert some backd dooror.

So essentially this creates a single point of failure dependency on the developer. We don't want to trust the develop the developer. We want to make sure everything can be visible and the user has the right to notice okay there is the update I can either approve it or uh you know uh uh disapprove it or at least I should know what's happening there so that I can make the decision to continue use this software or not. So a typical case is that okay now there is a new version we put the version updates on GitHub. So this is a pull request.

Everyone can vote on if the pull request is good or not. And if we get enough voting then okay this is good. We are going to push that to the CICD and it's going to be uh uh uh you know updated in the TE. And another thing is uh what if you even don't trust the voting that's also totally possible because you know uh participation in governance is really hard. Maybe for voting there's only five people voting then uh what we can do another very interesting idea is uh we can apply a time lock so I can update but the updates will only happen after say seven days or even one one months.

So during this period there is a p uh uh you know there's a public announcement saying okay this program is being updated and if you're curious uh feel free to invite your secure audit friends to come in and check the code and and you can leave if there's anything bad happening. uh and with the time lock some sometimes we can even skip the voting process because it give people the right to uh still you know reach quit from this system. So yeah on governance is really complex and uh interesting but uh what we do is we just give the option to the developer. So

there is a on governance contract but the contract is very minimal. You can point it to a mocky sick. You can point it to uh like a onchan voting or uh you know like a a time log. So really up to the developer to make such decision. >> The wait is over. The Talis test net is live. Be part of the revolution. Join now at testnet.talis.network. >> Relay low fees, lightning speed, and the power of multi-chain at your fingertips. Instant multi-chain payments. Choose any token on any chain. 75 plus network supported, more than 5 million users, 8 billion plus volume processed, 70 million plus transactions, fast,

reliable everywhere. Start swapping today only on Relay. Blockchain shortcuts, templates, and widgets launch without friction for builders by builders. Visit enso.build. Hang, we started this podcast very very you know straight into the weeds. Tees onchain governance agents agentic workflows. I'd like to just zoom out here and understand from your perspective, you know, what does the future of private AI look like? And if we don't get this right, why does it like why is this so important? You know, what is the other side of this world if we don't get this private confidential

open-source userowned AI system? You know, what does that look like? >> That would be horrible. Yeah, I think absolutely horrible. So um you know in the past people have been um dealing with the privacy problem for a long time but uh for say daily conversation on WhatsApp or Telegram people don't really pay that much attention because the the the the the impact is still limited. It it just it just uh impact a few uh conversation between like you and your friends but it's not the end of the world. But now you can see AI is becoming really powerful.

Initially I talk with AI just to learn something. But then you know we delegate some of our daily task to AI like hey open AI do a research for me. I'm going to use this research report for my next week uh team meeting. And now we even delegate code writing to AI. And uh to be frankly speaking uh our fa cloud has 90% of the codes produced by AI. So you know this this is like a shift of the power you know who controls the AI controls the world.

>> You know now today it's just our cloud but tomorrow you maybe at some point 90% of the software will be written by AI. But then how can we make sure AI will not be out of control. So one analogy I'm very happy to uh use in the past is AI is very similar to nuclear fusion. On the one side if you make that in a controlled way it benefits everyone right it's so powerful but on the other side if it's uncontrolled right it can be it can even you know just dorm the entire world. So here we want to make sure AI must be owned not by a single entity not some automated.

>> Okay. >> Mhm. >> Yeah. So so hang let's say we get the private open source AI thing right. Do you truly believe that that's going to stop the trajectory of where this is headed? Like like isn't it too late already? >> I don't think so. uh you know now AI is still like just a part of our uh life but be as long as like before AGI before AI taking over everything I think it's never too late so we can see a lot of people are trying to fix it uh and how to fix that so uh eventually we want to make sure all the AI can be controlled by not any single entity so we want to

eliminate single point of failure but how can we do that uh we want to make sure okay AI is available for everyone and everyone can participate in controlling the AI and if there is a bad AI we can also have a good AI to against it to uh you know like hatch all the risk but then the problem is how can we have such backup plan or even like a hatch to the centralized AI we want to make sure we have the AI inference then we want to make sure we want we want to make sure there's open AI model trained by the open world and in order to do that we need to have the open data. So you know this is actually a supply chain from the data collection to training to

inference to the AI agents and all the application build on AI. We want to make sure the entire process can be decentralized. So but the good news is there are a bunch of really good team behind all the those efforts. For example, one of our client is called VA. They are building the open data infrastructure and they run all the data cleaning data governance data trading using blockchain and TE. Next once we have the data we can train models and this is where near shines. So they are trying to build a very large infrastructure so that people can train the AI model in the TE so that you know nobody can inject any backd dooror in

this training process and the model produced by the training can also be monetized. So this is how opensource model can be competitive to the closed source model. They can earn money by doing something valuable for the community rather than just throw that to the community. let every everyone else to be a free rider and then once we have the monetizable private model we can run the inference and we can run all you know the application and every process can be secured by a combination of the web three technologies the blockchain MPC FH uh ZKP and TE specially for training and inference and the data intensive

operations >> hey the the scope at which you describe this is very farreaching. Like we talked earlier about how you can enable this in a browser. You know, you went all the way through to some of the vendors that are partnering with you guys and all the way through to some security audit friends that could come in and, you know, vote on whether the poll request would go live. So, so when you're thinking about this product and you see how far it stretches, who would you say Fail is for? What you guys are building? Who is it for? Is it for a bunch of different groups? Who are they generally when you're building this? Who are you targeting? Who is the person that should be taking it seriously?

>> I think everyone should take it seriously because AI is going to affect everyone. So everyone maybe they don't directly interact with us or interact with the underlying technology but we need a mindset so that everyone know okay this is really important. We need privacy. We need the decentralized control. We need uh you know avoid single point of failure. Then we give pressure to the big enterprises like open AAI, Anthroporic. It's not about like we as one company want to take over everything. We just want to make sure the trust problem can finally be solved.

It can be solved by us or can be solved by the other companies as long as we reach this goal. I'm very happy with that. So now I would like to invite like all the uh users either enterprise uh developers uh individual developers or end users to think about that like we really need it and we need to push every thing to this direction. >> Awesome. >> Hang in CEO CTO co-founder follow network big AI researcher gigab brain fellow privacy enthusiast. Thank you for your time. Hang it's it's been a pleasure.

>> Thank you all. Really nice to be here and uh yeah good place to talk talk about the privacy related uh topics. >> Absolutely. >> We appreciate you agreeing on Thank you for coming. >> Thank you so much. Thank you so much. I love this uh you know all the topic and all the questions. Really exciting. >> Thank you. >> Good H. Good. Well, keep up the good work. Thanks guys. See you.