JP Aumasson on Quantum Risk, AI Attacks and Too Much Crypto

Transcript

Welcome to Zero Knowledge. [music] I'm your host, Anna Rose. In this podcast, we will be exploring the latest in zero knowledge research and the decentralized web, as well as new paradigms [music] that promise to change the way we interact and transact online. This week, Terrun, Niko, and I catch up with JPasan, co-designer of Blake 3 and author of Serious Cryptography. JP is now working at Cloudflare after years working in web 3. He shares his updated take on the quantum risk to cryptography and his thoughts on how AI is changing cryptographic security. Specifically,

why he thinks symmetric primitives are safe from LLMs while the asymmetric side is already seen real AI assisted breaks. We touch on Poseidon and some of the security work that's come out recently around that. China's new NGCC competition and the speed at which AI is breaking the candidates and how AI is changing the way that cryptographic systems get built, allowing famously cautious cryptographers to be less conservative with security margins. He also shares news about a new hash function bounty project he's working on and why he would like to see Blake 3 everywhere and more. Hope you enjoy. Now Tanya will share a little bit about this week's sponsor.

Today's episode is sponsored by Sveright. Sveright builds the critical infrastructure that preserves your right to choose. Sveright has built two products, Argos and the Sovereight mining pool. Argos is a free tool that lets anyone with ZEC stranded in an old unmaintained wallet client recover their funds. Enter your seed phrase, generate a new address, and your money is yours again. No fees, no complexity. The sovereign mining pool is Zcash native and built on stratum v2 where miners choose their own transactions. Every block carries a verifiable signed attestation and payouts arrive in fully shielded ZEC. Two essential problems that have been overlooked finally

solved. Discover more at sovereight.com. So thanks again Sbrite. And now here's our episode. [music] Today, Tun Niko and I are here with JPasel. He is the co-designer of Blake 3, the author of the book Serious Cryptography, and he is now a cryptography engineer at Cloudflare. Welcome back to the show, JP. >> Thank you, Anna. Thank you for having me. >> Hey, Nico. Hey, Tun. >> Hey. Hello. Hello. >> JP, we've had a few touch points over the last few years. I want to just share sort of a rundown of some of these. You spoke at ZK Summit. I think it was ZK Summit 7. Uh, seven or eight. Yeah.

>> Yeah. >> I don't even remember. I know. It was about security auditing of ZK proof systems. >> Cool. You were a guest on the show in which we mostly talked about hash functions and you explained them to us and then you came back and more recently you were actually one of the experts that we had on the ZK whiteboard sessions also looking at hash functions. And I feel like that's I mean that came out about a year ago and yeah it's just it's really great to get a chance to catch up once again. I feel like since we last spoke you've made sort of a pretty big shift in your career. Do you want to share a little bit about what you've been up to? Yes, indeed. I've been for the last six, seven, eight years um contributing to the u so-called

web 3 ecosystem from a um let's say institutional banking regulated perspective working in a company where I was chief security officer and now I joined Cloudflare which many of you likely know. I joined three weeks ago start of September. Yes. And it's a very different very different experience because I I wanted to change for a variety of reasons and I realized that what I love doing what I do well I assume is cryptography cryptographic engineering security in a sense of you know managing security in an organization but in the space where I was it was very highly complaints driven

which I realized is not what I enjoyed the most and not where I'm the best. So that's some of the elements that brought me to um to Cloudfare and also if when I had this opportunity to join. It's a kind of an offer you cannot you cannot refuse given the the sheer scale of tourists and uh the quality of their engineering. >> Very cool. I feel like in what you're doing too, you're going to be basically interacting with technology that's touching a lot of the internet. Is that a shift from what you were doing before? I mean you were creating very very important hash functions before too. So like but does it feel like a different engineering scale?

>> Yeah, I'm I'm joking with some of my colleagues that I want Jafar to adopt free which would you know >> probably improve the performance of my system. >> Yeah. [laughter] >> Yeah. I really enjoy my my first weeks here, the company, the people, and to be honest, it's quite refreshing to to be in a different space on the web three blockchain space, which is, you know, it has um people who know me, who know who see me on Twitter know that I've been very skeptical of some web three projects and we see how it's evolving and we see that the best projects survive from a technology perspective, from a business perspective.

But I wanted to to see something different. >> So, >> here we are. >> It's funny though, you feel you've left crypto, but I feel like crypto's going to find you again or cry crypto scene. Is web [clears throat] 3 is going to find you again because >> you didn't go far enough, JP. >> I don't know. There are hush functions everywhere. >> Yeah. Also, isn't Cloudflare building a blockchain? [laughter] >> Yeah. >> I don't know. Don't expect that. [laughter] But you know sometimes I'm I'm good at prediction but sometimes you know the world surprises me like you mentioned hash functions before the the Ziki podcast and um I mean 10 years ago I really thought that hashing is a solved problem. We don't need anything more

than uh catch blake and uh the easiest thing to design and now as you saw we have all those uh debates about which hash functions to use in postquantum signatures which do a lot of hashing and is pos secure enough is pos secure enough what do we need for zk circuits so that's interesting but I don't want to spend my life working on hash functions >> okay you want to do other things Um, so yeah, one of the reasons I first reached out to you, I actually I've been trying to have you on the show for about six months, I think, and I had a really hard time getting in touch and I'm really glad I got a chance to. But one of the things that had prompted me reaching out

was actually your take on quantum computers and the advancements that had been announced maybe around like March, April, specifically with the Google paper, but also just generally the conversation that was going around on that. So this is something that I definitely want to cover with you. Um just to give maybe a rundown of some of the other things I want to talk about AI and how AI and cryptography are working together, how AI is like breaking cryptography, the newlyannounced Chinese crypto competition and some of the standards that have been presented. I want to hear about your thoughts on Poseidon. I know you sort of hinted at that. Yeah. So these are the things we're going to hopefully get a chance to talk about today. Let's maybe start on that quantum front. quantum. I feel like skepticism is going to be a a key word

in today's interview. [laughter] >> Well, I I like to be held accountable and two years ago, I think I I tweeted posted um the following opinion and I also repeated it in a multitude of public presentations. I was saying at the time that I don't believe that I'm going to see a quantum computer breaking RSA 2048 in my lifetime. >> Yes. >> So, I'm going to jump 44 in a few weeks. um healthy. And the question is uh do I still believe that? There was a big fuss in March this year when Google announced something and when Google announced that people paid a lot of attention and my experience was that

last couple years in my industry was warning people against the the quantum risk treating it like any low probability high impact risk >> in terms of risk management and it was often not really dismissed but you know when people don't read about it in the news then don't take UDS very seriously and rightfully so I get it but once Google made the announcement it was a major change now suddenly you had a million of articles they were like oh Google believe that a quantum computer will be out there in 29 >> whereas the reality is that it's not it's not the reality what Google rightfully assumes is that the risk is high enough maybe one over a thousand

one over 10,000 I don't know the risk is high enough for Google to justify their upgrading all their systems by 29. >> Yeah. >> And some of the progress was, I don't know, maybe unexpected, you know, in terms of the size of the circuits. >> Mhm. >> But what Google announced in itself was, to be fair, an incremental improvement compared to what researchers published a few months before. I think was a team of French researchers. And at the same time we had the oratomic launch and the claim that a quantum computer with neutral atoms would need way fewer cubits to to break RSA.

>> But in terms of probability I don't think it really materally increased the risk. It it's like a friend of mine was joking that blockchain has been the future of finance for five for 15 years and likewise a quantum computer has been perpetually 5 years away for the last 15 years. H >> I think that's been true then more than 15 years by the way. I think like since Fineman kind of did the talk in the 80s that's been true. >> So yeah there's a lot of investment but we see also a lot of investment in in AI because the returns are are more obvious.

>> Mhm. So I don't know I'm maybe less skeptical you know and again as purely my personal opinion you know of course not speaking for my employer but like we said in security you know P Gtfo let's see what IBM achieves relative to their road map let's see what the Google labs deliver let's see what those company listed in in the US NASDAQ deliver and then maybe we'll be surprised and maybe I will be oh damn I was I was wrong >> okay >> so has the chance gone from no chance of seeing one in your lifetime to maybe a small chance.

>> Let's say I've adjusted my prior as we say in Bison probability. >> Maybe instead of 90% certainty, I've raised it down to I don't know 75. >> Okay. >> Okay. Okay. That's substantial. >> I do kind of agree that unfortunately for better or worse, we're now in this era where like research breakthroughs are better as marketing than as research breakthroughs. that people are always kind of overhyping something even though there's often times very real truth the real truth you know in every good lie or every good ad there's always a kernel of truth >> I feel like and in some sense that that it feels like the Google thing was the beginning of the era where like every company is like kind of suddenly gaslighting you a little bit with this

where it's like there is really a discovery but like >> I don't think it's like as you know it's assume it assumes it's completely denovo didn't use other people's whatever and like in reality it's kind of like more mixed. >> I also think the fact of the matter is like we haven't really executed any quantum circuits beyond like trivial gates yet, right? Like if like let's zoom out a little bit like h how how large is the largest circuit we've executed? Not that big, right? Like even though we're getting these order of magnitude decreases in theoretical circuit size, it's like we still have a long way to go. And and I think like the incentives in quantum computing right now are obviously to be like, "Oh yeah, it's like going to happen in one year.

You need to give me $10 billion tomorrow." And and I think like I don't blame people in quantum computing, right? Like I don't know, I almost did a PhD in it and I would say they've had many rough years where like everyone got wiped out and like everyone in the field quit and did something else and like right now they have a bit of there's a little bit of a good light shining on them. So >> Terrun I want to ask about this actually like what you are saying like the Google and subsequent actually ECDSA.fail work has really improve the algorithms that interface with the actual quantum computer right but how far away are they? What you're saying is like you can't actually build very big circuits

and so yeah I've actually never seen that. Like it would be interesting to see that on a graph like how far away have we gotten or yeah have we gotten >> yeah let's make the ZK analogy right like >> in ZK this would be as if I told you in pre- gentry like 2012 paper that like I've shrunk the circuit sizes of like I don't know every hashing algorithm you need by like a thousandx and it's like great what's the large so but there's still thousands millions of gates and then you say like what's the largest ZK circuit that's ever been executed. You're like, well, additive homorphic encryption plus like maybe a couple gates, right? Like it's like under 10 gates pre that era.

>> Yeah. >> And so it's sort of like, okay, there's still this order of magnitude gap that it's like it's like we shrunk we we have two limits. We have an upper bound, a lower bound, and they've gotten a lot closer, but they're not like >> they're not overlapping yet, right? And like I think the speed at which people are telling you they're closing is like a little bit more marketing than truth. They are moving closer, >> right? But the speed at which they're moving closer I think is where the marketing comes in. >> Yeah, I agree with that. >> That's a good insight. >> You just sort of mentioned like would quantum computers break what was it RSA to was it 2048?

>> Mhm. Oh yeah, I know what you're getting at. >> You know [laughter] what I'm going to say, right? So, like this isn't So, you were you're worried about quantum computers doing it, but like let's maybe shift gears and talk a bit about the AI side of things because >> that's a great segue by doing >> I do feel like there's like two guns I' I've kind of been looking for like a meme for this, but like two guns pointed at the same place and like we don't know which one's going to get there first. You know what I mean? Like Yeah. >> Yeah. So I think also for for the audience the context here is that RSA which is one of the most famous public key crypto system has what we call challenges like for RSA of a certain key size we have this public public key

which has been out there maybe I don't know more than 20 years they were published by the RSA company back back in the days and a way to measure cryptonetic progress is to see which of those challenges are broken so we have RSA with a smoky size longer key size longer longer key size up to production type key size like 2048 and earlier that year someone posted on Twitter so it's going going to get very confusing [laughter] they posted on the factors also the solution of RSA-260 260 you're like oh that's a damn small key well no that's 260 bytes which is equivalent to I believe around 862 bits and what What's funny is that this

person I think it's penlu on Twitter they just posted oh here's a factor of RSA 360 and everybody was like what the how did they find that and turn out it was an employee I believe from um I forgot the name some AI company >> cognition >> yeah and then some people like oh but did they use AI so the reality from my understanding is that they use the state-of-the-art cryptonic techniques which is the general number field GNFS And they use the compute infrastructure of that AI company. So mostly based on GPUs and allegedly using AI optimizations. And I think it was last week that Steve Vice from Anthropic published the factors of RSA 896.

So 896 which is slightly bigger than the the previous one. And then again he really explained how he did it. He used the anthropic GPU infrastructure using the ideal idle GPU time. So then again it's an AI company but not it's not AI breaking RSA and there's a huge gap between those challenges and RSA even RSA 1024. >> Okay. Okay. >> It is interesting to me that this was idle GPU time because it means like these companies that have amassed a lot of GPUs if they put all their effort into it they could break something much bigger. Right. But they I mean it's not as profitable like on a cost.

>> We don't know how to how to evaluate the cost but I think I've seen some um some channel I was in Twitter someone stated that to break RS 1024 the CPU GPU dollar cost was estimated to around 10 million uh which is a lot but which is not a lot at the same time. M >> that's less than what was spent on the Navier Stoke solution, right? Like 15 million [laughter] >> right? >> But it's a lot more than my bank account. So if you're going to break my TLS session, it's not profitable. >> Yeah. And it's just just one key. But um >> yeah, >> and the question is uh that is legitimate to ask. You know you mentioned a stock and the other math problems and uh we we don't really know where factoring exactly stands in the

polinomial hierarchy in the computition complexity class infrastructure. I think it's widely believed that it's not easy to solve as in polinomial time in classical computer but you know who knows maybe some AI can improve GNFS make it slightly more efficient. I mean I I do think yeah one thing that is the two things I would say that are worth considering is that a lot of the number seieve algorithms were like hey we chose a particular class field we chose a particular ideal that happened to be easy to write the code for that we would scan across right so like you know you write the number field you look at the class elements you like do the kind of standard quadratic residue stuff but

it could be true that like the parameters people have been using in a lot of these algorithms were like kind of not that great. Like you could have found a weirder set of ideals, a weirder kind of embedding that works better. So that part I could totally believe AI could find a better thing. And the other thing is the implementations of these things could probably be optimized for different hardware platforms a lot better. And that I think those two things probably are really what if if anyone breaks 1024, it's going to come from those two things. >> Yeah. >> Yeah. maybe better way to paralyze it and as you mentioned optimizations for the this is the CPUs we have >> I mean from what I gather is like I this is kind of a joke that you've been playing with the fact that because these

less secure versions of RSA were broken that it would be like immediately all of these other ones but do you like you are seeing the acceleration of AI do you actually think that things that we do know and love could be broken very quickly like you know maybe yeah I'm just curious what you think on the on the breaking front. >> Okay, it's really good that you mentioned this. I have a a blog post bfsswa.blog and couple months ago I published a post titled why AI why lms are not going to break symmetric cryptography. M.

>> So why the AI models even the front tier models even the super AGI from my from my perspective are not going to break things like AES like chass like B3 just because they are mathematically secure. They are as hard as uh SMP hard problems. Let's say it's a rough approximation and I try to articulate why I don't think we'll get a breakthrough in terms of you know as attacks or shri attacks because it's not if we mention again stock or any math problem it's a space where you have you work with mathematical structures when they're complexity and they're all intertwined and it's really hard to to figure out but the point of symmetric crypto is to eliminate any sort of math

structure that could be used that's why for example In in a function like Blake, we combine the XR operation which is addition on on GF2 with the addition on larger number on the 32-bit numbers. So combining those two in addition with the bit shift breaks any math structure so that you can't really make any ma mathematical sense of that. It's just you know random so to speak you know major speech marks and since there is no structure to to analyze there is no structure for LLMs to exploit and make sense of >> wow >> cuz the high level >> unless they solve pal NP and finds a trapdo trapdo function for you >> which would be very fun. [laughter]

Can you give some examples of the symmetric crypto cryptography and then I want to also talk about asymmetric cryptography because maybe you have a different opinion there but give me some examples of that. >> So I think it's time to mention the the P word poson and posidon is actually interesting in a sense that it's so spous and other types of um of crypto crypto schemes and poson is really trying to have the best of both. So on the one hand it has a very clear well- definfined mathematical structure.

>> It's operating on field elements and it's doing field operations linear algebra. So things that are really clean from a math perspective. But the problem is that if you only have that you can describe the hash function as a list of equations and then breaking the hash is essentially boiling down to solving those equations. And to avoid that what Poseidon is adding in addition to those math person is one small portion which is a kind of Sbox which has nothing to do with the finite field or mathematical structure behind it. And because they combine this weird transformation is breaking the whole structure that give you equations. And

>> okay >> because of that the idea is that you go from simple equations to extremely complex dense and long equations. And the question was well is it secure enough? >> Having all those finite field operation and just a simple one a small one that is not math friendly. And as you may know, there's been a lot of research, another be from people from the ETH foundation, people like Antonio Seno and others. And it turned out that Poseidon, especially Pose Island 2, is not as safe as many expected, me included to be honest. Can I ask you though which like so is Poseidon actually a symmetric cryptography but then there's an additional thing that is asymmetric or

is it it's yeah I just I'm trying to categorize it where you're >> is purely symmetric in spirit >> okay >> in the sense that we're not trying to to develop a system with a trap door or back door something that you can invert only if you have um some some secret for example there's no discrete algorithm no factoring so it's working on typically from prime fields. So there's no no modulus to factor. You just have the parameters and and that's it. But what's interesting for people who have worked in the field of for example white box what we call white box cryptography and I spent part of my time when I was a researcher trying to develop backdoor symmetric cryptos schemes like a block

cipher hash function that was you know outwardly secure. However, if you knew some secret, some back door, you could break it for some deficient of breaking. And when you think really hard about that, you realize that what you're trying to construct is a kind of public keyy scheme whereby something is easy in one direction and hard in the in the other. Um, >> and there hasn't been any really convincing backdoor symmetric primitive from uh from what I know. H >> which also kind of dismisses the argument that AES or or have a back door which I've heard some people saying >> good conspiracies but to your initial question and I think so yeah because I

don't on symmetric crypto it has some structure that makes it circuit friendly and then as JP was saying like a tiny bit of no structure >> and that's where >> make it secure as a hash function >> but is where is the point where's the attack point then >> another reason one benefit of because you have this this structure It allows you to reason about the possible attacks about the limitation of of attacks which are many differential cryptois attacks. So you can prove that certain types of attacks would not work more easily than when your function is a complete mess of operations like uh like blree. Yeah. And the attacks develop on poson are really smart and sometimes kind of bypass the foundational security

arguments of the designers. Can we maybe specify what kind of attacks we're talking about? Like is Poseidon broken or like did we find a huge it depends yeah because when you look at the Boseidon paper is not one fixed hash function. So you can vary the number of rounds and one important parameter is the underlying finite field. So I think the security margin security level is very different whether you work on know large finite fields like 300 bits or a smaller one like 33 bits. So the attack techniques work a little bit differently is posit broken.

I think some of the attacks were so posit essentially a sponge function. So the attacks were what are what we call CICO cycle constraint input constraint output. >> So you fix some of the input and you fix some of the output and you should find a message. Your goal is to find a message that connects the two. So you can see there's a kind of preage attack in some way. But ultimately those attacks they show that you can do something that you you should not be able to do if the function was really structurless and insecure for this very general definition of secure which means nothing really differentiates it from a random function.

>> However the the system that you posidon I might be wrong. So my degree of certainty is uh is not 100% but I I don't think there's any you know meaningful impact on any uh other system that use today but yeah please fact check that and I'm I'm not I'm not fully sure. >> Okay so the issue here is researchers are exhibiting behaviors of poseon that are not supposed to be there. It doesn't break the systems that use it but it also kind of breaks our confidence in the design. Is that correct? I I don't know to what extent it's really you know invalidates the claims of the designers but let's say many were surprised to the point that the e foundation is to my surprise considering a function like blry

>> which initially we believe was much less you know circuit friendly >> than than poson but I I think it's a yukan project where they found super small black circuits I don't know how they did but you see really a sharp decline in the circuit size it's But asception as possidon in some settings and also full disclosure for the the audience I'm a co-designer of Dex 3 so I'm very biased I know [laughter] >> Nico is is this the flock work? >> No those are unrelated things. Um >> okay >> flock is work about proving many hash functions fast with a sort of dedicated proof system >> and it doesn't matter which one. It doesn't matter which hash function. I understood that the flock paper actually

also prompted partly the move away from Poseidon and I wondered if it was because it works better with Blake 3 or something like that. >> So flock I think was designed for binary fields which is you know bitwise operations and which is what our like classical hash functions do as opposed to poseon which was designed for finite field operations. >> Got it. So yeah, part of the shift comes from okay, actually we can do proof systems that work with bitwise operations >> or proof things about bitwise operations and those are still efficient >> and at the same time Poseidon and the trust in Poseidon starts to get shaky and therefore there's like an option that maybe didn't exist before that like that's why they were focused on Poseidon.

>> Justin Drake summarized it quite well at the time. He was saying we're dropping the snark friendly hash functions and we're picking up hash friendly snarks. But if is very conservative, I mean in a good way, very in the sense of [clears throat] very prudent. So like I was reading a buzz this morning of someone affiliated to E foundation arguing that oh B3 is maybe doing too few rounds. Uh I don't know it's maybe it's not secure enough. But I mean to be honest I I've been doing crypto analysis for 20 years now. a lot of it about hash functions and it's something where it's really hard to to have an opinion and it's very easy to have an opinion that is not really you know the optimal one

as far as R is concerned and because you have this feeling of security when you do a lot of rounds you have big security parameters like oh let's use RSA with 10,000 bits or let's use AES a Q256 instead of 2028 because what if what if what But I realized that cryptographers are not the best at risk management. >> They tend to be paranoid which is good in security but can you can also be unhealthy paranoid and when you argue when you argue what if without really reasoning about the likelihood of it it's not best approach at uh security engineering.

>> You end up with sometimes very inefficient systems. >> You have a famous article on the topic right? >> Yeah like shassri. So I mean that's one of my pet peeves and I I also feel kind of guilty for that. So SH I'll make it very short but Shri is doing 24 rounds whereas initially it was submitted with 18 rounds and part of the reason is historically I found an attack air quotes which is really not an attack but a property very artificial but which contributed to the getac designers you know increasing the round number to 24.

And the best attacks on chass are on three, four, maybe five rounds. And even then, so we're doing 24 rounds whereas maybe 10 would be more than enough. And it's not just hashing. It's in MLCM. So the PQ post quantum cryptography is turned on MLKM, MLDDSA. A large of the CPU time is spent doing hashing or random generation using KSAC. And we are wasting tons of CPU cycles. It drives me insane. But you know, that's how it is. is not going away. We had to live with that.

>> So Jiffy, are you telling us that you designed Blake 3, that you slowed down Ketchup, and now you're joining Cloudflare to push Blake 3 onto the world? Is this your master plan? >> Is that the evil plan? And you figured it out, okay, [laughter] from the beginning. [gasps] >> But jokes aside, you have a very nice article called Too Much Crypto that talks about this, right? How like cryptographers take too big security margins. >> Yeah. I'm curious if the sort of AI optimizations we're seeing now, especially on RSA, make you want to revise too much crypto or >> Not at all. Not. I mean, this article that you mentioned, it's #209/1492.

>> Wow. >> Uh, it [laughter] posted it uh in December 2019. I presented it at RWC Railroad Crypto in 2020 just before COVID fun times. And yeah, I'm making the point that Blake two then Blake three with so few rounds is widely sufficient in terms of security that Sha cha chaa 20 so 20 rounds would be as secure with eight rounds and that ketch doing 24 rounds would be as secure with 10 rounds. So I had a name for it. We call it kitten with a K. And seven years later history so far is proving me right at least not wrong. So maybe people many people's objection was not really constructive.

It was just what if objections it make me feels nervous to have raised number of rounds. Yeah. But you know confidence is good but you got to look at the facts and the amount of human time CPU time tokens thrown at this problem of breaking those functions and yeah I don't think ever in humanity in the history of humanity will break as or three or three >> and this I guess goes back to your point before is this because there's symmetric cryptography you just don't think that's going to happen >> ultimately I don't like to use this word exponential because exponential is relative to asytotic growth of a parameter. But if you see those functions as abstract uh problems, the cost of solving those problems

really grows exponentially relative to the input size and relative to the number of rounds. So it's not a matter of solving a problem. It's >> if you put it down in equations, the equations would be huge and um impossible to solve. >> Got it. I want to just go back to a kind of that original point because I had sort of asked what kind what is symmetric cryptography? Give me some examples. We talked about the Poseidon case which was a huge deal by the way in our telegram big telegram chat. We got to see all the arguments. It was very exciting. I feel like we maybe should debrief properly on that with some of the people involved eventually this year. I I had the second question which is like can you give some examples of asymmetric cryptography and do you feel

those are more at risk? Oh yeah, that's where it gets more more interesting. So historically I've been more involved into symmetric crypto. Why? Because I've never been very good at math and public crypto is essentially math. >> But it's way way more interesting both designing, analyzing, breaking. I mean that's really the cool part about crypto. It's all publicy crypto that you find in uh I don't know in proofs and threshold signature and and whatever. So the attacks are also way more let's say interesting from a math perspective.

>> So in comparison sim crypto it's mostly differential attacks. So differential attacks is very stupid. You take one message and you compute the algorithm and you see what output you get at the end. So differential attack is about like flipping one bit and see if the output is changing randomly or it's changing with some some pattern that you can expect. Ah and if you can figure out a pattern then you can break it somehow >> essentially. Um okay so it comes on different names you linear attack differential attacks rebound whatever but ultimately the same thing >> funny >> and say we were talking about AI. So there is the uh the NIST the US standard agency uh that is running a project a

competition for standards for new BQ uh signature standards. So we all know the selected MLDDSML came a few years ago 2022 uh but they're still running the competition to identify different types of algorithms and a couple months from now I think was in July anthropic announced that they they had found let's say very improved attacks in one of those candidates called hulk h awk >> um which led to the designer to withdraw the submission because the attacks is really superior to the state of the art.

>> Uh which leads us to conclude um well on this part at least that AI is a great tool to find attacks on asymmetric schemes. And another evidence of that is if you look at the NGCC so the next generation commercial cryptography competition from the government of China. >> Yes. The one that just came out recently. >> Yes. So, you know, China likes to copy things. You know, they're very respectful of what the Americans are doing. They tend to copy their the systems. It's no nothing new here.

>> So, they're running a competition really based on the on the mechanics of the NIS competition to the point that the call for submissions was not world toward but very very similar to the NIST documents. >> Okay. >> And a few days ago they that the Chinese government announced a list of candidates. >> Mhm. And interestingly like N it's all public. So the submissions are public the source code the specification and now it's a lot more fun when you have AI because I remember back in the days when we had the first competition the first day as a student you were so excited. Oh let's break all this functions you know I want to be the first to get my paper right to get my name and it was really a rampage >> but a lot of work I bet. Yeah, it was

really we're using our brain, our hands and um writing paper in Latic like like caveman [laughter] >> and and now you know what we did in days or weeks. It takes minutes or hours. >> Wow. >> It's exactly what some people did. >> So I tried it myself. I didn't have much time. So I opened an NLM. I think was GPT soul and I have the Daybreak Blue extension which allows to do slightly more offensive research and so I downloaded a bunch of submissions like maybe 10 I don't know and I asked uh GPT okay please look at those and tell me which one looked the weakest the the most fragile and it said oh this one and

that one then I said okay now >> break now break it please [laughter] it's like yes sir and 10 minutes Later it found an attack like you know that pretty killed some of the submissions but someone was much did much better than than I did. Maru Sarinon was I mean one of the best cryptographers that I know cuz he's great at math great engineering a lot of respect for Maru and so he he he downloaded like the entire catalog of submissions and broke with his AI models. I don't know which one he's using but I don't know how many he broke. He said a couple hours later he said 14 like 14 submissions out of maybe I don't know 50 60 whatever and he created a website ngcc.dev dev with reports of submissions of his attacks

and I'm reading it right now. Um so reports >> 142 active findings across 81 reports 65 impermission 49 design. So meaning like 49 have been attacked maybe not completely broken. >> And really what what what I find scary is that those submissions they are created by human beings. >> Yes. >> Which supposedly have access to computers and AI tools. And like okay why didn't they ask their favorite AI model to to a submission? were well so confident in their designer.

>> I bet you it was because they had to submit it two months ago >> and the models [laughter] have improved fast enough that it like I don't know >> I don't know what was the deadline by the way and also is what else I was wondering about so obviously May submissions are from from China some of those are from researchers including European or even American researchers and I was wondering if you're going to get in trouble for designing a Chinese standard but that's that's a different question. Yeah, actually how this also also made me wonder like how does NIST work? Like does it have to be Americans who submit? Like is there a rule on who's allowed?

>> No, I think it plays very little role. I mean, >> okay, >> even a submission with the best in general merits would have been co-authored by a Chinese or Iranian. I don't know if you would have made a big difference. And also I I think even back back then not all the NIST employees were US persons because you don't need a clearance for to work there. And you have as a matter of fact uh the US standards AES chass are designed by Belgian cryptographers which are not known to be in the an enemy of the US but you know these days we never know.

>> The inside joke is that it's always the Belgians. >> Yeah. that someone criticized their their designs and they went and broke everyone else's designs and proved theirs and they said take ours. >> Yeah. And and we can expect I mean it was published like three or four days ago all those submissions and in a month from now >> it's impressive. Are the attacks confirmed or is it just slop and the AI is gaslighting us into thinking it has attacks? >> No. I I think I think a lot of the findings are actually correct and but the downside of that is that it's really bad news for researchers because if AI is finding all the attacks and you don't get to find it, you don't get you know the enjoyment the dopamine kick of

finding an attack. >> I mean it's so addictive. I remember back then you know you really get this kick of you know like you get an exploit working on a computer makes you feel so good and you want to have to experience that again. And now the young you have all the low hanging fruits that young PhD researchers would break uh back in the days. Now the AI is finding everything. >> Yeah. >> So you don't get to do the work. You don't get don't get to publish the paper. So what what do you do with your life? [laughter] >> Oh god. >> This is this is interesting.

>> I I don't think this is a question that's only true for cryptographers for what it's worth. >> I feel like it's so funny. I'm I'm so used to hearing builders and like engineers and software engineers specifically being like what's what's the point of my job now? But I didn't know that the hackers would also be like, "What's the point of my job now?" >> Well, [laughter] it's just more that everyone's a script kitty. There's no, you know, that's that's how you have to view it. Everyone's an operator. You're just like a factory operator rather than, you know, like an art artisan. [laughter] >> I think one of the hardest problems from my perspective in computer science was designing web applications. I couldn't do that for my life and now just okay do that enter enter.

>> As a non-developer it's become incredibly fun to be able to build the things I always wanted and >> couldn't and now I'm just >> call but I don't care. [laughter] >> So JB, have you been doing more experiments with breaking things with AI besides the the Chinese competition? So once I got the daybreak blue access to GPT so I I think anyone can request it and um I got it in I don't know few hours. I did something completely stupid. I asked it. Okay, go to E print the ISR ein the preprint server. Go find some obscure ciphers for submitting in the last few months [laughter]

and tell me about it. And he first he gave me a list I don't know 20 and he said oh he said oh it said whatever. Oh this one looks a bit fishy. I'm like okay go break it. Go break it. And something really weird happened. There was this paper 200 pages. I kid you not, 200 pages and there was a huge cryptotic flow. I had to verify it myself to do the math by hand to verify that the AI was correct and it was total break of the cipher. I think it was a block cipher. Really stupid. Uh you know in very short when you have a cipher you you encrypt and then decrypt, right?

With this cipher there are some cipher texts that you could not decrypt that you could create by encrypting but not decryting. And what I was really wondering is okay the researchers they use it was very clear they use AI to generate lot of the text but for some reason they did not ask the AI is my cipher secure [laughter] which this is the first thing the only thing you want to ask >> it's very interesting that people are not doing that so the first time I was like okay it's really cool let's do a blog post about it and I broke a bunch of others I just emailed the authors and and one was also designed by I'm not giving the name but it was based on some very simple well understood construction you know LFSR linear back shift register

and it was designed by really author that looks very legit very expert in their field and there was a huge flaw then again the feedback polinomial was not primitive it was not even irreducible which is the very first security criteria you want to check when you when you do that kind of thing so when the AI gave me the attack I'm like come on you're alinating it can be That's stupid >> to do the math again by hand and in fact the polinomial. So yeah, a lot of fun. But um >> huh. So far we've talked quite a bit about the AI/cryptography and the breaking front like how it's showing vulnerabilities. It's speeding up the process from like publishing something and then have it be broken. It used to be a little bit more time in

between. But what about the other side which is like building with this stuff? We have recently had an episode on with Yukon and this auto research is really emerging quickly like really since you know March May like it's very a very fresh thing but have you been playing around with that? Is this something interesting to you? >> Do you really believe that an AI can do something better than Blree? I don't. [laughter] >> Have you asked? No. But [laughter] >> use daybreak blue and >> but I use daybreak blue to try to find better attacks on free but yeah I didn't find any but like I said it's probably not the best use case of finding

symmetric attacks because many attacks are really based on empirical findings on statistics and getting those statistic to run for a while but yeah to answer your question I have not I think two oct two or three years ago they're really at the start of the uh GPT D4. I asked it to design a cipher and it was not really doing a great job, but I believe that now >> so I don't think that it would do a better job than a top 5% cryptographer. It could do a better job than than maybe a top I don't know a top 50%. It would be better than the average cryptographer, but not as good because a lot of the ingenuity and quality is really in the details and some choices

that are here for a good reason and also based on experiments cuz it's engineering such a trade-off. You're not trying to do the fastest. You're not trying to do the most secure. Yeah. >> You're trying to find a sweet spot and also in terms of simplicity. It's really hard for an AI to to way to appreciate those trade-offs. So, I'm not saying that Bl is the best trade-off. It's one trade-off among among others, but yeah, really appreciating those nuances and also the sheer beauty of it. It has to look to look cool, to look simple. >> Yeah, in your in your world, you don't see you don't see it replacing kind of what you guys do yet.

>> But there's nothing to replace anymore. I don't think again I might be wrong, but I don't think we'll need many more new hash functions, many more symmetric ciphers. I mean we have a yes we have cha and then from this point it's really diminishing returns but yeah again for asymmetric crypto I think AI can be more helpful by helping us you know find the >> also on the building front >> also on the building front and also to better understand the limits of the attacks because when you don't know what you're doing when you're not really confident in your design you tend to be conservative because never know what's >> what are the best taxs >> so instead of spending so Instead of spending three, four, five, six years, you know, with thousands of human hours,

even more than that, of human time of cryptoysis, we can probably really shrink the timeline to assess the security of of a new cipher and to maybe improve it. Cuz the problem with those competitions is that you cannot really modify the ciphers. After 5 years, you've learned a lot about what works, what doesn't work. But you're not entitled to, you know, to learn from that. you can't say okay now I know that uh I only need 10 rounds of sketch so I'm going to stand that sketch with 10 rounds I'm going to make the small tweak so I don't know what the Chinese would be doing and I don't think they will do that but it would be much wiser so in terms of engineering to as you learn from the attacks as you learn from the AI findings and also human findings try

to converge toward this sweet spot of security efficiency and simplicity >> I don't know if you've heard about the term and this is really just starting to bubble up in our community, but like generative cryptography. Have you heard that? >> Oh, yeah. But not in a not in a good sense. >> Not in a good way. >> Yeah. I don't know. [laughter] >> Yeah. I'm curious what you make of it because it what you just described sort of sounds like that though, right? Like it's like you're building, attacking, building attacking. >> Does it stand for crypto designed by AI? Like crypto results?

>> I think it's both. So I will probably do an episode or two on this coming up. And I so I am not in a position right now to talk about to to explain it because I actually don't exactly know the definition. It's just sort of it's been sort of floated as this idea of having the build and the break happening [clears throat] really quickly together and so you're almost like you're constantly creating new cryptography like it's you're constantly building and breaking and building and breaking. >> Yeah. Such a waste of tokens. Um but [laughter] along those lines what what we're doing I say we but um I'm not doing much of the work had this idea with with Zuko from Zcash of doing a project similar to the you know you mentioned Yukon before like zk.gov golf

is this fail and what we wanted to do is something similar but for hash function attacks >> motivated in part by you know proving to the world that no AI can bring blree >> or or Kac for that matter >> so we were working with uh with ads on that and I don't know when the podcast will be released but probably around the same time we're going to announce a project >> cool but that's more of a bounty isn't it >> I think Our co-host Kobe is also on Hashm right?

>> I believe he is. Yeah. >> Oh, cool. >> Yeah. >> Nice. Nice. >> Yeah. Yeah. I don't know. I think I expect the unexpected. Also, the A models, we haven't seen the end of it. The models are getting better and better. And something that we did not mention, but it's also really important. Uh there's two ways to break uh a cipher. You can break the the math, the the specs, how it ought to behave, or you can break the software. >> Oh, yeah. So AI ask we know is really good to to find you know bugs in software and exploit them. So yeah, maybe we'll see more more attacks and um and many of those submissions broken either on paper, on software or both.

>> And I'm looking at Marcus this right now and uh some of the submissions are broken in three ways like on paper implementation and also side channel in terms of you know constant time constant timelessness. >> Actually mentioning this paper and implementation link a lot of people have been looking at formal verification these days. Have you at all? Not really. I know relatively little about formal verification and for a long time my my view is that you know in the process of quality assurance of software you had different levels of of testing. So you need testing, integration, testing, different ways to test stuff and that formal verification is really at the end of the curve where you get the missing

returns but where you need this levels of offer for certain types of software for example well critical infrastructure avionics and whatnot for cryptography I don't feel that it's been extremely useful because I think it's been most useful when proving properties of protocols rather than proving that one implementation of AES is actually correct and equivalent to other AS implementation. >> Uh you can also prove that certain classes of software bugs are not present but I mean if you have a buffer in your as implementation you're really having the wrong job.

>> Mhm. However, and maybe the reason why you mentioned that is lean lean for. So, one idea that I had was to formalize hash functions as lean statements. >> Mhm. >> And try to describe attacks as you know lean lin results. >> Mhm. >> And likewise try to automate that process finding new attacks that could be proven in lean but turned out this is not impossible. Everything is possible in theory, but empirically the type of attack differential attacks seem to be pretty hard to well again to automate and to formalize in lean >> um for a variety of reasons. So I'm not the only one. I'm not the smartest one

in that space. I wouldn't be surprised if some people have very interesting results, but I don't think symmetric crypto is the best use case for lean as opposed again to to asymmetric. >> Cool. I mean, I think the only thing I can really say is like formal verification used to be cheap. It's now 10,000 times cheaper. So, there's going to be something that previously was not. Everyone was like, "Ah, it's not worth doing because it's just like too expensive or too nuanced or whatever." And now it's like, "Okay, well, I can spend an afternoon and get a prototype." So, maybe maybe it's like it will find a way. I think obviously certainly in like proving protocol hacks, it's like been clearly very useful. I think like for instance just like I don't know there's

just like a million ZK papers and probably 95% of them are wrong or broken based on based on my very tiny sample of trying to reproduce lean proofs for like 30 almost everyone has had some type of issue >> related to that also tried to use AI to verify the proofs of York papers and it found small mistakes um >> that I'm not surprised a human reader can find those as well. >> Yeah. Yeah. I mean the products also so so hard in crypto so it's really hard to get right >> but most of the times these are like okay someone slightly miswrote the math but we know what they meant and if we put in a bit of effort >> the result is not incorrect just the proof is not fully fully consistent.

>> Cool. >> Interesting. >> Oh speaking of which Euro Crypto that's crazy they received around 1,000 submissions. >> Yeah. >> So >> well that's like the research that we're seeing for for ZK Mesh as well. It's just it's so much easier to produce papers at this point that you have but all it does is as a curator you just have more to go through and then you're trying to use tools to help you go through because there's more and so it becomes this funny funny setup. >> Although if everyone is submitting generating text with AI but not even checking if the construction is secure with AI then we have an easy AI reviewer which is [laughter] check that it's secure and kick it out otherwise. I think the bigger thing to me was more

just like the mapping from the paper to implementation was the type of thing I was trying to verify like does this code match what the paper says and like that was very you know it giving in all this extra stuff of like the RAM model might be weird they might have re you know the the exact sequence in the protocol might not be what that you use in practice because it's inefficient whatever fine but even then there were like some tiny errors where was like finding certain things that like overflowed without gates It's finding like like I was actually surprised by how many you know kind of papers that had like you know one academic implementation randomly on GitHub how much they actually did differ from the paper. Like I thought kind of

people in these like first implementations literally just tried to transcribe the paper and it was not even close like I was like I was like surprised by that. So it was like a good experiment for again something before I would have been like ah this is going this is like a whole month's project why would I do this it's just for a intellectual exercise whereas now it's like a I can do it in 10 minutes >> very cool all right I think we are at time JP I feel like we could continue talking for a long time there's always kind of interesting ideas and perspectives coming from your corner so I'm so glad you came back on the show thank you for covering with us sort of your take and maybe updated take on quantum from computers and their threat and obviously like the changing tides

with all of the AI both on the attack front and the building front. Yeah, thanks so much for that. >> Thank you Hannah. Thank you all. Thanks JP. >> Great chatting. Thanks for having me. It was really great chatting with you JP. >> Thank you. It was a really fun discussion. >> Nice. I want to say thank you to the podcast team, Henrik, Rachel, and Tanya and to our listeners. Thanks for listening. [music]