ZKPassport’s new Private FaceMatch

Transcript

Good morning, GMGM. Thank you everybody here for taking choosing [music] to spend your time at our third inaugural noon. [applause] >> All right, >> cool. Yeah, thanks for having us here and um thanks for coming to the the workshop. Looking forward to looking under the hood of what we built with uh with private face match. Really excited about it. >> Yeah. Um just as a kind of like introduction who here has used a passport.

Oh, that's a fair amount of people man. Good. >> Yeah. So you should have like some uh rough understanding of how it works at least from a user perspective. >> And for those that haven't, we're going to go over just high level like what we built and how it works. Um so you can kind of understand um what it's all about. And then we can then after that we'll go into the details of I guess also explaining what private face match is. um how we how we achieved it um while retaining like preserving user privacy um and then yeah I guess dive into the weeds a bit more and and actually uh unpack the technologies that we used.

So ah thanks >> cool. So yeah at a high level uh what is ZK passport? It is government attested identity. So everyone in this room your passport they all have an NFC chip on them and on that NFC chip contains your personal information embedded there and also signed over by the issuing government creating an unforgeable tamperproof seal. And this is the this is the key here. And then our circuits uh essentially verify that signature.

There's a there's a chain of trust from the the root certificates of the of the country. And this is maintained by um they're called Ical. They're like under the UN and they basically through diplomatic channels they will collect all of the different root certificates from the different countries and they put them in a master list and that's available for everyone to get. And then we have a uh certificate registry that we maintain where we basically get all of these different certificates. Um we across different master lists. So each country has their own master list of what they essentially consider to be a valid government root certificate. Um there's an overlap mostly and then we we go through we dduplicate those and then we merkealize it and then we store the

root onchain in the certificate registry. And so what you're proving against with a zika passport proof is that it's signed by a valid government. Um the signatures are you know are valid. There's the the root one. There's the intermediate one and then also the passport data itself. And so this whole chain of trust um the the root of trust is that that that route essentially for that Merkel tree. Um so you're not really you're not trusting us at all like the the provenence of the signature is is kept intact through the circuit. So we're just basically providing the circuit for that logic to be verified.

And then obviously because it's a ZK you can also have this like little um you know trick where you make it private and then now it's like you get the benefit of privacy too which ultimately means that you can have selective disclosure. So you can selectively choose to reveal um certain parts of your passport and that's what our protocol provides. So you can so when you plug this in as a developer you can you can specify what it is that you're asking the user to prove. Um and you can also have other cool uh approaches to this where it's like prove that you're you know from a country or from a set of countries. So for example uh Devcon connect they provided ticket discounts if you can prove that you're from LATAM. So we have a defconet integration and hundreds of

people use this to come through and get a 50% discount um on their ticket uh without revealing anything else about themselves except they're from LATAM and so yeah this is the flow you want to >> this is the interface I think all of you are familiar with it so you scan your passport or ID cards that works as well especially in the EU residence permits too you lo it loads into the app stored encrypted locally on your device never leaves it and from there you can answer to specific uh request from website such as was the case with def connect or as a case with the Aztec token sale. So that's the kind of request that's done is up to the website through our SDK. Uh

for example, in case of a stick uh stick token sale, you have sanction checks and privacy face match. So you have a good guarantee that's the person on the ID and some other such as being over 18 and other details. So this sanctions check that we do this is also a really cool uh ability to prove like full name and uh you know country and date of birth now because we have these these credentials to use and so with the uh the Aztec token sale what that ZKYC is doing is um with a sanctions registry we maintain a similar approach where we have like a route um we've merkeized four different uh sanctions lists from like the OFAC SDN and one from the UK and you're proving um completely privately that you

are not on that sanctions list which is a really really cool uh primitive to be able to do that and not have to provide your personal details to some provider that then checks themselves. It's kind of inverted. It's like um you know you give me the sanctions list um I'll tell you if I'm not on it um but what I'm saying that is is truthful because you're uh you know binding that in a ZK circuit. So this is how you'd be able to integrate with us is with um several lines of of JavaScript, TypeScript. >> Yeah, this is SDK. So how you would essentially communicate with the app. Uh the SDK takes care of everything. Uh the websocket connection to the mobile app and to an encrypted receives back the

proof verifies them for you including the public inputs and then you can use those proof either offchain or onchain. You can pass it to your smart contract. We have the smart contract verifier which has now been deployed on mainet and also base as well. We're going to deploy on other networks too. >> Yeah, we've just um just deployed a mainet which is exciting. So now we can go multi-chain and we've designed it in a way so that there's this universal address through like the create 2 um with a with like a salt. So what that means is there's a single address for the re verifier across all chains um even the ones we haven't deployed to yet. And you have this query builder as you can see where you can just like chain on different things to kind of add to what you're asking. So here it's like

career builder dot greater than or equal to um for the age 18 that this is like 18 plus basically and you can kind of keep going. There's like uh for for for nationalities um uh for sanctions list for um anything you can think of like to to reveal that's in any kind of combination of the passport data. Um, and then one feature we'll be adding soon to really help uh with integrations is like a form where it'll you'll be able to specify exactly what it is that you want your users to be proving any kind of combination. So like you know visually through a UI and then it'll generate the code for you. Um so like a code generator for Solidity. So then you can just and TypeScript. So you can plug

this part into your front end and then plug this code into your Solidity contract um to bind it to uh your particular service. Um and one other important concept here is you have this typically it's called a nullifier that we call it a unique identifier. It's a bit more kind of user friendly and the idea is that for every passport um and service the unique identifier will always be the same. So this prevents like double registrations. This is essentially how you achieve super resistance but it's scoped to the uh the web dom domain name of the website so that you don't have a global unique identifier that then you could have two services comparing knowing it to the same person. Then you can have a subscope. So perhaps you wanted to do

like voting so you have like a different poll. You can also like scope it again to a particular use case of of your service. And this is also the stuff that you'd bind to in the smart contract to ensure that it's actually for your service for a given domain name and like like use case. >> Cool. And then onto the title of the workshop. Um, private face match. So essentially this is a more recent feature. um a lot of like there's a bit of like there are other identity solutions DK ID solution but the main problem we have is that you can use a bunch of documents that may be or not your own. So you could let's say pick up your brother's or your father's or mother's uh documents. So you need a way

to prove that you're the person on the ID and that's what we provided with face match. So what it does is it will pull out the photo from your ID compare the camera feed from your phone to that photo using uh face embedding compare the cosign similarity and then also provide a live check. So you need essentially to look in specific direction we look at your gaze and that prevents essentially try to spoof the system uh using for example putting your ID in front of the camera that wouldn't work through the liveness check. So this is all done locally. The models are all stored locally. Some of you may have seen uh complaints about the size of the app. I believe um this is one for

>> the reason 170 megaby. >> We have absolutely no connection to whatever uh services that was intelligent services that was mentioned. Um the actual thing is the SRS. So this is the trusted setup for mastc 128 megabytes because the circuits we use are up to the subgroup to to the 21st and about 180 megabytes of uh models for private face match. We have a small one for OCR but it's not that big and some other artifacts essentially. And so yeah the real magic here is that you're essentially being able to prove that you are the same person on the document but never revealing a face to anybody. Um which I think is like really cool and really magic. This is like really the

magic of ZK, the kind of stuff it can do. Um, so everything up to that point is cryptograph is like is the cryptography part of it. So it's like completely sound um and secured. This is using a mobile TE guarantee. So it's still really strong. Uh, but it's on for iPhone especially. I would say like a state level actor would be able to to bypass this. Um, but you're not you're not doing it at a lower level. It's kind of like a nice icing on top. Um, and so that's what we're going to explore today is is how we actually achieve that through a service called Appest. Um, and then uh, yeah, we can also do some some I guess coding part of this as well. And so there's an equivalent on Android called um, play integrity. It's a very it's a very similar setup.

>> Uh, I suppose I can just dive into the explanation now, right? >> Yeah. >> Next slide. It's QR code. All right. So yeah, so the way it works is um, >> with Apple iPhone. Do you want to show the diagram or straight >> away? I mean, it's a pretty full on diagram. >> Yeah. >> But, uh, yeah. Yeah. [laughter] >> Yeah. We got some visual help. Uh, it's going to be a little bit scary at first. >> Uh, [laughter] but we can zoom in as we explain. >> Maybe I can explain it like myself first. >> Yeah, we can give a rough explanation and then jump into the details afterwards.

Um, it's slightly different than Android, but the principle is similar. >> Yeah. So, app test for iPhone. The way that it works, let me just bring this up as a nice. >> Yeah. This Yeah. So, the way it works is you So, every iPhone um on the secure element or the secure enclave rather um there's the there's a public there's a key pair and that's called the device key.

And Apple has the public key for that. So they know about basically like uh they have this key pair that Apple knows the public key of because they created the device in the factory, right? So they have a link, they know which devices are genuine um iPhones. And then during this process of app test, there's a new key pair generated on the enclave and then there's a certificate signing request where essentially that new key pair um is signed over locally by the key pair that Apple knows about the device device key pair. Uh it's sent to Apple. Apple can see that yes, this is a genuine iPhone because it's signed by the device key pair. And then they'll use their uristics to determine whether

or not it's jailbroken. So there'll be like thousands of of like efuses on an iPhone. So if you ever try to kind of get into it and tamper with it, they'll permanently blow. Um they'll use all these other uristics and checks things like uh which version of iOS is it? What's what's the bootloadader version? um all all these checks basically and they they do a really really good kind of best effort job to to essentially determine um that this is a non-jbroken um this is you know it's an authentic um Apple iPhone that's running the correct code. This is essentially trusted computing uh which is for iPhone and then also for for Android.

And by the way this can coexist with um I think this is a often kind of um it's not really like a dichotomy. I think it's a false economy of having trusted computing and then the alternative of like being able to sideloadad. Apple should open this up and I think what would be ideal is a way to have both trusted computing but also having the ability to sideloadad and it just means that you wouldn't get the same guarantees, right? But you'd be able to sideloadad anything arbitrarily. Um but then you you have no guarantees that it's essentially that it's running um under these these constraints. And so the constraints that exist Well, I'll continue the flow on. Sorry. So you get the the um the signed key pair, the signed payload back from Apple and then now you you have a guarantee that like at least from this signature from Apple

that the key pair on the secure enclave um is on a genuine device. And so then moving forward you can have the app perform arbitrary logic. So for example in our case you did the inference over the you know the camera feed when you're doing the face scan and it will compare that cosign similarity which is essentially a value that um allows you to compare two vectors. So we get one from the passport photo, one from the face scan. We compare the two. If it's above a threshold then it passes. And then over a custom data structure that we've got we we we save like the embedding value itself which is the actual face print. And then we save like the cosine similarity that was achieved like the average all these different

custom values and then we we sign over that data get a hash of it and we sign over that with the key pair then we um in our circuits we'll we'll prove the entire chain back to Apple's root certificate and this is the chain of trust with with um private face match. So for Google it's Google Play integrity's uh root certificate. Theirs is a lot more difficult. it has like uh five different um >> so yeah the the actual certificate chain varies from manufacturers and the signature algorithm are different so it's a bit more messy than iOS we needed to generate more circuits the similar way we did for passport but we generated a bunch of communations we had to do the same for Android uh and various so you can have like one two three intermediate

certificates so that varies a lot in terms of like circuit size and then you have to you need to add separately the play integrity logic which is separate from the k stationation. It's not a unified thing like on iOS. Uh and then you have like the custom OS like graphine which we do not support yet. Uh that don't use play integrity. Uh we would need I looked it up. Uh we will support later. You would need to check specifically the bootloader version. I will need to look at the security guarantee of that but apparently it's pretty good. Um so yeah, Android is a bit more messy being an open system.

Apple's a lot more elegant in their design and simple. It's just a root certificate that it's fixed, always the same, and they have an intermediate certificate that can change, but that's fine because you have the root anyway. And then after the intermediate that signs directly over the the payload of the of the signing key um on the enclave that was generated for this app, >> but also what I mean is that on Android, if you root your device, you're not going to be able to use private face match because u yeah, we can't check the guarantee of the device. >> Yeah. And that's why I think it'd be nice to have designs in the future that allow trusted computing and non-rusted computing to kind of coexist and then you just don't get the the guarantees. I suppose you'd still need to have like a

like rooting I guess kind of overrides it at the top level, but if you can almost have like Yeah, it's I guess there there's more involved than that. Um but then on the OS level the the way that the um the actual like uh the guarantees uh are enforced or provided is you have the kernel um communicating with the secure enclave. And so an app has a it's like a bundle ID that's signed by the publisher of the app which is us. Um as a publisher we're signed by Apple. This is like when you get the Apple account.

Um, and so you have this this yeah this chain of trust where on an Apple iPhone you can only run code that's that's signed by a publisher. And so when uh our bundle ID which identifies our app will request its own key pair initially the the kernel will forward the message through to the secure enclave and then it will generate the key pair but only for that app only for that bundle ID. So only our app can use this key pair going forward to sign over data. And so with that guarantee you can you can essentially sign arbitrary logic now and you have the guarantee uh with app test that it is from our app one of the apps that we published um because of all of these checks that take place. So it's

kind of like a hybrid almost of like a te that you get normally where it's like really like fla with like an Intel TDX and everything is done in that way. But this is also very similar guarantees but on a on a consumer edge device which is exciting because we can now do these really unlock these really cool primitives. Uh for example uh it would be possible to have like um as as a future get passport feature you can have proof of say like location. So maybe you can get the GPS coordinates um in the app and then now you can prove things over the GPS coordinates and you have the app test guarantee. So maybe you don't want to expose the GPS coordinates directly, but you could in a CK circuit you could do a mapping to like a city, right? Or within a radius of a certain

location. So um many cool use cases for that. Uh some other ones might be you could do like proof of IP address. Uh obviously it's not foolproof. you could use a VPN. But the idea is that at some point in time like you had this IP address because you the app was was able to like essentially you know have that um through contacting some external service and then maybe you don't reveal that directly but you have a mapping of IPs to countries um all the other interesting checks that you could apply. Um I mean it's kind of endless now because you could you could even tap into like um fitness data. Uh this would be a whole separate like product or protocol, but you could do like proof of like running miles. I'm pretty sure the

Apple Watch has a TE um in the same way with a like a score enclave. You could use app test for this. Different product, different protocol, but I think you get the idea, right? It's like it's like custom logic running inside an app by publisher and that's where the guarantee uh comes from. >> Yeah. >> Feel like that. [laughter] >> Thanks for coming to our workshop. >> No, let's dive in more. Do you want to show the um diagram more in detail? >> Yeah. So this is specifically for Apple uh where there's just the root certificate and so this is just kind of like an explanation of the Apple root certificate um all like the relationship all the way through

their key pair signs uh with a fixed signature algorithm with which helps us a lot which signs over this intermediate certificate which can change but also is the same algorithm signs a message hash of this uh uh credential certificate which encode which embeds the key pair from the secure enclave. Uh yeah, I mean it's this I think conceptually I've already probably explained it pretty well. >> Um but if you ever wanted to dive into the details more, we'll be publishing like a write up on pretty much what I've I've said already, but then also exactly like precisely how it works, including diagrams um and this kind of thing.

And so essentially in circuit what it all comes down to is verifying a bunch of hash and signature. Uh so if you use the noir CDSA library uh and the new RSA because actually Android they use RSA on Android. So Google is still using RSA for their root certificate but they're going to switch to a CDSA in February. Actually we need to be ready for that Michael. Ah >> but anyway um so that's [laughter] uh it's already it's already there. I just didn't generate the circuit because that's a lot of combinations. So I just need to press a button um and hope my laptop doesn't crash because a lot of no compile at the same time is very challenging for my computer. Um but yeah

so essentially um this like depends on on iOS for example the circuit is like still in the subgroup to the two to the 20 and on the on Android most of them are two to the 21st because there's a lot of signature and hash the most expensive part is really the hash actually uh like the SH 56 sh4 and I didn't see any SH 12 512 yet um so should be well hello sir And um one other cool feature that you get on iPhone which isn't on Android is you get this um they call it true depth.

And so you have this like infrared sensor that creates um like a dot projection they call it. So it's basically a bunch of infrared dots projected. So every time you unlock your phone uh on iPhone it's it's basically like shining these dots on your face and the distortion of the dots allows them to create a 3D model of your face which is really cool. So it's literally like you can generate a 3D model. It's a kind of a form of LAR. Um uh so then when you enroll with Face ID on Apple, they are creating a 3D model of your face from these distortions of this dot projection. Um and that allows us by tapping into that it allows us to have depth detection and and you can't bypass it by for example just putting a your

passport up to the phone during the like the face match uh part of the flow. So that's not on Android though, at least not on all devices. And so to defeat to to counteract that, protect against it, we have this livveness check where you have to like look to the left, look up, look right, then we do pose estimation to figure out um the direction the person's looking that they are actually looking in those directions. Uh and in a future version, we can do really interesting stuff. Um again to get maximum support because otherwise you're only going to be, you know, relying on iPhone. you want to also have uh Android supported, we'll have um things like for example, we haven't tested this yet, but we think

it's a really cool concept where you can have colors flash on the screen um and then we can have a look like kind of like zooming in to the person's eye and they the reflection of the color in the person's eye would have to match the the the colors that were like that were flashed like not too rapidly for in case like epileptics are trying to get through [laughter] the flow. You don't want to induce any seizures. Um, so maybe we should do some user testing to make sure that it's safe. [laughter] >> Um, and then the colors um would be in a different order for example. So that would be the challenge and that all of this is to say that with generative AI getting better and better. The whole point is to defeat defeat that and the ability to like generate you know a

generative AI video using like groore like VO for example Gemini um because you couldn't do it in real time I guess yet as it gets more sophisticated and more sophisticated it's going to be um more and more challenging but I guess the more friction you add at this layer makes it more difficult for scammers and um and they exist they're out there so we um we know we stumbled across recently um there was a screenshot someone sent through one of the groups of KYC scammer doing selling like Zika passport and un tech uh essentially like yeah like verification services. This is before face match a private face match.

So I think now they're going to have a more difficult time but they had a spreadsheet of like all the services and all the customers and like the like the URL that you'd be scanning to perform the um the rest of the verification. And so for so initially for Aztec for the sequences we had this sele resistance where you'd have to essentially prove that you're um you know holding a valid passport the only thing that there was being requested was this unique identifier. No other information was revealed but that unique identifier is always going to be the same per passport or national ID.

And so then that means that you don't have all these people like you know airdrop farming um sequences running all these different nodes because it's basically one person one sequencer or I guess more accurately one one passport one sequencer. So you know most people have one maybe you have two less likely you have three even less likely you have four. So it's it's you know and the government starts kind of raising their eyebrow like why do you have 10 passports? Um that's a way to have strong sele resistance still. Um and so yeah, so what they would do is they would offer these services and they would be having like a box full of I guess stolen passports, you know, like at their place uh and they get you to

like send the QR code to them, for example, and they would continue the flow on their end to to to complete the registration. Um but now with private face match, obviously you have to be the person on the document. So, um, yeah, unless they're like able to like like a shape shifter, um, they could morph into the different passport faces, uh, that's going to be challenging for them. We actually been meaning to do like, um, like undercover investigative journalism. We want to reach out to them and be like, "Yeah, yeah, please. I want to I want a KYC, how much?" And um, and just get an idea of like how much it costs and if they can still do it so we can um, you know, provide measures to counteract it. Uh, so yeah, that's next on the on the on the cards.

>> Yeah, >> my feeling is that they're not going to be able to overcome this anytime soon. Um, I know that I couldn't overcome it on iPhone. They're definitely not. There'd have to be like a zero day that would be able to achieve this to to actually like properly jailbreak it without detection. On Android, I have a feeling I probably could if I spent enough time on it. Um, but it would be very difficult. >> Yeah, that depends on the device. Some device has stronger guarant than others. >> So, take graphine OS for example. Um that's a different like parallel version we'll be adding very uh I would say not very soon but it's on the road map. We want to add support for this. Um and that would be a different route certificate I imagine.

>> I think I think it's still I don't know if it's still Google's but uh they definitely don't use play integrity. >> Okay. Yeah. And so then the idea here is that we would be able like the the the service provider that's plugging in here. They can specify what they're consider what they consider considering um acceptable. So they could choose only iPhone, only Android. Um I guess be more granular depending on the requirements of their service. Yeah. >> Yeah. >> Um and also I can give a demo. I'll give like a screen share if I can get it working of how I'll do an actual face match now. We'll do like a live one. So just give me a sec.

>> But feel free to keep talking if you want to. [laughter] >> I'm just like uh trying not to do myself too. Wait, >> I'll have to get to the right flow at the right time. >> You just have to hide your details. >> Sorry, >> the mic is working. Okay, perfect. >> Oh, yeah. There's a good time for questions, I guess. >> Questions. Uh, two questions. Two short questions. First question. >> Sorry, you only get one question. Perfect. >> Oh, question. Okay. >> You have two passports. Then you can get another one. [laughter] >> Then then the most important question. Uh, >> uh, can you maybe open or more uh request? Can you please open on the iOS app store the app more globally? So I have a Android Austria set it's it's it's working. I can download the app but

on iOS it's it don't let me download the app and it's just a one setting. So just you know one flag for the European >> not sure I actually understand what >> yeah on on iOS I can't download uh your app because my uh region is set to Austria >> on iOS for iPhone. >> Yes. And it doesn't allow me to download the app. >> Oh we haven't added it but Austria. >> It should be for the entire world. >> Should be like globally. >> Yeah. But you can test it.

>> I believe you. [laughter] You don't know that >> and then it just works. I'm like, bro, what the hell? >> But on Android, I can do, but not on iOS. Yeah. Yeah. >> Chat with us after the workshop. We'd be happy to have a look. We can fix that for sure. >> Apple just block country without telling us. >> That's weird. Yeah. Yeah. That shouldn't be the case. Thanks for >> But can I can I ask a technical question? >> Of course. on the uh so do you guys uh using actually the the actual signature of the passport and like actually a challenge to challenge the private key and to get a response.

>> Yeah. So so yes, great question. Um the feature you're referring to is called active authentication. >> Active authentication which is difficult. >> Uh it's it's just less supported. It's not diff we could definitely do it. It's just not on as many passports. >> Um but we could add this as a feature for sure. Uh yeah, that would be an optional feature cuz no passport uses it. Uh for example, my passport doesn't have it. Uh >> okay. >> So it it would be very nice because you could derive like some kind of secret from that. >> Yes. >> But then then the issue is like if not, you know, if only 20% of passports support this? Um what are you going to do? Require that as a as a service?

You're going to or probably not, right? And then if you're not requiring it, [snorts] then why even like bother using it? Because you want to let everyone in even if they don't have active authentication. Um, so you don't really if you're still be allowing people to get in without it, you're lowering it to the lowest security assumption, which is called passive authentication, which is essentially what I described before. It's the the root certificate check all the way from the the chain of trust, which happened as well for active, but with the difference is active authentication that it performs passive authentication first, which is the check that it's a a valid valid signed passport um from an issuing government.

But there's also a signature over this key pair. And then there's like a smart card on the um the passport itself with its own key pair. It can then sign things. >> So [snorts] now you can have the passport sign like arbitrary payloads or a challenge request um to add this extra feature. But again, not not all passports support this. >> Okay. Okay. >> Yeah, that's the challenge. >> Okay. Thank you. >> No worries. Thanks. My question would be what kind of model are you using to do the face match and can I run any arbitrary code inside the securing cliff? Maybe you mentioned it and I missed it.

>> I didn't get catch your first question. >> Like what kind of AI model are you using to face match? >> What kind of model? >> Yeah. How are you like face matching? >> Yeah, we're using um it's called um Arcface Resnet 100. It's a 150 megabyte model. uh a 512 dimension um vector emerges. And so that what that means is um the model itself um when it's doing inference against like the matrix which is the model um it's there are 512 different dimensions that it's checking against. I don't know them all but it could be for example the ratio between like the eyes to the nose to the mouth.

um I don't know the like the distance between the eyebrows and the eyes, but there's like 512 of these and combined these all make up um the the vector embedding which is the face print that uniquely identifies. So it's like a fingerprint, but it's a face print. >> Um and that >> what's the name? I want to like say >> the model it's called Arkface A R C >> F A C E >> Arface Resnet R E S N T. Yeah, but we actually did the implementation in rust and so the rust communicates with the swift and the cotlin to actually generate there's a there's yeah there's a bunch of uh different languages in our stack. So there's rust, typescript, c++ uh there is noir and um yeah just a bit

of python but that's just for scripts. [laughter] Yeah, it's a bit messy, but like you can that's a nice thing. You can run like some uh native code like Rust to C++ and just link it uh to whatever native language of the OS, Swift or Cotlin and then you have the React Native upper layer in TypeScript. >> Yeah. Uh >> so my second question was like can you run any code inside the secure enclave? So the secure enclave is not an open-ended like TE like you would have with [clears throat] Intel TDX. So it Apple or like the manufacturer of Android devices won't let you run arbitrary code. You can use that for cryptographic operations. So you can

generate key pair you can do signatures and all kind of stuff cryptographic um operations but you can't run arbitrary code. But the actual secular enclave allows you to get since you sign a bunch of stuff and you can attest as Michael mentioned to the key pair and that's from the genuine device. Uh but this the app itself will be guaranteed by that way that it hasn't been tampered with. So the code logic is kind of like protected in a sense even though it's not running in the T itself in that way like the secure enclave.

Then won't it be like manipulated by the client if it's not running inside the TE? >> No. The point is that because it's attested like the device is trustworthy and Apple will test attest to it. You know that that code cannot be tempered with otherwise if that was the case if that was a device that's not trustworthy and the code could be tempered with Apple would not attest to it. Similarly on Android you have like the same mechanism but the security assumptions may not be the same as on iOS. It's not your classic TE like an Intel TDX. >> It's um it's that's why I call it like a mobile TE.

>> Yeah. >> Or called Apertess as the service. >> Can you do this inside the browser? >> No. >> No, I don't think >> why not. >> Well, there'd be a lot of outrage against that, I guess, because it's um because in order to get um this working, you'd need to have like a trusted computing setup where from the bootloader it onwards, you need to have the OS keep control. So, you can't really root it. And so then for desktop computers it would I don't think it's ideal um because you want to be able to route your like your desktop computer. For for mobile devices this is ideal um because it's not like a general purpose computer. But yeah there's a trade-off here when it comes to like being able to own your own hardware and run your own your own code.

>> Mhm. >> Uh without the restrictions of like Apple. >> So in a sense like we're trusting Apple that it would only allow >> a trusted code to run Apple or one of the parties of the trust. Yeah, you're correctly responding when they say that this is actually a genuine TE. >> Yeah, I think it's going to become more and more common that within the ZK world, we're going to rely on like TE's at the edge like the edge devices. So essentially you want to have some information that's been attested that you can use for proving like it's to test it by some kind of signature like any kind of sensors like it could be a

camera it could be like a uh a thermometer like any kind of like input from the outside world that you want to test and use as a base to generate a proof like keep to keep private. So you need some kind of way to testing to it and like T's are pretty good for that. on on the other hand using a T on a server to have like privacy is not the same thing and probably not the best thing. Yeah, I think a lot of protocols are using this as like a cheat code like a shortcut um in place of actual engineering effort to properly create like cryptographic guarantees in a protocol and that should never happen at the protocol level only like as a layer above um as maybe on the application layer and in this example you know if you want to add private face match um

because you're never going to get the the complete guarantee that you get from ZK cryptography. Yeah. My next question would have been what if we run it inside the ZKVM the face match? >> Uh ZKVM or ZKML >> ZK VM because >> well ZK VMs are not quite ready for mobile. There are some that starting to work on mobile. Running an entire model on a ZK VM nowadays is probably not doable. But the the anyway the um the thing here is that in any system where you attest to something like cryptographic system the system as a whole is as strong as your weakest link here and the weakest link is the camera the device itself you need to trust the camera fit feed. So you're already

lowering your assumption to the device. So you may as well use that as the base. >> Yeah. So let's say that you ran ZKML um on this to get like a like a cryptographic guarantee that whatever image you're running it over the face print was derived exactly according to the the logic of the circuit um which is very expensive but let's assume that you could do that do it on mobile. There's like almost there's no point because the camera feed itself the input you could just fake the camera feed input right it's literally like a a sensor that's just taking in a camera feed of a person. So because you're already lowering it to that, you may as well just do the whole thing um from passport photo onwards in um with abot test as the guarantee. You've already got the hardware level like um that you're

running, you know, that you're not feeding in fake data uh as the as the lowest assumption. >> Thank you very much. >> Yeah, thank you. And so I can give a demo now of how private face match um looks when using it. >> Oh, sorry mate. [laughter] Go ahead. question please. >> I mean, you can probably fine. >> Can I give a demo and then we can continue cuz I'm really excited to show you guys. >> Just give a demo like we >> and then we'll jump straight to you guys. Um and also like you can try this yourself right now. This is live in production. Uh we haven't like publicly announced and and launched private face match yet. So um normally it would be at demo.zip passport. But to be able to just see it unlocked, you just have to

put question mark FM equals 2 at the end. Yeah, we just need to add you just need to add this to the website. >> We're going to be making [laughter] it like Yeah, it's a little secret cheat code. It's like a Yeah, please get a picture if you like the URL. Um and then yeah, so this is like choose your own adventure. You can try all these different types of proofs out for yourself. Um so in this one we're going to go private face match. And so this is me. Yeah, this is like a live demo. [snorts] >> Um so hit I'll select that. It'll provide a QR code.

>> Are you going to dox yourself? Yeah, I want to dock myself. The camera's down. [laughter] >> Is that a camera? >> All right. [laughter] So, just Oh, yeah. >> Oh, thank god. I enabled the feature that blurs my [laughter] 50/50. So, okay, cool. Do a QR code scan. So, everybody in the room. Very nice. Very nice. And then, all right, so we'll scan the QR code. Um this is a secure uh end to end encrypted websocket bridge by the way. Um and then it there's like a call back that gets sent to the the website saying yep user scanned it. So you can react to that and like show something else. Um this is asking whatever the

current credentials are uh being requested here. So this could be anything. In this case it's just private face match. So I'll go uh yep next to my face. And this is the liveness check I was talking about. Um this is a this is really fast now. Now, it never used to be this fast, but thanks to a lot of the work from Nile over in the corner there, um, optimize the out of it, it's really good now. It's almost like too fast. People get confused, but um, that's it. And so now this is generating a zero knowledge proof locally, verifying um, all of the signatures from Apple's root certificate all the way through to the intermediate to the um, this credential certificate which includes the public key pair. then the keep pair itself signing over the face match data um and then that gets once that's done that gets sent back to

the the service um as as a compressed proof that can then either be um verified on the back end so you don't need to do blockchain it could just be like a backend service for some kind of zika KYC stuff um for example with dev connect um you didn't need a blockchain for this so it just went to a backend service where it used the unique identifier to allocate uh a a ticket discount code, right? And then if you try to like register again, well, it it'll just give you the same ticket code because obviously um you're trying to like double dip with the discount codes.

Um and there you go. And that's like private face match um end to end how it works. So yeah, thanks. >> Oh, OKAY. [applause] >> Uh yeah, you had a question, please. >> Yeah, let's take some questions. >> Yes. Uh so I was curious um what's the minimum hardware requirement uh on a phone in order to like generate that proof? >> These proofs are pretty beefy. So like 32 GB at least on a No, I'm kidding. This actually used to be requirement, but there's been so much optimization thanks to like Zach and and others at Aztec um who by the way created. This is all all of our circuits are written in W. The back end prove is uh Baratenberg.

Um we managed to get it down to like 1 GBTE which is like truly impressive and this is only recently by the way like the last couple months. Um I guess they were like deferring a lot of the optimizations until the very end. Um and so now even like yeah lowerend Androids are able to do this these uh this proving. Um and our actual design we can talk about the design of the circuits. We have these four different uh subcircuits to generate these four different subroofs. Um the first three deal with the um signature verification from the root certificate to the intermediate and then over the passport data. Um and so we can actually c those first three. Um and then the last one is the disclose proof. Um and that's basically like pivoting off the first

three um to be able to like do the private face match one or do like a disclosure of like which country you're from or which set of countries you're part of. Um and this design also means that given the like there's a myriad of of different combinations of different signature algorithms from all the passports, we have like 600 circuits in total. Um and so you can kind of route through the combinations to create these you know a unique combination of the the sub subproofs [snorts] um specific to your passport. Um so actually as soon as you start scanning as soon as you scan your passport in these first three subproofs start proving in the background um because nothing else changes except the very disclosure part at the end. Um, so then when you go to actually prove it, it's very quick. Uh, only takes like a few seconds. And then

finally, these four subroofs are recursed into a final outer proof. We call it like a compressed proof. And that's what's then able to go onchain. Um, and is like EVM compatible. Um, and for that one cuz that's quite beefy. Uh, that used to be we have a cloud prover as a fallback. Um but because these these four sub uh four sub proofs are already you know sealed away um we can use the the cloud prover to recurse it if needed and so we try to do it locally first but if I think the requirements are like what so yeah the output proof so as we call it the compress proof uh the subcircuit size is uh goes from two to the 22nd to two to the 28th so this is beyond what mobile can do so this is

done on the cloud prover so you re recurs those z proof that already zka and hid all the inputs. So the final one is just like adding some glaze onto it so you can have a single proof that you can verify on chain because if you were to verify all those four sub proofs on chain that would be a bit expensive in ter of gas. Um yeah it's a nice background to show as we talk through the rest of it. >> Yeah that's the website. Um did you have a question mate? >> Yeah. Uh how is the like accuracy of the model that you use compared to like this runs locally, right?

>> This is what Yeah. All locally. >> How is that like how does that compare in terms of like accuracy and false false positives and stuff like that with like the most advanced ones that probably run in like a machine that is like way more powerful and they probably like take more variables or or not at all maybe. >> Yeah. No, good question. So there's actually a mobile version of um Arcface ResNet um which is also a lot smaller. It's like 10 megabytes something. But um we decided to use the non-mo version um which is 117 megabytes cuz it's way more accurate. Um and it it does it works on mobile as well. So we figured we just go with that. Uh and a lot of people like most people so far have been able to get through. Although I have spoken to to one person that said that they couldn't

get through because they have like a weird face or something. I don't know. Um I don't think that's >> [laughter] >> I know what the the face looked pretty normal. It looked fine, I think. So maybe yeah, maybe like um you know there's a there's we're going to explore like speak with her a bit more and understand exactly what the issue is. The threshold is like I believe it's like 7. Um this is called the cosign similarity. Um so maybe we have to lower that. Uh or actually maybe we can what we can do is have the service uh specify what their minimum threshold is for example, right? have a default that's same but then also allow you guys to like plug in whatever you want.

>> Well, that could be the difference between the regular and the strict mode. >> Right. So, we have Yeah, we have two variants right now. We have like a regular mode or a strict mode. And then so the regular mode doesn't do a livveness check. Um the strict mode does um because if I just put my passport photo in front of it, it would would work because it doesn't know that it's not a photo. No. >> In fact, we probably just get rid of that cuz >> No, no. So like the idea is to as we uh start to strengthen the model you want to have like different grade of it cuz like one may be like take more time but we have >> you saw how quick that was right it's much faster now.

>> Yeah. Yeah I know I know but you you may want to have some more checks later on but you can delegate the rest. >> So so what was strict what was [laughter] later >> it's a devolving protocol and if you have any opinions please let us know we'll be make it part of the protocol too. [laughter] >> Yeah. Hey, >> uh are you planning to separate the sort of because now you do of course the the passport verification checks and all the certificates and you have really the atestation level on the device. Are you planning to separate that at the station level on the device maybe as a separate library because I can see people would be very interested in leveraging the work you've been doing.

>> Yeah. on because you mentioned like uh fitness devices all of these kind of things which might not immediately be useful for Zay passport itself but might be useful for other applications. >> So that is something that we do plan um once we get more time because we've pretty uh swamped right now. Um so the mobile app is um with the code source is for now still closed but we do plan to pull out a big chunk of the modules like those and make it open source so you can actually use it. So you would have like this uh upper station service. You already have the part of the circuits public so you can like kind of guess but you still need the implementation mobile to figure out that that works. Uh it's not too difficult but that that should

make it easier if we export it as a module you can use. Uh, I think there's other feature we could probably export as module as well. >> Yeah, there's some that make sense for Z passport. I think we'll add those. Um, like the GPS coordinate one. Um, again, there's ways to obviously always port this like you could get um a GPS spoofer and just have it next to your phone and now you can be anywhere in the world. Um, and but it's but it's increasing the barrier so that um it's more like yeah, it's this is all like layers on top of the base cryptography that's insured. Um and some other things could be yeah I mean maybe like a device ID that could be scoped by a service. So so then the the thinking here was that with the KYC scammers um they probably use the same

phone for the like 10,000 like you know whatever 100 passports that they scan in. This would be like a way to have a device ID and you get privacy through scoping it to a service again. So a service can only see the device ID relative to their service. um if it's like the same device ID for these hundreds of passports um maybe it's a scammer and then you you have to use AB test on that to make sure that it actually is a device ID um yeah or maybe like you know a encrypted uh like IME number but but whatever we do it would have to always preserve the privacy of the user that's like the core value here of what we're building um and also give also have consent informed consent when you have the the pop-up right they need to know what's happening what's being

shared and they can either agree or they can disagree, >> reject it. Yeah. >> Yeah. Thank you. >> This is maybe more of like a product question than a technical one. One of the kind of like criticisms I saw during the token sale for ZK Passport is like >> the tokens out. What's what I'm first hearing [laughter] this >> if you haven't heard. No. Um I'm not shilling here. No. So the you know there's I would say folks that are just like well how do we know that only the ZK proof is being communicated in these over the wire here like are is there space for doing this in like an

airgapped scenario where there's verifiability of like what data is being transferred >> um I mean you could do this as a technical person have you heard of something called wireshark >> so if you run wireshark you will This would be Yeah. >> Yeah. So, if you run wireshack on the network, you will see all the data that's transmitted on the network. So, you'll be able to see what's actually shared by the app or whatever is on your network. >> So, you'll able to see if you have any passport data that's sent randomly, which this is not, but if you want to see it just for yourself, you can use that.

>> Yeah. Uh just >> Yeah. Wireshark is blocked by some network, so do it at home. Um but yeah. >> Yeah. And then you could basically install a custom root certificate on your iPhone. Um and then have that also in Wireshark. Um and then you'll be able to eaves drop eaves drop on all the traffic going through um your iPhone out of your iPhone um including into passport >> which so you can see it for yourself. >> Yeah. >> Cool. Any other questions? Yeah.

>> So if if I want to build on top of uh your application for example is like some sort of a exchange like half centralized half uh decentralized and I want to just use your solution. I still have to provide at one point to the authorities this is my list of the customer. Uh so this is like a nogo. So, so I'm just thinking of is there is there like any way to hey like there is still somewhere like a list but it gets only open when this and this is like or just uh expose like just like your last name or just like your first

and last name. Is this like something? >> Absolutely. So yeah, what you could do for this, and this sounds like an offchain use case where you have like a backend server, you just want to essentially like do a private KYC of your customers um that you yeah that you're not putting on chain. You could just request for example the first name and last name um and country. And this would be actually a really great way to have I mean that the fields then could autocomplete. You wouldn't even need to arbitrarily type in your name details because it would come from a government attested identity that attested over what your actual name and you know first name last name is. So then it can can pre-populate it. um you can store it on the back end and this kind of also opens up I think another really exciting area

of um a new kind of paradigm when it comes to like uh KYC that isn't this traditional um you know you send your photograph your passport in already they just look at it and go yep yep looks good and they like reattest to it and then they've got your data is like a verifiable KYC so for auditing um for services you know you have let's say let's say there's nothing's revealed you just have like the sanctions checks and all these other uh checks that take this um you can also have that proof to provide um the verifiability to the auditors that you did these checks at this time against these sanctions list.

Um but for your use case you can just have literally like um you can have the verifiability that that's the customer's actual name which is stronger than even having you know getting a passport and making sure that it matches what they put in and and it's automatic and like it's basically it's free because of what we provide. infrastructures there to tap into this and you're not trusting us, you're trusting the government themselves. Um, and then you're not revealing your whole passport data, just like you said, the first name, last name, country, um, date of birth maybe if that was for your service. Yeah, >> cool. Thank you.

>> Assuming that's the last question. Uh, Theo and Michael would probably stick around for a bit as well, so feel free to catch them. >> Yeah, come with us. We'd love to answer any other questions you've got. >> Yeah. Thank you, Michael. Thank you [applause] to the mele.