There is such a thing as a free lunch - the deep fake refund economy

Transcript

Thanks. Thanks again for for choosing to attend my talk and the one thing that I'm going to speak about today is essentially this notion of a new kind of fraud that is emergent. And when we generally think about fraud, you think about, you know, like stolen credentials or stolen credit cards and the like, but there's something that's different that's emerging in the food industry and also in industries that are operating at the intersection of physical and digital spaces. And that is essentially um manipulating the the manipulating images which which is essentially used for for refunds. So that's that's what I essentially mean by synthetic. So I'll I'll give you an example of this. I was actually pretty

surprised. So I was back home in India and you know, my mom was ordering some some eggs on like a food delivery platform, a pretty popular one, and the eggs were broken. And so she said she wanted to initiate a refund. Um [snorts] you know, to to the credit of the platform, they they actually wanted multiple images. But my mom had just just taken one, yeah. And so she asked me to kind of help her with this. So what what I did was I just used ChatGPT. I gave it the brand of the the the eggs and yeah, asked it to generate an image and that was that. So it put that it put that together. I I added those images and voila, everything just worked. The refund just just came in very very happily. So I mean, the the the key point here is um

the thing that people relied on is is perception and thinking about images as being anchored to to reality, but that is no longer the case. And so I I I always generally like to think about you know, like how you classify things in in general. And one one one one way that I thought was helpful to understand is is kind of thinking about this in in two axes, right? So who is submitting this this image? And the second axis is what is this claim that the image is supposed to support? Yeah.

And and so you can think about it from two different angles. So there's the customer and for the customer it's pretty straightforward. Like like me, you know, you can send put up an image and then you get a refund and that's that's great, right? But there's this also another perspective that that's popping up in in food delivery companies where the couriers themselves are kind of tweaking around and playing around with with the orders in in some cases. And what what they've and there have actually been instances where couriers have generated synthetic evidence and used that to essentially say they've they've actually delivered stuff in in the right manner, whereas it wasn't really the case, right? And so that's that's the the same point that I was looking to make. So so

you you now have essentially a new form of fraud where the the proof itself is is synthetic. Yeah. And so Um this this is really funny. So like the more I was diving into it, the more a lot of this was actually coming up like early 2024 and in later 2024 there was a bunch of people essentially on on TikTok and in subreddits who were basically showing how how you can actually use AI tools to to kind of you know, like beat the system, screw screw DoorDash and and all of that, yeah. And this is this is not just happening across like Western markets.

This is also happening in in India in in some in Southeast Asia. And yeah, it it seems to be working. And so the question is who is eating this loss, right? So it's not the users, obviously. Users are super happy about about all of this. It's um Um and yeah, I mean, there's this again studies about about how um Um I mean, studies are conducted across all all forms of online commerce and the the the striking point to understand is that food delivery has been the the the vertical that has reported the highest post-transaction fraud rates in in the industry as a whole. Yeah. And what ends up happening on average is you have

restaurants that report about 2 to 3% of their top line that is essentially tied up in in disputes. And it's it's pretty straightforward. So if you if you imagine even a market like Lyon in France or Utrecht, like you know, like small dense cities that that have a high throughput of food delivery, that that have high digital adoption, it's it's pretty you can easily get to like 10 million gross monthly volume and using, you know, like the 2% number here and assuming again like the 25% fraud rate which which might I mean, I just just pulled that out of a hat, but it it still is pretty massive in in terms of like the the amount of loss that is actually being generated in the system. And and yeah, this this is something that you need to realize is a massive industry, yeah. So

DoorDash itself has reported about a 50 billion odd GMV annually in in 20 2024. And the overall market is just just not showing any signs of stopping and there's more and more restaurants that are essentially using a lot of um capturing a lot of their their revenue through through using, you know, like food delivery platforms as such. And as I as I mentioned, yeah, like it's it's users are not paying for this. And when when you kind of think about food delivery platforms, they they essentially control like the identity layer, the payments layer and ultimately the dispute layer as such. And so what they they in the true they they are essentially the true definition of a platform, yeah. And so what what I'm is food delivery platforms are happy, I guess, in some cases to

issue refunds to users, but it ultimately basically means they charge the restaurants unilaterally. And it's the restaurants and maybe I guess the small business owners who end up eating all all these losses, yeah. Um And what's what's actually very interesting is also like this is the this is the last point that I had, which is I was speaking with a couple of food delivery companies who who basically said um it's it's it's when when they're entering a new market, they actually want to have a higher rate of refunds that they issue to users because refunds are the things that fundamentally drive user engagement way more than any other product feature in in the the food delivery app. Yeah. And so what what ends up happening is when when when when

they enter end up entering new markets, um they're extremely happy to just just keep issuing refunds. And what what what ends up happening is it's it's basically the restaurants who end up getting compressed, yeah. When when they actually start getting dominant in in in specific markets, that's that's when they maybe slowly start clamping down. But but you know, I mean, it's it's a very dynamic industry in any case, yeah. And so Um there's there's controls that that people have been putting in place, yeah. So you have like limits on refunds. There's there's people people try to do reputation for for users. There's manual review processes. Like but but again, these these don't work at scale, yeah.

These these are things that were um controls that were made for another era where generating, you know, like images was was not was was not easy, yeah. It was it was essentially a bottleneck. But but now it's it's it's it's no longer the case. You no longer have um the ability to um to to scale up processes like this, yeah. And and what's what's happening is um companies like DoorDash have actually started tightening the screws on on on the the couriers, yeah. So so they they So there's there's actually been there's actually like a couple of EP reports that came out on DoorDash which um on on, you know, like the the kinds of opaque and overreaching identity like

real-time identity controls that they were imposing on couriers and and the like where they were asking them to do like selfie checks, like include images of of their IDs or something along those lines. Um but DoorDash was basically saying that this this was all in response to an increase in the number of accounts that that were being created and shared and credentials that were being shared as well as, you know, like a lot of fraud that was happening on the platform. So there's there's a lot of admittedly, I mean, there's a lot of ways that food delivery platforms do have leakage, but this this is something this vector is something that is consistently increasing. And this is something that has been the case across like multiple companies that I spoke with, yeah. And so what what what actually needs to happen?

So um I mean, people have generally been asking, you know, like does this image look real, yeah? And that's that's something that's related to perception. But now I think and when when you even think about industries like journalism, for instance, where integrity of of media is is very important, what they try to ask is where did this actually come from, yeah. So there's there's a lot of organizations that are working on open standards like C2PA, for instance, who basically want to get people to answer the question where where did this specific piece of artifact come from? Did it come from a specific device or an emulator? Did this device have specific security

properties? Were there were there any additional kinds of metadata associated with um this is media that that that you can essentially use as a basis for for provenance. And so the the key idea again is images are no longer proofs, but they should be considered inputs. And you should stop thinking not about pixels, but essentially trying to think about creating a provenance chain and figuring out whether this is um increasing your confidence that this is in fact real or not. And so there's there's a couple of ideas that I kind of have in this product direction.

Um so there's there's some some work that that people were doing on what what are called attested sensors. The the idea again is is you could have like specific cameras even in high-end phones, what's what's happening is you have the ability to to kind of use a secure element that is on your phone and this is essentially something that that contains um it's it's hardware that's um generally not hard to tamper and it kind of contains like very specific capabilities that that enable you to kind of sign off on on specific things that that happen within the I guess the environment of your phone. And so the idea now is is to kind of move this this trust upstream like I was kind of

mentioning here. So what what I what ends up happening now is you could essentially in theory use some of these capabilities to actually create a so-called evidence packet. So you you can basically use some some of these these modules and and these elements on your phone to to kind of capture timestamps. You can try and capture, you know, there's this you can try and capture essentially things like um sessions that that you can set up with with the the platforms in an ephemeral manner and make make it interactive.

There's also other kinds of sensors on phones that are available. You have lidar sensors which which you can use for for depth sensing. So you can you can try and include all of these and and sign this together in in the form of an evidence packet and that that that's something that that you can send up on chain. There's There's again additional additional benchmarking methods that people have been working on which which basically say, you know, I believe you you know or maybe you're familiar with the concept of hashing, right? So it's it's something where you kind of take piece of anything really and kind of reduce that to a random number or a random string and there's these these so-called hash functions. So it's it's easy to kind of use hash functions to transform

inputs to outputs but not the other way around. But there's newer forms of hash functions which enabling people to also you know, detect changes in or essentially allow for specific changes to images that that you do. So for instance, you may want to basically take an image on your phone but but then maybe decide to crop things out. Um and so what what ends up happening is you you can use some of these these techniques to to essentially say like the content that you kind of captured is something that is authentic and the process you use to manipulate this in in terms of say cropping things out that are maybe sensitive is also an allowed operation. Yeah, and so so this is all something that fundamentally again ties

into the fact that you you you actually need to start thinking about the events around the capture of this this media and and then use it to start thinking about setting up provenance. Um there's actually a a pretty very straightforward thing that people have not very much been using. It's it's app attestation. So you have the ability for >> [snorts] >> for an app to to basically give an output that it is actually running on a real device and that it's not running on an emulator or or or rooted devices such. And this is something that's available on on Apple App Stores or Google Play Stores as such. And so this is again something that that you can essentially use to basically say hey, this this was actually captured. So you

you can basically think about the scenario where you can actually capture images inside an app and then use some form of app attestation methodology and you know, get get get this to also be part of the the provenance chain you create. There's there's some there's some issues, you know, like in in terms of this this flow because imagine like you you you order food, you're hungry, you you get some food that's that's spilled and so you you basically want to eat first. What you end up doing is you just take a snap and then you know, you want to kind of initiate the refund flow later. So this this doesn't kind of work with with with that flow in in mind and that is something that food delivery companies are happy to accommodate. But again, there's capabilities in in phones that that people should should be leveraging as

such. The third and the more obvious thing is again not not relying on things that are static but but essentially introducing some form of interactivity. And this is pretty familiar to people maybe who've been using banks or custodians or things like that, right? So when when you do like high when when you kind of do something that is important so people basically say like you know, like take a video, you know, like scan the food label, maybe show a clock in your room, show your face, move it around. And so the idea again is is there's this more entropy in in actually creating something that is more interactive as opposed to something that is static.

Again, these these like video generation is is also manipulatable but it's not so easy kind of increases the cost for for users who want to kind of use this on on that basis. So it's it's a it's a workaround in in some sense, yeah. But the the very interesting point about blockchains as such is they once once you kind of have blockchains as as a trust anchor. So there's these very many standards that I kind of was alluding to like C2PA, there's there's a couple of other image media provenance standards and they they work in in different contexts. They're maybe a little heavy to use but the idea is once once you actually start anchoring all of these as as provenance points and use a public blockchain to kind of create this, that's when you can

actually start thinking about looking at this from the perspective of insurance. So blockchains are great at at financializing things and and so there's there's again a lot of things that are generally available on chain like like insurance and re-staking that that that may enable you to actually price a lot of this fraud that is paid for implicitly, right? Right? And so what what am I talking about? I'm talking about essentially restaurants forming collectives. And so the idea here is you could basically have restaurants pool resources together, create essentially what what might be a shared pool like like a collective of sorts and then basically say you know, like as as any refund claims that that keep coming in a

given food delivery app, we we we try and see if there's like some form of provenance associated with with all of this, yeah? And if there is, then okay, that's that's fine. Then we will we will not pay the restaurants back. But but but then if there's something that is outside the accepted boundaries of provenance that that we associate with with images, then you know, we we can use that as as a quick means to to auto pay people back really. And and what what this actually is good for is the fact that you you first start pricing fraud explicitly and you you actually get a chance to make restaurants whole again. This also gives secondly restaurants more bargaining power as as

a collective to to work with food delivery platforms in in in one way, shape or sense. Um and I think the the the idea is as as I said, this this is actually a very nice approach cuz it it allows for pluralism in in a lot of standards. So there's there's very many ways in which you can start creating provenance. You can use CK proofs, you can use C2PA attestations, yeah? Yeah, and and so yeah, so I mean this this is again one one one specific way to to kind of get get about doing this. And so I mean yeah, this this is again the the the idea behind all of this is you you you stop you stop thinking about um you stop thinking about evidence as as being static in in a fixed point in space. You you actually start thinking

about introducing trust as a stack with with not just technical elements but but also cryptographic and hopefully economic elements that that you can then use to to start essentially thinking about this in a more holistic manner. Um and so yeah, I mean this is something that that we already know but but the idea again is there's there's faced ways in which food delivery platforms can actually do this. There's this ways that you can start introducing things like in-app attestation. You then move on to introducing I don't know, like some form of zero knowledge proof at the point of capture for for for creating um um for for creating this this evidence packet of sorts and then move on to actually insurance type model for for for merchants. Um and and so I mean yeah, this is something that

we were generally very interested in. So if if if as a company you are trying to create the rails for um for for for setting up a provenance chain, this is something that will hopefully enable you to to start thinking about redefining what trust means in an AI first era. So um there's there's some some some some some fun companies doing things like Rebellion. So you can check that out. But if you're kind of working in the space, would would love to kind of chat chat about this. So thank you for this.