Building ethical blockchain technologies - Doing good, while avoiding harm. - Dr. Maurice Chiodo

Transcript

[Music] Maybe I'll wait one more minute. Uh oh, there my slides. That's good. So, just as a lead in, uh I'm a real mathematician, which is why my slides are so lackluster and boring. This is a sign of the discipline I come from. I used at least two colors, purple and black. But all right, let's let's go for a start.

So my name is Maurice Koda. I'm a research mathematician. I'm trained in mathematics undergraduate PhD another PhD. And I've been a research mathematician from 2012 to 2023. That was the time I was writing my sort of research papers in maths. In 2016, I started the ethics in mathematics project uh ethics inmatics.com and looking at the ethical issues arise in mathematical work. I'm now a research associate at the center for the study of existential risk in Cambridge and I have industry experience as an ethics consultant uh for AI and blockchain startups which is why I'm here today and why I think I have some relevant stuff I might be able to to add. So just firstly what is ethics and

mathematics? Well the idea is that it looks at the sort of social and ethical issues that pertain to mathematics mathematical work and mathematicians. So basically when there's numbers in the room, what's going on and what might be the problems that come out of that? And this extends to all those who are trained in or working in mathematically powered disciplines and industries, technologies such as those from computer science physics engineering wherever there's numbers in the room. This is this is where the the concern arises. So the idea is that mathematics and mathematicians and and those similar don't live in a vacuum. They do work that has vast impact on society. it follows that well ethical issues will arise. If you're doing something, you might be doing good things, you might be

doing bad things or something in the middle. So let's focus on on sort of the purpose of today's talk, which is blockchain technology. So what does it mean to do good? There are countless ways to use new technology such as blockchain or AI for good. And there's a lot of goodwill and benevolence and optimism here. people I've worked with, I've seen are really trying to do good things with new technology. We can use our mathematical insight to to make some new stuff, to do some cool new things, or really help people, help the world. But another thing I noticed is that trying to do good doesn't always translate to doing good. The intention to help doesn't

always correspond to having actually helped. And I've seen this many times. So let me share some anecdotes of my experiences working with tech startups uh from AI and blockchain no mostly I but a few blockchain ones at the end I'll bring them as well who were definitely trying to do good quite clearly but in the process they kind of missed things here and there. So first anecdote about bias. So working with the team and the team said we're right about bias. I said, "Okay, that's that's a good start.

What are you what are you worried about?" They said, "Well, nothing. We don't know what bias is, but we read in the news that bias is bad." Right? Okay. So, the lesson from this is you might not know enough about your harm. It's not a given that somebody coming from a technical background understands what these concerns are. If you do a mathematics degree, computer science degree, you might come across little if any education on ethics, safety, these sorts of aspects. So number two, data. So I said to the team I was working with, where did your data supply get their data from? Pretty simple question. They said, we don't know.

Okay, well asked them, did you did you ask where they got data from? And they said no. and then sort of paused awkwardly and said, "Oh, do do you think we should have like, yeah, that might have been a good idea." So, lesson here is that you might not know enough about your data, or you might not know enough about asking about your data. That funny little awkward pause where they sort of thought, "Oh, maybe we should have asked where the data came from." And number three, boats. I love this one. team says our AI will control the entire global shipping network in real time.

That's impressive. So asked, do you have any experience in shipping or in you know routing things like this transport? They said no. I said okay well have you ever been on a boat? They said no. Right. So the lesson here is you might not know enough about your application domain. If you're going to create an entire system that red shipping network, that's a lot of boats. Society relies on boats moving around the world for stuff. Just look at what happens when the Sewish Canal blocks up for, you know, 10 days, whatever it was last year. Wre havoc. So if you're going to design a system that

that that sort of fixes or sends out all the boats in the world, you might want to know something about if you don't that's where problems arise. A lack of understanding of the domain you work in number one sorry next. So the team says we're making AI dash cam for car insurance pricing. you know, sort of see how good of a driver you are and then your premium goes up or down or something like this. Okay, nice idea. I said, where will you deploy that? And the team says, well, we'll deploy it globally.

Okay, that's that's that's fair. Uh, but you know, where is your train data coming from? And I said, well, mostly from my mom driving around leads right? So the lesson here is you might not know enough about your market. If you're going to make a system to deploy it on all roads around the world, having a data set just from leads might not cut it. You might need a bit of extra breath there. Yes, there are cars in leads. Yes, there are cars around the rest of the world, but the roads and everything work a bit differently. So antly these are all real. These are these are experiences I had with with tech teams. I wish I could

make this stuff up, but I'm a mathematician. I'm not creative enough to make this up. You have to believe me that this is actually real. Floods. So, AI predicts floods with 96% accuracy is what the team said. Um, and my colleague who was was from law in in the meeting, she was very impressed by this. Oh, wow. Um, me being a mathematician, I was less impressed. And I said, well, how's that 96% calculated? This is how to to determine when where or when there's going to be a flood. And they said, "Well, it's just total correct total correct predictions divided by total predictions." Like, well, that's going to be skewed, isn't it? So, wouldn't the no flood algorithm, the one that just says no flood all the time, input,

whatever, output, no flood, wouldn't that have about 95% accuracy given that most of the time it just doesn't flood, right? Team says, "Ah, well, I guess so." So, the lesson here is you might not know enough about your actual contribution. So it can be tempting to say look I've got this great technical tool and it does this awesome stuff 96%. But if you unpackage the mathematics behind that the 96 is mostly weighed by the fact that most of the time it doesn't flood. So just an octopus sitting there pushing the button saying no flood is almost as accurate in under this sort of measure this metric as your fancy new AI. And that's six privacy. So I asked the team this was

actually quite hilarious. asked the team describe your privacy features to me and the manager in the meeting said all that computation on user images is done on the edge on their device. Oh, great. But then the developer in the meeting said, "No, it isn't. We send images to our central servers for pre-processing." I said, "Well, maybe you two should have a chat." So, the lesson here is you might not know enough about what your product does and what the other people are doing. The manager was quite convinced that they had this privacy preserving tool that did everything on the edge and there was some computation on the edge.

But there was a point where the images were sent up to the central servers for pre-processing which broke the entire premise of all the computations on the edge. So in a marketing talk sure that that sounds great. When you speak to the developers realize no this is not actually what's happening your own problems. But unfortunately all these anecdotes are as as light-hearted as that. Sometimes it gets serious. So this is actually a this next one was a um a specific blockchain project encryption. The team says our citizen journalist app encrypts all data keeping those working in dangerous countries safe. So this is an app for people who are taking photos and whatever and

writing stuff in dangerous parts of the world. And so you want to somehow keep that safe in case you get caught. Some places don't like journalists. And I said, "Okay, well, what happens if someone is caught in one of these countries with your citizen journalist encryption app on their phone?" Because in some parts of the world it might not be the content or the photos that you have, but the mere presence of a encryption app on your phone that's called the citizen journalist encryption app designed to encrypt pictures and and whatnot could be enough to land you in a little bit of trouble or a lot of

trouble. And their response was, "Oops, we hadn't thought of that." So the lesson here is you might not know enough about where your product is used. It's one thing to say, well, it'll be perfect encryption and this works great in countries with excellent democratic processes and legal process, things like this. But in some countries, these processes are non-existent. And so just the app on your phone is enough to get you hauled off and and and disappeared for a bit or a lot. Last one, blockchain hash.

same same team this time. Our citizen journalist app again that's uses signed data hashes to the blockchain to indisputably prove authorship. So you want to take some sort of picture or something and you want to say well I took this picture but I don't want to tell everyone just yeah I took this picture. So I'll sort of sign it off to blockchain and later I can prove that it was me that that took this picture. I said, "Okay, well, can someone working in a dangerous country with your app later remove their authorship authorship signature if they're caught or if they're about to be caught? Can you somehow scrub the um the stuff of the blockchain in some way? Because maybe this is great. If you're not caught doing something that the sort of local government doesn't

like, but as soon as you get caught, you might want to quickly wipe everything off." And their response was, "Oh, I hadn't thought that one either." So the lesson here is you might not know enough about how dangerous your product is. This was created with the best of intentions. Help people in dangerous parts of the world do their citizen journalism. But without proper consideration, you can end up doing more harm than good. That's the point of the talk that just with the best of intentions certainly necessary. Don't try and do harm, but that might not be sufficient.

You could accidentally do harm. And having worked with lots of technically trained people over many many years, I understand that this this perspective of well I'm trying to help. So sure that's enough to to be helping. No, they don't. One doesn't equal the other. So how can developers think about safety? Well, there's kind of thousands of tech safety frameworks in circulation at the moment, which is a bit annoying because people want to technical people often ask, well, what's the what's the right way to do this? I've had mathematicians ask me, you know, great, thank you for informance of this. Um, what's the algorithm for ethics and safety? Like, no, it doesn't work like that for

starters. Um, people disagree or people have different suggestions. And there's this sort of ocean of of ways to do this out there. They're usually domain specific and more problematically they're usually endpoint oriented and they say achieve this great goal. Okay, how doesn't say teams tell me look we've had we worked with this framework it's great we agree with it but what do we do? How do we get to the end point that the framework tells us about? So my colleagues and I tried to create a generalized tool broken down into 10 key pillars. I'll show this in a moment that sort of follows the the fits into the workflow of any mathematically

powered project. Any project you're doing that has lots of numbers. This should be able to help you in some way. There's a link there. It's called the manifesto for the responsible development of mathematical work. A tool for practitioners and for management and for management is important. Remember few slides ago the manager saying, "Oh, we we do all the cubidity on the edge." And the developer saying, "No, we don't. We send the server." Managers need to understand this as well. So here's how it works. This is a very bridged version. Don't look at the left. Look at the right. First thing, should I be doing this? Next thing, who's on my team? Have I do I have someone who knows about boats? What am I working with? What data? Things like this. How am I working

with it? How will I do the mathematics that will optimize over? What am I recording? Do I know what's happening out there? Can I keep this safe? Who might criticize me? and where something goes wrong. I've given you this in sort of 10 small lines, but it unpackages to a 20page document, but it's clear for technical people who are going through a mathematical process to unpackage. And let's think about this, you know, does does this apply to blockchain? I mean, I've used the word mathematics here quite a lot. I haven't said blockchain anywhere, but that's the point. You can take mathematics and sort of replace with blockchain or AI or fintech or statistics operations

research whatever whatever your favorite part of mathematics is or mathematical powered work is you can put in there so work on blockchain is relevant yes very much so here's a take-home exercise for everyone to try I'm going to go through it briefly now but what are the ethical issues in creating a blockchain based public transport ticketing system. So actually the origins of this um 10 pillars manifesto come from a similar initial case study which was one of the ethical

issues in using AI to redesign a bus network. But you can modify this and say well so use blockchain based public transport ticketing system. Do I face any such issues? It's just tickets right? I mean does it matter? It's just making bus tickets that go on a blockchain. What's the problem? Well, there might be a problem. So maybe your motivation for this is you're thinking that well we're worried that travelers might not have a way to you know undeniably prove they had a ticket cuz maybe you buy a ticket in some sort of app and then the app malfunctions. You get on the bus and you

have nowhere to prove that you had the ticket. You don't have root access to whatever you know access to the the source code of the app. You can't see what it's doing and so you get fined or something like this or or charged for for traveling without a ticket. So this might be the problem that you want to solve. Okay, so start thinking well I'm going to use blockchain technology for this. How much energy or resource will this system consume per ticket? You know, electricity transaction cost, proof of work, proof of space needed blockchain compared to say a little piece of paper that people used to get when they got on the bus that said, you know, the date, the time, and the punch in the corner. Is it proportional? Are you now using, you know, a megawatt of

power to prove that one person bought a oneway ticket, two stops on on a bus? Is this fast enough for travelers to get on the bus in 10 seconds? Will it actually reliably work within 10 seconds 99.99% of the time? There's no point, you know, I buy my ticket, go wait 20 minutes for the ticket to issue because there's some blockchain registering process. Who will be left behind technologically? Who will or won't have the knowhow, the tech skills, or the hardware like a smartphone to use this?

You probably need your own digital device to use this. Does every sort of smartphone when they're traveling and their connectivity? What happens if the blockchain I'm using forks? Which one will we say holds the right uh the real ticket data? Blockchain is decentralizing, but you now might be centralizing the ticketing systems of not just one, but potentially hundreds or thousands of cities around the world onto the one blockchain. This is a central point of failure and a central point of power. Perfect controls that has massive power. Now, whoever has sufficient influence over that. How will you actually prove in a

courtroom that this system works if the concern is to you know travelers can prove that they that they um go if travelers can prove that they used that they had a ticket? Will a court actually look at this information? Can this be used by the defense or by the prosecution in a courtroom? Is it admissible? Who is your intended beneficiary? Is it travelers or is it the transport network transport company? Who is the one who's actually supposed to gain from this or is it both? And how do you reconcile their different their competing

interests? What aspects of ticketing do you need to know about beforehand? How much experience do you have with a public transporting system? Have you ever sold a bus ticket? You been on a boat, ever sold a bus ticket? How will you design such a system with if you don't have the right experience in that particular domain? How will you know if and when it's malfunctioning? I not properly issuing tickets or giving out free tickets. Is there a way to actually keep an eye on this in check? What will happen if you overwhelm the blockchain and blow out transaction times there? Okay, all of a sudden there's, you know, millions of tickets being loaded onto this chain every every minute. It's not designed for this sort

of uh this sort of data flow. How might existing ticket providing organizations react when you take their business? What might the consequences or push back be there? And what happens if the system fails? Can you have a backup? Is there a backup available or some sort of fallback mechanism if for some reason technology ceases to function? But ultimately, is there not already an existing system? Is there not already a way for people to get tickets? And is this an existing active problem that needs solving? Has anyone actually come to you and

complained about their ticketing system being unreliable or not being provable that they had a ticket? Do we need a technology like blockchain here? Maybe we do. But do you have evidence that we need that? Is there actually a problem that needs to be solved and it needs blockchain technology to solve it? So that's roughly how you can use this sort of understanding to take a seemingly innocent example like blockchain bus tickets and start picking it apart. And I invite you to go through and think more about what the I mean I gave a subset there. There are probably several other issues that come up in this technology.

So, thank you for your attention and feel free to get in [Applause] touch. But does anybody have any questions? All right. Well, um, thank you for the talk. I think ethics it's so important to consider and I hope that we all have a framework now to address that.

Thank you. [Music]