What Can Ethereum Do for India? The Exam Leak Challenge

Transcript

[music] [music] Thank you so much and thank you so much for attending this um very interactive workshop. Um my name is Utin. I am the co-founder of Agura Citizen Network and so we are trying to build solutions to help communities find consensus to navigate disagreement and to come together to collective decisions. And um so with uh me here today is my co-host Nicholas. He's also my co-founder at Agora and we also have uh Pri and she's a Mumbai based crypto analyst and she's also leading the um legal and governance

hub today and um originally um I pitched a talk at Defcon to present our tool and and then the Devcon team told us that oh it will be better if uh you can turn it into a workshop so people can experience with it and so on and um so Then I uh started collaborating with Pri to um make it into um workshop that has a very local topic um that's meaningful for the communities here and she told me about these um case um that I find just absolutely um well mind-blowing about the paper leak. So um before we start a

workshop just quick show of hands um do you know the case that uh we are mentioning? Yeah. Okay. Yeah. So, um I didn't know about it. Um and I was just uh trying to think like uh what what can we do and we started thinking about some very general topics like maybe some land land industry digital identity. There are some things that are already uh been failed by uh existing solutions. And then Pri told me about this um story that happened this year um where a national medical exam um the paper was leaked before the exam and which led to it being cancelled and more

than 2 million students had to retake it which is absolutely um tragic and this also led to massive movements on the streets. Um so apparently the uh minister agreed to uh completely digitalize um this uh process next year. However, will it still be enough um to just digitalize the exam? Will it prevent this from happening? And I think there are so many emotions um that are um that were caused by uh by these um incidents. And so here today we are not going to debate if the government should

have canled it or uh if um people who were impacted if they should retake it or that's beyond um what we are going to discuss today. So we want to um focus the uh energy the experience on what can we do now to prevent it from happening again. And uh if um blockchain could help and if blockchain could help um if Ethereum could help and there are no correct answers here and we are all welcome to challenge and it's okay if the conclusion is no we don't need blockchain no we don't need Ethereum and

that's fine too but we want to um come as a group to um propose solutions to stress test them and to uh eventually Um hopefully we'll all work out here with some concrete ideas and um so yeah what would an exam system need to guarantee for students to trust it? Again this is the challenge that we will all tackle as a as a group today and we are all one team. Um however this doesn't mean that we won't have different approaches. Um so there are two kind of two approaches that are pulling a little bit into opposite directions um that are all part of the

um values the properties actually um of blockchain and one is trans um one is privacy confidentiality and the other one is uh transparency. So for the first approach um we think of it like a vault uh how to protect the secret and so it stays safe and I will uh invite u pre um to uh share what you are thinking about this approach. So we will present uh these two approaches before we start breakout discussions.

>> Okay. So let me reiterate uh exactly what you said there are no right or wrong answers and we have to focus on finding a solution uh and and how we have basically designed this workshop is that I will put forth one solution and you know Nicholas would have one solution and we use that as a starting point and maybe we come up up with a third fourth fifth solution completely okay so it's not uh are you taking my side or his Right? Uh you know we have to figure perhaps you could see value in either or one or none but the point is we focus on the solution aspect of it and not any other aspect of things.

Okay. I would just want to reiterate it. So from my point of view I feel that whatever approach we take we need to make sure that there is a tamperproof element to it. And according to me, blockchain offers that. Okay. Uh we need to make sure that there are as few hands as possible that touch the paper till it reaches from the printer to the student. Okay? And that could be in any form, but there should be a record of it and there should be accountability at each stage and perhaps even a security check at each stage to make sure that you know we know who has access to the paper at

every time at every point in time and also we you know there are checks in place to make sure that there is a way to trace the accountability back. Okay. Again, how that really works is, you know, something that we would have to work together if you choose to work in this direction or, you know, using a hybrid of this direction. >> Thank you so much, Pri. And um next up, we have another approach. Um so these two approaches are not mutually exclusive and uh we will potentially need um properties of both. So next up Nicholas can you share a bit what you are thinking about this?

>> Yes. So uh the way I think about this problem is actually a little bit counterintuitive. I actually think about it um in terms of system design and I would encourage you to do so as well. What does it mean? It's a bit complicated uh when you hear it and you're not a software engineer. It's it's really about it's simple. It's really about thinking about the actors like who are involved first. Uh what are the trust relationship? Trust very important here. What are the limitations and and and also what technology exists and what are the actual properties of these technologies and then how can eventually these technologies can be

used collectively to enhance what the community can do without it. So uh instead of just thinking of the technology immediately, I think I would encourage you to use your knowledge. You have much more knowledge than me about it about what are the actual communities involved and because a solution for India is not going to be the same as the solution for China or for France or something like this because the society is not the the same. So the incentives are not the the same groups are not the the same and so that's why so the library why the library so just take the example of Bitcoin or Ethereum those are systems that are radically transparent it's a public ledger of

trans transactions and it's very counterintuitive to understand that it's secure because the public key of Satoshi Nakamoto is here available so if um [snorts] and there are billions of dollars there. So theoretically it's possible to brute force it and to try so many combination of numbers and get access to this to to this account. So it is actually now accepted this technology but it's very counterintuitive. It's very weird right because statistically it's possible there is risk the risk exists but it's mathemat mathematically accepted double.

So uh I I because of the understanding that okay who will spend you have to spend a lot of money in computers to be able to then crack this and it will take I don't remember I don't remember how many years but like at least uh uh uh thousands of years to to to statistically to be able to to crack it. So nobody's going to is going to do it. So that's why I will encourage you to understand for example what is the human cognition?

What's the limitation of human cognition when it comes to like reading papers when it come to like uh training for for an exams? What is the goal of the exams? Why are you passing the exams? What what are we testing? And then um uh uh contrary so to go into a little bit into the solution but I don't want to to say too much. Uh what would be uh let's say like if if all the exams the possible exams are actually available uh is it possible for for everyone to go through all of them and is it okay if there's a risk that the actual exam uh someone has already already train it because it was available. So that's the the thinking I

want you to to go through because this thinking can change radically uh the solution and maybe provide a better one than the the intuitive solution which is the for [snorts] me the more private private privacy centric one. >> Yeah thank you so much Nicholas. Um so um both of these approaches they are just ways um to uh guide you into uh to to assist you in thinking about uh how to come up with um some proposals and I just want to um say that um all the proposals well maybe you think they are perfect when you come up with it.

However we are going to have the group stress test it. So even if you yourself come up with your perfect solution, we are going to ask you later to attack it as well. And uh before um the um breakout session, we are going to ask you to scan this QR code and to do a quick round of voting on a list of statements um to uh quickly map uh what are you thinking about um as a group and we'll look at the result quickly before we go into the breakout. Um, let me know if you have issues um getting to the page and if you have uh internet connection

issues Is it perfect? Yeah. So once you uh arrive on um on this link, you are going to see a

list of statements and we invite you to click uh to vote agree, disagree or on shore on this uh list and then we are going to look at the results. um together. Um what you're all thinking about this I don't done it. It's okay.

Yeah, we're going to um give uh two more minutes for those who haven't been able to vote. >> Whoops. Okay. Okay. We already have two opinion groups with five people who are mapped in the groups. Yeah. We're going to give her one more minute.

right? Yeah, it's 11:20, so Oh. Oh, yeah, of course.

>> Um, yeah. Yeah. Yeah. Okay. Now we have three opinion groups. Um u for those who haven't been able to vote, it's it's okay. Um it's okay for now. We can do that later. Um but uh I just want to explain a little bit um what we were um what what we were doing in this um in this voting round. So I just closed the conversation I froze the

conversation. Um so what was happening is that as you were clicking through those opinions um the tool is um putting people who think in similar who vote in similar patterns in different opinion groups and this step is deterministic. It's based on machine learning learning algorithms to just quickly think okay these people vote similarly those people vote similarly and [snorts] then we are using LLM to summarize on what's a representative statement of each group and then we can see what's the consensus and what is a divisive um from this um conversation for example um for now based on what we all think

the most divisive statements are the safest exam is one where everyone can see all the possible questions but no one knows which ones will be asked. We see a lot of disagreement, some unsure, some agree as well. Um, [snorts] and then the next one is I trust the national entrance exams in my country to be fair. And then the third most divisive things are the safest exam is one where as few people as possible ever see the real questions. So we can see that um this uh this statement is kind of pulling towards the um library um approach for radical transparency. And the third question is pulling towards

the vote approach um to protect the secret which is very interesting because next we are going to ask those of you who want to um work in the towards the vote direction to sit uh on these sides of the room and pre will be facilitate your discussions and those who are who tend to agree more on the radical transparency ideas you'll be sitting on this side and uh we'll Nicholas will facilitate your discussions and the discussions will take um the form of um of um of of of a group uh workshops I think based on the people that we have

today. Um I think maybe you can work in groups of uh two or three people and then we are going to challenge you to design some proposals uh with this format to to to stop leaks or to do what who should do what as long as and we I we put a guardrail here and uh for example uh we have an example here like to stop fake reviews platform should only accept reviews from anonymous buyers as long as a platform can delete negative reviews.

So we are going to ask you to work in pairs uh or at most groups of three to come up with uh this kind of short statements and then we will ask you as a group to submit your statements um to the same link that we just saw and then we are going to let the group vote again on all of your proposals um before we stress test. Okay. So you can feel free to move around if you want to work uh for the vote idea or the library idea and you will be working in pairs.

>> The vote idea is a protect the secret. You'll be sitting in this room. Yeah. So who whoever uh is of the opinion that uh it should be secretive and uh you know the it should not be a library where people can where the the ultimate question paper is pulled out of a library of a few questions which is published and known for everyone. If you if you if you think that a library approach is better, you sit on this side and Nicholas will guide you. And if you think that the whole thing should be secretive, private and protected, uh, you know, sit on this side and work on the idea. The the the point is you need to stress test the entire thing and make

your case um on on why it should be this way. And also because mine ours is a halfbaked idea, there are a lot of holes and a lot of gaps in it. So you all need to make sure that you're plugging those gaps or perhaps you are taking the initial idea and moving in a solutionoriented direction. Uh ultimately you will have to your groups will have to uh sort of um convince the the whole audience why that is the best solution. So uh yeah you know so you have to make it a little believable right?

>> Yeah. Yes. And you have a seven minutes from now to come up with your proposals. Um, each group will need to do at least one proposal and if you have more ideas, we'll come to submit more statements to be voted on later. So you ready? Just sitting here putting

back. Should I solution So I would say >> yeah right now exactly the field right Yes,

>> the students are taught something for a whole year and now they are able to generate answers for the I'm not really sure. >> Yeah.

Everybody from the same happens. not there is a sense

This is very like something only that much go beyond that who is best

the one time. So, same question. We also know the who is asking questions are equally

the one in someone that I don't understand.

I have to be aware is that if you have the library, it could be that question because you know there's 100 questions and they're going to select 10 of them. It could be. But either way, if the questions are open, you will have a chance to examine all the questions and you can prepare and memorize more. Whereas if they are secret and all the questions are new and they are like surprised on the exam day, it requires more kind of critical thinking on the spot. So it it just varies different kinds of student.

>> Yeah. I I agree. I mean, it's definitely democratizing, [clears throat] [clears throat] >> right? But the question will be different from the previous exams. Yeah, I mean sure you cannot prepare all

of it but if you know I guess if you if you have a data set of questions then you can prepare according to those questions that I know >> I think more questions. [clears throat] Yeah. Yeah. Okay.

We should pray together. All right. >> Yeah. Yeah. >> People don't no question.

So we need to have a large set of questions that was >> last. >> Yeah, >> you can. Yeah. >> Yeah. I almost feel like I don't know. It could almost be like an algorithm. >> An algorithm that's that that's open source that people could see that that that picks the questions. You know what I mean? It's a similar to >> like Instagram. >> Yeah. But it's also similar to social media that has open source algorithms where you could you can see how the social media ranks different posts different content and >> thing

of at the moment I'm printing printed. So the this is also if I understand from India It's the generation of designation where

you don't have to have audiences. You have the >> We have two minutes left to submit your proposals. Um, yeah, if you disagree with your own group, it's okay. Just submit your own opinions.

>> Yeah, >> I guess one thing is like what we talked about that questions are randomly generated very close to the exam. >> Japan. >> Yes. Are the exempt always? >> Is it always the same date? >> Yeah. Yeah.

Um, you think one of them should put it in a show or >> Oh, that's cool. >> Who chooses which one? >> They will communicate in the morning that it's going to be set number three that they have to >> That's so weird. I'm just >> So what are your proposals? Just make

sure they're honored. Actually, we still might be

smart. >> Yeah. Yeah. Yeah. Sure. That's what I thought. That's what I thought. >> Yeah. Yeah. >> I don't It's quite [laughter] some AI has removed >> which is good. It's fine. Dr. [laughter] The question is like I assume people can't use AI at the exam, right?

>> Make sure like >> yeah that's where the the timing thing becomes important because >> it even if you have a few minutes advantage >> you can feed it to the AI and you can so quickly get the answers. >> Everyone has to bring that woman at the same time. >> Yeah. This is coordination. >> Yeah. This that this 3D. >> Yeah. Actually, I wonder if you could generate >> I mean, I know it's a bit hard because we have to print the thing, but it could be interesting if you already have people in the exam room and then you generate the questions.

>> Yeah. >> And print the split there. So people might have to wait like 10 15 minutes while they're in the room, but then you reduce the chance that someone has access to AI. >> Let me think system like blockchain because if anybody wants to try the system will fall down. When you are done writing your own proposals, you can also vote on the new statements that are submitted.

>> Okay, great. Um, they're also want to vote on Yeah. No, you first and then see like how what what comes up with this is not determined how discovered.

Yeah, whatever. These are new statements that are coming. >> No. So, just go one by one. Okay, we have one minute left. Yeah.

All of us. So understand whether or not Maybe I'm done with you.

So time's up. I'm going to close the conversation. If you haven't finished, uh you can do it later as well. And okay, I see that we have uh five different clusters. Um it would take a bit of time. Um yeah we have five different clusters uh which means that now we are all diverging in a lot of different uh directions. However um let's first see what's common ground. Um so despite that we think in five different directions. We all agree that if questions are drawn at random anyone

should be able to check the draw wasn't read. So that's the one thing we all agree on despite our differences. And [snorts] then um let's see what is uh dividing us. So um we can go to divisive and see these are the most uh these are the top divisive opinions. First one is moving need to computers will be enough to stop leads. Uh second one um the safest system is one where one can see all the possible equations. So that's the same as before. And then I accept a tiny chance that a student practice a question almost identical to the one on the exam. And then we have an proposal um that was proposed here by

you. Um that's also divisive. Um so like 1,000 more teachers will give a questions and AI will make a paper on spot. So there will be no reason for leads. Um may I ask who's comfortable to share why you disagree with this proposal? Uh because now we are uh moving on to the to the next phase which is to stress test uh your proposals. So I just want to know if you agree with it why you agree and if you disagree with this statement why you disagree.

Can >> we see the statement? >> Yeah. who who wants to share first. Yeah, >> I was not able to vote but like they would say that you know it will make it extremely tough for >> it's extremely tough >> because the students they already know if they are going to say that they're going to be any random question they're not going to be really random like the thousand teachers that have been selected for it they will be making

a set of question already so the questions will be coming out of those people is that the security on that that's why I'm very unsure because if that are involved in the system then there's still a potential >> I have so I have just me mentioned that we will use that blockch so that teacher don't know other teachers. So like it will be a decentralized platform like teacher don't know that who is so teachers are

got selected yet. >> Yeah. >> Uh so like uh if teacher don't know other teachers and all the peoples don't know who are the teachers that have selected this thousands of uh who that are selected in like in millions of the population who is who are that thousand teachers. So like it will be not so >> so I I disagree with this because I don't think blockchain does anything here. >> Uh like you can have you can have like a set of teachers and have just like them in an encryption pool right and and you don't need that that that's the reason that that's the reason why actually I disagree with

it. Um but uh yeah. >> Yeah. So I think we unfortunately we are approaching the end um the time that we already have. Um but I think we are all getting deeper in the solutions and uh we also know why people disagree with it and if people are questioning the the use of blockchain you can modify your proposals and you can modify you can iterate on the proposals and then submit it again for for voting. Um so I I don't think we have the time to to do it um during the the workshop but uh if you want to iterate uh on your on your proposed solutions um or if you are

inspired by your um opponent's solution um please feel free to do this and um after the workshop and um the conversation this link will stay open for one week and we are going to share your solution. um to the uh to the community also. Um >> yes. >> Um pre so um I just want to reiterate that this is uh was a fun workshop idea which is why I thought we should you know engage in it because I think when we started uh a lot you know it it was naturally a very uh debate kind of a format. it's A

versus B. But as you start to really get into it and actually design a solution, you do realize that you might disagree with somebody you initially agreed with and you might find agreement with people uh that you didn't imagine that you would. So when we really think of a solution from a working point of view, we realize how complicated things are and how intricately we have to think things through. And uh tools like these are super cool because I could see from the very first discussion that a lot of you had a similar idea to Nicholas which said that there should be a bunch of questions which are then randomly

assigned which is what he mentioned. So um when you were trying to poke holes in in what I was in in what my proposal was you tended towards Nicholas you know. So what I what I mean to say is that we should keep an open mind and tools like these are called deliberation tools and they help us with taking the discussion forward where we're not really so concerned about what we've agreed with. What we agree on is said and done and we work on what we do not agree on and we figure out why we don't agree on it. So, you know, just as this conversation was getting, you know, to that point, we've had to cut it off because I think we needed easily one one and a half hours

to do this uh you know, decently. But you do see how in which direction this was heading and what the ultimate goal was. I I just want to add if you if you want to continue the conversation I think something that is limiting currently the application it's work in progress is that it's better to create short uh uh statements that easy to agree or disagree with. So it's good to like break down your idea in short ideas so that people can clearly uh vote on something because otherwise they they're not sure because they oh I like that but I don't like that. If it's too complicated it it doesn't work. So, and as you can see also from a product perspective like this kind of

product are evolving and it's going to be better and better and it's going to be easier and easier for everyone to participate and if you have ideas also how to improve that if you have ideas about how to improve the product uh I'm interested [music]