ZKPodcast: Blockchain 101: Randomness and Random Beacons with Justin Drake

Transcript

[Music] welcome to zero knowledge a podcast where we explore the latest in blockchain technology and the decentralized web the show is hosted by me Anna and me Fredrik [Music] in this week's episode we sit down with Justin Drake from the theory foundation to talk about randomness how it's used in blockchain what a random beacon is and more today we're going to talk about randomness and

hopefully it's a one on one episode that sort of stays within the educational bounds and we can be useful here before we kick off hello Anna hello we have Justin Drake with us which actually is on the episode by People's Choice Award we made a call out on our Telegram channel and asked who the best persons talk about randomness is and everyone voted for you Justin congratulations oh wow great so welcome to the podcast thank you thank you for having me maybe you can just introduce yourself quickly Justin sure so I'm Justin Drake and I'm a researcher at the film foundation and I'm mostly helping

out with the affair and 2.0 effort and I've been with the infirm foundation for about a year and a half now when you're when you say you're working on earth to point out like why did your name come up for randomness like when we said who's the expert on randomness why did your name come up ah-ha so I mean I I look at all aspects of a film 2.0 and at one point in time I ran down the the randomness rabbit hole I'm trying to find effectively the the best solution for a film 2.0 and it turns out that one of the solutions that we looked into is so-called vdf verifiable delay functions and that is a whole story with lots of moving parts

but it's it's one of the you know research efforts that I'm leading a different Foundation and the idea of EDS is to try and make a randomness beacon which has all the properties that we want for a firm to point oh we want to structure this episode as a blockchain 101 episode on randomness and I think that through the conversation we will probably get has to touch on the work that you're doing Justin but to start off I want to define randomness and the reason I want to do this is that there is some confusion about the term here are the standard definitions randomness is the lack of purpose logic or objectivity of an event it's a type of circumstance or

event that is described by a probabilistic distribution randomness has also been defined as having the property that all possible outcomes are equally likely so to illustrate that last point let's take the number pi so pi actually is a series of seemingly random numbers pi definitely has that last property where all possible outcomes are equal so say you're looking for a specific three-digit pattern series it's no more likely that you would find the series 5 to 8 then it would be that you find the series 4 7 6

there's no pattern so this is great but it wouldn't necessarily satisfy the definition of randomness for cryptographic purposes in the case of cryptography one needs a form of randomness with a very specific quality that is that the random number is both unpredictable and kept secret with cryptography one is looking for an asymmetry of information you want to be able to perform some secure operation without an adversary being able to replicate it or derive the same results going back to the example of Pi the result of the calculation of Pi has no discernible pattern the actual number pi is very random the sequence of numbers appears with equal frequency therefore all number combos are equally likely and

so you could call this random but given that the method for generating this number is known and therefore predictable this would not satisfy the requirements of a cryptographic randomness so how do you add this unpredictability that you so need well this is a cool feature about randomness if you take a random number and mix it with another number random or not you will have the same level of unpredictability as the most unpredictable of the two so when you think of a computer generating randomness what it would often do is look to some outside real-world source to generate a random number and potentially combine that then with other numbers the source of this could be temperature or the timing of

some real-world thing for example CloudFlare published a really great article about randomness and how they generate it they actually have a wall of lava lamps in their office and they use a video recording of the lava lamps which would be fully unpredictable to generate randomness so particular pixels on the screen and depending on the color that they take this could then be used as a source of real real-world randomness cool so with this I think we can now jump into how randomness can be used in blockchain yeah so I think randomness in blockchain is an interesting topic that is related to cryptography obviously but also somewhat

separate like I think the first time anyone interacts with sort of randomness in the blockchain context is at least in the etherium world if you're trying to write a smart contract that does something like a lottery and they go okay how do I randomly choose a winner out of this set of people who are participating in the water and the first answer that they will think of is I'll use the hash of the last block because that looks like a random number and there's actually some problems with this and we've covered it slightly on the podcast before but not really dived into it so maybe Justin you can talk a little bit about what kinds of like randomness do we have and like say aetherium today

and what's the problem with that why is it not enough yeah so I think going back to definitions randomness for me is a non predictable source of entropy and one of the the big challenges in the blockchain space is that we want to do it in a decentralized fashion I mean there's other properties that we care about other than unpredictability for example we don't want the randomness or the source of entropy to be in Irbil and we want kind of a steady stream of it we don't want it to suddenly stop in the case of affair 1.0 the randomness that we get from the block hashes is actually falls under a

very large class of randomness beacons which I would categorize as commit reveal and this is actually very somewhat similar to the way that computers derive randomness by sampling from the environment so the idea is that some external actor to the blockchain will commit somehow and then they have the choice to reveal that local entropy which is external to the blockchain and put it on chain so in the case of proof of work you can kind of think of the commitment phase as being burning a lot of electricity to generate a block and

then once you have generated a block the block hash will look random it will be a good local source of randomness which is the right kind of externally to the blockchain and now the miner has the incentive to publish this block and include it on chain and basically include this randomness for everyone the main problem with proof of work and and all the other commits reveal schemes is that the the miner or whatever external actor it is it could be about the data for example and proof of stake has the option to not include that local entropy on chain and so by doing so they basically have the option to discard a

random number that they don't like they just don't tell the world about it and as such we say that they have the ability to manipulate the source of randomness so even though the block hashes look totally unpredictable and they satisfy the decentralized property and they also satisfy the liveness property which is that you will have a constant stream of blocks they don't satisfy the on man appealability property going back to the lottery example here that would mean essentially that you know the minor can participate in the lottery and then start mining a block and then he keeps churning out new hashes and and like

until he finds a proof-of-work that allows him to publish that block if he doesn't win the lottery in that block he can either keep mining and let's add another transaction and then keep mining until he finds a block where where he does actually win or he can you know say you know someone else won I'm just gonna throw this one way and then keep keep doing that so you can sort of keep churning to try to gain an advantage in winning and depending on how large of a mining portion this guy has he may or may not be more or less likely to doing you can actually affect the outcome of the lottery on average a lottery is one example but what other uses of on chained randomness do you see there is

so the use cases of generating random is on chain cannot fall in two buckets bucketz number one is just using the randomness for you know consensus layer use cases and the second bucket is for application layer stuff so the lottery for example would be at the application layer in terms of the consensus layer it's very simple with simply randomly sampling consensus participants to do certain tasks so if you take Bitcoin or film 1.0 the main task that needs to be done is block proposing so extending the blockchain with a new block and we want

that process to be random and fair but you can have more sophisticated reasons to use randomness at the consensus layer for example if instead of sampling a single monopolistic actor for short period of time you can sample the whole committee and so in affirmative we know for example we have the notion of the crosslink committee or attestation committees in this case though rent is randomness randomness is used to make choices sort of of like which actors get included that sort of the way it's been used in that case when you say a committee I'm just curious what that means yeah so the idea of committees is that you have a pool of participants and you have a

so-called honesty assumption so you assume that some fraction of these participants will be honest and generally the properties that this pool of participants can be very large because if you're working in a open decentralized blockchain anyone can just come in and become a consist consensus participant so generally it's not practical for everyone to participate at the same time one technique that is used quite commonly in proof of state systems is that you will have a notion of a statistically representative committee so if you take a if you take a committee which is large enough let's say a thousand validators

then it will be very representative of the global pool of but if you took a small lake that doesn't completely define where randomness comes in like the way I've understood it is you'd have a larger pool and in the sampling process it's almost like random numbers are picked I mean either you do it in some pattern like we picked the first 100 then the second 100 in a predictable way but if you want wanted to actually sample you need some sort of randomness to pick out random participants from this set and I guess the reason that you would do that is by adding this randomness it cannot be gamed you can't figure out who will be chosen for that

committee and then put your friends in that space I know it's not friends in this case but like yeah but exactly I mean if you had a thousand validators and you knew it was a hundred than the next hundred you could set yourself up because there's no like built-in civil resistance necessarily you could set yourself up with a hundred validator set them all in a row and then you know that by this block I will have total control over the production and so this is for we've just been talking about like the POS use of randomness but going back to what you'd said originally about proof of work like is proof of work a brendham general is that basically what it what role it takes there yes exactly and it's a extremely expensive random number generator you can think of that

basically it's a way to take a pool of Possible's in this case the miners and fairly sampling them one at a time and not only is it extremely expensive you know it could burn let's say 1 billion dollars per year it's also not perfect as manipulable and so it turns out that we can build randomness beacons which are much much cheaper you know they might burn on the order of 1 million dollars per year or so thousand times cheaper and also have better properties there are on many people so I yeah going back to proof of work and and this switch to proof of stake one of my

questions was so we have this randomness generation and for work we know it's manipulable so what is a better solution you just mentioned the random beacon but really what's the better way of generating randomness in this case when in that switch the proof of stake what changes what what capabilities do we gain so improve of take we have three options the first option is to go with the commit reveal family of randomness beacons and that's in the way replicating the the proof of work whereby you have Possible's who commit to local entropy and then they have the option to reveal or not reveal and all these systems have the so-called last

revealer attack which is when you're invited to reveal and after your reveal that's going to be the random number that is used by whatever application then you have the option to reveal not reveal and hence you can manipulate the randomness so there's two other classes of randomness beacons which don't suffer from the manipulation of attackers number one is a class around threshold cryptography and probably the most well known system here is definitely special and the other approach which doesn't suffer from the manipulation is basically taking a commit reveal scheme

and adding a third step which I call an extraction scheme so it becomes commit reveal extract with this final extraction scheme you basically prevent attackers from being able to make informed decisions about the reveal or the knot or the non reveal and hence not being able to manipulate the randomness you've just mentioned a few different sort of mechanisms for creating this but what is the input what's the like where where's the original randomness coming into this and where does it come from so in in all cases the original randomness comes from the external consensus participants so you know they will generate randomness in the traditional

way using their computer and so long as the honest players do that basically choose what looks to them like a random number then with the honesty assumptions you can have what looks random to everyone on the blockchain by going through the protocols basically in like a committed scheme it's up to each participant what they're committing I mean they're they're basically saying you know here's a random value trust me that it's random and you just have to trust them and what the source of entropy is can be up to each participant so usually it might be see spring or

something running on their computer sampling times probably from their CPU you know I there it's sort of funny like people can get crazy about this you mentioned the lava lamp situation and CloudFlare which is a funny one but there's an actual like device you can buy and plug into your computer that's like a Geiger counter or sampling the background radiation of the universe and that's like the best probably the best source of randomness that you can get but yeah and it could commit reveal scheme and things pretty obvious but to me it's not at all clear like in a threshold cryptography scenario what is the source of entropy because there's some message that sent out and then everyone just kind of signs off on that

and then you know the threshold is passed and some some signature gets forwarded onwards which is used as the random number but what is the input to all of this so the input is still random numbers generated on people's computer I guess the difference is that there's so a so called secret sharing scheme whereby with these local pieces of randomness you can generate a shared secret which is common to everyone but which has the property that no one knows it in order to read arrive it you need a certain threshold of participants to kind of collaborate and reconstruct it

so it's a way of basically having a secret which no one knows it just exists but it's unknowable and it split into tiny slices and each slice is given to a different validator so it's somewhat similar to you know fresh hold signatures sounds a lot like an NPC mm-hmm it isn't MPC yes so an NPC basically allows for every possibility in the NPC to have a piece of secret and then you want to compute a function basically by preserving the the randomness and the function will be a a mixing function which combines all the secrets and it has a another property

which is a threshold property which is that you can recover the secret if some fraction you know for example 1/2 well will collaborate and reconstruct it for listeners who haven't heard the episode we did on NPCs NPCs are it's short for multi-party computation and there's well link to the episode on that if you want to hear a little bit about it but like as I understood it the inputs are supposed to be random you're using an NPC to make sure that even if there's like a small proportion of the people or the participants who are malicious and potentially not giving real random number

are having some coordinated number even if they did that using an NPC the system would still be secure yes so it's actually not too much of a problem if you have a bunch of people who just pick you know pick zero pick a totally non random number so long as you have one single person who picks a random number then the DS the shared secret will be random the bigger problem with the threshold cryptography is that if you have a large enough attacker who can basically go above the threshold then they can compute ahead of time all the random numbers they can just go through

you know the reconstruction process and and predict the future the other problem with the threshold cryptography is that is so-called liveness problem so if you have so many people who are offline that you can't even reach the threshold then you can't generate the next random number and and your randomness beacon will stall just thinking before we dig deeper into there because we wanted to talk about the random beacon as like a separate entity here I'm I think I'm still trying to get to the I'm trying to basically the question I'm trying to come to is why do we hear about randomness using a block header as the input like let's sort of what I was talking about before with using the hash

of the block okay the previous block got in but there's there's two ways to use the block codes one is in the proof of work and basically the block header will encapsulate the the proof of work kind of result of the the puzzle you know then the nonce and that nonce will have entropy and so the the overall block header block hash will have entropy but in a proof of stake system you know it's it's free to come up with tons and tons of block hashes because you don't have to do the work so you can just you know shuffle around the transactions and then you'll get different block hashes and so in proof of stake using the block hashes

that's a completely broken solution I think that leads us pretty well into my next question which is what do we use randomness for in proof of stake like you say it's not you know to create the next block hash it like it doesn't mark the same way as proof of work so what do we use it for and and is there a way to bring the randomness you know from the consensus layer into the application layer as well so improve of stake you use the the randomness for the same purpose as proof of work which is basically to select a random block proposer in a fair fashion so by fairness we mean that your probability of being selected is

proportional to some sort of economic resource and you know you can go beyond that so there's the idea of sampling committees basically trying to sample the the total set of consensus participants which generally we call validators and only take a much smaller committee which is representative of the wider pool and assign them a specific task and this is how you get scalability with shardene for example so in the traditional model you know if you're in 1.0 every node will go through all the blocks and have to process everything which is unscalable and in the 2.0 model

what we're going to do is we want to find a smart way of basically splitting up the total validator pool into so-called committees and assigning one committee per shard so that you don't have this wasteful redundancy instead you're kind of cleverly spreading your security while spreading the validators across the shards and the idea that is that if you have enough honest validators in the global pool then this this property will be preserved in the committee's because the committee's are meant to be statistically representative of the global pool and in order to have this statistical representation you need to have a randomness and and this is where

the the randomness beacon comes in it's the seed to a so a called shuffling function which will allow you to sample committees so from what you've just said as I understand it randomness is used in POS for choosing the validator building a committee and potentially building a committee on the shards or did I misunderstand that a committee of validators for each shard yeah okay is there any other uses for randomness in proof of stake so um at the consensus layer no that's the main application just somehow sampling validators and it could be just a single validator could be a whole committee of validators but one thing that is totally

possible is to expose the randomness that it generated at the consensus layer to the application layer and then that application layer can use it for all sorts of things so the most obvious application is you know gambling and lotteries and games but you can do other things so one thing that you can do is that you can start building proof of stake systems on top of proof of stake systems so for example if you like file coin and that's a nice idea of being able to have decentralized storage and you want to recreate such a system on top of a theorem 2.0 well having an unbiased ball source of randomness will be a huge help to do that but it it's

also used for for it could be used for other things at the application layer so one kind of use case which fits in quite well with this podcast is in the context of stocks so stocks are quite large zero-knowledge crews and one of the reasons that large is because you have the so called fiat jamia heuristic where you try and reduce an interactive game which is like challenge response to something non interactive and you kind of derive randomness from the interactions between the Challenger and the responder what

this allows you to do as an attacker is basically try and simulate locally on your computer tons and tons of these interactions until you find one interaction which kind of breaks which which creates a proof which is wrong and so in order to counterbalance this you kind of have to increase various security parameters and so you have larger stocks if you don't rely on the theory heuristic and instead you you have a zero knowledge proof which is more interactive and kind of at each round you resample fresh on bias of all randomness then you're able to reduce the size of these stocks you know maybe reduce them by a factor of four

something like that so then yeah the the Stork would essentially use that on chain randomness generated by the beacon chain all right so someone who wants to kind of prove a statement would get a challenge which depends on the randomness and then the proof would kind of be broken down in time oh and in much smaller chunk where the aggregate communication is smaller than if they were to do it in one go in a non interactive way but because they're interacting with a block chain which is kind of this trustless entity which doesn't really is not a traditional counterparty you can still

think of it as a as a non interactive a large proof in a way it just struck me that having proper on chain randomness and I mean that this isn't necessarily tight footwork or stake or anything but if you have a huge lottery on chain and someone loses a ton of money they're basically incentivized to fork the chain before that happened and like they could put up if it was a proof of stake system they could put up a lot of stake previous to the loss and they just continue there it's interesting we should explain how this lottery thing

works because we didn't really I think we've mentioned it in previous episodes but as I understand it the idea here is that if you could predict what like a block number coming up is or if you see a block number before it happens or something you could place the bet on that number happening and then make the money from it well you could do that too I mean that's that's the simplest possible attack like a miner can place transaction they could front run the lottery basically by entering into the lottery in the same block that they are winning it yeah but that's like an incredibly dumb design of a lottery and like if the smart contract author just shouldn't be doing that so what do you what do you mean then what what kind of

attack is a lottery you disguise so basically if they place a bet then they if the source of randomness is manipulable they might not be able to directly say like I'm definitely gonna win here and place a bet but over time they are more likely to win than lose so in a repeated game they'll just keep losing keep losing and winning and then over time they'll win more than they lose because they have some influence over the randomness right so there's kind of two attacks on on on lotteries one is where your your source of randomness is manipulable because you're using for example a commit reveal scheme

you know you really shouldn't be doing that because even in that hacker with you know a tiny bit of influence let's say point 1 bit only 10% of the time they'll be able to kind of reroll to die that's very very bad because basically what that gives them is an edge so if they have a point 1 bit then that gives them like a 5% edge if they pass up a to an equal amount to what is currently in the pot so if there's if you have a 1 million dollar lottery and they possibly it with another million dollars well normally they should you know when you know half of the time but instead though they'll have an edge over everyone else

and so they'll be able to to basically steal from from the general public the other attack that Fredrik mentioned is kind of interesting which is well why don't you fork the the blockchain and indeed if you have a blockchain where the economic value is just much larger than the economic value backing the kind of the finality of the chain then that could happen I mean so one of the things that you want to try and do is avoid soft finality so you want to try and make sure that all the all the bets kind of have been placed and some sort of initial seed has been committed and

then after after this kind of initial phase there's a very strong finalization process and then after that you will try and deterministically extract the randomness from this seed which has been finalized at some point in the future in such a way that no one could could have predicted what this extracted random number is and so there's typically two ways of doing it one is using the the NPC approach where you basically have this secret sharing protocol where you split the secret and then what you do is that you commit to the shares that have been split on chain and then you have this finalization process which has much

more economic value at stake than the lottery and then after the finalization process you recombine the shares and then you get the random number and then the other approach is basically whereby you have a so-called vdf verifiable delay function and the idea there is that using computation using computation that is inherently sequential you can have a function which basically takes an input and produces a unique output so it's unique and deterministic but the process of computing this function takes a minimum guaranteed amount of time so let's say at least one hour and so what

you can do is you can commit to the input of that function run through the finality process within an hour and then go through this process of extracting the output of the vdf you're guaranteed that anyone who tried to extract the output of the vdf will only be able to do it after the finalization process and hence they cannot bias the input that happen pre feminization let's get back to vrf some ETF's in a bit because i'm i want to sort of tease that a bit talk about it a little bit but i think we should do a whole episode on that i think that and those

are super interesting topics alright now i wanted to get back a little bit to this idea of a beacon chain so we've mentioned it a couple of times already I think this is a term that is sort of evolved over time and I'm not actually sure I think in aetherium the beacon chain is sort of the same as the anything into the same is the kind of relay chain or whatever you want to call it but I've seen designs before of like a beacon chain being a completely separate thing that it's only purpose is to output random numbers it's just like a stream of random numbers that you can subscribe to over the network not really a block chain at all so how do you think

about beacon chain like what what's the definition to you and what's its purpose yeah so it's a bit of you know accident of research that happens so when I was studying randomness as you say I found all these so called randomness beacons which are you can think of as pure sources of randomness and I wanted to try and extend that concept and have a chain whose you know main purpose was to generate random numbers and I coined the term beacon chain and then people just started using the term for something other than what I intended they started using it for not just the randomness but

also the the other stuff the reason is that you can't really have a pure randomness beacon and a decentralized context you need to have at the minimum some sort of registry of a decent proof of state concept some sort of registry of who will be participating in the randomness beacon so you need to have a mechanism to maintain that registry and then you need to have you know a deposit mechanism so that you can register as a validator I need to have mechanism to exit and then you know you might have rewards to incentivize participation in the random speakin and then you start adding all these things and at the end of the day you start recreating what what I think would be a better term system chain so it's

basically the core shared infrastructure for all the shards which provide various system services and and and those services are randomness finality maintenance of the set of validators processing rewards and penalties including slashing conditions and all of that stuff so your your original idea was to try to have a random begin that sort of sat alone yes so additionally we were thinking of having a so-called shopping manager contract so a lot of the heavy lifting at the system level was done in a contract within if you're in 1.0 but you know even though that

worked great for things like like deposits and maintaining a validated registry it wasn't adequate for the purposes of randomness so I was thinking of basically building this external thing called the randomness beacon whose main purpose was randomness and then this whole thing snowballed and then we eventually realized that we want to put nothing in 1.0 we only kind of start with a clean slate and and and put all the system stuff in this external chain and I think the term is somewhat unfortunate because it gets cryptographers really confused because they're used to randomness beacons and they don't understand but the general public is fine you know they just think of some sort of lighthouse you know that

points the way forward for the rest of the system and that's perfectly fine I'm still curious they're like when you thought I understand throw this random randomness beacon I understand that it evolved and now you've referred to the beacon chain but if the beacon chain just eats 2.0 like where does it live somewhere or is it the entire thing is a beacon chain now now we kind of have three types of chains so we have kind of so-called legacy affirm 1.0 chain which what right now and then we have this totally separate parallel universe which is also a blockchain which is the beacon chain and that's one of the components of the wider FM 2.0 system and the wider film

2.0 system in addition to the beacon chain has shards specifically a thousand 24 shots and the shards you can think of them as user user level block chains like application level block chain so it has the user transactions has a user state it has you know user land blocks but the beacon chain is all about system stuff so it has system transactions system states system blocks and it it sounds like it goes way past randomness now yes everything and anything that is relevant to support the shot so you can kind of think of it a little bit like an operating system so on on your computer you have an operating system and then

you have applications that are built on top of the operating system and the operating system will provide basic functionality so you can I don't know read from memory right to memory and you know one of the basic functionalities that is exposed by the operating system is randomness and so and in a way we're just replicating this thing in a decentralized context is that is that so I how is like going back to just them the more general definition of randomness in computing I'm just curious if there's any examples you could give of how randomness actually exists in sort of the the usual OS so I think the the usual OS will basically have some

sort of API to various external sources in its environment so it might read the temperature it might look at the clock and you know go down to the nanosecond it might read user input from the mouse or from the keyboard or we might use during a random noise from the microphone and then it will combine all these things into a very easy-to-use API for the application layer and you know at the application that you use randomness all the time so I know if you're doing some sort of simulation then you would use randomness I say so this is like but it's still like feeding a random number into something else that would use that

randomness I mean the operating system well it depends on which operating system you're talking about but like the most operating systems have some sort of built-in way to get randomness like your developer doesn't really need to care they just call this random function and it gives them around a number they don't know where it comes from and it's the operating system level that does all be interacting with the hardware of like all the things that Justin we're talking about too that that's essentially like in your initial definition you said like the computer reads from its environments that's what it is it's reading temperature clocks noise in general in the system to try to generate a random number so that's so I mean in the

analogy like the system chain you know creates this randomness it is the source of this randomness and the the user level change they don't really need to care where it's coming from it's just you know I have a random number right here I guess but the question I had a little bit it was in the more traditional OS is there any need for using the random like randomness itself in its in its processing in its all right word maybe it's wrong here processing is probably not right but it's like in its own operation does it use randomness the way that the beacon chain would still be using its own randomness to define the validators mhm so you're asking does the OS use its own randomness for various things and all

most of them may the answer is yes and the reason is that when you think of it very abstractly in terms of you know algorithms then there's two classes of algorithms is like deterministic algorithms and then there's like randomized algorithms and it turns out that the second one is strictly greater than the other one like strictly more powerful and so you know you might have a sorting algorithm which is randomized or you might have a caching algorithm which is randomized you know you do read them in fictions or something like that and it turns out that if you want to try and smooth things out or try and avoid having any particular bias which may

come in with deterministic algorithms you can use randomness to have a superior algorithm for the job so I want to go back actually to a thing that you mentioned you were talking about these three ways of generating randomness in a sort of proof of stake environment one is a typical commit reveal the other one is threshold cryptography and the last one you mentioned was commit reveal extract this is actually something that's new to me and I'm not exactly sure how that works maybe you can explain that a little bit sure so the idea is to take a commit reveal scheme which is great on all properties except one which is that it's

manipulable and try and upgrade it somehow by adding this this extraction step to try and fix the manipulation aspect the idea there is that you want to try and use a so called vdf verifiable delay function to add a minimum guaranteed amount of time to extract a number which looks random given any input so you give it any input and it will take some amount of time let's say in one hour and after one hour you have a number which looks totally random if you make sure that the reveal

phase lasts less than the delay so in our case the reveal phase lasts less than one hour then any kind of bias that you want to try and introduce in the reveal phase by not revealing well that is not going to be meaningful in the sense that you will be biasing the randomness in the blind so you will affect what the final random number will be but you won't be able to affect it in a meaningful way in the sense that you won't be able to know if you're affecting it in a positive or negative way to you it would you'll just be resampling a new number so you can kind of think of it maybe as

a big jar with you know lots of numbers that are kind of written on pieces of paper and piece of paper are folded and you kind of you you put your hand in and then you mix and then you say okay I'm going to pick this piece of this this number but you don't open it and then you take it out and say oh actually no I've changed my mind I'm going to pick another one so you put you the number back in and then you pick another one so here you have changed the final outcome but it was in the blind so you have no way of actually meaningfully manipulating the final outcome I wondered does this a little bit to do with the idea of like mixing random numbers like are you combining MIT rent

are you combining a random number with another random number here yes yeah as I've understood it if you mix a sort of highly unpredictable number random number with a not quite a secure number you end up with the the resulting number will have a highly unpredictable value so it will actually take the more random of the two and will have that quality that's exactly right basically if you combine a bunch of random numbers it's not a weakest link properly it's a strongest link property yeah that's exactly right and so that's why the commit reveal schemes they kind of have this minority honesty assumption

you know you just need one single person in the group of participants to be honest and they will take an unpredictable number and so the combined mix will be totally unpredictable and so the idea of commit reveal scheme is that you have this commit reveal phase which generates this mix which is on unpredictable and then you have this extraction phase on top of it to make it on manipulable in addition to being unpredictable so going back to the lottery example I guess the commit reveal extract everyone commit you have the last guy in a normal in the normal model he can say if I commit my value to be this the outcome

will be that he has the the total privilege of seeing what the outcome of all of this will be and so can decide if he wants to put in his value or not but if you feed all of this into a vdf and then the outcome of the lottery is decided before the outcome of the vdf then basically it doesn't know until the outcome so you know it like you said he doesn't know if he should commit his thing or not because he doesn't know what it is until this vdf is done yeah that's exactly right so in the in the reveal phase which is where the bias comes in because I mean one thing that maybe I should make explicit is that the

commits reveal scheme only allows you to reveal the single number that you've committed to you have no other option then I've revealing what you've committed to or not revealing at all so you have so so-called one bit of attack surface when you are in attacker and let's say you control some fraction of the participants in the commission of your scheme you'll have some summit some amount of attack surface but this this amount of attack surface will be tiny relative to the unpredictable entropy which is added by the one single on esposa plant and then this this unpredictable entropy will overwhelm the

extraction process so it's basically impossible to to brute force the extraction process you have to go through this slow computation phase so we have these three models commit reveal communal extract and threshold what's the attack model for these three like if I wanted to break the randomness and actually be able to decide what the random number is what do I have to do for commit reveal you have a minority honesty assumption so you just need one single person's participate and you'll have this key property which is ability now what happens is that there's kind of a spectrum for manipulation so

if you have everyone is honest then you know no one will try and manipulate and you'll have a great random number but the more attackers you have so basically the lower your honesty assumption is the the greater the surface for for manipulation in in the threshold cryptography it's much more clear-cut instead of having this gradient the spectrum it either works perfectly always completely broken in the context of working perfectly basically you want to make sure that you have sufficiently many people who are honest and online at any given point in time if you don't have this the threshold then there's two

ways in which the randomness beacon fails in very dramatic ways way number one is you don't reach the threshold at all and so that's a lightness issue and you can't even generate the next random number and that's you know really problematic for example on the proof of stake system because you need to decide who's going to be the next person to create a block and if you don't have that then your beacon chain your your blockchain will stall and that's for example what definitey has right now and there's no way to get out of this right like you can't reshuffle anything because you you don't have a random number to reshuffle like you what do you

do can you just have to wait until enough people come online so you have various options you can you know just wait it out and see if people come back online another option would be to just fork at the social layer the blockchain and try and restart with some new parameters with new set of validators another option is to try and have some sort of hybrid model where you know in the default case you will use you know threshold cryptography but your blockchain has stalled for a whole hour you can try and have a fallback mechanism with commit reveal the problem way

these fallback mechanisms is that suddenly they introduce manipulation and the reason is that what you can do is you can if you're an attacker that's large enough you you will have kind of sufficient votes to go above the threshold well if you don't pass the bait you go below the threshold and so you can privately know what the next random number is and if you don't like it then you just wait it out and trigger the fallback mechanism and that will give you basically another chance at getting another random number yeah so it doesn't just introduce mini appealability of the fallback mechanism like if the fallback mechanism is commit

real then we have manipulable 'ti there but also like at the higher level where you can choose if you want to fall back or not which is interesting exactly and even forking out at the social layer that's also fallback mechanism and also a surface of manipulation is there any difference in the attack surface between the commit reveal and commit reveal extract or is it the same so the commit reveal extract is kind of perfect in the sense that it's totally unpredictable totally unknown to loop on manipulable and has shrunk liveness in the context of

minority honesty assumption but it depends on PDFs which is a complicated topic and we don't really know yet if it has its own attack vectors or not right I mean Vidya has definitely have their own attack vectors and there's a bunch of standard assumptions that I made cryptographic assumptions but there's also new assumptions which relate to hardware which can you can which are that that's the part I'm alluding to like if you design of EDF as an ASIC is your ASIC optimal or not and that sort of introduces some some questions but maybe we can like let's we're at time so

we need to wrap up unfortunately but I wanna I want to do an episode in the future with you Justin on VRS VDS all these things maybe we can just give a quick teaser and primer like what is a vdf high level what is a vrf and are they related or not so vrf is actually easy and we've we've kind of used them in the committe reveal scheme so basically you want to have a way of generating randomness locally and then presenting it to the blockchain in a verifiable fashion and so one very easy way to do that is you know first you commit to secret with a hash and

then you reveal the hash and anyone can verify that the secret that you've revealed matches the hash and you can use a valid function so if for example one which we use in the forum 2.0 is the idea of using a BLS signature so the commitment will be registering a public key and then the reveal will be a signature from that public key and so once you reveal a signature anyone can verify that the signature matches the public key so a PRF is nothing fancy it's you know we use them all the time

what's much more interesting and is the the vdf because that's a new cryptographic primitive and the idea there is to try and have a in a totally trust list and decentralized fashion have a notion of minimum guaranteed delay and by delay I mean physical time delay so you can you have a guarantee that you know a whole hour has passed and not only has a whole hour pass but also you have a mechanism of taking an input and producing an output at the end of this one hour where the output is is

unique and deterministic and looks like a random number and the way that these VD FS are implemented in the real world is using hardware so you have this notion of sequential computation where the every step of the computation depends on the previous step and so you can't paralyze things you have to do them one after the other and so there's no way of kind of cheating the system with you know data centers full of hardware and the new kind of exotic assumption that you have with the hardware is that an attacker is not much faster not too much faster than you are

so for example not more than a hundred times faster than you are and so the exciting thing that we're doing a different foundation in collaboration with all sorts of other blockchain projects is to try and build state-of-the-art hardware that will do this equation computation and have extremely high confidence that it's effectively impossible for an attacker to build equivalent hardware which is a hundred times faster than what we're going to build so listen thanks so much for coming on the podcast Justin and helping us to explore randomness yeah no worries uh it

was fun thank you for having me thank you very much and to our listeners thanks for listening thanks for listening [Music]