RADON: A Domain-Specific Language for Oracles

Transcript

I'm good afternoon thanks for coming to this presentation in which we will introduce the work we have done in this last year's in this crazy idea of a domain-specific language for articles what's radon in and so on the idea of this presentation is understanding what decentralized oracle Network are is and get you know Rayden why it looks like it does right now you're learning how witness as a decentralized oracle network uses Radin see in real life use cases for for this introducing sheikha which is a visual

editor for Radin and basically looking at the future of Oracle's decentralized oracle networks and radon itself so decentralized Oracle's wide limit matter why are they important right now in the blockchain space especially in this area so you have probably heard a lot about the Oracle problem it's 2020 I don't need to explain it anymore I think that you know it's basically marketing mumbo jumbo there's no formal definition for what the Oracle problem is nor for what an Oracle is at all so everything you

need to know for there for the scope of this talk is that smart contracts are deterministic they need to be deterministic because they are executed in all the nodes in the network in the theorem network at the same time and the result of the transformations they apply on the global world state of the etherium virtual machine as a world computer need to be the same yet the internet and data that you find in api's in any online data source it's not deterministic so the way to deal with that in the case of

etherium they made the decision to build a huge firewall between the smart contracts and the outside world that makes sense because you can in that way you're not polluting contracts with indetermination but then if you want to introduce data from the outside you need to do some sort of centralized data input you ask the creator of this Marc contract or someone that is allowed to do so I will need to introduce data which obviously breaks the tamper resistance and the censorship resistance properties of smart contracts which are the properties why we are using a smart

contract in the same place and why we are here after all so the question is how do you build determinism without introducing centralization the solution from our standpoint it's a decentralized oracle network decentralized standing for something that is a temper and sensor resistant and completely permissionless unless sorry at least in the same degree that Assyrian itself is or Bitcoin itself is an Oracle by Oracle we mean that is some sort of entity something someone acting as an information relay bringing information

into the contracts and also as a determiner or the designer let's say something that reduces the indetermination and leaves you with a single data point that you can fit into your contract securely and then when we're talking about a network is a pool of workers full of nodes that are a definitely run and from which we will draft we will select we will sample Oracle committees in an empiric table and secret way ok so that their main idea here is that the centralized network

are all about splitting power and so that no one can tamper with the data that we are fitting into the contracts and that we don't have a single point of failure which is weird what we're trying to avoid with with using blockchain technology so why why are we talking about that DSL at domain-specific languages for Oracle's specifically for for decentralized oracle networks basically because when you require determinism when that's a need for you you necessarily need to deal with things that need to be processed in explicit

way so that they're there there needs to be only one interpretation on how to deal with the data okay so human languages of course will not work you cannot say like write a prairie that says what's the weather like today in Paris that's perfect for Cortana or Alex or whatever but for an Oracle network that's simply not okay because it's open to contestation people can argue on what's the meaning of weather what's the meaning of today because at what time does today start or finish what's fair Paris is some neighborhood Paris or is it not

there are many open questions so most programming languages things like high-level programming languages and so on the problem with them is that they offer too broad semantics they they give they are great for expressing other kind of software any other kind of software but for very simple data processing features they are just too complex and they introduce a lot of challenges when it comes to the flow control instructions and so on so we need something that is totally focused on this domain which is data transformation and aggregation and something that is

more alive today map use paradigm to which many people will be familiar with coming from the area of big data and so on so particularly in our case we want it to be abstract enough so that it was not tied to a specific syntax in terms of coding so that we can write libraries for any language like wrappers or bindings and that and that libraries need to compile down to what we are talking about which is radon itself as a bytecode at something as something more I like to to

assembly for example we want it to be Turing and complete not no flow control at all no ifs no for no loops nothing like that we don't need that we just need to take data from somewhere process it and in some way I will get it in some way and deliver then the like the nicest way to do that is oriented to data flows basically using tape acid or point free syntaxes for the people that are more into parami language theory and you have a very small surface to provide only features that are focused in the domain

of transforming data one thing that was very important when you're outsourcing data processing or data retrieval and aggregation to someone else's computers you necessarily need to be able to make static analysis on the resources that execution will require and on whether their queries are well-formed let's say if they if they comply with some sort of a schema those two things are are like very paramount features that allow on one side

basically making pricing models for for this execution and in the other Garrett having some security guarantees so raid on itself is something that from our point of view is fulfilling most of those prompts and it was introduces in introduced in 2017 in the original Wynette white paper and as something that complies with all those requirements and radon itself the name apart from from a gas is it simply stands for retrieval abrogation and determination of technology so radon can be seen as a few different things you can you can think of it in different

ways or you can think of it as operating in diff at different levels the first one is that it's an abstract syntax or an intermediate representation and intermediate language like the one you see there which is just like using English words for expressing what it's doing just like assembly does for example and in the idea for radon is that you can express things in form of scripts whereas scripts are like basically pipe lines of course integral is either at one operator as you see here

or an operator with at Apple like at Apple of operate one operator and many arguments depending on what what operator you are using obviously then the LD remark here is that the return type of a script is that of this last call simply because all operators are applied sequentially so every operator just can operate on the output of the previous one it's like a pipeline like Linux pipelines like UNIX pipelines then it has an implicit type system which is strict if you want to change from one data type to another you'd need to

explicitly state so it's independent it's not using the type system of any other language although it's trying to basically create some uniformity between the origin of the data and the destination of the data in this case the earning of the data if it's ap is it use it's using basically Jason the the JavaScript type system and if you are delivering it into solidity you need to deal with the type shift of the type system of solidity so the types are

typical ones bad idea is that if you start with a with an with a particular data type and you apply an operator for which you know their output that you can basically reconstruct the whole chain of types and that you can do that statically before even having to deploy there the request or all right Rayden as a bytecode is what actually goes into wit net as a decentralized oracle network so radon is actually their bytecode for their for the Wynette Virtual mat machine let's say every operator has its own operator code and so on and it uses sibour encoding which

is pretty standard proficient really nice to work with so as I said before you can only operate on the output of the previous operator so in other terms it's like it was interpreted by one item death stack machine it means that you cannot have you don't have a proper stack or a properly you cannot put data on hold for use in later you can only operate on what you got from the previous operator then the idea is that you can create libraries that compile down to radon or that allows you to

write radon without having to deal with a bytecode or with sivir and anything at that and take for example with the winner requests jes library which is actually quite funny because it's doing basically it's meta programming that feels like if you were programming this is written using the JavaScript yourself and if you look at it it actually looks like if if it if you were actually running this code as if you are creating an object and parse in something and like actually operating in the data in real time but you are actually not by

running this what you are doing is actually mutating an object that when serialized will give you the back row okay so you're basically writing JavaScript that writes rhythm for you so Thank You Ellen our house justice playing what credo Nissan how it works but sorry bear with us well as I said Alan has explained what trade on is and how it works now we can say we can see how to use radon for our specific use case in in with it

basically when it has a request lifecycle that we need to review to to explain whole radon fits here basically with net is a decentralized oracle network and it has a pool of nodes that we call witnesses and those nodes are randomly selected we are critter affix sortition Albert a mechanism that usually uses peripheral random functions well the first step here is the data request step where the random selected nodes will be from a specific task then with a data they will commit a secret

commitment and they will reveal a value to avoid notes to copy each other finally they will go through the Italia's totally states where they will run a consensus algorithm and we will get data to to be used that can be using another contract we have using we use radon for one of the of these stages and we will use different or different many sources many sources scripts we will use sorry we will use many sources scripts to avoid a single point of failure in

the internet work and then we will use just an aggravation function and then we will use a terrifying some that are pretty similar than the aggravation ones but they are from bye-bye notes by mining nodes and the the requests are highly tuned Tony we can film a lot a request we can set or specify the amount of witnesses that are going to run out run or request or set the fees and the rewards that are going to be given to them to the participants of the network and we can also schedule a time stamp to specify when other requests can be can

be resolved and with that and with that we can see a full example of how what is the full cycle of a data request if we for instance if we will choose a to choose it if we will choose to to the number of witnesses they will be randomly selected and each of those nodes will run a they will fetch the data from the different URLs that we are specify then they will go through the source script that the ante input of the operational state of aggregation

function is the output of the other different source scripts finally finally for the witness notes they will run they will publish the secret commitment and then they will reveal the value finally the demeanor the binder notes will take the value given by the by the witnesses they will merge these data running the the tally script and they will they will publish a into a block basically here we are converting on the terminus and determining that data into the turbulent data that can be using in

a smart contract and now the question is how can we use a radon in other plug chains not just the wit net one who can be used for instance in the individual Network we have a smart contract that needs data the only thing that has to know about wit net is they have to create a data request and get the right coat of token of this issue of this request send it to the with net which interface that is a a smart contract and send also the the properties of this data request we can imagine they will

not which interface like a a postal notifier where we can a post that request and they will be they will be broadcast to the Windham Network these requests will be taken by the Brits notes the Brits notes we'll get the data requests and they will put it in in the wit net blockchain when all the magics all the magic will happen inside with net and then the resume that when the result is ready the originals will be listening and they will take the value and they will put it again in the interweb which interface there are there is also another

component that is called they would not block relay and the would not block relay it's well in there the function of the of a would not block relay it's validated the transaction that happen in the witness network within the DTM one basically he the winner Prague relay provides tamper uncensored resistant properties and my colleagues have given a talk where they explain in their whole who they will never block really works with that we can start writing or our core aware and data requests and we will start with with the freedom sources we

have to take into account that the witness sources are contained a URL where their data point that we are going to fetch the data and they also contain a trace on the script they are the present sources are run by the in the witness nodes and name and we have to choose wisely or all sources in that is really important because if we put garbage or if we select but sources we will put data or carpets data in in witness and we'd not we get this carpet set data out or another thing to take into account is all the sources scripts

will start with a string and then the result will be aggregated a will be an array that will be aggregated in in the next script or an expansion and also important that all the we need sources have to return the same type to be to be able to operate with them here we have an example of a social script we define with create a source instance in a source instance and we we pass it the the URL of open stamp and we explicitly explicit explicit say with that we are going to pass the string given as a

solution and this situation is a map and then we pick the last key that will get a float or we also can specify another wind source that we will also get a upper we will put an operator tap in the off position map then we will apply get map to get the Bitcoin price index and get a another map that we finally will get a float getting the right flow key the next function or the next component that we can write in in tradin he's an operator an aggregation function a

derivation function is built with one just one tree introducer and we can choose or write any filters apply any filters that we want to to that function in the rhythm erasin functions we will also be run by by the witness nodes and the entry of the aeration function is there a break is the result or the output of the retrieval States all the types of they have to be the same and the the output will be committed and later will be revealed here we have an example and for this case we create an instance of our witness activator and the filter stop we will only specify a

filter that will say ok filter by the standard deviation and then reduce all the ära for a mini bridge and finally we have the tally functions that there are also one purpose request and it also it's pretty similar the tally functions are pretty similar than the erasing world and they are also contained one producer on any amount of filters but they are run by the minor nodes instead of witness notes finally they are the final output of a tally function is data that can be using in the smart contract I said before and

the the final result of the tally functions is writing into integral with net blocks and as I said there are pretty similar than the original functions there here we choose a standard variation filter and then we would reduce the variability by the mean average but if we get an interval number we could choose mode or any operator that applies to in teller and something really remarkable about the tally tally functions is that day when we cried tally tally functions we

are designing or creating in its design by incent in the design incentives or incentive design we have to take into account that when we write Italia functions we are set in the rules for a selling point game where all the notes were that are not coordinated bits with them between them will try to behave to maximize their their profit and the best way to maximize the profit is just run the Italia script something something important to say it's that wait the three the tally functions that the notes that are not current that are not in consensus with

the other witnesses will be a last in the tally by the tie script if we design Italia scripts or tally functions could be tricky and we call if we choose a bad operators to to add to a pipeline we cook a hot party incentive to the to the notes so because of that we have chosen not have the ability or not allow users to add some dangerous operators that like get take or for sure because we could be given too much power to 13

nodes of the network but this is as I said this is extremely easy to to get with static analysis and we can't apply this operators with a winner present a library so coming down to earth so basically at the how it looks like when you're an Assyrian developer and you want to start by using waiting it for getting data into view into your contracts in a

totally trust less and permissionless way it's as easy as it can get which is basically having this you're probably familiar with truffle it has this functionality of unboxing things which is like having templates for solidity projects and and and and things like that and we have these truffle box that when you unbox it and compile it and you put your requests inside reading written news in this JavaScript library that is already I imported into the into the truffle box when you compile it what you do is what you will generate is something like this for example this is

an example that we built for for getting the result of a football match and your JavaScript will be compiled down into radon then she realized using their they Wynette binary compatible ike that and and they and the truffle box creates a solid a solidity contract around it wrapping it so that you can simply say okay that's ur i I don't I don't actually care how it looks like important thing is that I can create a contract that simply imports that contract the solidity comfort and you instantiate a request object which is a clarity that your contract automatically gets by simply saying that your contract

is using whether by an inheritance and you say new football request for example and we are instantiating the request that we just created using JavaScript then compiled into solidity when we do that we just we already have this frequents object-- we can reuse requests that's why it is like a two steps thing and and you can just say when post request take the back of the request set the fees that you are willing to pay for that request to be relayed in to intuit net and you get the request ID at any later time when the request is resolved and we're net you have you just simply

need to to call to win it read result with the request ID and you get a result object which is like a algebraic data data type so it can be anything but you already know what what it is because you know the data type so used you can cast it into the into the type that you know that it is and if for some reason something failed because maybe you requested only one source which is a silly and stupid thing to do but let's imagine you request one source and the source is down okay it me or it gives you a 404 HTTP area or whatever instead of getting a valid result you will get an error that you can handle and you

will get also an error message that you can lock using the the emit thing we off solidity and so on so one other thing that the traffic box does for you so that you don't have to actually deal with all the winner tooling is writing the migrations for you from what from your contracts for for any contract that you put in the contracts folder of the truffle box if you do it will automatically write a migration and if it's using with net or basically important with net or receiving anything related with win it as part of the constructor it will put it there if you

are if it detects that your constructor is receiving and an address and an integer it will also put placeholder for those arguments for example in this case it has the it has detected that is it's it's receiving the address of the wbi of the winner bridge interface contract so it automatically put puts it that so that you don't I need to actually do any any sort of DevOps thing for getting your contract to work with women this is actually super nice so one thing that we are especially excited about is something that Thomas

has been especially working working on is sheikha which is a visual editor for for rhythm I will be by your side so that we don't need to pass there yeah man well basically we are excited to announce or to present you say cut that is a visual editor to compose that request and you can easily create the data request or import them our choose your data regress that best fits of your needs that will be away away available in the windows marketplace you can download template use it in the editor and export it to to use it in in the

smart contract to deploy it to the window network and then use the value given in the inasmuch contract we will present in now sheikha and it will be available soon yeah sorry that it's not a live demo because we are using Linux and we the room doesn't allow using Linux because it uses some crazy belgian technology that we were not aware of and but yeah chica is basically there they with net wallet or the desktop with net wallet and it will be soon available with with this nice feature on the future of radon and Oracle's and win it

and so on we right now we net supports using HTTP or HTTPS sources only bad we we're thinking on supporting also other protocols like ipfs ficon and so on so that you can actually pull data from ipfs for example from your solidity contracts and operate on that and then do the the way back to also to ipfs to restore things there or for example listening to web 3 events that's something that you actually can do right out through web like explorers api's which is basically giving etherion the capability of reflection like

observing itself so that a contract can listen to other contracts events so that they can subscribe to the other contract events they can monitor the the state of other contracts and so on that's that's something that we consider that is super useful for for some advanced as a theorem use cases and also we could think of at some point supporting authenticated API is but that's something that it's quite challenging indeed and we prefer not to tackle just yet then one thing that we're working on is optimizing how epic code is serialized and compressed simply because

this nice feature of Sieber in which small numbers are more efficiently codified a we can we can leverage that and map enough set aware up codes to to a smaller ones so that you can save up like how up to a 50% of the baton as in in the example and maybe one thing we can do also is like trying to detach Raiden from Wynette because okay we we Raiden is the best thing we could him with at this point for or the use case that we had at

the when designed that they were net protocol but we think that it could be also useful for other domains like a for instead other Oracle's obviously but also for scrappers which is which is something similar at the end of the day a decent provides Oracle network is I like to button it that you can hire for scrapping things basically and also ETL tools which are tools that are used in in banking and many other industries for migrating data from the from one database into another and things like that and with this generalization we could come maybe in the future with higher order raid on in which you can bring

your own operators because right now you are the Braden only supports a closed set of data types and a closed set of operators but that's something that could be perfectly thought of so just to avoid the a classical question that we get often remaining okay so actually with net is on test net right now we know this as you know it's a separate chain that is sidechain to aetherium through this bridging scheme that that Thomas explained so it's right now sidechain to drink by and

girly you can actually use it from those two etherion test nets it's totally ready to play with it you have the truffle box is super nice we will also be creating other other tooling and support for for other developer tools other than truffle and right now all the solidity code and all the part also of the of the wind itself which is written in rust is undergoing audits and stress testing and so on and expected Maynard lounge is for four in a few months so around early next quarter nothing else for for today thanks a lot

feel free to to do reach out at any point with with any question suggestion that you may have and so on I don't know how we are in time that if we if we if something and someone has some that if we didn't explain something properly please please shut up I'm sorry sir she don't yeah thanks so much for the presentation I'm just curious in your data types I'm thinking if this is going to be useful

for financial use cases floating points are not going to be sufficient you're gonna need to have like big decimal kind of types is that something that you're thinking about supporting in a future version that's that's that's a great question and a great insight in indeed floats float floats floats are super tricky they're they're more or less a standardized as of today but the support for them is lacking sometimes we know we don't have floating in solidity so we are left with fixed precision numbers in the case of freedom how is it

implemented right now in the interpreter in rest is using f/64 so you have you you get idea of the precision that you have that may or not may be enough for some some use cases but that's that's something that's something that can be worked but then when you get it in solidity it's totally up to the user to to have it as like just multiply it by some power of 10 and get an integral that's one option that's the biggest thing that the solidity developers do right now from what I've seen and I bet other the other way is that you can

actually in with it with a current tooling and the current using with netlibrary integration there's there's like a data structures for for floats that preserves all the precision that comes from that actually comes from rest and you can deal with that in a fine-grained manner but it's not that convenient to work with obviously right I guess my question was more like floats themselves in 64 bits ones still remove some precision so you might want precise numbers so like actual decimal numbers not floating points if you're going to do if I had to do that today and I required full guarantee on that I will totally go with with the bytes and use a

serialization method that I know for sure that will preserve all the precision yeah sorry any other questions thanks for your question do you think the the financial cases they are going to use different kind of Oracle's only one that is right how do you think he multiple data sources yes or multiple even yeah that's interesting yeah that's that's something that we that we are actually like when in many cases when we when we talk to

users we try to make them realize that the that if you are using with and you're not try to win it in any way that you can you can use it in in parallel to other Oracle solutions more or less decentralized or with different security or likeness trade-offs and that's perfectly fine and that's even maybe the the smartest thing to do but then the the challenge comes when you when you your query multiple Oracle's first you need some way to query them through a common interface that's something that we're working on right now first through an organization that is called the alliance of decentralized

Oracle's where we are working together with teller with IX AG and with other products in the space for coming with that standard and we are drafting an EAP for for standardizing that and the other thing that that that that you need to deal with is obviously aggregation you need to do aggregation inside solidity so then you are you need to write aggregations using solidity so you're subject to the same challenges as we said before you are using a language that is maybe to reach for expressing that sorts of things and um there there there are many different examples for million icers and for things like that

they have again many open challenges many security issues that arise when for example when you use in the emelian that you are not actually stacking up the security of the multiple Oracle's you are putting them in parallel and in theory by compromising one you can manipulate or deviate the median by one position and if you compromise two of them you can actually decide the final result because you can use one for selecting what's the median and the other for reporting actually that value so it's basically if that's something that right now could

make a lot of sense in the long term I could argue that you're trying to replicate with net inside etherium in that something that it's defeating the point because you're basically they did one foundational idea behind with it is to bring these Oracle capabilities to theorem without being a burden to etherium without high in higher in gas prices and so on without having to compete with the transactions of the protocols themselves now so so that the Oracle is an external thing that doesn't compete with it with their with contracts of the users that which is which is important in this point and

that's why it's at site entry theorem I also have a quick question do you think one single source of data can ever be decentralized oracle for sure there's there's a clear point of failure there someone could could argue that that could be enough if the data if the API you are Korean is there is operated by the same entity as their country if the case mark on Turkey is being operated by but then what the what the heck why are they using a smart contract you're you're again trusting a single

entity so I'm not sure I'm not sure if there's an actual case for a single data source that's that's just too easy to break and the more if you are doing things that like some people are doing it some we are seeing some people do which is like operating for example open press fit basically having contracts that in that in their public estate they are publishing some data point that is queried through an Oracle or anything similar that's a problem because if you are a query in a single data source and then you have a contract that can be consumed by an unbounded amount of contracts the amount

of value that could be at the stake the amount of value that could be jeopardized if someone compromises the the data source is unbounded and if you cannot control that you cannot actually design what they make the security measures need to be for it for that mechanism and basically you are creating a big bounty for tampering with their data source okay do we have any other questions in the room anyone well thanks a lot guys thank you [Applause] you