#217 Andrew Trask: OpenMined – A Decentralised Artificial Intelligence Platform

Transcript

[Music] hello and welcome to Episode a destroyer which talks about the technologies projects and startup selling decentralization and the global blushing revolution my name is Brian Fabien clain and I'm a Hanoi today we are talking to Andrew Trask of open mind open mind is a very interesting project that uses tools from cryptography and blockchains in order to create decentralized AI that can use private data from different

people around the world to train AI models without compromising the privacy of the people contributing the data Andrew welcome to the show thanks great to be here so before we start there's a bit about your background and how you came to create open mind yeah so before I moved here I'm a PhD student at the University of Oxford and before I was a PhD student I worked for a really exciting company based in the u.s. called digital reasoning and they they they specialized in on-prem enterprise deep learning and AI services and they still do and and one of the things I came to really appreciate then

was was how difficult it was to get access to private data and so during my time here is a PhD I got really involved in AI safety conversations and really how open mind got started was a blog post looking for for the right tools to build safe AI and to try to contribute to that conversation that's how I came across a multi-party computation homework encryption and there's a kind of a blog post out from last last March and then we opened up the github repositories in in I guess July of last year and we used to been writing code ever since so tell us Nick why why is cryptography like homomorphic encryption and multi-party computation relevant to safe AI and AI in general

yeah that's actually a great question and and for I think most of most of their existence the conversation around AI machine learning and and these cryptography tools that sort of lived in isolation I mean there's been a handful of workshops at like nips or ICML where there'll be a few papers that discuss private machine learning but it's it's still a really small really small field but look growing quickly but the reason that it's important is that that cryptography is all about protecting really valuable digital assets right and also limiting the use of various digital assets so in in this most simple case with just asymmetric encryption I have a data file I wanna be

able to put it somewhere and I want only me to be able to do stuff with it and only me to be able to access it but in the context of machine learning often you want to have something that's a little bit more nuanced you want to have the ability to just do training or just do prediction or you only want to be able to do prediction if a lot of people are participating or you have some sort of digital asset that you want to interact with but you don't wanna be able to interact with in arbitrary ways and and that's where where the conversation between kind of cryptography and machine learning really gets started so the private kind of private machine learning is a field is all about either I need to put a model in a place where people might try to steal it or I have data that I need to

learn things about without actually having a copy of and so the the blog posts that really kicked off open mind was was talking about homomorphic encryption models where I can take a model that's fully encrypted and I can give it to you and you can make it smarter but you don't have the ability to predict with it and so it's it's sort of like a one-way gate for sort of intelligence right and and there there are various there are various long-standing problems in this field so the biggest one being efficiency but but that's sort of what the conversation is all about right I want to have limited ability so that the people who are participating in AI machine learning are protected in various ways

so you mentioned be free a I safety now I so familiar with what that is probably many people are but for those who aren't and aren't so deep into machine learning can you define what that is yeah I'm gonna do the best that I can I I would not call myself primarily any our safety researcher so so if if any of the folks that I know that are really as if you're researchers they might they might correct me on this but I'm a our safety really deals with in particularly AGI safety deals with the problem of how how do we as individuals with if we're lucky IQ of 150

limit the abilities and the the values and the goals of something with an IQ of 15,000 and that's actually it's it's closely related with a lot of concepts funny enough in political science where you have you know a large group of population and you're trying to make it so that these few people at the top that have controlled a lot of resources act in the interest of the population and so that's kind of a most of the AI safety conversation is around that we're very close to the eye safety and that we we we try to innovate on like the tools that you would use to do that I think that the most precise term for what we work on is probably AI security in to the extent that we're trying to build really good boxes and really good tools that that allow you to do different

things and and kind of give the AI safety community more knobs that they can turn right so so in the case of narrow AI one of those knobs that we're trying to bring online into the into general knowledge and and ease-of-use is the ability to train machine learning models on data that you can't see and train machine metamodels on daily don't have access to and that's like it's a really good example of us building a new tool that has been valuable for the AI safety community in the narrow AI space and obviously would like to keep going into the general AI safety space as well yeah maybe maybe we can come back to that in a second but I just sort of would like to understand this big picture now I you know we kind of started talking

immediate about this idea of how open mind a little bit approaches these things but maybe to give a bit more context as well there a lot of news these days a lot of kind of discussion and discourse around the issue that you know a few companies like Google Facebook Amazon are acquiring like huge amounts of data and acquiring a huge amount of power right you also see political backlash against this and with in Europe for example with things like this right to be forgotten or like data protection laws and stuff like that so what do you see I kind of as the current state like where the

trajectory is and how does that fit into open mind where the rubber really meets the road it's a big hairy issue there's a lot of players involved there's there's a lot of of kind of conflicting ideas and and and probably the biggest theme of 2017 as regulators really getting their hands getting their hands around it there was there was big big research interest from a US Congress and Parliament here here in the UK as well as the European European Parliament obviously with GDP are I think that the way that I see it the general theme is that a new asset came into the marketplace right data and and with with tools like for big data

becoming widely accessible companies started collecting it and and this went largely unregulated like all new tools and now we're starting to learn a little bit about the externalities of how it should be protected you know governments tend to be a little bit more responsive than than and reactive and then proactive so I think what we've seen is that this incredibly valuable asset has been aggregated as you know sort of like maybe oil was when it was first you know back in the because it would be the you know around like Andrew Andrew Carnegie oil and steel and like all these kinds of things the wake when they really came online and and we're sort of rounding the corner just like they did back then to significantly more more regulation now as far as where open mind fits in

with that I've watched many of these regulatory talks and the the one tragic thing is that no one seems to be aware from none of the regulator's seem to be aware of the notion that you can train models without having access to data like the conversation almost always boils down to well we have this trade-off between innovation and between privacy how how do we know how much privacy to give up an innovation to receive and and just even the existence of these these tools it's just it's it's almost like it's totally it's just not reaching the public discourse and so a huge part of what we're trying to do in the open mind project is just almost let people know that this field private

machine learning exists that that many of the tools are are ready for practical use and we're building software to make that as accessible as possible so in the short run if if we're very successful as a community you know that that discourse will change because people will actually realize that you don't have this nasty trade-off between privacy and innovation with with kind of these new pieces of technology okay you know that that's I think that's very helpful and makes sense now maybe kind of long one last thing on that I don't fully can you elaborate a little bit on the connection between private AI and AI safety because you know we've explained right why

multi-party computation and and these things obviously have implications for how AI works today in this centralization of data and power but how does that relate to safe AI so the real difference between those two things is narrow AI versus general AI so so AI safety is mostly concerned with general AI meaning they're concerned with something having an exorbitant ly high IQ that you know threatens humanity right but the the privacy conversation is mostly about narrow AI so it's about you know these are the things that the industry has optimized optimized just for certain business use cases for that leverage private data and that may or

may not be also defending the interests of the people who own the private data right or so so they are slightly different but the the the overlap is primarily in in in the tools that are involved because the tools that you use to mitigate the trade-offs in in narrow AI we hope we'll have broad implications as the conversation around AI safety for general AI unfolds because it's also a matter of like what's tractable now and what's what's where we expect to be important in AGI in the future yeah maybe one one fault on that I mean I I don't know much about AI but when I was reading maybe some sort of sense which no away like a over the

world and stuff it seemed pretty obvious to me how blockchain could play a huge role in that in that you could have maybe some kind of you know reputation system managed on a chain right where there could be some kind of you know way to turn off access for these a eyes or like some kind of economic management layer right that kind of controls AI and put some sort of governance over where I see see that also has a direction in the long term to handle this AI safety problem yes but but maybe not in the way that maybe not in the way that is most

obvious from science fiction literature the the thing that blockchain brings to the table in terms of governance is not really any new there's nothing new unlike the theoretical governance side right like it's still you still have the same problems with with value alignment like how do you get these super smart things to have the same the same values as us which is the same as saying how do we have our political leaders have the same values as us and so even for the normal kind of political alignment conversation it's not like there aren't new themes what what blockchain really brings is potentially is tremendous amounts of liquidity and transparency to that conversation so so whereas right now we we vote on basically everything

for the whole year with this one very discrete event right which is when we vote for one political leader to take power and and you could think of you know there could be scenarios in the future for AGI where where you know you could have very discretized events for for how you how you make your say and try to have the the AI to be aligned to your own values the really think about blockchain is that you can make that a lot more nuanced right like because you can do lots of transactions really really quickly so so particularly with with AGI what I'm what I'm really excited about is is the ability that that instead of saying instead of saying up this AG I did something bad let's let's pass a bill

and like turn it off right you could say hey this AGI is is behaving you know it's its value is misaligned a little bit in this direction let's let's curb the resources on that side and give it a little bit more reward on this side and and sort of direct it towards towards where we want it to go right so when I say increase liquidity I mean increase nuance in the way that we can interact with the with the two main resources that lives on right which is computing data and and so that's that's kind of why I'm most excited about about blockchain participating in the eye safety conversation it's it it gives us much more fine-grained control over in theory than than traditional kind of

governance structures based on sort of hierarchical voting schemes do and I think that's that's net new that's very interesting and obviously the the interplay with with blockchain being accessible to consumers to be able to participate is also extremely extremely exciting so um from EGA let's sort of return to narrow AI and an open mind so open mind is one of the first I think one of the first systems that that uses like blockchain and in order to do something interesting in order to deliver a capability that is that is new the way I understand it is generally with with traditional AI in

machine learning the way the industry works is there's a company in the center that collects data from a lot of diffusers it creates a used data set and then this data set is used to train models inside the company and once the model is trained it creates some kind of application or API that others can access and it monetizes basically the access to that model in some way now open mind is about flipping this architecture and tell us tell us how tell us like what open mind would allow and

how would models be built on open mind and monetized on open mind that was that was brilliantly articulated kind of description of sort of how the machine that industry works works right now and we do live in a world where this is this is kind of the structurally like all of our machine learning and AI tools assume that you've collected all the data into one place so there's this kind of not another option right and even from a business model standpoint the machine running business model is dependent on this idea because that's how all the resources and all the tools and all the kind of almost even just the ideas and the VCS heads kind of kind of sit and the story for how open mind participates in this really starts with how much

tools can define what people actually spend their time doing because because ultimately what we are is we're community and were a set of open source tools right and and so what we would like to see is people pick up tools like federated learning and really change to change that dynamic so if federated learning in particular right this is it this is a piece of technology it was developed internally at Microsoft and Google to to allow you to train machine learning models on data that's not centralized and they originally started working on this in the context of smartphones so so from a from a networking standpoint you didn't want to actually have to pull data from smart phones to be able to update update the model that like recommends the next word for for when you're texting

and and so what we would like to do is be able to make that that kind of federated learning technology widely accessible right so that instead of if I want to train a machine learning model right now I would go to you know a VC who and say hey I think that I can train a model on Fitbit data and that we use your heart rate to predict you know how way you're going to sleep that night or some of the lifestyles some of the lifestyle goal you're trying to optimize for then you would then you once you convinced that VC they'd write a big check you would buy the data you would you would you would kind of clean it train your model and then sell sell use that application this is all contingent on whether or not you can convince the VC this is this model is going to make

money you can kind of forget the idea of doing it for just social good right we're open mind changes that story is if if we execute on the vision that we've laid out instead of going to VC and go into this whole rigmarole without even like getting out of your pajamas to just walk over to your laptop sit down and open up you know your your Jupiter notebook and and train a model on private data that it will be readily accessible that there will be no no privacy kind of trade-off as an externality and that kind of your everyday person everyday machine learning developer will have access to these kinds of datasets and not just for commercial use but also for just

building a model for social good and that's you know the name open mind is really inspired by open open source to the extent that the open source community took something that was almost all like that that was built in in industry and only used for kind of making money and instead allowed large groups of individuals to collaborate on it in such a way that you can also spend time doing things for social good and in the machine learning space that means access to compute and access to data with tools like federated learning so so that effectively means that if if you're the engineer building model for Fitbit data so let's assume that these Fitbit devices they collect heart rates and sleep patterns so you are the engineer building the

model and I am a user of this Fitbit and my Fitbit collects my data then you can essentially create a model and I can I can help train your model with my data in such a way that my data always stays with me and never needs to be shipped to you you get a better model and I get compensated in some form for the service of training your model exactly and there's no privacy implication for me so yes there are always private the implications like like like from a cryptographic security standpoint you know like there's there's there was a new vulnerability against every almost every operating system on

the planet that came out three days ago but barring barring events like that that this system is designed to mitigate the need for machine learning engineers to actually see the data points that they're training and that's that's that's net new and and making that accessible has really really nice externalities like this this kind of seems amazing from you know from a crypto mindset guy like so like crypto people are generally very very privacy conscious and we have people in our community that don't participate in Facebook don't have like Twitter accounts that that probably you know like encrypt all of their communications they stay send

out the very very of these large companies collecting all that data and somehow some of there's an unspoken challenge that if the whole world were to be paranoid about data like that we just wouldn't build any good machine learning models right so so as the crypto community we want to make the whole world paranoid about their data but then in the current current sort of framework of how things happen if we do succeed as a crypto community then machine learning in today's pad I might have a hard time so in some senses like

open mind is this sort of the solution or something in the direction towards the solution where the crypto community fantasy can also be satisfied while also the AI community being able to build the models that we need in order to basically upgrade our our civilization yeah absolutely absolutely and and and really that the best part is that like those those tools have been invented that's that's like that's sort of the good news right the good news is that it's it's not like this is some mysterious hocus pocus thing in the future these things exist like there are there are there are conferences that have been publishing about them for four years and as as computation has gotten

faster there there are many of them that are ready for primetime we're just not using them like people people don't know that the that trade off is a false choice and that's that's like the real tragedy and and hopefully we can we can mitigate that so yeah I'm super excited about that there's always like discussion centered around principles like we are the crypto community like in principle don't like the idea of big corporates collecting all our data but then like capitalism has its own rhythm right so new technology or a new system will be adopted because it enables new ways of making money that weren't possible before like that's the

logic of capitalism I like principles be damned show us a way to make money can I quote you on that like speaking for the voice of capitalism so so so let's assume like yeah this is possible that you can like train train models to do various things without compromising the privacy of data what kind of applications do you think will be enabled by it that just aren't there today and this which people are man no top companies can make money this is my favorite question there are there are so many amazing things that we could be building if there wasn't this private like privacy

trade-off right I mean the first one that always comes to mind for me is is private data protects the things in our lives that are personal right things that are personal and things that we tend to try to hide are also some of our greatest pain points some of our greatest weaknesses and some of our greatest vulnerabilities right and and so when you open up the potential for people to innovate in that space without the privacy trade-off the kinds of products and services that you could that you could build without the threat that it actually you know that it takes advantage of the people that are participating right like like the types of products and services that become possible to build are are astounding and they give me goosebumps whenever I talk about them like like if you wanted to train a machine learning

model there was to help predict when someone was going to have some sort of mental disorder or breakdown or depression or the extreme versions of depression like self-harm like these are the things that no one tells anyone right and and being able to help predict that in advance and intervene and not in to me like not intervene in a bad way I mean intervene isn't like like reach out and be like hey are you okay right I or if people even know themselves like hey your your you're walking down a road that's likely going to lead to self harm or to despair like if you wanted to try to predict that ahead of time and give someone an app that's like hey predict when you're you're headed to

a bad place you would need to have access to unbelievably private data right underneath the current scheme like Norfolk company to a great that it would be insane like you would never ever want to trust a company to aggregate that detailed of a bit of amount of information but like that's like that's like you know home transcripts or something like it's like kooky spooky stuff right but if instead with federated learning you're taking the machine learning model to that person and you're letting them train it right and you don't have this privacy trade-off it could be possible for us as a community to work together to train models that actually help help predict and mitigate and trade-off some of these some of the most private and personal aspects of our life right that that's the hope that's the whole thing right

we're severely limited in our ability to help people with their most personal and most intimate problems because we don't know how to do it or we seem to not know how to do it without aggregating data and and and putting those people the very people were trying to help at-risk and and the tech open mind is about is about getting rid of that and and showing another way cool that's very articulate and good and clearly you're passionate at this point I know I want it I want to understand one aspect you better so I that argument makes sense to me right so I have a bunch of private data I I mean I would like to learn more maybe have it used in some way and now it may be possible to do that where we

possibly did have something like open mind but if you look at this overall kind of transformation of how the industry works that you know me her and you can't describe right where we have this shift from data and algorithm in a silo controlled by a huge company to you know data being kind of commonly available but encrypted algorithms being improved on this almost like public good so III see that one drive it could be individual saying no privacy my privacy is better protected in that world but do you think there are like big economic drivers that also pushing in that direction or you is it

kind of a conflict right where the economic drivers more like reinforcing the centralized word and you have a privacy kind of neat going in the other direction or can you see how you see these forces playing out against each other or supporting each other this is where open mind becomes real and become something more than just hey wouldn't it be nice because in this particular configuration it is more efficient less costly to do the same thing that aggregating data does and and here's here's why I mean that if you have an organization and entity anything that has to act as a middleman in the creation of value right like so so data is extracted you can

think of that like being oil pulled from the ground right and then you have this middleman that cleans it and curates it and gets it ready for production right and then then it gets gets distributed federated learning from a tech standpoint eliminates the need for a middleman by allowing machine learning specialists and and data owners to interact directly so from an economic standpoint I mean depending on how like I guess the computation shakes out that's just more efficient data and machine learning practitioners together can create a certain amount of value and right now they're doing it with with the middleman if you offer a technological ability to not do that you have a bigger margin and

if you have a bigger margin it means you can either undercut on cost or grow faster now we as open mind are not we're not like we're not company we're not even a non-profit we're a community that's trying to make these tools as accessible as possible like this is a movement that's going to happen and we're just trying to be there to help facilitate it and to make these tools as accessible as possible because we believe in the social consequences of this movement and yeah like like 10 to 20 years from now I hope that people refer to open mind like they do open source to the extent that it's a classification of tools that allow for the shared training and other ownership of models based on you know distributed private

assets being data yeah I mean I mean one thing I can also see it's just the aspect of having this very fine-grained economic layer to all of this right so that you can literally pay people a little bit of money for providing their data on fine-grained mechanisms for improving and I think that's maybe her ethically possible for a centralized company to do but much harder and and I just don't think it's naturally how like we've blocked and you have to build it like that right this is how it's going to work we have a centralized company it may be possible to try to build like that but it's not a natural way it would be built and and thus I don't think it's

it's gonna happen in that context yeah no I agree so let's let's drill down into what open mind is like so so what is it is it a protocol is in a system like describe to us the constituent parts of it so the it's an ecosystem and its community so so I'm the first to say that that the most valuable piece of open mind is the 2,000 people that are in a slack team and like this this next weekend there's gonna be a hackathon where there are people in 21 different cities that actually go to a cafe and meet together right that that's the part that really matters it's the awareness

and making these tools accessible but like so but you're asking about the software so so the the actual software itself is an ecosystem of libraries the the main one if you go to our github repository you'll see is actually a you know the engine or your unity hitties the word video game but it's it's a piece of software that's packaged inside of the Unity game engine and and what that is and we'll just call that a mine right and and that is that is a piece of software that is designed to hold an individual's data and protect it while allowing them to to be able to train machine learning models so so when when delivered you should be able to go

to the the kind of the Xbox gaming store and download a video game and have clear instructions on your television or on your phone or wherever you're downloading it to to to load in your personal data and on your behalf it will earn earn a passive income stream and it's responsible for downloading models from the blockchain train them locally and sending updates to that model back out to the blockchain so from if you're picturing it in your head it's kind of this this sort of graph right we're at the middle sits it's the blockchain and lolis I just go out to individuals who then with their private data send gradients back up to to to the head node which are then distributed back to

whoever's training the model technically what are the pieces like what are the different technologies that you're that you're using for it yeah so there is a deep learning library called sift and the reason we call it sift is it's like sifting through sand where you're you're keeping the intelligence that you want but you're leaving the the data behind and then there's a blockchain smart contract piece it's called sonar which is inspired by sonar which is your ability to glean information about something far away without without actually having access to it and packaged inside of sift is also the kind of the encryption pieces so that's actually it's a part of sis so it's an

homework lis encrypted and multi-party encrypted a deep learning library so you could think of it structured a lot like a torch or Kerris and actually we have interfaces now where if so if I took part in Karis if you're not familiar and you're watching this those are the main two of the main deep learning frameworks out there is also tensorflow and are the way that we're setting our sift library is that if you use PI torch or Karis you have an interface that is indistinguishable so so that's like a deep learning library that has the exact down to the method calls it's exactly the same but it can be attached to very large private data repositories in the backend so yeah deep lending library smart contracts system and I guess the only

other piece that's worth mentioning is a piece that we haven't yet released it yet but this is kind of the coming out in the next few weeks is a open grid which is basically the distributed computing at work that this will train on so if if you want to use kind of the GPU that's sitting in your your game console your PlayStation your Xbox or you have some other NVIDIA GPUs in my pet and you want to earn an income passively by training deep learning models you'll be able to do so and and the whole network will sort of sit on top of that compute lair but their monetization I mean let's say now I want to do that I want to provide my communicational capacity this network to contribute to better models so give my data or something like that the

there's no monetization built into this software yet right there's no token to pay people or you see that as kind of a layer that would be developed on top or do you see that different people would develop different approaches to solving this problem on top of these basic set of tools that you guys are developing perhaps the the interesting thing is that we're trying to build technology that allows a marketplace to happen and if you want to have a healthy marketplace it's a lot easier if anyone can bring their own currency right so so we've been kind of attached to this idea that that people should be able to trade with whatever they want to trade with

tokenization for us it looks a little bit more interesting when you look at shared ownership of the model afterwards but that's that's kind of interesting but even then that ownership is actually enforced through the cryptography like the multi-party computation shares as opposed to a discrete token so like tokenization is it's a lot it's it's a much more difficult conversation in this sense because first off whatever we bill it has to be not just useful but also more accessible than the next best thing right we're all about making making private computation federated learning as accessible as possible and so when we look at a token model it's like okay we could allow anyone to take to bring in any token that they want right or we

could add an artificial barrier to entry by having our own so like that that isn't we're still evaluating that but the the the consensus right now is that that we want to make software so that you can show up and if you want to you know hook it up to your your theory account or on cosmos you know if you want to use these atoms or something like that like it you should be able to trade with whatever you want to trade with does that make sense yeah and so are you guys think building a kind of decentralized exchange where you can allow the this marketplace to occur or what kind of infrastructure are you building to you know for that actual

marketplace and those actually can only transactions yeah so that that's the software that could support that kind of a marketplace or in exchange is is what we're most concerned with and actually do making that exchange occur efficiently is like the that's the main problem that would work on day to day it's all about okay if you have a data set and I have a model and I send you a model and you're sending me incremental updates how do we as efficiently as possible without without in executing too many transactions and losing money on gas or or without doing the cryptography wrong and then making the gradient aggregation process really really expensive for you know homework encryption and multi-party computation being inherently expensive algorithms like the main problem that we focus on

is making that that exchange of value as efficient as possible which is honestly a very academic pursuit like that that and and that's kind of the main theme if you hang out in a slack team you'll hear us talking about that most the time so this is the basic idea is obviously like so you want to build a model and there's a hundred hundred of us right Brian and me and two of them who have data that's relevant to train your model so you send us the model III train the model on my side that ship the update then Brian downloads the update model trains that model ships the update then person number 37 downloads the model trains the model ships the

update and yes loli as we keep training the model each each of us individually on our side the model keeps getting better and better and we are all compensated for basically training the model in in ESL or Adams or whatever the the currency underlying the system so this this can be this can be the compensation mechanism can be like a blockchain token like a standard token yeah or a stable coin because like like the biggest threat of a token for us is that the volatility is so much that no one actually wants to use the marketplace that we build for what we built it for right I mean I was extremely excited that one of the ways you introduced the project a few minutes ago was like one

of the first projects that's trying to do something real and new with the blockchain like like our community cares a lot about this thing being used for exactly what we built it for and and for you to like put your your bounty for training at machine the new model on the blockchain and for it to go up and down in value ten percent per day is unacceptable right that's just it it's it no one's gonna train a million dollar healthcare model that way it's just we need this to be used by real people trying to create real value and and so like if if some like if someone comes along and solves the stable coin issue or or you know one of those other innovations like it's it's not within our scope but we we don't have time resources to try to play Federal Reserve

with our own own cryptocurrency to create that kind of stable market like we we we kind of we kind of need that to be solved elsewhere to be honest but but that's that's sort of that's where we're coming from and I know most other projects are like ICO first and right software later and we're totally inverted from that standpoint to the extent that that we have we've raised no funding and but we're still you know a hundred and ten people of contributed code and two thousand people are in the slack and and you know our hackathon next week will be all over the world and like people are really really engaged involved but that's that's sort of we really wanted to be used for we're building it for

and that's kind of how we set up our priorities one thing that I don't understand is in in in all of these blockchain systems how to partition a resource into multiple different agents for a service they deliver is is an outstanding problem like if you look at Bitcoin there's a bunch of miners each of them is an economically distinct entity and then you have the reward which is like the block reward and you want to distribute this block reward into these entities in some proportion of the value contributed to them right so in the case of Bitcoin that is it turns

out that's pretty easy like you can measure the amount of work contributed by each of these miners and therefore you can partition this reward pretty efficiently now in your case you have the central company that wants to train a model and it has it wants to distribute a reward to all of these different agents in order to get the model train and then you have these different agents like me Brian that will contribute to training this this model and our contributions might have different value to the company right so maybe when the model is entirely new my dataset might be very similar to Brian's but I trained it first so that first bit of training is very useful but

then 100 other people trained it and Brian was the 101st and Brian's training is less valuable so how do you decide how much to give to each agent that trained yeah so we're super excited about our solution to this problem and the way that we look at it is is directly inspired from machine learning research so whenever you're gonna go to train a model the way that you objectively evaluate how accurate is is using a validation set that's also the best way to specify what kind of model you want to be trained right if I just say a sentiment model I could mean like we talked beforehand like I could mean a sentiment model for a hospital patient

so I give me a sentiment model for a movie review so I can mean a Senate model for for product reviews and like all these things like they're domain-specific right so the best possible way for someone to specify hey I want a model that knows how to do this is to provide a small validation data set that also gives us the ability to objectively evaluate how valuable each gradient is or each set of gradients is to the incremental increase in intelligence of that model so so the most expensive version you could think of like this you send in a gradient we update the model run run the validation set over that model and now we see that it's 1% smarter right or it's it's 1% closer to

the target goal and thus 1% of the bounty is allocated to you now well obviously we're working on our efficient approximations of this so that you don't have to run the validation set after every gradient right and that's what a lot of our innovations based on but but as a sort of high-level starting point that's what it that's what's more or less based on so incremental increases in the intelligence as measured by a validation data set allowed me to know up his data's crap up his data is really good I'm gonna pay him more right without me ever actually seeing the data the the other piece you mentioned about showing up earlier showing up later is solve using just a slightly different trick so on the one hand we actually do want to incentivize people to be first

because that's what helps makes the model get trained fast enough but because these models tend to follow up pretty you know it's it's a downward sloping curve right you can you can sort of kind of adjust just using a function that offsets sort of you know the first 50% receives this much bounty the next 30% receives this much bounty the next 10% receives it about Dean you can do that in a continuous way so that you can control how much you want to incentivize the first guy because anyone who's training machine any model knows the last 5% is where you spend most of your work okay so you can kind of adjust it and you can basically kind of over pay the later the improvements exactly the exact man and that's something that you guys would

decide or with that each each each person said it's submitting a model can decide that so okay so depending and how much they care like if they they just really need a model right now and they wanted to train really quickly well then and I imagine they would skew their their awards more towards the beginning of training but if they're looking for the state of the art model then they would they would reward heavily for the last mile right and they would really try to get everyone to geek out that last one percent and incentivize them to participate so this sort of assumes that there is a validation data set which is let's say let's say the model is to is to predict just repeat that Fitbit example you have

my heart rate during the day and using that heart rate or whatever biological data you're going to predict how well I am going to sleep in the night right or something like that yeah so if you want to make a model like that then you need to have a data set that is like very carefully curated like it's it's a set of observations that you have collected in the field that hey when people have had this heart rate during the day they did sleep well this this much percentage of the time and that highly curated data set becomes your validation set and then every time somebody ships an update to the model

you're going to use that q88 data set you have run it across the new model that comes and like measure the sort of performance of that model really well right yep so is it possible that a process like this can be attacked by creating some kind of biases that you aren't optimizing for like for example like your training your model to be very good at recognizing X but I am the person that is training your model you don't know me because that's the that's the advantage of the system you don't know who I am you don't know my data but I am training your

model can I like can I force some kind of biases to emerge in your model such that those biases are not obvious because they are not yes you did end up paying me but actually I have done a disservice to a model because I have introduced let's say some kind of racial bias like maybe maybe the bias might be that I don't know some for some kind of some kind of particular race their heart rates and their sleep patterns are coded in a particular way but that's not true across other groups and I as a trainer of that data present more and more data for one particular racial category to the detriment that your model performs

really well for that one category but then there are other categories where it just doesn't and I think that in order to screw with your model basically few things first this is an active high priority in the machine learning community period right I mean this is this is something that the machine learning community is actively talking about it's it's on Twitter it's at conferences it's all over the place like how do we have unbiased models on the one hand our system might add a little extra difficulty to that because you can't see the data but on the other hand the because it's a free market for gradients we we we've got some really good defenses against it because there's

such a wide variety of sources so even if even if one guy showed up with a data set that was skewed the unless he's responsible for the vast majority of the data there's going to be other people that are creating a more robust distribution right so so it's a lot easier to accidentally have bias if you're pulling data just from one distribution but if you're if you're opening up the accessibility and and making it possible to pull from lots of different different distributions it becomes a lot more difficult to hijack the model now that all being said what are we doing to help to help mitigate this this particular problem so the first one is is not all gradients get accepted it's actually only gradients that you contribute to in the

current the current plan for how we roll up it's only gradients that contribute to what ends up being the highest quality model so the that creates an added challenge for someone trying to add bias to the system because they don't need to just contribute gradients that are marginally good but also include their bias they to contribute gradients that end up participating in the highest model right so if you could think about having multiple different models that are being trained and only the top one will actually get you know pushed out to to the eco system it has to know which one to pick and and it also asked when it when it creates it so it creates like github Forks so if you're familiar with with github you know you create a code repository and then there's a fork off and there's a little improvement and a fork off and a

little improvement that's sort of how these models end up getting trained so it becomes significantly more difficult it's not just like one model and all the gradients aggregated and you just get paid for however two contributions even if you have good data you have to you have to participate in the best model to get included at all and your level of participation is going to be regulated based on how how how much accuracy you add to the model so there's there's there's already some very significant hurdles that you have to cross and it's even more difficult in the open mind ecosystems because we have such a wide distribution of data so whereas you can you can provide your your piece of it unless unless you can convince the system that that you should be

you should you should be the dominant amount of gradients which you can only do if you're also contributing good data it becomes very difficult so so this is still an active research question it's an extremely high priority in the machine learning community but we like to think that we sit at a pretty strong advantage by allowing data to come from multiple different sources and by not just not allowing any gradients but only gradients that contribute to the best model cool so what's the current state of the open mind project like how far have you gotten what are the main things coming up yeah so we opened our github repositories in July of last year so we're almost six months old

the we launched our first smart contract on a test chain in it was November that you could do basic federated learning with and I guess in the last six months about our community has grown to about a hundred and ten code contributors and about two thousand people on our slack Channel the next thing to be released here in the next few months are so we just finished our our kind of initial deep learning library interfaces for pi torch for Kuras and our standard auto grad system so basically you can do you can chain together any sequence of kind of machine learning computations and it will automatically back propagate

and update so the deep lending library is more or less in in alpha we we do have a blockchain or a smart contact up on the blockchain that being said we're working hard on performance improvement so I wouldn't necessarily for for kind of general availability or primetime it's on a test chain [Music] the thing that we're working most hard on now is and the next thing that we're and going into being end up releasing is the the compute grid so this whole system has to live on a pretty significant amount of compute or do the do distributed machine learning and so we're looking to release that part in alpha in the next few months next to two to three to three months what we'll have it internally before then you say you guys have something on a

test chain so is it in the end the idea that you guys will run your own blockchain and you guys have a like run on top of another blockchain do you guys have an idea what direction outcomes gonna go in yeah we're we've got a short list so the short list is aetherium tender mint and an in this really cool project called trillion and it might end up being a combination of several once again we're trying to make this as accessible as possible for us just being on one chain might might not be really enough we've also been approached by lots of enterprises who are interested in doing this on kind of their big data data warehouses which I actually view is even

though we're very excited about consumer data if if enterprises want to be able to protect their data and not send it around as much but still extract the same value from it I think that's pretty awesome the answer is we'll probably do multiple and and hopefully once again like I really my hope is that open mine will be referred to like open source it's it's a more a community and a class of algorithms than any one particular chain or one particular one particular protocol I've been a quiet onlooker on to your slack community and it's a huge community I actually love the way you manage the community like every day you are like commending members of your community for doing

something III get the impression you are a great manager although I haven't ever worked with open mind or whatever and you have this hackathon coming along like tell us self what are the interesting projects that are being done in your community like actual applications that people are building mmm so unfortunately the most interesting ones are being done by people who kind of want to keep it to themselves for the moment so I won't I won't spoil their surprise I think that the most interesting one that's public is based on reinforcement learning so the unity machine learning team reached out

a few weeks ago and they they've been particularly excited about us building a deep learning engine that actually sits inside of unity and so they actually commented on on one of our github issues you'll find and I'll just I'll share what's what's what they said which is more or less we want to build they've got this really cool project called ml agents you might have seen it is in the top of hacker news not too long ago and it's basically the ability to Train tensorflow based machine learning agents inside of unity worlds so like you can create a world inside of unity that an agent can interact with you know it might be balancing something or might be running you've probably seen some of the gifts running around and so unity has been working on their own project for that

but in order to do that they they have had to in to get it to work with tensorflow pay they literally render an image inside of unity and then shoot it out via a socket to tensorflow which then makes a prediction and then they take their prediction and shoot it back into unity and then iterate the game to the next frame and this and they're not unique in the way they do this everyone who does reinforcement learning kind of does it this way right there's sort of one rendering engine that sits over here and then there's whatever deep learning engine PI torch Karras whatever is sitting sitting over here so I think the most interesting project that's happening right now is potential for the machine learning engine to live inside of the world that and on the same GPUs in the same namespace has as the

the RL worlds exist right because then you can do all sorts of crazy stuff you can use machine learning models to help with the rendering process or to to make the world different right you could have so one AI that makes the world and the other AI that runs around in it right and you can do all sorts of crazy stuff and also you get better latency and better performance which allows you to do new stuff so I think the one of the most interesting projects is going on the community is rebuilding a lot of their reinforcement learning demos but with our deep learning engine based in the Unity game engine actually being used as the backend and there there's some really cool demos like highly recommend checking it out those unity guys they really know how to make well they know how to make unity

look really good so they know to build really cool games oh well so if people want to get involved in open mind or learn more about it what's the best what today took at checkout and how can they participate yeah so join the slack and say hi in general discussion there's a QuickStart guide that walks you through like like who we are what we do what the community is about what we're trying to build we've got around 200 github issues that are labeled good first issue and what those are designed to be or this was to be like the first code contribution that you make so each issue actually has a tutorial on like how to set up your dev environment how

to get unity install like maybe you've never built a video game before and like how to actually implement the piece of functionality that you're trying to implement so it's literally a tutorial for how to contribute code to the system we try to make that barrier really low and once you actually have your first merge pull request you become an official contributor on open mind and you your face appears in our homepage and you you are a member of our github organization and that's consider like the the formal process for for becoming kind of a member of they could have work so yeah come hang out it's super fun and we're in that we've got this really cool team map which I can send you the link for and it's actually it's my Twitter cover

photo as well that has basically every person all over the world with a little dot showing where everyone lives it's like it's in every time zone so when you go when you go hang out but every time zone but almost like a lot of time zones when so when you hang out in slack there's there's always someone on line to talk to and work on really cool stuff with and you mentioned that you guys are organizing some kind of like hackathon when is that happen yeah a week from Saturday so we have hackathons all the time we usually have them on Saturdays or Sundays and there is usually a in-person component and also a virtual component so we'll have a Google hangout that allowed lots of people in it but this this on Saturday the 13th we're going to have a hackathon and I believe there's

going to be 21 physical locations in 21 different cities around the world so there's probably one near you there's like a Florida Toronto California turkey Austria yeah London obviously because I'll be there yeah so so come to the hackathon and if you can't come in person come come hang and hang out and we'll get to know each other cool well Andrew thanks much was a pleasure learning about this super awesome project and I do agree with me even though I don't have a steep and insight that you seem to be this is B very vibrant and well-run and growing communities so I hope I'm gonna see lots of exciting news coming out of this project me too thanks for having me so thanks so much

Andrew and and of course thanks so much for a listener for once again and tuning in if you want to support the show you can do so by leaving us an iTunes review or sending us a tape as well and otherwise we look forward to being back next week thanks so much and we'll see you then [Music] [Music]