#275 Ben Goertzel: SingularityNET – The Global AI Network and Marketplace

20 Feb 2019 · 1:23:56

Ben Goertzel, CEO of SingularityNET and chief scientist at Hanson Robotics, joins Epicenter hosts Sebastien Couture and Brian Fabian Crain to discuss AI's convergence with blockchain. They cover the difference between AI, AGI, and machine learning, the risks of data and power centralization in AI, SingularityNET's blockchain-based AI marketplace, and the role AI could play in blockchain governance.

Speakers

Topics

Transcript

[Music] this episode of epicenter is brought you by cosmos cosmos is building the intent of of blockchains an ecosystem where thousands of blockchains can interoperate creating the foundation for a new token economy if you have an idea for a DAP visit cosmos thought network / epicenter to learn more and to get in touch with the cosmos team and by top-down experience a new way of hiring as top tell delivers only the top three percent of applicants including highly skilled blockchain engineers if you're looking to scale

your team with the very best town visit top.com / epicenter hi welcome my name is and my name is Pham 5nk today we have an interview with bankers Olaf who was the CEO of singularity net he also works for a company called Hanson robotics you may have seen in the news and so dancing her body makes this robot this humanoid robot named Sophia she's been featured and you know on TV shows Silicon Valley the sitcom and so it has become an ambassador for robotics so Ben is an expert in artificial intelligence and robotics and soon

learning that is a company that is building the marketplace for AI on unboxing so we talked to them about all kinds of interesting topics that we don't usually get a chance to discuss on the show since we primarily focus on flexing by talking about AI in a more general sense and what the future is there and that's how that ties into blockchain was a really fascinating conversation and Venice is a great speaker on these topics and does love thinking at a high level so it's really fascinating to get to interview them first we get a couple of announcements so one one thing we should mention is that we're gonna be as I would be at HCC on the March 4th

so this boxing week happening in Paris the whole week and so obvious a CC we'll also be having a meet up and meet up is on March 6th so on the Wednesday around 6 o'clock it's just gonna be a casual you know get together drinks meet up venue isn't totally figured out yet but it'll be announced soon and hope yeah I hope see you there is gonna be I'll be there are sunny might be there as well and we'll have some some guests and other listeners so have a happy to have you with happy to have you join us for that Meetup and you can sign up and register at epicenter dot rocks slash eath CCS epicenter dot rocks

/ ECC if you register there we'll send you the address for the venue when when it's announced and Brian I think you had an update on course one yeah but first of all that sounds great I'm so jealous I wish I could be in Paris too I think it's gonna be a really great conference I mean the first one that they had a couple years ago we were there together it was terrific and I wasn't there for the subsequent ones but it's now grown to the organizers have told me they're waiting for their hoping to have 1,500 people there it's going to be at the Kannamma which is this really fantastic venue and yeah 300 speakers so it's it's it's turning out to be quite quite a huge conference cool fantastic but yeah

so I did want to give a brief update on correspond so you know as many listeners know most listen oh so we've been mayor and I started this company together it's just over a year ago to work on kind of building Ballard's proof state networks and and finally now we are live in the first Network which is protocol loom so it's a new serum sidechain sort of a plasma cash chain and we did read an interview on this before as well and it's using tournament as consensus so we just launched there last Friday and so you you know anyone resolume can delegate to us we also wrote this in deaf kind of research report on loom so if people want to check that out that's

also available and we will put a link to our website in in the show notes so people can find everything there and then cosmos is also about to go live so which is also something where we'll have a about later on and that brings us to the other thing we wanted to speak about which is first of all we we have cosmos is starting to sponsor F Center so that's very exciting and it's starting to do so with this episode now we do need to make a disclaimer here which is that first of all I have some atoms and Sebastiao some atoms to Sonny has atoms and may here as atoms - right so the

episode our team is fairly heavily you know has has some atom positions and in addition to that of course Sonny works for the cosmos team and I used to work for cosmos team and then Mahan I fell so building a cosmos validator so we just wanted to give that disclaimer of France and people know that and fully aware of that and and that ties into a larger topic which we've had a bunch of discussions about but we haven't really taken the necessary action on and it's long long overdue for us to do that which is to have a better way of disclosing those kind of things so what we will start doing and we'll probably

have to up within the next week or so is it's just a page on a website where you know you'll be able to see all of the hosts and and they will list all of the tokens or you know other kind of investments they have in the blocked in space and then I think the other thing we'll start doing it's maybe in the show notes for every episode I mean this is something we have been doing generally like I'd say there's an episode and like somebody has this token then we've generally been mentioning that probably with the exception of Bitcoin and

hearing episodes but but that we just want to be more consistent there we really make sure we mentioned every time and also write it in the show notes so that I think that's mentioned the fact that we hold atoms I guess at least for me I mean I've always been interested in cosmos and I'm actually generally quite excited that they're launching and so having them become a sponsor it just felt like a really natural fit and so I think you'll see in the ad that no their intention to sponsor the show is sort of a benefit for both one because we really think the cosmos is a great platform and people should generally have interest in and also the cosmos team has always been very closely we've always been very

close to that team and I've always sort of appreciate what we've done and demahjay was whenever early guests sort of things yeah so but I think we've said enough on that and we'll have that page on our website and within about a week or so and we'll make a point of also mentioning in the show notes as you mentioned so without further delay here's our interview with Herzl hi so we're here with Ben Gretel and Ben is the founder and CEO of singularity net he's also chief scientists and advanced robotics and holds a number of positions in other organizations we'll all get to that today's interview offense hi thanks for joining us hey it's a pleasure to be here

thank you for joining us so yeah you have a very impressive resume so as I mentioned you're the founder and CEO of singular yet you're also chief scientist as Henson Robotics giveth PhD in mathematics you started a whole bunch of companies and lots of different areas and you're involved in some nonprofits and foundations as well so how did you how did you get here and what what does your trajectory look like and how did you get involved in AI and robotics I've been interested in AI robotics life-extension nanotech femto Tech time travel all these things since you know the early

1970s when I when I was a little kid reading science fiction books and you know now a few decades have passed and I found myself in a world where many of these apparently science fictional technologies are are gradually becoming realities and so it's really really exciting to me to actually be everyday you know concretely working on building Thinking Machines and networking people and computers together into a into a global brain and applying AI to longevity and nanotechnology it's it's astounding that we live in a time when these things are our realities and of

course it's also a bit scary and sobering at times because these things could go badly wrong or they could go amazingly right and you know I've been involved in a lot of different aspects of all these technologies many of which are converging together now so I did a PhD in in mathematics but even at that time I was very interested in AI biotechnology and a bunch of other things I distributed mathematics you know as Bitcoin says in math we trust' write mathematics underlie and alized everything that's the foundation of all modern science and technology so I figured learning a bunch of math couldn't be bad but since shortly after

getting my PhD I've been really which was 89 I got my degree I mean since then I've been working on AI in in various dimensions and aspects and now you know in the last few years that's really taken off along with a bunch of other other technologies and of course blockchain and cryptocurrency would you guys know a lot about this is all part of the mix so right now there's a insane number of different advanced technologies for manipulating creating different kinds of information that are all you know intersecting and then and pushing each other forward and you could talk about these for hundreds of hours without exhausting at all yeah that's certainly true so let's let's spend a little bit of time first on the topic of

AI which is something that I think you know we've can gently talked about a bunch of times but still it's I guess like like probably for many outside of blocks in space there's you know a big scary term that's a little bit hard to kind of demystify so how do you define AI and what's the difference between AI and you know terms that people use like machine learning and deep learning I don't think any of these terms are worth too much in in the end I mean AI I mean in what sense is it really artificial and so it's all part of nature and to some extent these systems are evolving and emerging instead of being purely

artificially created and intelligence we don't even have a good definition for among humans like there's not an IQ test that works across different cultures or ages of people than alone across different kinds of minds so none of these are very rigorous terms I mean machine learning again in essence all AI really is about machines that learn and reason and think that term has lately come to be used to describe particular types of AI algorithms that are trained on large amounts of data but then the term is also used more loosely so we always reinforcement learning a kind of

machine learning or not it's it's not especially it's not especially well defined and I mean deep deep learning again in cognitive science you know a guy named Stella Nelson wrote a book on deep learning what fifteen years ago which encompassed neural networks logic systems production systems many kinds of AI algorithms but now now the term seems to be used for what used to be called multi-layer perceptrons multi-layer neural networks which is really only one special kind of deep learning system and then in the broader sense so I mean what what deep learning originally meant was any system that just does hierarchical pattern recognition like recognizes

patterns within patterns within patterns within patterns in the world and uses those to take some action the deep learning systems being talked about mostly now our hierarchical neural networks which is one special kind of deep learning system in the broader case so I mean what we have is a lot of words with confusing definitions that that that shift over time and don't necessarily mean what they sound like they mean which which comes back to in math we trust right because the thing is the algorithms are doing are doing what they're doing and that there there is a real mathematical description to them which and they carry out practical functions but the the buzzwords associated with them serve mostly to to

sell things rather than to convey useful information okay okay that is helpful but then let's speak about one term and I think that's a term that maybe do have more relationship to which is a GIS or artificial general intelligence yeah well let me let me try to go to what I think are the foundation's here because I mean that there it is possible to describe these things in a way that makes sense it's just that things become marketing buzzwords in the hen tent and then them become become confusing so I think fundamentally you can think about a mind

or an intelligent system as something that's recognizing patterns in itself and in the world around it and then that system may have some goals which doesn't mean everything it does is goal directed but it may have some goals and it it then recognizes patterns regarding which actions will achieve which goals and in which contexts right so you have a pattern recognition system and it has goals among other dynamics and it's trying to learn it it's trying to recognize patterns and how to achieve what goals in what situations and you know they babies do that right babies are recognizing patterns in the world around them all the time and they have some goals like they want to get some

milk some food they want to run around and they try to figure out what patterns of activity will let them achieve their goals and what in what situations and then where I mean where deep learning comes in is the world we live in seems to be made largely of hierarchically composed patterns where you have patterns that build up in the more complex ones will happen the more complex ones I mean just like physics builds in the chemistry builds into biology builds into psychology builds into sociology so so we have a hierarchically composed patterns which means if you have a learning engine that is trying to recognize patterns in a hierarchy it may well succeed because our world seems to be seems to be built

that way now it happens that most of the AI is out there in the world now are able to recognize patterns in a very narrowly defined context and to achieve only a very narrow set of goals like say the original alphago could recognize patterns in go games and it could achieve the goal of winning a go game right and was it now alpha zero was a step beyond that these programs are all by google deepmind which is one of the more interesting AIE organizations out there alpha zero won't be on that because it can play a lot of different kinds of board games so it can recognize patterns in a broader scope of environments and

many many different types of board games and it can achieve more types of goals because the ways of winning chess or go or shogi whatever are are different right and still not nearly as general as a human being though because we can not only play a board game that we can recognize patterns and a huge number of other kinds of environments and we can achieve many many different types of goals like we can prove math theorems we can blow people up we can we can chase girls we can we can make art we can try this save starving kids there's a lot of goals we can work toward in a fairly rich collection of environments but we're still not like infinitely general intelligences you could you can imagine a mind that could recognize patterns in

four hundred and seven dimensional space we're very bad at that right we're much better at like two three or four dimensions so what we're still somewhat restricted we're good at recognizing patterns and some kinds of environments and achieving some kinds of goals better than alpha zero or existing AI programs but you could imagine some kind of mind that could recognize patterns in like a space of any dimensions and in things that just look like noise to human beings and they could achieve goals that humans can't even begin to begin to understand so I think you know totally general intelligence that could recognize any kind of pattern in any kind of world and could figure out how

to achieve any kind of goal by recognizing patterns have had to achieve that goal that's probably not achievable in this physical universe like you're totally general intelligence but we're much more general than any existing AI program like each of us can deal with a lot of different problems and if you give us something totally new to deal with like the internet didn't exist when I was born let alone when when the you know my DNA evolved that didn't exist when I went to school but I like everyone else was able to adapt to deal with this new thing right we don't yet have a eyes that can transfer the knowledge and adapt to deal with some very new type of thing they weren't programmed or or trained for right and I

think I think we will but we're not there yet so I think now the AI field is starting to begin the transition from narrow a eyes that do highly specific things recognize times and achieve goals and very specific domains toward more general a eyes that can just get with a broader scope of knowledge and a broader variety of goals and contracts for what they've learned so far the very different conditions and that I mean this will be really important you see that with like self-driving cars now are crashing into people because they're seeing situations that weren't in their training data I mean that's a failure - a failure to generalize right and then in financial markets when you have

what's called a regime change Oh suddenly the market is acting totally different than it was acting before well again current quantitative financial prediction systems and risk management systems have failed to generalize right though they were trained on previous market regimes when you give them a new market regime know that they're still acting on their on their previous knowledge no of course most people can't deal with a new market regime either but leasing foundationally we do have the ability to like go back to basics and deal with a radically new new situation and that's that's a big a big challenge facing the AI field in in

the next phase which I think we're gonna meet but it's it's still some research challenges there that's interesting I've never considered that way that you know I guess like humans and carbon-based beings are good at recognizing with certain types of patterns and I guess you could maybe differentiate so like humans are good at at recognizing certain types of patterns and acting on them and that might be different from like for instance the intelligence of a dolphin or another type of armor baby and then artificial or computer-based intelligence or like silicon based intelligence might be good at recognizing patterns and like you said

thank you multiple hundreds of side engines and figuring out you know actions based on it's kind of subtle if you think about it cuz like I mean we we evolved in this domain of like discrete solid objects like bouncing off each other and so on right and this probably led us to ideas about causation but if you're a dolphin in the water you are seeing things flow around and blend into each other you're not seeing so many solid objects bouncing off each other that probably leads to a quite different worldview now also like where each of our minds is stuck in an individual body or like our entire life until we die right and I mean in - in barring a

reincarnation and other other freaky things at least to it to a first degree of approximation right now if you're an AI that can put yourself between different bodies or occupy a hundred different bodies at a time or like fork yourself and roll back to your last version before a traumatic experience okay how does that change your whole outlook what kind of patterns you look for what goals you bother to achieve what risks you're willing to take right I mean I mean there's there's so many ways that were overfit to the exact environment we evolved in and the problems that we're trying to solve and that the other thing to realize like realize is we we're stuck without root access to our brains and

bodies which is which is pretty terrible right because I mean if you're an AI you can have like root superuser access your own brain and body if you think well I don't like the way I react in this situation just go in and fix the damn bug right but we we can if we want to fix bugs in ourself it's like years and years of a you know meditation or therapy or reflection rather than just going and and change the road piece of code right so I mean they there's a lot of a lot of things we take for granted now in terms of biases we have or restrictions that that we have which are not really intrinsic to being an intelligent mind but but I mean they're

just particularities of how we happen to evolve out of apes and in in Africa right right and I mean that's in in general this is sort of why I like a mathematical and conceptual view of things because how things evolved now there's some fundamental reality to it but there's a lot of historical contingency I mean you see the same thing with exchange and money and so forth I mean people take so many things for granted about how economies work which aren't necessarily intrinsic to the nature of like exchanging value in the community they're just how things happen to evolve for quasi random

combination of reasons this episode of epicentre is brought to you by cosmos the internet of blockchains we couldn't be more excited about the upcoming maenette launch and to see so many projects already building on it blockchain technologies are evolving fast and development shouldn't be one size fits all as a DAP developer you need the tools that will allow your dad to scale grow and evolve over time the Kosmos sdk is a user-friendly modular framework which allows you to customize or adapt to best suits your needs it's powered by tenorman core an advanced implementation of the BFT proof of state protocol Kosmos takes care of networking and consensus and allows you to focus on building your application in your language of choice if your 'i'm

smart contracts will be supported soon and the sdk makes it simple for you to connect to other blockchains in the cosmos network if you have an idea for a tab and would like to learn more about the cosmos sdk or if you'd like to connect your existing adapter cosmos there's a cosmos network slash epicenter for epicenter listeners the cosmos team will reach out to answer your questions and help you get started we'd like to thank cosmos for those supportive epicenter so let's talk a bit about about AI and data and this is the topic that has been brought a lot in the conversation about AI and the fact that AI needs large points as a theta to to train yourself and we kind of talked

about this so you today debt data is very centralized data is held and owned by by a small number of very large companies do you see this as a problem you know there's any type of repercussions that were unintended there or is there a better system that you think we could achieve yeah I mean the situation with the collection storage and use of data regarding human beings on the planet now is really pretty ridiculous I mean it's not that it's necessarily entirely bad or malevolent some of its really good and useful but overall the

ownership and control of data from the various centers we have everywhere is its centralized in in a pretty bizarre way I mean some of its good of course like a Google Maps it's it's pretty nice that it's collecting data on where everyone's driving to so you can like see weather where there's a traffic jam right so these are these are very these a very useful functions and I don't really mind sharing location anonymously of where I'm driving with Google map so we can tell everyone else where there's a traffic jam right I mean that that seems like a fair exchange but I mean in in the end the agency regarding use of people's data is in a very confused

state so like that this phone I carry with me everywhere right there's a tremendous amount of data coming through this phone onto the internet and it's all in a sense my data right it's data about what I'm talking about who I'm talking to like where I am where I'm taking pictures of but all this data coming from me through this device that I bought and then I pay a subscription to connect you the Internet each month like this data it's going sort of haphazardly into various databases owned by various large corporations probably passing along to various governments along the way and then this data is then being used you know for some useful

things like tell me when there's when there's a traffic jam right and and then it's being used to advertise things to me which doesn't matter to me merchants I never I've never clicked on an ad in my life I think but I mean it's it's it's it's being used by big companies to make themselves money and increase their ability to manipulate people as a whole are keeping the final click on their ads by studying me along with everyone else they're learning how to manipulate the minds of other people who do click on their ads and do read their their fake news so then yeah you got to ask like okay this data that comes from me through this device that I'm paying for I'm paying to connect to the internet

like what why isn't there an easy way for me to observe what this data is being used for and have some agency over what this data is is being used for like it if my data is being used you know to provide data to feel someone's political campaign I'd rather have it be used only for a candidate I agree with there's something right and you know it's quite women are reached technologically to put agency overuse of our data whether for AI are for basic statistics and in the hands of the human being who produces that data on the other hand it's not in accordance with the business model of

the large corporations involved in the phone and the Internet services behind it it's not in the interest of the business model of these corporations to provide that agency to the user except insofar as like government regulators forced them to but of course government regulators even when well-intentioned which is only a fraction of the time they can't keep up with the advances of the of technology now this I mean this right now is mostly an inconvenience and a sort of aesthetic and moral and Felicity but I mean as as you move from there I I toured AGI if it turns out

that these stores of data you know are critical for giving some parties a boost toward AGI more so than others right and then then this hoarding of data could actually have a have a more a more critical importance and in principle blockchain and related technologies give away to circumvent these issues by putting each individual's data in some you know online repository or distributed decentralized repository which is encrypted by their private key and then giving that individual agency over how the data is used and then there are fancy tools like homomorphic encryption and multi-party computation

which can be used to you know that a person give certain aspects of their data to certain other parties to use for certain things without giving all over the way so and yeah in theory the blockchain based decentralized ecosystem provides the technical tools and the sort of cultural uhm to to solve these problems and on the other hand the centralized ecosystem underlying you know big data and mobile phones and computers and embedded devices is as multiple trillion-dollar companies pushing things pushing things forward so that there's a the decentralized world

has the right tools but a big challenge on their heads here I'd love to speak a little bit about the concept of AI safety and gesture to take a step back here I guess like you know what what are some of the fears scenario say so so like let's say fear scenario today is okay AI is replacing all of these jobs so people become you know unemployed maybe it leads to accumulation of resources for a few people more and more and you have this like extreme inequality but like that that's like one fear scenario about AI maybe a different one is that then this AI starts to have its own objectives becomes more more

powerful gets more more resources and its objectives maybe it's hostile to humans or maybe it's like indifferent to humans and so you have these potentially like bad outcomes right and that may be extreme inequality maybe you just have like human beings becoming a kind of you know inferior species being exploited and then you know this is this is feel right like this IQ of AI Safety's like what are your fault on it you think this is an important field do you think efforts around AI safety are needed people are certainly right to be thinking about AI safety and really about the

impacts and implications of AI for the advance of technology and the growth of humanity in general because I mean looking at AI separately from politics and from all the other tech connected with AI problems probably doesn't make sense so people are certainly right to be thinking and worrying about it now whether the things that people will do about it will have a positive or negative impact is is a different question right like I mean bioethics is is a somewhat similar thing and in general it's easy to agree we should be thinking somewhat about the ethics of you know Jake engineering and the bio

hacking and so forth like No I don't want people to create like weird say artificial babies that have like a hypertrophied pain cortex oh they're just suffering and screaming with the billion times the level of suffering any normal human can have right so I mean clearly clearly there are some things that as a society we just don't want people to bioengineer because they're they're just plain old nasty and you're just creating suffering on the other hand in practice the role of most bioethicists seems to be just to say no genetic engineering is is bad don't make crisper babies don't upgrade your intelligence right so well in theory yes

there are things that are just morally bad to do by essentially any human standard and we want to reflect on what to actually do and what not to do not all possible things should be done on the other hand in practice bioethics seems very one-sidedly inclined to just push against advancing of humanity in new directions and to push against reduction of suffering in favor of maintenance of the status quo and I would say most most people who talk a lot about AI safety are not really thinking about how to maximize the odds of a beneficial

outcome for Humanity all things considered they more are thinking like how do we slow down AI development because we don't understand it and we're scared about it so I found myself disagreeing with almost everyone who's putting themselves out there as an AI safety pundit but that that doesn't mean I don't think AI safety is important look I don't want the Terminator to be roaming roaming the streets okay I mean I have four kids that I don't want an AI to be turning them into fuel or something right so let's talk about I was into Hanson robotics quite quite well because with progressive hands - robotics and you guys have built this this this robot named Sophia that I'm sure most most of our listeners have

seen at least at least once on the internet because she's had quite a few media appearances she's been up to befoul it I think it was a bunch of different conferences and what's the purpose of this robot and how does Sophia indeed was partly created and envisioned as sort of an ambassador and basa door of a ie love and love and and compassion and III I think that's been interesting to see because David Hansen who's a good friend of mine I mean he I've known them for a long time and I came to Hong Kong where

I'm living now in 2011 and he visited me here once and I ended up convincing him to come here and move his company or introducing him to some folks who helped inject funding into his company here so we've been talking about these things a long time but I think what's interesting is David is really a warm loving good-hearted person he wanted to create a robot that would emanate love and compassion and make people love it so that it would you know build a positive relation being humans and robots like proactively even before we have human level AGI so that as a AHS and robots get more and more generally intelligent that positive relationship is there on the other hand David he's an artist and

as such he can't help himself from poking people and provoking controversy a little bit and making making things a little bit creepy sometimes just because he thinks that that looks that looks coolest so I mean I would say Sophie out on all the Hansen robot sir they're driven by David's desire to build a sort of compassionate loving bond between human and AI and robot some level driven by David's Sonny conscious artistic desire to poke at people a little bit and make and make them a little a little uncomfortable right and the these come

together and in an interesting way and I think that's good because you know my my emotional orientation is optimistic and and positive so I mean not my intuition and feeling is that the technological singularity he's going to come out awesome and closer to utopic than dystopic but I I also think there's a fundamental uncertainty to all this so people are are certainly justified to feel a little bit uncomfortable and confused so I can in the end none of us knows what's going to happen we're on the verge of creating machines that are you know 10 20 100 a billion times more intelligent and capable than we are we'd be idiotic to

believe we could predict in detail how this is gonna is gonna come out I mean I I find this irreducible uncertainty beautiful and exciting and I see it as what humanity has been doing since the beginning like this is why we're less boring than cows and sheep right I mean we we decided not to remain monkeys and we we invented language and fire and wheels and and machines and money and Bitcoin and and AI and and AG I mean that's the trajectory world we're revolutionising ourselves over and over and we never know what's we never know what's going to happen next right and that's that's probably the essence of what of what it is tunde human and I

think David he baked some of that into Sophia as an artist along with the love and compassion which is quite cool yeah and so probably many listeners have seen the TV show Silicon Valley so there is this kind of basically inspired by by you and by Sofia that part there basically you know someone meant to be you is it's kind of playing this role but let's move to blocked right now when did you get interested in blockchain and why did you think that blockchain had you know kind of relevance to you know the future course of AI

I've been so I've been interested in crypto for a long time like since the early nineties now it's doing back with finite fields and cryptography tech and that that seemed like you could potentially be important just politically in terms of stopping governments from having like the ability to spy on everyone's information and and keep it unique uniquely for themselves Bitcoin I didn't like because just because proof-of-work annoyed me it just eats up the environment and waste energy unnecessarily so I did I didn't get involved in that when etherium came out that's the first thing where I thought well this this is actually cool like it did it did use proof of work but you

could see there was a will in the path to going beyond that and then you had solidity I mean you have a scripting language which basically lets you create this you know secure in encrypted decentralized world computer and I thought I thought etherium was a vision in the right direction and it was a reasonable software tool set although obviously immature at first and not not that mature still so once the theory em came out I started really thinking like how do we use this to create like a decentralized global AI network because I in 2001 I published a book called creating Internet intelligence which envisioned that you know a decentralized

global network of AI is coming together as a society of mind and I before that in in 95 I I posted some web pages claiming I was going to run for US president on the decentralization party platform which I I ended up not doing because I realized in time with a terrible job it would be to be President anyway but I mean the these ideas were interesting me for a long time both decentralized control politically because I always had a sort of anarchist socialist bent and then the idea of making a decentralized global AI network like Marvin Minsky's Society of Mainz but an economy of Mainz where the AIS are paying each other for work and

there's collective intelligence coming out of the of the whole network beyond the intelligence and the parts but in theory em seem like a critical step forward toward having a toolset that that would let you do this and so that then as soon as a theory miserea the idea of Dow's decentralized autonomous organizations which again they'd been spelled out in science fiction like in charlie Strauss's book excel around oh and a bunch of others but with the salinity programming language like wow you could script a dao in a short script right that's that's similar feeling to how when I first learned Java in like 1995 it's like wow you can create a webpage

or send an email with this much code that's power right solidity it was like that like you can you could create a decentralized corporation and just a little bit of code it's not the perfect language just like Java wasn't but I mean it it it really opened the door and once I saw how etherium worked I started thinking well how do we put this together with for example OpenCog which is my open source AI platform aimed at general intelligence or you know distributed neural networks and genetic algorithms or whatever other type of AI it seemed clear you could use aetherium as a basis for connecting together many different AI nodes and to some sort of decentralized AI mind and then this

logically this should be able to kick the asses of google amazon $0.10 in the IBM and all these big companies by you know the power of decentralized community and then when when I I met Simone a Jack O'Malley who was later to co-found singularity that would mean he had he had a blockchain development team in Italy and he'd been helping out a host of different blockchain projects so when I met Simone a who was really conversing with the blockchain world both technically in on the business level then we sort of put our heads together and we like roughed out what became the singular unit designed and then

started moving toward the initial token sale and then David Hansen already was a close friend I mean he he saw the vision immediately our first meetings on this were in the Hanson robotics office and in Hong Kong and David saw this as a way to get like a decentralized global robot mind cloud behind his robots because you always knew the intelligence is going to be in Sofia right I mean some is about seeing and moving but the cognitive parts the long-term knowledge you're going to be in the clouds but what cloud right do you have a million robots around the world and all the intelligence is running in Amazon's Cloud or it's using like Microsoft Azure API or do you have like a decentralized mind cloud that's owned and controlled

by all the people who were buying these robots right so that that was David was seeing it as a robot mind cloud but it was really the same thing that Simona and I were seeing with the decentralized blockchain based AI mine okay okay so would it be fair to kind of characterize this as you know you see this project or you see say I coming but then the question is yeah where do those RS coordinate where did they share information what kind of substrate do they run on and of course if you look at it today but it will be mostly controlled by companies like Google and Facebook and and with something like similarity net there could be you know

kind of open decentralized transparent accessible the mccr attic platform where you know a ice could coordinate a ice Kushida a ice could evolve that's kind of - yeah that's right so that I mean as I've said before what really excited me about the singularity net design and vision is seeing that two different goals which are very important really converge into one so what one goal is to make sort of a venue for many different AI components to join forces to make it collective AI money where the whole is greater than the sum of the parts so you

may have a one AI that like uses our OpenCog algorithm to generalize and abstract and reason you could have another AI that recognizes patterns in DNA data another AI that uses deep neural nets recognize patterns in visual data and you connect them all together into a mind that self-organizes in a dot so the AGI could be in the whole network not in any one particular node in the network and then the other thing is okay but if we're gonna have this network of a eyes like who controls that that network is it all sitting inside Google or Amazon or or is it just more like the Internet right which is not controlled by anyone it's a network of networks which is controlled by the different

participants right and so it seemed you could use blockchain to achieve both these goals to make a network of AIS it's controlled by the participants and it's sort of democratic self-organizing an open way and also make it so that the design encourages that guys to collaborate with the with each other and join Federation's with collective intelligence and and so forth and so this yeah of course is easier said than done but I mean we we did the initial token sale for this December 2017 we're launching like the initial the beta version of the platform the end of this

month the end of February after a simple alpha was launched December 2017 and then during 2019 posts the February launch of the beta we're gonna add more and more and more features to the network as well as adding more and more of own our own AI into the network and they're you know there's a huge the biggest part of the struggle remains in the future because I mean we have a beta version of the platform we have some nice AI we've put in there but still you know our competitors are our trillion dollar companies but the amongest server server farms you know Amazon has 10,000 people working on Alexa right so so - to

counteract that we need not only a good design in smart AI we need to attract a developer and user community which is even bigger and better than then the armies of highly paid employees at these big tech companies have and this I mean this is one of the reasons why I'm happy to talk to you guys and your audience because getting a community crystallized around decentralized AI is absolutely critical to really making the decentralized AI vision happen hiring is stressful let's face it it's a long process of sifting through resumes and

interviewing candidates without any guarantee of quality but it doesn't have to be this way companies all over the place are experiencing a new way of hiring with top-down if you go to their Trustpilot page you'll see that of the hundreds of people that have left reviews over 98% were four or five star ratings including one guy who wants to give his developer a bear hug that says a lot it's all this great feedback because they focus on their clients and their top priority is quality they only accept the top 3% of applicants including highly skilled blocks and engineers one of these engineers is Roddick astrouski Roddick has experience as a lead software engineer and data scientists for Sony and Expedia then he discovered blockchain and he became totally consumed with aetherium he worked as a

consultant for the firm start on chain and his time-locked app when the top quarter consensus you port and identity blockchain hackathon then he expanded his reach through top tile he worked with a bunch of clients on projects such as smart contract development and a POC leverages blockchain if you want to hire engineers like Roddick for your team go to top tile comm slash epicenter for a Norris trial a top tile director of engineering will deliver your next hire in as fast as 48 hours and you'll get $1,000 credit when you decide to hire we'd like to thank top towel for their support of epicenter let's go a little bit in depth on you know similarity net and what that looks like so can you speak about you know what are the different kind of components of the

system and you mentioned you know developers getting involved let's say now there is some AI algorithms you offer like how would they how would an interaction with singularity now look like well if you have an AI algorithm integrating it with singularity net is actually not especially difficult I mean it's a container based system like most cloud systems now so you you put your your AI in a docker container or lxc container and and and then there's a simple API to integrate it with which then let's let's your AI accept payment for services in our API cryptographic

token let's it and then announce what API it wants to use to get data and queries and then it can give responses in in JSON or whatever API if it wants so it's really just it's a system of containers and then there's a payment system using a token and I mean for cases where an AI outsources work to another AI which outsources to another a I there's a I mean there's a multi-party escrow framework on the back end and there's a system that allows a lot of AI the AI transactions to occur off the blockchain for Speed purposes but all that's really behind the scenes I mean from the point of view

of an AI developer it's it's really pretty simple to put your a AI in a container and take I mean 15 minutes to two or three hours to integrate integrate with the singular unit wrap wrappers so I get that right so I put my a algorithm into docker container kind of make it accessible through single arity net but let's say now I'm on the other hand somebody I have a bunch of data all love to get a better understanding of maybe what's actionable what it means so could I didn't go and basically say you know kind of hire the services of different of these AI algorithms to like get me resolved yeah

so I think the decentralized protocol could actually be used by anyone and I mean we use behind the scenes a component called drizzle which allows like decentralized search of any network of aetherium nodes so I mean you could if you are a reasonable scripture I mean you could just put out script your own query to go search the whole network and find any AI that broadcast is able to do the kind of thing that that you want now on the other hand we're making it easier than that so along with the beta we're launching just a marketplace user

interface which is a website and you can go to that website you can see what AI services are unlisted and and what sorts of things they do and you see their addresses and and so forth so I mean that set right now in practice that's a bit centralized right because I mean we make this web interface which lists a bunch of of AI on there and I mean we are legally liable for what we list there so we have to do some vetting of what we allow on there just like the Google Christ or those or something on the other hand the underlying protocol is completely decentralized and open so I mean for example if like we're

incorporated singularity net foundation which is building the singular unit Network now was incorporated in the Netherlands so suppose in Netherlands law said we weren't allowed to list on our user interface you know an AI based in Iran or North Korea or something then we'd have to take that off our interface on the other hand the decentralized network is whatever it is right so someone someone in Iran cube open another interface which is like an interface to all the Iranian and and North Korean they I know it's on the network or something right because they so this is the beauty of this architecture you have this decentralized protocol which is controlled by no one

and anyone can put an AI online and then it just announces it's there to the other AIS in that network and then it can be found by decentralized peer-to-peer peer-to-peer interactions so that's there which gives a lot of robustness to it on the other hand for ease of use we're putting up a simple website which just lists lists the AIS that are that are on there which then can be interacted with from a customer for you just like you're getting AI as a service from any other directory or somebody's website now that the beta still has some limitations in the sense that we accept payment in the beta only in our ati token which is an AR c 20

token and one of the things we're gonna do in the months after the beta is integrate the third-party no fiat to crypto payment system because because of course most most companies who want to use AI inside their their website of their product not from the crypto spaces they don't want to deal with crypto wallets and so forth at this point but this isn't isn't a really big obstacle it's just something we we hadn't hadn't done you that's more regulatory thing than there's a hard technical problem so you mentioned the AI hei token so what's the role of the token well the token is used by a eyes to pay

other AIS for services that they provide but having our own token economy lets us nudge the incentive mechanisms in interesting ways so as well as using it for payment of one a I buy another AI we we also will issue token bounties as as rewards for people who contribute eyes that are requested by the community and then we will implement later this year a curation market where if I want to rate your AI is good one way I can do

that is to stake some token on your AI and then if your AI comes out to be rated good buy a lot of diversity of other people I'll get some rewards whereas if your AI turns out to be horrible then they're not balloons I will lose some of what I some of what I staked so having our own token it's both inefficient a secure and private way to do transactions and it lets us do things with you know bounties for development and staking and curation markets which I think can sculpt and guide the economy of AIS and this is this is quite important because there's there's something in AI in cognitive science

called the assignment of credit problem which is one when you have a complex network of agents cooperating to do some function I mean how do you ensure that the agents like deep in the bowels of the network that indirectly helped achieve the function are actually getting getting rewarded right and the human brain somehow does this right like if you do something it gets you food or sex or money or intellectual satisfaction whatever is good you know that the neurons that moved your arms and legs don't get all the reward right there's reward that goes to the neurons deep in your brain that helped you get whatever those goodies were the US economy for example doesn't do so good a

job of assigning credit internally which is why bankers make so much more money than programmers or kindergarten teachers or artists or something right so we and arguably the Bitcoin and aetherium economies although they're really cool in some ways I mean there's a strong tendency towards like oligopoly and oligarchy and in these economies and that they don't necessarily do a brilliant job of assigning credit to genuine value either so by making our own token Amish economy and adopting the reward system and we hope to make the economy of a eyes operate better than other existing economies so

that the AI is really contributing most to the overall network and its intelligence and the value of deliveries the AI is contributing most to the overall network are actually getting rewarded significantly in that this is a hard problem we're like economic design matches cognitive science right it's it these are fairly subtle things yeah this is I would ask you this so if you take something like I'm not sure if you're familiar with the system the blockchain true--but so it's like a distributed computation blockchain edition distributed computation systems have been around for years and more recently you know people have embedded them in with blotching systems so that you can

have this reward mechanism and so with retrieve it you you have these actors at the network who watch for who who can potentially validate or verify the computations and so therefore there was an incentive to those providing and computations to provide correct computations because they have a there's no possibility of them getting slashed if they don't provide accurate computations now this is for computations that are somewhat trivial to to achieve with like even general-purpose computers but with with AI if I send some tasks to an AI and it it returns a result it returns some sort of data set how can i as a user or even

other users of the network verify that and also I think with the AI there there might even you know with gem like general intelligence becoming feel closer to reality you know a eyes could have their own kind of subjective subjective bias perhaps and so application might be different between one of the eye and another how do you test for that and how you verify that the resultant in LA is providing is actually actually accurate yeah I mean that there's there's clearly no general solution to that that problem just as there isn't among humans right because they eyes are going to be doing so many different so many different types of things I mean you could have an

AI that's proving tripping math theorems or coming up with science hypotheses to help with with biomedicine or or predicting the stock market or something right so then it's like is your stock prediction AI giving subtly biased predictions that it's then using to make money by trading itself from the background against what you traded there's something that I mean that there's a lot of subtleties that could come up and they're gonna be different for the different kinds of AI that that you're doing so I think that if you're doing a specific type of computation or a specific type of problem then you can come up with like a

formalistic solution for this right like if if you have an AI that's generating programs according to spec's you can do some like formal software verification to see that the software actually performs according to spec and if you're if you have an AR that's analyzing DNA data I mean if you if you have your own human DNA data you can do out-of-sample testing on that data to see if it's valid but there's really going to be no general purpose solution and singular unit is really a general-purpose network so we I mean we put a bunch of work into designing a reputation and rating system which is sophisticated and hard to game

and this is not part of the beta but it will be it will be rolled out rolled out later in 2019 but I think that that's been like a holy grail for every online marketplace right then we you really need to get you really need to get that right because in the end vacation that things are accurate unbiased or not too biased or with inappropriately biased I mean this is really hard and it's domain-specific and ultimately each person isn't doing that on their own and right it's like if the point of delegating tasks to any eyes come on our own study but what jiù

delegated to right so like if i'm if i'm want to verify that someone's AI for analyzing DNA data is accurate and then not many of us are going to you know write the code or even run the code without ourselves we're gonna go to some service that that does that and then which service did we trust like is it the singular unit foundation certified service well then that's like a centralized elite or or do you have a variety of you know competing services out there then you choose which one so in but then it comes down to reputation systems again because then

you're choosing the one that you think as the highest reputation maybe because it comes from Harvard University or from the the NSA or like who do you trust right so I mean ultimately even when there's a formal mathematical solution there you're you're placing trust in someone right now if something is simple and generic enough you could make verification into the protocol right I mean as it's done as it's done with with cryptographic checking and but I think checking if an AI is correct or not is just it's not gonna be that simple it's gonna be a variety of different algorithms for different checking different types of problems in different

domains and then you need reputation system to be able to place to know which there vacation checker to to trust and then people will try to game game that reputation system by giving a high rating to to bogus like very truth verification checkers or hexa so you need a machine learning based reputation police just to try to stomp out people gaming the reputation system and then you have to believe the machine learning based reputation police itself isn't corrupt right so I mean I mean this is the right this is the world that were that were that we're in but on the other hand like the real world economy

isn't all that clean and and safe either right like which major government is not is is is not corrupt and some in some serious way so I don't think like the AI and blockchain economy is not creating this problem this is a problem of human beings being right and this is this is just it's manifests itself and everything the human beings do okay so this is great so if he very much ties into another thing that you know I really look forward to addressing here a little bit so when you spoke about kind of the division mission of singularity net in a different interview I heard he

mentioned that tinglin air net has these two objectives first is objective of maximizing intelligence the other one this objective of kind of pursuit the maximum benefit for all beings I'm curious so you know we spoke a little bit now about okay how do you evaluate an AI and you know how can you check with it what they're doing is correct is in your interest so now I understand also the concept of creating this efficient marketplace for AI and so now I is a normal small business owner I can kind of use AI and maybe have something almost as good as Google and I don't trust them Foley or maybe something

better than Google right down the line but like how can you [Music] how can you make sure that this system is going to end up being a system that pursues this you know benefit for all beings and that kind of embodies is this value what we can't make sure of anything and I would say if if we don't create singularity net if I decide to go do something more relaxing with my life instead then how do you know for sure that you know sheesh imping Vladimir Putin Donald Trump Google IBM $0.10 all the companies out there how do you how

are you sure that those guys are going to create an AI which is for broad human benefit and if everyone stops making AI how do you guarantee that no one's going to like send synthetic viruses out there to poison everyone to death right so or that the proliferation of nuclear material in the you know Eastern Europe isn't going to be used to blow everybody up so I mean I think we're not in a point in human history where there's a great amount of certainty there's probably even more uncertainty than in the past and there there's there's always been a lot so but really that the question to ask is average are we better off creating a

decentralized you know benefit oriented AI platform like singularity net or are we better off I mean my gosh not having that there and then having all the other going on in the world right I mean that's that that's the question that's the question to ask so what I mean that's a fair point but then I mean that's just sort of rephrasing the questions a little bit so then I guess my question is well what are you doing in order such that you know this objective is and this value is embodied in the platform yeah I mean there there's two parts of that so what one part is in sort of the token ah mcc's of the singularity net ecosystem the other

part is in the AI is that the singularity net foundation itself is our building and putting into the network so I mean in terms of the token ah mcc's I mean there's curation markets and intelligent reputation system which is designed so that at least the agents that are interpreting value to the network are getting rewarded proportionally to that instead of having sort of game game theoretic dynamics where a few agents will accumulate all wealth which is what seems to be happening in Bitcoin and aetherium and is what happens in most conventional

economies also and then on top of that a certain percentage of the tokens that were initially minted are earmarked to be spent on benefit tasks as decided by the community which can be things like health care education medicine and so forth so there there's at least that nudging put in there to have a certain percentage of the token spent on things that are considered a broad benefit I mean then this is this is much like in government does when it when it spends some percentage of its well from on Social Welfare right it's just most most projects don't like wire that into their economic operation but then that the AI

is that we are putting into the network ourselves are largely benefit oriented so Sophie a robot which we talked about one thing we've been doing is using Sophia as a like a therapist and meditation assistant so that's you know that's not solving all the problems in the world but it's different than the Terminator right yeah I mean it's using a robot just kind of how people expand their consciousness and we where we're working on applying AI that's using OpenCog framework and wrapped in singularity net to analyze DNA data of people living 105 years or over to figure out what makes them live so long

to try to figure out how to extend other people's lives we're analyzing images of plants from China and Africa to try to diagnose spread of crop disease in early stages using using deep neural nets for image processing so of course each of these things is a drop in the bucket regarding what we need to do to massively improve this humanity but the hope is that by injecting these things in the network is at an early stage you're impacting the culture of the community because ultimately this is about the community that that you build around singular units so if if through curation rewards and benefit tokens and having a bunch of you know positive

beneficial stuff happening in the network and then you know our largest our largest development office is in Ethiopia in Addis Ababa where we have 20-something developers working on on singularity net so where we're trying to actively pull people from the developing world into development and use of the network so hopefully by all these things will be nudging the community in a in a positive direction which is really going to be the most important thing because I mean if if we're successful with this then you know five years from now the work done by singularity net foundation will be a relatively modest percentage of all the work being done to build the

pro/con the network over time and the AI is in the network will be mostly contributed by you know ever random people not not by people paid by singularity net foundation but then we are seating this comedian and we're seating this culture like you can see that in Linux right like Linus Torvalds and Richard Stallman and their friends from the old days right a very small percent of code in Linux right now but the culture of Linux is what it is because of how they started it right so we want to get you know beneficial motives and love and compassion and inclusiveness in the cultural DNA of singularity in the

community and then it will continue to be there in the code also and this is a bit you know soft and and fuzzy it's not like a mathematical guarantee of beneficial activity but I think that's that's how things actually actually have to work because in the end it's about the community of human beings who are going to be drawing developing this ongoing would well that's that's really fascinating and I think also the fact that you guys have actual people in Ethiopia working on problems in Ethiopia is really is great and a far far removed from what a lot of people the watching space are

doing this yeah Cardinal card on Oh is running a like a year-long education program where they're teaching a hundred young Ethiopian programmers Haskell that's called the programming language so I think well I've been I'm at this office in Ethiopia since what 2014 I guess doing AI outsourcing before we shifted them to singularity nits but now cardano's moved in there and yeah there's a lot of tech projects throughout various African tech hubs now so there's powerful forces of no centralization and wealth concentration but but yet there's there's also the the

opposite and that peer-to-peer and then positive globalization happening so it's a very interesting time where these two different forces are both both surging forward in powerful ways well so before we wrap up I didn't want to ask you one one last question and this is we kind of touched on this earlier when you were talking about a is making predictions so let's imagine the future now where you know a lot of a lot of the economy is is run on watching systems so you have you know powerful markets that exist exclusively on block chains and organizations and companies are interacting with these markets on block chains doing business on these markets

and these mark these markets are run by Dow so there are there are governance mechanisms in place which allow the companies that themselves that are operating on these markets to participate also in the governance through through stinky okay so you know the companies that use the markets also have staying on the markets and making participating governance decisions really protocol updates or things like this now it seems like there would be an incentive at this point and even for something like prediction markets for these companies to their stake to essentially delegate their stake to nai because the nei is going to make much better decisions on what types of governance or what types of proposals

that they should be making in order to maximize the network itself and also sort of like maximize their profits long term so it seems like there would be kind of a like a Nash equilibrium here where at some point you know if one company starts using an AI to manage their governance or to make predictions and other companies start using a eyes to make predictions and then when everybody's making predictions or making government's decisions with an AI as we move closer to general AI and it's like and you just have a eyes competing with the eyes and like I guess there's all extends charge now who's in charge of the world no no but nobody's in charge

right um which in some ways is good when you have presidents like Donald Trump out there right it's good that it's the whole collective self-organizing dynamic that's in charge rather than any any one person and who's in charge of Bitcoin and aetherium yeah we don't we don't we don't actually know that but it's clear it's concentrated in this small a small number of individuals and and investment groups who are controlling these things so yeah I think you know in the long term it's inevitable that you know if AIS are a thousand times more intelligent than human beings and of molecular nanotechnology and so forth it's it's

inevitable they're gonna have more physical power than humans right I mean it doesn't mean they're going to control every little aspect of what humans do in their lives but they're gonna have more more oomph than we do right so I mean it in the long term which may just be like decades from now we're gonna have two choices one is like you you wire into the network and become become one with a super intelligent global brain even if that means giving up many aspects of your legacy humanity or else you know you live in the in the people preserve like the squirrels in the national in the National Park and you know that the squirrels the squirrels in the park they can fight over girlfriends and hunt for

food and play and have fun then people are not trying to regulate every aspect of their little squirrel existence right on the other end if they run out of the park then I get rolled over by a truck right so I mean I think if you have a super human AI that's tremendously more intelligent than us either you join it or you're gonna remain living a happy human life hopefully with a lot of abundance provided for you but I mean in the end there's something much more powerful than you that does have some regulatory control when it when it needs to which could be good also right like if if the squirrels die of some plague will come in to give them antibiotics right and the same way of human society went too far awry a super AI that loved

us it would let us go about our business but if things went too far a riot might come in and and fix things so I mean that's that's a long term it's upload to the global brain or live in the people preserve right but I mean in the in the medium term it's going to be really really complicated and as you say there's going to be a gradual trend from human decision-making to AI decision-making but given human how profoundly so much of our political and corporate ecosystem is now I see that as a great opportunity to to improve things right I mean there's a lot of if the AI is written right it's going to do a lot better than the

individual humans and institutions and they're controlling things now so then it really comes down to you know creating the AIS that are going to be the the decision support systems for the the people controlling most of the world's most of the world's resources so [Music] reserve nature reserve is second I can chase whatever whatever thing is - and chase these on any encumbrance yeah yeah we're sitting we're setting aside to reach a region in that southern Ethiopia for this purpose so yeah yeah absolutely so I'm in the center of it all go to the

website singularity net dot IO and I mean that there you can find information on how to download and play with the with the beta if you're if you're a developer or we have a blog which has updates on their research pretty frequently we have a telegram discussion group which has a some percentage of interesting things on it and and some other things and so I think lots of lots of ways to get involved with the with the community and that you know I'm still as well as doing some actual work going around and speaking it at various conferences so I can meet meet some of us some of you guys listening there I think in middle of March we're having

token 2014 on conference here in Hong Kong so if anyone's there we can we can we can hang out but yeah this is all in the end while we're talking about building superhuman AI getting there is all about the human community right so we need people to be involved in many many different ways so join our communities online than the and then we're happy to talk to you about about what you can do to help out right well then thank you so much for joining us as a real pleasure talking and great and diving deep in this is really fascinating topic that we don't really always get a get much chance to discuss you're happy happy good fun yeah thank you for joining us on this week's

episode we released new episodes every week you can find and subscribe to the show on iTunes Spotify YouTube SoundCloud or wherever your listen to podcasts and if you have a Google home or Alexa device you can tell it to listen to the latest episode of the epicenter podcast go to epicenter TV slash subscribe for a full list of places where you can watch them listen while you there be sure to sign up for the newsletter so you get new episodes in your inbox as they're released if you want to interact with us the guest or other podcast listeners you can follow us on twitter and please leave us a review on iTunes helps people find the show and we're always happy to read them well thanks so much we look forward to being back [Music] [Music]