Golem
Devcon 0 · November 2014 · 18:29
Piotr Janiuk presents Golem's vision for a peer-to-peer market for distributed computing built around Ethereum.
Speakers
Topics
From Devcon 0, hosted by Ethereum Foundation.
Transcript
[Music] okay hi my name is Peter I work at imop in waro and we met with gab approximately in February I suppose and we had a short chat uh regarding our future collaboration and possibilities of us helping you with some parts of the etherum project uh it happens that I work at a company that specializes in compilers it specialized right then and it resulted in lvm implementation of evm that P is
going to describe but at the same time we had a little misunderstanding uh Among Us and when G described the eum our CEO remembered two phrases during completeness and virtual machine so after I don't know half an hour he thought well great that may be a super computer that's something that can distribute computation and I don't know locally run it so no not exactly it's more about the consensus and keeping a solid compact state not about Distributing computations so H okay so what can we do maybe we can provide
something like this and he called it Goen from Goen 14 it is a story written by stain L it's a machine that got some so large and complicated that it gained awareness of itself so okay nice let's have a world super computer if we cannot do it with each we can implement it ourselves U after another thought we come to a conclusion that it doesn't really makes sense because there are some attempts to implement something like this and they do not work so far well we have cloud computing but
this requires some structure over the machines and hardware and we cannot build a super computer or provide computing power out of the box using just would stay at homes of any people who would like to provide computing power so what to do we should give some incentive to them they should be able to make money and then maybe that that would work but still that's pretty hard because that would require registering into some services that would allow them to transfer value for the computing power that they would like to share so obvious idea is that it should be bued up Byer
or ethereum and that what we started to think about so the main idea is that we would like to provide users of regular PCS or whatever computers with possibilities of either Distributing their computation across a peer to-peer Network or to make some value by providing their uh resources either computation or storage or I don't know bandwidth for that matter uh and it should allow them to do it pretty easily
on both sides special specifica tasks may not be so easy but computation should be out of the box without any additional conf uration and they should get remuneration for computing something uh that's what's important is that we don't want them to uh get threatened by malicious nodes inside the network so all computations have to be separated from their host machines this is where virtual machines come into into role
and in general the task is some code well in fact any code that can be written in one of support languages that we can distribute across the network and run inside the virtual machine with some constraints because first of all we don't want to uh thread that Computing node and on the other hand we wouldn't like to create environment which would allow attack haers hackers I don't know whatever malicious Nots they may be in the network to attack other computers not necessarily the host one but some other we can imagine a scenario when someone prepares a Dos attack
distributed D of service just by sending a million tasks that would try to connect to some server so this cannot happen uh we started our proof of concept right now we implemented well I sorry for the noty graphics but that's how it looks like right now we implemented a few renderers as a backends and right now we have both open source renderers and commercial production renderers added to the project and they can you cannot see this here but right here uh there's a rank of
four Images the one shown here is the last one the last one is mental Ray I don't know if you know the under it's a commercial under and other three are two pbts and one vray vray is also a commercial and eror on the right P we see sap tasks which means that small tasks that were distributed across the network and are being gathered into a final picture so this is it um we believe that such framework can be used for something more than just rendering well in fact for any computation that can be distributed
and in in principle it should work as a map reduce computation model maybe bit more generic because we don't want just to distribute and gather results but to prepare a graph a CYCC graph of such tasks that can distribute their subtasks gather the results and based on that results send another list of tasks to perform under computation you can use it we we used it for rendering in for scientific calculations in hemistry it can be used for M integration easily even for Bitcoin mining if someone would like to do it the main problem with this approach
is that it require means of validating and Collective results because one once it's set to a network we have no idea who may be calculating and what value of the those results may have they may be completely invalid they may be partially invalid or simply valid so we need some means of getting the right results and collecting them into our the result that we need to use we can use so what they have in common Golem in etherum because we can use ethereum as a payment system but on the other hand there are some modules and parts of the source
code that seem pretty similar in both projects we would require a very good peer-to-peer module and transport transport layer along with node Discovery and resource transfer and very good resource sharing and management Machinery because when you write a program you don't only try to access local files or web and servers but maybe other computers in the network and so on so it should all be worked out but this that's the side of the Golem although I believe that some resource rank would be also required in
ethereum if parts of code are going to be sh across the network but from our perspective the most important thing is to use it room as a generic payment system because oh because because the traditional approach look looks like this a user have have to register to some service that allows him her to transfer value and this is a centralized solution requires giving out some private data that would be stored somewhere in some on some servers and what I don't know should be a problem
maybe it's not a problem you believe it is that microtransactions are not supported when we distribute a large task into the network it may be distributed across a bro subset of of this network and many nodes which may be well which is very difficult to handle when you use paper or similar Solutions and should be relatively difficult to implement with Bitcoin for example I don't know what's the scale of supported granularity of microtransactions in ethereum what is going to be but I believe that it might be much better than the current
Solutions we wouldn't like to flood ethereum with microtransactions so we at some point we have to decide what is the level uh at which we can process transactions because if it works out then it may be thousands hundreds of thousands a second so that wouldn't be feasible I suppose that's the another well it's not in groundbreaking neither in terms of the idea well although putting it all together is seem to be pretty difficult and in terms of iter contracts it's just I don't know
vanilla implementation of common contract that should take place in etherum it's just used to uh transfer value between participating nodes simple contract is as a contract between two nodes one that distributes task and the other that provides some resources for example computational power uh software as a service contracts may involve another parties for example programmers who provide their parts of code or in case of renderers companies that may distribute their rendering engines so that that that company or person also gets remuneration for providing solution and time dependent contracts for example
when you would like to H host something for some time and we have to pay in timely fashion or for hosting so from our perspective this is how it looks Golem andum are independent Solutions but to make Golem really attractive you have to be fully decentralized and well most more important easy to use and easy to install so forcing users to give out their data credit card data for example uh doesn't
seem to make any sense well at least much sense existing Solutions requires such registration and they doesn't seem to make uh a success with this attemp approach and as for ethereum I don't know this is just a guess but it's quite possible that that software as a service approach May attract more programmers to our eum I cannot guarantee that but it seems reasonable and that's how it looks if you just if you're not Bor with it I can show you how the graphic outline of the system looks like y
so it's it's not that complicated but it's not easy as well I don't know if there is anything similar to ethereum but we need a solid p top layer build on top of some transport layer uh we need network adapter to provide resource management and the global IO that would example that would allow nodes to read and right data in a safe fashion it has to be really well took over and implemented task protocol is protocol responsible for Distributing task across the network and gaing the results as well as
connecting it with a transaction system that's a payment system and that's this layer is all about the peer top Network sorry the upper layer is a client layer the center integration component was called the client before and it's responsible for defining tasks and put passing them to the layer that that's responsible for computation we can below this line we can test it just on a single machine running the task on a few processes or even threads so
that's good for testing purposes as you can see there's virtual machine that is separated by another border because we don't want to give direct access from the virtual machine to uh external world especially we wouldn't like to allow processes that might be written by someone to connect with any computer in the network above that we have task definition framework which describes in detail how tasks are specify yeah so you say that the individual instances uh cannot connect to each other so there are no shared resources so the there are shared resources but well I didn't
tell that explicitly the code that is run is just about any code you can write that we can share but we explicitly have to point which resources should be shared because we can for example try to download some data from a server and that's not a problem if you do it with one note but when you said distribute it over the network and every node downloads the same data that may be a threat to the server so in this case we can simply force uh the node that distributor task to download the data and then just distribute the downloaded data to Computing
notes what's sarting and grouping task interface is that interface responsible for creating a graph of map redu tasks and above that we have either user interface not similar to the one that I presented but the one one that should be easy to use and easy to specify tasks and multiprogramming language support which should I hope allow users to Simply write code using structure some some sort of standard standard library and control structures that are similar to uh existing control structures in languages for example parallel 4 that
should be a distribute the tasks and programmer shouldn't be really aware of what's going on down here okay and we believe that there may be many many many virtual machine flavors or maybe not technically but in terms of compute capabilities and that virtual machines should be defined by users which poses one problem that this does not fit the whole p feure as virual machines or the configuration fact uh would have to be distributed in a separate Channel not as a part of the
application but that may not be a problem at some point because we can allow users to distribute the data that messages that they have such machines and and interested party would be able to download it and use it okay I think that's it thanks pza [Applause]