Eva Beylin, Yaniv Tal : Decentralization in Practice via The Graph
ETHCC 4 · 22 Jul 2021 · 34:12
The Graph's Eva Beylin and Yaniv Tal discuss what decentralization has actually meant in practice for a protocol built to index and query blockchain data. Speaking at EthCC, they cover the tradeoffs of moving indexing infrastructure away from centralized hosted services toward a permissionless network of node operators. The talk treats decentralization as an ongoing operational challenge rather than a one-time migration.
Speakers
Topics
From ETHCC 4, hosted by Ethereum France.
Transcript
foreign can you hear me there we go Hey so uh thanks for having us uh Eva and I are going to be talking about Progressive decentralization today uh looking at uh what we've learned over the last almost four years uh building the graph um in case you don't know the graph is an indexing protocol for organizing blockchain data and making it easily accessible to applications and end users using graphql um we've accomplished quite a bit but in a lot of ways we're still just at the beginning so why you know worry about
decentralization well we in crypto are building the foundation for a modern Internet connected civilization and if we do our job right uh the things that we build are going to completely change how all humans cooperate and organize at a global scale creating a new wave of growth prosperity and opportunity and enfranchising people and systems that truly represent them and the only way to make sure that the things that we build uh actually last is to build them and to build protocols and applications in a truly decentralized way on top of truly decentralized
infrastructure so you know if you've been around the crypto space for a few years you've probably seen this type of strategy right where you come up with a great idea for a protocol you design this white paper it's super Byzantine fault tolerant you know has a teeny Genie coefficient you know you disappear for a few years and come out with a perfectly decentralized Network right uh obviously not um and you know in the early days it can feel a little bit like this right you're like we are a protocol right there's
nobody here but of course there is somebody there you know there's an initial team and even if you're building a decentralized protocol I think we should embrace the fact that these things are built by teams you you want to have a team of you know people with complementary skill sets that work well together that are committed for the long term um and uh and I think we should embrace that um so you know this is like roughly what you want Progressive decentralization to look like as an initial team um you're setting out you know okay we're going to build a fully decentralized protocol you know maybe
application um but instead of Disappearing for three years you want to break that down into uh Milestones each one of which is going to help prove something substantial and address some large area of risk so for example at the graph we started by building an initial version of graph node and the subgraph developer apis and we got that in front of developers within just a few months then we launched a hosted service that made it really easy for anyone to run their subgraphs in production only then did we finalize the design for the decentralized network so we had two full-time researchers working on the design for about 18 months before it was ready
then we launched we we built out all those Network components and we launched a test net then we launched the fully decentralized Network and just two weeks ago we launched a set of products it's the full self-service experience on top of the decentralized network so if you're using a subgraph on the hosted service today on ethereum mainnet you can migrate to the decentralized network so all of this has taken us almost four years and and I don't know that we could have done it without all these intermediate steps so the the path to Progressive decentralization can be broken down into
like roughly these five steps I'm going to cover the first three and then I'm going to pass it over to Eva so every protocol starts with a mission and you want to be really clear about what the mission is and a set of values for your community at the graph our mission is to organize and serve all of the world's public information in a decentralized way then from the beginning you want to be thinking about how are you actually going to be able to do this in a fully decentralized way and start breaking it down into individual components you know there might be some things that are harder to do fully decentralized from day one and that's okay but you want to be clear about how the system is going
to come together and you'll need to make difficult decisions from the beginning because it's really easy to you know paint yourself into a corner if you're not careful about this and end up in a situation where you actually will not be able to decentralize later so an example of this for us is we knew that subgraphs would need to run on a decentralized network of nodes which means that the computation needs to be deterministic so that meant that we couldn't allow developers to call out to random web services from their mappings because then that computation wouldn't be deterministic and we had a bunch of developers asking us for this feature over and over and we have had to keep telling them no and I think you know
making those difficult decisions and you know really having that commitment to decentralization the whole time is the only way to make sure that you will actually end up in a situation where you can fully decentralize so you know what your mission is and the second step is to raise to product Market fit so at this point you want to really focus and scope down to the bare minimum features that you need to put something in front of users at this stage you don't want to be thinking about things like growth and marketing you know I think there's a lot of pressure early on from you know folks like investors to like hey what are your
growth numbers and until you have product Market fit it's just too early for that at this stage you're really focusing on qualitative things so you want to make sure that your users identify with the problem you want to look at what's the friction in getting them up and running with the service see how their experience is and these are all really qualitative things you know using the service working closely with a small group of people iterating and uh and then looking at you know do they stick around right and that retention is really really key so after they go through this process are they really happy using the service um and are they so happy that they're
telling other people about it and if you can get even 10 people all the way through this funnel and you can see that it's really working you know congrats uh you're basically there you have product Market fit you can go out grab a beer celebrate and uh and it's time to to really hit the ground running and and fully decentralize and uh by the way that that process for us took about a year and hopefully you can do it a lot faster um you know building infrastructure is uh can be hard um so there are different aspects of decentralization there's you know the the technical aspects that go into your protocol design and then there's you know building out a community where you
actually have uh you know large number of independent participants that are contributing to the evolution of your protocol uh so that's the part that that I'm mostly going to focus on here so you've got product Market that you want to bootstrap an initial Community this is when you really go out there and evangelize a protocol right writing blog posts podcasts getting yourself out there and if you're good at marketing people that align with your mission are going to you know see that content and they're gonna you know come in through the door you want to provide really obsessive customer support for these people help them be successful and you want to start building out a set
of community leaders that are going to you know really help build out your community and and be these anchor points um you know Distributing power out and I think this is a really important part of the of the process and ends up making a really big difference and you know you should treat power like a hot potato right it's something you want to get rid of as soon as you can and so you want to really Empower and Elevate these people give them opportunities you know to speak to make decisions um and uh and that's how you really you know start decentralizing the power out and you know if these people are really good there's probably going to be a draw to also try to hire them and you know
we're kind of all starved for talent in the space but it's really really important to make sure that you you know have an active Community where people are building their own businesses external to the initial team and so you know making that decision is going to help you uh you know actually decentralize one one more thing I guess on uh on getting those uh Community leaders is if you identify someone that you think could be you know good at this get on a call with them and you know ask them you know uh you know what's your story What attracted you to the protocol I you know
what does success look like for you and the more you can understand you know their actual motivations the more you can help them be successful and and you really just want to do everything you can to support these people uh so what makes a good Community leader someone whose mission in values align it's really important you should be able to figure that out uh you know in in a single call Clarity of thought and communication this is something that you can see from people's writing and their Discord and it just makes such a big difference if you have people that are really thoughtful in all of their Communications that kind of sets the standard for all of the you know conversations and and discourse in your community um people with strong work ethic you know you need people that are going to
show up and actually work every day and and people that are kind you know it's it's easy to find people that are smart but you know are difficult to work with and and you want people that uh you know are genuinely nice people that are gonna you know go out of their way to help others uh makes a big difference okay so you've got uh you know the beginnings of a growing Community uh rallying behind your protocol uh you probably have a token and uh you're gonna think about how do you distribute value out to your early community members um you're going to run across a bunch of people on the internet that try to convince you uh you know to do a sort of fair launch which you know a lot of
times translates to you know give a bunch of people uh you know free money for for doing nothing and unless you're the mission or your protocol is to be a currency that has some kind of universal basic income you know I would push back on that you know your protocol has a mission and you want to use your token incentives to incentivize people to contribute value to the protocol and and push that mission forward um so you know I think that it's really important that Protocols are meritocracies uh so we want to start by B as open and inclusive as you can right catch that you know throw that net out far and you want you know people to be
able to find you be very open and inclusive anybody can participate but then you want to create commitment filters right filter people down to the ones that are actually you know care about your your protocol and your mission so for example at the graph we had a a series of programs to help you know distribute tokens out to community members and we require people to do a lot of work and we had forms that we would put out and people need to fill out these forms and we would go through them and you could really tell you know who's an active participant who's being thoughtful who's learning and growing and contributing um and that that makes a big difference and then you really want to have a culture of rewarding participants
proportional to the value that they're creating eventually you want all of these incentives to be designed into the protocols but in the early days it's okay for the initial team to exercise judgment because there's going to be a lot of creative ways that people come up with for you know how they can add value you're not going to know what all those things are ahead of time but if if you have a strong culture where people know that they'll be rewarded for the value that they're contributing they'll get creative and um and you know if if you're really uh you know have commitment to decentralization and and and you know the community sees that then you know generally they they get behind this kind of thing and it really
helps in the early days again creating this real kind of meritocracy um and uh you know as a protocol if you're able to design the incentives properly so that it really captures you know the the ways that people can contribute value then uh something really special happens which is that the participants in the network can actually stop thinking about the money and they can just focus on contributing value and pushing the mission forward and when that happens when you have that alignment it's a really magical thing and I think it's it's kind of the highest place that uh that a protocol can get to so to talk about how to create these incentives and how to scale
out I'm going to pass it over to Eva Balin [Laughter] okay so creating incentives uh as you've mentioned we had several programs to incentivize our community members and we had a test net and a curator program both of which were incentivized that started over six months before our Network launched and so what that allowed us to do was have a lot of knowledge sharing and people starting to actually develop the expertise that now a year later they are experts sharing that knowledge and recruiting new indexers new curators to our Network the next is future of work so we really
believe that the future of work is being a protocol participant and just because you're using passive Capital allocation that doesn't mean it's not a full-time job or a part-time job and so we're really stronghold about you know being an indexer curator delegator subgraph developer that's the future of work and your income can just be from the graph protocol the next one is knowledge and ownership incentives so intrinsic incentives are the best way to motivate people and what we realize very quickly is when you're building a protocol over a company everyone wants to have a stake in making that protocol and contributing and so giving people that ownership and allowing them to grow that knowledge to then be an expert in your protocol it really decentralizes away from the initial team
like you've mentioned empowering the community so allowing them to figure out which grants rfps which types of features are most interesting to them in creating that funnel of feedback with the either original team or the core developers and lastly cultural alignment so making sure that the people you're bringing into your community truly are positive for your network and your mission and aren't just there to stir things up So within the graph we have four key players we have an indexer this is who runs the graph node heavy devops experience very technical then you have a curator who can be semi-technical but ultimately you just need to understand in general what is happening in the DAP ecosystem so a curator will stake on subgraphs subgraphs are the open apis
that your favorite dapps use today and so for you to know which subgraphs are most high quality you really just need to understand the DAP ecosystem so for example the uniswap subgraph you know uniswap has a lot of users it's a pretty safe bet because you want that project to be sustainable and you can bet that the developers will maintain that subgraph the delegator role is the least technical so really you're just staking on an indexer so that way you can contribute to the security of the protocol without actually running the hardware yourself and they actually earn a portion of the query fees that indexers earn as well and each indexer can set their own delegation parameters so that's how they compete for the delegator's capital is actually what kind of features are they offering um you know and the the average cut is
around 10 percent that delegators earn from the indexer cut and the last role as a consumer so traditionally this would be adapt basically the one that's paying the query fees but long term this could be end users if we're able to start having gateways at the client side for you know a user on their mobile phone website or desktop whatever it might be but at the moment we expect dapps to be subsidizing those query fees similar to daps paying for AWS or other cloud services and lastly I'd like to talk about the graph as all-encompassing so really no matter what your skills are there should be some role you can do so if you're very non-technical but want to contribute to the data economy you can be a delegate or curator if you're heavily technical indexer might be the
right role for you the last one that's not listed here is the subgraph developer you know the open source soldiers and they're the most critical because all the open apis the graph Network itself has been built by them so the graph Network itself is you know a self-sustaining machine you have the indexers processing the queries you have the curators signaling on the subgraphs and that lets indexers know which subgraphs to prioritize whereas in a centralized world you usually have an intermediary like a centralized company telling you which apis are the priority here you rely on signaling from curators and so we find that actually the the gross amount of signal is is pretty fairly distributed across the highest
highest quality dapps most used apps so having launched the network in December we've already had over 150 indexers supporting the network and over 6 500 delegators and it's around the world so we're soliciting over uh you know 30 different countries but we're actively trying to grow in Africa and Asia and South America especially curating web 3 together so what's really exciting about the graph is it's really a mission that's bringing everybody together because one subgraph not only services that initial dat but perhaps every other dap that wants to query that API so I really encourage everyone to
consider the graphs Mission as United Mission because you know we wouldn't be here without the dapps and the dapps wouldn't be here without the graph but we already have over 400 curators only having launched two weeks ago in over 100 subgraphs that have migrated and so right now over the next few months the graphs Focus will be helping all those daps that currently rely on the hosted service to start migrating their subgraphs to the decentralized network and then having curators start signaling on those subgraphs so this is the data so far curation having launched again only two weeks ago we're finding that the live peer subgraph is seeing the most attention uh this is pretty interesting just from a historical perspective because life peer was one of the first adapts and a group of developers that really got into the
graph but the other Trend here is uh you know a lot of defy so most of the apps here are defy and also just uh you know understanding how opinionated people might be on the dapps that they want to support long term so you know when you curate on a subgraph you actually incur attacks and that's to disincentivize you from you know either being scammy or signaling on a subgraph you actually don't want to support and so to me this shows that our community has a lot of support for a curve and pool together and really wants to see those projects secede long term in looking at query fees we actually see that Uma and open are leading the way and this signifies that those subgraphs are receiving the most usage so query fees would be the the query uh you know
Revenue that is earned from you know processing those queries for those subgraphs and so that would be any applications that are using the Uma open Omen subgraph and you know are contributing to this value and so in the last month we've seen over 100 000 GRT in query fees which is really exciting and excited to see you know how that grows as we get more migration from the hosted service and this is a fun graph just to see the comparison and Trends so you know you can see Uma leading in fees but not leading in Signal you know what what kind of uh implications can we come from that and then on the other end you have rye and curves that have a lot more signal but not as many query fees yet the rise subgraph specifically I believe had a breaking change and maybe that's
why but other interesting uh you know observations is just the similarity across the board so you actually have quite a bit of consistency you know with dodo pool together enzyme M stable signifying that you know there's pretty even distribution around usage and also confidence so when we think about scaling out as I mentioned subgraph migration so helping the DAP teams that we know and love migrate to the decentralized network this is also beneficial for you as a user so if the DAP is relying currently on the hosted service you won't have to experience any more downtime if you migrate to the decentralized network we also want to start expanding the pie so we have a lot of sub graphs and dapps that are built you know on the same
types of protocols around the same ideas but we need to start growing the pie outside of the use case as we know today similar to what vitalik was saying earlier we want to start expanding the community more so we'll be launching an advocate program and also being more effective in building in public so this idea of protocols over companies and how do you start involving more people in the decisions around the protocol and lastly we're going to go gung-ho on decentralized governance 2.0 so you guys might have heard we did a grant to streaming fast for 60 million dollars over several years and the purpose of this grant was the streaming fast team is has significant expertise in streaming data improving indexing performance and so they'll be focusing on improving that for the graph node
we're also looking to you know invite anyone else who's interested in working on the core developer you know protocol or you know operations whatever it might be related to core development we're actively improving those processes second we're going to be expanding the grants program to be more decentralized so look out for more info on what that kind of Grant style might look like third we're expanding what the definition of a council might be so right now we have 10 folks on the graph Council representing all the five different stakeholder groups and this is sort of you know to enable that decentralization while having representation but we'll slowly start to expand that and see how can we involve more stakeholders and just widen the pie of decision making lastly a self-sustaining protocol you
know we are hoping that because we have such a wide community of skills and different expertise we can have people starting to teach each other instead of the initial team and so we're already starting to see this with over three organizations that you know are dedicated full-time to either indexing support subgraph development whatever it might be and really allowing them to start sustaining the protocol without the core team and lastly the trend of uh you know decentralized subgraph governance is really interesting and we're starting to see that Prevail um a few sub graphs like air Swap and maker Dow you know those communities are already migrating fully to a dow so any decision made about their tooling or their middleware is already in a decentralized process and we expect that as subgraphs surpass just being for one
protocol you know maybe a D5 subgraph you know we'll see a lot more decentralized governance of that and what is you know a decentralized governance of defied data for the world look like and so we expect to see a lot more iteration there either Dows multi-cigs maybe more bonding curves awesome thank you so much for having us and we hope you joined the movement and please let us know if you have any questions hello hi uh I'm I had a just chat with you about
potentially migrating ens my question is how do I simulate the clarity based on current usage on hosting that's first question the what's a plan for decommissioning in hospital service yeah so we've purposely tried really hard to not provide too much guidance about what where to expect the query fee Market to resolve I think because we've tried not to impact the query Market too much um you know it's an open competitive marketplace where indexers are competing to provide the best service at the lowest price and what we didn't want to do was randomly anchor people to a price because we just pulled it out of thin
air and instead we want the actual competitive Dynamics to play out and then see where those prices kind of resolve so I I totally recognize that it makes it harder to like plan because we don't know exactly what the numbers look like now we are starting to get data back but I don't know how much indexers have been fully optimizing their cost models and so I you know I feel like things could easily change by a factor of like 10 you know when you're at this stage of like you know early days and and establishing a brand new market so I could pull out a random number and you know out of a hat but I really think we should like watch the data come in and it's going to be an exciting few
weeks and then we should have a lot more clarity and we can you know tell people um you know more more concrete projections but but the general idea is it really should end up being cheaper right than running servers yourself um you know for one you have these service providers competing to provide the best service to the lowest price and there's also um you know Network incentives through new token issuance that's basically subsidizing their operations right so they get paid indexing rewards just for indexing subgraphs and that you know really covers a lot of their costs so that also allows them to to price really aggressively I would also add that you should just get more involved you know if you're interested because you as a subgraph
developer adapt developer are the demand side and so as much as indexers are still trying to figure out what they want to charge you you know as the user should also start maybe helping figure out what you'd be willing to pay for the um so I do think it's a two-sided process yeah add on to that point is that so currently everybody is using hosted service so as a Dap I have no insight about the analytics of how many query going so it's kind of two separate things and why is it almost like ethereum transaction cost and a gas price of two separency and gas price goes up and down but I kind of need a indicator of how much query ens is getting right now because you know our usage could be quite different to others
but we have no data points right now and it's different to how index size efficient to index yeah that's a that's a great point so we can definitely talk about how to you know expose some of those stats to the developers so that you uh yeah you know for example query reviews or query volume on the hosted service for sure question about price you see because I read on Twitter that rotkey had a problem since uh the next update you needed to pay individually for queries and my question is uh how do you guys
see that a work around the privacy issues so uh in this version of the graph Network there is a set of gateways that are geographically distributed that when you run a query it gets routed to the nearest Gateway and then from there to a nearby indexer and that's to allow the DAP developers to pay the queries so in from a privacy perspective that works somewhat similarly to like any other sort of web service so for example people that use the hosted service you know of course that traffic goes like to the hosted service servers you know ultimately I think in web 3 users are going to want to express preferences
around you know which service providers they use and so they'll be able to configure those types of things where they can say hey I only want to use these types of indexers or you know I don't want to use those um you know in the case of something like a rock key which I guess is is meant to be like a local first application you know graph node itself is open source anybody can run their own indexer themselves so if they want to you know do that for whatever application they have that option but in general you know it just would be impractical for like every person to run like the entire web you know on their machine right and so you always need to Outsource to some external service
providers and so when you have a decentralized network the users can choose which service providers they use for what don't know great thank you