Vector Search for Decentralized Systems

Transcript

Hello everyone. Uh it's really really great to be here uh to talk about I think a foundational topic which is simultaneously both so important but also very uh overlooked in the industry and what I'm talking about when we talk about AI we want to jump right into talk about agents. I want to talk right about generative AI but we forget something which is absolutely foundational. What what am I talking about? I'm talking about vector embeddings, talking about the vectorzation process and the whole uh story behind that. So, we're going to spend about 18 minutes and 48 seconds together talking about the vector process and why it is absolutely so foundational. But the way in which we're

going to begin that welcome, have a seat. Hey, good to see you. The way we're going to do that is we're going to start with what is such a common situation. I don't care if you are in web 2, web 3. If you're somebody's father-in-law or mother-in-law, you've all experienced the issue that you have your favorite thing and your favorite thing breaks and you want to find that favorite thing again you purchased years ago. So, you go on your favorite uh browser and you start searching for that thing. You don't remember what it was called, but you remember what it does. You remember the description of it. So, you start writing about it in natural human language. Do you think if you write about your product in that kind of language, what are the chances that the user, if the user is you or someone

else, hey, good to see you, will come into that product, will find that product. Well, pretty slim, right? That's a significant problem. And in fact, 61% of users are going to leave your application if they can't find what they need to find within how many how much time do you think? Just shout out some time. two seconds. You are Wow. Wow. You're trying I You said it, not me. I didn't say it. That's not It's recorded. I did not say that. Uh with actually about 5 seconds of time. You have about five seconds. Unless you're this guy over there. Then you better load that f you better be using service workers and load that application really

fast for you. So about five seconds you have to find what you need for that person. But the problem isn't just about ambiguity in the way people talk about things or search things. The problem is also with our data. Not only is not only are people ambiguous or don't describe things well or form data incorrectly or not well when they speak it. Data is also malformed. Data is also ambiguous. Data is all over the place. If you've ever had to reconcile data as part of a job, anyone ever had to do that?

reconcile like spreadsheets or reconcile data. Oh my god, I still wake up like in the middle of the night with sweats of remembering that time my first developer job where I had to reconcile like things old like uh uh basically CSVs that were like sales data and it was just all over the place. So data itself in both web 2 and in web 3 is ambiguous. There is data ambiguity from the person and from the data itself everywhere. So, first of all, before we get into how we solve that, my name is Ben. I work at Arbitum as a Devril engineer. And you can find me basically at all places. Just put in the prefix GitHub, Twitter, Farcaster, it doesn't matter what it is. Find me at hummus on rails at all places. Just

that's the handle. Very easy to find. And I'm uh the author of the upcoming book vector search uh with JavaScript coming out actually next week. So, if you sign up for that mailing list, which is my mailing list, I've not sent out a single email the entire time because I don't believe in spamming anyone, but I will be sending out the first email on that mailing list this coming week with a discount code. It is dropping on either Tuesday or Wednesday. I can't remember exactly, but I think actually Wednesday. Uh, coming soon. So, I I like this topic. Obviously, I spent a lot of time writing a book on this topic, which is a little crazy because it's about search, but whatever. I got into the rabbit hole and I didn't leave the rabbit hole. Um, okay. So, why do we want to talk about vector search?

Because we often want to talk about agents. We want to talk about AI agents. We want to talk about generative AI, but vector embeddings are actually the foundation for everything that we want to build on top of that. Vector embeddings is where we have to start. If we don't get that right, we're not going to get the rest of it right either. So, it's really, really important. But what actually is a vector? Who's heard of the term vector? These guys right here were like, "Yes, we we love it." Aden's heard of it. Anyone else? You've heard of term vector? Amazing. So, a vector is a list of numbers. That's all it is. An array of numbers. A vector embedding is a list of numbers that represent meaning about

the data. That meaning can be what? It can be contextual. It can be semantic. It can be any kind of meaning within that data. And what kind of data? Text, video, audio, images, any type of data. So a vector is a list of numbers. A vector embedding is data. Those lists of numbers that represent things, represent the meaning behind things. So when you do that, you take a search in natural human language and you turn that search, that query into an embedding to a vector embedding which then is used to find data related to the meaning behind what the person is looking for. That's pretty cool, right? That's a little

better than than traditional keyword search. And for many contexts, not for every context, but for many contexts, it's it's quite better. And so your data of your application goes from that into something that looks more like that. That becomes your data set, right? And so the search becomes its own embedding and your data set is its own embedding. And that that helps find the meaning and the relevancy of what people are trying to find. That all happens through a process that involves um closeness. Now, when we say closeness, what I'm referring to is not how close you were next to that really sweaty person at that beach party last night. What I'm talking about is mathematical closeness.

Closeness in what's called vector space. When you have two things that are near each other, that tends to signify that they are more like each other. Things that are farther apart would be what do you think does? Ah, guys, you you're brilliant. amazing are not so like each other. Things closer more like each other. Things farther apart less like each other. The way we get to that and that's kind of what we're talking about, right? There's a little bit of visualization. The way we get to that is with math.

Now, if you were like maybe all of you were not like me, but if you were like me when primary school, I hated math. Absolutely hated it. I thought I was awful at it. But I realized I was awful at math that had no purpose. Math that didn't feel relevant. And that's the way they taught math. It just felt like I was memorizing things and it was really boring. But math with purpose is actually utterly fascinating, at least to me. And so it's all revolves around math. And the math that we're talking about is math that came up in primary school, in middle school, in high school. This is math that you probably have all heard of. Things like ukitian distance, cosine distance, Manhattan distance, dotproduct. These are all ways in which to calculate the closeness. If

we had more time, we would talk about the actual ways this works in deep detail. And if you Google this talk, variations of this talk, you can find, not this exact talk, I love you. It's a unique talk just for this event, but variations of this talk, you can find a deep dive into each one of these and how they all uh work in relation to what we're talking about. But so you take all these math formulas and you essentially get that distance calculation. But how does that get to what we're talking about? How does that power AI? How does it actually help? Well, it helps in many ways because that is the source behind what takes data and turns it into a vector embedding which gives it the meaning, the context and gives us the

understanding. What is called an embedding model. An embedding model essentially takes all the data in it. It goes into this black box which my friends it is not a black box but it goes into what many people think is a black box. and out comes the vectors. Embedding models capture that context, the relationships between data, the meaning and the intent. And they do that through transformer layers. This here is an example of one popular uh embedding model. Uh I don't know if you've heard of it. You may have used it at some point today. It's called something like three letters like GPT. Anyone has heard of GPT? It's it's totally not used.

Everyone uses BERT nowadays popularly. No one uses GPT. But generative pre-trained transformers, right? So essentially you have this is an example of predictive text which essentially powers things like coding assisting a coding assistive agents uh text generation all kinds of stuff like that. You have your input it gets tokenized. Tokenized means it gets turned into its smallest little bits. Those go through different transformer layers and number of transformer layers which then in this example gives you an auto reggressive prediction of what comes next. That's an embedding model. They're really powerful and they do amazing things and you're probably using them all the time.

They're kind of basically like a neural network. See this guy right here? He's kind of reflective and thinking a little bit. He's probably thinking about a conversation he had on AI, I don't know, maybe today, couple days ago. Something I said reminded him of something and he's connecting the dots. He's doing what an embedding model does. He's doing what an embedding model does or I should say an embedding model is doing what he does. So, it's kind of like a brain. It's kind of like modeled after the way our brains work, creating connections between data. That is really, really powerful. And I'm going to say something that feels a little bombastic, maybe a little audacious, but I actually think it is very true. And it's it's

incredibly true and so important and why I actually care about this subject so much which is whoever controls the embedding models whoever controls those models ultimately will control the understanding layer of the internet. That is really true. Whoever controls the training of embedding models controls the understanding layer of the internet. That is significant. We use them all the time. But where do they come from? Who builds them? Who controls the access to them? Who has oversight over them? These are huge questions which honestly because we want to jump to agents so fast, we tend to forget

about this layer which is so so deeply important. So let's talk about that because as we mentioned embedding models are the understanding and context for things like retrieval augmented generation for rag which powers generative AI. It is the the the memory layer for agents. Right? So two of the most prominent use cases of AI that we have in our world today. Embedding models provide significant parts of what powers both of those areas. And just to briefly summarize in case you're not familiar, who is familiar? Who is not familiar with rag?

Ah, see this is why I put this slide up here. I love it. You are welcome and I love you all. Okay, so rag retrieve augmented generation is a basically taking a query. Let's say you ask um an AI chatbot um how do I cook fetuccini alfredo? And the AI chatbot, unfortunately, is not an Italian grandmother and it doesn't know how to make fetuccini alfredo. Without additional context, it will give you an answer as if it is a confident 45-year-old white man. And it will tell you with utter confidence exactly how to make fetucci alfredo. And you will end up making macaroni and cheese. And that's amazing. Mac and cheese is fantastic, but it's not fetuccini alfredo. but with context

given to that AI chatbot. Well, then now the AI chatbot has context on Italian recipes and it takes your query adds the context and generates the answer. That's retrieval augmented generation. That context though needs to be put into what? What are the two terms? Two words a vector embedding. Amazing. It needs to turn into a vector embed that context so it can find that context and add it in by itself. And this happens all the time. A lot of developer docs, for example, are being powered in their search queries by things like rag, right? When you're searching, often now you don't even realize it. You're searching and it's being powered by retrieve augmented generation to give better answers.

anyone ever used developer doc uh search bars in the beginning of like AI craze and saw how awful they were, how much hallucination there was, how much just like and you would go into developer discord and it was just like angry, frustrated developers trying to find something and the docs were like totally off. That was without rag and with rag you actually get much more accurate, grounded, contextually aware responses. Now when we get to agentic AI which I know is like the most awesome thing and everyone loves agentic AI we get to agentic AI vector embeddings play significant role as well vector embeddings well they take agentic AI what's the difference between a gentic

AI and a chatbot do we know it's three things aent a chatbot the difference between a chatbot and agents is a chatbot has memory it can access tools on its own and it can plan so it has the context It has tools that it can manipulate and use like a human or a chimpanzee or an anteater or something else. I don't know why I said anteater. It sounded fun. Um tools and it can make steps. It can plan out an order of action that it will take. And you probably seen these things. You understand these things. That's the difference between a chatbot and AI and an agent. Agentic AI. Where does the memory come from?

you know memory vector search vector embeddings are the memory layer of AI agents that's what gives it its memory so the memories are embeddings which helps it understand and ground itself so it's one leg of that threepart uh table that three-legged table of AI of agentic AI so vector embeddings without proper vector embeddings it can't get its memory can't get its context so it doesn't know which tools to use which means it doesn't know how plan. So, it's the basis, the foundation.

But you probably already know this, at least you know it from practice because you're probably using who writes code at all? Uh, basically everyone. If you don't, you're welcome as well. Um, if you're writing code at all, you probably have seen some of these things, right? the granddaddy GitHub copilot cursor windsurf uh code rabbit which is like automated agentic PR reviews um Kodto which is like another testing AI agent thing there is like so many of them and more ones just keep on coming out all the time so you're using these all the time you understand exactly what happens there but have you ever wondered when you're using them who actually owns the

memory who controls the waiting of the data, scoring the data, training the models that is giving you the output that you find so helpful. You ever thought about that? I would hope because we're in this space that you have. I would hope that since you're in this world of blockchain and web 3 and you care about the future of the digital common that you have thought about these things, but maybe the whole world's not going to think about these things. Maybe this won't be the predominant way that we access agents in the world. Maybe open AI and its competitors will win the majority space. I don't know. That's a pessimistic view, but maybe. But if they do, we must and ought to have a carved

out niche, an area for people who do care about like decentralization, who do care about privacy, who do care about uh consensus and people building together. So even if that wins, not saying it will, but even if it does, we must retain that space. And I hope you'll be part of that journey too because it's super super important. So the memory layer is actually decentralizing. And I just want to offer you three project areas that three projects that are working in that. There are actually many projects working in this area. Caveat, I don't work for any of these projects. I have played with these projects. I don't work with these projects. Uh buyers beware. You do you.

They're I think they're pretty great, but I don't represent any of these projects. So, one project uh three of them are one is called Arbius. It's a compute layer, decentralized AI compute layer. I'll give you QR codes in a second, don't worry. One is centralized AI compute layer. The second is called Carve. It's a data layer. And the third is Olif. It's basically a plug-and-play infrastructure stack. It offers both things like storage, uh decentralized virtual machines, and uh encrypted memory. all the foundational tools that you need to build out proper vector embedding models. You need your compute for the training, you need the storage, and you need um and uh and you need the VMs, you need to actually have it an executable layer. So all the things that

you need, these three are offering either all of it or some of it in their p in their package offerings. So, for example, uh for example, like you're really just starting to see essentially u this beginning to become a real possibility. you're really starting to see like as these ecosystems emerge and evolve like another option open up for you that is not just using Facebook's models or open AI models or Google's models but something from the community that is open that's transparent that is not opaque from because it allows you to go from things that are like gated closed systems where you don't know what's happening you don't know how things are getting weighted in go to gated closed systems into transparent

open reusable plug-and-play text stack. And so, let me just show you a couple links. This is a quick start guide for Carve. Uh, it just takes you right to get right get started with Carve. It takes a few minutes. I would scan that QR code. I see you did it. That's amazing. You're doing it right now. Nice phone. Don't know what it is, but it looks cool. Um, and you're doing it great. And then another one is from Olivecloud. Olaf. Another quick start guide. Again, we'll take you the Olivecloud's that plug-andplay full infrastruct. They're offering the VMs, the the storage, and the compute. The VMs, the storage, and the compute. Yes, I said all three. Uh, so they're offering all three as a quick start guide. And that is there for you as well. And really, if I can leave you with one thought, the one thought I want

to leave you with is that when you are using your next AI agent, when you're asking a developer chatbot, how do I do something? No matter what it is, I want you to take a step back and think about who owns that memory layer because that memory layer is integral and will kind of basically dictate the future of the internet that we all use. So, thank you very much. I would encourage you to also join our developer newsletter. Uh we have launching it actually next week. My book is launching next week. The newsletter is launching next week. It's a big week next week. Um, and you can sign up for the pre-release of the newsletter. It won't be more than once a week, probably once every other week into your inbox with developer tools, tips, new projects, etc. So, thank you.

And if you want to stay in touch, you can find me at all places on Rails. Really appreciate it. Have a great rest of the day. [Applause] On time. You didn't hear that?