Vector Isn't Enough: Hybrid Search & Retrieval — Jeff Vestal & James Williams, Elastic

Thank you for coming. Thank you for coming. My name is Jeff My name is Jeff My name is Jeff Estelle. I work at Estelle. I work at Estelle. I work at Elastic. I am an AI architect. Elastic. I am an AI architect. Elastic. I am an AI architect. We chose new We chose new We chose new job titles this year to be job titles this year to be job titles this year to be relevant. relevant. relevant. This is James. Would you like to This is James. Would you like to This is James. Would you like to introduce yourself? introduce yourself? introduce yourself? Yes, I'm James Williams. Yes, I'm James Williams. Yes, I'm James Williams. I am a solutions architect I am a solutions architect I am a solutions architect specializing in specializing in specializing in search. So I search. So I search. So I help clients with help clients with help clients with all the technical all the technical all the technical issues you'll issues you'll be looking at today. This is a hands-on This is a hands-on workshop, so I don't workshop, so I don't workshop, so I don't plan on plan on plan on talking much. talking much. talking much. I've been doing a lot of I've been doing a lot of I've been doing a lot of speaking on trips lately, speaking on trips lately, speaking on trips lately, so I'll be so I'll be so I'll be speaking less now. We speaking less now. We speaking less now. We will do will do will do practical work. practical work. practical work. I want to point out that in I want to point out that in I want to point out that in the upper right the upper right the upper right corner you see a corner you see a corner you see a short URL. This is short URL. This is short URL. This is ELA elastic. This is a ELA elastic. This is a ELA elastic. This is a workshop environment. Ahem. A workshop environment. Ahem. A workshop environment. Ahem. A brief note. brief note. brief note. I wasn't told I wasn't told I wasn't told how many people would come. how many people would come. how many people would come. So I randomly So I randomly So I randomly guessed 75. I guess guessed 75. I guess I was a little off. I was a little off. I was a little off. So, 75 of you will get things up and running So, 75 of you will get things up and running So, 75 of you will get things up and running pretty pretty pretty quickly. Those who are a little quickly. Those who are a little quickly. Those who are a little late, late, late, please wait a few please wait a few please wait a few minutes for everything minutes for everything minutes for everything to start. Overall, to start. Overall, to start. Overall, this will put you in a this will put you in a this will put you in a learning environment learning environment . And then I'll make a . And then I'll make a . And then I'll make a little introduction. I'm little introduction. I'm little introduction. I'm just saying this just saying this just saying this while everything while everything while everything loads. Um, loads. Um, loads. Um, we'll get to that we'll get to that . You go here, . You go here, . You go here, click "start," and click "start," and click "start," and it will start it will start it will start the environment for you.

the environment for you. the environment for you. While it's While it's While it's loading, loading, loading, tell me, who is familiar tell me, who is familiar tell me, who is familiar with Elastic? Elastic, with Elastic? Elastic, with Elastic? Elastic, Elastic search. Okay, Elastic search. Okay, Elastic search. Okay, not bad. Good turnout not bad. Good turnout . And how many of you . And how many of you . And how many of you have used Elastic have used Elastic have used Elastic to build to build to build applications based on RAG, applications based on RAG, search-augmented generation? Perfectly. Who among you has already Perfectly. Who among you has already gotten tired of the term RAG over the gotten tired of the term RAG over the gotten tired of the term RAG over the last 2-3 years? last 2-3 years? Fantasy. I know I Fantasy. I know I said it said it said it about a thousand times about a thousand times about a thousand times at many at many at many conferences, and my conferences, and my conferences, and my former boss got former boss got former boss got really angry when we really angry when we really angry when we kept mentioning RAG kept mentioning RAG . The fact is that . The fact is that . The fact is that the generation with the generation with the generation with the addition of search has the addition of search has the addition of search has not disappeared anywhere. not disappeared anywhere. not disappeared anywhere. Fortunately, it simply Fortunately, it simply Fortunately, it simply faded into the faded into the faded into the background, but as background, but as background, but as agents became agents became agents became more capable and more capable and more capable and efficient, they efficient, they efficient, they took over the took over the took over the task of task of task of obtaining data and obtaining data and obtaining data and analyzing it. So the models analyzing it. So the models analyzing it. So the models are getting are getting are getting smarter. You know, smarter. You know, smarter. You know, Fable came around for a moment, Fable came around for a moment, Fable came around for a moment, teased us, and disappeared, teased us, and disappeared, teased us, and disappeared, but the need to but the need to but the need to load load load information into these information into these information into these models still models still models still remains. It remains. It remains. It could be, you know, could be, you know, could be, you know, things that are things that are things that are time-dependent and changing, time-dependent and changing, time-dependent and changing, confidential confidential confidential information, pricing information, pricing information, pricing data, anything that the data, anything that the data, anything that the model hopefully model hopefully model hopefully hasn't been trained on, whether hasn't been trained on, whether they tell us so or they tell us so or not. But there is a not. But there is a not. But there is a need for this. And this is where need for this. And this is where tools like tools like Elastic Search come to the rescue. It is this level of Elastic Search come to the rescue. It is this level of Elastic Search come to the rescue. It is this level of data acquisition that data acquisition that data acquisition that makes these agents very makes these agents very makes these agents very effective. And effective. And effective. And the trick, or the trick the trick, or the trick , is probably that , is probably that , is probably that if you " if you " feed" these models, no matter how

feed" these models, no matter how feed" these models, no matter how smart smart smart they are, they are, they are, garbage data and garbage data and garbage data and confidently tell them confidently tell them confidently tell them that this is the answer, that this is the answer, that this is the answer, they will almost certainly they will almost certainly they will almost certainly believe you, because that's what believe you, because that's what believe you, because that's what you're telling them. you're telling them. you're telling them. So we want to be So we want to be So we want to be able to provide able to provide able to provide accurate information and accurate information and accurate information and do it right. do it right. do it right. So that's what this So that's what this So that's what this seminar is about. I'm sure there seminar is about. I'm sure there seminar is about. I'm sure there are 20 other are 20 other are 20 other workshops that workshops that workshops that explain how explain how explain how to build agents, and to build agents, and to build agents, and we have those too if we have those too if we have those too if you go to our you go to our you go to our site, but this workshop site, but this workshop site, but this workshop will remind you about will remind you about will remind you about vector search, vector search, vector search, semantic search, semantic search, semantic search, why it's important, where it's why it's important, where it's why it's important, where it's useful, and also where useful, and also where useful, and also where it breaks down. And then it breaks down. And then it breaks down. And then we'll move on to the we'll move on to the we'll move on to the BM25 lexical search. BM25 lexical search. BM25 lexical search. This is the same keyword search This is the same keyword search we've been using for we've been using for decades. Where decades. Where decades. Where it remains it remains it remains relevant and relevant and relevant and useful today. And useful today. And useful today. And also, again, also, again, also, again, where it sometimes fails where it sometimes fails where it sometimes fails . And then . And then . And then we'll combine them into a we'll combine them into a we'll combine them into a hybrid search. To hybrid search. To hybrid search. To get the best of get the best of get the best of both worlds. We will both worlds. We will both worlds. We will make combinations make combinations make combinations using RRF, which is a using RRF, which is a using RRF, which is a ready-made solution, and ready-made solution, and ready-made solution, and then try a then try a then try a slightly more complex slightly more complex slightly more complex linear combination, linear combination, linear combination, where these balances can be adjusted a little where these balances can be adjusted a little . And then, if . And then, if you have time, we'll you have time, we'll you have time, we'll try try try using the using the using the Elastic Agent Elastic Agent Elastic Agent Builder platform, where you can Builder platform, where you can Builder platform, where you can see what see what see what happens if you happens if you happens if you get bad get bad get bad information and tell information and tell information and tell the agent that it's the agent that it's the agent that it's correct. This doesn't correct. This doesn't correct. This doesn't work. That is, it work. That is, it work. That is, it works, but it gives a works, but it gives a works, but it gives a bad answer. And bad answer. And bad answer. And if you provide if you provide if you provide the correct...If you the correct...If you the correct...If you 're really quick 're really quick 're really quick and get to the end, there's a and get to the end, there's a and get to the end, there's a bonus section about, bonus section about, bonus section about, sorry, sorry, sorry, re-ranking. So, re-ranking. So,

re-ranking. So, reranking is where reranking is where reranking is where it becomes useful. it becomes useful. it becomes useful. The seminar The seminar The seminar essentially consists of four essentially consists of four essentially consists of four main sections plus main sections plus main sections plus bonus sections bonus sections bonus sections or tasks. Each of or tasks. Each of or tasks. Each of them has two parts. There is a them has two parts. There is a them has two parts. There is a part in the part in the developer tools console where you developer tools console where you simply run API simply run API commands in Elasticsearch. commands in Elasticsearch. commands in Elasticsearch. Each of these environments Each of these environments , if you are in it , if you are in it , if you are in it and and and click "start", click "start", click "start", launches a launches a launches a serverless serverless serverless instance for you. And then you instance for you. And then you instance for you. And then you upload the upload the upload the data there. This is a small data there. This is a small data there. This is a small data set, it's a data set, it's a data set, it's a workshop, so nothing workshop, so nothing workshop, so nothing complicated. You'll complicated. You'll complicated. You'll run a short run a short run a short version of each version of each version of each in the in the developer tools console, then developer tools console, then switch to the switch to the switch to the Python notebook tab and Python notebook tab and Python notebook tab and run the more run the more run the more verbose version. I'll verbose version. I'll verbose version. I'll shut up in a minute and shut up in a minute and shut up in a minute and let you let you let you work on the work on the work on the workshop, and every 20-25 workshop, and every 20-25 workshop, and every 20-25 minutes we'll minutes we'll minutes we'll check in on how you're check in on how you're check in on how you're doing. I will briefly doing. I will briefly doing. I will briefly summarize the tasks of summarize the tasks of summarize the tasks of the chapter you are the chapter you are the chapter you are working on. To working on. To working on. To get the most out get the most out get the most out of this workshop, I of this workshop, I of this workshop, I strongly recommend strongly recommend strongly recommend taking the time taking the time taking the time to read the contents of the Jupyter to read the contents of the Jupyter notebook. It notebook. It notebook. It explains why explains why explains why documents documents documents appear in the appear in the appear in the order they do, how to order they do, how to order they do, how to set it up, how set it up, how set it up, how to optimize it, and why to optimize it, and why to optimize it, and why it's important. If you it's important. If you it's important. If you just hit " just hit " run all" on your run all" on your run all" on your laptop and take a quick laptop and take a quick laptop and take a quick run, you'll run, you'll run, you'll be done in 10 be done in 10 be done in 10 minutes, but if you minutes, but if you minutes, but if you really get into the really get into the really get into the nitty-gritty, it'll probably nitty-gritty, it'll probably nitty-gritty, it'll probably take longer take longer take longer than this than this than this workshop. So, workshop. So, workshop. So, I'll let you I'll let you I'll let you start with that. One of start with that. One of start with that. One of you should start right away you should start right away you should start right away . Some of you . Some of you . Some of you will need a will need a will need a few more minutes. In the few more minutes. In the few more minutes. In the meantime, has anyone meantime, has anyone meantime, has anyone started yet? Sometimes it started yet? Sometimes it started yet? Sometimes it just takes a few minutes, just takes a few minutes, just takes a few minutes, it happens. Yes? We it happens. Yes? We it happens. Yes? We understand.

understand. understand. OK. And if you're OK. And if you're OK. And if you're stuck, raise your stuck, raise your stuck, raise your hand. I'm essentially your hand. I'm essentially your hand. I'm essentially your assistant, so I'll assistant, so I'll assistant, so I'll check what's check what's check what's going on in your going on in your going on in your system and maybe system and maybe system and maybe help you figure it out help you figure it out if it's something minor. if it's something minor. If you have any If you have any questions, don't questions, don't questions, don't hesitate to raise your hesitate to raise your hesitate to raise your hand. I'm happy to hand. I'm happy to hand. I'm happy to provide more details provide more details provide more details about the exercise if you're about the exercise if you're about the exercise if you're looking at something and it's looking at something and it's looking at something and it's a little a little a little unclear, or if you unclear, or if you unclear, or if you want to ask a want to ask a want to ask a question. question. And all of this is also And all of this is also available in a available in a available in a repository that should repository that should repository that should be public. Yes, I be public. Yes, I be public. Yes, I made it made it made it public. So you public. So you public. So you can run these can run these can run these notebooks outside of notebooks outside of notebooks outside of Instruct, after a workshop, Instruct, after a workshop, Instruct, after a workshop, or anytime you or anytime you or anytime you want. want. I missed this a bit I missed this a bit , but the main , but the main , but the main change in RAG over the last change in RAG over the last change in RAG over the last year or so is that year or so is that some some vector database vendors— vector database vendors— and we're in that category— and we're in that category— have added have added have added inference inference inference directly into the directly into the directly into the vector database. So you vector database. So you vector database. So you will see that the will see that the will see that the embedding generation embedding generation embedding generation happens happens happens inside Elasticsearch inside Elasticsearch inside Elasticsearch serverless. We have an inference service serverless. We have an inference service called Elastic called Elastic Inference Service—a very " Inference Service—a very " creative" name—that creative" name—that powers the powers the powers the models models models used in used in used in these workshops. So, these workshops. So, these workshops. So, we have Gina models we have Gina models we have Gina models through an acquisition through an acquisition through an acquisition that we made, that we made, that we made, so now we're so now we're so now we're actually doing actually doing actually doing our own our own our own model development. And, um, they model development. And, um, they model development. And, um, they focus focus focus mostly on mostly on mostly on dense vector dense vector dense vector models and models and models and reordering models. So

reordering models. So reordering models. So you will get you will get you will get an idea of ​​new an idea of ​​new an idea of ​​new technologies that were technologies that were technologies that were not available in Elasticsearch a year ago not available in Elasticsearch a year ago . That's all I can . That's all I can . That's all I can add on this add on this add on this matter. Have matter. Have matter. Have we heard of chunking? we heard of chunking? we heard of chunking? We have, we We have, we We have, we help with help with help with chunking. So, you chunking. So, you chunking. So, you know, we're going to know, we're going to know, we're going to ask about this, ask about this, ask about this, feel free to join in feel free to join in . He is the manager of a . He is the manager of a . He is the manager of a team of team of team of search specialists. And I'm just search specialists. And I'm just search specialists. And I'm just walking here. Um, so as walking here. Um, so as walking here. Um, so as James mentioned, James mentioned, James mentioned, we have, you know, a we have, you know, a semantic text field semantic text field in Elasticsearch. So, there are a in Elasticsearch. So, there are a in Elasticsearch. So, there are a few ways to few ways to few ways to do this. do this. do this. Semantic text Semantic text is a kind of is a kind of is a kind of automatic and automatic and automatic and simple way. Um, so simple way. Um, so by default we by default we by default we split at split at split at sentence boundaries and break it sentence boundaries and break it sentence boundaries and break it into a certain number. I into a certain number. I into a certain number. I can't can't can't remember what number it is. remember what number it is. remember what number it is. 250. That's it 250. That's it 250. That's it . And this, you . And this, you . And this, you know, can be know, can be know, can be customized. So, by customized. So, by customized. So, by default, default, default, the breakdown is by the breakdown is by the breakdown is by sentences. You can sentences. You can sentences. You can split at split at split at word boundaries. This can be word boundaries. This can be word boundaries. This can be changed. Um, the Gina model changed. Um, the Gina model that that that James mentioned, we, you know, are James mentioned, we, you know, are James mentioned, we, you know, are now part of Gina, or now part of Gina, or now part of Gina, or Gina part of us, it Gina part of us, it Gina part of us, it doesn't matter. Um, it has a doesn't matter. Um, it has a doesn't matter. Um, it has a very, very large very, very large very, very large context window, but context window, but context window, but you won't always want to you won't always want to you won't always want to fill it up for fill it up for fill it up for various reasons. Um, and various reasons. Um, and various reasons. Um, and so you can so you can so you can configure chunking configure chunking configure chunking to the word level, to the word level, to the word level, paragraph level, page boundary level paragraph level, page boundary level , etc.

, etc. We have a We have a semantic semantic semantic mapping field in Elastic mapping field in Elastic mapping field in Elastic that essentially takes a that essentially takes a that essentially takes a piece of text and piece of text and piece of text and breaks it into chunks. breaks it into chunks. So if you send So if you send us five pages of us five pages of us five pages of text, we'll break text, we'll break text, we'll break it up into chunks of 250 it up into chunks of 250 it up into chunks of 250 tokens with tokens with tokens with one one one sentence overlap and store sentence overlap and store sentence overlap and store it in the system for it in the system for it in the system for search. There are frameworks search. There are frameworks search. There are frameworks that talk about that talk about that talk about semantic chunking, semantic chunking, semantic chunking, and there are other names for what and there are other names for what and there are other names for what I would call I would call I would call large large large context window chunking. context window chunking. We are actually in And We are actually in And this is discussed in the this is discussed in the this is discussed in the lab work lab work lab work in the following chapters. in the following chapters. in the following chapters. We have a technique We have a technique We have a technique we use we use we use called "late called "late called "late chunking" that can chunking" that can chunking" that can work with large work with large work with large context windows context windows . This is not yet . This is not yet . This is not yet represented through represented through represented through Elastic Inference Service, but it is Elastic Inference Service, but it is Elastic Inference Service, but it is something that Gina models something that Gina models something that Gina models support. This is support. This is support. This is the closest I the closest I the closest I think we have to think we have to think we have to semantic semantic semantic partitioning, but we partitioning, but we partitioning, but we don't really don't really don't really need this need this need this construct, construct, construct, as the as the as the semantic text field semantic text field semantic text field allows us allows us allows us to achieve the same thing as to achieve the same thing as to achieve the same thing as semantic semantic semantic partitioning by simply partitioning by simply partitioning by simply partitioning the data and partitioning the data and partitioning the data and then then then searching it in a searching it in a searching it in a semantic style; semantic style; semantic style; you will see this in you will see this in you will see this in practical classes.

practical classes. So, we solve this So, we solve this problem using problem using traditional traditional partitioning and partitioning and partitioning and hybrid search, hybrid search, hybrid search, rather than any rather than any rather than any exotic exotic exotic semantic semantic semantic partitioning scheme. Please. partitioning scheme. Please. OK. Yes. So I made it Yes. So I made it available for a week. available for a week. available for a week. Because I know how it Because I know how it Because I know how it happens. So it's happens. So it's happens. So it's available throughout the available throughout the available throughout the week. Here. And this is week. Here. And this is week. Here. And this is nowhere to be found. The seminar is nowhere to be found. The seminar is nowhere to be found. The seminar is in my repository, in my repository, in my repository, for better or worse. for better or worse. And all the notebooks And all the notebooks are here. So are here. So are here. So if you if you if you only want to run only want to run only want to run notebooks, you can notebooks, you can notebooks, you can connect them to connect them to connect them to your serverless your serverless your serverless cluster or cluster or any other any other any other cluster that has cluster that has semantic text support. semantic text support. But yes, the workshop itself But yes, the workshop itself But yes, the workshop itself will be available for a will be available for a will be available for a week. And what did I week. And what did I week. And what did I stop at? And so, as I stop at? And so, as I stop at? And so, as I mentioned, the first mentioned, the first mentioned, the first part, the first part, the first part, the first half of each half of each half of each lesson, is the lesson, is the lesson, is the tools where you tools where you tools where you run run run the mapping: on the left, the the mapping: on the left, the the mapping: on the left, the input data for input data for input data for your API, on the right, the your API, on the right, the output. And then you'll output. And then you'll output. And then you'll move on to part move on to part move on to part two, where you'll two, where you'll two, where you'll open a open a open a Python notebook and it will Python notebook and it will Python notebook and it will automatically automatically automatically connect to the connect to the connect to the Elastic Search environment Elastic Search environment Elastic Search environment that was deployed

that was deployed that was deployed for you. for you. I see them I see them letting people in as letting people in as letting people in as others leave to others leave to others leave to make sure the hall is make sure the hall is make sure the hall is full. The URL in full. The URL in full. The URL in the upper right the upper right the upper right corner is what you corner is what you corner is what you need. need. need. All the All the All the seminar content is there. And there's a seminar content is there. And there's a seminar content is there. And there's a second URL—this is a second URL—this is a second URL—this is a repository with the repository with the repository with the source code for all the source code for all the source code for all the notebooks for notebooks for notebooks for today's today's today's workshop. These are the two workshop. These are the two workshop. These are the two most useful resources most useful resources . So, if you're . So, if you're . So, if you're coming in and haven't coming in and haven't coming in and haven't started the workshop yet: started the workshop yet: started the workshop yet: the top right corner, and the the top right corner, and the the top right corner, and the bottom right corner bottom right corner if you're interested in the if you're interested in the if you're interested in the Python notebooks that the Python notebooks that the Python notebooks that the content content content you're you're you're working on today is based on. I'll working on today is based on. I'll working on today is based on. I'll probably probably probably repeat this repeat this repeat this message every time message every time message every time I see I see I see someone come in. This is a someone come in. This is a someone come in. This is a very good question, very good question, very good question, and I will repeat it so that and I will repeat it so that and I will repeat it so that others can hear it too others can hear it too . Is the main goal of . Is the main goal of . Is the main goal of this series of classes this series of classes this series of classes to show that to show that to show that BM25-based search—the BM25-based search—the traditional traditional traditional lexical search that lexical search that lexical search that we've been using for a long time we've been using for a long time we've been using for a long time —and —and dense dense vector-based semantic search, or should vector-based semantic search, or should vector-based semantic search, or should the two be the two be the two be combined to combined to combined to create a hybrid create a hybrid create a hybrid search? This is the key search? This is the key search? This is the key thesis that we are currently thesis that we are currently thesis that we are currently advocating as a advocating as a advocating as a company. And the only thing company. And the only thing company. And the only thing we add to this we add to this we add to this that is obviously valuable that is obviously valuable is the ability to is the ability to is the ability to use use use re re -rankers on top of this.

-rankers on top of this. But yes, that's the main But yes, that's the main point. We strongly point. We strongly point. We strongly believe in this approach believe in this approach , and it's consistent with , and it's consistent with , and it's consistent with everything I've seen since before I everything I've seen since before I joined Elastic. joined Elastic. Even if you Even if you Even if you use only a use only a use only a specialized specialized specialized vector database, vector database, vector database, you often have to you often have to you often have to combine it with another combine it with another combine it with another search engine and search engine and search engine and perform the perform the perform the equivalent of a equivalent of a equivalent of a hybrid search. hybrid search. hybrid search. The advantage of Elastic is The advantage of Elastic is The advantage of Elastic is —and you see this —and you see this —and you see this in labs—that it's a in labs—that it's a in labs—that it's a single API request. We single API request. We single API request. We take care of all the take care of all the take care of all the output processes: output processes: output processes: from data breakdown from data breakdown from data breakdown during loading to during loading to during loading to real- real- real- time search, where the query time search, where the query time search, where the query is converted into a is converted into a is converted into a vector, doing all the vector, doing all the vector, doing all the work for you “under work for you “under work for you “under the hood”. We find the hood”. We find the hood”. We find this very valuable, this very valuable, this very valuable, especially for especially for especially for large-scale projects large-scale projects large-scale projects where latency, where latency, where latency, accuracy, and accuracy, and customizability are important. You can You can change the slicing settings. Can change the slicing settings. Can change the slicing settings. Can they be customized? Yes they be customized? Yes , you can. The , you can. The , you can. The documentation has documentation has documentation has information on how information on how information on how to change them and what to change them and what to change them and what partitioning parameters partitioning parameters partitioning parameters can be adjusted.

can be adjusted. can be adjusted. We have very detailed We have very detailed We have very detailed API documentation where API documentation where API documentation where all this is described; The all this is described; The all this is described; The workshop doesn't cover these details workshop doesn't cover these details workshop doesn't cover these details , but , but , but you you you can find all the can find all the configuration options in the Elastic documentation. Some of configuration options in the Elastic documentation. Some of our clients our clients our clients bring their own bring their own bring their own vectors and perform the vectors and perform the vectors and perform the partitioning themselves. partitioning themselves. partitioning themselves. We support this We support this We support this through standard SDKs. through standard SDKs. through standard SDKs. So whether you So whether you So whether you use Python, use Python, use Python, Java, .NET Core, or almost Java, .NET Core, or almost Java, .NET Core, or almost any modern programming language any modern programming language like TypeScript, you like TypeScript, you can still can still can still use Elastic use Elastic use Elastic in the format of these in the format of these in the format of these workshops. It's just that you workshops. It's just that you workshops. It's just that you as a developer will have to as a developer will have to as a developer will have to put in put in put in a little more effort. a little more effort. a little more effort. I'm sure you're I'm sure you're I'm sure you're moving forward moving forward moving forward and you can and you can and you can continue to continue to continue to work at your own work at your own work at your own pace. I said I pace. I said I pace. I said I would would would check in on progress from time to time and check in on progress from time to time and check in on progress from time to time and talk a little talk a little talk a little about the lab about the lab about the lab work. The first one is work. The first one is work. The first one is dedicated to this dedicated to this dedicated to this little diagram that little diagram that little diagram that explains how explains how explains how vector and vector and vector and semantic search works and semantic search works and semantic search works and what you get from it what you get from it what you get from it . So, when . So, when . So, when you use an you use an you use an embedding model embedding model embedding model connected to Elastic— connected to Elastic— we mentioned Gina as an out- we mentioned Gina as an out- we mentioned Gina as an out- of-the-box option— of-the-box option— of-the-box option— you can of course you can of course you can of course use your use your use your own. You can own. You can own. You can connect to connect to connect to other other other withdrawal services. We're just withdrawal services. We're just withdrawal services. We're just making it easy for you to get started making it easy for you to get started making it easy for you to get started with Gina, with Gina, with Gina, but you can but you can but you can use use use your own models and your own models and your own models and semantic text.

semantic text. But, the great thing about But, the great thing about this this this setup setup setup is that is that is that when you create when you create when you create an embed during an embed during an embed during indexing, you indexing, you indexing, you create it create it create it based on your based on your based on your document. However, when you document. However, when you document. However, when you run the query, run the query, run the query, you need to you need to you need to use the use the use the same same same embedding model, otherwise it will embedding model, otherwise it will embedding model, otherwise it will be, you know, like I'm be, you know, like I'm be, you know, like I'm trying to speak trying to speak trying to speak Portuguese. Portuguese. Portuguese. Nothing good would Nothing good would Nothing good would come of this. come of this. come of this. So when I started, So when I started, So when I started, when we started doing when we started doing when we started doing this a few years ago, this a few years ago, this a few years ago, in the early stages, in the early stages, in the early stages, you had to you had to you had to keep track of the keep track of the inference ID and the inference ID and the model itself. And then model itself. And then model itself. And then the developers the developers the developers had to constantly had to constantly had to constantly make sure that make sure that make sure that everyone was using the everyone was using the everyone was using the right model, and right model, and right model, and it was all quite it was all quite it was all quite difficult. Therefore, difficult. Therefore, difficult. Therefore, embedding this embedding this embedding this directly into directly into directly into the identifier of the the identifier of the the identifier of the field itself helps in two field itself helps in two field itself helps in two ways. First, ways. First, ways. First, you you you only need to set this up only need to set this up only need to set this up once, and since you once, and since you once, and since you set up an index, set up an index, set up an index, other people can other people can other people can use the use the use the same index to same index to same index to load their load their load their data or queries, data or queries, data or queries, right? So you can right? So you can right? So you can set this up set this up set this up once, once, once, whether it's the Gina model whether it's the Gina model whether it's the Gina model or some other one, and or some other one, and or some other one, and every time you every time you every time you use that use that use that field, it becomes field, it becomes field, it becomes just another field just another field . You simply either . You simply either . You simply either send data or send data or send data or make a request. So, make a request. So, make a request. So, when you execute when you execute when you execute a query, as you see a query, as you see a query, as you see here, the text comes in here, the text comes in here, the text comes in and it can be and it can be and it can be broken into pieces broken into pieces broken into pieces depending on the size depending on the size depending on the size and settings. It and settings. It and settings. It goes goes goes to maybe the Elastic output service, to maybe the Elastic output service, to maybe the Elastic output service, which is standard, which is standard, which is standard, or to another or to another or to another output service, and that's perfectly output service, and that's perfectly output service, and that's perfectly acceptable. We then acceptable. We then acceptable. We then generate embeddings generate embeddings generate embeddings and pass them back and pass them back and pass them back to Elastic to execute the to Elastic to execute the to Elastic to execute the HNSW ANN. We have other HNSW ANN. We have other HNSW ANN. We have other options, such as options, such as options, such as disc ANN, but by disc ANN, but by disc ANN, but by default we

default we default we use HNSW—a use HNSW—a hierarchical hierarchical hierarchical navigation world graph. navigation world graph. navigation world graph. It essentially allows you to It essentially allows you to It essentially allows you to quickly navigate quickly navigate quickly navigate to find to find to find nearest neighbors nearest neighbors nearest neighbors among matching among matching among matching documents. Then you documents. Then you documents. Then you get your top K get your top K get your top K documents—5, 10, or documents—5, 10, or documents—5, 10, or however many you want—and however many you want—and however many you want—and that allows you, if that allows you, if that allows you, if everything works well, everything works well, everything works well, to find documents or to find documents or to find documents or parts that are semantically parts that are semantically parts that are semantically similar to what you're similar to what you're similar to what you're asking about. And as you asking about. And as you asking about. And as you learned in learned in learned in the lab, it's the lab, it's the lab, it's not about the exact not about the exact not about the exact keywords, but about the very idea of ​​what keywords, but about the very idea of ​​what keywords, but about the very idea of ​​what you're you're you're asking about. So if asking about. So if asking about. So if all of us in this room all of us in this room all of us in this room ask, let's say, ask, let's say, ask, let's say, how to set up how to set up how to set up security, TLS in Elastic Search, security, TLS in Elastic Search, security, TLS in Elastic Search, we'll all phrase it we'll all phrase it we'll all phrase it a little differently, but a little differently, but a little differently, but the idea is the same, and so the idea is the same, and so the idea is the same, and so hopefully you'll hopefully you'll hopefully you'll get the same get the same get the same documents. And you've already documents. And you've already documents. And you've already noticed where it sometimes noticed where it sometimes noticed where it sometimes fails. Um, one of fails. Um, one of fails. Um, one of those cases is a those cases is a those cases is a very specific very specific very specific keyword, keyword, keyword, especially if it's not a especially if it's not a especially if it's not a word, but, for example, a word, but, for example, a product identifier that has no product identifier that has no meaning, it's very meaning, it's very meaning, it's very specific, then specific, then specific, then semantic search semantic search semantic search stops working stops working stops working because it doesn't because it doesn't because it doesn't need it. And that's what the need it. And that's what the need it. And that's what the second second second lab is for: lab is for: lab is for: we're adding lexical we're adding lexical we're adding lexical filtering by filtering by filtering by BM25 keywords BM25 keywords BM25 keywords and all the other and all the other and all the other traditional traditional traditional search techniques we've been search techniques we've been search techniques we've been using for a using for a using for a long time to improve long time to improve long time to improve this process. So I'm this process. So I'm this process. So I'm sure some of sure some of sure some of you have already moved you have already moved you have already moved on, but I'll drop by in a on, but I'll drop by in a on, but I'll drop by in a few minutes and we'll go over the few minutes and we'll go over the few minutes and we'll go over the second second second lab. And again lab. And again lab. And again , if you have any , if you have any , if you have any questions, don't questions, don't questions, don't hesitate to ask hesitate to ask . Yes, so pages.

. Yes, so pages. . Yes, so pages. Yes, Yes, Yes, pagination. So if pagination. So if pagination. So if you have, for example, 100 you have, for example, 100 you have, for example, 100 results and you results and you results and you want to get want to get want to get 10 at a time. Is that what 10 at a time. Is that what 10 at a time. Is that what you're asking about, or...Oh. you're asking about, or...Oh. you're asking about, or...Oh. Yes, you can Yes, you can Yes, you can do that. There is, I do that. There is, I do that. There is, I forgot the exact name, forgot the exact name, forgot the exact name, but essentially it is a but essentially it is a checkpoint identifier. And checkpoint identifier. And then the search then the search then the search continues. And yes, continues. And yes, continues. And yes, you don't need to you don't need to you don't need to search again, you don't search again, you don't search again, you don't need to need to need to restart the search restart the search restart the search for that part. It for that part. It for that part. It continues, and that's what continues, and that's what continues, and that's what our pagination API does our pagination API does . So if you . So if you know about this know about this know about this opportunity, you opportunity, you opportunity, you use it, we use it, we use it, we send you this send you this send you this key, and that actually key, and that actually key, and that actually means where you means where you means where you start and where you start and where you start and where you continue. The continue. The continue. The official official official name for this seems to be " name for this seems to be " point in time." point in time." point in time." If you look at the If you look at the If you look at the Elastic documentation, Elastic documentation, Elastic documentation, just search for "point in just search for "point in just search for "point in time" and it shows exactly how time" and it shows exactly how time" and it shows exactly how to use to use to use it for such tasks. it for such tasks. it for such tasks. Okay, good Okay, good Okay, good question. Um, so the question. Um, so the question. Um, so the resume field was created resume field was created resume field was created specifically for this specifically for this specifically for this seminar. It was seminar. It was seminar. It was generated by a large generated by a large generated by a large language model (LLM). language model (LLM). language model (LLM). It is not part of the It is not part of the It is not part of the system itself.

system itself. system itself. The real reason is The real reason is The real reason is that when I first that when I first that when I first tested this and outputted the tested this and outputted the tested this and outputted the entire text field, it entire text field, it entire text field, it was too long and was too long and was too long and cluttered everything up. That's why cluttered everything up. That's why cluttered everything up. That's why we made this we made this we made this little summary field. little summary field. little summary field. But you But you But you can can can use, use, use, for example, the for example, the for example, the ingest ingest ingest pipeline in Elasticsearch, pipeline in Elasticsearch, pipeline in Elasticsearch, call the call the call the logical inference and logical inference and logical inference and say: I want to say: I want to say: I want to create a summary of create a summary of create a summary of this document or I this document or I this document or I want to extract possible want to extract possible want to extract possible queries. I could do that queries. I could do that queries. I could do that . So, you . So, you . So, you can do this can do this can do this while loading while loading while loading data. It's just for data. It's just for data. It's just for this specific this specific this specific case to make it case to make it case to make it easier. But the easier. But the easier. But the vector field vector field vector field you're looking at is the you're looking at is the you're looking at is the header, yes. header, yes. When we delve into the When we delve into the labs, labs, labs, we won't be we won't be we won't be using this using this using this construct, but in the construct, but in the construct, but in the next labs there next labs there next labs there will be API calls to will be API calls to will be API calls to LLM and it will be the same. LLM and it will be the same. LLM and it will be the same. You just You just You just plug this into plug this into plug this into your your your data processing pipeline. data processing pipeline. data processing pipeline. Question. In Question. In Question. In RAM? RAM? RAM? That's a great question. That's a great question. So, in terms of the work So, in terms of the work we've done to we've done to we've done to make HNSW make HNSW make HNSW more efficient, it more efficient, it more efficient, it started 2 years ago, started 2 years ago, started 2 years ago, even over 2 years even over 2 years even over 2 years ago, when we ago, when we ago, when we implemented implemented implemented int8 quantization, which I int8 quantization, which I int8 quantization, which I think is still think is still think is still standard. So by standard. So by standard. So by default we default we default we take float 32 vectors and take float 32 vectors and take float 32 vectors and quantize them quantize them quantize them before storing them in before storing them in before storing them in memory. We still memory. We still memory. We still store the full store the full store the full vector, but vector, but vector, but we only keep the we only keep the we only keep the quantized vector in memory.

quantized vector in memory. quantized vector in memory. We also have a BBQ, that's right. We also have a BBQ, that's right. We also have a BBQ, that's right. And we have a blog that And we have a blog that And we have a blog that talks about BBQ talks about BBQ . It's a little over a . It's a little over a . It's a little over a year old, maybe even a year old, maybe even a year old, maybe even a little over a year old, little over a year old, little over a year old, where we compressed the where we compressed the where we compressed the HNSW graph vector that we HNSW graph vector that we HNSW graph vector that we use even more use even more use even more heavily, effectively to heavily, effectively to heavily, effectively to one bit. And under one bit. And under one bit. And under the hood, the hood, the hood, complex mechanisms work to complex mechanisms work to complex mechanisms work to support this and support this and support this and maintain a high maintain a high maintain a high level of level of level of search completeness. And then we search completeness. And then we search completeness. And then we just introduced just introduced just introduced disk BBQ, which disk BBQ, which disk BBQ, which almost completely almost completely almost completely eliminates the eliminates the eliminates the memory overhead of memory overhead of memory overhead of vector search. vector search. The only difference, I would The only difference, I would say, between disk say, between disk say, between disk BBQ and regular BBQ BBQ and regular BBQ BBQ and regular BBQ is that in is that in is that in Elastic we have Elastic we have Elastic we have the option of not the option of not the option of not using HNSW using HNSW using HNSW as the underlying as the underlying as the underlying search engine. And search engine. And search engine. And the newest form of the newest form of the newest form of this is disk BBQ, this is disk BBQ, this is disk BBQ, which is which is which is Elastic's own development. Elastic's own development. Elastic's own development. Nobody implements this anymore Nobody implements this anymore Nobody implements this anymore . . Graph-based search. Do you want Graph-based search. Do you want to answer this? to answer this? So, graph-based search So, graph-based search . To be . To be . To be clear: we are not a clear: we are not a clear: we are not a graph database.

graph database. graph database. There are quite There are quite There are quite specific graph specific graph specific graph databases. Graphs databases. Graphs databases. Graphs have their have their have their purpose. purpose. purpose. They usually They usually They usually require a lot of require a lot of require a lot of resources to resources to resources to support and support and support and maintain. If maintain. If maintain. If you do that—great. you do that—great. you do that—great. However, for the vast However, for the vast However, for the vast majority of people, as majority of people, as majority of people, as we see, this is not really we see, this is not really we see, this is not really necessary, since necessary, since necessary, since the ability to quickly the ability to quickly the ability to quickly search among a large search among a large search among a large number of documents number of documents number of documents is provided is provided is provided by quantization. by quantization. by quantization. Especially now, in Especially now, in Especially now, in the world of agents, where the world of agents, where multiple requests can be executed. So, multiple requests can be executed. So, in one of the later in one of the later in one of the later labs, labs, labs, multiple multiple multiple query loops are executed. Just query loops are executed. Just query loops are executed. Just two as an example, but you two as an example, but you two as an example, but you can go in, can go in, can go in, get the first stage, a get the first stage, a get the first stage, a sample of documents, and sample of documents, and sample of documents, and then see, " then see, " Okay, they're all Okay, they're all Okay, they're all related to this tag related to this tag ." And then search. ." And then search. ." And then search. So, this is where So, this is where things like filtering things like filtering and some of our other and some of our other and some of our other search methodologies help. And search methodologies help. And search methodologies help. And yes, it does occur, but yes, it does occur, but yes, it does occur, but we don't often see it we don't often see it we don't often see it as a real obstacle as a real obstacle as a real obstacle that needs to be that needs to be that needs to be addressed. addressed. And the last thing I'll say And the last thing I'll say about this: we're increasingly about this: we're increasingly about this: we're increasingly seeing seeing seeing machine learning models machine learning models machine learning models that perform that perform that perform classification "on the classification "on the classification "on the fly," which is another fly," which is another fly," which is another way to solve way to solve way to solve this graph this graph this graph problem. This works in problem. This works in problem. This works in some situations, not some situations, not some situations, not all, but Neo4j and other all, but Neo4j and other all, but Neo4j and other graph databases have graph databases have graph databases have been around for a long time. I do been around for a long time. I do been around for a long time. I do n't know of any that n't know of any that scale very well. We are scale very well. We are trying to trying to trying to target target target multi-billion multi-billion multi-billion vector vector vector datasets, which means datasets, which means datasets, which means we are very careful we are very careful we are very careful not to implement not to implement not to implement a design that will a design that will a design that will prevent us from prevent us from prevent us from scaling at scaling at scaling at that level. And we have that level. And we have that level. And we have n't felt the need to n't felt the need to n't felt the need to move on to that yet, move on to that yet, move on to that yet, although it's often although it's often although it's often discussed. This

discussed. This discussed. This was the main topic was the main topic was the main topic last year at this last year at this last year at this conference. I conference. I conference. I remember being here and remember being here and remember being here and seeing a seeing a seeing a lot of talk about the Count's RAG lot of talk about the Count's RAG lot of talk about the Count's RAG . It will probably continue to be like this . It will probably continue to be like this . It will probably continue to be like this . Ahem. I have . Ahem. I have . Ahem. I have n't seen a technology n't seen a technology n't seen a technology that has mastered this yet. And that has mastered this yet. And that has mastered this yet. And maybe there's someone here maybe there's someone here maybe there's someone here from Neo4j who will object from Neo4j who will object from Neo4j who will object to me and say, "No, no, to me and say, "No, no, to me and say, "No, no, no, we've solved everything." no, we've solved everything." no, we've solved everything." I think they decided, I think they decided, I think they decided, but not on the but not on the but not on the scale scale scale we work on. Yes. Oh, so we work on. Yes. Oh, so we work on. Yes. Oh, so you came from their you came from their you came from their session. Perfectly. OK. session. Perfectly. OK. There, in the back? I think There, in the back? I think the question was the question was the question was that we can that we can that we can use use use classification models classification models classification models to get some of the to get some of the to get some of the results that can be results that can be results that can be obtained obtained obtained using graph using graph using graph traversal. So, at Elastic, traversal. So, at Elastic, traversal. So, at Elastic, we have the concept of we have the concept of we have the concept of running running running machine learning models machine learning models machine learning models on what we on what we on what we call ML nodes. call ML nodes. You can You can upload any upload any upload any classification classification classification model that you have model that you have model that you have determined works and determined works and determined works and connect it to Elastic connect it to Elastic connect it to Elastic this way. Another this way. Another this way. Another option is to create option is to create option is to create your own model for your own model for your own model for which you have which you have which you have external output, and external output, and perhaps perhaps perhaps use it use it use it instead of our instead of our instead of our machine machine machine learning technology. And I learning technology. And I learning technology. And I hope that in hope that in hope that in the future we will the future we will the future we will implement more implement more implement more complex solutions in this complex solutions in this complex solutions in this area. But we have area. But we have area. But we have nothing to announce right now.

nothing to announce right now. nothing to announce right now. We are focused on We are focused on We are focused on taking taking taking existing models and existing models and existing models and pushing the boundaries of pushing the boundaries of pushing the boundaries of what we can do with what we can do with unstructured unstructured data processing, dense data processing, dense data processing, dense vector vector vector embeddings, and embeddings, and embeddings, and reranking. We'll reranking. We'll reranking. We'll probably move on probably move on probably move on from that once from that once from that once we improve the we improve the reranking mechanism and the reranking mechanism and the dense dense dense vector models that are currently vector models that are currently vector models that are currently in in in development and, you know, development and, you know, are working quite successfully in the market. I'll add I'll add classification classification classification models to this, and you can also models to this, and you can also use use things like NER models, things like NER models, things like NER models, i.e. i.e. i.e. Named Entity Recognition Named Entity Recognition . So, if you're . So, if you're . So, if you're extracting entities, extracting entities, extracting entities, trying trying trying to identify to identify to identify people's names, locations, people's names, locations, people's names, locations, company names, or company names, or company names, or products, you can products, you can products, you can run them "on run them "on run them "on the fly" during the fly" during the fly" during query execution: query execution: query execution: when the query comes in, when the query comes in, when the query comes in, you pass it you pass it you pass it through the NER model. And through the NER model. And through the NER model. And NER models are very fast NER models are very fast NER models are very fast because they are small because they are small . These are really . These are really . These are really small models. You small models. You small models. You extract these extract these extract these entities, and again entities, and again entities, and again , now with , now with , now with agents and agents and tools like that, you don't tools like that, you don't have to write have to write have to write complex scripts on complex scripts on complex scripts on Painless. The agent can Painless. The agent can Painless. The agent can take this data and take this data and take this data and create a query that is create a query that is create a query that is better better better tailored to it.

tailored to it. tailored to it. So there are a lot of So there are a lot of machine learning capabilities machine learning capabilities that we don't talk about as much that we don't talk about as much that we don't talk about as much as we as we as we used to because, as used to because, as used to because, as James said, we've James said, we've James said, we've shifted our focus a bit to other shifted our focus a bit to other shifted our focus a bit to other things, but they're still things, but they're still things, but they're still in the stack and very in the stack and very in the stack and very functional. Oh, functional. Oh, functional. Oh, the filtering the filtering the filtering is on. This is is on. This is is on. This is great. Here is Sunil. How are great. Here is Sunil. How are great. Here is Sunil. How are you, dude? One of you, dude? One of you, dude? One of our outstanding SA ( our outstanding SA ( Solutions Architects) in the corner. Solutions Architects) in the corner. Solutions Architects) in the corner. You can ask You can ask You can ask him all our him all our him all our questions. I questions. I questions. I will redirect my attention now. will redirect my attention now. will redirect my attention now. For those who just For those who just For those who just joined, except Sunil, joined, except Sunil, there is a short URL for the seminar in the upper right corner. You URL for the seminar in the upper right corner. You URL for the seminar in the upper right corner. You can can can run it. It run it. It run it. It opens up a kind of laboratory for you opens up a kind of laboratory for you , a , a laboratory laboratory laboratory environment. And also environment. And also environment. And also deploys a deploys a deploys a serverless Elasticsearch cluster, serverless Elasticsearch cluster, serverless Elasticsearch cluster, fully fully fully configured for you. configured for you. configured for you. So feel free to So feel free to So feel free to go through it. go through it. So, there is one main So, there is one main parameter for parameter for parameter for mutual RRF, and it is...I mutual RRF, and it is...I mutual RRF, and it is...I forgot something, I can't forgot something, I can't forgot something, I can't remember it off the top of my head. I don't remember it off the top of my head. I don't remember it off the top of my head. I don't know. You...you know. You...you know. You...you open it. I don't open it. I don't open it. I don't know if we're bringing know if we're bringing know if we're bringing this into the RRF attributes for the this into the RRF attributes for the this into the RRF attributes for the search module.

search module. What is the default constant What is the default constant in... in... in... Constant. Rank Constant. Rank Constant. Rank constant. This is something constant. This is something constant. This is something you can you can you can customize. I don't customize. I don't customize. I don't know what the know what the know what the range is. This is what the range is. This is what the fifth fifth laboratory, the laboratory, the laboratory, the optional one, is dedicated to. optional one, is dedicated to. optional one, is dedicated to. She's all about...Well, She's all about...Well, She's all about...Well, sorry. The fifth sorry. The fifth sorry. The fifth laboratory is laboratory is laboratory is reordering. RRF is out there somewhere reordering. RRF is out there somewhere reordering. RRF is out there somewhere . Sorry, my . Sorry, my . Sorry, my brain went blank for a second brain went blank for a second brain went blank for a second . Um, this is somewhere around what . Um, this is somewhere around what Tanu was talking about, Tanu was talking about, a little more a little more a little more examples, and you examples, and you examples, and you can change can change can change things like that, so. things like that, so. So, I don't know if that So, I don't know if that answered your answered your answered your question? question? question? Yes. Yes. Yes. Okay, cool. There Okay, cool. There is a question here. is a question here. Ask. Ask. What is the recommendation What is the recommendation for using for using rerankers when your rerankers when your data corpus is data corpus is data corpus is small? So, small? So, small? So, how how how effective is it or how does it effective is it or how does it effective is it or how does it work?

work? work? That's a great question. That's a great question. That's a great question. There's actually a table There's actually a table There's actually a table that explains some of that explains some of that explains some of these decision trees in, uh these decision trees in, uh , a later seminar , a later seminar . It seems to be in the . It seems to be in the . It seems to be in the fourth fourth fourth lab. lab. But the short But the short answer is: for answer is: for answer is: for a small corpus of a small corpus of a small corpus of documents, my documents, my documents, my main rule is to main rule is to always test always test always test what can be achieved what can be achieved what can be achieved using a combination of using a combination of using a combination of BM25 and semantic BM25 and semantic BM25 and semantic search. If it search. If it search. If it delivers on target delivers on target delivers on target completeness or accuracy, completeness or accuracy, completeness or accuracy, uh, or whatever relevance goals uh, or whatever relevance goals you have, stop. you have, stop. Every time you Every time you Every time you add something on top, add something on top, add something on top, you get you get you get additional delay. additional delay. additional delay. Only do this when Only do this when Only do this when necessary, and the necessary, and the necessary, and the next, uh, you know next, uh, you know , expansion step , expansion step , expansion step would be to use the would be to use the reranking model. And, uh , then all that's left to do is , then all that's left to do is , then all that's left to do is choose the one choose the one choose the one you think will work you think will work you think will work best for best for best for your case and your case and your case and test it test it test it to make sure to make sure to make sure it gives a tangible it gives a tangible it gives a tangible result. Uh, that would be the result. Uh, that would be the result. Uh, that would be the standard standard standard optimal approach. There are optimal approach. There are optimal approach. There are people who people who people who would change this would change this would change this recommendation a little bit and recommendation a little bit and recommendation a little bit and say, "Listen, say, "Listen, say, "Listen, just just just use BM25, use BM25, use BM25, regular search." Uh, regular search." Uh, regular search." Uh, and then add and then add and then add semantic search semantic search semantic search if it's not, if it's not, if it's not, uh, relevant enough, but uh, relevant enough, but uh, relevant enough, but I'm a big fan of I'm a big fan of I'm a big fan of dense vectors, dense vectors, dense vectors, so I would so I would so I would implement them into implement them into implement them into the architecture right away. And, uh, the architecture right away. And, uh, the architecture right away. And, uh, that way I don't have to that way I don't have to worry about worry about things like, you know, you things like, you know, you things like, you know, you get into weird get into weird get into weird lexical lexical lexical search situations when search situations when search situations when you start adding you start adding you start adding synonyms or synonyms or synonyms or resorting to resorting to resorting to advanced advanced advanced lexical lexical lexical tuning techniques that

tuning techniques that tuning techniques that might not be might not be might not be necessary if necessary if necessary if semantic search semantic search semantic search works well with works well with works well with lexical search. Please lexical search. Please . . So now people are So now people are moving on, and moving on, and moving on, and that's great. Uh, let's just that's great. Uh, let's just that's great. Uh, let's just summarize the second summarize the second summarize the second lab, lab, lab, which, again, which, again, which, again, was about your was about your was about your favorite lexical favorite lexical favorite lexical search. But I hope search. But I hope you understand that it has you understand that it has you understand that it has its, uh, its, uh, its, uh, advantages. Uh, we've only advantages. Uh, we've only advantages. Uh, we've only barely touched on the topic of barely touched on the topic of barely touched on the topic of lexical search, lexical search, lexical search, because, you know, because, you know, because, you know, for about 15 for about 15 for about 15 years, or a dozen years, or a dozen years, or a dozen years at Elastic, that was years at Elastic, that was years at Elastic, that was all we had. So, all we had. So, all we had. So, if you've ever if you've ever if you've ever visited the visited the visited the Elastic search documentation sites Elastic search documentation sites , gone to the , gone to the , gone to the search section, and search section, and search section, and clicked on the clicked on the clicked on the search API, you've seen an search API, you've seen an search API, you've seen an incredible number of incredible number of incredible number of ways to configure ways to configure ways to configure lexical search. As lexical search. As lexical search. As James pointed out, before James pointed out, before James pointed out, before semantic search came along, semantic search came along, semantic search came along, we we we had to do had to do had to do a bunch of weird things a bunch of weird things a bunch of weird things to get to get to get relevant relevant relevant results. Luckily, results. Luckily, results. Luckily, we don't we don't we don't have to do have to do have to do that much these days, at least that much these days, at least that much these days, at least usually, although we usually, although we usually, although we still do from time to time still do from time to time still do from time to time . But . But . But lexical search has lexical search has lexical search has its advantages, although its advantages, although its advantages, although it often falls short it often falls short it often falls short when when when you need to search for you need to search for you need to search for something beyond something beyond something beyond specific specific specific keywords or phrases. Its keywords or phrases. Its keywords or phrases. Its advantage is that...I advantage is that...I advantage is that...I don't even know if I don't even know if I don't even know if I should say it. I should say it. I should say it. I meant to say that it's meant to say that it's meant to say that it's usually faster, usually faster, usually faster, although now, with although now, with although now, with quantization and quantization and quantization and things like that, it's things like that, it's things like that, it's not always the case, but not always the case, but not always the case, but it's it's it's kind of faster by default. So in kind of faster by default. So in kind of faster by default. So in lab lab lab number three, which is probably where number three, which is probably where number three, which is probably where most of you most of you most of you are already are already , you move on to , you move on to , you move on to combining them.

combining them. And I will say this: And I will say this: while I am a big while I am a big while I am a big fan of fan of fan of dense vector search, dense vector search, dense vector search, lexical search lexical search usually has lower usually has lower latency in most cases. And that's latency in most cases. And that's latency in most cases. And that's because there's no because there's no because there's no ML ML inference going on there. So every time inference going on there. So every time you you you use use use semantic search with semantic search with semantic search with dense vectors, dense vectors, dense vectors, you're adding a call to, you're adding a call to, you're adding a call to, you know, the inference you know, the inference endpoint. And endpoint. And endpoint. And most often, it is this most often, it is this most often, it is this inference endpoint that inference endpoint that inference endpoint that becomes a bigger “ becomes a bigger “ bottleneck” than bottleneck” than bottleneck” than the rest of the system. And this is the rest of the system. And this is the rest of the system. And this is another reason why another reason why another reason why we integrated we integrated we integrated inference inference inference directly into the directly into the directly into the data platform. data platform. data platform. Because during Because during Because during end-to-end end-to-end end-to-end testing, it doesn't testing, it doesn't testing, it doesn't matter how fast matter how fast matter how fast your vector your vector your vector database returns database returns database returns vectors; if vectors; if vectors; if the inference the inference the inference you depend on is you depend on is you depend on is slow, then your slow, then your slow, then your search will be slow too. search will be slow too. I think this question is I think this question is changing as changing as data platforms integrate data platforms integrate more capabilities into more capabilities into more capabilities into their systems and their systems and their systems and models models models improve. And improve. And improve. And each developer each developer each developer will have their own will have their own will have their own vision of this vision of this vision of this process. Our approach is that process. Our approach is that process. Our approach is that everything everything everything starts with starts with starts with traditional search, traditional search, traditional search, which is quite logical, which is quite logical, which is quite logical, given our given our given our history. And on top of history. And on top of history. And on top of that, you add that, you add that, you add semantic semantic semantic capabilities. And then, capabilities. And then, capabilities. And then, if necessary, you if necessary, you if necessary, you add a add a add a reranking level. As reranking level. As reranking level. As for where exactly this for where exactly this for where exactly this happens, that's where the happens, that's where the difficulty lies. As a difficulty lies. As a developer developer developer trying trying trying to build to build to build an architecture around an architecture around an architecture around this, you have an this, you have an this, you have an extremely wide extremely wide extremely wide range of options. And range of options. And range of options. And depending on the stack depending on the stack depending on the stack you start with, you start with, you start with, you will have your own you will have your own you will have your own vision of what vision of what vision of what should be done at the

should be done at the should be done at the data platform level and what should be done data platform level and what should be done with with with some kind of middleware some kind of middleware . I . I use the term " use the term " middleware" loosely middleware" loosely middleware" loosely , but there is , but there is , but there is a whole concept of an a whole concept of an application runtime that disappeared application runtime that disappeared with the AI ​​boom and will with the AI ​​boom and will with the AI ​​boom and will become become become necessary again. It either necessary again. It either necessary again. It either integrates integrates integrates directly into the directly into the directly into the data platform, where data platform, where data platform, where we perform we perform we perform inference and other inference and other inference and other tasks that tasks that tasks that typically require typically require typically require coordination with coordination with coordination with external services external services , or there will be , or there will be , or there will be platforms that platforms that platforms that will try to implement this will try to implement this using the using the data platform only data platform only data platform only as a plug- as a plug- as a plug- in. I remember the in. I remember the in. I remember the good old days good old days good old days when I was just starting when I was just starting when I was just starting my career in IT: we my career in IT: we my career in IT: we were doing object- were doing object- relational relational relational mapping, and the mapping, and the mapping, and the database was considered, so database was considered, so database was considered, so to speak, a variable. You to speak, a variable. You to speak, a variable. You change your change your change your credentials and it doesn't credentials and it doesn't credentials and it doesn't matter because matter because matter because you're not you're not you're not using SQL anymore. I using SQL anymore. I using SQL anymore. I believe we believe we believe we are at the very are at the very are at the very beginning of the AI ​​era beginning of the AI ​​era , where for , where for , where for search purposes, once you search purposes, once you search purposes, once you start writing code, start writing code, start writing code, you become you become you become locked into a locked into a locked into a chosen stack. And that chosen stack. And that chosen stack. And that dictates a lot of different dictates a lot of different dictates a lot of different design decisions design decisions that you have to that you have to that you have to make, especially make, especially make, especially when you start when you start when you start to scale. Our to scale. Our to scale. Our advantage over, advantage over, advantage over, say, a simpler say, a simpler say, a simpler stack becomes apparent stack becomes apparent stack becomes apparent when the when the when the data volumes grow and the data volumes grow and the data volumes grow and the latency requirements latency requirements latency requirements become, you know, I don't become, you know, I don't become, you know, I don't want to say want to say want to say tighter, but the SLAs tighter, but the SLAs tighter, but the SLAs become really become really become really tight. That's when we're at our tight. That's when we're at our

best, because we best, because we can do things that can do things that can do things that other platforms can't other platforms can't other platforms can't , and we have a , and we have a , and we have a level of level of level of customization flexibility, customization flexibility, customization flexibility, especially around especially around especially around relevance, that relevance, that other other data platforms can't achieve. data platforms can't achieve. This is a very good question that we discuss in that we discuss in that we discuss in the lab, but I don't the lab, but I don't the lab, but I don't think we give an think we give an think we give an exhaustive exhaustive exhaustive answer key of, "Here's how to answer key of, "Here's how to answer key of, "Here's how to build the scales," build the scales," build the scales," based on what you based on what you based on what you can tell about the can tell about the can tell about the structure of your structure of your structure of your data or the nature of data or the nature of data or the nature of the searches that the searches that the searches that you know you know you know your users will perform your users will perform your users will perform . The short . The short . The short answer to this answer to this answer to this question is: it's question is: it's question is: it's actually hard actually hard actually hard to do today. The longer to do today. The longer to do today. The longer answer—and this is what answer—and this is what I think you'll see I think you'll see I think you'll see from us—is that from us—is that from us—is that there's a there's a there's a missing piece in the missing piece in the relevance setup that has relevance setup that has n't yet been addressed in a way that n't yet been addressed in a way that n't yet been addressed in a way that makes our makes our makes our solution completely solution completely solution completely turnkey. turnkey. By the way, I have a By the way, I have a reference reference reference architecture that I architecture that I architecture that I presented a year ago. presented a year ago. presented a year ago. It needs It needs It needs to be refined. This is to be refined. This is to be refined. This is based on Mastra, which based on Mastra, which based on Mastra, which embraces the concept of embraces the concept of embraces the concept of automatic automatic automatic tuning. Hmm, but tuning. Hmm, but tuning. Hmm, but in an ideal world, you in an ideal world, you in an ideal world, you could take the could take the could take the search queries that search queries that search queries that go through the go through the go through the system, telemetry from system, telemetry from system, telemetry from that that that user user user experience, and experience, and experience, and click data, likes click data, likes click data, likes or dislikes— or dislikes— in effect, telemetry in effect, telemetry in effect, telemetry from users— from users— from users— as well as any other as well as any other as well as any other business metrics business metrics business metrics integrated into your integrated into your integrated into your script to script to script to adjust adjust adjust relevance, which relevance, which relevance, which could provide could provide could provide different weights for the different weights for the different weights for the linear model. Hmm, linear model. Hmm, linear model. Hmm, the system could the system could the system could switch between a switch between a switch between a random forest (RF) random forest (RF) random forest (RF) and a linear model with and a linear model with and a linear model with certain weights

certain weights certain weights depending on the type of depending on the type of depending on the type of input query. Or input query. Or input query. Or you have all sorts of other you have all sorts of other you have all sorts of other types of techniques. The one that types of techniques. The one that types of techniques. The one that I think is more I think is more I think is more common than even common than even common than even those two is simply those two is simply those two is simply creating different creating different creating different sets of filters sets of filters sets of filters depending on the incoming depending on the incoming depending on the incoming query. So, Sunil query. So, Sunil query. So, Sunil designed the initial designed the initial designed the initial architecture for architecture for architecture for this. We called this the this. We called this the this. We called this the intelligent intelligent intelligent search level. Hmm, and search level. Hmm, and search level. Hmm, and this is what Azure Search this is what Azure Search this is what Azure Search was trying to implement was trying to implement was trying to implement in its product. This is something in its product. This is something in its product. This is something you you you can already do with Elastic today, can already do with Elastic today, can already do with Elastic today, using the using the using the agent builder construct agent builder construct agent builder construct that was covered in the that was covered in the that was covered in the last last last lab. lab. lab. Hmm, but this is a slightly different Hmm, but this is a slightly different Hmm, but this is a slightly different technique than just technique than just technique than just adjusting the weights "on adjusting the weights "on adjusting the weights "on the fly" based on the fly" based on what we see. You what we see. You what we see. You can have LLM can have LLM can have LLM analyze the query analyze the query and then perform and then perform and then perform a search with different a search with different a search with different filters based on filters based on filters based on that analysis, and I that analysis, and I that analysis, and I actually actually actually like this like this like this approach. I think it's approach. I think it's approach. I think it's fantastic, and I'm fantastic, and I'm fantastic, and I'm glad we have glad we have glad we have this kind of design in this kind of design in this kind of design in Elastic, and it's all Elastic, and it's all Elastic, and it's all what the what the what the agent builder is designed to solve. agent builder is designed to solve.

Yes. Exactly. And that's where Yes. Exactly. And that's where I hope—and this is not something I hope—and this is not something I hope—and this is not something we've already we've already we've already announced—that at announced—that at announced—that at some point you'll some point you'll some point you'll see the concept of see the concept of see the concept of auto- auto- auto- tuned tuned tuned relevance that relevance that relevance that can be enabled by can be enabled by can be enabled by having attributes or having attributes or having attributes or settings that you settings that you settings that you specify to specify to specify to activate, and maybe activate, and maybe activate, and maybe even to reinforce, even to reinforce, even to reinforce, depending on the depending on the depending on the use case and use case and use case and what you see in the what you see in the what you see in the actual operation of the actual operation of the actual operation of the system. And to make it system. And to make it system. And to make it clear to everyone clear to everyone clear to everyone in the room, we have in the room, we have in the room, we have n't released n't released n't released any technology any technology any technology that would do this yet. But that would do this yet. But that would do this yet. But this question this question this question comes up quite often when comes up quite often when comes up quite often when working with clients, and working with clients, and working with clients, and it's something that, if we had it's something that, if we had , would greatly improve the , would greatly improve the , would greatly improve the situation and situation and situation and make life easier for make life easier for make life easier for developers. Do you developers. Do you developers. Do you agree? Okay, I'll agree? Okay, I'll agree? Okay, I'll bring this up bring this up bring this up with our with our with our CTO again CTO again CTO again and get him to and get him to and get him to say "no" again. say "no" again. So, we'll just So, we'll just go over this go over this go over this point, if you don't point, if you don't point, if you don't mind. Past, if mind. Past, if mind. Past, if not, it's not, it's not, it's okay. Um, yes, we're okay. Um, yes, we're okay. Um, yes, we're trying to combine trying to combine trying to combine two different two different two different methodologies here. And the essence of methodologies here. And the essence of methodologies here. And the essence of the problem is somewhere the problem is somewhere the problem is somewhere here. For example, BM25 has here. For example, BM25 has here. For example, BM25 has a scale, it rates a scale, it rates a scale, it rates documents from zero to documents from zero to documents from zero to almost almost almost infinity. Whereas infinity. Whereas infinity. Whereas vector search vector search vector search usually works in the usually works in the usually works in the range from zero to range from zero to range from zero to one. And so when one. And so when one. And so when you have these scales that you have these scales that you have these scales that don't match, don't match, don't match, it's obvious that a bad it's obvious that a bad it's obvious that a bad result in a result in a result in a BM25 lexical search BM25 lexical search will almost always outperform a will almost always outperform a good semantic good semantic good semantic search from a purely numerical

search from a purely numerical search from a purely numerical perspective. So an perspective. So an perspective. So an interesting interesting interesting game begins: how to combine them? game begins: how to combine them? There are several different There are several different ways. We won't ways. We won't ways. We won't go into the go into the go into the details of choosing on the fly details of choosing on the fly , but there are different ways , but there are different ways , but there are different ways to understand the basics. to understand the basics. RRF is designed to RRF is designed to work quite work quite work quite well “out of the box,” up to a well “out of the box,” up to a well “out of the box,” up to a point. But point. But point. But this is a good first this is a good first this is a good first step. You don't need to step. You don't need to step. You don't need to know how know how know how to configure. You don't to configure. You don't to configure. You don't need to know need to know need to know the details of your data. the details of your data. But the idea is that there But the idea is that there is essentially a is essentially a is essentially a table here. You table here. You table here. You reject the grades. You reject the grades. You reject the grades. You are running two are running two are running two searches in parallel. You searches in parallel. You searches in parallel. You take the order. There's take the order. There's take the order. There's a bit of this a bit of this a bit of this ranking constant there. But ranking constant there. But ranking constant there. But essentially, if your essentially, if your essentially, if your document has a high document has a high document has a high ranking in both ranking in both ranking in both cases, it's cases, it's cases, it's probably a good probably a good probably a good match. If match. If match. If not, then a kind of combination comes into play not, then a kind of combination comes into play . And you . And you have this pretty simple one, where is have this pretty simple one, where is have this pretty simple one, where is it? I scrolled it? I scrolled it? I scrolled past him. Um, past him. Um, past him. Um, the assessment is here. And the assessment is here. And the assessment is here. And then you have a table.

then you have a table. And in fact, you are And in fact, you are not returning a list of not returning a list of not returning a list of documents documents documents sorted by sorted by sorted by rating. It's just a rating. It's just a rating. It's just a ranking order. ranking order. ranking order. That's why there's a That's why there's a That's why there's a ranking constant ranking constant . Because you just . Because you just . Because you just get the rank of get the rank of get the rank of documents. And documents. And documents. And can you see here can you see here can you see here where it works? Yes. This is where it works? Yes. This is where it works? Yes. This is described in this table described in this table , where the documents , where the documents , where the documents arrive in different arrive in different arrive in different orders. But by ranking orders. But by ranking orders. But by ranking them, you hope to them, you hope to them, you hope to get the best get the best get the best answer for yourself. answer for yourself. answer for yourself. But this doesn't always But this doesn't always But this doesn't always work. By the way, work. By the way, work. By the way, James mentioned this a second ago James mentioned this a second ago James mentioned this a second ago . . . Filtering is a very Filtering is a very Filtering is a very powerful and powerful and powerful and effective way effective way effective way to help with this. to help with this. So if you can So if you can filter your filter your filter your data by data by data by keyword, geolocation, keyword, geolocation, keyword, geolocation, date, anything that date, anything that date, anything that narrows the data set to narrows the data set to narrows the data set to fewer fewer fewer candidates, you will candidates, you will candidates, you will generally get generally get generally get better results. And better results. And better results. And because you have a because you have a because you have a smaller volume, you don't smaller volume, you don't smaller volume, you don't just throw your just throw your just throw your nets at everything at once. You nets at everything at once. You nets at everything at once. You filter and filter and filter and narrow your search, narrow your search, narrow your search, right? It's like, if right? It's like, if right? It's like, if I want pizza today, I want pizza today, I want pizza today, I'm, you know, not I'm, you know, not I'm, you know, not just going to search for "food," just going to search for "food," just going to search for "food," but if I did, but if I did, but if I did, I could I could I could filter filter filter the results by the the results by the the results by the query "pizza" and query "pizza" and query "pizza" and at least get at least get at least get only results with only results with only results with pizza or something like that pizza or something like that . So, this is where . So, this is where . So, this is where filtering comes into play, and filtering comes into play, and filtering comes into play, and this is something that Elasticsearch this is something that Elasticsearch this is something that Elasticsearch does very, very does very, very does very, very well and very quickly, and well and very quickly, and well and very quickly, and we have many different we have many different we have many different ways to ways to ways to implement it. But

implement it. But implement it. But it happens that this doesn't always it happens that this doesn't always it happens that this doesn't always work, and then work, and then work, and then you can switch to a you can switch to a you can switch to a linear combination. linear combination. linear combination. And here we are talking about And here we are talking about And here we are talking about minimization and minimization and minimization and maximization. There are different maximization. There are different maximization. There are different things that can be things that can be things that can be done. But this is where done. But this is where done. But this is where you normalize the you normalize the you normalize the scores. So instead of scores. So instead of scores. So instead of taking taking taking order into account, you instead order into account, you instead order into account, you instead just normalize the just normalize the just normalize the scores. So everything scores. So everything scores. So everything has the same has the same has the same range of scores from range of scores from range of scores from zero to one, and you zero to one, and you zero to one, and you can combine them. And can combine them. And can combine them. And here are some examples of here are some examples of here are some examples of settings. This is, settings. This is, settings. This is, again, an again, an again, an introductory level introductory level introductory level setup, setup, setup, for sure, but you for sure, but you for sure, but you can adjust can adjust can adjust the weights and see what the weights and see what the weights and see what works well. And that's works well. And that's works well. And that's where where where assessment sets come in, and that's assessment sets come in, and that's assessment sets come in, and that's where, you know, we where, you know, we where, you know, we do a lot of do a lot of do a lot of different workshops and different workshops and different workshops and so on, and the so on, and the so on, and the intelligent intelligent intelligent query layer and query layer and query layer and things like that do things like that do things like that do it more automatically, it more automatically, it more automatically, not in a very complicated not in a very complicated not in a very complicated way, but essentially way, but essentially way, but essentially you have an you have an you have an assessment set that assessment set that assessment set that a person can a person can a person can build today using a build today using a build today using a language model, and it's language model, and it's language model, and it's based on the query, based on the query, based on the query, "Here are the "Here are the "Here are the documents that I documents that I documents that I need," and then you need," and then you need," and then you run run run experiments and experiments and experiments and see what weights see what weights see what weights help you help you help you get those get those get those results as results as results as accurately as possible. And I'll accurately as possible. And I'll accurately as possible. And I'll continue here. Did you continue here. Did you continue here. Did you want to add something?

want to add something? want to add something? Sorry. Sorry. I'll add one more point, I'll add one more point, because this is actually a because this is actually a because this is actually a technique that we technique that we technique that we used in the used in the intelligent intelligent query layer architecture, query layer architecture, query layer architecture, which we which we which we talked about quite a bit last year and the talked about quite a bit last year and the talked about quite a bit last year and the year before. year before. Elastic supports Elastic supports the concept of the concept of the concept of search patterns. So search patterns. So search patterns. So technically you can technically you can technically you can have three different have three different have three different search strategies. search strategies. search strategies. One that uses One that uses One that uses RF, another that uses linear, and RF, another that uses linear, and RF, another that uses linear, and perhaps another that perhaps another that perhaps another that uses a uses a uses a combination of linear combination of linear combination of linear or RF with a reranker. or RF with a reranker. or RF with a reranker. And if you have a level And if you have a level And if you have a level above, let's call it above, let's call it above, let's call it LLM, which has a simple LLM, which has a simple LLM, which has a simple prompt: "Based on prompt: "Based on prompt: "Based on this type of query, I this type of query, I this type of query, I want you to want you to want you to redirect it to redirect it to redirect it to this search pattern." this search pattern." this search pattern." Based on this Based on this Based on this other type of query, I other type of query, I other type of query, I want you to want you to want you to direct it to direct it to direct it to this search pattern. this search pattern. this search pattern. You can literally You can literally You can literally get the equivalent get the equivalent get the equivalent of what I call “ of what I call “ poor man’s auto-tuning,” where you poor man’s auto-tuning,” where you literally literally literally switch between switch between switch between different different different search strategies depending on what the search strategies depending on what the incoming search incoming search query looks like. And you have something— query looks like. And you have something— it doesn't have to it doesn't have to it doesn't have to be an LLM—but you have a be an LLM—but you have a be an LLM—but you have a way of analyzing that way of analyzing that way of analyzing that search to decide search to decide search to decide what search pattern what search pattern what search pattern to call. Most to call. Most to call. Most developers who developers who developers who build an advanced build an advanced build an advanced use case use case where they don't have a where they don't have a where they don't have a single strategy for single strategy for single strategy for all queries will eventually all queries will eventually all queries will eventually end up with something like this end up with something like this end up with something like this . It's just a . It's just a . It's just a question of question of question of how difficult it is to how difficult it is to how difficult it is to actually actually actually program. And in the program. And in the program. And in the case of Elastic, many case of Elastic, many case of Elastic, many people don't know about people don't know about people don't know about search patterns, but search patterns, but search patterns, but it's a powerful mechanism it's a powerful mechanism it's a powerful mechanism that can be that can be that can be used to used to used to reduce the amount of reduce the amount of reduce the amount of boilerplate code that boilerplate code that boilerplate code that has to be written has to be written has to be written for this type of for this type of for this type of architecture. This is not

architecture. This is not architecture. This is not covered in the covered in the covered in the labs labs , but it is worth at least , but it is worth at least , but it is worth at least understanding that such a understanding that such a understanding that such a concept exists in concept exists in concept exists in Elastic. Elastic. So the table at So the table at the end, again, the end, again, the end, again, be a little be a little be a little skeptical, but it's a skeptical, but it's a skeptical, but it's a good way good way good way to start, right? to start, right? to start, right? For example, if you don't For example, if you don't For example, if you don't know know know your data very well or it's, your data very well or it's, your data very well or it's, you know, a fairly you know, a fairly you know, a fairly unknown set of unknown set of unknown set of queries, you can queries, you can queries, you can start with RF. This works start with RF. This works start with RF. This works quite well. It's not that quite well. It's not that we didn't invent RF, we didn't invent RF, we didn't invent RF, but we but we but we use it. Other use it. Other use it. Other places use it too places use it too places use it too . And . And . And beyond that, you can beyond that, you can beyond that, you can start start start using other using other using other ways to combine ways to combine ways to combine data. But you don't data. But you don't data. But you don't always need to always need to always need to do hybrid do hybrid do hybrid search. Sometimes search. Sometimes semantic search is enough. semantic search is enough. Sometimes BM25. But Sometimes BM25. But Sometimes BM25. But we usually we usually we usually find that it is a find that it is a find that it is a combination of combination of combination of semantic and semantic and semantic and lexical search that lexical search that lexical search that helps search helps search helps search work work work most effectively. And most effectively. And most effectively. And then lab then lab then lab four is where, four is where, four is where, despite the name, you despite the name, you despite the name, you still still still load it into the load it into the load it into the model. But we'll model. But we'll model. But we'll show you exactly how.

show you exactly how. Again, as I Again, as I said at the very said at the very said at the very beginning, LLMs are beginning, LLMs are beginning, LLMs are extremely, well, " extremely, well, " smart"—not the smart"—not the smart"—not the best word, but best word, but best word, but they are effective at they are effective at they are effective at providing answers, providing answers, providing answers, but whenever you but whenever you but whenever you need to give them need to give them need to give them data that goes data that goes data that goes beyond their knowledge beyond their knowledge beyond their knowledge acquired during their acquired during their acquired during their training, you can training, you can training, you can pretty confidently pretty confidently pretty confidently get them to give the get them to give the get them to give the wrong wrong wrong answer by giving them a answer by giving them a answer by giving them a data set and data set and data set and saying, "This sky is saying, "This sky is saying, "This sky is orange." And then orange." And then orange." And then they they they will usually tell you that the sky is will usually tell you that the sky is will usually tell you that the sky is orange. Um, there are orange. Um, there are orange. Um, there are ways around this, ways around this, ways around this, but this is where we but this is where we but this is where we show that show that show that information retrieval is still information retrieval is still information retrieval is still important for important for important for agentic responses. agentic responses. Quick poll: is Quick poll: is there anyone who hasn't there anyone who hasn't there anyone who hasn't passed their third passed their third passed their third lab yet? lab yet? Okay, great. You Okay, great. You are moving are moving are moving forward confidently.

forward confidently. forward confidently. Amazing. Amazing. It's also worth mentioning It's also worth mentioning that this is where we that this is where we that this is where we move to Agent Builder move to Agent Builder , Elastic's agent platform , Elastic's agent platform , Elastic's agent platform , which has the , which has the , which has the ability to launch ability to launch ability to launch agents, tools, agents, tools, agents, tools, and skills. You can and skills. You can and skills. You can connect to connect to connect to many major many major many major language models. Elastic language models. Elastic language models. Elastic Inference Service has all Inference Service has all Inference Service has all your favorite language your favorite language your favorite language models, but you can also models, but you can also use your use your own. She is able own. She is able own. She is able to run these to run these to run these tools tools tools directly to directly to directly to work with data in Elastic work with data in Elastic , and although this is not , and although this is not , and although this is not covered in this covered in this covered in this workshop, we do have workshop, we do have workshop, we do have workflows. workflows. Workflows are Workflows are essentially any essentially any essentially any ordered set of ordered set of ordered set of tasks you tasks you tasks you want to accomplish, want to accomplish, want to accomplish, including including including decision trees and the like. Agents decision trees and the like. Agents decision trees and the like. Agents can invoke these can invoke these can invoke these workflows and workflows and workflows and also access also access also access external resources. external resources. external resources. You can access You can access You can access other systems, other systems, other systems, call MC Patrols, call MC Patrols, call MC Patrols, call APIs.

call APIs. call APIs. Call out whatever Call out whatever Call out whatever you want. This is again you want. This is again you want. This is again beyond the scope of beyond the scope of beyond the scope of this workshop, but this workshop, but this workshop, but since we have since we have since we have a little time, this is where the a little time, this is where the a little time, this is where the agent gets agent gets agent gets the opportunity to start taking the opportunity to start taking the opportunity to start taking action. That is, not action. That is, not action. That is, not just finding just finding just finding answers, but also answers, but also answers, but also reacting, for example, reacting, for example, reacting, for example, during an incident: during an incident: during an incident: by analyzing by analyzing by analyzing telemetry or telemetry or telemetry or trace data, the system trace data, the system trace data, the system can decide: "Yes, the can decide: "Yes, the can decide: "Yes, the system resources system resources system resources are running out." It are running out." It are running out." It needs to be needs to be needs to be scaled. So, scaled. So, scaled. So, you can call a you can call a you can call a workflow, workflow, workflow, for example, to for example, to for example, to automatically automatically automatically scale a scale a scale a Kubernetes environment. This is Kubernetes environment. This is Kubernetes environment. This is just one just one just one example. But this is where example. But this is where example. But this is where we move on to we move on to we move on to Agent Builder, which Agent Builder, which Agent Builder, which introduces you a little to its introduces you a little to its introduces you a little to its capabilities. Therefore. capabilities. Therefore.

One of the most exciting One of the most exciting aspects of Agent Builder for aspects of Agent Builder for software developers is software developers is that everything you that everything you that everything you see in the see in the see in the Kibana interface is available through the API Kibana interface is available through the API . So if you decide to . So if you decide to . So if you decide to make a quick make a quick make a quick prototype prototype prototype using our using our using our Agent Builder mechanism in Agent Builder mechanism in Agent Builder mechanism in Elastic, you don't Elastic, you don't Elastic, you don't have to have to have to rewrite all the rewrite all the rewrite all the code from scratch to code from scratch to code from scratch to create a real create a real create a real application. You can application. You can reuse the reuse the same endpoints in same endpoints in same endpoints in your application and your application and your application and make the same make the same make the same calls in your own calls in your own calls in your own user user user interface that you interface that you interface that you control. And you control. And you control. And you get the added get the added get the added benefit of being benefit of being benefit of being able to remove these able to remove these able to remove these Agent Builder components and Agent Builder components and Agent Builder components and plug in another plug in another plug in another framework or framework or framework or set of APIs that you've set of APIs that you've set of APIs that you've built or built or built or are using – we'll are using – we'll are using – we'll be compatible with be compatible with be compatible with that too. So our goal is to that too. So our goal is to provide as provide as provide as many ready-made many ready-made many ready-made solutions as possible, but not to solutions as possible, but not to solutions as possible, but not to limit you so that limit you so that limit you so that you are not forced you are not forced you are not forced to build applications to build applications to build applications exactly the way we exactly the way we exactly the way we want them. We are very want them. We are very want them. We are very flexible. Almost too much flexible. Almost too much . So when you . So when you . So when you ask us ask us ask us a question, the answer a question, the answer a question, the answer often starts with the often starts with the often starts with the words "it depends on the words "it depends on the words "it depends on the situation." Is situation." Is situation." Is there anyone who there anyone who there anyone who came recently and came recently and came recently and doesn't have a link yet?

doesn't have a link yet? doesn't have a link yet? Okay, I apologize. Okay, I apologize. Okay, I apologize. That's what I thought, start there That's what I thought, start there That's what I thought, start there . Uh, so it's . Uh, so it's . Uh, so it's in the upper right in the upper right in the upper right corner. Is that short, e l corner. Is that short, e l corner. Is that short, e l a s t? And just in a s t? And just in a s t? And just in case anyone... case anyone... case anyone... came by later and didn't came by later and didn't came by later and didn't hear me say it: hear me say it: hear me say it: access will be access will be access will be open for a open for a open for a week. The workshop itself, and week. The workshop itself, and week. The workshop itself, and the link “blue by James” the link “blue by James” is a repository that is a repository that is a repository that contains all the notebooks. There contains all the notebooks. There is other code there, but the is other code there, but the most important thing for most important thing for most important thing for independent work independent work is notebooks. You is notebooks. You is notebooks. You can connect can connect can connect it to any of it to any of it to any of your your your Elasticsearch clusters. Elasticsearch clusters. Elasticsearch clusters. Free, well, Free, well, Free, well, almost. If almost. If almost. If you connect another you connect another you connect another service. But the data service. But the data service. But the data will load and you will load and you will load and you will be able to go through everything. will be able to go through everything. will be able to go through everything. So, it will be So, it will be So, it will be available until someone available until someone available until someone deletes it. Will not deletes it. Will not deletes it. Will not delete me. delete me. By the way, thanks to By the way, thanks to our esteemed our esteemed our esteemed architect here, architect here, architect here, Sunil. We wanted to Sunil. We wanted to Sunil. We wanted to make sure make sure make sure we provided you with bonus we provided you with bonus we provided you with bonus content, additional content, additional workshop materials. There are workshop materials. There are other other other workshops available online in the same workshops available online in the same workshops available online in the same format as the one we format as the one we format as the one we used used used today, where there is essentially a today, where there is essentially a today, where there is essentially a pre- pre- pre- configured configured configured environment for you to environment for you to environment for you to walk through the walk through the walk through the concepts and concepts and concepts and learn more about learn more about learn more about how Elastic works “ how Elastic works “ under the hood.” We under the hood.” We under the hood.” We strongly strongly strongly recommend that you follow recommend that you follow recommend that you follow this URL and this URL and this URL and take advantage of the take advantage of the take advantage of the workshops. This is workshops. This is workshops. This is essentially free essentially free essentially free tuition. And once tuition. And once tuition. And once you complete these you complete these you complete these workshops, you'll workshops, you'll workshops, you'll probably increase probably increase probably increase your market your market your market value by, well, I don't know,

value by, well, I don't know, value by, well, I don't know, $100,000 to $200,000 $100,000 to $200,000 $100,000 to $200,000 a year. This is a a year. This is a a year. This is a guaranteed guaranteed guaranteed salary increase. salary increase. Okay, while people Okay, while people are working on this, I'll are working on this, I'll are working on this, I'll do two things. So, do two things. So, do two things. So, one of the... Sorry, I one of the... Sorry, I mentioned a few times about providing mentioned a few times about providing new, relevant, new, relevant, new, relevant, fresh and any fresh and any fresh and any proprietary data to these LLMs proprietary data to these LLMs proprietary data to these LLMs as a way of augmenting as a way of augmenting – that's where the " – that's where the " A" in RAG comes in – A" in RAG comes in – augmenting their augmenting their augmenting their knowledge. Another thing knowledge. Another thing knowledge. Another thing that often comes up and is that often comes up and is that often comes up and is becoming more becoming more becoming more relevant, relevant, relevant, especially as agents especially as agents especially as agents start to act start to act start to act faster, is faster, is faster, is the ability to the ability to the ability to cite cite cite sources. So, models are sources. So, models are sources. So, models are very good at generating very good at generating very good at generating information, and often information, and often information, and often it is correct. it is correct. But they are also great at deceiving you, not because deceiving you, not because they have they have they have been specifically taught to do so, but because been specifically taught to do so, but because been specifically taught to do so, but because when there is a fine when there is a fine when there is a fine line between the two other line between the two other line between the two other lines you lines you lines you ask about, and the model ask about, and the model ask about, and the model doesn't know the specific doesn't know the specific doesn't know the specific answer but has answer but has answer but has enough enough enough surrounding surrounding surrounding information, it can information, it can information, it can give an extremely give an extremely give an extremely convincing convincing convincing answer that, if answer that, if answer that, if you are not an expert, can you are not an expert, can you are not an expert, can easily be misleading.

easily be misleading. So, to summarize, So, to summarize, one of the things in this one of the things in this one of the things in this lab that's lab that's lab that's on the on the on the screen right now is screen right now is screen right now is the ability to force LLM to the ability to force LLM to the ability to force LLM to cite their cite their cite their sources. Most sources. Most sources. Most models do not have models do not have models do not have the ability to the ability to the ability to cite cite cite sources because it is not a sources because it is not a sources because it is not a database. These models database. These models database. These models do not look for do not look for do not look for information in information in information in tables and do not tables and do not tables and do not extract it from there. extract it from there. extract it from there. They generate They generate They generate tokens based on tokens based on tokens based on their learning. In their learning. In their learning. In any case, when any case, when any case, when providing LLM with factual providing LLM with factual providing LLM with factual answers for her to answers for her to answers for her to analyze and analyze and analyze and formulate a formulate a formulate a human-friendly answer human-friendly answer human-friendly answer or in any or in any or in any other format, she other format, she other format, she may also cite may also cite may also cite her sources. This is her sources. This is her sources. This is extremely important, extremely important, extremely important, for example, in for example, in for example, in law, where, as law, where, as law, where, as you have probably seen in you have probably seen in you have probably seen in the news, lawyers the news, lawyers the news, lawyers get into trouble by get into trouble by get into trouble by making up answers, making up answers, making up answers, citing citing citing non-existent cases, or non-existent cases, or non-existent cases, or distorting facts. distorting facts. So, if you can So, if you can get a model get a model get a model to handle large to handle large to handle large amounts of complex amounts of complex amounts of complex data, but still data, but still data, but still cite the source, it cite the source, it cite the source, it makes them much more makes them much more makes them much more useful and useful and useful and much more reliable.

much more reliable. So when you So when you get to... get to... let me let me let me try to start this. try to start this. As James pointed out, As James pointed out, one of the nice one of the nice one of the nice features of the features of the features of the Agent Builder platform Agent Builder platform Agent Builder platform is that is that is that it is also managed it is also managed it is also managed via an API. That's why the via an API. That's why the via an API. That's why the lab work in lab work in lab work in this case was this case was this case was slightly modified to slightly modified to slightly modified to show how all this show how all this show how all this can be built can be built can be built programmatically. So you programmatically. So you programmatically. So you can create can create can create skills, tools, skills, tools, skills, tools, and agents programmatically, and agents programmatically, and agents programmatically, allowing you to allowing you to allowing you to create, delete, create, delete, create, delete, scale, and scale, and scale, and modify them as modify them as modify them as needed, needed, needed, leveraging all leveraging all leveraging all the capabilities the capabilities the capabilities built into the stack. built into the stack. So, we move on So, we move on here, and here you can here, and here you can here, and here you can ask questions. ask questions. ask questions. You know, we'll move on You know, we'll move on You know, we'll move on to the review. We have to the review. We have to the review. We have many ready-made many ready-made many ready-made skills “out of the box”. skills “out of the box”. skills “out of the box”. So, these are the skills. I'm So, these are the skills. I'm So, these are the skills. I'm sure everyone here sure everyone here sure everyone here knows this, because you're at an knows this, because you're at an artificial intelligence conference, artificial intelligence conference, right? But a skill right? But a skill is many different is many different is many different things, essentially it's things, essentially it's things, essentially it's explaining to a model or explaining to a model or explaining to a model or agent how to perform agent how to perform agent how to perform a task in a certain a task in a certain a task in a certain way. When to way. When to way. When to use use use tools, how tools, how tools, how to create to create to create presentations, how presentations, how presentations, how to create other to create other to create other services. This is all that services. This is all that services. This is all that can be written down to can be written down to can be written down to help him. So help him. So help him. So he doesn't have to he doesn't have to he doesn't have to figure figure figure everything out on his own. Some are everything out on his own. Some are just ways to just ways to just ways to speed up work.

speed up work. speed up work. Something you need when Something you need when Something you need when you want to get you want to get you want to get very specific very specific very specific results. If you results. If you results. If you want to give him want to give him want to give him basic instructions basic instructions basic instructions as shown here. We as shown here. We as shown here. We provide many provide many provide many ready-made features and ready-made features and ready-made features and skills for you. skills for you. skills for you. For example, if you For example, if you For example, if you want to create want to create want to create graphs, graphs, graphs, visualizations, or visualizations, or visualizations, or perform searches. perform searches. perform searches. But we also But we also But we also build build build tools. The difference is that tools. The difference is that tools. The difference is that the tool the tool the tool performs a specific action. performs a specific action. performs a specific action. The tools we The tools we The tools we provide out of the box provide out of the box provide out of the box are, for example, are, for example, are, for example, launching a search, launching a search, launching a search, viewing viewing viewing workflows. I was talking workflows. I was talking workflows. I was talking about workflows. about workflows. about workflows. This is a way to call This is a way to call This is a way to call external external external services or services or services or run something in a very run something in a very run something in a very specific specific specific deterministic deterministic deterministic way. But you can way. But you can way. But you can create your own create your own create your own tools. If we tools. If we tools. If we go to the agent go to the agent go to the agent that you created, that you created, that you created, for example, and for example, and for example, and look at the overview, look at the overview, look at the overview, it's a pretty simple it's a pretty simple it's a pretty simple agent for the purposes of agent for the purposes of agent for the purposes of today's today's today's presentation. He has presentation. He has presentation. He has one skill. It one skill. It one skill. It describes how describes how describes how to diagnose to diagnose to diagnose the problem you are the problem you are the problem you are looking for. So it's looking for. So it's looking for. So it's pretty short pretty short pretty short compared to ready-made compared to ready-made compared to ready-made skills. But here you skills. But here you skills. But here you can provide can provide can provide as much information as much information as much information as needed to as needed to as needed to help him. This help him. This help him. This helps to find helps to find helps to find the cause and find a the cause and find a the cause and find a solution. It's about solution. It's about solution. It's about looking at looking at looking at error codes. Another thing error codes. Another thing error codes. Another thing he can do— he can do— he has one he has one he has one tool. Yes. So, tool. Yes. So, tool. Yes. So, here we have a here we have a here we have a search tool.

search tool. search tool. But if I add other But if I add other But if I add other tools to tools to tools to show create a show create a show create a new tool. Ahem. new tool. Ahem. This is always an interesting This is always an interesting question that I question that I question that I like to ask. Who like to ask. Who like to ask. Who here knows or has heard of here knows or has heard of here knows or has heard of ESQL within Elastic? Good. ESQL within Elastic? Good. ESQL within Elastic? Good. We have one. Perfectly. That's pretty much what We have one. Perfectly. That's pretty much what We have one. Perfectly. That's pretty much what I I I expected. I was expected. I was expected. I was pleased to hear that pleased to hear that pleased to hear that many of you are many of you are many of you are familiar with Elastic and familiar with Elastic and familiar with Elastic and Elasticsearch, that's good. ES SQL Elasticsearch, that's good. ES SQL is a newer is a newer is a newer query language for Elastic. This is ES query language for Elastic. This is ES query language for Elastic. This is ES Pipe QL. So, it's an SQL- Pipe QL. So, it's an SQL- like language. You like language. You like language. You have have have SQL-style syntax, and the pipe character is SQL-style syntax, and the pipe character is needed to needed to combine commands. You combine commands. You combine commands. You start a process, start a process, start a process, it is passed on to the it is passed on to the it is passed on to the next stage, and so next stage, and so next stage, and so on. That's it. So you on. That's it. So you on. That's it. So you can can can use use use ES QL tools to ES QL tools to ES QL tools to query query query documents in Elasticsearch. documents in Elasticsearch. documents in Elasticsearch. Index search Index search is a pretty interesting is a pretty interesting is a pretty interesting tool. In ES QL, you tool. In ES QL, you tool. In ES QL, you execute a specific execute a specific execute a specific query. You write a query. You write a query. You write a query and get query and get query and get specific fields.

specific fields. specific fields. Index searching is, in my opinion Index searching is, in my opinion Index searching is, in my opinion , a , a , a documentation tool. You documentation tool. You documentation tool. You specify a specific specify a specific specify a specific index or set of index or set of index or set of indexes, describe indexes, describe indexes, describe its purpose, its purpose, its purpose, field mapping, and the field mapping, and the field mapping, and the fields themselves. If you have, fields themselves. If you have, fields themselves. If you have, say, a say, a say, a product index and a product index and a product index and a price index, it will tell you how to price index, it will tell you how to price index, it will tell you how to combine them. As in ES combine them. As in ES combine them. As in ES QL, you can QL, you can QL, you can combine indexes combine indexes combine indexes with each other. He with each other. He with each other. He talks about it. talks about it. Agents, because they Agents, because they can can can iterate and iterate and iterate and explore data, are explore data, are explore data, are usually able to usually able to usually able to figure out the figure out the figure out the indexes in your indexes in your indexes in your cluster. So, if you cluster. So, if you cluster. So, if you ask a question ask a question ask a question and the agent doesn't and the agent doesn't and the agent doesn't know anything, they will use the know anything, they will use the know anything, they will use the search tool, search tool, search tool, look at the mappings, look at the mappings, look at the mappings, or refer to the or refer to the or refer to the mapping tool. mapping tool. mapping tool. It will find all the It will find all the It will find all the indexes and their indexes and their indexes and their mappings, try mappings, try mappings, try to guess which ones are to guess which ones are to guess which ones are related to the products related to the products , and try to combine , and try to combine , and try to combine them. In any case, them. In any case, them. In any case, he will go through he will go through he will go through the options, and this costs the options, and this costs the options, and this costs tokens. And we all tokens. And we all tokens. And we all love spending love spending love spending tokens so much. So, this tokens so much. So, this tokens so much. So, this tool significantly tool significantly tool significantly speeds up the process and speeds up the process and speeds up the process and helps guide the helps guide the helps guide the agent in the right agent in the right agent in the right direction.

direction. direction. Workflows—I've Workflows—I've Workflows—I've mentioned this mentioned this mentioned this a few times, we a few times, we a few times, we can perform can perform can perform tasks in an tasks in an tasks in an orderly manner and orderly manner and orderly manner and interact with other interact with other interact with other systems. And MCP—if systems. And MCP—if systems. And MCP—if you want to access an you want to access an you want to access an external MCP external MCP server, you can server, you can server, you can do that. do that. do that. I should also note that I should also note that I should also note that the tools in Agent Builder the tools in Agent Builder , both built-in and , both built-in and , both built-in and custom, custom, custom, are presented as MCPs. are presented as MCPs. So, Kibana provides an MCP server. So you can server. So you can server. So you can use use any tools any tools any tools from there if your from there if your from there if your agent wants to agent wants to agent wants to connect through MCP connect through MCP . And lastly, you do . And lastly, you do . And lastly, you do this in a lab, but this in a lab, but this in a lab, but this actually shows this actually shows this actually shows how an agent can how an agent can how an agent can do multiple, um do multiple, um , multiple cycles. So, , multiple cycles. So, , multiple cycles. So, first he first he first he loads the skill loads the skill loads the skill to understand how to to understand how to to understand how to answer the answer the answer the question of question of question of why he keeps why he keeps why he keeps getting error 137. Why does getting error 137. Why does getting error 137. Why does this happen and how? This is a this happen and how? This is a this happen and how? This is a two-part question two-part question , isn't it? Why is this , isn't it? Why is this , isn't it? Why is this happening? Why is happening? Why is happening? Why is my cluster crashing with my cluster crashing with my cluster crashing with exit code 137?

exit code 137? So he needs to So he needs to find the answer to find the answer to find the answer to this, and then—how do he this, and then—how do he this, and then—how do he change the settings change the settings to prevent this? to prevent this? to prevent this? This is a two- This is a two- This is a two- part question. And again part question. And again , he launches this , he launches this , he launches this skill and starts skill and starts skill and starts looking for the answer. It looking for the answer. It looking for the answer. It goes through a few goes through a few goes through a few cycles, diagnoses cycles, diagnoses cycles, diagnoses the problem, and tells you the problem, and tells you the problem, and tells you why error why error why error 137 is occurring. In reality, it's the 137 is occurring. In reality, it's the 137 is occurring. In reality, it's the memory manager memory manager memory manager running out of memory and stopping running out of memory and stopping running out of memory and stopping you. Um, then he you. Um, then he you. Um, then he tells you how tells you how tells you how to adjust the to adjust the to adjust the JVM settings to JVM settings to JVM settings to fix this. But he fix this. But he fix this. But he also talks about also talks about also talks about prevention. So prevention. So prevention. So since the answer since the answer since the answer was in Elastic, and he had was in Elastic, and he had was in Elastic, and he had the opportunity to query the opportunity to query the opportunity to query it, run some it, run some it, run some tools, etc., tools, etc., tools, etc., he was able to give you he was able to give you he was able to give you the answer and the answer and the answer and help you move help you move help you move forward. Yes, I mean it forward. Yes, I mean it forward. Yes, I mean it . . Yes, and one more thing I'll Yes, and one more thing I'll add: if you add: if you add: if you 're in 're in 're in active active active prototyping mode and prototyping mode and prototyping mode and trying to get to a trying to get to a trying to get to a search logic or search logic or search logic or query language that query language that query language that really solves your really solves your really solves your problem. This is actually a problem. This is actually a problem. This is actually a pretty good pretty good pretty good tool for tool for tool for developers and developers and developers and debuggers to debuggers to debuggers to figure out what figure out what figure out what search queries search queries search queries will help get the will help get the will help get the data they need. And data they need. And data they need. And the last thing I will say the last thing I will say the last thing I will say about ES | QL: I'm a about ES | QL: I'm a about ES | QL: I'm a big fan of his.

big fan of his. big fan of his. At first I was a bit At first I was a bit At first I was a bit skeptical because Elastic skeptical because Elastic skeptical because Elastic Query Language had features that Query Language had features that Query Language had features that ES | QL. I ES | QL. I ES | QL. I think it's think it's functional enough for functional enough for you to at least take a you to at least take a you to at least take a closer look at it; closer look at it; closer look at it; especially if you are especially if you are especially if you are building agent-based building agent-based building agent-based search solutions, because search solutions, because search solutions, because in ES | QL has great in ES | QL has great in ES | QL has great concepts that are better concepts that are better concepts that are better suited to suited to suited to solving certain solving certain solving certain search problems search problems search problems than Elastic Query Language, which than Elastic Query Language, which than Elastic Query Language, which should be considered should be considered should be considered a superset of what a superset of what a superset of what ES | is. QL today. Um, and one more ES | is. QL today. Um, and one more ES | is. QL today. Um, and one more thing that thing that thing that makes ES | QL is makes ES | QL is makes ES | QL is fantastic, is fantastic, is fantastic, is the ability to the ability to the ability to integrate integrate integrate parameterization into the parameterization into the MCP toolchain and MCP toolchain and connect it to Agent connect it to Agent connect it to Agent Builder, which is a great Builder, which is a great Builder, which is a great addition that is addition that is addition that is technically possible technically possible technically possible to implement outside of ES | to implement outside of ES | to implement outside of ES | QL. But the way we QL. But the way we QL. But the way we designed ES QL, our designed ES QL, our designed ES QL, our Agent Builder, and MCP endpoint Agent Builder, and MCP endpoint is the optimal path is the optimal path is the optimal path for many, let's say for many, let's say for many, let's say , agent-based , agent-based , agent-based search scenarios that search scenarios that search scenarios that you might have you might have you might have . So if . So if . So if you haven't checked you haven't checked you haven't checked out ES QL yet, out ES QL yet, out ES QL yet, I highly recommend you I highly recommend you I highly recommend you do. And if you're do. And if you're do. And if you're trying to keep up with trying to keep up with trying to keep up with some of the some of the some of the innovations we're innovations we're innovations we're implementing in implementing in implementing in search, the first place to search, the first place to search, the first place to look look look is the ES QL documentation is the ES QL documentation is the ES QL documentation to see what to see what to see what new, original new, original new, original search engines we're search engines we're search engines we're developing.

developing. Because we're doing some Because we're doing some really interesting things with really interesting things with really interesting things with federated search federated search federated search and potentially and potentially and potentially addressing addressing addressing different systems within the different systems within the different systems within the Elastic Query Language, that is, Elastic Query Language, that is, Elastic Query Language, that is, queries that are not queries that are not queries that are not considered considered considered traditional for Elastic traditional for Elastic , and these are , and these are , and these are ongoing developments right now. But ongoing developments right now. But ongoing developments right now. But you'll see how it you'll see how it you'll see how it comes into the product, comes into the product, comes into the product, and when it does, ES and when it does, ES and when it does, ES SQL will be your entry point SQL will be your entry point SQL will be your entry point into these capabilities. into these capabilities. Now I'm talking about Now I'm talking about things like things like things like being able to being able to being able to perform parquet perform parquet queries on queries on queries on data external to Elastic data external to Elastic data external to Elastic using an using an using an ES SQL call, which can be ES SQL call, which can be ES SQL call, which can be extremely powerful extremely powerful if you also if you also if you also combine this with combine this with combine this with data from Elastic. data from Elastic. You can see You can see that the agent used ES that the agent used ES that the agent used ES SQL for this call. SQL for this call. SQL for this call. So, here's what it So, here's what it So, here's what it looks like. It can looks like. It can looks like. It can perform actions perform actions perform actions like branching, and like branching, and like branching, and here it combines here it combines here it combines two search threads. two search threads. This is what the This is what the fuse function below does. And fuse function below does. And fuse function below does. And he is able to ask he is able to ask he is able to ask these questions and then these questions and then these questions and then return the answer.

return the answer. return the answer. You see that there are You see that there are You see that there are pipes that allow: pipes that allow: pipes that allow: you launch a request, you launch a request, you launch a request, execute a certain execute a certain execute a certain phase of it, get a phase of it, get a phase of it, get a response, and it response, and it response, and it moves on to the moves on to the moves on to the next stage in next stage in next stage in this pipeline language. this pipeline language. So, it is very flexible in So, it is very flexible in setting setting setting the parameters. Is that so? the parameters. Is that so? the parameters. Is that so? Parameterized. It Parameterized. It Parameterized. It is parameterized, and is parameterized, and is parameterized, and it tries it tries it tries to achieve everything to achieve everything to achieve everything that the Splunk query language that the Splunk query language that the Splunk query language tried to do tried to do tried to do with Splunk in its with Splunk in its with Splunk in its heyday. And then heyday. And then heyday. And then combine that with combine that with combine that with SQL concepts, which SQL concepts, which SQL concepts, which are obviously are obviously are obviously proven, and I proven, and I proven, and I think the last thing think the last thing think the last thing you'll notice in ES SQL is that you'll notice in ES SQL is that you'll notice in ES SQL is that queries have become queries have become queries have become more concise. more concise. more concise. They don't have as much They don't have as much They don't have as much markup markup markup to to to get the get the get the results right, which is results right, which is results right, which is another advantage of another advantage of another advantage of this pipeline this pipeline this pipeline structure. And there are other structure. And there are other structure. And there are other companies that companies that companies that use the same use the same use the same technique to technique to technique to build search. build search. build search. This usually This usually This usually applies to applies to applies to unstructured unstructured unstructured data sets, logs. data sets, logs. data sets, logs. Um, you know, I would say Um, you know, I would say Um, you know, I would say that was the main that was the main that was the main reason we reason we reason we started building ES started building ES started building ES SQL—to provide a SQL—to provide a SQL—to provide a best-in- best-in- best-in- class search experience class search experience class search experience so people could so people could so people could get rid of all get rid of all get rid of all their Splunk. And now we've their Splunk. And now we've their Splunk. And now we've moved on to the point where moved on to the point where moved on to the point where this construct is this construct is this construct is so useful that so useful that so useful that it should be it should be it should be available across all available across all available across all pillars of Elastic: search, pillars of Elastic: search, pillars of Elastic: search, security, and security, and security, and observability. And observability. And observability. And it is a first-class it is a first-class it is a first-class component for all component for all component for all three pillars in three pillars in three pillars in the future, the future, the future, becoming the main becoming the main becoming the main engine for our engine for our engine for our innovation.

innovation. So you see, when a hybrid search hybrid search query is run, the response query is run, the response query is run, the response shows you essentially an shows you essentially an shows you essentially an audit trail audit trail audit trail of how it arrived of how it arrived of how it arrived at those responses. at those responses. at those responses. So not only does it So not only does it So not only does it provide provide provide answer sources at the bottom answer sources at the bottom answer sources at the bottom that you can click on, that you can click on, that you can click on, but if you want but if you want but if you want to dive into the to dive into the to dive into the fascinating world of fascinating world of fascinating world of JVM tuning, you JVM tuning, you JVM tuning, you can do so. Um, I'll can do so. Um, I'll can do so. Um, I'll never never never do that again. But, um, just do that again. But, um, just do that again. But, um, just kidding. One of the kidding. One of the kidding. One of the nice things nice things nice things is also that it is also that it is also that it shows—we shows—we shows—we limit it to five limit it to five limit it to five results—it results—it results—it says it found five says it found five says it found five results. Um, it doesn't results. Um, it doesn't results. Um, it doesn't clutter the UI clutter the UI clutter the UI here for here for here for you, but you can you, but you can you, but you can click on that and click on that and click on that and go to the go to the Discover section. This is kind of a Discover section. This is kind of a Discover section. This is kind of a place where we place where we place where we review documents. review documents. review documents. This is the usual This is the usual This is the usual interface for searching interface for searching interface for searching in Kibana. It in Kibana. It in Kibana. It has changed a bit in recent has changed a bit in recent has changed a bit in recent years if you have years if you have years if you have n't used it in a while, n't used it in a while, n't used it in a while, but now you can but now you can but now you can use use use SQL there. And you see here that SQL there. And you see here that SQL there. And you see here that he returned five he returned five he returned five documents. If I documents. If I documents. If I close some of these close some of these close some of these panels, it will build a panels, it will build a panels, it will build a nice nice nice little graph for you. I don't little graph for you. I don't little graph for you. I don't know if this is the know if this is the know if this is the most useful chart most useful chart , but you can see the query at the , but you can see the query at the , but you can see the query at the top, as well as the top, as well as the top, as well as the actual documents, and you actual documents, and you can go in and can go in and see the whole thing see the whole thing see the whole thing . So, here's the . So, here's the . So, here's the gist of it, and we gist of it, and we gist of it, and we were looking for part of this were looking for part of this ...Oh, here it is. Search by ...Oh, here it is. Search by ...Oh, here it is. Search by URL. So the URL, again URL. So the URL, again , because we provide , because we provide , because we provide the URL as part of the

the URL as part of the the URL as part of the document, we can, you document, we can, you document, we can, you know, essentially know, essentially know, essentially provide a link to provide a link to provide a link to more information more information without having to guess. without having to guess. I think this room I think this room closes at 4:20, closes at 4:20, closes at 4:20, right? right? right? Yes, the timer shows 18 Yes, the timer shows 18 Yes, the timer shows 18 minutes to... minutes to... minutes to... 18 minutes, which means 18 minutes, which means 18 minutes, which means we have a hammer we have a hammer we have a hammer that's about to that's about to that's about to hit us. Yes, that hit us. Yes, that hit us. Yes, that means you have a means you have a means you have a little time to little time to little time to ask questions. ask questions. ask questions. Yes, and there is...Yes, I forgot Yes, and there is...Yes, I forgot Yes, and there is...Yes, I forgot about the question, about the question, about the question, sorry. Um, there's sorry. Um, there's sorry. Um, there's lab lab lab five about... five about... rerankers? The light rerankers? The light just burns out my just burns out my just burns out my brain. Um, so brain. Um, so brain. Um, so reranking is an reranking is an reranking is an interesting thing that I interesting thing that I interesting thing that I think a lot of people think a lot of people think a lot of people don't fully understand don't fully understand don't fully understand why they why they why they use, use, use, so we added it so we added it so we added it to explain why to explain why to explain why it's needed. And if it it's needed. And if it it's needed. And if it loads. I know loads. I know . Yes, we'll see if . Yes, we'll see if . Yes, we'll see if it appears. Maybe there's it appears. Maybe there's it appears. Maybe there's something wrong with me.

something wrong with me. Reranking is Reranking is taking... So, if taking... So, if taking... So, if the queries we've the queries we've the queries we've done here and everything else is the done here and everything else is the done here and everything else is the first stage, let's call it the first stage, let's call it the first stage, let's call it the search stage, search stage, search stage, then in the second stage then in the second stage then in the second stage you can perform you can perform you can perform reranking. So reranking. So reranking. So if you run if you run if you run vector search, vector search, vector search, semantic search, semantic search, semantic search, hybrid search in the hybrid search in the hybrid search in the first stage and first stage and first stage and you get a large you get a large you get a large data set, then you data set, then you data set, then you can take that data can take that data can take that data and rerank it. And and rerank it. And and rerank it. And reranking has... I reranking has... I wish this wish this wish this thing would work. I thing would work. I thing would work. I think it's me. think it's me. Reranking Reranking combines the query and combines the query and combines the query and documents during documents during documents during evaluation together. So evaluation together. So evaluation together. So usually with vectors, usually with vectors, usually with vectors, you know, you you know, you you know, you load all load all load all your data whenever you your data whenever you your data whenever you want, and then when you want, and then when you want, and then when you query, you do query, you do query, you do a search, and it just a search, and it just a search, and it just does HNSW or one does HNSW or one does HNSW or one of the algorithms for of the algorithms for of the algorithms for disks and returns a disks and returns a disks and returns a set of documents. But set of documents. But set of documents. But the request and the documents are the request and the documents are the request and the documents are not actually not actually not actually evaluated together.

evaluated together. Reordering Reordering combines them, but combines them, but combines them, but since they since they since they are executed together are executed together are executed together during the second stage of during the second stage of during the second stage of the query, you cannot the query, you cannot the query, you cannot do a pre- do a pre- do a pre- embedding. So you embedding. So you embedding. So you pay the price of pay the price of pay the price of having to having to having to perform the inference perform the inference perform the inference right at the time of the query. right at the time of the query. Now the way Now the way this is done to this is done to this is done to help this help this help this process is to process is to process is to not not not re-rank re-rank re-rank your entire corpus of data. You your entire corpus of data. You your entire corpus of data. You run the first run the first run the first stage to get a stage to get a stage to get a smaller data set, and smaller data set, and smaller data set, and then rerank then rerank then rerank using a model using a model using a model to, well, to, well, to, well, rerank that rerank that rerank that small small small data set. That's all. I data set. That's all. I data set. That's all. I was trying to lead by was trying to lead by was trying to lead by example, not just example, not just example, not just talk. Oh no. That's talk. Oh no. That's talk. Oh no. That's all. My lab all. My lab all. My lab laptop was dying. So laptop was dying. So if you go if you go if you go here, you'll see it's here, you'll see it's here, you'll see it's right at the end. Um, right at the end. Um, right at the end. Um, again, the points again, the points again, the points below about when below about when below about when you can add a you can add a you can add a reranker. If reranker. If reranker. If you have that kind of you have that kind of you have that kind of delay budget. If you don't...

delay budget. If you don't... if you can if you can if you can take a little bit take a little bit take a little bit more time to more time to more time to let that let that let that second stage second stage second stage run and run and run and generate the generate the generate the embedding, because embedding, because embedding, because the accuracy, you know, whether it's the accuracy, you know, whether it's the accuracy, you know, whether it's answer one or answer one or answer one or answer two, if answer two, if answer two, if it's a critical step, it's a critical step, it's a critical step, that's where we pay, that's where we pay, that's where we pay, you know, a little bit of you know, a little bit of you know, a little bit of delay that delay that delay that comes to the fore comes to the fore comes to the fore . If you . If you . If you prepared your prepared your prepared your documents documents documents well enough in the first well enough in the first well enough in the first stage and let the stage and let the stage and let the agent work with agent work with agent work with them, then you probably them, then you probably don't need a don't need a don't need a re-ranker. re-ranker. re-ranker. But this becomes But this becomes But this becomes relevant when you relevant when you relevant when you need to consider need to consider need to consider sequence or sequence or sequence or order. AND order. AND AND we often see this in agent search usage scenarios. We agent search usage scenarios. We see many see many see many situations where people situations where people situations where people check the assessment check the assessment check the assessment based on the first, based on the first, based on the first, second, and third second, and third second, and third fragments of the fragments of the fragments of the document they receive. This gives you a document they receive. This gives you a document they receive. This gives you a very narrow window of very narrow window of very narrow window of opportunity to opportunity to opportunity to get the right get the right get the right answer if you answer if you answer if you only feed only feed only feed a small set of a small set of a small set of candidates to a large candidates to a large candidates to a large language model language model language model instead of a broader instead of a broader instead of a broader set. And this is where set. And this is where set. And this is where a lot of the a lot of the relevance issues arise. You relevance issues arise. You are giving too much are giving too much are giving too much context. It's like a multiple choice test context. It's like a multiple choice test where two or three where two or three answers look answers look answers look equally correct equally correct equally correct to the LLM and she just to the LLM and she just to the LLM and she just chooses one. And if chooses one. And if chooses one. And if it chooses it chooses it chooses the wrong one, then your the wrong one, then your the wrong one, then your users and beta users and beta testers of your new testers of your new testers of your new agent agent agent search system complain that search system complain that search system complain that the search doesn't work. Very the search doesn't work. Very the search doesn't work. Very common. Very, very common. Very, very common. Very, very common. And, you know, common. And, you know, common. And, you know, often such a often such a often such a re-ranker re-ranker re-ranker will actually be very will actually be very will actually be very useful.

useful. So there are two points or So there are two points or two types of two types of two types of re-rankers re-rankers re-rankers that we, at least, that we, at least, that we, at least, support. This is a support. This is a support. This is a pointwise pointwise pointwise approach. This is what approach. This is what approach. This is what most most most reorderers are reorderers are reorderers are today, and this is what the today, and this is what the today, and this is what the V2 model is. So, the V2 model is. So, the V2 model is. So, the point approach point approach point approach combines the query and the combines the query and the combines the query and the document together, but document together, but document together, but evaluates them independently. evaluates them independently. He looks only He looks only at the request and the document, at the request and the document, at the request and the document, nothing else. The nothing else. The listwise listwise approach found in the approach found in the approach found in the Genie V3 model takes your Genie V3 model takes your Genie V3 model takes your set of candidates— set of candidates— for example, up to 64 for example, up to 64 for example, up to 64 documents or 64 documents or 64 documents or 64 candidates—and candidates—and candidates—and considers them all considers them all considers them all together. And so, if together. And so, if together. And so, if the answers or the answers or the answers or potential candidates are potential candidates are potential candidates are very similar, meaning very similar, meaning very similar, meaning they all say they all say they all say pretty much the same thing pretty much the same thing pretty much the same thing , this is where , this is where , this is where the list approach the list approach the list approach can outperform the can outperform the can outperform the point approach because point approach because point approach because it sees the nuances. it sees the nuances.

He can see He can see nuances within this nuances within this nuances within this list because he list because he list because he sees multiple sees multiple sees multiple documents at once documents at once . This requires a bit . This requires a bit . This requires a bit more memory or more memory or more memory or time, because time, because time, because you need to generate you need to generate you need to generate embeddings for all of embeddings for all of embeddings for all of these candidates these candidates these candidates at once, hold them at once, hold them at once, hold them in memory, and in memory, and in memory, and perform the evaluation for perform the evaluation for perform the evaluation for all of them, as opposed all of them, as opposed all of them, as opposed to processing them one by one to processing them one by one . So, if your data is . So, if your data is . So, if your data is quite diverse, quite diverse, quite diverse, and the answers and the answers and the answers coming in are a fairly coming in are a fairly coming in are a fairly clear set of clear set of clear set of candidates, where you candidates, where you candidates, where you know that 1, 2, 3, 4, 5 are know that 1, 2, 3, 4, 5 are know that 1, 2, 3, 4, 5 are quite different, then the quite different, then the quite different, then the pointwise method pointwise method pointwise method handles it. I do handles it. I do handles it. I do n't know. Sorry, I n't know. Sorry, I n't know. Sorry, I said, "Here it is." Hmm said, "Here it is." Hmm , then the point method , then the point method , then the point method usually works quite well usually works quite well usually works quite well . But again . But again . But again , if you have a , if you have a , if you have a set where potential set where potential set where potential candidates are very similar, the candidates are very similar, the candidates are very similar, the listwise method will listwise method will probably probably probably work work work best for you. And somewhere here best for you. And somewhere here because we want to because we want to be useful, you be useful, you be useful, you don't need to don't need to don't need to run, um, although it's run, um, although it's run, um, although it's a two-step, a two-step, a two-step, two-phase process, two-phase process, two-phase process, you know, phase one is the you know, phase one is the query, phase two is the query, phase two is the reranking. You reranking. You reranking. You don't need to manually don't need to manually don't need to manually execute them as execute them as execute them as separate API calls. We separate API calls. We separate API calls. We can do it all can do it all can do it all at once. Um, you at once. Um, you at once. Um, you do it in

do it in do it in the lab, it's in the lab, it's in the lab, it's in the retriever, you can the retriever, you can the retriever, you can do it. Can you do it. Can you do it. Can you rewrite it? Can you rewrite it? Can you rewrite it? Can you use SQL? use SQL? use SQL? Yes. Yes. Yes. Yes, okay. Um, so you Yes, okay. Um, so you Yes, okay. Um, so you can do that in can do that in can do that in SQL too. But "in one go" SQL too. But "in one go" is when you specify is when you specify is when you specify the reranger you want the reranger you want the reranger you want to use, to use, to use, the model in the query, and it the model in the query, and it the model in the query, and it performs both phases performs both phases performs both phases for you. So you for you. So you for you. So you don't have to don't have to don't have to manage the inference manage the inference manage the inference settings settings separately again. You don't separately again. You don't need to call the need to call the need to call the inference API, then query inference API, then query , then , then , then rerank. You rerank. You rerank. You can just can just can just launch everything at launch everything at launch everything at once and we'll take care of once and we'll take care of once and we'll take care of it for you. So I'm going to it for you. So I'm going to it for you. So I'm going to bring up the bring up the bring up the slide again with a few slide again with a few slide again with a few links. Again links. Again links. Again , in the upper , in the upper , in the upper right corner is what right corner is what right corner is what we did we did we did today. This is available today. This is available today. This is available throughout the week. The throughout the week. The throughout the week. The bottom blue bottom blue bottom blue repository is repository is repository is where the code where the code where the code for this is stored, notebooks, for this is stored, notebooks, for this is stored, notebooks, and the like. And and the like. And and the like. And then there are two red ones in the then there are two red ones in the then there are two red ones in the middle. Um, middle. Um, Sunil's secret Elastic environment. Oh my Sunil's secret Elastic environment. Oh my God. Also with a God. Also with a God. Also with a typographical error typographical error . Um, everyone is going to . Um, everyone is going to . Um, everyone is going to take a picture of this.

take a picture of this. take a picture of this. So, and then another So, and then another So, and then another I'll just prove that we're live I'll just prove that we're live I'll just prove that we're live . . . Blame it all on Blame it all on Blame it all on the lighting. the lighting. the lighting. I do everything on the go. I do everything on the go. Still, I was wrong. OK. Um, and other OK. Um, and other online seminars. So online seminars. So , we have many other , we have many other , we have many other online seminars that online seminars that online seminars that are completely are completely are completely free. And free. And free. And things like working with things like working with things like working with streams, if you have streams, if you have streams, if you have streams of streams of streams of data coming in, it's an extremely data coming in, it's an extremely data coming in, it's an extremely efficient way to efficient way to efficient way to process that data. There is process that data. There is process that data. There is another one on RAG, another one on RAG, another one on RAG, lexical search, lexical search, vector search guides. vector search guides. If someone If someone If someone is doing is doing observability work, for example, observability work, for example, things with open things with open things with open telemetry data or finding the telemetry data or finding the telemetry data or finding the causes of causes of causes of problems, etc. We have problems, etc. We have problems, etc. We have laboratories for this laboratories for this laboratories for this . So, some . So, some . So, some laboratory work on laboratory work on laboratory work on monitoring. There monitoring. There monitoring. There should also be should also be should also be security issues here. Security security issues here. Security is a big platform is a big platform is a big platform or an important part of or an important part of or an important part of our platform. We our platform. We our platform. We do everything: from do everything: from do everything: from attack detection to attack detection to attack detection to detection rules. I detection rules. I detection rules. I know that know that know that new rules are emerging.

new rules are emerging. As we add more and As we add more and more more more agent capabilities agent capabilities agent capabilities to the stack. Have a question? to the stack. Have a question? OK. Let me OK. Let me ask a quick ask a quick ask a quick question. If you question. If you question. If you stay here, do stay here, do stay here, do you you you automatically get a spot in the automatically get a spot in the automatically get a spot in the next seminar? next seminar? next seminar? I just wonder I just wonder I just wonder if that's why people if that's why people if that's why people stay. stay. stay. Of course. I'm Of course. I'm Of course. I'm trying to figure out trying to figure out trying to figure out who's still working and who's who's still working and who's who's still working and who's just resting, which is just resting, which is just resting, which is perfectly normal. Yes perfectly normal. Yes . Anyway, I'll go . Anyway, I'll go . Anyway, I'll go . . We'll assume that We'll assume that most of you are most of you are most of you are just resting and just resting and just resting and not waiting for help, not waiting for help, not waiting for help, but if that's not the case, but if that's not the case, but if that's not the case, you still have 7 minutes and 57 you still have 7 minutes and 57 you still have 7 minutes and 57 seconds. seconds. Sunil, do you want Sunil, do you want to demonstrate something to demonstrate something to demonstrate something for 7 minutes? for 7 minutes? I hope this time I hope this time has been used to has been used to has been used to good effect for those good effect for those good effect for those still in the room, and we still in the room, and we still in the room, and we would love to hear would love to hear would love to hear your feedback. We didn't your feedback. We didn't your feedback. We didn't tell you, but there tell you, but there is a is a feedback form at the end of the workshop.

feedback form at the end of the workshop. feedback form at the end of the workshop. So when you're done, So when you're done, So when you're done, you can you can you can tell us tell us tell us how cool the how cool the how cool the workshop was or workshop was or workshop was or how much you didn't how much you didn't how much you didn't like it. And if you like it. And if you like it. And if you say something bad, say something bad, say something bad, it will hurt our it will hurt our it will hurt our feelings, but we feelings, but we feelings, but we will make the workshop will make the workshop will make the workshop better for others in the better for others in the better for others in the future.