From LIKE to Love: Adding Proper Search to Your Django Apps

Speaker 1 [00:07]

Yeah, hello. Thanks for joining my session today. And yeah, like I mentioned, there probably has been a lot of people speaking about semantic search, but I would like to briefly speak about search in general and describe the ways of how to bring that effectively into a Django application. This is slightly different from retrieval augmented generation. Probably that was the topic that most of the people were covering yesterday, at least. And before we start, let me just make a quick introduction. I'm Kasper Lukavski, and I work at Quadrant. We are building a vector database or vector search engine. That's the name I prefer. But I also run a nonprofit organization that focuses on AI education. And you can find me on GitHub and Twitter at this handle, and this is my website, which is not that up-to-date, unfortunately, but I try to write more about the stuff that I'm interested in. And before we dive into semantic search, I really want to focus on the basics of search because maybe some of you are just not familiar with the historical approach to search. Let's make some things clear. SQL-like operator is not a real search. I know that some of you may have used it. I did that too in many of my site projects, but that doesn't solve anything. like there is so much intent that we can encode into our documents and queries and that will be only like matching the exact term and we never, we won't be ever able to find all the possible documents which are relevant. But I get why people decide to use that approach if you just see the Django's documentation and the section which is about search. First of all, it's marked as an advanced topic. I don't get it. it's a fundamental functionality of any application or majority of applications and even though Django has a separate section on search it's very limited it mentions just some of the some of the methods that are available but Django tries to be database agnostic so there are not that many ways of making a proper search with the default functionalities of it so if you just look at the Django docs then you maybe just start to ignore the building a proper search mechanism and maybe well if you just build a personal blog blog then well in the worst case people will just miss that your great post about vibe coding or whatever topic you you will be posting currently but in many cases the lack of proper search mechanism means losing money because if your customers cannot find the item that they are desperately looking for they they would probably choose a different vendor. And it's also pretty important. I feel like one of the reasons we web developers often ignore proper search mechanisms is that we strongly rely on search engines like Google. And the user experience provided by Google or any other search engines out there, like Flexity, for example, is hard to beat. I agree. Probably we won't be even able to get close to it. But on the other hand, especially with these AI overviews that Google has added recently, generated by AI, you might be just losing visitors on your website if you just let them use Google if they want to find something on your website. They will just get this AI-generated bullshit and probably won't visit your website either way. Yeah, they won't be reading your posts. Maybe they won't be buying your products, because that will be all made up. So, you need a proper search if you treat your project seriously. And there is obviously a lot of mechanisms that you can use right away. There are these built-in Django capabilities and probably many of us use Postgres as the default database to build Django applications. I know there's like a huge fan base of SQLite. This is not that advanced. So let's just have a review of what's available by default and what we can solve with this method. And what we have are just custom lookups and some database functions. And the simplest way is just to use contains lookup. So we just check if a particular model attribute contains a phrase, which is actually a like operator in SQL. And that might be fine. But what if all your queries are lowercase and your documents are, like, case-sensitive? I don't know, maybe New York means the city, but I don't know, maybe your mom just bought a new dog. Well, that might be also a case, and that really matters. So that should be captured by the whole semantic of the data, not necessarily by the presence of particular keywords. But, okay, there is something else for you. If you just want to ignore lower and upper case letters, you can just use this case insensitive variant of it. But what if your query is just a longer phrase and, well, all the words, all the individual words from the query occur in multiple documents, but the phrase as a whole does not occur in any of them. In that case, you will be just missing. You won't be ever able to find anything. There is another idea how to solve that. We can also use treegrams and also remove some of the accented letters. That's especially important for some of the languages, maybe not for English, but if you work with German, that should be also the case. So all these umlauts just have to be removed, and if you just apply treegrams, then treegrams consider all the combinations of three letters and compare how many of them appear in both search and and source strings so inquiring the document still that doesn't solve the problem what if you have a vocabulary mismatch that happens quite a lot like if your data is prepared by the domain experts they would be prone to using specific terminology but if your users are not experienced in that area, they will be describing their intents in rather shorter and simpler terms. So another question is how do we score using any of these methods? How can we say that this is the best match? Are we just choosing the shortest document all the time or I don't know like it's not that easy to say that a particular document is the best match because search is ultimately also about ranking and ranking is based by scoring the documents in some way. There is also Postgres full-text search that might be might be used for that. Just to give you an example like Postgres full-text search is based on some sort of vectors so we represent our data our text as vectors and we break the break the documents and queries into lexems so we We do some sort of normalizations. So if we have this sentence, for example, the quick brown fox jumps over the lazy dog, and if we convert that into a vector, then that will be the output. So words are normalized, so we somehow transform each individual word into its root form. that case jumps became jump lazy it's just spelled differently and some stop words like there and over are just entirely removed from the from the created vector and this numbers actually represents just just the order of this word so how they occurred in the input data so this is some sort of normalization that helps to just reduce the ambiguity and like different forms of the same word and that makes this full text search quite efficient that's like keeping only hopefully only only essential information needed for search but still that doesn't cover the semantic of the of the of the data and obviously if you really want to implement a proper search then traditional relational database such as Postgres is not the best tool for that There is also like a pretty big search ecosystem, however I was quite surprised like the two major projects which are which are built around proper search for Django are Django high stack and Django elastic search DSL and all of these tools are focused on keyword based search and if you just see the check the high stack documentation they actually missed some of the releases so so it's not really up to date and it's usually considered to be implemented when it comes to e-commerce applications that's where we care about search most because we want to sell something and people should be able to find what they need. And if you build this kind of application then you no longer have a transactional database that keep the transactional data like all the purchases but also you have another system which is a full-text search engine and there is like a sync process that puts data from the transactional DB into the search engine. That might be done different ways but these libraries would typically handle that on your own so there is no need to worry about. Still if you create a product that aims to be used by users coming from all around the globe you should be also prepared to handle multiple languages. These are all translations of the same example that I used before, but in case of full-text search you would need to set up a separate search pipeline for each language that you have and on top of that you will also need some sort of mechanism to detect the language that you want to support. That means some probably automated translation of the product descriptions or attribute descriptions because we are not speaking about e-commerce only. But different language for me is also a different terminology already mentioned this example documents in your systems are described by domain experts but your search might be used by everyone so in that case there is also a different language being used by both two groups and if you want to set up a multilingual search using traditional approach then you need to have lots of heuristics to get things done like this normalization and split in text into into the root forms and probably I just don't need to I don't need to convince anyone that search really matters especially if we when we started building some LLM based applications and it is like quickly turned out that we need to have like a proper retrieval mechanism so search is important but there are different kinds of search different kinds of queries obviously the easiest ones are the ones where a person knows what they're looking for. So in that case, if I'm looking for a USB-C cable, I just bought a new iPhone and right now I just need to change my charger, then I would rather know how to describe the product I'm interested in buying. And this is the simplest example, And that assumes that the user knows what they're looking for. However, I can also describe the same item in a completely different way. If I'm not an expert, I can just try to describe it. Well, actually, I would need to be an expert in order to describe it. But there might be some people who just don't know exactly what they look for. And in the first case, in the previous example, let's say you are selling a very specific niche product like some car parts. and each part has a specific identifier which is assigned to it and your customers will typically know the exact identifier that they are interested in. In that case, this traditional keyword-based search should be enough. However, if many of your queries come from LLMs, they are prone to use natural language like queries like this one. Definitely longer ones than just a single two-word space-based queries. So, if you want to handle both cases, then keyword-based search is not the best option. And on the other hand, we all got used to some applications like Google Lens, for example, that allow us to search by image. If I don't know, you have a picture of a product that you're interested in, then why would you even consider to write a query? You can just send a picture. Nowadays it's pretty popular. keyword-based search is limited to text only and obviously there are some other ways to allow to explore this the set of items you have set of set of objects you want to search over like faces and search is actually a pretty complex beast it's not that simple but somehow I feel like web developers including me just treated that's not as seriously as we should and except for this keyword based search mechanism there are also some modern ways of how to do search and all this actually also that the previous methods are all about building some representations and this representations are built in a way that makes search and retrieval just efficient so let's speak about different representations because as a software engineers we are dealing with different presentations on our daily basis for example if we represent some if we represent words in our applications we need to simplify it somehow so we define some models and this models does not do not capture all the details about a particular topic about a particular subject in that case this is an abstract user model from Django obviously it is not like a whole picture of a person but But that's just enough to handle the cases that we need to handle, like logging in the user. This is a loser representation of a particular person. We are just not describing all the details, all the subtle details of a particular user. And well, in math, we also have different ways to represent the same information, like polar and Cartesian coordinate systems. ways to describe the same information and the same goes for semantic search because semantic search is a method that we would like to speak about in semantic search we also rely on some representations that have some specific specific traits so if you have a document the same example like before and if you have a properly trained embedding model those are usually just available there are like many SAS like tools for that even open AI so do like even this big LLM providers also have some some embedding models available and if you pass this input data it should produce a single vector representation and an important fact is that no matter what's the length of the input data obviously there is like a maximum length this is some sort of compression so we cannot expect to pass like a 900 long book and expect to receive like a thousand numbers that will cover all the details of the book. So a reasonable input will be always converted to a vector of the same length and that's a pretty useful property because currently that allows us to compare these vectors somehow. And two different embeddings should be close to each other in some sense if they represent a similar input object. That's how these networks are just trained. And similarity is usually measured with cosine distance that has a lot of benefits. There are various options like Euclidean distance but cosine distance is, I would say, like 95% of the project I've seen used cosine distance because that's like the metric that the models are optimized for. And in our example I would expect this vector to be similar to that one because they describe something that's hopefully truly German and I'm not sure if that's the best example of the traditional German foot probably not but indeed if we check like individual dimensions of these vectors they're not identical but they are close to each other and if we compare a cosine distance those are vectors we can do it those should also be hopefully not that far away from each other that that's because they represent and some similar concepts. And if you think about that, well, vectors are just some sort of multidimensional numbers, and if we just limit ourselves to dimensional space, this is how it could look like. This red dot is just a query, and blue dots are documents. So cosine distance might be calculated because there is like an angle between any two points in that space. we are just selecting the highest matches by the lowest score, like if you compare, if you calculate the cosine distance between the point to itself, then it will be like always perfect match, so we are just trying to get the highest score possible. And in that case, that might be just counterintuitive, those are the closest entries, not necessarily like Euclidean distance base. And a value of one means perfect match, negative one is like completely opposite however in reality you never deal with such negative values it's usually more like 0 to 1 range like how do we how can we like define like completely opposites sentence to any given sentence like it's not like a negation of it well really hard to imagine like practically it's never never never experienced and we are always looking for the best matches so So the similarity is pretty high in that case. And cosine distance has this pretty nice feature that this is normalized to a specific range. And we can almost translate it to percentage score, which is really nice also if you just want to display that information to the user that this is 85% match to your query. So just to sum it up, neural embeddings are used like the most popular way to implement semantic search. and actually the only way that proves to work, but maybe there will be something else in the future. It's like with artificial intelligence and machine learning. Machine learning is just an approach to achieve artificial intelligence and the only one that we know to work quite well. So it represents data with vectors that are close to each other if they represent similar concepts. And just to wrap this up, If you write lots of emails at work, hope this helps, should be really close to just stop bothering me. Obviously, that's something that we have in mind when we write the one on the left-hand side, but we just want to be polite. And similarly, in that example, thanks in advance means I'm already thanking you for doing me this favor, even though you haven't agreed to it. That's what we expect, and that's also depending on the model you choose. You can also fine-tune it. Building these models is actually way easier than building LLMs or fine-tuning LLMs. These models usually have way fewer parameters than LLMs, so training that in-house is not such a big deal, so to say. So you could probably achieve that. These general-purpose models won't be that good at it. And finding these closest matches is actually K-nearest neighbors' algorithm. Well, like one of the most basic algorithms of machine learning that has one huge disadvantage, it doesn't scale well, mainly because if you have a query and a set of documents, in order to find the closest matches, you need to calculate the distance to all the documents. And queries are not known upfront. That means that you cannot cache these distances in any way. So imagine you have millions of documents. For every query, you would need to calculate millions of distances. That won't scale. Like, KNN works practically well if you have dozens or hundreds of vectors, or if you just do one short comparison, then that may just run for a few days, so to say, and then you have, like, the exact results. That's totally fine, but in a running system, you cannot expect your users to wait several hours to get the best shirt that they are looking for and ANN is actually a family of algorithms that tries to approximate a nearest neighbor search and we do that with like success nowadays HNSW is the most commonly known algorithm that does exactly this major major like disadvantage is that it's memory intensive so we just need to like be able to afford it obviously there are some ways to optimize it. Still, vector databases became a new category of databases or search engines that support exactly this. And when it comes to practical implementations, that's something that... I'm building some side projects for my own, like mostly in Django. None of this is actually a huge success, so I can't share it, but what I found out was that I couldn't really implement search because there was nothing, like, created specifically for semantic search. I would need to set up, like, elastic search instance or any huge search engine, which was, like, a huge overhead because I was mainly dealing with hundreds of examples, like, or I was doing some reverse image search. And that's not, like, a practical way of how to do things. So I created this package. still not that popular but hope it will get more popular after after this talk at least and this is actually pretty pretty pretty simple approach of combining Django ORM with with semantic search so I I have just two three screenshots of code that's required to run it in your project so that let me just show it step by step so first of all obviously you need to install the package that's relatively simple and then you need to make some changes in your settings so do there are two components that you need in order to set things up so first of all you need to have this back-end that's how I call it and back-end is a vector search engine or anything that supports this this nearest neighbor search if you just deal with hundreds of examples maybe storing that in memory is just fine for you and you don't really need to spin it up but obviously I use quadrant because I I like this product not only I work here but but also I feel like this is a really nice product and it's also open source so I'm just self hosting it most of the times you said set up the configuration and the other thing is the embedding model that you would like to use and actually it's pretty nice that we had this previous talk about celery because that's a pretty important thing. These embedding models are lightweight compared to LLMs, but still, those are machine learning models that has a few millions of parameters at least, and they are not that easy to be run in a web application. So there are some ways of how to overcome that. But for a very naive approach, that should be just enough. This all-mini-LM L6v2 model is just a pretty lightweight one. You don't need to have like a GPU to run it, even a really like a low-end CPU should be enough. And then you need to have a model that you want to be searching over. In that case, that would be just a product. My product has multiple attributes, but name and description is actually most important to me because those are textual attributes I would like to be able to search over. So, for every model I want to, like, set up the semantic search for, I need to create a corresponding document class. So that's actually based on some commonly used Django libraries, like Django rest frameworks, for example. We are also creating, like, separate classes for each model that we want to handle with that. So that's nothing that's strange. if you work with Django, you define a model, you define the indexes. So this is actually like a mapping of fields to the corresponding vectors. You don't really need to think about these vectors that much. You won't be looking at these high-dimensional numbers. It's more about letting the system know that I want to be able to search over a description and the name. So I may have multiple indexes created on a single model. And how it works in practice, because right now we just defined it, that should be already registered and the library itself will handle the creation of all the necessary collections, for example, in Quadrant or any other system that you would like to use. And then if you want to search, first of all, when you create an instance of your model, the instance of the document is also automatically created and stored in your search engine by by using signals. Then you can search over the documents using the search method that is exposed by the object manager of the document you created. And the search method returns a query set. So this is compatible with Django's ORM. You can also apply some filtering on top of that later on. And this is as simple as passing a name of the property you would like to search over. And then a query, which is a textual query. You are not passing any kind of vectors. These vectors are created automatically by the library. So it's as simple as this. Like these three code snippets should be enough to set up proper search for a model. But of course, there is no free lunch, except for PyCon. But there are some challenges of how to set this up properly. And obviously, embedding models are the bottleneck of the whole search pipeline right now. Because embedding models are also based on transformers. That means that if you have an input data, it is divided into tokens, and each token gets input token embedding. This is like a static mapping. So there is like a dictionary mapping a token into its input embedding. It has nothing to do with the context, so it still doesn't understand the whole context of the data. It just knows that this is a particular word and two words which have similar meaning should also have a similar vector representation. That also means that the longer the document or the query, the longer it's going to take to process that information. Because what happens next is a set of stacked layers of the transformer model with attention mechanism that takes a sequence of token embeddings and produce another sequence of token embeddings which should be more and more contextualized when we pass it through multiple layers. At the very end, we receive a sequence of output token embeddings that we pool to create a single vector representation. This pooling is usually like taking an average of all the output token embeddings. But as a user, I don't really care how things are processed internally. If I create a Django application, then I would probably just stick to one of the SAS providers or just set like an API on my own. There are plenty of open source models that I can host. But there is a pretty interesting concept that was actually created even before the transformer models. If you remember or heard about Word2Vec, that was a pretty nice idea that words might be mapped into vectors and then the similarity might be compared on the word level. However, Word2Vec had a big disadvantage because it was calculated for a single word, so we couldn't compare documents that easily. However, static embeddings have made a huge comeback, and it turns out that we can have a very simple mechanism that actually imitates the behavior of the transformers. obviously it's not that accurate compared to the transformer model but still in many cases we can just accept having that loss of precision because ultimately what we care is to get a single vector representation per text and this static embeddings might be just 400 times faster than the typical open source models and they still provide like a pretty decent quality compared to the to the transformer based ones that that ones can be easily used in your Django application you don't really need to use like a celery queue because that's not gonna take that much time I'm actually doing that with static embeddings and like it doesn't have that much impact on the latency because it's just a simple mapping and really encouraged to have a look at the article it's pretty long description of how they did that, but ultimately as a web developer you can just take an existing static embedding model and use it in your applications. There are also some multilingual variants available. And that might be done because Django Semantic Search already supports sentence transformers, fast embed and open AI embeddings, but I'm gonna implement some more providers pretty soon. And there are also some other things on the roadmap because Semantic Search also has some things that might be just unclear for the newcomers because vectors do not capture everything like if you have some images of your products and if you just convert them into vectors there are also some attributes that won't be converted into vectors that easily like the manufacturer of the specific piece of clothes or the color might be covered but the fabric would probably not same Same goes for the price. Price might be also varying over time, so there is no way that the image can capture that information. So we need filters, and those filters should be applied along with the semantic search. We don't want to, like, perform semantic search, find the top ten results, and then remove all of them because they do not fulfill the criteria defined by these simple filters. Like none of the examples is just cheap enough for the particular person to be considered to be both. So that's something I want to implement directly into the system so you won't be able only to filter that after the semantic search phase. You receive like a query set so you can do that. But you will be able to just do like a filtering applied exactly once the semantic search is done, which is a pretty powerful mechanism to my mind. Also I focused on text embeddings, and I know that text embeddings are important, but semantic is not only about text it's also about images and any data modality you could possibly think of there are models available for both text and image so you can search for images based on text queries and vice versa and also any kind of data that could possibly possess like like videos and audio signals might be also supported I'm not claiming that there are so good models that you can use them out of the shelf still that's completely doable so if you in your case you would like to find some similar music based not only on the description of the music but also based on the real sound you can also do it and obviously that brings us to celery embedding creation is just a pretty time and memory consuming process, so some sort of background processing is needed and that's also something that I'm currently working on, so hopefully that will be part of the next release. Support for Celery or maybe some other frameworks too, so the embeddings, once you index the data may be just computed not in the web server process, just not to block the whole application, but instead in the salary worker. And if you find that interesting, I also would like to announce a hackathon. We are not organizing it, but I just found it interesting. Many European countries are just speaking about sovereignty nowadays, and there is a hackathon happening in Paris in June that will be focused on multiple projects, but Docs is one of them, and And Docs is like a European alternative, open source alternative to Notion. And they also would like to build semantic search or search for the notes. So if you would like to participate, please feel free to let me know. Maybe we can build a team and implement that together because I will be definitely trying to attend the event. And if you have any questions, I'm happy to answer them right now. And this QR code points to my LinkedIn account. Thank you very much.

Speaker 2 [34:49]

Okay, we do have I did neglect to mention this but I guess at the end of day two everybody's familiar with the process of questions We do have a few questions Are you familiar with sonic it offers suggestion and fuzzy search for a lot of languages and is a single rust executable

Speaker 1 [35:11]

Well, I had this name, like, I'm not sure if that provides semantic search, too, like fuzzy search and semantic search are just different things. So I'm not really sure. I would need to check this out. I heard the name. Unfortunately, I can't tell that much.

Speaker 2 [35:24]

it's just a maybe a suggestion for you then our approaches like TF IDF a

Speaker 1 [35:25]

It's just a

Speaker 2 [35:31]

worthwhile approach if you don't have enough resources to host a DL model have you compared with more traditional approaches yes

Speaker 1 [35:40]

Yeah, sure. So TF-IDF is actually one of the methods to implement keyword-based search, so it still lacks, like, the real intent. Like, the main difference between keyword and semantic search is that semantic search is thought to not only understand words and find documents that have an overlap between the terms of query and the document, but also to capture the semantics so you can use different language or maybe just different words to express the same idea. So that's an alternative and actually like this full-text search implemented by Postgres is probably based on some sort of TF-IDF. I would probably advocate for using BN25 which is just a different variant of TF-IDF but yeah that's an alternative but it's still keyword-based search.

Speaker 2 [36:27]

Okay, and will the library remain tied to Django? It would be great if it could be used as a standalone package.

Speaker 1 [36:36]

Actually, if you just check the sources of this, it's relatively simple. Like, you have two components, like a mining model and this backend for doing search. You can implement that for any application. But this is done specifically for Django, because I felt this is a common issue, that there is no easy way to build that for Django applications. But yeah, I just took and combined multiple, multiple other libraries. So it's relatively simple. I wouldn't create another layer of abstraction just to provide that search. I wanted something that would be tightly coupled with Django ORM.

Speaker 2 [37:14]

Do the embedding models use semantic ontologies, or can they be trained on anything?

Speaker 1 [37:22]

they can be trained on anything like it's not that easy to train them I have to say that like you need to create create a data set and hopefully a data set of a really good quality and this data set is usually like two different examples and some sort of similarity measure between them and you don't need like tens of them but you need hundreds of thousands to to be able to really build a good model you can of course fine-tune them but still you can do it for any kind of data you could possibly have still it's a manual process to probably properly like label this data to make it usable for that but it's doable of course

Speaker 2 [38:00]

Okay, I'll wait to see if there's another question 530. I guess people are Yeah, okay. Well, thank you for everybody for sticking with it till 530 and thank you to Casper for this presentation