RAG for a medical company: the technical and product challenges

Speaker 1 [00:04]

Hello everyone, very glad to be here today. So we're going to talk about building a RAG for a medical company. And we will both talk about the technical and product challenges. So with a quick show of hands, who here is familiar with RAG? That's great. Just to make sure I have the right level for the talk. I see there are basically pretty much half of the room who does not really know it, so I will try to cover the basics as well. So very quickly about myself. I'm an engineering manager and the general leader at SICARA. It's part of the Theodore Group. It's a service company building data science and data engineering products for their clients. And from my experience, I first worked as a data scientist for six months in OCR and adverts in a company called Realitix. And then I've been working at Seacara for 3.5 years. Some projects I did, just to give you a quick idea. So a lot in vision with object detection videos matching to similar products. So it was already a lot of vector databases before the Gen-I height. Risk detection in all platforms. Iraq for medical company, which we are going to talk about and the error detection invoices, which is another generic use case So the Vidal which is the basically our source documents for for this project It's it's a company It's very weird because we don't see the picture on the left Okay, no, it's better So the Vidal, it's basically the reference for medical knowledge for all health professionals in France. So it's not really well-known here in Berlin, but in France, if you're basically over 30, 40 years old, you will see it on all the doctors' shelves, it's a red book, where they are basically looking for dosage, secondary effects, and stuff like this. So it's been around for a while, there are a lot of users, mostly professional users. It's in a lot of countries, and there are over 30,000 monographs. So a monograph is a document which says for a given drug what are the signal effects, what is the dosage, et cetera, et cetera. So we just asked this question, and now let's do a quick demo. Just as a quick disclaimer for the demo and all these presentations, basically in the code all the documents and the prompts are in French, but basically I used Google Translate on the website and on everything that's going to come afterwards, so it looks like it's in English. But basically, so if I come here and I say something, how long should I stop Eliquis before operation? So Eliquis is a drug. So imagine you're a doctor. you have someone coming to you who has a question about doing an operation because he broke his hand or something, and he's on some drug, and you want to make sure you are allowed to stop it before actually operating the person. So you will ask the question, you will see the documents which will be found. It's basically in the Vidal, so in the 30,000 reference monograph, it will find the documents which look the most similar. So this is the first part of Frag, which is retrieval, finding the most similar documents. And I feel like the connection is not great, but it's finally working. And it will find the documents, and then on the documents, it will try to answer the question. So here, the answer is not great, but I assume it's due to Google Translate. So, and then from the official source, so here it's from the Vidal, you can see highlighted in yellow, the Apixaban, which is the medical name for Helichis, here. It needs to be stopped 48 hours before surgery, so you know you need to stop this drug before actually operating your patient. And if I just show you as well the demo I prepared, which is very similar, it's just to show you that the AI also formulates an answer, but since the page is transcribed live with Google Translate, sometimes there are some problems. I think I'm just going to go on my phone for the Wi-Fi. So, yeah, here where you can see that the AI formulates an answer. A Pixaband should be discontinued, et cetera. But since the answer is generated with AI, you know, there can be a lot of hallucinations, et cetera. It's very important to also base it on a proper document and highlight in yellow the answer so that you know where to look. So, this is basically the end product, now we're going to look at how we build it. So a little bit of theory. This is a monograph, so it's a document which can be over 40 pages with a lot of information. So it's for the Prezista, which is a drug, I don't know for which kind of sickness. But you can see it's very long, here you can see the small slider here, so yes. So finding the right information in such documents can take quite a while, and there are, for the same drug, there can be a lot of monographs, because there can be a lot of derivatives, which all contain different information. So for the same active ingredient, you have the generic name and the brand name. The brand name is basically the main one, and then the generic name is all the derivatives that comes out of the main drug. You have the dosage form, so do I eat it, do I drink it, etc., etc., and all these combinations leads to a different monograph. As we saw, a monograph can be very long. This is why finding the right information in these documents can take over three minutes to a doctor. It's not that long, three minutes. But when you go to the doctor, in France, it takes around 10, 15 minutes. The fact that they spend so much time finding the right information instead of focusing on the patients can be a problem. So our first thought to do such thing is can we just do it with CHAP-GPT? So it does work, how long should I stop the leak before an operation, it provides the good answer in 30 seconds, there is a lot of useless text which makes it basically longer to read, and more importantly, there is no sources provided. We know generative data is subject to hallucinations, so a doctor can accuse it to basically give a drug to a patient. So we can use retrieval augmented generations. So retrieval augmented generation is really from a base of documents retrieving the right information which is what we saw in the demo we did so we are retrieving the documents on Alicus and then from these documents asking the question of the user to asking the question of the user on these documents to answer the the question so with a quiver which is a retrieval of multi-generation solution of the shelf so you just drag and drop your documents and then you can see it It cites the document which allows to answer. It gives a good answer in 30 seconds, and the answer can be customized as well. So it seems quite good. But is it performant enough? So to know if it's performant enough, we started collecting a data set from real FK professionals. So the way we did it is doing our user interviews at the start of the project to validate our convictions. We also asked them what kind of search did you do in the past week in the Vidal to basically collect a data set of 100 questions that the doctor asked themselves over the week. So this is really for evaluation purposes. We are not gonna train on 100 questions. It's not that big, but it's a good place to start. So this is an example of user questions. Are the side effects of Clamoxil in liquid form and capsule form the same? I'm not a medical expert. I'm not really sure what Clamoxil is. But really, what the user is asking here is, are the different forms the same? It took 30 minutes and 16 seconds to the doctor to find the information in the VDAL. The answer obtained was yes, and here are the monographs he used, which contains a part of the answer. So then we are gonna define a metric. So we have a question, it goes through our copilot, it gives an answer, and we define three levels. Okay, if a doctor, which will help us to validate our results, consider that it's OK. I don't know if the system specifically says, from all the information you have given me, I haven't found anything relevant. So it's really in the prompt you specify that. You can just say, if you don't find anything in the document provided, just say you don't know. And KO, if the model says, consult a doctor, because it does not make sense to say, consult a doctor if you're already a health care professional, if it's a false information or an incomplete information. And our initial objective was to have zero KO questions out of these 100 questions, and the least I don't know answers. So this is the architecture we used initially. And this is basically a typical architecture for an off-the-shelf RAG, where you have your monographs, you have your documents. Then you're going to split your documents in a quite random way. So there are different methods. We took kind of a dummy one to start with, where we just, every 8,000 tokens, we would split the documents. Because we are using OpenAI Ada, an embedding model, and the limits of this model was 8,000 tokens. We put it in a vector database. We're using Qdrant. It's one I really like. It's actually based here in Berlin, because it's both open source and managed. You use the same client for your experiments and for deployment, so it has a lot of advantages. And then a health care professional in consultation asks questions about the medicaments. They will write their question in the search bar. The question will go through the embedding model. So basically, the embedding model is a model that represents, that will represent, give a vectorial representation of the question. The same way it will give a vectorial representation of all the subparts of documents you will have in the Vidal. And then you will compare the representation of your questions to all the representation of the subpart of the VDAL to retrieve the most similar documents, so the most similar chunks of the VDAL, basically the ones which are relevant to answer your question. You put them all in a prompt and then ask GPT-4 Turbo to, from these documents, answer my question with a prompt saying, you're addressing health professionals. If you don't find any information, say you don't know, et cetera, et cetera. So this is the performance we have at the beginning. So it's not too bad, considering we haven't put any specificities in it. So we really took a dummy thing. So we have 40% of right answer and 20% of answers we are judging as KO. But it's definitely not enough to bring value to in production, to users which are health professionals. So just checking the time. So here's an example of a KO answer from our data sets. Are the side effects of clamaxid in liquid form and capture for the same? So this is a question we just read. So as we can see, the model says it's important to consult the leaflet or a healthcare professional, which is something we don't want. And also, among the sources retrieved, we retrieve sources on Clamoxyl in the capsule form. We can see it here. But we don't see anything, there is no sources retrieved about Clamoxyl in liquid form. So we cannot give a complete answer to this question, because there is not the all the documents we need to answer this question. So the answer is incomplete. This is an example of a don't know question. What precautions should be taken when prescribing detentiel for diabetic patients? And you can see that among all the sources none of them corresponds to or anything relevant to the question. So the model just answer, I have not found any relevant information. Could you provide more context or rephrase the question? So the model specifically says it doesn't know. And finally, an okay question. So this is about tablets of propanolol. And it found the right sources to answer the question and the answer it has given is actually good according to a doctor. So once this initial model is created, we can already ship it to build a prototype, which will be used by beta users to start gathering their first thoughts on the tool. So to retrieve the, to monitor this prototype, we used Longsmiths, which can be really easily deployed. It does not have to be used with long chain if you're not a fan of the tool, but Longsmiths is really complete, really great. It allows to basically, every row you see here corresponds to a user interaction. And when you click on a specific row, you can see all the model interactions which happened. So here, first, there was first a model which rephrased the question. Then we had some similarity search. Then another model which answered the final question. And what's great in Longsmiths is that you have this button called the playground, which allows to replay a prompt. And this is really great, because you can actually put this into the hands of a business collaborator, which will be able to iterate on the prompt engineering. Because as I already specified, I'm really not an expert of medical knowledge. So while our product owner was really an expert in it, so Allowing this person to play with the prongs was a major plus for the project, as we will see in a couple of minutes. Performance, we will try to raise it. So the first thing we can see is that, as we saw, we used a naive segmentation of chunks. So again here, before putting the documents into the embedding model which will create the vectorial representation, we need to split it in subparts because there is a context limit in the embedding model. So the thing is that since we just split it every 8,000 tokens, sometimes the start of the section was missing. So here, it starts with the end of the section on a fertility, pregnancy, and breastfeeding. Then it contains a section on driving and using machinery, and only contains the start of the side effects. And the end section of side effect is also missing. So you don't get the entire information. The other problem with that is that this is not only the information is split in the middle, is that the embedding, so the vector representation, it will try to represent something which is a bit on fertility, something which is a bit on driving and using machinery, and something which is a bit on side effects. So it will be, basically, the ending vector in the semantic space will be somewhere in between all this section, and it will be neither close to a question on machinery or a question on side effects, it will be a bit far from everything. So what we did is that since we know how the HTML works, of course it wasn't that easy because the data quality is not always perfect, but basically you password the HTML, you put some custom rules, and then you can decide that all your chunks are always starting with the beginning of a section and ending with the end of a section. And on top of that, so you're not, we don't have the split problem anymore, but you also, a chunk is one section, so your embedding model will only represent a single section, Which means the corresponding embedding will be very close to questions on this specific section. So just doing this, it improved a lot the performance of the model. It reduced the errors by 30%. And if we look, for instance, at the questions which failed earlier, it's not failing anymore. It managed to retrieve the liquid form of the Clamoxyl. And so we have all the information needed to answer this question. So in only four weeks, we managed to, so sorry, data in French, but I hope you can do a bit of translation. It's December and January. In only four weeks, we managed to basically reduce all the errors to 0% and drastically reduced the I don't know answers. What we can see is that the largest improvements really happened during the third week. So what happened then? It was a very easy trick. We just added in the prompt. So basically we added a prompt to reformulate the question, basically because sometimes how long to stop Eliquis before operations, how long before surgery should you stop taking Eliquis, etc. Rephrasing these questions helped the model and what we did as well, and this is really what helped improving the performance, is we asked the GPT when rephrasing the question to include the international non-proprietary name. Sorry, it's quite hard to pronounce. It's basically paracetamol for doliprane. I'm not sure if doliprane is a drug in Germany. It's not, OK. So basically, sometimes you have a drug, And then it's also known by its drug name. There is a molecule and a drug name. This is kind of the mapping. Sorry, I won't be able to give any good example in German. But basically, I would have never had the idea to do it because I did not know what it was. And it's actually the product owner which thought about this improvement playing around with Longsmith and the Playground replaying the prompts and seeing it was working well and seeing, sorry, it was working well so it tried on a couple of results it saw it improved it then it gives the tips to the data team we run the battery of tests with all the pipeline and we see it actually improves the performance. It's actually great because it's the first time that I see a product owner contributing to the RAD operation in such a great way. So it's definitely changing the way we are interacting with each other. Why did it work so well? Because then when we, if we go a bit, if we come here, then although in the prompt we only talk about helices, the retrieval parts, here we are adding also the apixaban word, so the retrieval model knows it can output documents about helices or apixaban. Because while we are rephrasing the question with GPT-4 Turbo, while GPT-4 is a huge model and has been trained on a lot of data and so knows how to do the mapping between Eliquis and Epic Saban, the embedding model is a much smaller model, trained on much less data, and so it may not know how to do this mapping. So basically we are extracting some knowledge from the LLM to help the embedding model. Of course, later on what we want to do is directly use the VDAL APIs to do this mapping, which will be more resilient, etc. But this was a much easier way and a faster way to poke the feature. So after the two improvements we just mentioned, the two changes are the split of chunks, which is now better, and the reformulation of the question with the INN. So of course there are a lot of other small improvements, I won't go into all of them because we won't have enough time. But at the end, we end up with 88% of okay answers, 12% of I don't know answers. Is it enough to gain adoption? So to ensure that the product is adopted by our users, we rely on the theory of Martin Kagan, which wrote a book called Inspired. Who here is familiar with this book? a few people. So basically it's inspired a lot of theory on product developments. It's helped creating some great products in the Silicon Valley like Dropbox and a lot of others. And it basically highlights four risks which if you don't address one of these risks your product basically won't work the first one is value basically does your users perceive value in using your product so do the does the doctor perceive a benefit in using it usability will doctors be able to use the tool feasibility can we build do we have the right resource to build and maintain the product. And the last one is viability. It's basically the business plan. Do you have a business plan? Is this solution viable from a business standpoint? So we will focus on usability and value for the end of this presentation. So starting focusing on usability, will doctors be able to use it? So before building this model, we made two assumptions. The first one is that AI is never 100% reliable, so we'll have to cite the sources. And the second one is doctor only trust reliable sources, so we will need to display the VDAL directly in the tool. So this is what we did at the so here you can see that it's citing the answers and you have directly the HTML from the VDAL displayed in the response. The doctors are used to it so basically the tool will look familiar to them. To build this interface we use the Chainlits. It's a framework that you use with Python which is very useful. You can basically deploy a chatbot very quickly. And actually all the front components are built in React. So if you want, you don't have to, but if you want to customize the front-end as we did, you can basically reuse the Chainlit React components and basically tune them. So you have your Python back end and your React front end, which are interacting with each other to basically do all the right logic and display the right answers. But then when we put it in the hands of the doctors, the first thing they are telling us is I don't understand how the tool chooses the sources to give me the answers. Can't I choose? The second thing we noted is that the cited monograph is too long. I can't see where the information is taken from. So what we did to address the first problem is that we made the sources selectables, so that the doctors can have some control on the tool. And they knew what the answer would be based on. So they can either use to choose a generic or brand name. So Cordaron and Amiodarone. Amiodarone is a generic, and Cordaron is a brand name, I think. So it's a derivative from Amiodarone. They can use a specific dosage if they want to. For instance, Eliquis 2.5 milligram or Eliquis 5 or they can choose to base their answer on documents, which is what I did in the demonstration at the beginning. The monograph is too long, I can't see what information is taken from, which was another feedback. Indeed, here we are citing a single section, but the section can be up to five pages, So you don't really know what was used to base your answer on. So we basically decided to highlight in yellow the part of the document which is used to answer the question. And how we did it, so we are using open AI functions. So basically we have a prompt which says respond concisely to the question based on the following documents. You put all the documents. And then cite the short passages from the documents on which the answer is based. And basically OpenAI functions allows to retrieve JSON formatted answer. And in the JSON you will ask to have the start of the passage on which the answer is given and the end of the passage. You don't ask it to cite the full thing because then you will waste a lot of tokens. And then it says find the minimum at the start and two tablets every day at the end. And then you can just find it in the document and then use some mark tags in the HTML in the front end to highlight the part in yellow. The second focus we had was on the value. So does the doctor perceive a benefit in using the product? So simply, we looked at the search time they had. So for instance, for the specific questions, of course, here I'm overfitting on specific examples to give concrete examples. But we had also the overview on the entire interviews, of course. Can I use the Lovenox if my patient has a glomerular filtration rate of less than 32? I don't know what it means either, no worries. It took 2 minutes and 42 seconds to the user to find this answer, and they obtained the right answer looking for it. And so we compared it with our search tool. So I typed very slowly to simulate the doctors using the... It can take a while. I will fast forward a bit. So first, it's rephrasing the question. So they can choose to base the answer on a specific document, or all the documents. Here they are choosing all the documents. And it sometimes takes a while. The streaming is not activated due to a buggy version of OpenAI, but after around 51 seconds they have their answer the right information of the Vidal is highlighted for them so it took around 50 seconds so it went from 2 minutes and 42 seconds to 50 seconds so there is definitely a gain when you know they spend 10 minutes with their patients every time Thank you So what's next? We still have some feedbacks. It's a new tool. There is a real challenge for doctors to be able to use it. So for instance, we put it in the head of someone else, and they just type Cordaron. Because the way Cordaron is a drug, that's the way they are used to search in the Vidal. They don't know. They need to ask a question. was like, oh, I actually need to ask a question. Then we were rephrasing the question because we are basically displaying the rephrasing of the question just to make sure if it's buggy, then the doctor can see it, et cetera. But then the reaction was like, does it think I'm an idiot? To save cost also, because OpenAI is really, really much cheaper than serving your own model when you don't scale, when you have a few tens or hundreds of queries a day. But it's much longer. It's much more expensive when you have thousands of queries a day. So we were considering deploying, using an open source model. So at the time, it was a couple of few months ago, the state of the art of the open source was Mixtral, the Mixture of Experts of Mistral, and the state of the art of closed source was GPT-4. So comparing basically SOTA and SOTA, we see that Mixtral is really not as good as GPT-4, and you really lose the magic of GPT-4. Also, the evaluation is done manually with the help of the doctor, and it really limits the size of data sets and the frequency of evaluation. So if you want to expand our data sets, we need more automatic way to evaluate. So one thing is to set up automatic metrics for retrieval. If we have the documents, stuff like hit rates, I'm short on time, so I will go quickly on that. Or using LLM as a judge for your generation, basically an LLM to evaluate your LLM, but still combining it with some manual evaluation because it can be dodgy sometimes since you're using AI to evaluate AI. You can see how it can start to be dodgy. And of course, we need to separate our evaluation and test sets. we are just using 100 questions, we need to separate them to make sure we don't overfit in some features which are specific to the evaluation sets. So very quickly on the conclusion, because I'm running out of time, to ensure we build a product that works, we focused on the entire machine learning life cycle, starting from data preparation, engineering, evaluation, deployment, to monitoring and evaluation. I think it's very important to point that out because since poking has become very easy with LLMs, sometimes all the evaluation parts and stuff like this kind of go in the bin, which is why sometimes some products struggle to work well. Thank you very much, and I hope you enjoyed the talk.

Speaker 2 [38:23]

Thank you in a way for your whole presentation and thanks to all of you for your questions You have wrote on Slido if you don't have a question just read through them and you can vote them actually up to get them Answered first the first question is how do you meditate? Hallucination and prompt rephrasing for example in GPT generates an incorrect INN

Speaker 1 [38:48]

This is a very good question. It actually happens that the ANN is not completely wrong, but sometimes there can be some confusion. So at the moment, we are not mitigating it. What we want to do is basically basing ourselves on the API of the Vidal, which contains this mapping, which will make it much more resilient. Just to give some ideas, we did this project a few months ago and we are going to take it back in a month to basically continue it.

Speaker 2 [39:23]

How do you handle the tables and the documents?

Speaker 1 [39:26]

So, this was one of the performance problems, actually the tables were badly handled, and we simply transcribed them to CSV, so comma separated values, so we just separate them with a comma, since there are never hundreds of columns, but it's more a lot of rows, actually GPT is not that bad if there are only a few columns.

Speaker 2 [39:57]

Since you are running on GPT-4, you are probably hosting on Asia. How do you make sure you are GDPR compliant, especially since you are dealing with medical data?

Speaker 1 [40:10]

That's a good question, basically all this, right now it's only a prototype so we haven't really given the thought about GDPR, Vidal has already a lot of infrastructure and everything in place to deal with GDPR in the medical context. So this will be really taken into account in the second part of the project.

Speaker 2 [40:43]

How do you optimize rack, recall, and retrieval if the number of documents increases? What were your findings? What is your total number of documents in your DB?

Speaker 1 [40:53]

Again, a great question. So basically what we saw is that there are still 12% of questions where the model says I don't know. Actually all these questions, it's because they are on a lot, it's basically what are the secondary effects between a list of ten different drugs. So we can't answer right now these questions. What we are planning to do is basically reusing a tool of the Vidal which already exists, which allows to compare different drugs. And basically if we identify there are too many drugs, it's usually, it's always actually that kind of questions. So we will basically call this tool, so do an API call to call it, and basically use the LLM to generate the right call for this tool so that it just, instead of displaying an answer, it displays the interaction comparison tool.

Speaker 2 [41:55]

Have you explored integration with patients records to always inject relevant medical history into queries?

Speaker 1 [42:03]

Sorry, can you go, Anna?

Speaker 2 [42:04]

Have you explored integration with patient records to always inject relevant medical history into your queries?

Speaker 1 [42:13]

No, because it's really, it could be done later in time, but it's really something when you have a patient in front of you, and the idea is not to replace the doctor, but kind of to help them. So we want to do something end-to-end, but really just answer simple questions. But it's something that might be done maybe in the future when we want to be more specified on the usage.

Speaker 2 [42:42]

There are so many more questions left, but the next tab is coming up. Thank you for answering our questions. No, thank you