Are LLMs the answer to all our problems?

Speaker 1 [00:07]

So let's see if the Chris is really working because I have a limit of 50 euros because it's for free so please connect because I want to see if you're using LLMs and How and everything so yeah would be great if you would answer a few questions Let's see So we need to hurry up a little bit on this quiz So first question is coming up So I want to know why do you use an LLM or a large language model like ChatGPT or anything? Do you use it for your daily tasks to create articles, to create LinkedIn posts, Instagram posts? Or do you use it for your own product development? Or is it more just for fun to create Ghibli images or some hero image for you? Or do you avoid an LLM? So, yeah, please give me your result about it. I mean, have you, did you do, you can't, why, previously, is it still, I mean, okay, then just raise your hands, maybe, who's using it for your daily task? Oh, quite a lot. Okay. And for product development in your own product as well? Someone avoiding LLMs? Great. I mean, I love this answer as well. And just for fun, creating a Ghibli image. Who created a Ghibli image? Great. Okay. Then we go just through like this. And which of the LLM are you using? And I would just go through the most popular one. using ChatGPT for all these tasks? Okay, and who's using LAMA? LAMA model, Meta, Meta AI, and Mistral? Oh, a few of them. Cool. That is pretty cool. I mean, if you ask me, I always use LAMA or Mistral because of certain reason you will see later in the presentation. And what do you think? Does your LLM have less than 10 billion parameters or higher what do you think higher higher okay good great so it is all I wanted to see to get a feeling about the audience and what you're really using and why so now let's go through the presentation so we already have heard during the keynote just right before that large language models they have a lot of downsides and I would like to go through them and I I hope that in the future you can really take all this knowledge with you and check if you really need a large language model. So one of the negatives part, as we already have heard, is that it has a big bias. I mean, all the LLM models have bias in their database and everything. So this image I just created one week ago with DALI. So you see here very well what kind of bias is still available in these kind of models. I use the prompt here to create a picture of a legal assistant. I use always this kind of example because I work in the legal area. I talk to a lot of judges, legal assistants. And to be honest, I know that they don't look like this. I mean, they are pretty young. They are all female. I mean, they are very stereotyped as a woman, I think. But they don't look like this. And if you ask or if you say, okay, give me a picture of a judge instead of a legal assistant, what do you expect to see maybe you don't want to see of course everything what we expected we see old men and i mean i talked to a lot of judges in the court and they are pretty young i mean they are pretty young and i think they are equal male and female so this is really not the reality but at the end we see that the data it was taken to be trained is completely biased and if you ever run your own AI model let's say independent if it is like if you create a language model or something else if you normally pick just a random database but you're not taking care of the data that is included and this kind of data is always biased it's super biased and if you see for example just simply taking the ratio between male and female in the database for images you see most of these databases, they are not equal, showing women and male. And if you just check the upper one, the databases of the upper one, you see that if you look for distribution over different countries or people from different countries, you see that most people are white on these images. And what we see normally in science is that most images in the Internet, and the whole Internet, represent male, white male, between 30 and 50 years old. So if you ever try to have like a face recognition AI software, keep it in mind to adapt your AI system to it. I mean, for generating images, as I have shown, it is just for fun and you know what you want to see and if you want to have more diverse images and everything. But this can have also legal consequences. So if we take a look just to America, I mean, this newspaper was just from some years ago. It was not recently taken. but they still use face recognition in the police departments to find criminal people. And since the black people are underrepresented in the databases, we have here wrong decisions. And in the European Union, this is always a big case. And we always try in the European Union to be more equal, to reduce the bias and to avoid these kind of mistakes because, I mean, they have legal issues here. So this is why we have the European AI Act. And I know if you have worked together with AI systems and tried to set up your own AI development, you always come up with the AI Act and always say, oh goodness, the AI Act is destroying innovation. And yeah, I mean, it doesn't help, doesn't innovate and everything. But at the end, the AI Act is here to protect our rights. It says that we are equal and that we need to protect the humans and need to protect our rights that we have. So what they did, the European UNIS, is they have like different categories for the AI systems. They have the AI category like in the lower ones, so where you don't need to have any regulations, you just need to show, okay, that users interact or interfere with an AI system like a chatbot, that you say, okay, you're now talking to a chatbot or not to a human. And then we have a high-risk AI system and all these AI systems that you maybe use, for example in courtrooms, maybe might be in a high-risk AI system and the top of the pyramid of the high-risk systems are the AI systems that are forbidden here in the European Union. And the example I have just shown you to use AI within the police department is one of these forbidden AI systems in order to protect our rights and that this kind of thing will not happen here in the European Union. So everything that is together with face recognition and in real time in public spaces is forbidden here in the European Union. And I think it's a very good idea to have it and also like emotion recognition and also to say, okay, if a worker is good or not, or if like having a social scoring is also forbidden here in the European Union. So next time, if you hear about the European AI Act, please consider it is not here to block innovation, it is here to protect our rights. Another regulation that we have within the European Union to protect our rights is GDPR. Have you ever heard about GDPR? Do you like it? I mean, sometimes it's like, oh my goodness, GDPR again. This is maybe most of the reason why projects are failing or not starting because of GDPR. All the people say, oh, we need some information and we violate GDPR. but at the end it's really protecting our rights, because at the end if you put in some information, personal information, you have the right that all this information is forgotten in the AI system. If you ever use LLMs, large language models, you know that everything that you put into this AI system will never be forgotten. And this is a big issue. So it's a big conflict with GDPR. If you ever use a large language model in your product development, take care to make masking of personal information or make an anonymization process in order not to take any personal information for training. Because if you take it once, you cannot remove it anymore. And Catherine is talking about it tomorrow, I think. It will be a great talk, I'm sure. So just go to this kind of talk, how we can forget about this kind of information. And now with this meta-AI you have on WhatsApp, I think it's a risky thing. I mean, at the moment, they are not taking any information to use it for training, but they will. They already mentioned they will use this data and information for further training. Keep it in mind. Another conflict we have with large language model is the mass data collection. You know that large language models, they use or need, require a big amount of data to be trained and to be good, and also to reduce the bias. but in GDPR it is written you should only use as much I mean the lowest number of data as possible and always explain why you use this kind of data so it's a big conflict here with GDPR and I cannot give you an answer what is right or wrong because this is still under discussion and it's still an open issue and still discussed under lawyers another point that was also mentioned in the keynote is copyright and I think most of you are not aware of copyright and how it affects your work and also your usage of large language models because you have two different types of copyrights within your large language model systems so I mean every data you take more or less or you can take data from the internet for your training large language model training in general we have like a fair use so take any data that you see use it for your training it is allowed but on some of data. There's a blocking installed so it is not allowed to use this kind of data for your training. So also crawlers should not see this kind of data because they are protected under copyright. If you build your own AI system keep track of it. Don't use this data for your own training. And at the end what you also need to take care if you're a user so if you generate for example Ghibli image you might violate copyright as well because if the output of what you just created or the AI created is too close to an original one that already claims copyright you may pay for it and at the moment Ghibli the company behind Ghibli is checking if you violate copyright if you're creating these Ghibli images yeah so please check out if you create a Ghibli image and in the future really distributed to the world because at the end it's not chat GPT or open AI who's taking the course of your lawyer it is you you have to pay it at the end but especially for the beginning I mean what I just mentioned taking all the data and put like a block on it and so that course cannot see this kind of information this is something you should be aware of but to be honest none of the search engines are aware of. So this study here is just from a few weeks ago, which shows that all the search engines, like ChatGPT Search, Perplexity, or Perplexity Pro, they still find information out of documents that are behind a paywall or blockers, which means they violate somehow copyright here in this sense. So if you ever use a search engine, and if you use this information out of the search engine, And be careful if you violate your copyright as well. And it's not only violating copyright, but also the issue if these citations you're using or you're getting from these search engines are really right or not. Because what we have seen here in this study, I mean, please also check it out. I think you made some photos of it, but here's the source. You get the details about everything that is in the study. What we see as well in this study is that most of the citations are not right and some tools like ChatGPT, they are really confident showing you citations that are not really the right ones. So always be careful, always check where your citations are coming from, if they are right or not, be aware of it. So this is the one. And you see that for every system it is pretty the same and I mean, to be honest, I was a a little bit disappointed about Gemini, that it's so bad. I mean, I don't know. But what was also pretty interesting is that how bad Copilot performed. I mean, many of you in your company, you use maybe Copilot and the performance is not that good. Really disappointed. So, I mean, I still use a search engine. I'm not using Google because of certain reason. I will just tell you why in a second about carbon footprint, but let me come to this. Anyway, I mean, if you use large language model independent as a user to create images or text, or if you use large language models for your own product, please always think if it is really required, if you need to use a generative AI models. Pretty often with customers, I see that they use generative AI for everything. They use it to get some resources, citations, create images, text articles, everything. But it's not always necessary. I mean, we have something, for example, translations, we have DeepL already. I mean, and it's efficient working on translation. And it's, so it does not consume as much energy as, for example, ChatGPT, yeah? And it's good and in translation and also in improving your writing you have also deeper right here it's working perfectly fine i always use it so please always try to think if there's an alternative a good working alternative to these we always have for some customers i mean in the legal area many customers just want to have an extract of information of documents pdf documents and here we could do it with generative ai but we could also use a simple name entity recognition for example the one that is produced by Ines and her open source project it runs on your laptop, you can install it on your laptop and so what is the profit from it is it is small, keeps data privacy because all your data stays within your laptop it takes less resources because it doesn't need any or doesn't require any high performance computer and it doesn't cost much money so always think if you really need a generative AI system And as I mentioned already, generative AI system, they have a big carbon footprint. And now I have a second quiz with you. You have to work again a little bit with me. I want to figure out if you can imagine what the carbon footprint of certain AI models is. So in order to get a feeling about carbon footprint, I give you here some numbers. So you as a human or a person, you need around 10 ton of CO2 equivalent if you live here in Germany for one year. So one person, 10 ton kilo CO2 equivalent. One US car in its whole lifetime, including the production as well as the fuel for its whole lifetime, requires around 57 tons of CO2. Really by producing, having all these energy consumption, fuel and everything. So what do you think? What is the carbon footprint of training and image recognition AI? So ImageNet trained with ResNet. What do you think? And now again your thumbs. Do you think it is less than a person in Germany, in between a person and a car or higher than a car? What do you think? You can install it on your laptop. Okay. Okay, so I think most of you decided it's more than a car. It is 0.001 ton CO2 equivalent. So just training face recognition, yes, to train the model. You can run it on your laptop. I mean, you don't need a high-performance laptop. Your laptop is more or less like cooled down by the air around, so it doesn't require much. And now the big question. What do you think is the carbon footprint of GPT-3, training GPT-3, just training of GPT-3? Do you think, again, it is less than a human, in between human and car, or higher than a car? Okay, perfect. I think you got the right feeling. So you see, you could have 10 and run 10 cars. It's the same amount of CO2 as training one GPT-3 model. And now you can say, okay, the training is just happening once. But if you worked in science once before, you not only train once. You train several times until you are happy with your result. And at the end, the models become bigger and bigger. I mean, this is what we see at the moment. We see it with LAMA. We see it with GPT. They become bigger. And I think it's getting worse. And the point is, the number I've just shown you, they are inofficial. They are not published. I mean, they are published numbers, or people try to calculate it, but they are not published by OpenAI. And only LAMA published its information about carbon footprint, but not OpenAI, and I think it has a reason, yeah? And if we think, okay, or if you tell me, okay, but inference doesn't take that much carbon as, for example, training, I mean, you're completely right, yeah? But here's just a comparison between the carbon footprint of a text-based request and the image-based request for 1,000 each. So you see that for a text-based request, you produce 5 grams, and for an image-based, you produce 200 grams CO2. So it is 40 times more. Just keep it in mind next time when you create an image. so and what we see in the last weeks is that people were creating images like crazy even sam altman said okay please calm down a little bit i mean we have to run the servers and everything and create the infrastructure that you can produce the image your image and just imagine what kind of carbon footprint it took to create all these ghibli images yeah and i mean the ghibli company behind wasn't happy with this. But Sam Altman was pretty happy because you gave him a lot of images and pictures to be trained on, yeah? Just keep it in mind. So what you can take from this talk is always think if you really need a large language model. They are really good in supporting us as humans. I mean, they are good in supporting with finding some ideas, how we can structure a presentation, or just get some feedback about an article or text. I sometimes use Lama or Mistral to say, okay, just give me an information or feedback about the article I wrote. Do you think it is consistent? Do you think the audience I write it for, it is a good article or not? I mean, but don't always try to use it for fun. I mean, at the end, we live just on this planet and we only have one planet. So keep it in mind for the future. And I hope you could take something with you and you learned a little bit. And I hope I'm happy to connect with you. via LinkedIn or just chatting now.

Speaker 2 [20:40]

Thank you very much. I already have some questions on Slido, and I remind everyone, if you have a question, the format is you post here, and then we ask the question out loud. So one of them here is that generative AI is often given by C-level management, even by engineering point of view, this keep it simple principle is the basic for good software engineering. How do you deal with that, if it's a given at the C-level management level? So if

Speaker 1 [21:07]

So if I get like in large language model and then how I deal with this I

Speaker 2 [21:12]

I think the question is if the management assumes that you're using that as a tool in terms of lowering your time or keeping it simple, how you manage that. If somebody who asked the question is here, maybe they can correct me if I'm interpreting this correctly or not.

Speaker 1 [21:32]

The question is always why you use it. I mean, it makes sense. In some work, it makes sense. But in daily work, it's always the same case. I mean, if it doesn't make any sense, then don't use it. What we see in programming is sometimes for newbies in programming, it supports you, it helps you. But at a certain level, the AI is not as good as a good programmer. So there's no reason why you should use it. Always think about if you should really use it or not and what are like the security levels of everything. I mean, you could just, while programming, just add a code and this code is somewhere in the cloud and used maybe by OpenAI for further training so everyone knows what you're coding. It is always also security-wise not a very good idea to use it always. So I think, I mean, but in general life, I think always make your own decisions. I mean, you can improve your work, but always keep it in mind if you really should use it.

Speaker 2 [22:34]

Thank you. Another question here is about, it's a big question, how do we find balance between protecting rights on one hand and then fostering innovation to keep Europe competitive to other countries?

Speaker 1 [22:46]

That's a very great question. As I said, the AI Act is always seen as a blocker for innovation, but in general, the AI Act has a very great article. It's about the AI sandboxes. And if you really think about further on the sandboxes, they are a great booster for innovation. So the idea is to create like a little sandbox, like servers or something, databases, and to train your own AI on it and test everything over there. So it would even improve your innovation because you could just go to a safe space let's say, get all this data or train on there and check if this use case is really suitable for government, for courts or anything and everything is within a protected room. So and I think it's a big improvement to what we have already. I mean we cannot simply take any data from a court because it is not secure at all and I mean I mean, they are not digital, they are not available in a digital format. And if we would have something like the Sandbox, it would be a big innovation booster, I think. And to be honest, there are these high-risk AI systems, but most of the AI system that we have here within Germany, they are not falling under the high-risk environment. Because the point is that these risk systems are based on the whole European Union. And Germany is not one of the problematic states for using AI systems, to be honest. And this is also what the European Parliament said. So I think you're safe using AI and also training AI.

Speaker 2 [24:26]

Thank you. I also have a question from the viewpoint of the person who doesn't know anything about legal. What are the strategies and challenges of enforcing these new laws that come with AI nowadays? Because this is not a simple thing to just enforce and then expect everybody to abate, right? So what would be the challenge and then what would be the way to actually enforce the laws that come with new technology?

Speaker 1 [24:51]

I think you should always have a legal counsel within your company giving you advices. The big issue is that you should have a legal counsel that is really focused on AI related regulations, which is a little bit difficult because it's a new topic and it's not taught in the universities. But at the end, I mean, really don't worry. I mean, most of the things you are just doing right, more or less. And I think in Germany we have a good feeling about data protection and safety and everything. And most of the cases, they are just big news. And at the end, if you're really working on it, there are other issues than regulations. Also in the AI Act, we have talking about that you have to teach your employees about AI, that you must teach your employees about AI, but this is not really a must. Keep it in mind. You should, but it's not a must. So it's good for everyone, for every developer who knows what AI is and how to use it, what are positive and negative parts and what kind of regulations, but you don't have to. It's not a must. And it doesn't, I mean, a developer doesn't need a regulation course, a deep regulation course. He should just know the basics and that, so just, yeah, I mean, try to go down a little bit. It's not as big as we all think.

Speaker 2 [26:17]

all right there are a few more questions but I'm just going to pick the one that is closest to your talk right now because you already had similar content so there's a question if you can talk about the trends in LLM energy efficiency and we understand that the larger models can be less efficient and they use more energy but could it also change in the future and could we have more efficiency so that this carpet footprint is actually pretty

Speaker 1 [26:38]

actually reduced? I mean we have seen it from China with DeepSeq that we have more efficient AI systems so I think in the future also with better GPUs that's it becomes efficient and more efficient but my worry is at the end at the moment AI must be a business case. At the moment it isn't. OpenAI just produces all these AI system because they have a lot of money from investors but in the future they have to make money. So the question is what is the balance between and also OpenAI has to be more efficient and reducing the energy cost. So hope in birds let's say that they try to make everything more expensive but also the hope to make it more efficient energy wise.

Speaker 2 [27:26]

Thank you very much, so there are some more questions, but unfortunately we don't have time to ask them I thank you very much again. Let's thank the speaker again