Loss Function Theory 101

Speaker 1 [00:02]

All right, awesome. First, a disclaimer. This one is a 101 talk, so it's an introduction. If you're very good in mathematics or you're a statistician or something, we'll leave. No, just joking. You can stay in the room, but don't expect too much, okay? Before I start, my name is David. I work for the FDI Research Center for Information Technology. We're a non-profit research institution located in Karlsruhe, and we work on applied informatics. and we have a mission of doing knowledge transfer that means we try to transfer universitary research impulses into the industry in the private sector with many interesting applications cyber security robotics assisted living smart mobility systems autonomous vehicles and last but not least also the energy sector and i'm telling you this because well to fulfill our mission to transfer something we obviously need some private partners and we're always looking for companies that either wish to just get in or start a project with us or that want to be participate in some publicly funded research project so if you're interested in this oh and in this particular thing well don't hesitate to approach me or go on our website there's a lot of nice people on there which all work there which can answer you every question you probably might have around this all right let's go on um what are particularly doing and i was very pleased about the talk before because it just introduces or motivates what i'm actually trying to do is um well we have an energy transition to uh to face that means we will have high share of renewable energy generation in the future energy sector and that also means that we cannot go any further with just you switch on anything as the moment you like but because we will have energy or electricity available once the wind is blowing or the sun shines and that means we can either really push a lot of money into storage technology or we try to flexibilize our demand and that's what we're trying to do we try to develop algorithms to actually make demand more flexible in my particular case it's about building so i try to optimize energy consumption behavior of buildings to make it more efficient but also to um to make it such that it stabilizes the energy grid and as i'm not trying to do it with one building because one building won't stabilize the grid obviously um i try to do to develop approaches that actually work on a large number of buildings and that motivates for me using machine learning or particular reinforcement learning actually but i'm facing the same issues that you taught before so i'm all about tackling the sample efficiency issue okay so while i was starting with machine learning a few years ago i just picked up some some tutorial books and i've read like okay you have a regression task go for mean squared error or you have a classification task then go for cross entropy And anyway, if you once you apply your loss functions, you need to optimize to get a good result. And you should apply stochastic gradient descent. Our little disclaimer, you can apply usual gradient descent without the stochastic part too, but I will use it synonymously here in this talk because it doesn't really matter to my point of view. But the question I ask myself is why? So I really try to understand what's happening if I apply such a system Because if I just feel like the guy in the comic who just has a huge pile of machine learning something and poke in it all the time, then this might sometimes work. And I might find an algorithm that actually solves chess or something by chance. But I don't really understand how it worked anyway. And that's what really bothered me. So what I'm trying to do is actually I've really tried to get into the theory and understand why do we use those last functions? why are they the recommended approach and why do we why does it work at all and i tried at least a bit of sharing that with you so you might hopefully have an intuition afterwards not all the math i will show you a lot of math by the way um that's not too important but hopefully an intuition what actually happens behind the scenes so let's get going first thing we're going to do is mean squared error and in the beginning of any data science project you probably have some data Okay, so this is our sample data set for the moment. We have n vectors of input features, and we have n corresponding vectors of targets. So we denoted the input data of features as capital X, and the other one as capital Y. And you might, for a simple example, just that one you see above there. So it's just the x-axis is our features, the y-axis are our targets, and what you see is just some samples we observed. And every point, every green point, is one particular example we observed. So what we're now going to do is, or what supervised learning is all about, at least in my opinion, is finding a function that generalizes those observed data. So what you see in the plot is actually a blue line that is the true data-generating function, which we do not know in reality, but we try to fit a function that actually comes close to our unknown function, or at least come close to that, I think, to what we think later will be the true data generating function. The nice thing is if you perceive supervised learning as this, is you don't have an artificial intelligence problem, which seems like, oh shit, how should I, sorry for that, how should I solve artificial intelligence? But all you need to do is curve fitting. And curve fitting looks much more nicer from a mathematical point of view, I think. One thing we can observe in here is that our points that we observed are not from our true data generating function, but we have some noise on top and that noise is usually like any real process will have some noise in there too the question is just how large it is and the noise adds a stochastic element into our problem so what it means for us is we cannot just fit the points directly we need some statistical tools to do so and the tools that we're usually applying are called maximum likelihood estimation maximum likelihood estimation is for now just a very simple concept it says that we should use those parameters or the best parameters that we can find for a function approximator are those that maximize the likelihood of observing our targets if we give in our features into the system and adjust at the parameters okay but that sounds a bit abstract probably and It doesn't really give us a way to compute stuff. So if we assume IID data, which is a common assumption, especially in machine learning, then we can compute this as just a product of all the probabilities for every single example that we've observed. So every point in our chart, we've got to compute one probability for it, and then we're just going to make a huge product out of it. I'm correct from a mathematical point of view. The point is, it's not very nice to compute, because you're going to multiply a lot of zeros, and that's going to be, in the end, it's just zero, or close to zero. The point is, what usually people do is they will make a logarithm around it, like introduce a logarithm here. That doesn't change the position of our best parameters we find, but it just changes the value of the probability function. And so it's okay from a mathematical point of view. and what people also do is i don't know you know machine learning people they don't like maximizing stuff they like to minimize error um so they put a minus on top of it and then they're going to call this negative log likelihood and the point is why i'm telling you this is that this term is quite common in machine learning literature you will find it on all over the way as soon as someone is a bit more fancy then we'll talk about negative log likelihood so you have an idea hopefully now where this term is coming from okay so this was a bit mathematic obviously I'll try to give you an example how this works okay so we're going to make I think the fundamental and most easiest example I consider is fitting a Gaussian distribution to some observed data points we have a probability density function of a Gaussian and please note now that for those who are strong in mathematics they know that this is wrong what if they know that like the notation is wrong this is a for a discrete probability distribution, please ignore it. It yields the same results. I don't want to explain all the differences between continuous and discrete cases here too, because it doesn't fit into 30 minutes. But apart from that, the general intuition remains the same. What you see here is the probability density function of a Gaussian. That means, this is the blue curve you see here, and the blue curve is just giving us an intuition about in an area under that curve, how likely are we to observe data here. So speaking of that, we are very likely to observe data here and not very likely to observe data here. And what you see also here is some green points, should be 10 of them actually, that are samples derived from this distribution. So our problem that we're now trying to solve is how can we reconstruct that blue curve that we don't know? And what we're going to use is we're going to apply maximum likelihood estimation. That means we're going to, in that case, minimize the probability or the logarithmic probability of observing data points from a Gaussian distribution while we optimize our mean and standard deviation as parameters. And if we're going to do that, you get this result. The nice thing about Gaussians is that Gaussians are actually very simple. As we heard in talk yesterday, too, if you got into that Gaussian progress talk, Gaussians are very simple. And that means if you have a Gaussian distribution, you can actually solve this equation with conventional classical extreme value analysis, like you did in 12th grade, I think. And if you do so, if you apply what we've just learned before, maximizing probability, you will receive just an estimator. The maximum likelihood estimator for a Gaussian distribution is actually just the arithmetic mean. So it's not very surprising at all. Okay, same for standard deviation. You just do differentiation, set to zero, solve for standard deviation, and the maximum likelihood estimator is, well, what you usually or at least nearly compute a standard deviation, apart from some minus one that's usually added here. Okay, and if you do that, actually, and you reconstruct the Gaussian distribution, you see that from our 10 points, We reconstructed our Gaussian. The original one was with a mean at 10 and a standard deviation of 1. And the reconstructed one is 9.95 and a standard deviation of 1.05. So it's quite fairly okay. But that not so much data we perceived. So, Yao, you're trying to apply machine learning, don't you? And you may tell me, well, David, I don't want to fit Gaussians. I don't care. I want to have a fancy neural network. so how does it relate well let's try to generalize that so what we all we do is to generalize this to general machine learning under supervised learning in particular is all we're going to assume is that we have a function now that generates a mean and that function might be our neural network or linear regression or whatever doesn't really matter so far and then we are going to assume that our points have been sampled from this function but with some standard deviation around it with some gaussian noise and we're going to apply our function here a function approximator with our parameters and the features as mean and going to put our true observed values here and what we're going to have is now the probability of observing data at any point on our function approximator and to give you a more of intuition how this looks like how this might look like is um have a look here this one in this plot the the right curve is actually the um the mean of the function approximator uh the green dots are samples and the contour plot in the back is the probability of observing data okay and what you see here is that your probability of observing data in that particular case is just like a tube around our mean so that's at least something okay um this is the hopefully uh last slide with a really lot of math on it maybe maybe not um what we now do if we're going to apply this okay this general framework is we're just going to try for parameters max make a maximum likelihood estimation again plug in our function and then we receive some function which i which i do not expect you to read too much the point is we can simplify it a bit we can reinsert our an estimator for standard deviation that we computed before for gaussian and what we'll receive in the end is this one here and i think that one looks quite familiar to most of you because that one is just ordinary mean squared error so the very point of the slide is actually this one that minimizing mean squared error is equivalent to applying maximum likelihood estimation that under the assumption of gaussian noise okay so that's why mean square error actually works so the one one other thing we have to solve now is we have to uh solve the differentiation problem and what we're going to do there is we're going to apply stochastic gradient descent and the reason for that is actually that um if we or if we would try to apply an analytic extreme value analysis that means that we have to for every function approximator every time you change something in your neural network, you have to make a recomputation of your analytic approach to find maximum values. And I think in the most complex function approximators, you probably won't find no analytic solution at all. So what people do in that, indeed, to circumvent this is they make a stochastic gradient descent, and that just means we compute our loss, in that case the main squared error, at a particular starting point. We're going to follow that gradient, we're going to compute the gradient that is just the differentiation at that point regarding to the loss function, the parameters follow the gradient some distance and then we do the recomputation again until we met some stopping criterion and hopefully at least somewhere got close to some minimum of our loss function okay so what does it imply? Well, the point is that if we comply in main square error and stochastic gradient descent, what it yields us is a universal tool for fitting any differential function to data, assuming Gaussian noise and utilizing maximum likelihood estimation under the hood. The point about Gaussian noise is it's a reasonable default choice for real distortions. Much of measurement error is actually or often assumed close to be Gaussian noise. but you should be aware that there actually problems exist where this is not the case okay one particular example is meantime to failure of machines which is often assumed to be viable distributed and if you try to predict one of those systems where you have a very strong assumption that your error is not gaussian distributed then this might not work or might at least not give you good results then there's maximum likelihood estimation if you apply classical maximum likelihood estimation you have two guarantees that are very very nice well the first one is that maximum likelihood estimation is actually the asymptotically best estimator in terms of convergence if you have if your number of examples reaches infinity so that's good because it tells us okay this is the best estimator you can get and on the other hand it also tells there's a guarantee that we can find a true generating data generating function if our approximator can resemble it and if we have infinity data. Infinite data? Okay. Don't worry. The problem with that is that relies on the assumption that you actually found the extreme value. And if you are not able to do so, which is the case for Stochastic Gradient Descent, then this might not be guaranteed anymore. You might still find or you can hope for that it still somehow applies, but you don't have any guarantee for it. And that's actually the catch if you apply stochastic gradient descent that you should be aware of. Okay, so indifference to other optimization methods where you have, for example, a convex problem and you can be guaranteed to find the extreme value. One last thing about how does it work. Well, I told you in the beginning that I'm trying to give you an intuition of what actually happens. And I tried to do this with this slide. What you see here is a training process for an example we've seen before. And this one here is just the initial values. So the initial values as I defined them. This is the logistic function we're trying to fit to data. The green points here are actually our observed data again. And you see here that our fitted function is very far away from this one here. The contour plot is the computer standard deviation. And what now happens while we train is actually that the mean of the function approximator gets closer and closer to our real data. And as it does this, it can actually reduce the standard deviation. And that means that the points that are actually close to the mean have a higher probability value. and that's what's really happening under the hood if you apply mean squared error okay so once we hopefully got a little further here next one is cross entropy well cross entropy is usually used to classify images or as a classification loss function and say you have those images of cute animals i'm not sure if i've ever seen a cute fish but it could be close to that um and what you're not trying to do is solve or your data would look like this you have like your features are actually your images um and your targets are uh just a one hot encoded label which tells you what is it so this has a hundred percent probability of being a dog this has a hundred percent probability of being a cat and so on so forth um what we're now trying to do is oh no sorry um one thing i should tell you about this is that we can assume this as a probability distribution too so similar to our gaussian we have seen before we can also assume that this is a multinomial distribution and that multinomial distribution describes us the probability of observing that particular category and it has nearly no constraints apart from that every probability must be equal to zero or larger and that the sum of all probabilities must be equal to one. So what we're actually trying to do is this. We have our target data and we have a model or predicted distribution and we try to make them equal. And at the moment you see there that this doesn't match so good. So what we actually need to do is we need to compare probability distributions. And the thing which is usually or often applied to compare probability distributions is called Kohlberg-Leibner divergence. And it's a bit, okay, this one's really the last math slide, I promise. It falls a bit from the sky now. But the point about Kohlberg-Leibner divergence is that this here, this particular term here, can be interpreted as also a maximum likelihood estimator. But it's the maximum likelihood estimator that those distributions are actually identical, or yield identical samples. Not going too much into detail here, the point about the Kohlberg-Leibniz divergence is that it actually, that you can transform it, and you can make two parts of it. One is called Shannon entropy. It's relevant for information theory, but it's not dependent on our system. So we can just drop it out for our optimization problem. And the second one is called Kress entropy. And the point I was trying to make with the slide is that given the assumption of Kohlberg-Leibniz divergence, you can actually find a reason why you should use cross-entropy, because minimizing the cross-entropy term here is equivalent to minimizing Koeberg-Leitner divergence, which makes the distributions as equal as possible. Okay. Practical example. Computing cross-entropy is actually very easy. All you do is you take the PC value here for every category. First example is 1.0 times the logarithm of the probability of observing it from your predicted distribution, that's 0.3, that's this term here, and then we're going to make the same one for the second category and for the third one, and so on and so forth. So that's all it does. And what does it tell us? Well, you could say that cross-entropy in combination with multinodal distributions is heavily used for loss function as a loss function for classification. However, cross entropy is actually not limited to this okay so um you can the point is about that cross entropy can be used to fit arbitrary probability distributions um also not limited to discrete distributions as multinully but you can also use it for continuous cases and also interesting is that if you apply a Gaussian distribution to your cross entropy what you will receive in the end is actually called it's just maximum likelihood sorry, main squared error so that's a bit of a point so actually cross entropy is actually the meta loss function and you can kind of use it if you have for example other loss functions or other noises then you could use this as a starting point to generate your own loss functions which would probably find better results, well hopefully at least okay so that leaves me with a summary so what why do we use main skirt error well because minimizing mse is equivalent to maximizing the likelihood of observing our data from our function approximator while we adjust our parameters and assume gaussian noise and gaussian noise is usually a reasonable default choice also why do we apply stochastic gradient descent because applying stochastic gradient descent removes the requirement for symbolic differentiation and this allows us a relatively free choice of our function approximator. Why do we use cross entropy? Well, because minimizing cross entropy is equivalent to maximizing the likelihood that two probability distributions are identical. And what else should you know? Well, you should know that some applications may benefit from assuming alternative error distributions and you could use cross entropy as a starting point. then and you should also know that some functions may allow more robust methods than stochastic gradient descents to minimize the loss especially if you can assume convex behavior that's all from me thanks very much one last thing before we go to the slides if you ever wonder well there's so much information on there where resources from this talk is actually inspired by by me reading this book by ian goodfellow where all main findings are based on this publication except especially from these sections pointed out here and i also use some medium article towards data science.com which inspired the gaussian example and now i'm happy for remarks and questions so go ahead thank you very much i can really recommend that book as well um can you take us through the intuition of the two terms of the callback labla of the the shannon and cross entropy like how those um line up and also how would you would you use the cross entropy if we were to say to use a ga a weibull distribution of noise on the example you had before how would we go about that with the cross entropy yeah Well, common intuition about Like Shannon entropy is often called in information theory As the maximum amount of information or bits you use To compress an information So if something is very probable Has some discrete probabilities of happening Say an event, yeah Say you roll a dice And you want to know what is the most effective way of compressing this information in a bit way or any other computational method then this gives you an a lower bound for it and actually cross entropy is related to the usual shannon entropy by telling you how what is the lower bound if you try to compress this information but with a wrong assumption of the distribution so how much information will you get Is it very clear for you? No. Just say that one more time. So it's the difference between the optimal... This one here is lower bound for the optimal compression rate. So if you have probabilities and observe events, and you try to find the minimum compressed version of how you can encode this in bits and bytes, for example, then this gives you a lower bound here. And this one here, but it assumes a certain probability. So, it will make a higher or larger bit sequence for events that occur less often. So, that's why it, in the end, will be highly compressed. And this one here is basically the same, apart from that, that it gives you the information about how much, how is the lower bound if you assume a wrong probability distribution. So, that's pretty, well, at least one interpretation point. hi thanks for the talk and so as you said using a custom loss function makes sense if your noise is not gaussian distributed so how do you know in practice that your noise is not gaussian distributed so it seems like first you have to fit the model and then in order to quantify the noise and how it deviates from the novel so from the from the model so would you say just use assume a Gaussian noise and then look at the noise and then look at the distribution of the noise? Well, that could probably work. I think it's actually a thing of domain knowledge. I think that's probably the best starting point. If you have some theoretic knowledge about what's happening under the hood, then this might really help. If you model a physical process, for example, you might know this. Or another example could be you know that you have a sensor, but there's a sense of software and it just cuts away unlogical values so you know okay you don't have a clear gaussian anymore because shifted a bit and you try to find another distribution which fits it better i wanted to ask you about cross entropy how would it work to fit the model if the targets are not one hot encoded like classes one zero zero but for example the dog can be point eight and uh cat 0.1 and fish also 0.1 like this one yeah but uh not clearly 1 0 0 but uh you know it's the same concept so if you have like here a 0.8 and 0 0.1 and 0.1 here then you would just have here a 0.8 and here a 0.1 and here a 0.1 and in fact that's what people actually doing because um if you compute this function here and you have a softmax function in uh before as your output function or neural network then you will have an infinite you have to have an infinite input to receive a one so actually there's some um some approach where people try to arbitrarily reduce those um those values here in order to uh prevent exploring uh exploding weights but it still works nevertheless so even even though if you're not very sure about this and you can also use a sample where you're not sure about how does if that's really a dog, if you say it could be a dog but could be a cat, then you could also assign 50-50 probability and let the still use the sample Okay, let's do that So yeah, time is up Let's thank David again and if you have more questions Thank you.