Are you sure about that?! Uncertainty Quantification in AI

Speaker 1 [00:02]

So welcome everybody to my talk about uncertainty quantification in AI. Before I start, a few words about myself. I'm a principal data scientist at Innovex. Innovex is an IT project house, medium-sized, like we are 350 employees. And being a mathematician, my main activities are mathematical modeling, so somehow translating practical use cases into the language of mathematics to then eventually solve them. I'm also a big fan of putting things really to production, so not just stopping at some proof of concept, but really showing the value of machine learning of data science by deploying it and gaining then more revenue or some things. Also other special interests I have, I really like everything about recommender system, which is a hot topic nowadays. Also uncertainty quantification, of course, this is why I'm here. also causality because i think those two points uncertainty quantification causality will be and become much more important in the future i'm also a fan of the python data stack of course that's why i'm here and i'm a maintainer of a little project of my own pi scaffold so if you ever want to like set up your own project in a standardized way to to package your project to ship it then please have a look at pi scaffold so the work i'm going to present today is actually not my work but it's the work of one of my students so he mainly did all of it and he yeah i was supervising him in a master thesis now he's an esteemed colleague of mine and he's also written a really cool blog post about this and if you want to get even more more details you can even download his master thesis. So he's a mathematician, a little bit shy. This is why he's not here. I am. I just like to go to conferences. So this is the ground we're going to cover today. First, we're going to start with the motivation. So why should you actually care about uncertainty quantification? What is it all about and why it's important? Then I'm going to present different methods. We're going to start with something like really mathy like Gaussian processes. If you have been to some of the other talks yesterday, you heard already a lot about Gaussian processes. So we're going to use this as a baseline, then compare to four other methods which are more based on deep learning approaches. We're going to evaluate those five methods together in some experiments to show the pros and cons of each of those methods and some quantitative qualitative rule of thumb when to apply which and of course in the end some short wrap-up and conclusion so let's start with the motivation so you surely all know those nice deep learning classificational networks where you put in some some image and then you end up with the classification if it's a cat a dog a hot dog or whatever and you might wonder actually so yeah the guy is talking about uncertainty quantification as if it's something new but actually already here in those networks we have some probabilities right we have the soft max layers we get us probabilities and in case of this cat it's 90 so it seems to work if we put in another picture maybe of this dog the network is a little bit less certain about it but what actually happens if we put something completely different in there some kind of image we have never trained on some compute some complete new class like a bird so in this case you might assume that the network will tell you something like 50 percent a dog 50 percent a cat in a way telling you i have never seen anything like this before but actually what might happen that you end up having 70 percent cat or 80 percent a dog so it's not it's not guaranteed what kind of what kind of results you're gonna get so you could get anything because there was never an image only like similar to this in your training set and the network has no way of telling you i'm completely uncertain about the outcome so please don't rely on my on my outcome on my prediction and i mean it's all yeah fun and games if we are talking only about classification of cats and dogs but you might think that if you do autonomous driving and if you have a classification like this for traffic signs then it becomes really important that if you put in some pictures that are not a traffic sign that somehow your um your classification model has a way of telling you so i'm not i'm not certain what it is and just doesn't give you just some random um answer and so why is this so why doesn't it work so um beginning of the year i heard a really cool talk about the intuition about this at the triple ai um the paper is called learning and the unknown And in this case, there was given an example of four different classes, like those four colors. And what does a deep neural network do? I mean, a deep neural network, we have all learned that it helps you do the feature engineering. It finds some internal latent space. But in the end, a deep neural network does something really simple. In this internal representation that is shown here, it still finds those decision boundaries. Like here, the circles would be the training examples, and we have these decision boundaries. And within these decision boundaries, it will give you a high certainty about an instance being in this class if it's far away from this decision boundary. So this is all fine. If you end up with the instance you want to predict somewhere around here, but in case like with our bird, what happened, we somewhere ended up completely far away. So we just zoomed out in the right picture and we ended up somewhere in this open space. And due to the fact how a neural network is constructed in case of a classification, those decision boundaries go up until infinity. So if you land somewhere in this open space with some new instance, you might get a really high certainty that you are in the green glass, for instance, but actually you are not. You're just somewhere completely different. And so this is for classification problem, but still you have those probabilities in case of this softmax. But it's actually even worse if we look at another problem, if we look at regression problems. For instance, here we made a simple 1D function. We generated some observations. Those are the crosses. And we let neural network learn the interpolation of these data points. And you see that the fit, it looks quite well. right so it's um we chose a mean squared error to to minimize so we will end up having a kind of mean but already looking at this you see that we only for each point we get only a single function value right so the deep learning network it just has one output so we have no notion of the fact that somehow there seems to be noise in our data, right? So here, so two or three or four points, which are really close in its point, they are quite different in the function value. So the network is not showing us this uncertainty. It's just giving us a single point. And we're going to hear in the next presentation, for instance, that if you want to do some optimization with the results, you're going to need the whole distribution and not only a single data point. So this is the interpolation case. So we had data from 0 to 10. But now what happens if we look beyond this interval? So again, like in the classification problem, what happens if we look beyond our data points? So what the network does is it just gives us some results. And due to the fact that we chose the ReLU activation function, You know, the ReLU is linear for positive numbers, so this network will come up with any kind of linear function beyond the 0.10 and below zero. If we compare this to the actual true function, then, of course, it's completely different. I mean, how would it know otherwise? How can it know? But, again, the main point is the network did not tell us that it's completely unsure. and in the interval it worked quite well but outside you just end up with random number and this is something you don't want to have in a production system for instance because if this was multi-dimensional you could not as clearly see that you are now in a region where you had no data points, right? So what we rather would like to have is a network that gives us something like this like confidence intervals Like those shaded areas are 0.5 sigma standard deviations in each direction, and that we could get something like this, because then we would see, okay, here, between 0 to 10, this is the mean, but we have some random noise in the data, and these are the confidences we have about our prediction. and if we go outside this region we see okay we are getting more and more uncertain as we move away from our trainings data so now implicitly we've already talked about two kinds of uncertainty and i want to define those so first of all this intrinsic noise is called aleatoric noise so this is the noise due to the fact how you might measure the data maybe you have some measurement errors maybe you also um just don't have each covariate for instance so let's assume you're shooting an arrow and you have all the data about your angle the thrust how far your target is away and so on you have all the data that you could physically like um like use newton or whatever to to calculate where the arrow is going to hit the target but if you do the same if you shoot the same way twice you will still have a little noise because due to the let's say wind or whatever different pressure in the air and if this is if this is something you cannot measure then you will have some kind of intrinsic noise and this is the aleatoric noise and the most important thing to remember about the aleatoric noise, if we collect more of the same data, then we will not be able to decrease aleatoric noise. In contrast to that, we have the so-called epistemic noise. So this is noise due to the fact that we lack certain knowledge, that we lack data, that we lack knowledge about the underlying process. And this is something we can, of course, improve if we gather more data and in some uses cases you could imagine it makes even sense to be able to to split your uncertainty into an aleatoric part into an epistemic part because if the aleatoric if you only have aleatoric noise and you know the the noise is still too big for your use case the uncertainty is too high then you might try and look again maybe you need more covariates you're missing some features whatever or let's say the other way around you have a lot of epistemic noise then of course you would just try and gather more data at these points so now let's uh let's talk about uh the methods so what can we use to actually accomplish this we're going to start with the baseline the gaussian processes they're like really mathematical and then as we go on we're going to relax on different mathematical assumptions we're going to ignore distributions and so on as we go away and the methods itself become i would say more and more heuristic especially with the monte carlo drops dropouts but let's first start off with something more mathematical with the gaussian processes so i'm going to give you only like a really small like intuition what a gaussian process is maybe or raise your hand who knows what a gaussian process is already okay so quite a lot so to um to give a repetition so um a gaussian process is a kind of distribution and if you would normally think of some some random variable if you pull one sample from a random variable you end up having a number or vector or whatever but if you ask for a sample from a Gaussian of a Gaussian process you end up having a whole function so this is the is the difference and a function so what's really fascinating is I mean a function has a kind of infinite number of points right so you have a lot of points and a lot of function values so how does the Gaussian process accomplish this that it gives you back a whole function so actually you're never practically dealing with infinite number of points right so for a function for us It's enough to have a finite number of points, a finite number of function values. And there comes now the relation of a Gaussian process to a normal distribution in. Because if you ask the Gaussian process for a finite number of points, then it will give you back a normal distribution. And then, yeah, we are back at something we know. So the normal distribution in a multidimensional case has a mean vector and a covariance matrix. And since we could ask the Gaussian process for any kind of points, we have to kind of generate those vectors and this matrix for all kinds of points. So this is why we have functions here. So I see it as a kind of generalization of a normal distribution. And so, as I said, we have instead of a mean vector, we have a mean function and covariance function, which gives us then some covariance matrix given some points. And the things we kind of put in there in the discussion process, if we set something up, like the prior information is, for instance, if we are looking for functions that have a certain trend or if we are looking for functions that are really smooth. So if we want to express the notion that if two points, when they're close together, the function values should really highly likely be also close together, then this is something we can encode into the covariance function. But a picture says more than a thousand words. So it's actually clear if we look at some examples, let's take this exponential covariance function and you can see that this would generate really smooth function because if this is really close, then the covariance between two points is really, really high. and if we now take just the zero function for the mean and this is how a Gaussian process would look like the distribution so again we have for the shadings we have the visualization of the distribution and those colored lines are just random samples from this distribution and the dashed line is the mean If we somehow know that we are rather looking for functions with a trend, then we could, for instance, choose this mean function and have a distribution that looks like this. So far, I've only talked about how to come up with Gaussian process, how to construct it, how to put a priori knowledge into it. So far, we have not seen any data. So you might wonder how does now inference work. So let's assume I have some data points. What happens now? So again, this is like if we have no data at all. so it's the prior let's assume we get three different data points here there and there and now we can like due to the fact that we are dealing internally with a lot of normal distributions and they have a lot of nice property so we can just condition on those points and calculate the posterior and then our distribution looks like this so the resulting posterior. So in this case, just for illustration, I chose a perfect interpolation. So we see that now the mean function just interpolates those given data points. We also see, due to the fact of the covariance function, that if we go in the middle between two points, then the uncertainty is quite high and as we go closer to one of the points the uncertainty becomes small and again the colored lines are just now random some samples this also works of course with noisy observations i mean this is something how we do how we would do it in in uh in practice and there we see then that there's still uncertainty even if we have a point here and there and for instance here we had like put four points of yeah four data points at eight so you see that the uncertainty is smaller than for instance here um so just to show you a little formula how this looks like so So the axis would be your data, the data we are having, and x star would be the data points where we want to evaluate the function. So this is kind of where we want to do our predictions. And the only important thing to actually notice here are those two terms, because they are really computationally intensive. So if you remember maybe your computer science classes or whatever, if you have to invert a matrix, this has cubic complexity. and if this covariance matrix is as big as the number of data points you have, then it's completely unfeasible if you're doing anything related to big data. So there are different ways how to get around it, but this is seen as the major drawback of Gaussian processes because for small data problems, they actually work really, really well. And if I kind of get your interest on this, If you want to read more about Gaussian processes, then there's really cool tutorials of Christopher Fonsbeck. He has some really nice talks about it and how you can do this with PyMC, which is a really cool library, by the way. So this Gaussian process, as I said, we're going to use this later as a kind of baseline, a mathematical baseline to compare the other methods to. And now to the first deep learning-based approach, how to get uncertainty quantification. you all might know the dropout technique right dropout is a way that is used in most modern machine deep learning models to regularize your um your network so it works by randomly activating and deactivating certain nodes during the training time and this like intuitively like spreads the information more over the whole network instead of being just local and um yeah helps to generalize during training time and the idea of this paper was of 2016 icml was okay what if we don't just use this during training time what if we also use this during prediction time meaning what happens if we um if we do a prediction with a new value here we also randomly activate and deactivate um the internal nodes and run this a hundred times and then we'll get a hundred different outputs and of these hundred different outputs we can do just uh things like calculating a mean calculating a standard deviation to give us an uncertainty right and the idea is if the network is quite sure then having dropout during prediction should give us almost the same values while if it's really unsure then it should give us a wide spread of values so this is the basic idea already of monte carlo dropout so if you train this you You have to be a little bit careful if you also do weight decay, because this probability of making a dropout will play a role in this formula. But all the details are also in the master thesis. So this was like the first approach. Another approach is how you can achieve uncertainty quantification is called deep ensembles. It's from a NIPS paper of 2017. the basic idea is what if we don't take a neural network that has only a single output like normally but what if we would try to generate a whole distribution and how do you get a distribution you fit the parameters of a distribution like in a case of a normal distribution the mean and the variance and then of course in practice you're going to ask yourself okay i got a single value as a target point how would i now kind of fit the standard deviation i mean this is something i don't have in my data but it turns out if you just look at the negative log likelihood of your normal distribution you can just use a custom loss function so you would not just use a mean squared error but you can use this to actually fit it and what we see here we got some mean squared error here and we kind of if the network says that standard deviation is the variance is really high then this becomes low but on the other hand this term then becomes high so you got this both terms kind of fighting each other and this is how it eventually finds good parameters for normal distribution and fits it so this already sounds good like okay we have a way to fit a distribution so we get some uncertainty but it turns out that the variance is quite high if you do it like this first of all you have to be careful how you train so you cannot regularize with dropout you can do something like adversarial training like finding new data points that are really close to your original data points doing pseudo data points and so on and but still you will get a high variance in dependence of your initial initialization of your weights so this is where the the ensemble part comes in so what the paper suggested is that you do this now like really yeah with many many networks let's say 5 to 10 or 20 networks you train them and you get a lot of different parameters for your normal distribution and then you just average them again so this is the basic idea this is where the ensemble comes in of course then you decrease your your variance again in your estimation of this and as you already see if i got to to train many networks then this becomes computationally really intense and this is why simon so the master student came up with the cool idea so what if we now combine the two methods the monte carlo and the deep ensemble method and just call it dropout ensembles basically um taking a single network of the deep ensembles but instead of having that multiple times what if we just think of each instance in this ensemble is just another dropout configuration and by this we will also get different means and different variances and we can do the calculation of those central means and central standard deviations again so this was kind of where what he came up in his math thesis as an additional beneficial suggestion so as the really last method i'm going to show a really easy one so quantile regression which can also be done with the help of neural networks and it's a really simple method but yeah works surprisingly well so again to explain what this method does what is again a cumulative distribution function i've given here the definition so you're kind of given some point you're kind of looking for the probability tau that they are showing up points in your random distribution that are below or equal a point and the quantile is actually just the inverse function so given some some tau some probability you try to find that point y so that this equation is fulfilled and you might wonder okay so how if i given that um if i want to find the quantile function and i mean if you have the quantile of course you could perfectly um measure and quantize your uncertainty because you know everything about distribution if you choose different taus and you might wonder okay how can i use now some just neural network to actually fit these values and this is quite easy as it turns out so you just take a custom loss function and what's special about the loss function that it makes a difference between kind of a right error and the left error So if my estimation of the quantile is below the actual value or above the actual value. And then another distinguishing point is that we don't have like normal some squared error or something nonlinear. So we have at least like something piecewise linear here. We just take the absolute value and then we do some weightings between those left and right arrows. So for me, actually, to be honest, when I first saw it, it was kind of, OK, this really works. And I'm going to show you in some simple example that it's actually really easy. So if we take, for instance, the median, so 0.5, then tau would be 0.5 here and here. So we can just ignore it for a moment. So how this would look like now in 1D, let's assume we have four points at 0.1, 0.2, 0.3. 0.5 and 0.8, and we assume that our median is here, wrongly, then we would have those four errors, yeah, so 2.01, 0.02, and so on, and since we call this now kind of right-hand side errors because the error is in this one, the example on the right side, and the error is in total 1.6 so the some back provocation would maybe tell us okay we got to move on a little bit higher let's say we move on to 0.1 in this case we decrease the error four times we have four data points by our step width which was 0.1 and we end up having 1.2 still not right so we go on one more point and now we see that what happens is that from here to there we decreased by 0.3 but increased now we have now the the left left hand side error and we are basically also um done here um if we since we have now a even number of points so any so according to our minimization function any point between those two will do for a median so the interesting thing and now the actual intuition comes in if we go on one step further and you see that between those two slides what happens as we increase the error on the right hand side by decrease by 0.1 we directly increase it so this is this piecewise linear of the of the absolute value and so we are just trying to find points to find a point where there is the same amount of data points on the left-hand side as there is on the right-hand side and now comes in the weighting so if we would now assume the 75 percentile then we can just accomplish this by weighting the right-hand side error more than the left hand side error so in our case it would be three times and that means to get this like tipping point as we've had before we would need to jump somewhere in this region because now as this arrow weights three times then it's equal to this arrow and this is the whole intuition why this quantile regression actually works now to the experiment to compare those methods we took a function from a PhD thesis actually that someone suggested to evaluate the function, the weakly function that you've seen before. And we generated the data points with 0.5 standard deviation and generated different number of points and let it run. Here again is our data set with the, yeah, this is five sigma in total confidence interval and the mean. And for all our methods, which are based on deep learning methods, we chose two hidden layers with 20 relo neurons, five networks only for the deep ensembles because it already took quite a while to train. for all the different experiments and their variations. 100 iterations to make in this dropout, the different predictions for each point, and atomizer. And of course, we chose the best hyperparameter for each of the methods to be fair. For the Gaussian processes, we used the squared exponential covariance function you've seen before. and also there was done some hyperparameter optimization to get the right aleatoric noise. So if you want to compare something, of course, you might think, okay, what kind of error functions, what metrics you want to look at? And first of all, we have the mean squared error for the mean and the standard deviation. Then we looked at the mean negative log likelihood. So, this is the function that the deep ensemble method is directly trying to minimize and the mean Kullberg-Leibler divergence because this is also often used. So, what were the results? So, especially for the matrix of the mean squared error, so how good our mean actually is and the standard deviation as well as the mean Kullberg-Leibler divergence, we see, what was cool to see being a mathematician of course that the baseline, the Gaussian process was much better than the other ones but if you go for a deep learning method then the deep ensemble which was also computationally quite intense turned out to be the best for the interpolation experiments. So you might wonder how does it now look did we achieve our goal i mean we wanted to have interpolation and extrapolation so just by looking with the eye so interpolation looks good i mean this were the numbers shown before but extrapolation the monte carlo drop out um at least what we wanted is as we move away the uncertainty increases this is also with the deep ensembles and um with the with the dropout ensembles and the quantile regression but somehow in our example they really all had problems in this like narrow arm of the function because here they definitely underestimate the increase in uncertainty so at this uh at these regions and uh what's what you can also see is all the dropout methods you have this this kind of uh chick um mean so this is due to this um that you have the the drop out and also the the um uncertainty the confidence intervals they look kind of strange in those kinds of methods so for the deep ensembles like only qualitatively looking it looks quite well and the quantile regression if you can compare this to the gaussian process so this is actually what i would have liked to see also for the um for the deep learning based models because here you see directly okay we are moving away from the 10 and the gaussian process by its definition goes back to its prior so you end up being in a complete like uninformed state you might say okay this is kind of unfair because this is some a priori knowledge that we've put in there but in many use cases you have this knowledge and then it's nice to see that yeah that this does exactly what you wanted to do talking a little bit about like how efficient this the whole thing is how data efficient i mean as you we have here in the y and the x x is the number of data points and one of the measures the karl divergence and the gaussian process needs only little data to be quite good actually and then kind of like levels out the deep learning based models like quantile regression deep ensembles the more data you give the better it gets like typical for deep learning methods what we found quite interesting is um that um all the dropout based methods they kind of improve up to a certain point but then hit a barrier and go on so we assume this is due to the fact that there is this dropout i mean it gets also some noise into the prediction that might be unwanted and then one would have maybe to play a little bit more with the fact that maybe in in lower levels you want a different dropout rate than in higher levels and of course then it becomes really problematic to optimize but yeah this is something we found okay it's not behaving so nicely there how about heteroscedastic noise so what happens if the noise changes throughout the function this is also something you might have then Gaussian processes are quite bad because I mean they the one of the assumption is that it's homoscedastic so if you if you um yeah if you don't adhere to this um to this assumption then it becomes quite bad in this case deep ensemble could still um handle the situation quite well how about a non-gaussian noise here we chose uh exponential noise um there was not that much of a difference between the different methods so i would actually would have expected that the gaussian process is much worse in this case because it also i mean it has a normal distribution as one of its assumption but in this example it worked quite well and another thing i just want to mention so why did i give all this uncertainty split definitions about aleatoric and epistemic noise and the cool thing about the ensemble methods is that you can now do a split like this um you can um like split up the the the this averaging of the um of the different um parameters of your normal distribution into an aleatoric part and into an epistemic part and this gives you than confidence interval of your aleatoric and your epistemic noise. And this is really cool because, for instance, we see here that during the interpolation, the epistemic noise is actually kind of zero. The aleatoric noise is blue. So we might be happy at that region in a practical use case, for instance. But here it's just the other way around. Here the epistemic grows. And this could tell us in some use case, okay please get more data for this region to improve our uncertainty so to sum the whole thing up um if we just maybe look at the columns so my kind of takeaway after this master thesis was that if i have a small or medium-sized data set i would definitely and i can make the assumption that i got some homoscedastic noise so if i can assume the normal distribution and that it's not not changing then i would definitely go for gaussian processes because they're only really bad if you have a lot of data and you also can get this nice uncertainty split that i just showed before um if it really has to be big data and if you have a lot of compute power that you can parallelize then deep ensembles they work also really pretty well but as i said their computation is really intense the quantile regression is a really really simple method so if you have like no time and you don't have as much compute resources and you don't care about the uncertainty split then you could go for this method it's really easy to implement and the dropout methods they are mathematically really interesting so i yeah uh suggest you to to read the papers because they even have a nice link to the uh that this drop out um uh this monte carlo drop out so this paper that they have a nice link that you could even think of this as a deep uh gaussian process so there's an interesting paper but practically we found the results not really convincing and the own methods that yeah like we came up during the master thesis is also quite yeah it's also quite quite nice it has done certainty split but in some cases yeah it's kind of comparable to the monte carlo so i want to conclude just the the last two slides so our conclusion is that you can do uncertainty quantification also with the learning based method especially if you're looking for the aleatoric uncertainty they are not really capable by estimating epistemic uncertainty we saw this in a simple 1d example and i mean in practice you're going to have like multi-dimensional spaces and if it doesn't work in 1d it will not work in in in higher dimensions so be careful with this gaussian process on the other hand are really cool and i think maybe in some cases you should rather use them instead of using just some some deep neural network and of course a combined solution would be really great because we're going to need uncertainty quantification three things we haven't really looked into but want to look into are beige neural networks so the case when each weight in your neural networks are distributions on their own then there is sparse Gaussian process approximation this is kind of a trick how you can do Gaussian processes on big data so instead of like using each data point you come up with a good set of supporting points and this kind of relieves you from the fact of inverting a huge covariance matrix and then there's also Gaussian process on top of neural networks this is something where I also need to read up more um so this is something we would like to um look into so um i hope uh you learned a little bit um about uncertainty quantification in ai thanks for your um attention and um yeah i'm happy to hear your questions thank you for this nice talk so probably we have time just for one small questions and you can continue discussion probably after yeah or so please questions hi thanks for talk it might be confidential but for what do you use it in practice or do you use it for anything in practice some some kind of uncertainty estimation for your yeah predictions yeah you in in cases where you actually um yeah in cases where you do for instance a demand forecast and then you kind of let's say you actually it will be quite similar to the next talk but uh to give a little um like like like pre-spoiler um if you do a demand and then you actually don't care just about the prediction what the demand will be tomorrow right if you are the person like needing to decide how many apples or let's say sushi is needed the next day in a store right and someone would give you just a regression would say well confidence interval then you might need like 50 to 70 apples and this is the distribution so what would you do with this now so you would actually need to run some optimization on this so okay so to say the other way around if a normal neural network would just tell you with a mean you need 70 apples tomorrow this is not you would not order 70 apples because maybe it is like this that if you have too many apples then you have to throw them away but this costs you less than if someone buys an apple so you have an asymmetric like cost function and if someone gives you only a single point you cannot really apply this cost function that it's for you better to actually have a little bit more apples than uh than less apples and if you want to optimize this then you can you need a distribution and this is then the case when our customers ask us for methods that give them whole distributions um and this is basically so a lot of it's mostly about demand forecasting for um yeah for retailers okay so thank you let's thank the speaker again