Bias Meets Bayes: A Bayesian Perspective on Improving Model Fairness

Speaker 1 [00:07]

Thank you so much and welcome everyone. Good to see so many faces. Indeed, my name is Vince. I'm a staff data scientist at Blue Yonder. And today we'll talk about a pretty curious topic because we're going to meet two very interesting concepts. One is fairness, something that has been a hot topic for quite a while and still remains a major point of debate and constant struggle in machine learning. And another one is the powerful statistical framework that is Bayesian statistics. And this talk will be perfect for anybody who ever trained the machine learning model and wondered, is it fair? Does it make sense? Is the decisions that it's making actually reasonable? Is it discriminating in any way? And those are questions that both you probably asked in the past, and if you didn't, you probably should have. So good to see you here for this reason. And Bayesian statistics is something I have quite a history with, that statistics is part of my education, and even though I've been data science and data science consulting for more than five years now, that's something I've always enjoyed both applying in practice to teaching other people how to actually make use of it. And it's not the biggest field perhaps that you would encounter, but I think it's one of those misses where once you know how to use it, this will equip you with really great tools. Let's get started, and we'll try to cover quite a few important things in 30 minutes. It's almost an impossible task, right, especially if you haven't seen much of Bayesian statistics before, yet I'll try to balance between making sure everyone understands what this is about, while giving enough technical details, while also not drowning too much in some things that are perhaps optional. All right, well, let's imagine a totally hypothetical but a pretty curious situation. So let's say we have a society with two groups, orange and purple. Orange is the majority, purple is the minority. And there's some major problem that we're trying to address with machine learning. For instance, we're trying to flag risk for individuals from both groups. And let's say we so happen to observe that the model flags purple individuals much more often than the ones from the orange group. Could happen for several reasons. Maybe that's exactly what the data suggests, maybe there's something fishy going on, but just the fact that we have some kind of a situation with multiple groups in a situation of high stakes, and flagging risk is something both very relevant and also something that can lead to major consequences. Think about policing, healthcare, lending, essentially there are many applications that follow this form. At the same time, this is not essential because what I'm going to talk about would practically apply to a variety of models, so I use this particular one as just a striking example where this will be useful. The most curious case would be if, even though we actually observe this model discriminating in practice, what if actually in reality, if we would have access to the code behind the world, right, we would know that actually there is no difference. Maybe individuals are actually fairly the same from the two groups, so perhaps it's some issue in the data, or the feedback loop, the way we were going through everything. So let's get to the bottom of it and see what could have gone wrong and what we could have done better. And for instance, part of the reason why we might observe a result like that is essentially very often, even though in reality there is a lot of similarity between such individuals, The way the data is collected, the way perhaps historical prejudice unravel into the training data, it could so happen that the model essentially starts to pick up some patterns that are rather problematic, and we might not actually have a good idea to either understand that these patterns are there, whether they were picked up or not, and what we could have done better. But in general, we would agree that we would want to be conscious about it and cautious about it and in general deliver a model that is as fair as possible to the best of our ability. And this is what we will try to do. But first let's agree on what is fairness, because it's also a major point of debate, and of course you could argue that fairness is something fairly subjective, but I think there's a core in this concept that we could agree on here, and this is what I highlight here at the beginning. Basically we would call a given machine learning model or model in general fair if it behaves similarly for various groups in the problem, or more specifically, the errors that it makes do not discriminate between groups. So we are as likely to make mistakes for one group than for the other. Because if we have a model that essentially, let's say, has a lot more false alarms for one group than the other, that's a problematic model. Together with it is the notion of bias, something people also tend to disagree on or misunderstand. In this case, I also give the following definition, essentially, that is consistent over or under prediction in certain groups, or in case of training data, that is essentially particular patterns that invalidate the truth that the model might be able to pick up on. And that's basically what we'll try to evaluate. Are various models that we train fair? Is there bias in the training data? How we can measure whether that's been influencing the model, and what can we do in the end to make things better? And to help us with that, we'll use several metrics. Of course, there's a whole family of them, but we'll focus on the most known ones and perhaps the most intuitive, but also quite helpful ones. That is, things such as false positive rate gap. That's something I just talked about. So if we have two groups, we want to see what is the false positive rate for each of the two groups. And if we see a big gap, so essentially, again, more incorrect false positives for one group than the other, then it's a signal that something unfair is going on in the model. The same can be said for true positive rate, even though here depends a little bit on application. So first tip, I would encourage you to think about exactly what is at stake. Are you caring more about one type or the other type of error and focus on that one respectively. Well, for the sake of this application, we'll focus primarily on false positives today. But again, be aware that this can be situation specific. And another metric that will help us is essentially how well calibrated the model is. So we'll try to see essentially for each group, let's say there's a certain probability being predicted on average for the negative outcome that we're trying to deal with, is that probability actually corresponding to what we are seeing in that group in reality? And there is a big mismatch, so miscalibration, that is again something that will indicate that there is something unfair going on in the in the model all right well before we actually get to see any models let's talk about why Bayesian framework is actually applicable here how it can be useful and for anyone who hasn't seen it much or perhaps have been a while this was a useful talk on it yesterday so maybe some of you have been there so this would have helped but don't worry if you haven't essentially there are some key ideas here on a high level that But I would encourage you to catch here if you want to go more in depth. So I'm not going to talk about how exactly to apply bias rule to update your priors and things like that. But on a conceptual level, it's important to understand that essentially the whole paradigm here is that we very explicitly model what is going on in the problem at hand. We do this by setting priors or prior beliefs for the full set of parameters that we care about. And then, most importantly, we use data as information to update those prior beliefs. And that leads us to what we call a posterior. So that's essentially updated beliefs based on the evidence that we've seen. So this is quite crucial, right, because this already quite closely ties to the concept of fairness, right? So we can essentially think of what prior beliefs could be fair beliefs, and then we can use the data to either support that or drift it a bit away from that belief. An important feature here that is not true for many other approaches that is we are very explicit with what we're doing so we can justify exactly by saying this is the prior, this is the real-world facts and evidence to support why we're doing it this way and perhaps we're wrong but the data will either proof or invalidate this, essentially. Alright, so again, the way we're going to tackle fairness problem with the Bayesian approach is by setting fair priors. For instance, we could say that we believe that at baseline all individuals from all groups are the same, there is no group coefficients or individual differences. If we're wrong, we're going to learn it through the data, right? So that's the key principle here. Important remark, so I'm not going to go a lot in detail into other approaches in other fields in the topic of fairness, although super interesting that would explode the conversation today quite a bit. So again we're talking mostly about problematic situation and the Bayesian framework applied to address fairness, but it doesn't mean that this is the only way to tackle this problem. That said, I will make some arguments for why this is a particularly useful and powerful framework and it can be more effective in certain things than its competitors. All right, well finally some math. So essentially to make a point here today I have run a certain simulation, something that you can easily recreate and of course there's a variety of problems that you can simulate and the results would depend a little bit on a particular problem. Here I've picked a particularly curious situation that mimics a little bit what we've outlined at the beginning. And at the same time, it provides a good benchmark for how we can compare different models against each other. So let me summarize a little bit what is going on, and also on a conceptual level, in case the math is not accessible for someone. But in this case, it should be fairly simple. So again, we have two groups. orange, majority, let's say 90% of the population, and purple, minority, 10%. In terms of real numbers, and importantly, this is the training data samples, so let's say this is what we end up with before we train the model, that is 450 for majority and 50 for minority. Again, in this case, I choose relatively small samples on purpose to illustrate specifically how certain biases would really propagate this type of framework, and this is also where the Bayesian approach would really shine. That said everything I say still holds on larger samples. Just the problem becomes more complex So what is important here, right? We again talking about flagging risk so the true data generating process that creates this risky situations in my simulation is essentially simply based on a logistic formula, right so we get Let's say the real process is there some parameter X that we simply draw from normal distribution and we apply a logistic formula to it to convert it into probabilities and that essentially provides us with the data points that we can further expand on. What is very important here you see that this is actually not dependent on the group or any other features right so that's the key sort of underlying assumption something we wouldn't per se know in real life but in this case for the sake of comparison we'll be able to tell this is the reality and let's see how close we can get to it now on top of this reality we also have two other things so one one thing naturally is the two groups so essentially individuals belonging to one or the other group which again in terms of outcome doesn't per se matter but we'll see in a second that matters in terms of introduction of bias so that is what will make the problem interesting and there's another feature that I call here proxy feature Z essentially that is something that might appear as a useful feature if you don't know what is going on but then reality that is just some mix of X and belonging to the group, in this case purple, with some noise, right? So this will naturally correlate with the outcome, and you might think that this is a useful feature, while in reality you see this doesn't actually explicitly participate in the data generating process. This will be important because you will see this will be both a feature that very likely would end up being included in your model, again, because it's correlated with the outcome, and it could also bring some information about the group, even if you didn't include the group explicitly in your model. And that is, again, a major problem that often happens. So think about things as anything related to one's job or healthcare access or perhaps geographical location. Essentially, any information that isn't per se sensitive but might be correlated to other sensitive outcomes as well as outcomes that we care about. All right. So what we end up with once we run the simulation is essentially two sets, as I said, right? And you see that the real probabilities are the same, so 36% in my example, but the probabilities that we see across the training data, or in this case the samples that we'll use to actually train the models, are different. So where does the difference come from? And that is important, probably one of the most important things here to capture. This is where we actually introduce the bias. There are several ways you can think about it, But essentially, let's say while for the majority there is no systematic pattern in how we select the outcomes, we just randomly draw them from population, for the minority there is some process that essentially biases data collection towards collecting more negative outcomes than positive outcomes. There are several reasons why this can happen. Perhaps there was already some type of prejudice existing that has affected data collection and then propagated further on over the years. What also happens is types of feedback loops. So you have a wrong belief that produces problematic data, that produces problematic model, that produces problematic predictions, that produces more problematic training data. So you can think of a variety of situations. And in each particular case, it will probably be slightly different. But the important aspect here is how it affects the problem. So again, we have more negative outcomes in this case than we would have otherwise had if we would have drawn randomly. I'll let this sink in for 10 seconds. All right, well, let's have a look at several models that, in this case, we'll try to use to address the problem. We'll start with our simplest benchmark. So we'll essentially just apply logistic regression to the observed data. And importantly, we'll try to do something that is called fairness through unawareness. So we'll just say, OK, we don't want to bother with the group. That sounds kind of discriminatory. So we'll just leave it out. So we'll just use x and z, which both seem to be correlated with the outcome. So let's see how this model performs. The important thing that we do not know here is that z is actually not only not really important for the problem, it is actually going to pick up some of the group's information implicitly. So here's what we will see with this model. So on the right, I have a violent plot for probability distributions for two groups compared to each other. So in the world where things are perfectly symmetrical, we'll see them essentially mirroring each other. In this case, we have a clear discrepancy. And let's have a look at a few metrics that we'll later compare with other models. So first of all, false positive rate for purple is about 28%, relatively high. We could also measure this for the orange group. This is something we are a little bit less interested in here, but we'll do this anyways for the sake of fair comparison. And one important thing here, when we evaluate these models, again, because this is a simulation, I'm not doing it on the biased training data. I'm doing this on unbiased real data, right? You can see it in the way as we could, I don't know, conduct an experiment to actually independently draw some samples from the population to see whether we actually predict well in a world whether the data is not biased, doing it on the training data itself essentially will only tell you how good are the models at picking up problematic patterns in that data. So that's not really something we're interested about here if we want to measure fairness. And one more important thing to see here is accuracy, right? So we'll often see a bit of a trade-off here, so more fairness, less accuracy, and the other way around. So that's something we'll keep an eye on across models as well. And last but not least, so what I call miscalibration here is essentially the difference between the true rate, so again this was 36%, and what the model predicts on average. So 8% in this case means that on average the model gives 44% for that group, so more than it should have. All right? So yeah, conclusion. On the surface it could look like a reasonable start, but we see that we already introduced a problem without even per se noticing it, and there's not so much in the outcomes that would tell us that a lot is going on, but we'll discover this in the process, and here I think the second example is the one that is more striking because it is an approach that is commonly done without much thinking, kind of as a good rule of thumb, but you see that here it can make things a lot, a lot worse. So essentially what I'll do here, right, so we have imbalanced data set. We have majority, we have minority, so in a lot of cases you would either downsample the majority or up sample the minority to kind of equalize them a little bit to address that problem. So probably at least most of you have either tried it or done this. It's considered a reasonable practice in a variety of cases, but let's see what kind of impact this has here. So what we're going to do is we're going to down sample the majority to the same amount of samples, 50 in this case. So we have the same amount of samples now. So let's see if that helps fairness and the rest or not. Try to make a guess for yourself. It actually makes things a lot worse. And what is the intuition here? So again, the training data for purple is biased. So essentially, by increasing the relative importance of that part of the training set, we essentially make the impact of this bias much larger. So essentially, something that potentially could have looked like an innocent improvement of our modeling approach can actually make things a lot, a lot worse. And in this case, as you can see, it exacerbates unfairness quite a bit here. So we have 41, almost 42% false positive rate here. And accuracy actually also drops, because now we also start to make pretty bad predictions for the majority group as well, because it's again influenced by the biased data. All right. Well, now the interesting part, that how we're actually going to use the Bayesian approach here to address the problem. So this is important part again if you get the technical and intuitive bits here I think you will get the rest as well and this is the most important baseline model. The next model will be a little bit more technical so if you think this is a challenging topic this is the one to get your focus on. But essentially on the overall conceptual level it's not something vastly different. It's more the approach that we use to evaluate this is quite different in this case. So, again, we are still talking about the logistical approach to the problem, but the way we're modeling it now is a lot more explicit and the way we're going to learn parameters is through Bayesian approach rather than the way we did it before. What is most curious here is we actually now explicitly including the parameter for the group, so in this case, beta purple times the group indicator. This might seem controversial, but as you'll see in a second, this is actually our way of saying, okay, let's make a certain assumption, that is, the group should have no impact. We'll capture this assumption with a prior, and we'll let the data tell us whether that's something that should be invalidated or not. And the nice thing here in terms of interpretation also, you're very explicit about what you're doing. You're saying this is what we believe, this is how we let the model pick this up, but if we're wrong, if there's something wrong with this assumption, this will essentially get support or invalidating evidence in the process. So let's see how this model performs. Again, I'm skipping here the exact process of getting the posterior distributions and such. But again, the idea is that we are introducing this additional coefficient, that is one thing, and the way we estimate coefficients now is using essentially bias rules, so we get not just point estimate, we get whole posteriors for each parameter, and this will have some nice positive side effects as well. And maybe very briefly, again, I didn't want to go here too much into code, but what kind of PyCon would it be without any Python, right? So if you perhaps have seen PyMC, one of the most prominent libraries for Bayesian statistics, this is essentially how you define this model. This is nothing radically complicated. In fact, if you compare the formula on top to the code itself, it's essentially a one-to-one match in a style similar to NumPy. So it's really quite relatively straightforward to get into. Right. But here are the results. An important thing here, the way we choose the prior, of course, really, really matters. We can choose different types of prior distributions for the parameters, in this case I've chosen quite a conservative one, so essentially, as I said here, it's centered around zero and we want the variance to also be relatively conservative, so we want to really have good evidence that there is a group effect in order to actually create a significant coefficient there. So what this does, as you can see here, it actually makes things a lot better, even though again we do have explicit term for the group, so one could have thought that this actually makes things worse, but that's not the case. So the false positive rate goes down to just about 10%. Interestingly, accuracy stays the same, or even maybe slightly higher than the first model. And miscalibration is also a lot more moderate, and you see on the right on the violent plots essentially the distribution's not the same, but a lot more symmetrical than they have been before. And again, why did that happen? It's because we essentially regularized the model through applying this conservative belief that there shouldn't really be a group effect unless there's a lot of data to support it. And guess what? There isn't actually a lot of data to support it, so we just learned that perhaps we need more evidence to really introduce the difference between groups there. And a really nice way that you can look at this is essentially directly looking at the posterior distribution of posterior distribution of the parameter here and we can say with 95% confidence interval here that the parameter is somewhere between minus 0.11 to 0.19 so essentially zero somewhere in the middle so we can say here with large degree of confidence we're actually not sure whether there will be any effect at all in this case all right very briefly about fourth model that's a bit of an extra here but I think still worth mentioning and then we can wrap up for the sake of time so it's important to mention that you can dig much much deeper in this direction essentially you can introduce much more complex models that explicitly model not just an individual group effect but perhaps structural differences between groups so we could have had a model that actually as different intercepts for each group but we can connect them together by essentially drawing them from the same distribution that is also known as hierarchical approach in Bayesian statistics it's a little bit more complicated so again I would only recommend diving into it if on a conceptual level that is clear and you're dealing with perhaps multiple groups and there's There's quite severe imbalances between them, but as I'll show in a second, this can end up being even more effective. So to summarize, again, we mostly focused on the third model. We can see here how much it has decreased, essentially decreased the false positive rates, and also accuracy actually stayed quite competitive, and if you would go all the way down to more complex models such as the hierarchical approach, you would essentially make things even better you see here that essentially we have close to zero false positive rate, so a lot more potential there to unlock.

Speaker 2 [26:01]

that should be the end sorry we need to take questions also we have some questions there in essence the payroll on the group include an L2 penalty on including the group could need to achieve something similar using a rigid regression

Speaker 1 [26:31]

Yes, that's a good question. So maybe to reiterate, can we achieve a similar result to Bayesian approaches with regularization? So say logic with penalization. That is true. That is indeed a quite powerful competitive alternative. Important difference where I would still advocate for Bayesian approach is that here we explicitly model our belief on fairness, while with penalization, it's a little bit implicit. So we just say, hey, we want to punish the model to avoid certain behaviors, but we don't really know what is going on and can only essentially guess based on the outcome instead of being very transparent about how we approach the problem. And I think for the topic of unfairness, being explicit and transparent is really quite important.

Speaker 2 [27:13]

Thank you. Can you eliminate all biases? How do you make the difference between desired behavior and actual behavior?

Speaker 1 [27:21]

difference between

Speaker 2 [27:22]

Can you eliminate all biases? How do you make the difference between desired behavior and actual bias? Sorry desired behavior and actual bias

Speaker 1 [27:33]

So I'm not quite sure I understand exactly the difference between...

Speaker 2 [27:36]

desired behavior and actual bias.

Speaker 1 [27:39]

And what kind of difference are we talking about?

Speaker 2 [27:42]

I'm not sure.

Speaker 1 [27:56]

So, there are two imbalances we're talking about, right? One is majority-minority, the other one bias-no-bias for the minority. As I said, we usually would not know whether there is bias or not, minority-majority we do know, right? But I think it's a fair assumption, though, to say that there is a high risk of bias in most of such problems, especially where there's some ethical concerns or potentially past prejudice existing. So, I think it's a fair assumption to say that there is likely some form of bias, so So let's pick a conservative approach that would really need a lot of support, like in this case very conservative priors, to actually invalidate that.

Speaker 2 [28:34]

If the data is biased and unfair, do you update your fear prior into unfairness? How does BASE help?

Speaker 1 [28:44]

So, did I understand correctly, like, in the situation where everything is unfair, would a fair prior actually produce fairness?

Speaker 2 [28:52]

Yeah, he said, if the data is biased and unfair, do you update your fear prior into unfairness? It's just like, do you?

Speaker 1 [29:01]

Sweetie thing right so priors are not some kind of magical peel that immediately solves your unfairness problem It's a tool right and it's as any powerful tool that requires responsibility to use it very carefully, right? So in this case, that's something that you would really want to carefully think about Because again if you use it in the wrong way or perhaps introduce a problematic assumption Then essentially you can also make things worse right or you can can take a very broad prior Then essentially you will not really help the model to be conservative in this case In fact, you'll just provide it with a group factor to introduce even more discrimination based on on it So again powerful tool, but in cautious hands that understand what they're doing. Thank you

Speaker 2 [29:40]

Thank you. I think this one is similar. How do you introduce a meaningful prior while preserving fairness?

Speaker 1 [29:52]

Yeah, that's a good question. So how do we introduce a meaningful prior, and in general, what is the rule of thumb for priors? Well, here we had a good example, right? So for the, really quickly go back. What we have here is essentially prior that is centered around zero with a very narrow distribution around it. So we say we are quite confident that it's probably around zero. So again, the distribution, he says that I think it's quite unlikely that it's far away from zero. importantly I'm not saying that this is impossible I'm just saying I would want to see a lot of data to actually to actually make me believe that there is a substantial difference between groups and that's a fair approach

Speaker 2 [30:32]

Thank you so much. Because of time, we will stop it there. A round of applause for him again.