Thou Shall Judge But With Fairness: Methods to Ensure an Unbiased Model

Speaker 1 [00:03]

So a very good morning, and thank you for coming for my talk, Doshal Judge, but with Fairness, Methods to Ensure an Unbiased Model. So I'm Nandina Sriraj. I'm a data scientist at Scientist.ai. So let's move on to the talk. So before I start with my talk, I will just like to quote a phrase from the book, To Kill a Mockingbird. So people generally see what they look for and hear what they listen for. So human beings are, like in general, human beings generally make judgments. But we as human beings must make sure that the judgments that we make are fair. Likewise, our ML models or ML algorithms also make judgments. And we have to make sure that the judgments that our ML model makes are fair. Thus, it brings us to our topic, though shall judge, but with fairness, methods to ensure an unbiased model. So I'll just give you an agenda of today's talk. First, we'll be starting with the introduction. Then Then we'll be moving on to what bias in ML models means. And then we'll be seeing what model fairness is. Then we'll be moving on to model fairness mitigation in three main stages of the ML models pipeline. Then we'll be seeing some useful fairness evaluation matrices, followed by Python code to gauge the fairness of a model trained on a law school data set, followed by conclusion. So let's hop into the introduction. So let me throw some light on the growing importance of machine learning and artificial intelligence in today's scenario. So big data and machine learning are pervasive and have a very rising influence on the current society. So our world is getting automated day by day. So most of the daily tasks that we do are influenced in one way or the other by some AI or some machine learning algorithm. So let's take this job screening platforms. Even job screening platforms in the preliminary stages use AI or ML. So yeah, in every day-to-day task, we can find some ML algorithm. And in today's scenario nowadays, even ML algorithms are getting used in decision-making scenarios. Let it be of an individual, or let it be of an organization or a business. So there is a very stressing need or a very pressing need to make sure that our ML algorithms or our machine learning models are fair and ethical. So before we move into machine learning models' fairness, let me introduce you to the term bias in ML models. So yeah, bias is kind of a vague word, so let me try to define it to you. So we say that a bias has occurred when our ML algorithm displays inaccurate or inappropriate results due to flawed ML assumptions. So we all know that our machine learning model is built upon a number of ML assumptions. So when these assumptions are flawed or are actually faulty, This will lead to our ML model making predictions that are actually wrong or inaccurate and decide predictions. So a bias can exist in any shape or any form, and it can be introduced in any stage of an ML model's pipeline. So let me try to brief you the ML model pipeline. So it starts with business understanding, then we collect the data, then we label the data, we train the model, we deploy the model, and then we evaluate the model. So a bias can be introduced in any of these stages, let it be the business understanding stage or the data collection stage, or maybe the last stage model evaluation stage. A bias can get introduced in any of these stages. So let me try to define bias. Bias is a phenomenon when our model, our ML model, consistently make predictions that are skewed due to faulty premises or maybe faulty ML assumptions, as I told. And as I already mentioned that the ML algorithms are even getting used in the decision-making sector. So these bias can lead to significant losses or significant mistakes too. So we need to make sure that our ML model is free of bias. So let me give you a brief example regarding of bias. So let's take this example of COMPAS. COMPAS is Correctional Offender Management Profiling for Alternate Sanctions. So let's forget the full form. Let's take it as COMPAS. So COMPAS is an ML algorithm that was used in the state courts in the United States to predict if a person is reoffending or not. And it has been seen that the African-Americans had a higher chance to be predicted as reoffending than they actually did. And the white Americans had a lesser chance to be predicted as reoffending. So this is a decision-making scenario. And we don't want such kind of bias in such a state court scenario. So the bias needs to be removed. So that is why this topic becomes relevant. So let me try to brief you what are the types of biases that can exist. The first one is bias in data. So we can actually divide bias in data broadly into three. The first one is historical bias. Historical bias is the bias that is inherent in the data or that is seeped into the data. Even if we collect data from perfectly sampling environments, there might be some bias that is actually inherent in the data or that has actually seeped into the data, even if we make sure that everything is right. And this is known as historical bias. The second one is representative bias. Representative bias occurs from the scenario or the population from which we are actually gathering the data. So maybe the population from which we are collecting the data is not representing all the population that is desired. So let me give you an example. is the data collected from smartphones. So the data collected from smartphones are under-representative of the data of people with lower income or the people who cannot afford smartphones. So their data is absent in the data collected from smartphones. So their data is not getting represented. And this is representative bias. The third one is measurement bias. Measurement bias occurs from the noise in the environment from which we are collecting the data. So maybe the labels are getting labeled incorrectly, or maybe we are getting wrong feature values. So due to the noise in the environment, we may get wrong values. So this leads to measurement bias. So measurement bias, historical bias, and representative bias together constitute bias in data. Now moving on to bias in modeling. So bias in modeling can be actually categorized into two, the first one being evaluation bias. So evaluation bias occurs in the model iteration the model training phase. So for every ML algorithm that we use, so we have set certain benchmarks to qualify like this model is good. So if it crosses certain benchmarks, okay, we say that, okay, this model is good. If it's not matching certain benchmark, we say, okay, this is underperforming. So these benchmarks, we actually tend to generalize these benchmarks, but these benchmarks may not be applicable for each and every case. So there might be scenarios where these benchmarks may not be applicable for this particular scenario. And And this leads to evaluation bias. And the second one is aggregation bias. Aggregation bias occurs when different populations are combined inappropriately to train a model. OK, I'll try to explain that with an example from the health care sector. So let's take the example of a model used to predict whether a person is diabetic or not from the levels of HbA1c. So it has been seen that the levels of HbA1c differ across various ethnicities. That is the level of HPA1C is different across different ethnic groups. So we can't get like a generalized model to predict whether a person is diabetic or not across all these ethnicities. And this leads to aggregation bias. So aggregation bias and evaluation bias totally constitute bias in modeling. Now moving on to the third type of bias, bias in human review. So this happens when a human reviewer or a human person actually overrides the predictions that are made by the ML model. model, and he induces his bias into the ML model, and that causes bias in human review. Now moving on to the umbrella term of model fairness. So model fairness is another vague term, which I'll try to define it for you. So it can be broadly defined as the absence of prejudice or preference for a particular individual or a particular subgroup or a particular group or community. So it can be understood as designing or creating algorithms that are actually free of bias, of outside bias and gives us accurate and desired predictions or desired results. So this is how we can define model fairness on a broader scale. Now moving on to model fairness mitigation in three stages of our ML models pipeline. The first stage being data mining. So in this stage where we collect the data, the first thing that we need to do is identify our protected attributes. So our protected attributes in our case may be like religion, it can be race, it can be be ethnicity, or it can be gender. There can be a number of protected attributes. So first step should be to identify. We should know our data set. So we should be able to identify the protected attributes. And we need to measure the data distributions of this protected attribute. And we need to ensure that every subgroup inside a protected attribute is having a good representation or enough representation in our data. We need to ensure that no subgroup is being underrepresented, or no subgroup is being overrepresented in order to prevent the representative bias that already I mentioned. So in some cases, we get data sets where we don't have enough data for a subgroup. In these cases, we may resort to techniques like oversampling, or we may use some techniques to create synthetic data set. And our whole motive should be to have a very balanced data set. That is a really key point to eliminate bias. Now moving on to the model training phase. In the model training phase, we need to actually identify the metrics that we are going to use to evaluate our model's fairness and bias. And we need to use them while training the model. As we go on, I'll be showing you a few metrics that can be used. There are a number of metrics that you can use. But I'll be pointing you out a very few metrics when I go on. But you'll get more clarity on this as we go forward. So the last phase, that is the post-processing and deployment phase. In this phase, we need to compute the fairness matrices on the test data set. And also, it's really essential to know the distribution of data in our test data set. So we have already identified our protected attributes, our sensitive attributes. So we need to get the distribution of data for our test data set for that particular protected attribute. We need to see whether, OK, this particular subgroup is underrepresented or overrepresented in our test data set. And another essential thing is to see the performance metrics of this protected attribute. So we can use accuracy, or precision, or recall, whatever needed. So we need to see whether these performance matrices are actually performing well for every subgroup inside our protected attribute. And then there are a lot of open-source tools that are available, which will help us to detect bias and eliminate bias. So I'll be mentioning a few tools as we go on, too. So you may use them to identify and mitigate your bias, too. And another key essential point is to evaluate and keep track of your production's fairness metrics. That is also really essential. Now, as I mentioned, these are some of the tools which you can use to detect and mitigate your machine learning model's fairness. The first one is, these are all open source tools. The first one is Microsoft Fairlearn. So as we go on, I'll also be showing a demo with a law school data set, how to compute fairness matrices. I'll be using the first one, that is Microsoft Fairlearn. But IBM's AI fairness 360 is equally good. There is Google's What If tool, and there is Equitas. All of these you can use in order to calculate or compute the bias. And they also have techniques to mitigate them. Now moving on to some useful fairness evaluation matrices. The first one is error rate. So these are actually technical terms that you can actually use to compute your model's fairness. First one is error rate. Error rate is the degree of prediction error in your data set. So for all these fairness evaluation matrices that I'm going to say, you can actually set a threshold. There are actually a set of accepted values for the same. For error rate, the accepted value is 0.1. These are kind of hyperbarometers that you can tune according to your use case or according to your data set. But yeah, generalized accepted value for error rate is 0.1. So you can say that. If the ML models, the error rate that you're getting is greater than 0.1, it's kind of alarming. And then if it's below 0.1, OK, you're good to go with error rate. So you can define your fairness metrics and the threshold for the same. The second one is equalized odds. Equalized odds ensures that the accuracy is high for each and every subgroup inside your protected attribute. And it ensures that your model is not just favoring the majority subgroup, but also the minority subgroup. And equalized odds helps us to punish the model if it only favors the majority group and if the accuracy is only great for the majority group. Maybe your overall model's accuracy might be like 94% or 95%, but maybe for a particular subgroup or a certain subgroup, the accuracy is really low. So equalized odds helps us to identify that. The next one is demographic parity. Demographic parity says that a model is fair if the number of positive predictions of a particular subgroup matches its group membership percentages. So I already told these are all like technical terms, like technical matrices. Bear with me on that. And now this next one is overall selection rate. Overall selection rate is the number of elements that are selected or the number of candidates that are selected to the total number of candidates under consideration. And the last one is the area under the ROC curve. So ROC curve is like a graph that represent the performance metrics of a classification model in all possible scenarios. So area under the ROC also gives us a good understanding on how fair our model is performing. Now moving on to the Python code to gauge the fairness of your ML model. So I have used a law school data set and a classification model. So the data set had 20,427 rows. The protected attribute that I used is race. There were race ranging from 1 to 8, 1, 2, 3, 4, 5, 6, 7, 8. The target variable that I used was bar passed or not. It's like a binary variable, 0 or 1. Now moving on to the model that I've used, I've used a simple decision tree classifier. I just divided the data set into test and train. I just use a simple classifier to make the predictions. Now moving on to the portion of our interest, that is using Fairlearn to gauge the fairness matrices. So basically, Fairlearn is an open source package. And I already mentioned the matrices, like overall selection rate, demographic parity, equalized odds, error rate, et cetera. So Fairlearn has inbuilt libraries. You just need to import Fairlearn, call the function whichever you need, pass the parameters. So that's all that you need to do. And it'll do everything for you. Your job is done. And now, as I told, it's very important to compute the performance matrices too. And we need to analyze how your model is performing for each and every subgroup inside your product attribute. Fairlearn also allows us to do the same. So Fairlearn has something called metric frame. So in metric frame, we just need to specify which all performance metrics that we need to use. I've used these, like accuracy, precision, FPR, FNR, true positive rate, true negative rate, selection rate, mean prediction count. There are a number of matrices more. You can use any of them. So just import metric frame, pass which all performance matrices that you need, just call the function. And it will give you the results. Now, moving on to some visualization about the data set and the results. So this is the data set that I've used. And this is a distribution of the protected attribute rays. You can see rays from 1 to 8. And you can see that our data set is not very balanced. So you can see that 4 is having a very red one, is having a very high population, while 3 and 6 are having a relatively lower population. And now this is how Fairlearn gives you the result for fairness matrices. So you'll get values like these. You can set thresholds. There are accepted values. You can also set thresholds according to your need. You can tune them. But yes, I told for error rate, say the threshold is 0.1. For here, the error rate is 0.08. It is below 0.1. OK, your model is performing fair. So likewise for all these, like overall selection rate, demographic parity ratio, parity difference, equalized odds difference, everything, you can set thresholds and check if your model is performing fair. Now, analyzing the performance matrices for each and every class of this protected attribute. So you can see that they're all bar charts, and they're having eight bars representing each one of the rays from 1 to 8. Let's just consider the first graph, first blue graph for accuracy. So you can see that for my particular, this model, the accuracy for every single ray, it's above 0.5. So it's having pretty good accuracy for each and every rays in my case. But it might not be the case in your case. Maybe in your case, the accuracy might be low for a particular ray. You'll be able to identify from this. So yeah, if you go to the last graph that is count, you can see that 4 is having a very big bar, whereas 3 and 6 are having a little bit lower. So in this case, what I could infer is that 3 and 6 is having lower population. I need to create more synthetic data, or I could do oversampling to create a more uniform data set. So likewise, you can infer from all these graphs what you can do for your particular use case. Now, moving on to conclusion. So let's sum up what we have learned. So we have seen what biases and what different types of biases are, and then we have seen model fairness, and then we have seen model fairness mitigation in different stages in your ML models pipeline. We have also seen the various tools that can be used and various metrics that can be used to gauge an ML model's fairness. We also saw a small demo on how to use Microsoft Fairlearn to gauge the fairness of your ML model. Let me conclude my talk by quoting a phrase that was said by Professor H.V. Jagdish from University of Michigan. It is up to us as responsible data scientists to make sure that we are using the power of technology to do the right thing. Thank you.

Speaker 2 [19:22]

Until we wait for some extra questions on Slido, I want to thank you for your great talk. Thank you. Okay, let's come to the first question I saw. Is Vogue Culture the solution to fairness in ML?

Speaker 1 [19:37]

Can I see the question?

Speaker 2 [19:38]

Is vogue culture the solution to fairness in ML? Like when we are all, when all people are ethic unbiased, is ML also unbiased?

Speaker 1 [19:50]

No, we can't say that. So according to me, as far as I know, we can't say that. There's no situation which is free of bias. It's something ideal. We can't say that this situation or this scenario is completely free of bias or completely ethical. So as of now, it's something ideal and it's something hypothetical. We can't say that there's something completely ideal. So according to me, in every situation, some of the IR bias can exist. It can be used in any part of the pipeline. So even if everyone wakes up or something, but we need to ensure that this needs to be ensured in the pipeline.

Speaker 2 [20:27]

How to identify protected groups if they are not available in the data due to privacy reasons?

Speaker 1 [20:36]

Okay. So actually, I didn't get the question.

Speaker 2 [20:41]

How to identify protected groups if they are not available in the data due to privacy reasons?

Speaker 1 [20:50]

So there is nothing like straightforward protected groups or something. So I hope this answers your question. So there are some generalised protected attributes such as race, gender, religion. And even if these are just some question that comes to your mind, we can just remove it, right? But no, this can be implicitly also implied through some zip codes or something. This can be implicitly implied. So I think how you can identify them is that there are certain straightforward protected groups that are available, that is, race, religion, that you can find a set of, when you Google it, you can find a set of protected attributes that you can get. And for privacy reasons, I'm not understanding, if the data is there, if you can identify what are the column names or what are the columns under consideration, you will be able to identify the protected attributes. It's pretty straightforward.

Speaker 2 [21:43]

How do you know that the fairness metrics are not biased by themselves?

Speaker 1 [21:51]

That is a pretty, like, that's a tricky question. I'm afraid I'll be able to answer that. But, yeah, like, we're not just using one metric, right? So we are using a combination of fairness matrices, combination of performance matrices, combination of data distribution, like, everything. We're not just looking into one particular thing. Since it's a combination of many things, like, even if one of them is biased, it's okay. we can get gorgeous from the others it's we're not just concentrating on one thing so yeah

Speaker 2 [22:28]

A model predicting credibility predicts men are more credible than females simply because of the gender pay gap. How to trade off accuracy and fairness?

Speaker 1 [22:47]

Can I?

Speaker 2 [22:48]

A model predicting credit credibility predicts men are more credible than females simply because of the gender pay gap, how to trade off accuracy for fairness.

Speaker 1 [23:01]

Okay. So, yeah, this is, like, in this case, I'll just rephrase the question. So, the question comes in, like, some of the question. So, we have this gender as a feature in this particular data set that he has mentioned. And for gender, like, yeah, earlier, there's a pay gap between men and women. So, as mentioned here in the question. So how to like, so it's something historical, right? It was something that happened in the history. It's kind of historical bias that the question, the person is mentioning in the question. And he's asking how to trade off accuracy to prediction, like to fairness. There's nothing like to trade off accuracy for fairness. The thing that we need to do is find a balance between both. Like we need to find out like a proper balance between accuracy and fairness. And as we know, in fairness also, we are looking into the accuracy. So it's kind of, it comes into that also. So there's nothing that you need to trade off your accuracy for fairness. It's better to find like a balance between both. And yeah, ensuring a balanced data set, I think, won't compromise on your accuracy. And even if like, the main thing I would suggest is to find a proper balance between your fairness and accuracy. You don't need to like give off your accuracy, just go behind fairness. is, you can actually find a balance between both. That is the ideal scenario.

Speaker 2 [24:23]

Patrick is asking, how can I make sure that I don't miss any protected attributes or groups?

Speaker 1 [24:32]

That's a good question. So basically, it's finding particular attributes is kind of a, it's straightforward, but then it can become tricky in some cases. But the main thing that you can do is that understand your data set. So that's the first thing that we can do. Understand your data set, understand the columns under consideration, and as I've mentioned, there are a lot of like typical bias, like for data bias, modeling bias, or bias in human review. See your data set, see which bias applies for you. See the scope of each and every bias. So let it be historical bias or gender. So when you go into each and every bias, you can see, okay, this particular attribute can come under this bias. This particular attribute can cause this bias. So go through these different types of bias, identify scenarios, understand your data set, identify scenarios where a bias can get introduced, and then you'll be able to find which all features are actually your protected attributes. That's like a step-by-step procedure in finding your protected attributes. But yeah, there is like a list of protected attributes used in general, like race or ethnicity or gender. But this is more of a systematic flow, like to go through each and every bias, possible bias, each and every possible, like go through your data set, your columns, and identify, like, which column can cause which kind of of bias. Understand your data set. That's the sum up.

Speaker 2 [25:59]

Morena is asking, could you elaborate a bit more on how AUC can measure fairness in an ML model?

Speaker 1 [26:08]

Okay, so basically I hope she's mentioning about the ROC curve. So basically you will have a curve, it's a curve that will actually depict the performance of a classification model in all possible scenarios. So it will give us the area under the curve. So we can see that if the area under the curve, there are certain values, I'm not actually remembering the values, but yeah, there are certain values for which your AUC is considered to be good, there are certain set thresholds. So I also told that these metrics have set thresholds, accepted values. You can see that if your area under the ROC curve is falling to your particular set threshold, then it is actually considered to be fair, else it's not. And okay, as I told, it's like a kind of a hyperparameter that needs to be tuned according to your use case. So we can't actually generalize everything. But yeah, tune your hyperparameter according to your use case. but there are, like, certain set values for each and every, like, each of these fairness matrices. You can feel free to use them. But, yes, understand your data set, see if that's applicable for you, like, understand everything, and then, yeah, use them.

Speaker 2 [27:19]

The next question is, what do we thrive to achieve in the case of fairness being the representation of our world or creating a model that will influence our world in a positive way?

Speaker 1 [27:34]

Can I see the question?

Speaker 2 [27:36]

What do we thrive to achieve in the case of fairness being the representation of our world or creating a model that will influence our world in a positive way?

Speaker 1 [27:48]

Okay, yeah. By fairness, what I mean is, like, what fairness generally means is, like, absence of bias. It's not, like, converting the world into a positive way. Like, in all the scenarios, as I told, a bias can get introduced. It's not, like, making the world more fair, like, making the world more positive or something. It is like we'll have bias in most of the real world scenario. We can't get an ideal scenario or something. There are chances that a bias can get introduced. The focus of fairness is to eliminate the bias or reduce the bias as much as we can. At least we can make the model perform better. We are not trying to make an ideal scenario. We are trying to make our model perform a little more better.

Speaker 2 [28:40]

The last question is, can we optimize for fairness at the moment of training? Can we optimize for fairness at the moment of training?

Speaker 1 [28:51]

Yeah, like, some of these we can understand from the, so as I told, like, in every stage, we can actually find out and start our optimisation for fairness. So in training stage, before training stage itself, like, we can understand the data set, right? We can see the data distribution. So we can eliminate, like, data distribution if some particular subgroup is underrepresented. We can actually do oversampling, or we can do undersampling, or we can create synthetic data set. into some kind of these things even before we go to production. So in the model training phase also, if the model actually comes with a better data set to train the model, that is actually ideal. We need not wait till the production phase. We can actually do this before our training phase itself, making our training data set a little more balanced. So that is what we can do before the production phase.

Speaker 2 [29:41]

Thank you for the great insight into ethics in ML.