Breaking AI Boundaries: Fairness Metrics in Unstructured Data Domains

Speaker 1 [00:06]

All right. So let's start. Yeah, I'm Daniel. And I will today talk to you about how you can ensure fairness, especially in domains where you deal with unstructured data. But before I really dive into that, I first want to talk about what unstructured data is and also what I mean with fairness. So when I talk about unstructured data, I talk about, for example, image data, audio data, large text documents, and stuff like that. So basically data that is not that well-structured, like tabular data or data you would normally find in database systems. And why is that an important type of data? It is mainly because, for example, in my case, I work a lot with automotive customers. So there you have a lot of, for example, testing and simulation and stuff like that. And these processes are all centered around physical systems. And when measuring them, you just generate a lot of this unstructured data. For example, audio recordings of a car you maybe want to do testing for or pictures of some parts you want to analyze for anomalies and stuff like that. So in general, I mainly deal with those use cases. So what about the term fairness? I will keep it pretty casual, not too academic. and what I mean with that is you basically want to make sure that your machine learning model is working the same way or equally good for different groups that are relevant for your use case. And I can probably show this in a really brief example. For example, here we have a use case where you want to basically classify if a patient is susceptible to heart failure based on like some medical parameters and yeah it is pretty obvious that you want this model to work equally well for example younger people older people female male people and so on and how do you now ensure this in this like pretty trivial case it's tabular data so you would simply slice your data into for different groups for example you could measure okay for the people where the attribute sex is female i will get an accuracy of 80 percent for the male category i would get 85 so i could simply compare them like that but for unstructured data it is not that easy and therefore like i have another example in this case this like a data set is pretty popular for emotion recognition it's called effect net and there the goal is you basically have portrait pictures of people and you have a label that can be for example anger or neutral or happy or something like that and And yeah, could of course be part of a larger system, and also there you want to ensure that the groups in the data are treated fairly. Yeah, but how do we ensure that now? And yeah, you quickly realize, like before we had this tabular data example, and then we had these attributes like biological sex or age group explicitly given, we don't have that here. basically like the case in most industry use cases I dealt with, at least. And yeah, especially if you are like, this is a pretty obvious example, because it's directly affecting humans. But my talk is also about use cases that are not directly affecting humans. So how could we apply this framework here? And the key is we have to discover these important sub groups in the data. And this is basically what my talk will be about. So how can we identify groups where we want to measure how well our model works? But before I come to a practical example, I want to quickly dive into the question. You may ask yourself, is this relevant for my use cases if I have this indirect human impact? And basically, my answer to this is yes. To keep it short, usually you have use cases with not-so-structured data, and then you have this human impact axis. And there are use cases where you have a high human impact. For example, you could have a medical diagnostics use case, like MRI data or something like that. Or you could have another use case where the human impact is not so direct. For example, if you want to measure or to judge the vehicle comfort by the classifying brake noise in vehicles, like a typical use case I usually deal with, it's not that important from an ethical standpoint. But still, judging these subgroups in your data, finding them, seeing how well your model performs, will just give you really, really good insights where your model fails, and also gives you some actionable insights, like what to improve in your data and your model. Yeah. So what is typically the existing workflow I see when I am, for example, consulting with customers and so on? Usually, people not look into their data a lot. Typically, they have picked some global target metric, and that is also important because you have to compare things and have to measure things. But they are really driven by this metric, typically. They usually just measure it in some report and then iterate on their model and their data with not so much looking into the real data and model problems. Maybe there is some, I would say, like, primitive reporting where you say, like, the worst five examples or something like that, but that's about it. And, yeah, like this, you can, like, not get actionable insights on how to improve your model, and you also cannot ensure that people that are maybe somehow represented in your data set are treated fairly by your model. and I see like several really important components to improve over this one is being able to interactively analyze your data so like going beyond the static reports like just having like the worst five examples or something that's usually not enough you should be able to really browse through your data recognize patterns and stuff like that so that's one component we will like soon see what I mean with that in practice then also usually what also keeps people from applying these principles is it is just really like time-consuming so it can be helpful to also have mechanisms in place to allow automatic detection of these issues so that's just like to even make people use it otherwise it's usually not done and the other part is maybe like a little less important but still nice like if you can get effective explanations like you know okay these data points work badly and you get an concise explanations to even like more quickly like find an explanation why your model fails that's also really useful so So let's look at our updated model evaluation and, to some degree, fairness insurance process. Usually you first start out with the raw data. In case of our emotion recognition case, this is these raw images. And the first problem we have to deal with is we have to get representations we can't even process this because you probably know, yeah, that's not that easy. We need some sort of features or vector representations to even make the data searchable for these problems. And what I usually do in this case, or we experienced works well for us, simply one way is using model embeddings. And a really useful resource for this, That is, for example, the Hugging Face Model Hub. So there is a lot of models that also capture different properties from the data that can help you just transform your pictures in these numerical vectors that, for example, could be a general image similarity, could be also more specific, for example, having properties of people's faces or something like that. Or you could simply also try to more explicitly extract interpretable features from your data, which is shown below here. There you could, for example, say, OK, the biological sex of these pictures shown here is male. The race is black, and so on. So this will just make the search a little less bothersome later on, because you have this interpretable features instead of just numerical vectors. So what do we want to do next? Basically, our goal is now to identify the subgroups in the data. And therefore, we want to find clusters and measure how well these clusters perform according to our target metric. And the first step is finding the clusters in the first place. So this is just finding groups in the data. So it's like some dimensionality reduction plot on embeddings that you see here. And you can basically identify, OK, this is a group of similar images, basically. So they have something in common. Typically, what I use for this clustering, I usually use a combination of dimensionality reduction approaches and clustering. So for example, just PCA, UMAP, and then some hierarchical clustering approach works pretty well for that. Why hierarchical clustering? This has a simple reason, because you will try to measure the metric drop on the clusters. And if you look for really small clusters, you will usually get a large metric drop. But the cluster support, so there's number of samples really small, that's usually more related to errors, like really outliers in the data set. If you look for larger clusters, it will usually be more related to larger unwanted biases. So this hierarchical clustering gives you both, and you can sort of switch between the different partitionings of the data. That's cool about it. Yeah, the next step is pretty trivial. We found those clusters where we have some similarities, and now we just measure how well our model performs on it. We can probably just use our target metric, or maybe if there are some imbalances and so on, we have to choose something a little more sophisticated. But basically, it's just like measuring the metric you care about on these clusters. And yeah, then you basically have a list of clusters where you're generated from your unstructured data, where your model is not performing well. You can simply order it and say, OK, I will care for the ones, for example, where the model works worst. But there's one crucial step missing, and this is finding out why is the model failing. Because now you have just like, OK, my model fails on certain clusters. But of course, you need some interpretation and also need an action you can take to mitigate this. And yeah, basically, what proved to be really efficient for us is just interactively analyzing these results. So you can basically just look at your cluster list and then just check, OK, what type of images are in this cluster? And then try to visually find similarities between those images and make a good guess why the model is probably failing. I have to say that this can be substantially easier, more easy, if you have these interpretable features. Because then you can do a search or a clustering based on these interpretable features, and can also here explicitly look at these attributes. And this will make the interpretation a little easier in some cases, maybe in some cases, but not that, cannot capture every problem because you make some assumptions when choosing these interpretable features in the first place. Right. Yeah, that's basically the typical process that we choose here, like what we also use in customer projects. And now it's demo time. I hope it works. Usually not, but let's see. So how does this look when we do it interactively? Just make sure that this is loaded. Yeah, in this case, you see I have basically this data set in a data frame. And in this case, I have the image. So, I could use model embedding for doing this process, or I could also use attributes I extracted. So, this is the other way of doing it. That's actually what's implemented here, because it's also a little bit faster. You may ask yourself now, how did he get these attributes? In this case, it's a little bit maybe experimental, but I used the Lava model. So, like this multimodal LLM that basically takes images and text and just asks, okay, for this picture, extract, for example, the biological sex of the person or just tell me, does the person wear glasses and so on. So, yeah, that's a pretty nice trick. I did here you could also use like specific like more specific models that are just like doing for example Like age classification or regression or something like that so That can vary how you do that but yeah, we didn't then have our data frame and then what we did we captured that like in a This process in a like small open source library. You can also look at it's called slice guard And basically, what you do here, you pass in the data frame, you tell it, okay, I want to find similarity or similar clusters, cluster of similar data points on these features, just like in this case, these interpretable features. This is my label, this is the prediction, and this is my metric. And what it will do, it will simply execute this clustering process and the measurement of the metric for you. And later on, you will be able to view an interactive report about this. There are some other things it does. It does some preprocessing, for example, it will recognize that some columns are categorical or normal and stuff like that and does the proper normalization to make the clustering right. Yeah. So now it is running. It will tell us something that some features are categorical and so on. And it will tell us, for example, the overall metric value here is like 0.85 roughly. and yeah now what we can do is just look at an interactive report of this and I hope that works now it looks like that I have to just quickly configure the view such that you see something yeah basically okay nice picture shows that randomly yeah so basically you can then just like have a cluster list here where you also see like the metric so you see okay like our metric was 0.85 for overall data this works worse for example it has like 98 samples and has only 0.72 and this like considered to be a cluster of similar samples according to the attributes we passed in. So if I click on it, I could simply browse through the data and then just try to make a good guess, like what is maybe a similarity. I get some additional hints by the tool, which are, it tries to give me like a feature importance for these like categorical features, says, okay, here's like the glass attribute seems to be pretty like describing for this cluster and also the age group could play a role. So in this case it's like more senior people with glasses which seem to work worse. And of course like if we had the time we would have to look more into it. Like if this is really the pattern here you could for example look at other plots like like a confusion matrix, histograms of certain, for example, labels, predictions, and so on. But it gives you usually a pretty good guess, and you can further look into that. So yeah, and probably I should also mention that the preprocessing part is a tool called Sliceguard. The viewer part is also like a standalone data exploration tool you can also look at. It's called Spotlight. We also put that on GitHub. It's also an open source tool. You can check that out. It's basically what was first developed out of our experiences in customer projects, and now it's like an open source tool we offer. Yeah, let's go back to the presentation. So what were the results, for example, in this data set? When we looked at the attribute view, we saw that, for example, you see the bars are almost equal like in hate, but for example, the senior group has some drop, like older people, more senior people. The model worked like roughly 10% or almost 10% worse. There were also some differences related to the race attribute, which was mostly due to mislabeled data. So for example, there was a lot of non-angry people on pictures that were labeled angry, which is bad. But also a lot of findings where it was more like, for example, people wearing glasses, wearing hats, everything that covers up your face maybe in addition to some race racial features so and this like of course a like example to make it like pretty like obvious how this works but usually i don't apply it on this like human-centric data sets right uh was also a little bit more mentioned in my abstract, but the example was a little bit far from that. Usually, I apply it on, for example, also data sets with audio recordings of cars or, for example, predictive maintenance case. You may want to check, OK, is it working equally well for all my machine instances I have or for all my machine types I have or something like that. This is also similar. And also there, these principles just hold true. And you can pretty much switch out just like the embedding model. All right. The embedding model probably needs a tool like Spotlight, where you can, for example, look at image data and audio data equally, or also at the same time. It's like a multimodal thing. Yeah, you maybe now ask yourself, how can we mitigate these biases? And there, I certainly have to tell you, really highly use case-specific and problem-specific. Typically, there will be more technical measures, like you can simply change the balance of your data. You can probably switch out some labels. But there's also other stuff, like, for example, if some group of data is underrepresented, you maybe have to acquire more data, or you really have to change your labeling process to rule out these labeling issues. You maybe also saw in this data set, right? So that could also be something you want to look at. Yeah, getting started, I have to keep it brief. You saw the tools. Have a look. Maybe it's useful for you. It's those like Sliceguard automatic detection and Spotlight for interactive exploration. Yeah, my conclusion is maybe like, OK, I rarely see people doing this on infrastructure data. If you use the right tools or take advantage of the open source tuning available, For example, also from hanging face and so on, it's pretty easy, or not easy. We've said too much, but it's doable. I feel like in a lot of use cases, it would be better if people cared for this. So think twice if it's necessary to do this in your use case and otherwise profit from the performance gains. It's also not bad. And yeah, of course, now I made a reference to Jenny LMS. Yeah, in this use case, it's also important because we see, for example, our tools getting a lot more traction because those models rely on unstructured data for training and output unstructured data. So I feel like that's also something that will increase in importance somehow. All right, so that's it.

Speaker 2 [24:13]

Thanks a lot, Daniel, for this great talk. And thanks for sending in the questions. The first one is, many studies show the dangers of inferring characteristics like gender, race, and emotion from images. Is it okay to tackle fairness with problematic labeling?

Speaker 1 [24:31]

Yeah, that's a good question. And I feel like there's not so clean answer. I should just be sensitive to like that models can output bad things. Also, for example, the approach I took, like I did like this pre-labeling or automatic labeling of certain attributes. and yeah of course that's why for example I think there has to be like other approaches as well like for example interactively looking at the data like not just blindly trusting stuff so that's what it's basically all about in the first place and that also is true when setting up these like automotive automatic processes like you have to think twice like where there's something really critical and if you can do it automatically or need like maybe manual review.

Speaker 2 [25:21]

best actually to use a combination of various methods, right? And then another question, how do you make sure initial representations are not biased, for example, by stereotypes? Wouldn't this distort the entire evaluation setup?

Speaker 1 [25:37]

Yeah, for sure. I think that's like a pretty similar answer to the one before. It's only like, of course, like you can have no perfect evaluation setup. There's tools that can help it. I see like a lot of people don't even apply it because it's too much like work. So, you can at least try to make it easier for people getting into this, right? But of course, yeah, if you have the resources to do like a complete manual evaluation, you can also take this approach.

Speaker 2 [26:12]

Thank you. And then how to interpret problem clusters in case of huge data where it's not practically easy to visually detect them.

Speaker 1 [26:22]

I think that's pretty much related also to the difference between interpretable features I mentioned and the embedding-based approach. The good thing about interpretable features is that they can help getting pretty concise explanations. For example, if I have a large text document or a cluster of large text documents, it's really hard to find the pattern just by quickly looking at it. And then you can rely on, for example, supporting techniques like topic modeling, where you, for example, get the list of the most prominent topics in those documents to get a better grasp of this large data quickly. So that's also something like a bachelor student of mine recently implemented.

Speaker 2 [27:09]

Next one, how does LAMA multimodal classify someone's biological sex? How reliable do you think this prediction is purely based on an image?

Speaker 1 [27:19]

That's a good question and I can probably not answer that because I didn't evaluate it

Speaker 2 [27:27]

And then the last one here is this fairness strategy only applicable to categorical data

Speaker 1 [27:33]

No, it's not, and that's what I think should be said. There is this interpretable feature way, but usually a lot of times you use the embedding-based way. You will just create model embeddings from some image classification model or something, or an audio classification model. It could also be a text embedding model like Sentence Transformers is a good library for that, for example, and then you can still apply this clustering approach yeah you find the problem clusters you still will have more as we heard in the previous question have more trouble like getting the final interpretation of the like wide model fails but it is applicable to unstructured data pretty pretty well yeah

Speaker 2 [28:22]

Any more questions here in the audience? Anything that we haven't covered here? If not, obviously feel free to reach out to Daniel, ask him directly. And thanks a lot, everyone, for being here.