The Battle of Giants: Causality vs NLP => From Theory to Practice

Speaker 1 [00:02]

Hey. Hi, everybody. Thank you for... Okay, let's figure this out. We good? Okay, we should be good now. Hi, everyone. Once again, thank you for the introduction. We'll be starting. And before we start, I want to ask you two questions. How many of you are familiar with the field of causality? So it's around 40, 50%. Okay. How many of you are familiar with deep learning? Deep learning like most of you. Okay. Perfect. to go. How many of you are familiar with directed acyclic graphs? Okay. Okay. So we will do it for both groups, for those familiar and not familiar. Great. Let's start with an example. So imagine you have a friend. She's a researcher in some field, whatever the field is. And She comes up with an idea that addresses a long-standing problem in her research area. Some novel solution to a problem that people were trying to solve for a long time. And she publishes a paper. Now, let's call her Jane. You have another friend whose name is Jake, and Jake reads Jane's paper, he publishes another paper that builds on top of her discovery, and adds a couple of ideas. But they are not that novel, they are not that revolutionary as her ideas are. And they do it in a very short time period. Now 12 months passes, and Jake's paper gets 573 citations, but Jake's paper gets 977 citations. So it's almost twice more. Now there might be many reasons for this. So maybe they have different affiliations, and Jake's affiliation is better, more popular. Maybe they publish in different journals. Maybe they have different writing styles, and Jake's writing style was more compelling to the readers, or maybe easier to understand. And there might be more factors like this. But one factor, and there's a research like this, a potential factor can be Jane's gender. So there's a research that is showing that papers that have authors who are easily identified as female by their names, for instance, are cited less frequently there than papers by the male counterparts. So now sometimes we might be interested in answering a question if gender could be a factor here. So was it about Jane's work? Something like a journal, something like an affiliation, or maybe about her gender? Or to be precise, the perceived gender, right? Her perceived gender. How people perceived her gender. So we can ask a question like this, holding everything else constant, would Jane have been cited more often had she been perceived as being male? And this question is a counterfactual question. And we ask those questions and similar questions in the field that is called counterfactual fairness. And this is one type of questions that the intersection of causality and NLP is trying to tackle. There are more questions, we'll see other possible questions in a second. And before we go to the meat of this presentation, let me tell you five words about myself. My name is Aleksander Molak. I love traveling with my wife, running, and I enjoy good vegan food. For the last six years, I was helping Fortune 100, 500, and 5,000 companies building machine learning systems. And I'm currently finishing my book on causality and machine learning. So if this is something that is interesting to you, you can scan this QR code and get to a page and subscribe for the updates on the book and also for a newsletter with curated causality-related content. Okay. So how are we going to structure the presentation today? So we'll start with an overview of what is causality. For around 50% of you, this might be a little bit boring. But I'll be asking questions, so maybe you will have an opportunity to have some fun. Then we'll talk about LLMs and causality. What's the connection? And we'll start talking about the large language models that are popular today and give an overview of the intersection of how they work and how causal they might be. Then we'll talk about two streams of causal NLP. Then we'll introduce the architecture we'll work today with. It's called causal birth. And finally, we'll have some time for questions. Okay, so what is causality? This is the definition that Wikipedia gives for causality as understood in physics. is the relationship between causes and effects. What are your thoughts about this definition? Is it a good definition? I'm pretty sure you have opinions. It's a bit large. A bit? It's a bit large. A bit large. So not specific? That's what you mean? OK. Yes? The relationship may not be true. The relationship may not be true, mm-hmm. So what you're saying is that this definition does not say anything important about the relationship itself? Yeah. We are testing the relationship, maybe, but we don't know yet. OK. Great, thank you. Any other voices? Yes, go on. It really sets it apart from a correlation. Does it really set it apart from the correlation? Very good question. Doesn't say anything about direction, because I think intuitively, one would assume causation leads to the effect and not the effect to the cause. So what you say is that this definition is not very precise when it comes to the direction of this relationship. Is it circular because causality and cause are the same? It's circular. I also think so. So circular in definitions means that it defines the term causality using the term cause. That likely is defined by using the term causality. So it's self-referential, yes? More steps to a diagram, right? You would have a treatment, and you could look at that, and then also have several factors that are like some other, not counterfactuals, but co-founders, and so on, and you could look at that, and that would make a difference, and make it more specific. So what you're saying is that it only talks about causes and effects, but doesn't talk about other elements that could be involved in a system and could modify this relationship. Is that correct? Great. Thank you. These are all great ideas and great comments. What do you think what causality is? And this can be about intuitions. Don't worry if it's correct or not. It's more about understanding what are your intuitions. So like around 50% of you said that you have some experience with causality, so some understanding. So let's get some intuitions. Maybe from the people who rose their hands or the other half. Okay, I see you want to say something, don't be shy, no, no, that's really, okay, okay, anyone? Yes? Maybe to a certain extent, quantifying the effect that a cause has on something else, and to the other extent, also learning about these relationships or discovering the structures, even coming up with ideas of what could influence what. that a cause has on something else, and to the other extent, also learning about these relationships, or discovering the structures, and even coming up with ideas of what could influence what. So you said it's about quantifying. So causality is about quantifying how one element influences other element? That's what you're saying? On the one hand. On the one hand. Okay. And on the other hand, maybe coming up with ideas as to what could have an influence on something else. And coming up with ideas what could have influence on something else. So that would be discovering causal mechanisms. And the first part you mentioned would be about just quantifying the strength, yes, strength of effect. Okay. That's great. Do we have any more voices here? here yes mm-hmm finding the real root cause of the effect yeah that's that's that's very good how could we do it how could we operation operationalize those ideas both the free ideas that we that we heard about how could we easily define them and an experiment could help us mm-hmm so experiment so what would be an experiment for instance? An A-B test. So intervening on one parameter in the system and looking what is changing in the system, right? Great, thank you. That's a great intuition and this is very similar to the definition of causality that we will use today as our main definition that comes from Judea Pearl and this definition says it's a very simple definition so maybe even embarrassingly simple but I think it's beautiful because of its simplicity and it states a is a cause of B if B listens to a so this is what you said about a B test we change something we intervene on the system it's a it's it's a targeted intervention so we only intervene on one aspect of the system and we observe how other elements in the system in this case B is changing. Perl also came with came up with something is called the letter of causation for those of those of you who are not familiar with this and he said there are three rungs on this letter. First is association so this This is everything that we quantify statistically, purely statistically. That would be correlations, that would be dependence. By saying correlation, I mean linear correlation. So dependence would be more complex relationships between functions or objects. Then we have intervention. So this is what we have just discussed. So we change, we tweak one element in the system, and we observe how other elements in the system are changing. And then we have counterfactuals, and counterfactuals would be like doing, in a sense, like doing an intervention, but in the imagined world. In Pearl's teaching, each of those rungs has a separate mathematical set of tools that That allows us to talk about those rungs, okay? So we will see that in association, we can use purely statistical language. For interventions, we will need something as do operator, okay? And do operator means that we are changing something. Do operator can be expressed purely statistically sometimes, not always. And finally, for counterfactuals, we have another set of notation. An important aspect of parallel causality are directed acyclic graphs. So these are graphs where you have nodes. You have directed edges. So there's always an arrow, which signifies the direction of influence. And they are acyclic, which means that there are no direct cycles nor indirect cycles, which means that, let's say, this is perceived gender, this is style, this is number of likes for a Reddit post maybe. So there is no path in this graph. There cannot be a path in this graph that goes from perceived gender to itself. It can be through other nodes or directly. If this condition is met, the graph is acyclic. And we will make use of those structures today. Okay. Why model causality? How to increase the number of non-commercial space launches per year. So there are probably many answers to this question. One answer that is a data-driven answer is to increase the number of PhDs in sociology in the U.S. This answer comes from the first rank of the ladder of causation, so association. And it comes from this data that was collected by Tyler Virgen. That's one of the graphs on his website. And as you can see, the correlation between those two, so non-commercial space launches and sociology index rates in the U.S. are very strongly correlated, and this correlation is strong over more than a decade. So you can also see this, you can also see this as a scatterplot. Maybe that's less impressive, I don't know. And so this answer, most people would say that this answer, increasing the number of sociology doctorates in the US, is not causally related with space launches, right? Most of us would say so. Most likely there is a common cause to this. So maybe in those years there was something happening in the global economy that was influencing both variables, or maybe some other mechanism. And now, this leads us to an interesting conclusion. When we think about the state of contemporary of modern NLP, we can often hear people talking about models hallucinating. What does it mean that a model is hallucinating? Language model. I mean, they're inventing stuff. Basically, I ask them to write a function with a specific package, but the package is too new. They don't like, so the model doesn't know it. So it will just invent stuff that isn't there, and obviously wrong. So the model, so we will ask the model to do something, and the model will invent this on the fly. That's right? That's what you're saying? Yeah. But now, one thing is that we don't need large language models to experience hallucinations from a model, and linear regression will be enough for us. Which means that a very simple model that we can use, that many of us probably have used in the past, can actually give us results that are hallucinations. And this comes almost directly from the fact that we often mix the ladders, the ranks in the ladder of causation. So let's see why. Let's imagine a fire alarm system, and in our world we have just three variables, fire, smoke, and alarm. And the alarm will react to smoke, okay? So this is one class of fire alarms. I'm not an expert on this, but I checked it. One class of fire alarms that react to smoke particles. So they have smoke detectors, and they set off when the smoke is detected. Now, symbolically, we could express it this way. So we have fire that causes smoke that causes alarm. The interesting fact about this is that fire and alarm are independent given smoke, which means that if there's smoke, the alarm will set off with some probability. If it's a probabilistic system or if it's fully deterministic without errors, it will just set off. And it doesn't matter if there's fire. So we can imagine a situation where we just open the window, and there is fire outside, the smoke is coming inside, and the alarm starts. Now, this property is not a property of alarms only. We can generalize it, and it will work for any variables. So if you have a variable A causing B and B causing C, if we control for B, A and C will become independent. Because the only dependence or the only variability that C has in common with A goes through B. Do we have any questions here? No, okay. Now this structure is called a chain. It has its name. And now if we think about different structures that are possible with three variables, We'll get two more. So we have a fork where B causes A and C. And what will happen if we control for B? What would be the dependence slash independence pattern here? So in the previous one, we said that if we control for B, A and C will become independent. But they are dependent if we are not controlling for B. So now we have the second pattern which is called a fork. What will happen if we control for B? Will A and C become dependent or independent? They should be dependent, right? They should be dependent if we control or if we don't control. I mean, if we can't talk for b, like, if we create b, we should expect, like, some relationship happening. Mm-hmm. Or something similar to what a and c. Mm-hmm. So what's your name? Michele. Michele. Okay, so what Michele said, that if we change something in b, then we should expect some differences in a and c. That's okay. But if we don't, if we cannot intervene on the system, if we only can observe it. Now, think about statistical control. We observed these three variables, okay? And now, if we look at the correlation, let's say it's everything is linear, very simple. If we look at pairwise correlations, would A and C be dependent or independent without any interventions? Do we have any guesses here? Yes? After controlling for b, they should be independent. After controlling for b, they should be independent. And without controlling for b? Well, it depends on the correlation coefficient. So if there's positive correlations, there might be a scenario where they're canceling out correlations. But in most cases, they would be dependent. Theoretically, they could cancel out, yes. But that would be very rare, but probably in the real world, highly unlikely. Yes, so this is a very good answer. So controlling for B, A and C will become independent, but if we are not controlling for B, we are just looking at pairwise correlations, they will be dependent, most likely, very likely. So now the independence pattern between within chain and within fork is the same, right? So if we control for the middle variable, it renders the variables on the left and the right independent. Now, the third structure is called a collider. What will happen if we control for b here? So collider is very special, because with collider, the pattern is reversed. So we have two causes, A and C, that are causing B. And if you look at the correlation coefficient between A and C, or any other measurement or metric of dependence, A and C will be independent pairwise. But if you control for B, they will become conditionally dependent. OK? You can think of it that controlling for B enables the flow of information through this variable. This is the least intuitive one, I would say. But it is very important in causality. So the difference between these behaviors between collider and both chain and a fork makes causal discovery possible. But it's also crucial in causal inference. Because if we control for a wrong variable in our system, we can actually enable information flow in the system that is not real. And we'll see this in a second in the regression analysis. So what I did here, I took three variables, A, B, and C. I generated those variables according to those three patterns that we have seen, a chain, a fork, in a collider, and I run an analysis of A predicting C, controlling for B. And as you can see, for chains, B is a significant predictor of C, but A is not. So if we haven't control for B, A would be a significant predictor for C. But if we control for it, A becomes independent. The same, as you suggested, the same happens for a fork. B is a significant predictor of C, but A is not. But for Collider, the pattern is different. Both are significant when we control for B. So to see this in practice, let's do some coding. So we'll need five steps for you to open the environment. You can do it in your computer. We'll also do it on the screen, so you don't have to. So, go to bit.ly slash PyData Berlin 23. And this should open a GitHub repo. Then you need to copy the address of this GitHub repo. Something wrong here. Wow. Not so bad. It might be case-sensitive, yeah. Is it? Yeah. Okay, good. Thank you. So this is case-sensitive. You need to say pi with capital data, capital and Berlin capital letters. You have it, and you need to copy the link from the browser when the repository, when you open the repository, and then go to collabresearchgoogle.com, and then choose GitHub, there will be this window, and paste the link and click the search icon, and then it should open the GitHub link Now, I'm trying to... Oh, okay. Oh, that will be difficult, but I only see this here. Okay, does everybody, does anyone need some help? No, okay. Okay, so you can run the first cell. It will install some stuff. Before you do it, you can change. You can make sure that you have GPU. Anyway, cancel. Can you help me, is it GPU now? Not now. This is GPU? Yes. I'm not sure if I turned it off. Okay. So, we can run this code. And we have some stuff, random seed and so on. And then we have data generating processes. Okay. So, let's see if it's executed. It's still executing. Okay, so first we build a DAG like this, and DAG stands for Directed Acyclic Graph. As we said, we have X causing Y, and we have W causing X and Y. So you can see that there's a structure here. What is the structure between X, W, and Y? We have the collider, fork, and chain. This is a fork, yes, exactly. So it's a fork structure. So now, statistically speaking, x and y, if we remove this path, and this is a fork, x and y will be still related for this path, right? Because we said that in a fork, if we don't control for the middle variable, the variables on the ends will be related. So now if we run an analysis and we don't control for w, then our estimate of the influence of x on y will be biased because we will have this influence here on this path, the true path, but also there will be some influence going here. So we need to control for w in order to get an unbiased estimate. So let's generate some data according to this graph and run an analysis, okay? So what we're doing here is we say that w is some random vector, okay? X is caused by w and some noise. And Y is 5 times X plus 2 times W plus some noise, okay? Now we run linear regression. This is Stats Models API. And we can look at the coefficients. We can look at the coefficients. And you will see that the coefficient for X is 4.99, so it's almost 5. The coefficient for w is 2.007 something. So it's almost 2. It's almost exactly, there are almost exactly the coefficients that we designed there. This also shows us that linear regression can be interpreted causally if certain assumptions are met. One of these assumptions is that there is no hidden confounding, which means there are no more variables like w that are causing both x and y. If we are sure about this, linear regression can be interpreted causally. OK, let's see the second data generating process. The second process is different. Now we have x causing y, and both x and y are causing w as well. So what will be the structure between x, w, and y? We had a chain, a collider, and a fork. Again? Maybe a collider, because both are, again, they're both in W. Yes, exactly. That would be a collider. That's a very good answer. So they are both causing W. Now, if in this analysis, because we said before, collider has a conditional independent structure that is reversed comparing to forks and chains. So now, if we control for w in this analysis, let's see what's happening. So first, x is some random noise, okay? y is 5 times x plus some random noise, and w is 5 times x plus 3 times y plus some random noise. so we expect that the coefficient for x will be 5 but it is minus 2 oh this is pretty far away there's also some coefficient for for w which doesn't make sense because w has no influence on y right but if we go to the rank one of the ladder of causation Which is association it makes sense because they share some information. There is some variability in Y and X In W and X and W sorry W and Y that are shared and it's shared because they are related But the direction of this relationship Causally speaking is reversed Then what we see comparing to what we see here So as you can see our linear regression is is hallucinating now, right? there's a there's a strong positive influence of X on Y the coefficient is five but the regression tells us that it's minus two okay do we have any questions here or comments okay those structures do you feel this is clear yeah yeah yes yes yes so so X X is the the treatment and Y is the output and W is some additional information mm-hmm great any more questions no okay so it seems we have clarity regarding this good so let's go back to let's get back to the presentation. And let's talk about large language models. So where are we today with language models and causally speaking? This is one of the examples from my book in chapter on counterfactuals. And it goes, John has coffee sensitivity, he drank coffee in the morning, and his stomach now hurts. Okay? And the question is, would his stomach still hurt if he had not drank the coffee? So in the book, we have a formal treatment of this example, we have a set of equations, and we show how we can retrieve counterfactuals in in certain cases, purely analytically. But here, we just asked Chad GPT to answer this question for us. And so Chad GPT has, no, his stomach would not hurt if he had not drunk coffee, assuming that his coffee sensitivity is the only cause of his stomach ache. If John has a sensitivity to coffee, then drinking coffee can cause adverse reactions in his body, such as stomach pain. Therefore, it is likely that the coffee he drank in the morning is the cause of his current stomach ache. If he had not consumed the coffee, he would not have experienced this specific reaction. What are your thoughts about this? It seems sort of correct, because it's also specified that if this is the only cause, so it seems like given the information in the prompt, this is the less outcome based on the information that the model has available. So you say that this seems to be correct, especially that the model also gives us this restriction that assuming that this is the only cause. I agree with that. I think this is a really impressive answer. And before the chat GPT, that would be, you could only hope for answers like this from large language models that we had previously. But, you know, as they say, hope is not a strategy. This is one of very good answers, very good causal analysis, very good causal answers that I got from ChatGPT when I was testing it early on for causal questions. I was very impressed by this. Here we have a tweet from Judea Pearl, who is also doing tests lately with Chat, with GPT-4 and so on and so on. and he gave, like, a set of examples to the models. Some of the answers were correct, some of them were incorrect. But it seems that many people got very good answers, but this is not always the case. So, now I gave the model the description of the graph that we've just seen, seen, the one with a collider, and I asked if we should control for w. And the model said to determine whether we should control for w in the model that seeks to understand the impact of x on y, we need first examine the relationship between x, y, and w, and then it gives a long explanation, and it concludes, therefore, to understand the impact of x on W and isolate its effects, it's advisable to control for W in the model. And it says this is because the variation in Y that is explained by W is not due to X alone, but also to the noise and the contribution of Y. So this is an argument that seemingly sounds very reasonable, because it says there's some variation in Y that we can see in W, but we cannot see it in X. And so, if we want to make our estimator reduce the variance of our estimator, we should control for W. But the problem is that W also opens the path between X and Y, and as you have seen, it It changes the coefficient essentially in an arbitrary way. So you can get effect where there is no effect, reverse sign, and so on and so on. So this argumentation is incorrect. And there are more examples like this. So there are essentially two papers from the last two weeks. The first one comes from the Microsoft research team from Chongchang and the team that works on NLP and causality there. The second comes from Technische Universität Darmstadt, from Moritz Willig and Matej Zercewicz and their team. And they both go, they both present similar conclusions, and they say that the large language models might give us good causal answers sometimes, but they do not understand causality by default. Moreover, the team from TU Darmstadt proposed a very interesting formalism where they say that the model is actually learning a metastructural causal model strictly from language. So an interesting research was performed by Amit Sharma from Microsoft. He took a famous causal data set that is called Tübingen causal pairs. You have pairs of two variables. And this data set is often used to benchmark causal discovery algorithms. And he passed the questions about all those pairs to ChatGPT. And he got accuracy of 97%, I guess, if I remember correctly. the best state-of-the-art algorithms go up to 83, maybe. Do we know if it was actually trained with the information as well? JORGE FELIPE GONZALEZ- Yeah, so that's a very good question. So the question is if we know if it was trained on this information. So as they say here in this lengthy description, the model likely was trained on this information, right? So it had information about the connection of altitude And temperature maybe took it somewhere from Wikipedia or another source. And the same is most likely true for other pairs. And this is the conclusion that the researchers from Darmstadt do, or make, that the model is learning a causal model of the world, but strictly from language, from statistical dependencies in language. And they call it, they formalize this mathematically in a pretty elegant way, and they call it a meta SCM. They also say that they propose that those foundation models, those large language models, might be exploiting a loophole in the causal hierarchy by Perl. So CHT, it stands for this letter of causation and so on. And this is very closely related to another research by Michaela van der Schaar and her lab, where researchers proposed rank 1.5, which is between association and intervention. So sometimes we might have models that might give us useful causal information, but without theoretical guarantees. And they have a very interesting matrix where they put static and time-sensitive models, and so on and so on. So the conclusion of this is that although those causal Although those large language models can be helpful regarding causality, we cannot get guarantees from them at this stage. And some of the proposals for the future work are to combine those models with implicit or explicit causal modules where a model can use a module like this to get computations about causal relationships and then translate them back to language and return the output. When you say a module, do you mean something like the plug-in stuff like that? Like it outsources something like a library which does that and interacts with the library, the library checks its reasoning, and then the model converts it back to textual language? So the question is, what do we mean by modules? Is it about a library? Yes, exactly. So, so that's one of the, that's one of the solutions proposed by, by Jung Chang and, and, and the team. They say that there might be a, like a causal module, a library, some code that performs causal reasoning for us, and then the model translates it back to natural language and, and returns the output for us. Okay, so the large language, large language models that, that we have today, they are are not causal by default. So what can we do with NLP and causality? There are basically two broad streams of research. The first one is about causality in the real world. So that would be about questions about does altitude cause changes in temperature or the other the way around? Does gender cause articles to be less cited, or more cited, or perceive gender? Anything like this. So any relationship that we have in the real world. And the second stream is about causal influences within the model itself. So what would have happened if we changed this embedding? What would change if we rephrase the question? And so on and so on. So we will focus today on the first one, on the real-world questions. And there are basically four scenarios in this category. first one is text as outcome. An example of this could be we perform an educational program and we want to check if this educational program influenced the readability of the text that participants are producing. So maybe people are writing essays, then we run a program and we want to understand if this program has impacted the readability of those essays produced by the participants. The second scenario is text as confounder. Confounder, which means a common cause, something that can cause the treatment and cause the output, as in the first example that we've seen in the notebook today. And here, we could say how perceived gender affects number of citations. So the example from the beginning. And I built a DAG here, because this is not that intuitive. At least I did not find it that intuitive in the beginning. So here we have perceived gender. Here we have topic of the article that can be operationalized as an abstract. So abstract has information about the topic. And we have a number of citations. And here we have gender, the true gender, not perceived gender. And then we know that gender might affect the distribution of topics, at least in today's society. And it also affects the perceived gender. And so this path, if we don't control for gender here, this path is open. So we could control for gender, or we could control for topic. And any of those nodes will close the confounding path here. And finally, we have not finally, but almost finally, we have text as treatment. So maybe how the way we communicate influences someone's choices. For instance, you could think about a use case where we transcript psychotherapy sessions, And then we want to see how the ways that the therapist is choosing to communicate translates to the outcomes of the client. And that's finally text as mediator. How does perceived gender affect number of likes mediated by writing style? So we have this direct effect here. And here we have also mediated effect. So if we are interested in so-called natural direct effect, so only this one arrow here, we should control for style. But style is hard to define, right? So we can take a model, build embeddings, and then maybe we can try to train or fine-tune these embeddings in a way that will emphasize the relevant parts of the text. OK, so let's talk about an architecture that is called causal BERT. This architecture has been introduced by Victor of H and colleagues in the article called Adapting Text Embeddings for Causal Inference. It's an article from 20 or 21. And the basic idea here is that we take a large language model. In the original work, it was BERT. In the implementation that we'll be using today, it will be distal BERT, so a slightly smaller architecture. Then we perform some modification on this model. We'll talk about this in more detail in a second. And we want to predict 8, so-called. Eighth stands for average treatment effect. And it's defined as a difference between counterfactual outcomes. So counterfactuals, we talked about counterfactuals briefly in the beginning. Two counterfactual outcomes, we say that we assume that we have a binary treatment. And so this is the expected value of the outcome under treatment. and some other variables, relevant variables, minus the expected value of the outcome, given no treatment and the same set of variables. And this equation is for continuous variables, but this can be also for, we can also say this is probability, or this is a distribution. This is a distribution. And do it for categorical or binary variables or anything else we want. Now, what's the problem with this? So we said we take the outcome under treatment and the outcome under no treatment. So what can be problematic here? It's hard to estimate because you have either a treatment or no treatment. So the answer was it's hard to estimate because we either have treatment or we don't. and this is so-called the fundamental problem of causal inference that we only observe in the real world, we only observe just one outcome. So either somebody took appeal or they did not. And as you said, it's hard to estimate because now we essentially need to think how to build this alternative world in a way that is still realistic. So how do they approach this? So they propose a model that builds three different artifacts. Document-level embedding, which is just a regular embedding within the language model. Then we have a mapping from the embedding to propensity score. What is a propensity score? In the binary case, the likelihood that the treatment is assigned? You said that this is the likelihood that the treatment is assigned in the binary case. Yes, that's a good answer. So, this is the probability or sometimes likelihood that given a set of values for X for our variables, the treatment is assigned to this particular unit. And then we have mappings from embedding to the outcome. So, this model learns those three things. And it does so in a way that goes like this. So we have this distal bird Then we learn embeddings Like traditional so this is MLM stands for masked language model. What is a masked language model? moving at random certain words from sequences or from sentences, and then we try to predict these words and fill the gaps here. Yeah, perfect. Thank you. Exactly. So mask language model is this objective that we use in training large language models, when in unsupervised manner, we just remove or mask certain tokens, And then we are trying to predict those masked tokens given the context. And this is how we learn the embedding. And now we use this embedding to estimate the expected value of outcome given treatment and the embedding, the expected value of outcome given no treatment and the embedding, and finally, the propensity score, as you mentioned, which is the probability that the treatment is assigned given the embedding. There are a number of assumptions for this model. For those of you familiar with causality, they won't be probably surprising. So first one is no hidden confounding, which means that we have this graph X, Y, and W causing both of them. We observe all the nodes W, in W. Then we have positivity. is positivity? Mm-hmm. It's the positive probability of being treated given a set of variables, that variables take a given set of values. Thank you. And finally, no leakage slash consistency who knows what its consistency or no leakage I cannot hear you. Can you say? Okay, I could not hear you. I will go to you. All possible values of control are also present in the treatment data set. That's positivity. That's a very good description of positivity. In other words, consistency is slightly different. So it says consistency is a more complex thing, but let's focus on no leakage. This means that the treatment by one unit is not influencing the treatment for another unit, okay? So if we say this is treatment for you, so you get the treatment, but if you did not get a treatment, you are not influenced by her treatment, okay? Do we have any questions here regarding the assumptions? No, okay. Okay, so let's do some coding. Let's go back to the notebook. Okay. And we will use the implementation from read present from Microsoft who implemented causal BERT using PyTorch. The original implementation was in TensorFlow, which I actually wanted to use TensorFlow 2 implementation, but I had some challenges with running it in Colab. So we'll run this PyTorch stuff. This is not available as a library, but as a repository. So you can say, you can type causal bird PyTorch in Google and you should find this repository. Then we'll import the data. It's called test data, and it comes from read-present as well. And as you can see, this data has four columns. So this is text, just the free text. Then we have treatment. We have confounder indicator. And we have outcome. And the outcome is binary. And this implementation only works for binary outcomes. So let's instantiate the model. This wrapper is very high level, so we don't need to do too much low-level coding. And as you can see, it's downloading the model from Hugging Phase. And what's inside? You can see that it's causal bird from pre-trained. And causal bird, if you look into this class, we'll call distal bird. And then use pre-trained distal bird for training. Now we have a bunch of parameters here. So let's go through them. We have batch size, which is batch size. That's pretty self-explanatory. We have G weight, Q weight, and MLM weight. So those weights are weighting the cost function in the training. G is propensity score. This refers back to the original paper when they call propensity score G, which is not very intuitive, maybe, if you're familiar with the literature. But this is what they do. Q is the embedding. Q is the embedding under treatment. So I wanted to go back to the presentation. I'm not sure if that will be easy. OK. OK. So G is this thing. We are trying to estimate the probability of treatment given the embedding. Then Q are those two things, so the probability or the expected value of outcome given treatment and given no treatment. And MLM, objective, this is the traditional objective for the embedding model. And now those, all three of them has their own loss components in the training, okay? So the influence of the embedding, you can decrease it, you can basically weight them. You can balance them. You can make the mask language model objective more important, less important, and so on. So the authors in the original paper, they conclude that using mask language model is beneficial. They also compare it to other approaches like bag of words and so on. And so these are the default values proposed by Prezant in his repository. Now if you want to use this in practice, my recommendation would be, if you can afford this, to split your data set into training and validation and try to find optimal values for those losses empirically. Okay. So now we can train the model. There is a train method in the Kotlbert wrapper. We take texts, we take confounders, treatments, and outcomes, and we define the number of epochs we want to train for. Can I ask a question? JAN-FELIX SCHWARTZMANN- Yes, of course. If you open the data frame where we see one of the text examples, could you explain what the exact task is? JAN-FELIX SCHWARTZMANN- Yes. Because I read the text, and I'm not sure what is actually supposed to be. Yes, yes. So that's a great question. So while it's, let's see if it's training, it's training. Okay, so that's a great question. The question is what exactly the task is. So this data, let's talk about this data for a second. So this data comes from read-present and I don't know its exact source. These texts are about music. So these might be comments from some subreddit. or maybe another online forum. Now, we have some treatment here, which is maybe in a similar data. So first of all, I'm guessing a little bit, because this data set is not documented anywhere. I wanted to bring you another data set from a real world problem, but unfortunately, it was impossible. So that's why we're using this one. But now, similar data sets, very similar data sets, usually are structured like this. So text is some text from maybe an online forum. Then the treatment is maybe if people have specified gender in their profile description. The outcome will be maybe a like or a lack of like. Or I don't know if somebody reposted this, anything like this, some action that people take. And then confounders here will mark the confounding content in the text. OK? Does it answer your question? So confounder, what is the confounder? So confounder is something that influences both the treatment and the outcome. Now if we had this, so let me show you in the diagram, I think it will be easier. The outcome in your example would be a light or something. Yes, I think that's, this is how we should interpret this particular data set, yes. So some action. So for instance, we have gender here. Then we have perceived gender that is coming from your profile description or your icon. And there we have some outcomes, some user action. And then we have text. And maybe this text is influenced by gender as well. Because maybe people differ culturally, or I don't know, whatever, or biologically in some way that influences their choices of the topic. And then we can have maybe other confounders. Maybe there is something more in your profile that is also influencing your perceived gender and the user action. And then we can provide a categorical variable that will mark this type of confounder. OK, maybe in this particular implementation, we can only provide categorical variables. So that would be like 1, 2, or 1, 2, 3, 4, 5, and so on. Discrete values. So what would it mark? What would 1 or 2 mean? So this is arbitrary. It can be anything that you think is a confounder in your setting. anything additional to the text. Like a profile picture, maybe one, and maybe like two. MATIAS NIEMELAER- Maybe a profile picture, yeah. Maybe a profile picture, or maybe something else in your description. So if we had this example, x and y, x causing y, and then we had w. So w doesn't have to be just one variable. It can be a set of variables. So we're not in the text as confounder scenario, but we're in a different one? MATIAS NIEMELAER- No, this is text as confounder, but they might be also different confounders. OK, they might be a broader set of confounders. Make sense? Not really? So you can have two variables that are causing the treatment and the outcome, for instance. Yeah, I get that. In abstract terms, it makes sense. I just wonder, in applied settings, like? JAN-FELIX SCHWARTZMANN- So that could be a profile picture, for instance. I think this is a very good example. So you have text, but you also have a profile picture. So profile picture will influence your perceived gender because people will say, like, hey, it's a girl on the picture or it's a guy on the picture. And it can also influence the outcome because maybe people find you attractive, right? And they will be more likely to click a like under your comment or repost it. But the text of your comment also is also important. I guess what I'm a bit confused about is that I think I thought finding or confirming these causal relationships is what we want to find out. But deciding this particular thing is like a confounder is where you're kind of assuming . So that's a very good point. Now, in causality, we usually talk about two broad categories of tasks. So one is causal discovery, which is about discovering, hopefully, as automatically as possible, causal relationships between variables. And then we have causal inference, which is just quantifying those. So everything we talk about today here is about causal inference. So we had this idea, one of the assumptions was no hidden confounding. This might be a difficult assumption, but if you assume that you know all the confounding variables, well, these methods will be helpful for you. If you say, like, I have no idea, maybe there's like 500 other confounders, then the results of this will be highly risky to use them in practice. Does it answer your question? Yeah. Yeah? You've got a question online. Yeah. So in reality, we are going to have a lot of confounders. So should we have to include all of them in the model? JAN-FELIX SCHWARTZMANN- So the question online was, in reality, we'll have a lot of confounders. So should we include all of them in the model? That's a very good question. Often, we are not able to exclude all the confounders. If we want those methods to work and have all the guarantees that they might have some of them have more some of them have less then we should include all confounders in the in the model now from my practice in the industry I can tell you that usually the process looks like this that you start with some model with your best expert knowledge that you can gather all over the place then you build a model then you generate some data from from this model some predictions from this model and then you compare this to some new data and then you iterate and you improve the model so this is usually the process that that that I see in the industry and that and I also recommend because in many cases especially when we talk about like open-ended complex systems like I don't know social media whatever social topics and so on it will be very very hard to control for all confounders now another way to work with those methods is to run experiments so if you are able to run an experiment an experiment will actually will basically cut where is the treatment here so the experiment you can think about running an experiment as cutting off all the arrows that are coming into the treatment in the graph so we are removing this arrow by by running an experiment and by experiment i i mean um randomly assigning perceived gender in this case your treatment and now your data is unconfounded if you if you run an experiment So, hopefully, that answers the question. Do we have any more? Right now, not yet. Not yet. Okay, good. Okay, let's get back to the model and see... and see if it has trained. Okay, it's finishing. So put two epochs here, might not be enough or it might be enough. Okay, so the training has concluded. So now we can take the model and it has a method called ATE, Average Treatment Effect. We pass the confounders in text and this will return the average treatment effect for our set. So we are doing an inference task, not a prediction task. We are checking the average treatment effect on the same data set. You got another question? So you mentioned that the four scenarios of text as outcome, text as confounder, et cetera, all these four scenarios, and what is this? So this is text as confounder as well. This is text as confounder. And this method can be also used for text as mediator. But as far as I understand, this particular implementation, it only works for text as confounder. Could you give an example again, once again, for text as confounder? Text is confounding? Or if I think of your diagram, I wonder, would the name of the author be the text that is confounding, or? No, that would be, oh, just let me, okay. So now, text is an abstract of, I don't know, abstract of a paper, okay? And there is this path between gender and abstract. that that opens the path between perceived gender and a number of citations so if we control for for topic we'll close this path and now only this path remains which means that we now have the effect of perceived gender of numbers of citations but if we don't control for this and we don't control for this and I assume that the real gender is unobserved here then this path is open. And this will change the estimate for this effect here. So, any time you can think of text influencing like your treatment and outcome, this will be text as confounder. Why my treatment? Yeah, why your treatment? Why would it influence my treatment? Well, it depends what your treatment is. Like, any time it happens, it will be a confounder. Now, of course, it depends on what your treatment is. But it might affect your treatment. I'll try to think about another example when this is happening, and we'll try to give it to you. If not now, we can try to talk after the talk. Yes? I don't know, I find that a bit confusing. I think the terminology of treatment, it doesn't mean that you need to intervene to change it. It just means that it's just an observed variable. Maybe that's the thing. Yeah. It's difficult to keep, I don't know. Yeah, that's a very good comment. So the comment was, for the people online, that the treatment doesn't have to be, I don't know, a drug or something or something. That can be just one of the observed variables that you are interested in the effect of this observed variable on another observed variable. So there's a strong connotation about the treatment being something that we manipulate, or like something hard, and so on. So this might be useful for some people. Thank you. That's a very good comment. OK. And so we have it. And the average treatment effect here is minus 0.22. The thing is that, and this is pretty, the thing is that this is pretty stable. I run this model like, I don't know, like a dozen of times and it was always very similar to this. So, maybe minus 25, minus 24, minus 22, which is a very good sign. Okay, so let's go, let's get back to the presentation. Okay, so a couple of conclusions from the paper. Language modeling helps. Which means that the authors have tried also using different approaches. If I remember correctly, they're using Bag of Words and this other stuff. And some other stuff, I'm sorry. And language modeling in terms of using large language models helps. In particular, when we adapt those embeddings to learn this more complex objective function that models counterfactual outcomes and the propensity score. Also another thing is that custom pre-training helps. So they notice that not only, let's take a step back. What happens when we train this model? We are in so-called fine-tuning regime, which means that we are taking pre-trained embeddings and then we are learning our objective function starting from those pre-training embeddings. What the offers suggested here is that before we do that, a good thing to do is to take your data and run a couple of runs of unsupervised pre-training using this data. And they noticed that this also improves the estimates. Like, in the normal, like, regimen for LLMs? Again, again? Like, to run the supervisor as a, like, normal LLMs do? Yes. Like, just to predict the next token, blah, blah, blah. Yes, exactly. So the question was, should we do it in the same way as if we would train a large language model from scratch using this masked language model objective? And the answer is yes. so that's exactly what they say you start with the pre-trained model but you still run just the mask language mask language objective for a couple of runs using your data set and then you do fine-tuning using the method with the complex loss function that we that we've seen in action today do they explain why Yes, my intuition would be that it is necessary to give the model information on what you are writing, like what your, the actual language, what is being used, like the text. So the question is... So you have the text, like in the memory of the language of the LLMs, then you ask it how does, like, the inference work, like, and stuff like that. So you have to have the information. Yeah, so the question is why this is the case. So I think this is something that we often see in the industry, that when you adapt a model to a new task, for instance, I used to work at Iron Scales, which is a cybersecurity company, and we're training large language models to predict certain qualities of emails. Now you can imagine that email language might be slightly different than general language on the Internet. The distribution of emails might be slightly different than the distribution of overall language on the Internet. And then not only fine-tuning this model for our classification task, but also training this on this email language in particular before we train the model on a task might be beneficial because it makes the distribution that the model learns more similar to your domain distribution. Does that answer your question? Yes. Yeah? Perfect. Have you ever heard of the, like, theory and interpretation of LLMs model as simulators? Again, again? Like, have you perhaps maybe read about the theory or interpretation that LLMs are actually simulators underneath the hood? So the question is about the theory that LLMs are simulators. No, I haven't read a paper like this, but I can imagine that you could probably say this about any generative model to an extent the idea would be that like this works because the actual in order to predict correctly the next token you have to have like the uh the better the word model you have the better your token prediction so eventually you're able like to predict personas we say like you know when you break the model and they start like going on a ramble stuff like that but also probably the same reason why you need the step before because like Actually, now the model has enough information to simulate what state actually makes the language to the mails. And then we apply the encoder inference and the encoder read. That's why it really helps. So the comment was that if we think about large language models as simulators, the better representation of language we learn, the better outcome we can expect. And we can also learn more specific things like personas, you said, right? And so on and so on. And this is likely related also to the observation that the authors had here. OK, so we conclude here. And if you want to learn more about causal inference and natural language processing, this is a great paper by Amit Feder from Technion in Haifa and his colleagues from all over the place, as you can see. And this is a great overview of work on the intersection of causality and NLP. I also have a QR code for you here, so you can just make it like this. Yeah? Before we conclude, I think you have quite several online questions. OK, we have more questions. Good. So how often do you test for different but valid DAG configurations that explains the observed data. Once again, how often? How often do you test the different DAG configurations? The different what? DAG configurations. DAG configurations. DAG configurations that explain the observed data. OK, so the question is, how often we test for the structure of DAG? So DAG is the directed acyclic graph. That's a very general question. So in this sense, it's hard to answer, but it's good to test for this, right? Maybe let me take a step back. There are different use cases. So far, we are talking about like social models that are trying to model some social phenomena. And these are often open-ended and complex systems, and in this sense, it's very difficult or might be very difficult to exclude all the hidden confounding or control for all the hidden confounding. But not all systems are like this. So, for instance, there are very successful implementations of causal models in the industry where you have, for instance, systems like production lines that are virtually almost closed. And so the influences from the outside are very limited. and often those systems also are well documented when it comes to how information in those systems flow. So this might be a very, very different case. But if I had to give you a general answer, well, I would test to the extent where I feel this model makes sense and fits the data well. that's i know it's broad but i guess that's the best what i can give in in the general sense okay i guess there are more specific questions here so there are some studies stating that propensity score performed poorly to decrease the imbalance between control and test do you also use alternatives such as So it says that there are studies that show that propensity score can perform poorly. I'm not sure which studies the person asking the question is referring to. There are certainly studies that are showing that propensity scores are a very bad choice when it comes to matching. So matching is another type of an estimator that can be used in causality. But matching on propensity scores can lead to very detrimental results. So this should not be done. In other cases, I don't know. Perhaps there are studies that also show some problems with propensity scores in a broader sense. But I'm not aware of those. Here, in this work, people usually work with propensity scores, if anything. Okay. And I guess we have one last question. So in the case of text as outcome of treatment scenarios, how would you quantify the ATE since the text is not a numerical variable? That's a great question. So the question is, in case of text as treatment or outcome, how would you quantify ATE, given that text is not a numerical variable? So in general, we've seen the formula for ATE that is for binary variables, and in our case, the outcome is also binary, but it doesn't have to be the case. So we can have continuous treatments and continuous outcomes as well. that's the first thing the second thing is that if we treat text as as as outcome we might sometimes just say hey if there was some feature in the text right so maybe if certain set of features in our embedding space has certain values then we just say okay this is one otherwise it's zero so it can be binarized sometimes and and also it can be a continuous variable. When it comes to the treatment, it can also be a continuous variable. Moreover, it can be a multivariate variable or multidimensional variable. So we can have an embedding space, perhaps cast down to some lower dimensional space. Like, I don't know, maybe we have eight dimensions. And these eight dimensions, continuous eight dimensions, define our treatment and this is also possible and this can be done so there are models that that can take multiple or multi-dimensional continuous treatments and this could be also generalized to natural language although i don't know papers like this i think at this stage and you got requests um about like could you send us slides and some papers to the discord channel For example, in this A5, A6. For how long will this code be open? I'm not sure. I think it stays open. Okay, so if it stays open, I will do it later today. Otherwise, I will add some more papers. So there is one paper, but I will add more papers to the repository that we used for the talk. Okay? I don't see anything coming, so that's it. So as an outcome, you will get a lot of applause for the ending. Great, thank you so much.