Black Hole Stars: An Astronomical Mystery (Mostly) Solved with NumPyro and JAX

Speaker 1 [00:02]

Yeah, thanks so much to the organizers for having me here. I see that I have lured a good number of you with the sci-fi-sounding title. That's good. But seriously, I think I really want to try and take this as an opportunity to let you, the public, know about some of the cutting-edge research that's going on in astronomy right now and kind of what's being enabled by Python and how we as scientists are using it. And I'm also going to try, where possible, to endeavor to use real images, as real as they can be, from astronomy. I'm going to try and cut back on artist renditions and actually show you kind of plots from papers as well, at least to try and give you a sense of when we look at the data, how do we actually interact with it and how we look with it. Okay. So, let's talk about what we're going to do today. This is really going to be a story about discovery, right? Recently, a couple years ago, NASA launched its flagship mission, the James Webb Space Telescope in collaboration with European and Canadian partners, and this is really going to be a story about something that we found that was new when we launched a $12 billion mission into space. And I'm also going to talk about how we solved the problem with Python, just kind of fun. And maybe it'll be something that you can take away to your own work too. But if not, then you can stay for the cool astronomy. And so really the roadmap is we're going to take a couple slides to go through some astronomy 101 just so we're all on the same page. I promise it'll be very brief and hopefully not too boring. Then I'm going to talk about little red dots. These are the things that have been newly discovered with the James Webb Space Telescope, and you may have even read about them in the news. Then we're going to take that brief detour, not really a detour, but on the way of understanding this new phenomena, we're going to talk about this modeling problem that I mentioned, and then I promise we will talk about black hole stars. Okay, good? Great, okay. Astronomy 101, what is a star? Okay, so raise your hand if you have a good idea of what a star sort of is. This is great. We'll skip this slide. No, I'm kidding. So stars are basically balls of hydrogen gas. And in a nutshell, there is an inward pressure from gravity because of all this material there. And the only reason the star doesn't collapse and just disappear is because you have an outward pressure from fusion in the central core. And basically what we're seeing here is a reprocessing of something like millions of degrees Kelvin in the central core where fusion is happening until all of that reprocesses, reprocesses out, and you get a ball of gas that to the outside just kind of looks like a uniform 5,000 Kelvin temperature. And actually, again, this is a diagram. This is an actual image of a star that we have resolved. This is Betelgeuse, where what you're seeing, the fuzz here, is not a blurring from the atmosphere or instruments. This is actually resolving the size of the stars. Probably one of the few stars we can actually do this for. It's pretty cool, but stars are usually so far away, we can't resolve them. Anyway, so stars. So now we're going to talk about galaxies, which is basically just taking stars, well, about 100 billion of them, sticking them all in one place. Here's a galaxy here. That's another galaxy right there. And we can see that actually just from this example of two galaxies right here, this is the Whirlpool galaxy and its companion, you already have a pretty wide diversity in not only the shape but also color of these objects. This one here is blue with some yellow in the middle and some spots of pink. This one's mostly yellow and sort of roundish where this one's spiral. And I really want to convey to you the fact that, you know, if you think about our universe, or maybe the way that I think about our universe, is that it is the basic building block of our universe, our galaxies, and that our universe is full of them. This is an image taken by pointing the Hubble Space Telescope at a dark patch in the sky, leaving the shutter open, and every single point of light in this galaxy that you can see is a galaxy except for, like, three. And this is what you... If everyone will do me a favor and stick out their hand, open their pinky, look at their pinky nail, and cut it in half, that's about the size of this image on the sky. So you move over again, boom. So our universe is full of galaxies, and my research is trying to understand the story of galaxies, how we go from galaxies 14 billion years ago, which is the age of our universe, all the way until the present day, or where some of these more well-formed galaxies might live. Okay. One of the really essential parts of that story, and that I specifically specialize in, are quasars. And now I'm not going to prove this to you, and you're going to take my word for it, but I'm going to say that all massive galaxies, asterisk maybe, host a supermassive black hole. This is actually an image of a black hole taken by the Event Horizon Telescope. Or I should say, of course, we cannot take a picture of a black hole itself. The black hole does not emit any light. But we can take a picture of the gas around the black hole, which does emit light. And that's what you're seeing here. And this exists at the center of every galaxy. It's about 100 billion, sorry, it's about 100, sorry, it's about a million times the mass of our sun. So I don't want anyone to come away from this thinking our galaxy orbits around the black hole. The black hole is relatively small compared to the 100 billion stars of our galaxy. Yet, I do want to impress upon you that this is actually the black hole in the center of our own Milky Way, and that's why we were able to get a picture of it. You see here, there's a little bit of light around it, and it's actually a very, very, what we might call a dormant black hole, In that light, or sorry, the gases that are funneling into the black hole, it's a relatively small quantity, and so then it releases very little energy. As a material falls into the black hole, before it crosses the infrared horizon, a lot of energy can be released. Now, we're very lucky that ours is dormant, because here's what a galaxy looks like when you turn on a black hole to the max, and we call this a quasar, where the black hole is undergoing rapid accretion and releasing massive amounts of energy. You're going to see a galaxy, sort of diagonally, we're looking at a spiral edge on, and I want you to take a look at what's happening to it. These are two jets of material that are being blown out on galaxy-wide scales simply by this small, compact object at the center of the galaxy. You are tearing holes in a galaxy because of the activity of its supermassive black hole. I'm talking a little bit too much about quasars because I do love them, but I really want to get across that you should see this as a really powerful force for shaping galaxies over cosmic time, and so we really think they're critical for galaxy evolution. However, one big open question in astronomy that we don't know the answer to, we have a lot of ideas, but we do not know the answer to, is how you make black holes this big at the center of galaxies. We do not know. We know how you make little black holes, we do not know how you make ones this big, and how you put them at the center of galaxies. We just don't know. We have ideas. So we'd like to know how they're made. And one last little thing I'll say is that if you look at the Milky Way in X-rays, and I really want to show you some more real data, is there's very good evidence that there's these bubbles, essentially, above. This Milky Way is here. Below and above the Milky Way, you actually see these bubbles being carved out. And that sort of look like remnants of what might have been a quasar in our own galaxy. So you can kind of take this, rotate it. it. You can kind of see the similarities. Of course, this is much harder to do. It's in our own Milky Way. So you're sort of stuck with the problem of trying to understand something you live in. Anyway. Okay. That's it. Astronomy 101 done. Okay. And you all brought the homework, right? Because I am... No? Okay. So I want to talk about this story of discovery. We launched the James Webb Space Telescope. This is the last time we ever saw it as we launched it off of an Ariane 5. This was actually done by the European Space Agency. We're launching it to its eventual destination about three times further than the moon. I don't remember that. But we launch it, and then it unfolds in space. It's actually very compact right now until, again, this is an artist's impression, we get the James Webb Space Telescope. It's about the size of a three-story building. From tip to tip, it's about a tennis court. It's about three stories tall. Massive. Had to unfold in space. And I really want to communicate to you that this was the flagship mission, and it currently represents our best telescope in space. Our best telescope, really, period. And so, for example, just to kind of show you, it's really remarkable in a way that I could spend hours talking about, but just to show you, this is the best we could do before, and this took, like, hours, and this is what we can do in minutes. Wow. Cool. Okay. But, I mean, really, that represents a fraction of its capabilities. So why would you ever launch $12 billion of metal and other things into space? Well, because there are questions that astronomers want to answer. For example, that question of where do black holes come from? That was one of the questions that we thought we might be able to shed light on with the James Webb Space Telescope. There's a myriad of other questions as well, everything from understanding planets outside our own solar system to galaxies, how stars are made, et cetera, et cetera. But these are all questions we knew we had going into launching a flagship mission. But I really want to try and convince you that the reason that it's worth making these investments into the unknown is because we want to search for the things that we don't even know we don't know. These are the unknown unknowns. The surprises that happen whenever we build an instrument or a research facility or in astronomy, a new telescope that push the frontier, we always find things that we didn't even know we didn't know. And that's where the discovery is really exciting. And so I want to show you these little red dots. Essentially, every time we pointed the telescope somewhere and took a sufficiently deep image, we found things that we know are at the distances of galaxies, so they're incredibly far away, and as the name might imply, they look like little red dots. And what I want to get across to you is that there are a lot of them. So here's a list. Oh, sorry, no, I think we have a more updated list. Sorry, I think we have a couple more. I really want to get across to you that no matter whenever we point the telescope, we see these things, and we did not know about them before. They are ubiquitous in the universe. And a couple of things that we have figured out about them is that essentially, before the James Webb, we could really probe the universe from now until if our universe is 14 billion years old, we could probe back about 12, 12 and a half billion years. There's some wiggle room in that. That last billion years is really that reason we launched the Webb Telescope to really understand the formative time in a galaxy's life, right? If you think about a child, going from the evolution and the growth they do in the first year of their life is unlike anything else in the rest of their life. And this is really analogous to galaxies. We're understanding that imprint of everything that happens in that first year is the missing link to understand them. So I'm not going to talk about everything we've learned from the James Webb Telescope, but I do want to talk about the fact that this really was an exciting discovery that that got reported in the news because our early theories were, well, they break everything we know about astronomy. We think that they don't fit into our models of the universe. We were very naive back then. They don't really break our universe too much, but we do now have a more exciting way of describing, in my opinion, that they don't necessarily break the universe, but they potentially have given us a lens onto something entirely new and maybe help solve that question of where black holes come from. Okay, so where do we go from here? All right, now we're going to take a little detour from the science and the nice pictures. I'm going to show you something that's a little bit more what we have in our papers, which is going to be great and we're all going to enjoy it. And so the thing that we do next is instead of taking images, we actually get a lot more information from a technique known as spectroscopy. And very briefly, for the Pink Floyd fans out there, all we want to do is take the light that we would have collected in the image and we sacrifice all of the spatial information that we had. that we had. We no longer see what it looks like, which is fine. The little red dots are essentially unresolved, in the sense that we actually don't see how big they are. Then we split it into its component colors, which can tell us a lot. Again, I could spend hours talking about this, and I won't. And from now on, I'm going to show you plots that kind of look like this, where we have energy on the y-axis, essentially how bright it is, and at what colors it's bright. And on the left, we're going to have blue. And on the right, we're going to have red. And normally, I would say, well, actually, this doesn't super map to what we know. But actually, in this case, it's actually relatively close to blue and red that we actually see with our eyes. And so I just want to show this plot that really was one of the first, one of the really impactful spectra, again, a data set derived from spectroscopy, of a little red dot. It's codenamed The Cliff because it looks so different from every galaxy that came before. So on the next slide, all of the colored lines are galaxies that are sort of normal. And the black line is going to be the little red dot. Wow. Now, to an astronomer, this is extraordinarily striking. And so that's why I want to show it to you. This is essentially the known limit of the parameter space that has come before. And we finally found something that is more extreme in essentially every dimension. So again, I think you guys can actually handle this. You can handle the data. What's also pretty remarkable about them is that you'll notice that actually a lot of these galaxies have these lines in them, these big, big bursts. This actually comes from elements themselves. So most of these are produced by hydrogen gas. So we can actually probe the physical conditions of hydrogen in these galaxies by zooming in on these lines. And that's exactly what we're going to do. One thing that's fantastic about these lines is they're incredibly unique for galaxies. Not necessarily unique in astronomy, but unique for galaxies. Because they not only show emission, a bright line going up, but they also show absorption, a line going down, and a broad line, which can imply very fast-moving gas. And so you get something that looks like this. Up, down, broad. Great. And we see this all the time in little red dots in different ratios. So here, this isn't two absorptions. It's just one absorption that kind of, because the line is coming up in the middle. But we see down, up, broad. And actually, this was a paper that came out a couple of days ago with the most extreme version of this. And again, this is an astronomer's wow moment. Wow. The amount of absorption in this line is ludicrous. It's really remarkable, and I wish I could spend the time really getting into this with you guys. But anyway, I want to communicate to you, you don't have to necessarily appreciate it to the same level that I do, but that understanding these kind of graphs and modeling these lines is imperative to us understanding the physical conditions of what's going on. However, we have run into a problem in our data known as undersampling, in that much of the data that comes from these telescopes is undersampled. And what I mean by that is that the size of our pixels is bigger than how fast the function changes. And I'll show you what I mean by in a second. And there's good reasons we did this. But when you want to fit a model to data, like we have maybe done in our y equals mx plus b, doing this in, I don't know enough about the German education system, whatever you do it, you ideally have x, which is the points that you observe at, as well as y, which you make your predictions and you compare that to the data you have observed. And I'm going to make the case that for this kind of data, we actually have two kinds of ways of thinking about it. One is evaluation, and one is integration. So if we have our true model here, which is that line I showed you before, the way our detectors actually work is that at each interval, we measure the average in the function. However, when we evaluate our model, in this case, it's a normal distribution, which we're all very familiar with, if we were to fit y to x, we would fit at each of the points. We would evaluate our model at the points and compare it to the data. And so you get something that looks like this. Here's the integrated curves that kind of mimic what the detector is doing, where the pixels lie. And the pink dots are what you actually evaluate the model to be. Now, this is all well and good until we make our pixel size big, especially compared to the size of the function. Can anyone see the problem here? Disaster. And so when we do inference on this, we actually get the wrong answer. So the blue curve is fitting to the blue lines. That's actually mimicking the whole process. The pink curve is fitting to the pink points. And if we compare it to the actual values that we put in, so the width, for example, the width of the line, we're off by about 20%. This means that any physical intuition that we gain from this is off by 20%. Not good. Or maybe even squared. Whatever 20% squared is. OK, so how do we solve this? This is not a new problem. We've dealt with it in the past. One of the most traditional ways of solving this is supersampling. So essentially, you take your model, and instead of evaluating at each of your known data points, you do it 100 times or however many times in between. And then you take the average. This is basically Riemann integration. But this can increase your memory footprint and your compute time by the number of super samples that you do. Not good, especially because you can maybe optimize this a little further, yada, yada. But this makes it noticeably slower. And this is really important because speed for us really matters. Because we're not just trying to optimize to get to a best solution. We actually usually want to do inference, which means many, many more likelihood calls in order to really get an understanding of the error distribution for any of the parameters we measure. As a scientist, we don't want to just give you the answer. We want to tell you how sure we are of the answer. And so we end up doing a lot more Monte Carlo methods, which are a lot more compute usage. So how do we solve this? Very briefly, I built a Python tool, which is very much dedicated to astronomy. But maybe if you are interested in taking some of the techniques, it'll be of interest to you. Basically, it's called Unite, or the Unified Line Integration turbo engine. In astronomy, we love our acronyms. So the goal was, essentially, all we're doing here is we're not going to supersample. Because we can analytically describe all of our functions, we just compute the analytic integrals. And so the only overhead here is you have to do the math on paper. And especially not just for a normal distribution, which we all could probably write down the integral for, or at least Google. But we also have to do the integrals for all of the analytic functions a family of different profiles that we see in these lines. But don't worry. We've done the math for you. Great. But then all we do, please don't look at this, and then all we do is we basically build this system of doing the integrals fast and efficiently in JAX. We use NumPyro, which can leverage those immediately to do the inference. And so we get a very performant, accurate, because now we're actually not super sampling is still an approximation to the actual process. Here we're replicating it exactly. And a scientific in that in the sense that we get the inference for free here. Way of approaching fitting these lines. What's also really nice is that I haven't really shown you this, but when we take these spectra, we sometimes take a low resolution one and a high resolution one. The low resolution one has high signal-less noise, but of course you lose resolution. High resolution one, you have much more data, but it's very noisy. And so we can actually generalize this model. This is one thing that was very helpful to actually fit different instruments simultaneously. So I'll show you an example of that. This is a real-world example that we actually did. And the question was, can we find objects with those big, broad lines? Don't worry about the absorption for now. And so I'm going to show you a real spectrum. These two lines here are for reference. This is the line we're looking for a broad line in. All right, let's go raise our hands. Who thinks that there is a secondary, a normal distribution component underneath this line, same one in both, that we can detect? Who thinks there's a broad line there? Oh, right, you're going to be right. Morgan, of course. He's cheating. It might help for me to draw the line if you fit it with just a narrow component under there. Maybe now we think it's a little bit more sure. But what really hits home and what really makes us much sure is that if you were to fit two models, one broad, one narrow, to this data or this data alone, you'd only be about 95% sure that a broad line is there. That is not good enough for scientists. We don't like that, or at least for astronomers. We like to be a lot more sure. But if we combine the spectra, we actually get about a one in a million chance that this is rare chance because of the fact that we're leveraging more data and we can combine it in the same inference model. OK, just going to, so with Unite we get to be more confident. Unite has also been used a couple of times to do some other cool things. It's fitting more little red dots here. Actually, this is the most distant galaxy ever known. And the way that we actually know precisely what its distance is is because we were able to fit its lines with code like this. And so currently, the most distant galaxy we know of was fit with this code as well. We've managed to fit some cool extreme little red dots here. You can kind of see them. It's kind of fun stuff. And then here's a big library of where it was used to look for those broad lines across a wide variety of different spectra. OK, so what are they? I'm going to try and convince you that they are not things we've understood before by showing you spectra that may make no sense to you. But I will try to convince you, or at least have you believe me, that they are not, for example, just galaxies dominated by stars. One way that I'll convince you of this, this is the Andromeda galaxy. I have put the size, or at least the upper limit of the size, of a little red dot on here. Can anyone see it? Those in the front row have an advantage. It's right here. So they're about that big. Now, galaxies in the early universe can be much smaller, so this is a little unfair of a comparison. However, they're very, very small. Again, we've never seen how small they are. We only know the upper limit. And they actually had to blow it up a little bit so it would be easier to find. Also, if you try to explain their spectra with stars, here in black, we have the model. In gray, we have the data. And if we go, if we really zoom in, we see that there's this discontinuity here, which we really should be able to fit. And so again, stars don't make sense. They can't be galaxies with stars. Here's another example really showing the same thing. Here's the data. There's the model. And we see that really around this specific region, they break. There's a lot of good reasons why that's important. These are sort of known galaxies in one measurement of, basically this is showing the same thing again, but they of course are much more extreme. Taking my word for a lot of this, guys, I appreciate it. So are they galaxies dominated by stars? Too small, can't get the right shape, and also broad lines are not common in galaxies. Aha, then maybe they are quasars. Quasars are extraordinarily small and they do have broad lines. However, here in black are the data points and in yellow is the best fit for a quasar, and you see that especially at the reddest colors, they really don't match. So there's very good evidence to suggest they are not typically quasars. I have convinced you all, even though I can't convince all of my peers in the field, that they in fact, I'm teasing, I'm teasing, they are in fact not associated with typical quasars. However, myself, as well as many others in the community, and my peers have noticed that there are similarities between the spectra, sort of shown here in black, and star-like spectra. So this is the spectrum of a star, and I'm going to simply claim, and we can argue about this afterwards, or I can try and convince you more, that, in fact, if we recap what we know, that there are, one, red extragalactic phenomena with unique observables never before seen, for example, these broad emission lines and absorption lines, spectral breaks which is essentially that mismatch I didn't mention to you but it's sort of implicit the fact that they were discovered with Webb is that they are very easy to find in that first billion years of the universe but they disappear afterwards and they're not described by typical galaxies or quasars I haven't proven this to you because it would be a little tough but they share a lot of similarities with stars but of course they're as bright as a galaxy so they can't just be a star and so the idea that we have been putting more and more effort towards and that is seeming to bear the most fruit is that these objects These are essentially massive stars, but instead of being held up by the power of fusion at the center, they are held up by the power of black hole accretion. We don't obviously have a real picture of any of these, or at least this close, but this is sort of the artist's impression of what's happening. At the center, you have a black hole, which is accreting extremely rapidly and blowing energy out, just like we saw in a galactic scale. However, it is being constrained by the fact that it has an immense, and I'm not talking about a normal star. I mean, it's not even a star, but I'm talking about a cloud of gas around it, an envelope of gas around it, a cocoon of gas around it that is so heavy that it stops this pressure from the middle from escaping, and you end up with something at least in equilibrium for a little bit. And actually, if this is to be true, this could be one of the ways that you help explain the problem of where these supermassive black holes come from in the first place. Conclusions. Okay, great. And then we can do questions, and of course I'll be here today, tomorrow, and you can come find me and we can talk about astronomy. The James Webb Space Telescope is amazing, and it is really revealing to us, not only answering questions that we knew we had before, but it is shedding light on questions we didn't even know we had. And the little red dots are the best example of this. They really do break the mold of what we knew. There's good evidence to suggest that they may be something similar to black hole stars, a massive massive system that behaves like a star, but with a compact object in the middle providing a different source of energy, not from fusion. Could they help us answer the question of how they formed the first supermassive black holes? We don't know. And I want to really say that in astronomy, we really love using Python, and so there's a lot of applications. So there really is a bright future in astronomy, pun intended, with Python. But I think that's my time, and I'd love to hear from you guys.

Speaker 2 [25:50]

Thank you Rafael. There's time for questions. So please go drop in your questions at talks.pycon.de. So there's one I'm going to read out to you. Even though you said you're getting exact solutions, I was wondering if you're using variational inference in NumPyro. And if so, do you have any advice for convergence diagnostics?

Speaker 1 [26:12]

It depends what you're trying to do. I think, in general, we still like MCMC and NUTS as our really final, really not approximating our press tiers, but ideally sampling our press tiers as well as possible. We do sometimes use SVI to maybe initialize and get a little bit closer to a global minimum so we can understand at least the local behavior. But then really the gold standard for doing model comparison in astronomy is, unfortunately, nested sampling, which is a nightmare and we definitely can't do that on everything. But the fact that we are able to run our likelihoods much faster makes things like nested sampling actually really possible. So for whoever answered that, I hope that answers the question.

Speaker 2 [26:49]

Okay. There's a popular one. Are they really just called little red dots?

Speaker 1 [26:55]

Yes, and pretty much everyone in astronomy hates that. So it's a very generic name. The problem is, again, we like to name things. It would be great to name things after we know what they are. But the problem is we saw them first, and so the only descriptor we can have is a physical one. Maybe when we all agree on what they are, we can call them something like black hole star. But, for example, quasar, the name quasar, does mean quasi-star because the first time we observed them, they looked like stars. and someone was like, that kind of looks like a star. I guess it's a quasi-star. They are not star-like at all. And so we are very sometimes stuck in astronomy. So the name Little Red Dot will almost certainly stick around. It is not by no means the worst galaxy name. Green Pea is probably one of the weird ones. Any others? Yeah, there might be some other weird names. But, you know, actually, I know the guy who named them or put it. He only put it in the title. He took it out from the entire paper, and it has stuck around so aggressively. And what can you do? What can you do? A lot of inertia.

Speaker 2 [27:57]

Well, that has been answered. So there's one more. Is there a proposed connection between little red dots, black hole stars, and the black holes at the center of galaxies?

Speaker 1 [28:09]

Yeah, exactly.

Speaker 2 [28:09]

Yeah.

Speaker 1 [28:10]

So the idea is that we know how to grow black holes very efficiently, but essentially there's a limit on how fast they can grow. So if you take something that's a million, a million is actually a little small, we could probably do that, but we know of black holes that are a billion or a trillion times the size of the sun. And if you say, how fast could you possibly have grown? And you just trace that back to the beginning of the universe, you get a number at the very beginning of the universe that we don't know how to explain, right? Usually the way we make black holes, I should have included this in Astronomy 101, but one of the ways that we make black holes is you take a very big star, it dies, it collapses, makes a black hole. That black hole is about the size of a star. And so even growing at an exponential rate for its entire life, it cannot reach the masses that we see in the present-day universe. However, a black hole star might be able to go even beyond that limit that we normally impose of exponential growth. We call it super-Eddington accretion. but essentially that actually might be able to have a sustained massive growth phase that actually goes from a relatively formable black hole to something very big and then the normal universe takes over after those first billion years, more or less.

Speaker 2 [29:18]

I'm going to take one last question, but feel free to talk to Rafael after the talk and ask your questions there. Is there an explanation for why we see the red little dots much less now than we used to see 12 billion years ago?

Speaker 1 [29:32]

Yeah, phenomenal question.

Speaker 2 [29:32]

Yeah, phenomenal.

Speaker 1 [29:36]

I mean, again, this is a lot easier to answer once we know what they are. But in the paradigm where there are black holes going through this extreme growth phase, essentially the conditions only exist at the start of the universe to allow this to happen. So the beginning of the universe is a lot denser, a lot more gas-rich. And if the idea is true, the black holes are therefore a lot smaller. So you can form this system, but by the time they grow fast enough, they essentially exhaust that star.