Are we free-threaded ready? Looking at where free-threaded Python fails

Speaker 1 [02:14]

If you see free seats next to you, maybe it's a good idea to move to that so there's more people coming in. Thank you. all right uh welcome back or welcome everyone uh we have two very nice talks starting with Choi Ting Ho, who is a developer advocate for JetBrains. And she will be talking about, are we free-threaded ready, looking at where free-threaded Python fails? Let's welcome her with a warm applause.

Speaker 2 [03:31]

Thank you. And, yeah, thank you for the almost full room that, like, you know, I'm surprised. I think people are really interested in this topic. So, but first of all, I want everybody to be, like, understanding what's going on here. Who has heard about the comment that Python is slow? Yeah, lots of you. Who actually think that Python is slow, you can keep your hands up. a few of them, okay, I like your bravery and, you know, your hot take. Yeah, like, I would say that when I heard about the comment that, oh, Python is slow, I want to actually, first of all, define what is actually slow mean, right? Like, you can comment on things, but let's just get our definition clear first. So, I do a little bit of, like, digging, like, why Python is slow. So I think a lot of the opinion is that because Python has this thing called global interpreter log, which I call it GIL, but I know there's different pronunciation for the acronym there, but I just call it GIL here, just so you know this is what I'm referring to. It's actually when Python was born in the 90s, wow, I think it's older than than some of you here. So you all look very young, that's why. So when Python was invented in the 90s, it's actually a genius design because at that time, a lot of the computer only have one core, which compared to now, if you walk into a huge electronic store, let's say for example, Apple, you go inside and then get a new MacBook, whatever, it's definitely have more, way more than one core, like you may have like 16, you know, some of them may be like, oh, we have some GPU for your, you know, AI stuff and all these crazy things. But at that time in the 90s, one core is kind of the standard. So everybody, if you are lucky enough to have a personal computer at home, you would have a computer that is quite basic compared to what we have today. So unless you are a researcher that have a huge cluster which luckily at that time I think when I was in uni we also have a cluster in the university that you can do some like more kind of CPU heavy computation with that cluster which because it's a cluster you can do some HPC stuff but let's not get into that because this is not what we are going for today. So when Python become quite popular among the data science community when data science actually become a thing, which is like maybe around 15, 20 years ago, you know, then we, a lot of data scientists think that it's not enough because we actually want to do research. We want to have more computational power because, you know, we're handling a lot of data, we're handling a lot of numbers, we need something that is not just working on one core. So, oh, by the way, the global interpreter is the reason why Python only, one Python process only work on one core. It can't have true parallelism. But we will dive more deeper into it later, of course. So yeah, like, researchers want more computational power. So that's why NumPy was born and all this, like, you know, basically NumPy is a game changer at the time. Like, you know, it allows a lot of, you know, more computational power that can be used. It's actually, it's achieved what it tried to achieve by having all these binary extension, which is, you know, written in another language, not Python, and then it, you know, compiled into a binary, you know, a binary file that can kind of, you know, talk to the C Python interpreter and that's why we can have parallelism. So if you want to achieve that kind of, you know, power, you have to go outside of Python. And, you know, today if you think about it, it's like, oh, yeah, but we want Python to be better, more powerful. So maybe it's time for a change. So the community, I think we all have this feeling we want Python to be better and we We want Python to be working better in the modern era, which we have a lot of cores available, not just locally, but if you run something on the cloud, again, open a huge amount of things you can do. And also, Python goes from a very hippie-ish kind of scripting language that you may just use it in your pet project into something that could be fundamental in some company. It becomes a very important language in a lot of business use cases, especially if a lot of data is being handled. We want to take Python to the next level, so maybe we should remove the GIL. We can have true parallel processing available in Python, which is a huge step for the community. However, after actually this idea of removing the GIL has been floating around for maybe around ten years, multiple people have tried to propose something that may work. However, it takes quite a while for discussion and a lot of whether the community are ready for this huge change. But fortunately for you and me, free-threaded Python, it used to be called no-guild Python, but then there was some technical reasoning that that was not a good term, so that's why we call it free-threaded Python, but generally the idea is that this is the Python version that doesn't have the guild and it can achieve true parallelisation and can do multi-threading. So that was introduced in Python 3.13. So it's proposed in a PEP 703. Maybe you'll see that in some kind of quizzes down the line when you go to a Python conference because this is quite an important PEP that you may want to remember it. So yeah, that's the PEP that proposed, let's remove the GIL, let's achieve three-threaded Python, and this is the starting point. So when it's 3.13, free-threaded Python was introduced as an experimental build. So, of course, it's not the standard. So when you get the standard 3.13 CPython, it's going to be not free-threaded. However, you have the option to either build that free-threaded build of Python, Or you can get it from UV, you can get it from PyEnv, like a lot of those management tools provide that for you to get it and try it out yourself. And one more thing about it is that at that time, around the same time when this is announced, Also, the steering council member, which is the technical leader of CPython, announced that there are plans to make a three-threaded version of Python the default Python. When you get Python in the future, it will just be three-threaded, and that is the future. What has changed? Does it affect me? If, let's say, tomorrow, or not tomorrow, but let's say, you know, 3.15, it will be like, oh, free-threaded, yay, but, you know, does it affect me? Do I have to make any change to my code, that kind of work? So the good news is that it may not affect you, but let's look at what has changed. So, first of all, like I said, now we can actually achieve true parallel processes in Python. So right now, if you are using the standard version of Python with the GIL, you can still use a library call threading. But however, that is not true concurrency. Later, I will show you. If you do some profiling and look at the frame graph, you will see that all the threads are not working at the same time. You will know what I mean later when I show you. So the only way you can achieve kind of a parallel processing with Python is to have multi-Python processes. And that may not be an ideal design for you, because the overhead is quite big, and then sometimes some data are not shared that easily. So now, with this free-threaded build, you can compare and then see things could be faster, and you can see the multi, like, different threads are working at the same time. However, if you are one of those amazing developers who, you know, maintain a, let's say, open-source library that are written in other languages and have those, like, binary extensions that achieved this kind of parallelism in the past, now, if you are adapting it to the new build, you may have to make some change because in the past, when you write those, let's say your library in C, in Rust, you may assume that, oh, Python has the GIL, I don't have to worry about all these errors that could come up when you are dealing with concurrency programs. Now, we can't assume the GIL will be there because some of your users may be using this three-threaded build. You may have to handle things a little bit differently or if there's a part that having the GIL there is essential, you may have to mark it so, you know, at least it's thread safe for your user to use. And also, because this is a huge change and it involves a lot of, like, kind of lower level manipulation of how things work, the C API, which is those kind of how C Python, the Python interpreter, communicate to all these binary extensions. So those APIs are changed. So there are new ones, and there are also a lot of recommendations of how you can adopt. Those are actually the CPython core developers, they have documented very well. There are some resources at the end of this talk, don't worry about it, you will find them, I will give you those links later. So yeah, I said that a lot of you may not be affected because I know a lot of people using Python don't care about parallel processing and for them, Python is still a good tool for them because they can just quickly script something, so they don't necessarily need this or if they are using async, async is a different thing, it's not different from threading because it handles IO bound program instead of the CPU bound program, so you may not care about all these things, right? And it's actually fine, because, well, even if, again, like, even if 3.15 is all refretted, still, you know, your Python will still work more or less the same. And also, you, you know, and you may have the option to, you know, put the guilt back in. So, again, like, don't worry too much about it. If you don't worry about this kind of thing. Python is still good for you. However, if you are maintaining those libraries that have binary extension, you may have to look into how you can migrate and take advantage of using this free-threaded build for you and your user. And, again, maybe it's not in 3.15. I don't have any news about that. But, again, like, from what I heard from the leadership of the CPython development, it will be a future. Right now we are in phase 2, that's what they said, it's 3.14, FreeThreadedPython is actually not experimental anymore, so actually, you know, you are welcome to use it in your production if it works for you. So that's phase 2. However, phase 3, which is the next phase, we will have FreeThreadedPython as the default. So we're heading there, I don't know when we'll be there, but we will get there at some time. We have some insider news there, almost certainly not 3.15, so don't panic, but we're heading that direction, just don't know when. Cool. Right. So one of the other things that kind of affects this speed that like when we are heading like two- or three-threaded as the default, is that is the community ready to adapt it, right? Because like I said, we have a lot of libraries that we use and love, and they may have binary extension, but if they are not three-threaded ready, then maybe the whole project doesn't work as well as we hope it should be. So because of that, a lot of members in the community, this is actually, I get this beautiful wheel picture from Hugo. Hugo is the release manager right now for CPython. So if you go to that website, you'll see the most updated version of it. I grabbed this picture a few weeks ago, so it's kind of more or less still quite updated. So you'll see that only around half of the most popular libraries which is like 360 of them are chosen. So like around half of them offer a free-threaded will. What does it mean? So will are actually those extensions that I talk about is actually a packaged binary that has, you know, that will make your library just, you know, you can plug it in your Python code and it will just work. So that's actually a good thing because if you get it from PyPI, you would, let's say you pip install a library, you would get the wheel so you don't have to build it from source again or, you know, because not everybody have the right compiler for those things. So having three-threaded wheel is quite important for you to be able to have a compatible things that work for your three-threaded Python. So only half of them already. Some of them are kind of getting there. They have some generic Python wheels, so it should be quite easy to have a compiled wheel ready for three-threaded Python. But they are not there yet. So those are a small number of them. And there's 41 of them are actually like, again, they don't have that available yet. So the wheel that they offer on PyPI are not compatible with three-threaded builds of Python. So you may ask, right, oh, I use so many libraries. Can I check if it works on three-threaded build? Yes, you can. If you go to PyPI, just search that library, pandas, NumPy, whatever, you will see that whether if they have the three-threading tag on the page, on the left-hand side of the page, then yeah, well, it's ready. So the maintainer set is ready. You can use it with your three-threaded Python build. However, sometimes you may be like, I just want to quickly check it because I already install some library on my free-threaded environment, so can I just quickly look? Yes, you can. If you check the will, if you have your .env, your virtual env, and then look into it, do some digging around, you'll see those wills that you install in that environment will be there. And then they should have, so the will files will name something like dash, something that's dash and then .so. If you look in one of them, it should say that what, you know, environment it works with. So, for example, this one, cp314t. So, the t there means that it's 3-threaded Python. So, cp means cPython 3.14 3-threaded build. So, it works with that. So, you probably, because in that environment, you have the, you know, 3.14 cPython 3-threaded build. So, that's why pip get this for you. So you'll be like, yay, I get the right thing for it. Good. Another thing you could do, you can test it yourself. Again, like Hugo published a blog post of how you can test free-threaded Python build with any libraries, whether you get it from PyPI or whether you have it yourself because you build your own library. You can test it. So just get free-threaded Python and test and see what happens. That's one thing to do it. And if you are like, if it's an open source library that you really care about, maybe you want to make some report, some issues, or any finding, or if you have any use case or anything, you can also share your use case. You can also help fixing any issues that you find if you are, you know, if you have the knowledge to do so. So, again, always encourage you to contribute to the open source community. So, however, I know that, again, like, not everybody really cares about free-threaded Python, and so you may want to do some sanity check whether this is for you, and if you have CPU-bound processors, which means that you have to handle a lot of data, you may have some, you know, Monte Carlo simulation or doing some machine learning things that, you know, need a lot of computation, yeah, you may have to look into, like, oh, whether free-threaded Python is helping you, because you're already using those, like, numpy and those, like, so they are already doing some parallelization, but, you know, you may also, you may have some other things that you may want to see, check whether that helps you. And also, check whether all your dependencies are ready, because if one of your dependencies is, you know, if they mark it as, no, this is not for free-threaded Python, instead of of creating some like crazy memory things that can be fixed. Python is very nice, it will put the guilt back in so your program is still safe to use. However, you won't be taking advantage of the free-threaded build. So yeah, I think that there's some misconception out there that people think, oh, we have free-threaded Python, means everything's gonna be faster, right? Not really, because to make this change, actually you have to make some change in the design of your code. So a lot of mistakes could come up if like, because the guild was put there to protect all these mistakes to come up, right? So if you are not familiar with how a truly concurrent program are designed, how the data are shared between different threads, you may encounter any of those problems, right? Like, mutable data, you know, fresh safety, deadlocks, all those things could happen. Some of them actually could already happen in some async code, so if you have, you know, dip your toe in those, you know that, like, oh, you have to think more about those kind of things, right, so, yeah, so that also happened. And, yeah, multi-threading is not always faster. Sometimes I do some experiment and check, Let's say you have a program that takes 0.05 seconds to run on one thread. If I have five threads running it, is it just like every five x faster, right? So it would take 0.01 seconds. No, because there will be overhead. When you do multi-threading, your operation system needs to juggle between those threads and make sure things are working fine and all these things. So yeah, so there will be overhead. and also there will be design change. And, for example, if you are sharing data, you have to manually put some lock in there, so those mechanisms also will cost some time. So, yeah, it's not always faster, and so, you know, just a caveat that you need to know. So let's have a quick look. I have a little bit of time, so let's have a quick look of what does it look like if you, for example, try to do some, you know, multi-threading and stuff. So I have an example here. It doesn't do much. It just calculates something very big here. So it's a CPU-bound work, but it's doing something silly, just 10 to the power of 7. However, we would spawn five workers on five threads to work on this. So right now, I'm using UV, but I'm also using a standard Python build. So it's not three-threaded Python. So let's double-check. So I'm running it on this VN. So this VN, I know, you know, if you check, I don't know if I have time to show you, but if you check, you see that it's 3.14, not 3.14T, so it's not free-threaded Python. So I can run this sampling profiling. Again, this is recommended by the CPython, you know, the free-threaded community, so this is actually quite a good profiling, too. So if I run it, okay, it's running something. So, it starts a worker, and it's doing something, and I can actually see the profiler, so it will open it on a browser, and you see that this is the five workers. And I hope you see it, like, I'm sorry if you are sitting quite far, but, you know, so you can see that, oh, there's all these little spikes, what are they? Actually, you can see those spikes are not overlapping. So again, we are not truly doing multi-threading, but actually, we are juggling between these five threads. So that's why they are not really overlapping. And the time it takes to run is, let me switch back to PyCharm. OK, here, cool. So the time it takes to run is, OK, let me make this bigger. In total, it's 3.73 seconds. So OK, this is standard build. Let's switch to free-threaded build. So I have another interpreter here. OK, just terminate that. I'm not running it anymore. I'm also closing this. So I'm now using another interpreter. So let's double check what it is. It is the T one. So it's inside that pi and the other vn T thing. If we look at it, we should have C. So there's a 3.14 T here. So this is the free-threaded build. Trust me. Right, so you will see also from the profiling that is actually different. So now this is the TBL. Oh, this is different now. It seems that they are way more overlapping, so these five threads are actually working together. You can see there's a little bit of the main thread here. It's like this is spawning all the threads, and this is the overhead that I talk about. You spend a little bit of time orchestrating all these workers. So since we don't have too much time, we can't dig into a lot inside here, but we can have a look at how much time it takes. It only takes 1.62 seconds to do this. So it's not five times faster, but it is faster. So yeah, so if you have a CPU bound process, you can maybe gain some speed from that. So again, I highly recommend you use the sampling profiling. What I did here is just that I install it, and then because you can run it on command line. I just set it as a run script because I have PyCharm, and it's nice. So yeah, so you can do that as well. So I have one minute left, so I'm going to show you the last conclusion. Because if you are falling asleep and don't know what's going on, at least remember these three things. First of all, Free Feather Python is the future. Be ready for it. You may not be affected. You may not care about it, but still, this is an announcement, basically. This is going to happen. If you are curious and if you think that may help your work, maybe learn more about it. Learn about multi-threading. Learn about what does it actually mean and maybe do some profiling on your code and study more about it. Another thing is that if you have any use case, again, it will be very, very good to share so other people can be inspired. And learn more. And also, if you have any hot takes, please have a civilised discussion online. I know there may be a lot of threats on the Python discourse that people are talking about these things, which is great. So these are the useful resources. The QR code is linked to this slide. So again, I know this is very information heavy, but if you take a picture of this slide, you'll have everything. So yeah, I highly recommend check out the things by Hugo and check out the py3threading.github.io. That's where all this information about free-threaded Python lives. So, yeah, cool. Last things. If you can spend two minutes, tell me how you use AI at work and all those things. That would be great. I'm just trying to understand what people are doing these days with AI. So that's it. And thank you very much.

Speaker 1 [30:04]

Thank you Chuck. Let's go to the questions now. So there's one popular one Will something change regarding the garbage collector? If yes, how do you see Python developers prepared for this change?

Speaker 2 [30:17]

Yeah, so the garbage collector, if you want to look into the technical detail of what has changed, I think one of the things you can dig into is that PEP 703. Another thing is to look at the release notes of 3.13 and 3.14, look at the three-threaded build session there. I'm pretty sure there were some changes for the garbage collection. However, I'm not going to dive into technical detail here. First of all, I'm not the expert. I don't want to give you wrong information. Second of all, it's quite, you know, tech-heavy, so maybe some of you is already falling asleep. I don't want to, you know, give you too much, like, information dump. So if you are interested, again, like, look into those resources and we can have a discussion afterwards.

Speaker 1 [31:06]

I would like to give you the mic because the online participants are not going to hear it.

Speaker 2 [31:13]

So one easy thing I think we can say is that it's now partly inside of the thread happening and partly shared between threads. So basically the objects that are created inside of the thread are now part of the thread.

Speaker 1 [31:25]

created inside of a thread and deleted inside of a thread.

Speaker 2 [31:27]

inside of a thread are very fast to create and delete and only objects that need to share

Speaker 1 [31:34]

that are shared across

Speaker 2 [31:34]

that are shared across thread boundaries, they need more work to be cleaned up afterwards. There's a lot of change in the garbage collector, actually, and the way the garbage collection is done, but that's the basic idea. Yeah, the reference counting and a lot of things. Look into PEP 703, you will find it's a gold mine. There's a lot of things there. I enjoy reading it myself.

Speaker 1 [32:00]

Okay. Next one. Python is a gateway language. All CPU-heavy libraries are implemented in other languages like Rust. Do you think there are still relevant use cases which can benefit from that?

Speaker 2 [32:13]

Yes, I am a big fan of PyO3, if you're talking about Rust here. So I think, again, it's sometimes beneficial to write those codes in another language. For example, Rust, we have Rust Consider. More of Rust Save, because if you don't do anything that is unsafe in the Rust code, then it's safe. I know, it sounds weird, but if you know Rust, you know what I'm talking about. So, yeah, like, there are some benefits of doing that. However, I know that, you know, some of you may, like, maybe, oh, I just want to have something that works in Python because, you know, you may not be a software developer. You may not want to learn another programming language. So having Python, having more capability and, you know, still keep Python very easy to use for a lot of people is still good. However, you know, I think Python is a little bit like, you know, it's like my friend Deb have a very good analysis. It's like a capybara. I just be friends with everybody. All the animals are hanging out with capybara, right? So Python is good with anything, you know, and it's very flexible and people can use it for a lot of different use case. So whether you are a, you know, software engineer, you want to really like customize and polish everything you want to write in another language that's fine but for someone who is just like I'm a data scientist I just want to handle my data and that's also fine too.

Speaker 1 [33:44]

All right, there's still a lot of questions, but we're running out of time. So I'd like to invite you to talk to Choyk after the talk. And with that, let's thank Choyk again for a very nice talk and for giving us updates. Thank you. There's one last talk in this session.

Speaker 2 [34:03]

session.