AsyncIO vs Threads: who survives in the No-GIL Era?
Python concurrency for I/O-bound tasks traditionally involves a choice between multi-threading and AsyncIO. Multi-threading utilizes OS-level threads that are serialized by the Global Interpreter Lock (GIL), while AsyncIO uses a single-threaded event loop to manage coroutines. The primary advantage of AsyncIO is resource efficiency; a coroutine requires approximately 4 KB of memory compared to the 4 MB required by a thread, allowing AsyncIO to handle over 10,000 parallel requests more effectively than threading. However, AsyncIO introduces higher cognitive load because a single missing await keyword can stall the entire application. Furthermore, many asynchronous libraries, such as Motor or iofiles, are merely wrappers around thread pools, offering no real architectural advantage.
The release of no-GIL Python (starting with version 3.14) changes the trade-off by allowing multi-threaded code to execute CPU-bound tasks in parallel across multiple cores. While AsyncIO remains superior for high-concurrency I/O, threading becomes the preferred choice for CPU-heavy workloads, such as data science and machine learning. In web services, where CPU-bound tasks like Pydantic serialization typically account for 20% of execution time, no-GIL threading can provide significant speedups without requiring a full codebase rewrite.
Benchmarks indicate that for under 1,000 requests per second, performance differences between the two approaches are negligible. However, as the ratio of CPU-bound work increases—for example, when moving from MongoDB to Redis—the performance gains from no-GIL threading become more pronounced. The long-term evolution of the language points toward virtual threads, which aim to combine the resource efficiency of AsyncIO with the parallel execution capabilities of no-GIL threading.
This description was generated by Open-Source AI using the transcript of the session and the original submission contents.
This session took place in track Python Language & Ecosystem and was classified suitable for novice domain / intermediate python by the speaker.
Submission
The proposal as submitted by the speaker before the conference.
Concurrency in Python is full of stereotypes: "threads are useless because of the GIL", "async is always faster", "just make everything async". This session replaces opinions with mechanics and measurements, and updates the story for Python 3.14's free-threaded (no-GIL) build.
What we'll cover
1) How things actually work under the hood
- A Python thread is an OS thread (pthread_create/clone). The OS scheduler runs it like any other thread - the GIL only matters when Python bytecode executes.
- asyncio is also scheduling: one OS thread, many Tasks, cooperative switching at await, and readiness notifications via epoll/select.
2) Why IO-heavy workloads often look "equally fast" in threads and asyncio
- both models hide IO latency by switching while waiting;
- the real difference shows up in scalability and cost: per-thread memory/stack + OS limits vs lightweight Tasks.
3) When "async" is secretly a thread pool
- aiofiles delegates file operations to run_in_executor();
- Motor (async MongoDB driver) runs the synchronous PyMongo core in a ThreadPoolExecutor;
- frameworks like Django must bridge sync and async worlds (sync_to_async), adding overhead and sharp edges.
4) Benchmarks that mirror real services
- 100 / 1,000 / 10,000 concurrent IO waits: why "10k threads" fails but "10k tasks" is fine;
- memory and CPU overhead comparison (what you pay for concurrency);
- a microservice-style endpoint (FastAPI-like) in sync/threaded vs async mode.
5) What changes with free-threading (no-GIL)
- a high-level view of what CPython changes to make it possible;
- rerunning the same benchmark with and without the GIL;
- when an interpreter upgrade can deliver "async-rewrite-level" gains for mixed CPU+IO workloads.
Takeaways
- a practical checklist for choosing threading vs asyncio vs multiprocessing;
- performance vs resource-usage intuition you can apply to real services;
- guidance on how to read "async" claims in library docs.
Transcript (auto)
Auto-generated from the recording utilizing Open-Source AI. Speaker labels (Speaker 1, Speaker 2) reflect diarization, not identity. Timestamps refer to the recording.
Speaker 1 [01:19]
Well, good afternoon, everybody. We're here to listen to a very interesting talk, I think, by Igor Nokhin from K2 Cloud. He is going to talk about async I.O. versus threading. Who will survive in the age of the non-GIL? Well, give him a warm applause. Igor?
Speaker 2 [01:51]
Hello. I'm going to tell you about concurrency in Python, threading, asyncio, and what changes with the three-threaded on OJIL Python release. I should note that I'm a big fan of threading. I've been doing programming in about eight years, and the main goal is simple. I want to add more intent to the way we choose an approach, not because it's trendy like asyncio, AsyncIO but because it fits the real workload. A bit of context. I work in K2Cloud. It is one of the biggest cloud providers in Russia. We've been working on it since 2009. We built it from scratch with the KMQM. And the reason why they picked this topic is simple. They're growing, hiring new engineers, I do a lot of technical interviews, and I keep seeing the same pattern. Many junior developers know nothing about threats. They use only AsyncIO approach, they do not really understand the GIL, and actually the problem is that they treat thread-based code as a legacy and don't even want to know about it. And this is a little bit disappointing, because Python 3.14 was released not so far, and it brings a stable, no-GIL version, which significantly improves the speed of multi-threading code, And this may have a huge impact on the big enterprise systems that has been built all day with threat in mind. And this one creates a question. Do we really need to rewrite, I don't know, from scratch, huge production systems from trading to a single model? To answer that question, we should start with a quick theory recap. In programming, not only in Python, we in general have two types of tasks. We have CPU-bound tasks, which are related with pure computations, a lot of Python code running, and so on, and we have input-output-bound tasks, which are related with waiting. For each type of task, we need to choose the appropriate solution in Python. For CPU-bound tasks, we shouldn't even choose between threading and asyncio, because the only choice is multiprocessing because of the drill, because Python is unable to run codes across the course, and so on. But for input-output bound tasks, for me, the both approaches are quite the same, because actually they use the same K mechanism inside of them. Let's explain. If you talk about multi-threading, for each Python thread, we should note that CPython creates a real operation system thread. Then operation system scheduler, which knows nothing about Python, will try to run that threads. And as I said, Shadware knows nothing about Python, so it may give CPU time to several threads at the same time. When thread starts running, it tries to log the drill, if it does successfully, it continues to work, if not, it goes to sleep. And important note, it does it effectively, because CPython uses thread conditional wait function, which allows us to not waste and to not burn our CPU time for nothing. As a result, CPU parts of code effectively serialized and we gain speed boost from overlapping input-output operations. Of course, there might be some ineffective scheduling when interpreter forced thread to release the drill until the thread didn't reach input-output operation. It happens every 50 bytecode operations or by timeout. It depends on the interpreter realization. What about asyncio? In SyncIO we have only one thread, it's busy by the InventLoop, and the main idea is to keep that thread as busy as possible. So we don't have threads on the left side, we have tasks or coroutines, we have Linux on the right side, and the schema of InventLoop is the following. InventLoop runs first task or first coroutine and does it until the moment when it reaches the first await keyword. It doesn't switch to other coroutines, it just runs only one coroutine. When it finds the first await keyword, it actually continues to run that coroutine. It goes down until it finds the real input-output waiting. In that moment, when it finds the waiting operation, EventLoop creates a special socket and delegates waiting of that input-output operation to that socket. And EventLoop runs EPOL Linux operation to get the results from all created sockets. It allows EventLoop OSMK solution to not waste our CPU time. Then we receive all results from radio sockets and continue to run other coding. As a result, the final solution looks like our trading version. CPU parts are serialized and we are unable to run any other task or any other coroutine until we hit the input-output weighting. So if you will compare these two approaches, they will look similar. The only difference between them is that switching tasks, because the operation system scheduler switches tasks in trading, We do not know, we don't know to know about it, but in AsyncIO, tasks switching is our own job. And this is the key problem of AsyncIO, because when you lost your await keyword, you will not only stall one loop or one request, you will stall the whole event loop or whole application, which means that AsyncIO has higher cognitive load and requires more discipline, because you must not only know your code and how many CPU codes you run inside your own code, you must also know the code of all the libraries that you use inside. Because sometimes the support, asynchronous support in that library is not so good as we may think about them. For example, operation with files. Asyncio doesn't provide built-in operation with files, asynchronous operation with files, So people often use iofiles library. This code looks like asynchronous because it has asyncdef, it has await keywords, but the truth is that isn't really asynchronous. Because iofiles is just an asynchronous wrapper around thread-based code. Because if you look inside the documentation, at the bottom it says that iofiles delegates operations to a separate thread pool. It means that we mix both worlds and our event loop that must be as effective as possible will just idle and wait until the thread pool will do the real operations with our file system. And this situation happens not only with small libraries which are related with files. The same story might be found in some database drivers like motor because in our cloud we We used PyMonger, synchronous driver, and we would like to get some speed boost. We would like to find some modern solution. We found that motor, and documentation says that motor processes tens of thousands requests per second. And this is it. This is something that you're looking for asynchronous library, but the problem is that motor uses thread pool inside. So it's just a wrapper around PyMonger. it runs PyMong inside the thread pool, and as a result, you still use the good old threads. The same story and the same pattern used to have a lot of database drivers, and some of them still provide only a synchronous-looking API, like MySQL. Also problems with, I think, AI might be found in some web servers, web frameworks, like Django. Django is a huge project, it's a long story, and a lot of enterprise projects have been built on top of it. And not so far, Django released asynchronous support. Not everything good with it, because, for example, Django has problems with middlewares. Because not all of Django middleware is asynchronous. Django has to mix both worlds, and it relates to the speed of your code. Because changing synchronous context to asynchronous is a complex operation. You have to pay the price for switching that context. But even if you would like to write a middleware that will support both ways, you will write quite complex code with not really good decorator, you have, right, more codes and it's not cool. Another thing and another problem with asynchronous Django support is object relationship model. Because despite the fact that Django has added some asynchronous methods with the A letter which indicates that this is asynchronous method, some key mechanism of ORM and some Some key mechanisms of working with our objects, like transactions, still do not yet work in the synchronous mode. You have to use a help tool. And these things lead us to the following problem. Django has unsafe calls when we're mixing synchronous and asynchronous chem. Because when you use threading approach, you are ready for some race conditions, for mutexes and other stuff. But Synchronous by design was created to avoid us from these types of things. Because you have only one thread, you have no some race conditions, and that's quite said that we get the worst thing from both worlds when we're mixing synchronous and asynchronous things. But if we have problems with asynchronous, why is it so popular now? Why do we rewrite our code bases from trading to asynchronous support? Because when we are adding asynchronous support, we have to rewrite our codebase from the scratch. And the reason is that some libraries has a good designed asynchronous support. And the example of it is PsycPG. In the third version they added asynchronous support, and they do it the right way. The right way of doing it is duplicating your codebase. They had connection class, they added AssumeConnection class with the same interface. They had cursor, they added async cursor, they had execute, fetch one, and other functions, and they just rewrote them using asyncdef in their declaration and await keyword inside. The only file that has really been changed is waiting.py with the waiting responses from databases. The guys replaced while to select waiting loop with the built-in async-wait-for function. And this, I don't know, one, maybe 200 of unique lines of code followed us to big performance boost, because synchronous version of PsycPG are able to handle about 800 of requests per second, but asynchronous version are able to handle around 2.5 thousands of requests. requests. It's three or four times higher number of requests. How did it happen? To show it, here is some small example of which simulates input-output operations with Google time sleep, with trading, and I think a version. And if you will try to measure the speed with different number of requests, we will get quite same results. Because when we have below thousandths of requests, trading an asynchronous version will show you quite similar results. The difference will be about 10%, but it's not so much, because you have one thousandth of requests per second. It's a big amount of requests, and not each system may reach it. But it's a speed test, benchmark, so we have moved further, and if you increase the number of requests by 10 times, I think the version will do it for 1.05 seconds. But threading version fails with runtime error because it's unable to start the new thread. Why and how did it happen? Actually, to fix it, we need to limit the number of our threads to 1,000, and the final time will be 10 seconds. Why? Asynchronous is faster, and why it's able to handle more requests. Actually, because Asynchronous is not about the speed. It's about the effective. Because each thread takes four megabytes of memory. It holds variables and other stuff. But each coroutine is just a socket. It stores four kilobytes of memory. It means that AsyncIO is more memory efficient. It's about 1,000 times more efficient than threading. That means that you have handled more asynchronous requests with the same amount of resources. And unfortunately, asyncio has less context switching and it has less scheduler overhead. Because as I said before, in the threading, operation system scheduler knows nothing about Python, knows nothing about what threads are ready to work, but asynchronous works only with those coroutines which are ready to be worked. But this is noticeable only with one thousandth of tasks. But this is the comparison before the drill, how no drill influences of all of that. Drill function, drill version, available to get the speed boost only by overlapping input-output operations. But Node.jl version scales our CPU part too. And this one may have a huge impact even of our web services, because modern web services has a lot of CPU work. Because we will try to check how we handle our requests. We have some modern routing frameworks, like FastAPI. We use Pydantic Insight, which do a lot of serialization and deserialization work. We handle our requests with some business logic. We then often convert this data to some database objects, and only then we store them into the database. After storing data to database, we need to do the same number of stages, but in reverse order. If you will try to measure how much time we spend on input-output operations and CPU operations, think that, in average, we will get the same fraction. We spend 20% of time for CPU-bound operations and 80% of time for input-output operations. And NodeGL release helps us to speed up this 20% of time of our CPU parts. And another point that makes asyncio worse that threading in NodeGL release is that removing GL means locking tons of small logs instead of one global log. How does it work? Before NodeGL release, the thread has to log only one log before it starts to work. But with the NodeGL release, thread has to log a lot of different logs inside some native libraries. It makes the single thread code slower because we received some over time for it, but at the same time it makes our multi-threading code much faster because we parallelize the CPU part too. But this means that if you will return to our original question, what to choose for CPU bound task, threading or synchro, the answer is threading. Because threading is easier than multiprocessing because threads have shared memory, it's easier to synchronize them between each other and so on. But maybe write our web services on multi-threading because we have just 20% of time. And the The answer is yes, but we have some conditions, and I have a story for you. In K2 Cloud, not so far, we released a new region. Region means separate data center in different cities. For example, we have one region in Moscow, we have another region in St. Petersburg, and connectivity between these regions are not always guaranteed. That means that you should have some reliable message broker. We thought that it would be nice to realize some Amazon simple queue service inside and then expose it publicly, because we are making the same thing as AWS, but why don't we realize it by ourselves? As a start, we're beginning to do it with MongoDB because we use it heavily inside, but we hit the limitations and switched to Redis, and Redis is the thing that perfectly fits for this configuration, because Bredis holds all the data in memory. That means that input-out operations might be ten times faster. That obstacle shifts the fraction between CPU and input-output times from 20 to 80 percent to 40 to 60 percent. It means that fraction of CPU time was increased. As a result, we performed some speed tests inside with some of our internal benchmarks. Threading version with the drill showed 2.8 thousand requests per second. It's some basic value, I think. I think your version showed 3.5 thousand requests per second. It's a big amount of requests, but it comes with the costs of rewriting the whole codebase from trading to asyncio manner. But what will be changed if you just change the interpreter version? Asyncio with the non-GL version showed quite the same amount of requests, 3.5 thousandths. But what about the trading version? Actually, the trading version showed the same amount of requests per second, like as in KO. But the K point is not about the numbers. K point is about the decision. If you have a huge code base that has been written with threats and no-jill build is suitable for you, it might be not a good time investment to rewrite the whole codebase from trading to Async.io. You should just stay and wait until trading version will become more widely supported. And conclusions. Async.io is not about speed performance. It's about effective resources usage, and up to 1,000 of parallel requests, it doesn't even matter what to choose. Trading or asyncio, you will get the same performance, but trading may get up to 4 gigabytes of random access memory. It has bigger costs. But if you have more than 10,000 of parallel requests, asyncio will be cheaper for you because it's available to handle more requests on the same resources. But do you really have 10,000 of requests per second? It's a big amount of requests, and I don't think that every system reaches that amount of requests. And at the same time, if you have 10,000 requests, you'd rather to scale horizontally than vertically. The other thing is that no-jail approaches become more effective with the CPU-bound operations growing. As I showed you with Redis, the more CPU operations you have, the more performance boost you will receive. As a result, the best field for using NodeGL is data science in machine learning. The average field of using them is our web services, and I think the last thing we should use NodeGL's solution is some modern system with large language models, because the fraction between CPU parts and input-output parts might be 5% to 95%. And the last thing about the future, because Node.jl opens up an opportunity for virtual threads. Virtual threads is the thing that has been already realized in many other languages like Java, Golang, and so on. It's a system where you handle CPU parts like threading with Node.jl does, and at the same time you handle input-output parts like I think I.O. does it with the sockets. It combines both worlds, it makes the Python more powerful. So this is it. Thank you for listening. I'm ready for questions.
Speaker 1 [24:11]
Well, Igor, thank you very much for this interesting talk. There are no questions. Oh, there's a question here in the audience.
Speaker 2 [24:22]
I posted it as well, but I'm happy to repeat it. So how do you judge the difference for us with threads and async IO when it comes to the control we have? So with async IO, we can control when we are being kind of put to sleep with the await. And with threading, it can be at any arbitrary point in time, which requires maybe locking or some other mechanism to make sure that we don't manipulate shared resources in the wrong way. so how do you judge it with regards to this problem the main advantage is asyncio to be honest is the thing that you don't need to use mutexes semaphores and other stuff for me this is the real benefit of asyncio you just do not think about race conditions and so the other But the things, to be honest, they are quite similar between these two approaches. And even if you will change your coroutines manually with await keywords, it's just an illusion, because you don't know exactly what happens in other coroutines when you type or when you reach await keywords. That's it.
Speaker 1 [25:42]
OK, actually, there are a couple of questions. One question is, isn't the question more about if the problem is more too heavy, async, or more CPU-heavy threading? Uh, what? Is the question more about if the problem is more IO-heavy, async, or more CPU-heavy threading?
Speaker 2 [26:05]
Yes, maybe, maybe, because it might be the different view on that problem, because if you have, as I said in the last slide, if you have the system which has more and much more input-output tasks, you should use AsyncIO, because despite the fact that threading may speed boost your input-output operations too, like AsyncIO does. AsyncIO will be more effective if you have a high load system. But if you have a CPU-bound system and most of your tasks are CPU-bound, you definitely should use threading with NodeGL.
Speaker 1 [26:50]
Okay, thank you. Another question, does threading no-gill Python use multiple CPU cores? If so, the comparison with single CPU async IOs is unfair.
Speaker 2 [27:05]
Maybe, but single CPU is the only option for passing KO.
Speaker 1 [27:12]
How do you judge the problem that with threading we don't have control when we are interrupted? Can this lead to issues in practice?
Speaker 2 [27:22]
I think the same question.
Speaker 1 [27:23]
Oh, that was your question, sorry. As many CPU-heavy libraries are implemented in other languages, like Rust, which don't work inside the Python thread, is the CPU-heavy use case still relevant for Python developers? Since it's outside of Python, we can view it as an IO use case.
Speaker 2 [27:45]
I think the question is more about our virtual threads, because they will help us to use both approaches together. But another option that really works in the current Python version is to write some parts to the rest. Yes, that's our reality. It happens from time to time. one of that example because a lot of Python ecosystems and a lot of Python tools you're writing to Rust just to scale some CPU parts.
Speaker 1 [28:22]
You said async I.O. causes more mental load, but isn't threat safety much harder to achieve since you have no control when a routine is interrupted? Yeah.
Speaker 2 [28:34]
Yeah, okay, this is a little bit different problem, you know, when you start rewriting your big code base from threading to asyncio, unfortunately you have to change your software developers, because the software developers that has been developed your code, they knows only threads, they knows nothing about asyncio, they don't have enough experience to write the same code base in asyncio manner. And I've talked with some maintainers of Python, and the asyncio code itself is quite complicated. It's hard to fix the bugs inside the CPython, inside the asyncio style. So when I say that asyncio requires more discipline, It means that we don't have, I think, enough programmers with, I don't know, 10 years of experience writing Python code in async style. Because it's more a problem of practical usage of that async.io. It's much easier to write the code in a trading manner.
Speaker 1 [29:46]
Okay, one last question. Is it possible, or maybe are there already approaches, to provide an abstraction similar to Go routines in NoGill, Python, where...
