Rustifying Python: A Practical Guide to Achieving High Performance While Maintaining Observability

Speaker 1 [00:07]

Thank you so much. Hello, everyone. There are actually a lot more people than I thought there would be. Thank you so much for joining me today. Whether you're here in the room or tuning in online, it's a pleasure to have you with me. My name is Max, and I'm a software developer working at SAP. In today's talk, I'll share with you the journey my team and I took to migrate key parts of our Python application to Rust, and how this helped us improve performance by over 200%. Before we begin, because in the recent days I've seen that it's difficult to see the slides, in case you have difficulties, scan the QR code and you'll find the slides in PDF format on my GitHub page. Now, if you have strong negative feelings about multi-language code bases that you would like to keep, this talk might not be for you. I'm sure that if I ask you right now for a specific part of your Python application that's a major performance bottleneck, most of you would be able to name one within the next minute. And while I won't ask you, I would like you to take a moment and think about it. Think about the main Python application that you are currently working with and which part of it is the major performance bottleneck. My goal for today is that by the end of this session, you will be equipped with the knowledge to assess where Rust can improve your Python application's performance and how to integrate it in a reliable and observable way. We will focus on three things. First, we will see how we can identify which parts of the application are suitable for migration to Rust and why. Second, we will migrate those parts to Rust and deploy them in production. And I will show you how not to do it by sharing some of the mistakes that we've made over the past two years. And third, throughout the entire talk, we will distill a list of lessons learned when it comes to migrating code from Python to Rust. Now, not all of them will apply to your specific use case, but I hope that some will help you avoid mistakes that we've made. Before we begin with the journey, let me set the stage of our story with some exposition and world building, if you will. It's the beginning of the year 2016. The Brexit referendum was not due for another month. The number of people globally using mobile devices to access the Internet overtook those using desktop computers for the first time, and the small team behind the CI-CD infrastructure of one of SAP's core products, the in-memory database SAP HANA, well, we came to the realization that off-the-shelf CI-CD tooling just wouldn't cut it anymore. A strategic decision was made. The team would build a custom task execution framework with built-in data management. Almost a decade later, and the now much larger team is still building and operating set task execution framework. Our production cluster spends more than 2,000 virtual machines and bare metal servers across a mix of public cloud providers such as AWS and GCP, as well as SAP's private cloud. With over 1.2 petabytes of memory and more than 250,000 CPU cores, the cluster executes an average of around about 170,000 tasks per day. That was the prologue, and we now move on to the main story and jump to the year 2023. As I have promised in the beginning, we will now follow the journey my team and I took to migrate key parts of our Python application to Rust. We will focus on two major problems. Some unexpected low cluster utilization on one hand and some increased data management overhead on the other. One beautiful Monday morning a member of our team, this one actually, noticed some unexpected drops in the Grafana monitoring panel that shows our cluster utilization. And while we did have enough workload ready to be processed during the entire time frame, well at times it seemed that we were unable to get it out onto the cluster fast enough as to keep the utilization high. And while this was not a common phenomenon, we had seen similar issues before. Such occasional times of unexpected relative low cluster utilization, they are generally an indicator for some performance bottleneck in our task scheduling. And they are by an extent also an opportunity for us to further increase the efficiency of our framework. While our team was still investigating the low cluster utilization issue, a stakeholder reached out to us about another problem. They had observed that the total runtime of their task was near twice as long when executed by our framework than when run locally. And while some overhead is expected because we have integrated data management, the scale of the described issue, it also prompted us to investigate. We will now follow the steps that my team and I took to analyze those issues, identify which parts of our system were performance bottlenecks and which of those would be suitable for migration to Rust. We will start with the cluster utilization problem. Let me briefly describe to you how our task scheduling works, really on a high level here. A microservice, which we call the scheduler, receives so-called resource offers from the cluster. Such a resource offer originates from one of the hosts in the cluster and describes the attributes of that server. Think of things such as the compute resources, CPU and memory. The scheduler will periodically determine which of the tasks it knows are processable and then try to match these tasks to the resource office. Once a task has been assigned to an offer, we make sure that all required payloads or data the task requires is available and finally we launch the task in a Docker container on the host. The problem that we observe is a low cluster utilization, despite having a high number of processable tasks. That means that the part of our code that we should be looking at, well, it starts with retrieving the list of offers from the cluster, and it ends with the part where we actually launch the task on the host. And what you see on the slide, it's actually a small part of the visual representation of OpenTelemetry traces. Let's now briefly talk about each of these spans. And even though this is a talk about migrating code from Python to Rust, the question, should I build this with Rust, is actually one that's best left for last. In my opinion, it does not make much sense to talk about a solution before one has understood the root cause. The first span you can see here is for the getResourceOffersFromCluster function, and it's already quite fast, a couple of milliseconds, so let's just move on. The second span is for the getProcessableTasks function. It performs the necessary database queries to compute the list of processable tasks and retrieves their data from the database. As the data is stored as a serialized JSON string, we must first deserialize it on application side before we can actually work with it. And as you can see, it's this deserialization step that takes most of the time, over one minute here. And we could jump ahead now and say, hey, let's use Rust to make this faster. But maybe we should first try to understand the root cause. In this case, there are other factors that play a role than just the computational effort in desalizing the data. For instance, how large is a single CLS task? Does it maybe contain lots of redundant information? How many tasks are we even retrieving from the database with each iteration? And do we retrieve the same task maybe multiple times? These questions, and others like them, must be answered before we should even start to think about Rust. And in our case, we had lots of redundant information per task. Each iteration could retrieve well over 100,000 tasks, and most of these tasks would get retrieved over and over again before we would finally be able to put them on the cluster. By fixing these issues, that is, getting rid of the redundant information by improving the serialization format, limiting the number of tasks that we retrieve per iteration, and caching deserialized task data on the application side, we were actually able to reduce the span from over two minutes down to 32 seconds. And this not only got us a step closer to our goal of improving task scheduling overall, But as a side effect, it also more than halved the size of a database table from three terabytes down to one. And I think there's a first lesson learned here. Just because we might have a silver bullet, Rust in this case, it's still worth the effort to try and understand the underlying root cause before we jump to solutions. Or to put it bluntly, just because you have a beautiful hammer, it does not mean that every problem is best solved by hitting it on the head, right? And other reasons why Rust might not be the best choice here include the fact that Rust does not yet have a true SQL alchemy equivalent, and also that others have already done the work for us. For example, with fast JSON parsing libraries that are written in Rust, such as OJSON. The third span is for the match tasks to offers function. It takes the list of processable tasks and the list of resource offers and then computes a numerical score for each offer. The lower the score, the better the offer fits the task. And offers that don't fit, well, we just put them aside for lower priority tasks. While this is the part where we spend most of our time, each individual child span is quite short. It's just we have a lot of them. We did ask ourselves here similar questions as before, like how many tasks do we have, how many offers do we have, are they still valid, are there maybe duplicates? Well, none of the answers here showed any issue. So we started to actually look at the matching implementation. And spoiler alert, that's the part that we've migrated to Rust for major performance improvements. But before we get there, let's talk about the last span. The fourth and last span is for the prepare payloads function. As I've mentioned earlier, our execution framework offers integrated data management. That means that any data that your task needs in order to run or any data your task might use, we take care it gets where it needs to be, when it needs to be there. And this is the place where we make it happen. In practice, that means that we sequentially send HTTP GET and POST requests to our data management microservice. The duration of these requests is well within the expected range. The problem is just we, again, have quite a few of them. Now, if sequentially doing something takes too long, the next best thing is to do it in parallel, right? Well, Python has what's called the global interpreter lock, as I'm sure all of you know, the GIL, which does prevent true parallelism. Python 3.13 adds the free threading mode, but even today, this is still an experimental feature and not all libraries support it, and back in 2023, that was not really an option for us. So is this another part that we should move to Rust? Well, we've decided against it. By the use of Python's multiprocessing module, we parallelized the sending of these HTTP requests without having to worry much about the gill and without having to switch away from Python. Instead of spending multiple minutes on sequential requests, we now handle up to 16 requests in parallel and managed to reduce the time spent here to an average of 23 seconds. And as you can see, we have again decided against treating Rust like a silver bullet for all our problems and instead opted to stay with Python. And I think that's another lesson learned, quite similar to the one from earlier. Whenever possible, we should try and utilize the capabilities of our existing tool set to their fullest potential. Migrating parts of your application to Rust, it's not just a question about how well a certain problem could be solved with Rust, but also about the effort involved, both in terms of the initial migration, but more importantly, later on, when it comes to maintaining the Rust code in the long run. Team members must be willing and motivated to learn the new language and commit to it. Rust is great, but maybe tune your engine before switching cars entirely. Let's dive deep now and see how we went about migrating offer matching code to Rust. Our first step was to recreate a simplified version of the match tasks to offers function. As you can see, it takes a list of tasks and a list of offers and computes a mapping that it will return from offer name to task name where the offer is the best fit offer for the task. For each task, we iterate over the list of offers and compute a score if the offer fits the task. We keep track of the best offer score, and before moving on to the next task, we assign the task to the offer with the best score and then remove it from the list of offers. To verify that this proof of concept here works, we could have taken our production data for testing, but we decided to write a script that generates us some sample task can offer data. And we'll actually see later on why that was a good choice. We can use PySpy to generate a speed scope of our matching function and thus get the runtime of all involved functions. And with the spend durations from our traces as our benchmark, we can also make sure that our POC is not oversimplified. With an average of 83 milliseconds and 2 milliseconds for the respective functions, we are somewhat Not faster than our benchmark, but not so much faster as to have us worry about having missed something. At this point, we were not sure where the large difference in total time came from, but we decided to move on for now. As we've seen with the prepare payloads function earlier, trying to parallelize can be an effective way to increase performance. So let's do that. We have here a function that uses a multiprocessing pool to compute the offer scores in parallel. That means that we still iterate sequentially over the tasks, but we split the work of computing an offer score onto multiple threads or processes. And first, we will actually be using a thread pool, mainly as a way to confirm our theory that it is the GIL contention that's holding us back when working with a single Python process. Could I please get a quick show of hands who thinks that our theory is correct and that it is the GIL that's blocking us in a multi-thread approach? In other words, who thinks that the multi-thread approach will be slower than the Python single-thread approach? Okay. Only a few people. That's interesting. Our theory was correct, actually. The total time increased more than tenfold, but also the individual times themselves increased. And the speed scope shows us, and I hope you can see it with that light, there are gaps. You're supposed to see the gaps. The speed scope shows that the Python global interpreter log prevents true parallel execution, and the individual threads, well, they are blocked from execution unless they have the gill. Now, let's move on to a process pool, because that will effectively ignore the gill, and see how that works out. Again, please give me a quick show of hands if you think that the multiprocessing approach will be faster than the Python single thread approach. Okay, that's much more people. It's actually even slower. At least when it comes to the total execution time. Why? Well, because of inter-process communication. The current implementation is not very smart. And for each task, it sends the task itself, but also all available offers to the processes in the pool. And while we could improve this, for example, by improving the data structure that we share between the processes or just the way how we share the data, we should also consider that adding more and more Python processes also induces additional resource demand. And compared to the prepare payloads function from earlier where we did choose Python multiprocessing, well, the degree of parallelism that we would like to achieve here, it's much higher. And as this talk is about moving Python code to Rust, let's talk about Rust. We have translated the single-threaded Python implementation to Rust. And we're using PyO3 and Maturin to produce a Python module that we can import and call from our Python process. The implementation is essentially the same as the Python one, just a different syntax. Now last show of hands for today, who thinks that this Rust implementation is now faster than the Python implementation? Okay, that's great. Most of the people are convinced Rust is faster. And you're right, it is. At least the functions themselves are. I mean, the total run time, we're still at two seconds. That's like the Python single-threaded approach, right? Well, the Rust implementation is faster, but calling into Rust from Python, that adds a certain not always negligible overhead. Sending data across the language barrier, our task and offer data in this case, it can become quite expensive depending on the amount of data that you send around. The multi-threaded approach in Rust, we're using Rayon for those of you who know, suffers from the same problem. But on top, the parallelization, well, it also comes with some overhead. And that further reduces the total run time. And that's actually where it pays off that we chose to write a script in the beginning that would generate us arbitrary test data. So let's scale up, so to say, increase the number of tasks and increase the number of offers that we try to match. Please take a moment and think about what you expect from both the Python and the Rust implementations. Now, our Rust implementation is much faster, and by a lot. The single-threaded Rust implementation is actually over 30 times faster than the Python equivalent. And the multi-threaded implementation is almost 50 times faster than the single-threaded Python implementation. And as you can see, we are also much closer to the benchmark time that we had initially. And not only are the Rust implementations much faster, but they also scale better. I mean, while the runtime of the single-threaded Python implementation increased more than a hundredfold, the runtime of the Rust implementation, that only tripled or rather doubled, right? And I think we have two lessons learned here. When working with outside dependencies, that could simply be data as in our case or it might be an HTTP endpoint, anything really, we should make sure that we have an easy way to modify it and play around with it. And we should play around with it. Change the data structures, change the amount of data, and so on and so forth. And the other lesson learned, the other lesson is scale. In our production scenario, we actually have a much higher number of tasks and offers than what we showed initially. And in the future, that scale will most likely just keep increasing. Everything comes at a cost, and migrating parts of your Python code base to Rust for performance reasons, that might only be worth it if you have the scale to counteract these costs. Now, while this POC is very similar to our production setup, it is still missing one crucial aspect. You see, our Python process is not just doing offer matching, it is doing lots of other things too. So, let us look at the numbers again after we have added some background threads. And let us not only look at the performance of our scheduling, but also of these background threads. As we can see, Python implementations got slower, except for the multiprocessing approach, of course. Rust implementation, on the other hand, fast as ever, right? Well, what about the background threads? They basically stopped. And why? Well, it's simple. We forgot to release the GIL. So while our Rust part is doing just fine, every other Python thread is blocked until the Rust part finishes. And not only does this deteriorate the performance of other background threads, but if one of these background threads is, say, meant to answer HTTP requests for health checks of your application, the entire application might get killed, right? Here's how this should look like. PyO3 makes it really easy for us to release the GIL. I hope you can see, otherwise, the slides on GitHub. And while that does actually decrease the matching performance somewhat, the impact is negligible. I mean, we're talking about milliseconds here. The Python background threads, on the other hand, they are as fast as they can be. And I think that really shows that moving from Python to Rust, or honestly to any other language here that allows you to drop the gill, it will not only improve the performance of the part you moved, but most likely also of the entire application overall. With that, we have successfully improved our task scheduling performance by reducing the entire matching duration from over eight minutes down to less than one. And we've also managed to increase our cluster utilization from 60 to over 70 percent. Let's get back now to the other problem, the data management overhead. As our production cluster is spread across multiple cloud providers and thus physical locations, we have dedicated cloud storage solutions for each location. And once the Docker container for the task is launched on a host, well, all the data that's required for the task's execution, it first needs to be downloaded from the nearest cloud store to the host before we can actually do some work. And after the task execution is done, we still need to upload any data that might have been produced. Think of block files or test results or binaries. The problem that we observe is a long delay between container start and workload execution start, as well as between workload execution and container shutdown. In other words, we have a poor upload and download performance. And that, of course, means that we should look at the part of our code that handles data up and download performance, but it also means that we might want to look at, let's call them external factors, such as the read and write speeds of the local disks, or maybe the network bandwidth between host and Cloud Store, and maybe even the rate limit of the Cloud Store in question. Let's start with what I call external factors. Based on our log files, we could calculate that we have an average data transfer rate of 480 megabytes per second. That's actually nowhere near the limits of neither the host local disks nor anywhere near the limits of the cloud store. And based on monitoring data from our network team, we could also confirm that we are not reaching the network bandwidth limits. So it is our fault that things are slow. Similar as before, we have a PLC that recreates a simplified version of a production setup, In this case, copying data on disk. And we have, again, a script to generate random data. As you can see, the Python and Rust implementations, they are very similar in terms of performance. And while using asynchronous code to read and write concurrently does increase our throughput, the difference between Python and Rust, it's still negligible. And that's because the main limitation here is I.O. Rust offers an easy way to use the IOUring Linux kernel API, namely GLOMIO or GLOMIJO, I'm not sure how they want it to be pronounced, and that does significantly improve performance. But is simply using IOUring already the best we can do? I mean, Python most likely also offers a good IOUring abstraction, right? So why go with Rust? And to answer that question, again, we must try to better understand how our simplified setup differs from our production setup. Our production setup is really copying data on disk. It's either reading from disk and writing to CloudStore or it's reading from a CloudStore and writing to disk. And does also help with socket read and write calls, but as we've seen before, Rust truly excels when it comes to parallelizing things. And in our case, most of our CloudStore solutions actually offer an S3 API, So we can split our data into chunks and download and upload those chunks in parallel. And I think we have another lesson learned here. As the only thing we need to pass between Python and Rust in this case is a storage URI, well, the overhead of calling from Python into Rust is quite small. So use Rust where parallelism matters and costs of entry are small. When dealing with large amounts of data, the cost of crossing the language barrier, it might not be upset by the performance gains of Rust, unless you have the scale, as we've seen before. But in this case, well, when your workload allows for high parallelization, and little to no data needs to be passed around, moving to Rust will increase your performance. Now that we have two working PLCs, one that improves our task scheduling performance, and one that improves our data management performance, let's talk about how to get them production ready. Had certain character from the Lord of the Rings been with us at the time, he might have told us one does not simply deploy to production. But when we saw the potential in our Rust implementations, we initially got a little carried away and we were overexcited and maybe focused a bit too much on getting it out in production quickly. The end result was a poorly designed API that needed multiple breaking changes and some very stressful bridge calls, during which we found out that our breaking changes are now breaking our workload. And there are, again, a few lessons learned here. When designing an API, think about the interface, not only from a usability point of view, but consider also how future changes might break it and how this will be dealt with. Or even better, how it can be avoided. And while it might double the test scope for a while, make sure the new implementation is at least as well integration tested as the old one was, if not more. The pitfall that we ran into was that we forgot or rather we were ignorant of how much Python and more specifically the libraries that we were using actually did for us, especially when it comes to handling certain quirks of APIs, the S3 API, for example. End-to-end integration tests will save you here. And while it might be obvious, I think it must be said. said, write your unit tests as close to production as possible. We had a bug during as part of our data management that would cause a rust panic because we called a function that was expecting a Python byte array, but we gave it a memory view object. And our unit tests, they made sure byte arrays are supported. But our production code just wrapped our rust object in a Python buffered reader and that one internally uses memory view. We could have just tested with the Buffett reader, but, yeah, we didn't. Especially integration tests for the Python to Rust interface should cover every aspect. For instance, the initialization of the observability stack in a Rust extension was misconfigured for a while, and it left us blind for a couple of hours before we actually found the issue. And while we are on the topic of observability, in a Python environment, were used to Sentry events and OpenTelemetry traces. But how does that work in Rust? And how can we integrate it with Python? Well, both OpenTelemetry and Sentry offer official Rust SDKs. So that's easy. And the Rust tracing crate from the Tokyo project is not only very powerful in its own right, but it also can be taught to speak OpenTelemetry. And configurations such as a Sentry DSN or ingest keys, anything you might need there, can easily be made available across the language barrier via environment variables. What becomes interesting, though, is when we want to be able to follow our execution flow across the language barrier. Here's what we did. Say we have this do work function, which is a Python function that is being traced. It calls another Python function, but it also calls a Rust function. Ideally, our traces would look something like this. The Rust function and anything it might call is available in the same way as a trace Python function is. It has the correct parent span attributes and it is in the correct trace context. While we could achieve this by just passing all the relevant information, such as the current span ID, into it, that would not be very satisfying, right? Well, what we did instead is we used PIO3. See, PIO3 does not only allow us to call Rust code from Python. But also the other way around. We can call Python code from Rust. Let's take our single threaded matching implementation as an example again. We have here the instrument macro, which is basically the same as the tracer decorator in Python. It comes from the Tokyo tracing crate. And the function body also remains mostly unchanged. We just added a call to the set Python span as parent function. And that's where all the the magic happens. And I don't think that it makes much sense to give you a detailed walkthrough of what that function does. I still have included it in the slides, so if you want to have a look at it later, feel free to do so. For now, let's just quickly go over it and see what it does. We first get the current tracing Rust span. And here we can already exit early if tracing is disabled. Otherwise, we continue. And we retrieve the current Python span context. How do we do this? When we first call this function, the function calls from Rust into Python and imports the open telemetry trace Python module. The module is cached so that later calls don't need to be imported. We then call the necessary Python functions from that module so that we can get the current span context, the Python span context, that is. And from that, we can then extract trace ID and other relevant information later on. If the trace ID is zero, that simply means there was no active Python span context. And we can again exit early. If there was, we build a Rust span context from the Python span. By setting the is remote flag to true, we tell OpenTelemetry that this context was propagated from a remote parent, our Python application in this case. And lastly, we just create a new Rust OpenTelemetry context from that remote span, and we configure it as the parent context of the current span. The beauty of this approach is that the caller, it doesn't need to care about tracing. It just works. And the same approach can also be applied when calling Python functions from a Rust application. Let's move on now and come back to some of the mistakes that we've made and some of the unexpected issues that we've encountered in the past two years. A very basic thing is how to build the production wheel. When we started with our POCs, the speed of the Rust implementations did not really live up to our expectations. Using the release profile when compiling the Rust code is what helps. I mean, obvious, right? Another thing to watch out for is version compatibility. Our Python application supports all Python versions from 3.9 onwards. But certain PyO3 features that we wanted to use, they required a newer Python ABI. So make sure that the PyO3 and Rust C versions are compatible, yes, but also make sure that everything still stays compatible with the oldest Python version that your code is still meant to support. When we introduced the Rust extensions, we made sure to guard the new code paths with feature toggles. This way, as long as the toggle stays off, the old implementation is used and everything just works. And once we turn it on and things go sour, we can always turn it back off again, right? Well, that's what we thought. We spent multiple days with the new Rust extension enabled, and everything just worked fine. On Friday evening, we checked our monitoring one last time. Everything still green. Everything running smoothly. It worked. Well, until it didn't. Monday morning, our production cluster was down since 36 hours. While the feature toggle did help us to quickly mitigate the issue once we've become aware of it, it did not prevent the outage over the weekend. Maybe we went a bit overboard, but we have a lesson learned here. Prepare for failure. Anything that can go wrong will go wrong. In addition to the feature toggle, we also wrapped any calls to our Rust extension in an accept exception block. This way we make sure that we get sent events and logs about the issue, but we also make sure that our legacy implementation jumps in and prevents the outage. That will of course be removed from the code base alongside the legacy implementation once we are confident of the new implementation. And yes, having unit and or integration tests that cover all possible cases is preferred, but the reality is you just can't foresee every problem. Especially during the migration phase, keeping the legacy implementation as a fallback, that's worth a hundred feature flags. Now why did our implementation break over the weekend? Because it was too good. Turns out if you manage to put much more workload onto your cluster and parallelize most of the data transfer, reaching your network bandwidth limits and overloading a cloud storage system, That's easier than you think. For eight hours, we were maxing out our routers, causing large-scale package drops, not only for us, but also for other SAP internal development teams. We also overloaded the cloud storage system with more than 100,000 parallel requests per minute for multiple hours. That resulted then in such fun things as data getting overwritten by out-of-order request processing. We are since then in very close contact with the vendor to get that fixed. And while the issue was at first caused by how well our system performed, the reason it persisted as long as it did, that was partly our fault. You see, we had configured a timeout of 30 seconds for most HTTP requests, and that meant that we would keep bombarding an already overloaded cloud storage system with retry after retry if one of these requests takes longer than 30 seconds. And the last lesson learned for today. Prepare for failure, yes, but do also prepare for success. And more importantly, prepare for success to turn into failure. Know the theoretical limits of your systems and try to understand what might happen once you reach them. As a result of the major outage that I just described, we have extended our network bandwidth to over 20 gigs, and we are onboarding additional cloud storage providers to help distribute the load. It was also an important reminder for us to have proper timeout values and retry strategies in place. Now that we've reached the end of the journey, let's recap. Everything I've just shared with you was our journey to drastically increase our task throughput. When we started our journey in 2023, our framework managed to execute 170,000 tasks per day, And we had an all-time high of 380,000 tasks. In 2025, we now have an average of 340,000 tasks per day and an all-time high of almost 1 million. And we've achieved this not by forsaking Python and hailing Rust as our savior, but by strategically analyzing our performance bottlenecks and choosing the best tool for the job. And I hope that you will be able to draw from the mistakes that we've made and the lessons learned that we've derived from them, so that you can improve the performance of your applications in a similar way. Thank you so much for your time.

Speaker 2 [37:57]

I think that is not enough. You can still clap hands for Max. That was an amazing presentation. Please, let's do it again. Thank you. Let's slide into Slido. The audience has some questions for you. So the first question is perfect voice and storytelling for YouTube tutorials. Very nice. That's your first question.

Speaker 1 [38:27]

Thank you.

Speaker 2 [38:36]

So the second question is Was there a reason for using multi processing over usually more lightweight and economic essential to concurrently perform HTTP requests

Speaker 1 [38:50]

I'm sorry. I did not get that

Speaker 2 [38:52]

Was there a reason for using multiprocessing over usually more lightweight and economic essential like async?

Speaker 1 [39:00]

Yes, okay. Yeah, I got that.

Speaker 2 [39:01]

Yes. Yeah, I got that. To concurrently perform HTTP requests. Yeah.

Speaker 1 [39:04]

I think we have seen that in a talk yesterday, that asynchronous code can help you, but it still most of the time operates on one thread. And in our use case, our main application that is handling those HTTP requests, it has more than 200 background threads already, and if you have just one thread that is trying to do asynchronous HTTP requests, that still will not help you, and adding more background threads will also not help you. That's why we chose the multiprocessing approach.

Speaker 2 [39:34]

Good. The second question is, what was the impact on memory usage?

Speaker 1 [39:41]

Ooh, that's a good one. Maybe contrary to popular belief, I guess, it went down. You see, in Rust you have fine control over how to organize your memory. And that makes it very easy to reduce the memory footprint after applications.

Speaker 2 [40:02]

Okay What are good data structures to pass to rust do pandas data frames work?

Speaker 1 [40:11]

They might surely work, yes. I think it's less about the data structure that you're parsing, but more about the size of it in total. And also what you're doing with it later on. I mean, maybe to give you a rough example, when I, maybe I can go back to that slide. When we increase the number of tasks, right, where is it, there it is. When we increased the number of tasks, the size of that thing, I mean, 20,000 tasks, that was like 10 megabytes. And 20,000 offers was like another 20 megabytes or so. It's quite a lot, I would say. But since we were able to parallelize a lot of the computational effort on Rust's side, it was still worth it. So yes, crossing the language barrier, the time it takes to do so increases with increased data size, but if in the end you can really benefit from Rust, it might still be worth it. I think the most important thing to take away is try to understand what is really limiting you and then build POCs, play around with data and try to get a feeling for what can help your specific use case.

Speaker 2 [41:26]

How would you compare the profiling benchmarking tools between Python and Rust? Currently, I mean something like Criterion, Cargo, Flame, Graph in Python.

Speaker 1 [41:38]

Okay. I mean, for CargoFlameGraph, there is PySpy for Python. So I think that would be a good alternative. And that's actually what I've used. I've used CargoFlameGraph on the Rust side and PySpy on the Python side. And I've also just used the built-in tooling of Python's standard library, which allows you to kind of measure CPU times and also memory consumption. Yeah. For other tools, honestly, I don't have any recommendations.

Speaker 2 [42:07]

All right, let's take two more questions What was the motivation to implement the application in Python in the first place? Go and Rust would also have been good choices for the whole microservices

Speaker 1 [42:20]

Good point. We started that thing in 2016, and we wanted to get it out in production quickly, so Python was our way to go. Back then Rust was not yet as popular as it is today, and I think we just had a preference for Python over Go.

Speaker 2 [42:41]

Okay, did you run into problems with the features of Pio 3 was there something not supported? What do you need it?

Speaker 1 [42:49]

Yeah, the memory view and PyByte array thing, for example. As I said, we support from 3.9 onwards. For 3.9, you need PyO3 does not out-of-the-box support memory view, because that only comes in with the buffer protocol, which needs a higher Python ABI version. So we actually had that problem, and it broke our production system.

Speaker 2 [43:15]

Okay, let me take the last question Which tools did you use for measuring the performance of the implementations? How did you interface rust in Python?

Speaker 1 [43:26]

As I said, for the performance measuring, mostly PySpy and CargoFlameGraph. I hope that answers the question. If you have more questions, honestly, just come talk to me later, or you can also write me on LinkedIn or wherever. Happy to talk to you.

Speaker 2 [43:46]

Of course, let's give it up again for Max, please, for an amazing presentation.