Speeding up Python with Zig

Speaker 1 [00:04]

Hi everyone, I'm Adam, software engineer at Delivery Hero, and the title of my talk is Speeding Up Python with ZIG, a tour through ZIG, the Python C API, and packaging challenges. Now, before preparing for this talk, I had never written any ZIG, and I wanted to do a quick show of hands to find out if anyone here has written any ZIG before. Okay, so a couple of people, but what about C? Has anyone written any C in, say, the last year? Okay, most people. Actually, I don't expect any knowledge of Zig we'll see in this talk. I will mention pointers a few times, but you don't really need to understand what they are. But first, a confession about the title. If you read the description in the program, you should understand what I'm about to talk through. But when I say speeding up Python with Zig, I, of course, don't mean the Python programming language itself, but what I will demonstrate is speeding up the Python library ecosystem with Zig, and specifically with extension modules written in pure Zig that would normally be written in C. So the full title might be speeding up the Python library ecosystem with extension modules written in pure Zig, but as that's a bit of a mouthful, I stuck with this snappier title. So a natural question to ask at this point is why. And to motivate this talk, I'm going to mention something, as this is a sponsored talk, something which is important to companies. I mean, there's all sorts of justifications for making applications use less CPU and memory and work faster, for example, environmental. But for a company, a big reason is money. This is a Slack channel at Delivery Hero called Performance Wins, where teams post latency and resource consumption reductions that they've achieved. And our CTO, Christian Hardenberg, is in this channel and will regularly comment on these improvements. And why is he in that channel? I can't really put any better than Christian himself. Performance wins are cost wins. So we use Python at Delivery hero in a number of different places, obviously data and machine learning, but also image serving, delivery areas, customer support, chat, marketing, Q-commerce, checkout, somewhere in the region of 10 billion HTTP requests per month are handled by a Python backend. So any kind of improvements we can make to Python application performance actually saves us money. The other part of the question why is why Zig? And judging from the show of hands earlier, you may also have the question, what is Zig? So it's a compiled system language. In the words of its creator, Andrew Kelly, Zig is a system programming language intended to replace C, which is a pretty ambitious goal. And I think to realize that goal, Zig has a few killer features to interop with C as to replace C, you probably need to interop with the mountains of C code that exist in the world. And I will just quickly talk about some of those features now. The first feature is that you can directly import from a C header file. So the C import built-in function can be used to directly import symbols from header files, a header file being, of course, a file with extension.h, which contains C function declarations and macro definitions. For example, Python.h, which is a file that ships with Python and exposes a C API that we can call. And this is a snippet of Zig here. We're declaring a const variable pi and importing the Python header file, while also defining a macro in line 2 and assigning it to Py. Now one cool thing about this is since Python 3.2, we've had something called the limited API, which is a subset of Python's C API. So any extensions that only use this limited API can be compiled once and work with multiple versions of Python. And that API is nicely documented in the Python documentation. So second feature. Here we have a function in Zig that can be called from C with the call convention C style, which allows Zig function to be called from C. It has primitive types for specific ABI compatibility with C. Shorts, ints, longs, et cetera. Even has C pointers. Even though you're not supposed to use these, Zig can automatically translate C code, which which, of course, it needs to be able to do to understand a header file. But there are some differences in the way pointers work between ZIG and C. For example, ZIG doesn't allow null, or it has an explicit optional pointer type, and it has a different syntax for pointers to arrays rather than pointers to single items. But the ability is there to declare a C-style pointer. And finally, you can even use the zig-cli to translate C source code, including macros. So I actually used that functionality when I was making this demo to understand how zig viewed the Python header file. So I've been talking a lot about zig's ability to interrupt with C. So why not just write extension modules in C? I mean, the silly answer to that question is then the talk would have been called Speeding Up Python with C, and I also like to make life really difficult for myself, so that's why I decided to talk about Zig. But more seriously, I think Zig has a few kind of modern conveniences that differentiate it from C, and I secretly kind of hope there might be a renaissance in writing extension modules in Zig rather than C. So, for example, it has a rich and comprehensive standard library. That's something I think Python really benefited from. An official package manager soon. Zig is only in version 0.9, and the idea is in version 1 there will be an official package manager, so it will be easy to build code from other dependencies. Has a cross-platform tool chain and build system, so they take cross-platform building very seriously. This is an interesting post by Andrew Kelly there where he basically downloads the Windows version of Zig, runs it in Wine to compile a Linux binary, and then runs it. And finally, it has no macros. Maybe that's a slightly controversial thing to mention as a feature, but I guess it's Andrew Kelly's belief that macros do not aid maintainability. Various warts removed. Well-timed execution, which is a bit like C++ templates, but nicer. Optional types, discouragers and all pointers. Just various modern conveniences. And one final point is I think Zig shares some philosophical and cultural alignment with Python. This is the zen of Zig, which you can print out from your terminal by typing zigzen. A few of these line items remind me greatly of Python. For example, communicate intent precisely. It could be explicit. It's better than implicit. Favor reading code over writing code. Readability matters. Only one obvious way to do things. That's familiar. And I rather like the last one, together we serve the users, which is a nice reminder of why we're all here. So hypothesis of that lengthy intro, we can and should continue to improve performance of the Python library ecosystem, and Zig can help us do that. So now we'll get into some more technical details. Of course, this isn't just a talk. There is a repository that you can go to in my GitHub account called XAML, and I've structured three pull requests there, kind of in the order that I'm going to tackle things. Firstly, the README, I mean, it is my intention that this would actually be a fully functioning YAML passing library. Currently, I haven't tested it on different operating systems, but it is written in pure ZIG. It does import the Python header file directly. It is only compiled using ZIG. And yeah, let's just dive into what the simplest possible C extension is, and then I'll show the ZIG version of that, and we'll pretty much see it's not that different. So the additional boilerplate to write the ZIG extension is not very much. So, of course, to begin with, we import the Python header. Here we've got a function that returns a PyObject pointer. On line 7, we're returning an integer with the value 1. That's all our module does. It returns from the load function an integer with the value 1. Next, we have the methods of the module defined, the method's name, the flags. Meth, no args, is a macro which tells Python this function shouldn't expect any parameters. and some documentation for that function, and a sentinel value which says that's the end of the methods. And as I said, this is all part of the limited API, which is rather nice because we can depend on this being stable across different versions of Python. Then we have to define the module itself. So we pass the methods to this module def struct, struct assign this macro pi module def head in it to the base the other properties don't really matter and finally create the module itself and one final ingredient we need to compile that and Python even provides reasonably nice way of doing that without even having to worry about clang or GCC we can just declare an extension with the the C source file, give it to the setup function, and then when we run pip install, it just works. And what that's actually doing is two things. It's building a shared object file for the module, which is a binary file that Python can understand, and it copies this shared object file into the site packages directory of the Python environment. And then you would be able to import that and assert that, indeed, the load function returns 1. So how does it look like in Zig? Very similar. Here is the snippet that I showed earlier. We're importing the Python header and defining the PySizeTclean macro. This is additional boilerplate. It's also not necessary, but unlike C, the symbols in the header file are not kind of implicitly in scope here, so I've taken the parts of the Python header file that I'm using and assigned them to const variables there. Here we have the load function that we had before, but it is slightly different. For example, Zig understands that the way Python will call this function means that the self and args pointers could be null, so we have have to declare those pointers as nullable with the question marks, and of course the call convention needs to be C-style, but the return type is the same. It's a PyObject pointer. Another difference between Zig and C is that Zig doesn't permit unused variables, which is why we're assigning self and args to these underlines, and line 17 is pretty similar. We're returning a PyBuild value, building an integer, casting the 1 to a C-style integer, And yeah, that function does the same. Here again, it's very similar. We're just declaring an array of PyMethod defs, again, using the macro meth no args that Zig can use, and Python will understand that the load function doesn't take any arguments. Here we get a slight difference. So if you recall from the C version, our base, mbase of our module definition had the PyModuleDef head init macro assigned to it. When we try and do that in Zig, we get this nasty error, compile error, unable to translate C expression, unexpected token L brace. So what's happening there is Zig's ability to understand macros is not perfect. It's also improving over time, so there is a path to enabling it to understand more macros, but in this case, it doesn't. However, it was fairly easy for me to go into the Python header file myself as a human being and understand what that macro is doing, and essentially what I've done here is inlined what the result of the macro would be in the module definition. And finally here, we create the module, export the function so it's visible from outside. Again, there's a slight difference from the C version, because Zig also cannot translate this PyModInitFunc macro, but as it turns out, that just collapses to a PyObject pointer. So it's easy to kind of inline the result of that macro. So one final ingredient, pass XAML module.zig to our extension, get ready to install, and Of course, that doesn't work. Python has no idea what zig files are. Error unknown file type dot zig. In fact, the idea that you would be compiling with something C-like is fairly deeply embedded into the assumptions of setup tools. When you get to the heart of setup tools, there is this class here, the C compiler, which I initially thought maybe it would be a good idea to subclass this, but as I said, the assumptions are pretty coupled to C, but we can subclass at a slightly higher level, which is what I've done here. So the build-ext command is subclassable, and here I've written a little zig builder, which basically just does three things and uses some of the methods and properties that you get from the build extension command. So for example, by subclassing this command, I know where I need to build the shared object library. I know if it's a debug or release build. I know where the full expected path and file name to actually build the shared object file is. And I know which directories to include when I compile in Zig, so where can Zig find the Python header file, for example? And by plugging that into our setup.py file and overriding the command class of the build extension with ZigBuilder, we do, in fact, have an installed module that, as before, we can call the load function and get the integer 1. So that's all very well and good, but it's just a module that returns one, so not super useful in real life. I kind of wanted to see whether this applied to a real-life problem could yield some speed benefits. And the problem I chose to apply it to is YAML, which doesn't sound very exciting. and i wanted to explain why i chose that problem spoiler alert i it does work but this is the delivery hero website in singapore foodpanda.sg this part of the website the help center which is embedded in the apps as well is a python back end and it's essentially a tree structure that lets the user either resolve a problem themselves or finally drill down into a chat or a case. And that help center is embedded in 50 countries, 84 different languages, five different use cases, customers, vendors, riders, devices, and shoppers, making about 700 distinct flows or 700 distinct versions of that trees, all with their own fairly complicated logic embedded in different rules for when a user is allowed to create a case or a chat or refund an order. And the way we manage that is in a back office, and you guessed it, that tree is defined in a big YAML structure. Some of them can be hundreds of thousands of lines long. So in In summary, the team I work on passes a lot of YAML, and one of the slowest endpoints in that application is the endpoint which validates that this YAML structure is correct. And having dug into why that is, it turns out, actually, the slowest part is just passing YAML, which was somewhat surprising to me, because PyYAML is written in C, it's installed in the correct way, so I was not expecting it to be that slow. I mean, I wouldn't think that 100,000 lines of YAML should take significant time to pass, but that's what I got. So I picked this problem to apply my prototype. So this is a proof of concept. It's in the GitHub repo. I don't want to read this code too closely now. It doesn't handle a lot of edge cases. It only handles one particular structure of YAML and may have bugs. So please don't use this in production, as I've liberally commented here. And most of the credit, all I've really done is written the kind of Zig C interop or Zig Python interop part. The actual YAML parser itself, the credit goes to someone else. And that YAML parser, just to show the kind of, I guess, immaturity of Zig right now, even that YAML parser is not 100% complete. For example, it doesn't handle comments. So then having kind of mashed together the Zig YAML library and my Zig Python interrupt stuff, this is a little benchmark script where we create a million-line YAML file. and pipe it into PyYAML and our prototype YAML parser written in Zig and then assert that the result is indeed the same dictionary back in Python land and happy to report that, yes, XAML, this proof of concept YAML parser is very fast. Also, yesterday did a little bit of benchmarking against some other alternatives, the non-C version of PyYAML and RooAML. And I wouldn't want you to take away from this talk that PyYAML and RUAML are slow. This proof of concept doesn't handle many edge cases and would be slower in practice, but I suspect not ten times slower. And therefore, I will just leave then with the conclusion that we can and should continue to improve performance of the Python library ecosystem. I think problems that maybe seem well-solved or libraries which appear to be, you know, essentially coming into this talk, I wouldn't have expected that there could be many speed improvements you could make in a YAML passing library at this stage in the Python ecosystem, but maybe that assumption isn't always true, and Zig is a good approach to help us do that. has some promising attributes that make it interesting. Thank you. My name is Martin. I will guide you through the questions. There was three times a very similar question. Why can we not use something like Cyton? We can, yeah. Or it was also about Rust or Citon, basically. Yeah. I picked Zig for this talk because I was interested in the philosophy of Zig. And of course, there are lots of libraries that use Citon and Rust. So there's ORJSON, which uses Rust for passing JSON, CYTON, I think UVLOOP uses CYTON to bind with LIBUV. So there's probably libraries that everybody's using in production that use those approaches, but I just wanted to talk about an alternative, and I think Zig has some interesting qualities. So yeah, and in the words of its creator, it really is intended to replace C. There's another question. Do you use it in production? If yes, does the SIG introduction and usage in production run against the team performance in terms of velocity? No, we don't use it in production, no. Delivery Hero very kindly gave me some time to just work on this to make an interesting talk topic. Yeah. It's my secret hope to get this library production ready, publish it on PyPy, and use it for parsing YAML in production. So maybe if I come back in the next years, I can tell you whether I've been successful doing that. Maybe one last question. Can you tell us more about SIG? What are some of those interesting qualities? Interesting qualities that differentiate Zig from C. I don't know what the question is intending, yes? Yeah, well, it's, I think it has a few safety features that C doesn't have. So nullability of pointers being one or explicit optional types, error unions. The compile time execution is interesting. Essentially, you can define a block of code with the comp time keyword and that code will be executed. And it's no different from ZIG code. It's just ZIG syntax. And that basically gives you the flexibility to do anything that you do with C++ templates or advanced types in a higher-level language because you can literally just write directly ZIG code to enforce whatever you like at compile time. So that's one feature. The memory allocator feature is quite nice. You can really finely control how memory is being allocated and used and buffered. What else is interesting about it? I mean, to be honest, I'm also just a bit of a fan of Andrew Kelly. who is just a programmer I really respect. I consider him one of the greatest yak shavers of all time. He wanted to write a digital audio workstation and then got distracted and had to write a C library for real-time audio. And then instead of getting back to the digital audio workstation, he realized that he had to make a programming language to replace C. So, yeah, I just kind of respect that level of yak shaving. And so his involvement just makes me interested in the language. Yeah, which is not a technical reason, but it's more of a fan reason, I guess. All right. Thank you, Adam. Okay, thanks.