Cython to speed up your Python code

Speaker 1 [00:03]

Hello everyone. I'm actually happy that there's still so many people here, although I already had, you know, part of a talk earlier this week. How many of you were in the talk on Wednesday? Almost no one. That's cool. Okay, cool. So I can tell you entirely new things here. Okay, so this talk is about Cython. So I'll start with a tiny bit about myself. Stefan Bienel, as you heard. I'm a software or data engineer, if you want. I'm also a trainer, so I do Cython trainings. So if you want me in-house in your company, I can come over and teach you Cython. Or do consulting for you, so look at your Cython code and tell you how to improve it. That's what I do from time to time. I'm a Python user since 2002, one of the Cython core devs, actually one of the founders of the project in 2007, when we forked Cython from a previous project called Pyrex. Who's heard of Pyrex? A couple of people, maybe six, seven. So it's actually, it's nice to know for historical reasons, but otherwise all you need these things aside them. So I do training, consulting, but I also work for a company called TrustU, where we do big data processing for hotel reviews, so we do text analysis, we run analytics, statistics on the data that people write on the Internet. So anyone of you who is staying in a hotel now, maybe you get an email afterwards asking you to write a review for the hotel. If you got it at, let's say, Booking.com or HRS or so, they'll ask you for a review afterwards. And if you provide it to them, we'll quite likely collect it afterwards and integrate it into our system. and then we can tell people how that hotel performs. We can actually tell the hotel how they perform and how they can improve. So this is something we do. So we analyze this data, as I said. We summarize it. And if you go to trusty.com and enter a hotel name there, maybe the one of the hotel where you're currently staying or the one where you're considering to stay next time for your vacation or so, check it out. We'll tell you how that hotel is actually being perceived by guests who stayed there and what real guests have to say about that hotel. So this is the place to look, and you get all the information there as opposed to a subset of the view on other sites on the Internet. Okay. A lot of the tools that we use for data processing are part of the Python data ecosystem. You probably know many of those, right? So, NumBy is probably word, SciPy, Scikit-learn, Pandas, spaCy, if you're into native language processing. All of these tools are partly or even entirely written in SciThem. Who knew that? No one's here, okay, cool. Interesting. Cython is actually a very widely used tool these days. It's a programming language and people use it, especially in the Python area, to write high performance data processing code. I said all this is part of the Python data ecosystem. Why is it an ecosystem? Well, it's an ecosystem because it's integrated. All of these tools have some way of talking to each other, of exchanging data. What you heard in the keynote this morning, there's a way, for example, to exchange data between OpenCV and Cyclic Image. You can pass a NumPy array from one to the other and apply image processing algorithms from OpenCV to images that you loaded through scikit-image and processed there. So that makes it an ecosystem because it tries to integrate the code that we have through the data. And the way the data is integrated is through NumPy. NumPy is kind of the one place where all the data comes together and then you can run different algorithms on it from different libraries. And they're all able to exchange their data. But NumPy only integrates the data, so how do you integrate the code? I mean, there's a lot of code out there that people have been using for decades sometimes, right? For data processing. There's, who here has any experience with Fortran, for example? Couple of people do, okay? So this is still a language that people write code in. Why? Because it's a very efficient language. You wouldn't probably write a web application with it that does fancy things here and there. You would probably use Django for that or Pyramid or something like that. But still, people use it for implementing efficient data processing algorithms. And people have been doing that for the last 40, 50 years. So for quite a while. And they have this code lying around. they're still using it, it's been debugged, right? In 50 years, you have a lot of time for debugging your code. It's rock solid, it works, it's very, very fast. Why should you throw it away, reimplement it, and invest another 10 years to get all the bugs out of the new code? You wouldn't, right? And so, you need ways to integrate all of this code into the ecosystem, into making all of this code available, usable, with the data that we have today. And Python has been an excellent way to do that. Python's always had an excellent C API, a way to talk to C code and through C interface to Fortran code also. So all of this code has been reused and has been connected to Python. And a very good way, it's not the only way, but it's a very good way to do that is Cython. So Cython is an excellent way to integrate code in the Python ecosystem. Why Cython? Well, it's a programming language. It's a very pragmatic programming language in the same sense that Python is pragmatic. It tries to keep your focus on the functionality that you're implementing rather than having to implement all sorts of boilerplates or syntax here, syntax there. So you can really focus on what you're trying to achieve, implement the functionality. and it applies in the same way to Python and Cython. Why? Because Cython is actually Python. It's a Python compiler. It's an optimizing static compiler. It's production proven, widely used, and it's really all about getting things done. So the programming language allows you to move freely between Python, it compiles Python, but at any time it allows you to do C stuff, arbitrary C stuff. So it mixes C data types into the Python language. And at any time, you can say, I'm doing Python stuff here, but what I'm really operating on is C data types, C arrays. I want to call it C function here. And you can freely do that at any time. That's how the language works. So you can write code that's as Pythonic as you want and as low-level as you need it. So what is Cython good for? Well it integrates native code into Python. It's used to speed up Python code in CPython and a surprisingly large number of people actually use it to write C code without having to write C code. So we write the C code so you don't have to. The main topic for this talk today is speeding up Python code. Skip that and go to the intro. Okay, quick intro to Cython. Who's used the Jupyter Notebook? Quite the majority. It's actually an excellent way to do interactive data exploration, data processing, interactively trying out code, playing around with it. And it has Cython support, so I don't know if all of you know that, but the Jupyter Notebook supports way more than just Python. It supports Julia, it supports C++, as we've recently seen in another talk. It supports various languages, more than 15, last I checked, probably way more than that these days. And it supports Cython. So all I have to do is, I have to tell Jupyter to load the Cython extension, load xcython, and that's that for me. And then I can take a quick check how my environment looks like. I'm using Python 3.7, the latest Cython release, the somewhat recent NumPy release, and the C compiler. Because that is something that becomes relevant now. Normally what you do in Python is you have a Python file and you say Python run this file for me and it just runs it, right? That's it, done. When you use Cython, Cython compiles your Python code to C code and then you use C compiler to compile it into a shared library into an extension module and then you can take that, import it in Python and use it. So there is a build step involved. Okay, so it is not as edit run as in Python, it's edit compile run. A quick intro. So this is a tiny bit of Python code. I'm using the sine function from the Python math module and sine of five is apparently this number. I can do the same thing in Cython. As I said, Cython compiles Python code and in order to tell Jupiter that this is not a Python cell, it's a Cython cell, I just add the Cython cell magic. So when I run this through Cython, what the Cython integration does is it takes the cell, drops it into a Cython module, runs Cython over the module to translate it into a C file, Then runs the C compiler in the background to compile the C file into a shared library, into an extension module for Python, imports that module, executes it while it's importing it, and then does whatever Jupyter normally does. So it augments the namespace. So the cell magic in Jupyter does all the complex build setup and all that for you. So this is what happened when I hit Shift-Enter to run the cell. Cython compiles the code to C. And since it's C, you can do C things, right? It's not just limited to what Python allows you to do. You have all the power of C at your fingertips. So instead of using the Python sign function, which is an object, I can use the C sign function. from the math header file, from libc. Okay, and that's what I'm doing here. So I'm telling Cython I want to use libcmath, and I take the sign function from it, and I just assign it to a variable. So do that, and then from Jupyter I can run the function. Anyone surprised? Yeah, some, okay. So what happened here is, Scython saw that the sign function here is a C function, which takes a C double as input and outputs a C double. And when I'm assigning it to a variable, this is actually a Python variable, so it says, okay, I have a C function here. The code tells me that the user wants an object instead. How do I map a C function to an object? Well, I just wrap it, okay? And it's easy to wrap this function, it's kind of obvious that the input of a C double maps to a Python float type, in and out, and so it just auto-generates a wrapper for me that allows me to call this function. And so the signed function here, when I say type of signed func, it's a signed function. This works for simple functions. It obviously doesn't work for functions that take pointers or do any C stuff that is not compatible with what Python can support. But for functions like this, this is the fastest way to wrap C stuff for Python. Internally, what this pretty much does is it generates a function, and I can spell that out as def function. which takes a double in, calls the C sign function, and then returns the result. So this is kind of the spelled out version of what I just did, what Cytan just did for me automatically. Can run the same thing, and it gives the same result. So once I start spelling this out, I can see this is pretty much Python code with one tiny difference. I've typed my argument, I could do it like this, works the same. I just typed the argument because I wanna make it explicit that I wanna double in and this is a kind of, it's a bit of a syntax extension that Cython allows me to spell out the argument type as I would in C signature. Now once I'm starting to do this, I can start making this wrapper a bit more interesting. I can move more functionality from the Python side into the C level. What I could obviously do here is, in order to calculate the sine of 5 squared, I could say 5 to the 2, or 5 times 5, and then I get sine of 5 squared. I can do the same thing at the C level, and that is much faster, why? Because when the CPU can actually see what it's doing, that is multiplying two numbers, it can do that in one instruction. In Python, taking an object, multiplying it with another object, is a lot of instructions, okay? And then some memory allocation, and then some more instructions. So one CPU instruction for this is much faster. So when I'm moving the square down to the C level here, and by doing this, so I'm writing a function square sign, and I just say, well, take the sign of x squared, then the whole expression gets evaluated in C as fast as the C compiler can implement it for me, as fast as the CPU can run it. And so, from the Python side, I get the same result. I pass in my five, I get the result of five squared sign. And the whole thing is evaluated in C. That's also something that Cython allows me to do now. Even if I'm just wrapping C functionality, I can make my wrapper as thin or as thick as I want. You wouldn't see it from Python, it doesn't make a difference, right? but I can implement a bit more functionality in C, in Cython, right, by writing it in Cython, and it gets translated to C, it runs fast. I can add more C functionality down there, and it all gets translated into C, it all runs fast. I don't have to do anything in Python anymore. I can drop it all below the Cython frontier into C. That's totally my choice. It's a programming language. I can do anything above or below. I can do it at any point where I like. One nice feature that Cython gives me is Cython-A annotates this for me. I pass in my code and it outputs a little HTML snippet which shows me how Cython interpreted my code. And the nice thing here is I can click on one line and it tells me what C code it generated for me. This is a simple line, I can see it took the variable X, caught the C sign function on it, and then used the C API function from CPython to convert the C double result into a Python float object. When I do this, when I click on the other line, the signature line, then there's a lot of code that I really don't wanna write myself. That is code that Cython can generate for me. And there's a lot of argument unpacking, argument conversion, error handling in there. All of this is code that is obvious from the Python signature, and I can leave it to Cython to do the right thing. So this totally has Python semantics. It's called as a Python function, as a function object. It does the usual input conversion. I can pass in an integer float. I get a type error when I pass in a string instead of anything that can be converted to a double. So it's all nice and safe, and Cython does it for me. You can see that there are two shades of yellow here. I hope you can see it. Yeah, you can see it. And that is another hint that Cython gives me, is the amount of Python interaction that is associated with these lines. There's little Python interaction here. There's a CIPI function that is called. There's a bit of error handling in there. And there's a lot of Python interaction in the signature. So when I'm using Cython to optimize my code, when I'm passing it my Python code and I'm telling it, you know, make it run fast, but tell me how I think about it, then it outputs this HTML snippet and I can look through my annotated code and see, okay, there's a lot of Python interaction going on there. If I want to make it faster, I should maybe try to replace this with C operations, which are faster. And the less yellow I get my code, the faster it's probably going to run. So this is a really nice optimization helper. Okay. What should I show you next? Next, I'll give you an optimization example. So, optimizing Python code. I'm using a text processing example here. Who knows DFLIP? Not so many people. you should look at the standard library. There are lots of nifty little tools in there. DiffLib is one of them. What DiffLib does is, a lot like the Unix diff tool, you give it two sequences, arbitrary sequences, might be lists, might be iterables even, I think, it's probably supported, two kind of sequences, even strings, right, two strings, and it's going to compare them and output the differences between the two sequences. Really nice little tool. it's in the standard library, it's pretty versatile, and here I'm going to optimize it to make it run faster when I compile it inside them. I'm using a little benchmark from the Fuzzy Wuzzy tool, does anyone know Fuzzy Wuzzy? Yeah, two people over there, three. Fuzzy Wuzzy is a tool for a fuzzy comparison. That's pretty much what it does. It has a faster backend also, but it can just rely on DefLip and makes a pretty nice benchmark here for DefLip. So that's what I'm using it for. Okay. So I have a little benchmark, I can show you that. As in show. Basically what it does is it has a couple of strings and compares them in a fuzzy way to look for differences. It does that in loops, it runs it a couple of times, and so it exercises Fuzzy Wuzzy, it exercises DiffLib, and what I'm gonna use as a benchmark here is basically just the time that the Fuzzy Wuzzy benchmark runs overall. So when I run it, I'm not gonna wait for it, it's gonna take 51 seconds, and this is the time that Python simply needs to run fuzzy-wuzzy, to run div-lib on the data that it provides, okay? So, the first thing I'm gonna do now is I'm going to compile div-lib with Cython. I have a command somewhere here, yeah. Can you see that? Can make it a bit bigger, is that okay? Yeah, okay. So that's the Cythonize command, which is a nice command line interface, and that's also a function with the same name, which allows me to compile Python code. So I'm compiling DiffLib down here, and I'm providing a couple of compiled directives. I'm compiling in Python 3 mode, so that Cython knows that the code I'm compiling there actually uses Python 3 semantics. because Cython has support for Python 2 and Python 3, so I have to be explicit here. And the dash i down here triggers an in-place build, so Cython would normally just generate the C file, and when I say Cython-i, or Cython-i, then it does an in-place build of the whole extension module for me. So it runs distutils in the background, and takes the C file and compiles it to an extension module. So I'm gonna run that, and run Scythonize. It runs the C compiler, takes a while. Takes a while longer. Runs the linker and now it generates a .so file for me for Python 3.7, which is a compiled DiffLib. And I can run the benchmark again. And you'll see that it runs quite a bit faster. We were at 51 seconds before, almost 52 seconds. 51.7 or so. Takes a while to run. Takes a while longer to run. That's why I didn't run it before, it just takes a while. Okay, and we're down to 34 seconds. So, if we take 52 by 34, that is, sorry, the other way around, 34 by 52. then that's 65% so we already have a speed up of 35%. More than a third, not too bad. Because I didn't actually do anything here, right? I just called siphon and it cut off a third of my processing time. Not too bad. But there's more that we can do. We can optimize the flip. So, how do we know where to start optimizing? How do we actually know where the bottleneck is? Well, we use C-profile. Okay, so we profile our code. So, C-profile also takes a while to run. Who has never used C-profile here? Ever, never, never before, not yet. Okay, that's about 10 people. So what Cprofile does is it kind of instruments your code, your Python code, and it logs whenever you call a function and you exit a function, and then it just takes the time that it takes to execute that function, okay? And that slows down your code, sometimes by quite a bit. Sometimes by quite a bit. Simply because it does a lot more, right? Depending on how many function calls you have, it just takes ages. And then in the end, it's going to spit out lots and lots of information about all the function calls that it recorded. And up here, you can see that the bulk of the time the time to spend in function findLongestMatch, okay? So, I'm gonna start optimizing that function. As you can see, the top two functions here add up to 20 seconds, this is actually already the compiled code, okay? Add up to 20 seconds, and both of them are functions in DFLIP, okay, yeah. So when I look up the function, the top function find longest match, you don't have to understand exactly what this function does. Okay, basically what it does is takes the two sequences from index A to index B and compares the two sub-sequences. So it inputs as basically sub-sequence A, sub-sequence B as indices and then compares the two sequences and it does that in loops, a couple of loops and then looks for the best match between the two sub-sequences, more or less what it does. Okay, how do I make this faster? First thing I can see is there are a couple of loops that run over integer indices, which represent indices into the sequence. So the first thing I can do is I can instruct Cython to turn these integer loops into C loops for me. How do I do that? I can type my variables. So when I say I should be, well I can actually say I should be an int, but int in Cython is a Python int, and I want to be explicit here, I want a C type, and I could use a Cython int, a C int, but the sequence can be arbitrarily long, And int in C is a size-bound type, okay? It's at most 32 bits, which might not be long enough for the sequence I get. So what I'm using instead is I'm using a type size T, which in C is defined as an integer type that is definitely large enough to hold any sequence you could ever hold in memory, okay? be it in a 32 or 64-bit system. So it's not happy with the Cython because that's not imported yet. So let's do that. Import Cython. Cython is a magic module that gives me a Sys to the C types that Cython can use for compilation. So that's an I, that's a J in the same way. That's also a K somewhere. So I start typing my variables, and now when I run Cython again to compile my library, you'll notice the dash A option here, so I'm already enabling assess to the annotated source output because I'm optimizing, so I'm interested in what Cython makes of the code that I'm passing it. and here it is, quickly reload it, and you see there's a lot of yellow in there. So much of this code has actually not been converted much. Converted down to C, so it's not very optimized code. There's a lot of Python interaction going on. Okay, nope, not here, yeah. Okay, it's compiled, I run the benchmark again. It's gonna take a while, while that is, we can look at the code. Okay, and you can see here that Cython has actually converted my Python for loop into a C for loop. And on the way there, converted the indices that I got in as arguments to C integers also, okay? So it did that conversion for me. And inside of the loop, things are now running with I and J and K as integer variables. But you can still see that a lot of the operations done with I, J, and K are still in dark yellow. And that is because there are other variables that depend on them, and the thing with C is Integer operations in Python are nice. You add one to a variable and you get x plus one. In C, when you add one to an integer variable, you may get x plus one, or you may get a very negative number, okay? Integers in C are size bound, so they sometimes just wrap around if they get too large. So C is unsafe where Python is safe. And what I'm doing here is I'm instructing Cython to generate code for me of which I know it's safe, but Cython cannot automatically know that it's safe. So I'll type some more variables to make sure that all the operations done with INJ, especially on these three variables here, which are used to store the current best known index in the system are also run in C. Okay, so I'm typing some more variables. Okay, and while I'm doing that, the timing is done and it got a little faster, but not much. So let's do that again. Now I type more variables and you'll see that things are becoming less and less yellow. For example, this line down here is now entirely evaluated in C, okay? And that is because I allowed Cython to turn it into C by typing the variables that are used here. Also the condition here is now C. This one is not. So there's a BH below, okay, those are the arguments that are passed into the function. So next thing I'll do is type the arguments. And eventually I'll end up with code that has enough types to allow Scython to generate efficient C code from it, even though I'm not actually changing the algorithms in there, I'm just saying, you know, it's totally enough if you use C semantics here. Don't change the loops, it's all fine. I tested them, I can test them in Python, right? There's still Python code, it's totally normal Python 3.6 code. I can still run it in Python. But Scython, when you compile it, Please use a C integer for this variable. Please use a Python list and not an arbitrary object. I can type my variables and allow Cytan to apply optimizations that would otherwise not be obvious. So, bit more white. I typed the arguments, so this turned all white now. And I'll run the timings again. And we're down to 28 seconds, okay? So we're already down from 52 to 28 seconds, which is almost twice as fast. And there's still more we can do. For example, this B2J here, that's a get method call, and when I look up what it's used for, it's actually a Python dictionary. So I can tell Cython this is always a Python dictionary and never anything else. And that allows us to allow Cython to apply optimizations for this specific data type, rather than having to assume that it's some object that probably has a get method. It might not, but it probably has, right? So I'm manually specializing the types that my code uses to allow Cython to specialize the code that uses these types. So in this, when I click on this line, there's a lot of C code being generated here, but somewhere down here, you can see there's a dict-specific method being used, a function being used now. Okay, previously it was an arbitrary call, a get method. So the more and more I advance with this, the less Python operations I will have in my code. And keep in mind, this is still Python code. I can still totally normally run it in Python. I can run PyLint over it. I can edit it in any Python compatible editor. I can do coverage analysis on it, anything I like, all Python tools available, but when I run it through Cython, it gets visibly faster. So, up, 28 seconds. It doesn't always get visibly faster, but it already got visibly faster. And a couple of more things down here that I can do now. That would even make it faster in the end. I'll stop here and show you a couple of more things along the way. Are there any questions to this so far? Am I able to use MyPy with this? Yeah, in fact, yes. With the newest version we have included MyPy, what's it called, sheds, something, so the type declarations that MyPy can exploit, so it understands the CYTHEN types. So when I write this CYTHEN type here, then it understands, oh, that's probably something that's similar to Python int, and it can analyze this code. We're also wondering when I don't use CYTHEN. No, so MyPy, MyPy's type system, So the Python type system does not know anything about size-limited types. Sizen does. And the C compiler does too. So you have two levels in the end where you can get warnings about things that you probably did wrong or errors if you really did something wrong. So it's caught at these levels but not caught at the Python level. So a Python int, as I said, is not size bound. So it doesn't help if you type a variable as Python int. There is a Python int, yes. So in order to use an integer, I would just, so C int, I would just use Python int. Okay, the difference is just int is, well, it might not always be large enough from the value space to. Yeah, I can use long. But long might also not be large enough. For example, on Windows 64, a long is actually 32 bits. Yes, so this is the normal Python 3.6 typing notation, right? So you can type variables this way in Python, and instead of saying, you know, this is an that would be a Python type information, Python type annotations, but it's not going to be used by Cython compilations. So for Cython, I can be more specific, I can say this is not an arbitrary int, it's exactly the C type, please use it and adapt the code that you generate to this variable type that I'm giving you. Okay. the time annotation compression in my whole project and want to say, okay, every integer is now going to be a Python S size C. Then you still have to tell it what is an integer and what isn't. So it's actually good to be explicit. What you can do is there's a Python compiler directive which is infer types. So Python has some kind of type inference, but it's limited for integers. For integers, it goes the safe route and says, well, if you add two integers, even if they're C integers, the result might not fit into the C integer, so I'll be on the safe side and assume it's not a safe operation. But if you say info types equals true as a compiler directive, then it goes like, user said so, I can do this, okay? Okay. Yeah, I'll quickly show you one more thing. Where am I? Somewhere here. Okay. Yeah, for example, malloc, when you want to use, so as soon as you start leaving the Python space, okay, so previously I talked about optimizing Python code, you stay in Python space, you keep everything Python compatible, you keep it runnable in Python, all fine. As soon as you start calling C functions using external C code, you'll leave in Python space. And that's a special syntax for that. So as soon as you do that, your code is no longer Python compatible, but then it's Cython code. You would change the extension from .py into .pix, and then you can use an extended syntax that allows you to declare and use external code. Most of your code will still be Python, but you can use additional features that are Python-specific. So here, for example, I'm calling the cmelc function to allocate C memory and assigning it to a typed variable. So this cdef statement here allows me to type variables in C notation. There's not really a way to express pointers in Python. So this pointer is using C notation, and then since allocation can fail, and this is always something you have to take care of in C space, in Python space it's all nice, right? You do something, it fails, you're an exception. Wonderful, right? In C space, if something fails, you probably get an error return value, And if you don't care, the error will be lost and no one knows about it, but something might go wrong later. So I'm taking care of that. I'm saying, you know, if what I got back is actually false, so if it's a null pointer in this case, then I raise a memory error. And this is also something very nice in Cython, right? It's mixing C operations with Python operations. And it's mixing them freely. So I'm doing a totally C-ish thing, I'm allocating C memory here, I'm calling malloc. And if that fails, I raise a Python exception. Then this is a lot how your citing code is going to look like. You write mostly Python stuff, but sometimes you do C stuff, or maybe you do a lot of C stuff, and sometimes you do Python stuff in between, and you do whatever is necessary to get your functionality implemented to get errors handled, and to get your stuff done, okay? This is what the pragmatism in the language comes from. And if you do it really right, then you probably don't want to just say free after doing something else. You probably want to use a try finally here. And in the finally clause, you want to call free, okay? Another Python thing that you're using. Okay, and then I'm through with my time, and I'm happy to have you listen. Thanks. Yeah, so thank you for this awesome talk. As you said, we're almost out of time, so we barely have time for any additional questions. Maybe one quick question if we have. All right. Thank you. Did you try to get this DiffLib function even faster? And if yes, how much did you get the speed up? I actually didn't really go further than this. It gets a bit trickier afterwards because if you want to keep it Python compatible, then you're bound to the Python data structures, for example, that you're using. And as I said, you get it in a sequence. might be a list, but it's not necessarily a list. It might be a deck, for example, or a tuple, right? So there are different Python data types, so it's difficult to make this faster. You can. There's something called fused types, which is kind of like generics. So you can write a Python function once and declare it as accepting a set of different input types, And then Cython will just spell it out into the different type functions for you. So it will duplicate your code based on the input functions. So that's something you can do here. And then you can obviously have a general object kind of fallback. So this works. So there are a couple of more tricks that you can pull. But I think a factor of two is pretty okay for adding seven lines of no code. Okay, thank you. I'm afraid we're out of time. The next session is in two minutes. I'll be outside for the rest of the conference. Thank you all.