PyO3 101 - Writing Python modules in Rust

In recent years, Rust has been getting more and more popular over other similar programming languages like C and C++ due to its robust compiler checking and ownership rules to make sure memory is safe. Hence there are more and more Python libraries that have been written in Rust natively with a Python API interface. One of the tools that have been driving this movement is PyO3, a toolset that proves Rust bindings for Python and tools for creating native Python extension modules.

In this interactive workshop, we will cover the very basics of using PyO3. There will be hands-on exercises to go from how to set up the project environment to writing a "toy" Python library written in Rust using PyO3. We will cover a lot of specifications of the API provided by PyO3 to create Python functions, modules, handling errors and converting types.

Goal

To give developers who are not familiar with PyO3 an introduction to PyO3 so they can consider building their Python libraries with Rust to make use of Rust's memory-safe property and parallelism ability.

Target audiences

Any developers who are interested in developing Python libraries using Rust. It will be an advantage if the attendees are comfortable writing in Rust. However, attendees are not required to be familiar with Rust as all the Rust codes will be provided. Basic knowledge of Python will be assumed from the attendees.

Outline

Part 1 - introduction and getting started (40 mins)

  • What's the difference between Rust and Python (5 mins)
  • Why using PyO3 (5 mins)
  • Setting up the environment (exercises) (15 mins)
  • Starting a new project (exercises) (15 mins)

Break (15 mins)

Part 2 - Creating a simple Python library (50 mins)

  • Creating Python modules (exercises) (20 mins)
    • Generating documentation
  • Creating Python functions (exercises) (30 mins)
    • How to create function signatures
    • How to deal with errors

This session took place in track Programming & Software Engineering and was classified suitable for novice domain / intermediate python by the speaker.

Transcript (auto)

Auto-generated from the recording utilizing Open-Source AI. Speaker labels (Speaker 1, Speaker 2) reflect diarization, not identity. Timestamps refer to the recording.

Speaker 1 [00:05]

Yeah, thank you so much for coming, because I know that on, you know, pre-talks, like, the program, I said that, like, oh, you don't need to have any experience in Rust, so just a sanity check, who actually have experience in Rust, you maybe put up your hand, so I know how deep I have to go to, or how, okay, okay, so it's more like half and half, that's good, but don't worry, I expect But nobody can write Rust, so if you already know Rust, you may have found it like, oh, you can do it yourself, that's great. You can maybe kind of try something yourself, you don't have to follow everything because you know how to write the Rust code, and you can do a little bit more. But for people who doesn't write Rust, it's absolutely fine. You can follow. And also, I have put up the setup instructions before. I hope that people have already done it, but if not, I will quickly show the steps that you have. So first of all, I hope you already have Rust installed on your computer, and also if you are using Windows, you may... Because I'm not using Windows, and someone told me that, oh, if you're using Windows, writing Rust and then using Windows and compiling could be a little bit tricky. So yeah, I just don't know how to solve Windows problems, so I can't help you with that, I'm So, without further ado, I'm Chuk, I contribute to open source, I love open source, you know, I love the community, I love Python community, I organize events, PyData London is, you know, yesterday, if you have been to the lightning talk, it's happening in June, so that's also our 10th anniversary, so, you know, if you're into data science, you can come, and, you know, there are also EuroPython, of course, this year we're going back to Prague, and yeah, that's great. And now serving the Python Software Foundation as a director and right now I am a Python consultant so yeah, happy to talk more about that but let's jump into the workshop. So this is what you're supposed to have done beforehand, so install Rust or if you have Rust installed before but you haven't used it for a while, update Rust. I'm going to use Python 3.12, and I think the setup today will be compatible with Python 3.8 or above. So I'm just recommending everybody to use the latest version of Python because, you know, Python have a life cycle, and then after a few years, then, you know, maybe Python 3.8 won't be, you know, at the end of life and won't be supported anymore. So keep updating your Python, please. So another thing, we'll use a virtual environment. You don't have to use pyenv virtualenv, which is my setup. If you have CPython, the standard would be .vnv. You can use that, but again, I just assume that you have some experience with Python and know how to do it with the tools that you're using, so I'm not going to go into detail like, oh, if I don't use pyenv, how should I do it? I assume that you know your tool. Again, Windows, I'm sorry. I will show you the repo, which you can, if you want to do the exercise again, all the information is there. If you can't try it now, if you don't want to install a virtual machine right now, maybe you can try that at home, right now maybe share with your neighbours, or just follow along. But again, I am trying to not spend time solving Windows problems, I'm so sorry. So first of all, assuming that you have done what I mentioned in the previous slides, the first thing is set up your virtual environment and then install Maturin. So Maturin is the tools that are going to be used a lot today. So what is Maturin? Actually, you will realize that we don't have to pip install Py03. We actually pip install Maturin. So Maturin is a kind of a tool that will use to compile a Rust code, and then it will be packaged into a Python library. So that sounds like magic, right? But again, if you know Rust, you know like Rust is a language that needs to be compiled. And then after the compilation, and actually it will be something that like, you know, will make it into kind of like some extensions, some C extensions that you have, for example, NumPy and those kind of things. So there will be a Python API, so you can use that in Python code, but it's a compiled executable file at the back of it. So that's how all the magic happened. But you don't have to worry about those. Today, we are just going to be, again, the workshop is one-on-one, so it's going to be a very brief and just like a taster of using PyO3. So again, a lot of deeper understanding. Of course, if you want to use PyO3 for something that is in your production, you have to write Rust code and those. So again, I'm not going to teach you Rust today. It's not the purpose of today. Today is to taste how to use PyO3. And I hope that if you found it interesting or something you want to do in the future, you can find a path that you want to go. You can learn more and then you can learn Rust and then kind of make it into something that you could use at work, for example. So, that's, you know, assuming that you have done the Pitman store, everything's good. So, first thing, you know, we will start a project. So, Maturin has also come with a command that's called init. It will just create a whole project structure for you. So, you can now, like, do it. You know, I would open a new directory. I call it Pyro 3101. one like you can use the same name then it will be convenient because all the code just assume that you do the same but feel free to do your own but make sure that you know what you're doing you have to kind of adopt it to your own version so run this command and then you will see what happened so now I'm going to also show you what happened so this is the ripple that I prepared oh okay and I can't make the left hand side bigger I don't know how sorry so um but don't worry like again like this is actually is on github so later you will see it but what happened is that um after you run that uh command a maturin init you should be able to see that you would have um some files that got set up for you so first of all there will be a cargo.tomo file so now you you may inspect that file so we just like get a sense of what's going on and how things work. So cargo.tomo file is, well, it's not pyproject.tomo. We also see pyproject.tomo. So we all know pyproject.tomo file is for Python packages. So cargo.tomo is actually for Rust. So cargo in Rust is kind of like peep in Python, but not exactly the same. But again, I put it into easy to understand terms. Cargo can manage all your packages in Rust. So there's also this cargo.tomo file that kind of describes the setup of this project. So you can see that package. So this is just like default the version to be start with 0.1.0. Of course, when you're actually doing it in production and then you have a second version, then it won't be 0.1.0, so you do semantic versioning. So you do you, and addition again, like the year is 2021 by default, but you can change it to 2024 if you want. But I'm not going to do that, those are kind of not very important to make things work, so I'm going to skip over those. What important is this line, the PIO3, 0.20.0, so this is the PIO3 version that we're using, so actually PIO3 is not a Python library, it's actually like, you know, Kaggle is getting it so it's actually a rust package so um the version we're using is 0.20.0 so this is uh i see inc is the latest version and also you can see that uh there are some other information there as well um the library name again this will be the the folder the name of your folder again like you can use other name but you know make sure that everything stay consistent um so also the the The create type, you can see, again, this is essential. Don't change it. So again, there will be a lot of advanced detail and stuff, so I'm not going to go into it. But just don't change it, OK? You can change the name, but nothing else, please. Maybe the year or end version, but nothing else, please. OK, so that's the cargo.tomo. Just have a look. You won't have the cargo.log yet if you're just starting the project. The next thing you have is the private project.tomo. So I'm sure this is more familiar to you all. So this is information that PyPI will use when you publish your library. So of course, we are not going to publish it today. Otherwise, PyPI will be full of our trial today, which is not good. So we're just going to try it locally today. So it requires Python 3.8. Again, if you develop it for not supporting Python 3.8 or other requirements, you can change it. But again, today I'm not going to dive into all the details of these small things. We just want to have a taster and make it work and then see how it works. So, okay, have a look at these two files. Any questions? One second, we gotta give you a mic.

Speaker 2 [10:04]

Bigger font, please

Speaker 1 [10:06]

I can make this side bigger, but I don't know how to make the other side bigger. I'm sorry. But you should be able to see in your directory. So this is like the, I'm just talking about there's two files that got generated. If you run Maturin in it, you should have these two files, not the log, cargo.toml and private project.toml. That's already better. Yeah. So you have it, you're supposed to have it in your computer if you're following along. If you don't have, you know, if you don't bring along your computer, it run out of battery or you're using Windows, you can't follow, maybe ask your neighbour nicely, can they share it with you and you can have a look. So yeah, so I haven't done anything yet. So we just type Maturin init and this is generated for you. So okay, all good? Any more questions, except that it may be a bit small and difficult to see. So yeah, look at these two files, just have a quick look. you know, get a sense of how things work. Next things for people who have no Rust experience, I'm going to just, you know, explain it again, like usually when I do workshops, I don't know how experienced people are. So if it's like very tricky for you, just bear with me, you know, just quickly explain this. So Python and Rust is very, very different, okay? So Python, we never come, well, I mean, you technically, I know that there is like some for a niche project, they can compile Python, but nobody is actually doing that. So Python actually is a language that's supposed to be interpreted. So for example, when you give your colleague a Python script, it won't work on their computer by default. They have to have a Python interpreter in their machine. So for example, everybody uses CPython, it's a kind of standard distribution. You get it from pypi, no, python.org, and then you, it's like standard version. You may use some other versions. There are other versions, like, for example, some people use something that is like, because CPython actually interpret Python into C, which is, you know, another language. But some interpreter, you know, interpret Python into something else, you know, like something that is like super popular In the last few years, it's PyScript. Let's say PyScript actually interpret Python into WebAssembly, and then it's run on the browser. So that's how Python works. Python is super flexible. We just don't have to really define the type of anything. You sometimes feel like everything works with Python. So it's designed to be easy to use, to be an educational programming language teach younger people, for people who are not engineers, to maybe be able to use a little bit of scripting language to help their work. For example, that's why now we have data science and other things. So Python is super flexible, but I know some people don't like it, that's why in the community now we have people start typing their Python code. So that's more similar to how other languages, for example Rust, work. So Rust is like, you know, kind of like C. It's considered to be a lower level language. It needs to be compiled. So it will have a compiler. If you install Rust, follow the link that I've sent, that you have done, that you will already have a Rust compiler. So it's, you know, it's cargo, yeah, everything is cargo in Rust, which is great. And also Rust is strongly typed. So in Python, when you start a variable, you don't have to say what type of this variable is. But in Rust, you have to. Again, if you have never done any programming in C or something, you may find it a little bit different. But it's OK. We got to get used to it. So other terms that use in Rust, we will have crates. Crates is actually, it's very hard find a Python comparison, again, because those two languages works very differentially. So if I have to really quickly explain to someone who code in Python, it's kind of like, you know, an other Python file that you import into your other Python script. So but it's not equivalent, like, you know, I know, you may be like, No, no, but I know, Ross is not like, I know. But, you know, just put this simply. And again, we are not diving into the technical detail today we're just like trying to have the limited amount of time and get a taste of how things work so if you want to study and understand rust more in the github repo i put the link to other learning resources that you can learn more if you want but again today we have limited time so bear with me um there are also macros uh we will come across some macros when and we try to like, you know, write some code today. So macros is kind of like a, like, I'm just quoting the Rust book, it's like, it's metaprogramming. So metaprogramming is like, we would use some, for example, you'll see some like macro that actually take a function and it will generate some code that modify that function. So it's kind of like a decorator, but instead of doing it in runtime, do it before, like, it would do it while the code gets compiled. Again, like, if you only do Python, that may be a very new concept to you, but for people who have experience coding in other languages, that may be more familiar. So there are two types of macros, and I will dive into detail after when we came across them. So again, like, macros are some things to help your code to be shorter so you can reuse the code so you don't have have to write the same thing every single time. So those are the good things, kind of like shortcuts for you to write good code. And I think Russ will do is panic. So I hope you don't panic when you see panic. So panic basically is like something that goes wrong. There will be a failure message, and your program will get terminated. So that's panic. It happens at runtime. It won't happen in the compiling stage. In the compiling stage, you will get error message from your compiler. And one thing that I like about Rust is that the Rust compiler is very good at telling you what's go wrong. So it will give you a suggestion that this is how you can fix your code. And most of the time, if you follow the suggestion, it will work. So if you see an error message at the compiling stage, follow what you have been told, and you will probably be fine. Don't panic, OK? Right, so this is an example code that when you type Maturin init, it got generated for you. So after you type Maturin init, you will have a directory structure get built for you in your directory. From the SRC source directory inside, there's actually this script. It's already written for you. So before we write our own code, let's look at this example code to understand how things work. Then we can write our own code. So first thing, you can see that first line, use, So this is, again, like in Python terms, for folks who only do Python, easy to understand, this is like an import. So if you're using things that's provided by Py03, you will have to write this line. So again, don't change it. Keep it there. You will probably be fine, OK? So yeah, remember, it needs to be there. Next thing, these two are actually procedural macros. So what are they? Do they look like a decorator to you? It's kind of like something that put in front of a function. Because the fn actually is like writing a function, right? So instead of df, you have fn, which is fine. Just change of characters. But these two procedural markers, it's kind of like a decorator. So what it does is that it will take the function that you're writing after as an input, and it will generate something to replace it and make it work, make it into a thing that would work with PyO3 and have your Python interface and your Python function can call it, OK? So that's why we need them. Again, I'm not going to dive into the detail. Just understand that we need these two to work. So first one, whenever you write a new function that's included in your Python library, you have to put that in front. And then the key thing is that after you write that function, you have to put it into the PI module. Because when you package the whole thing, then your module includes those functions. So if you forget to put it in there, you'll see the error message after if you don't put it in there. But today, everything you write, we will be writing as a PI function. And then after we write it, we'll put it into the PI03101 module there. But you will see how it worked in a bit. Now it just sounds a bit like kind of mystery, but it will be more clear when we do things. The next thing you'll see, I highlight this line because there's the wrapPy function exclamation mark there. So this looks like, oh, I'm just calling a function called wrapPy function. But there's an exclamation mark in the function name. What was that? That's actually not a function. It's another type of macro. Again, macro is just like a way to modify your code. So instead of writing a lot of complicated things to include the sum of string that function into your Python module, this give you a shortcut to just write it like a function. You know, when we write function, we reuse the code, right? We simplified it so we don't have to write the same code all the time. So it's kind of like that for this macro. But again, the difference is that technically macro is like it will do the modification, the code modification during compiling time. Function is like for runtime. But for simplicity's sake, just treat it as if it's a function. Just remember that it's a macro. It's not a function. It's a little bit different. But we can use it kind of like a function today. So that's it. That's the thing. And then we just copy and paste this line over and over change the name, so don't worry too much about the detail. Next thing is you can see that, again, in Rust, everything needs to be typed. So this is kind of like what we do when we try to type things in Python. If you are already doing that in Python, this should look a little bit familiar for you. So yeah, so just remember that it's not optional in Rust. We have to do it. We have to decide on which type we want and state it there. Hopefully, we'll have time to touch on the type conversion at the end. But again, I will show you the repo. Every information is there. But again, we have to type everything, including the return result, which is, in our case, because we're using PyO3, we'll have the PyResult, which is, again, provided by PyO3. So again, for simplicity's sake, today, everything is, you know, output will be a PyResult. is, you know, you can see the first one is string, it could be something else, but everything is a pi result, okay, next thing, why I have okay here, why, so, because pi result actually is a result, so in Rust, a common practice is that for each function at the end, it will return a result, it's either okay, so okay means everything is fine, and the other is, of course, everything is not fine which is an error. So in Rust, at the end it will return an OK or an error. That's why we have OK at the end. In Rust you can see that actually in Rust code if there's no colon at the end means that that is going to be returned. So at this to function At the end, it will return this OK object. It's a result object, or actually enum, but I'm just saying object for Python people. So yeah, so you will return an OK. So again, if you don't quite understand, just follow the pattern for now, and then you can study a little bit more after. So I'm highlighting this line again, because there are some question marks at the end. So the first thing, you see the colon. So this is kind of saying that the next line is another thing, kind of like a line break. And also, this is not going to be returned. So there's a colon at the end. And then there's the question mark. So what are the question marks? The question mark is kind of like a shorthand. So if, for example, the wrapPy function somehow have an error there, and then it gives an error at the end, so that error will get directly passed as a return. And then we just terminate it. We don't have to worry about what's come after that line. We just return an error and pass the error out. So that's the shorthand for the question mark. So you can see it as like, oh, it may or may not work. So that's why there's a question mark. So again, I can't cover all the technical detail. Just remember, this is what's going on here. So this is the example code. I hope it's not too scary. There are some familiar things in it. If you write function in Python, you're just like, maybe you don't have the brackets. Just remember the brackets. Remember those semicolons and follow the pattern. You may be fine. For today, at least. Today, you'll be fine. I promise. OK, so now we have looked at all the things that we got by default. Let's try to build it and see if this default thing, this function that was written for us, some are string, if it works. But let's try that first. So we know how to try things out. So that's the process that we will do throughout the whole exercise and workshop. So if you type, so we're using Maturin, right? So well, we're keeping story. We better use it, right? So Maturin help, and see what Maturin can do for you. So if you type Maturin help, you will see. So this is all that will be in your prompt. I know it's a bit small, just bear with me, or type it in your computer, Maturin help, then you will see this on your computer. So, you will see that we have a lot of command, there's multiple commands that you can build your package. You can build it and publish it, or just build it and like all this, but today, don't worry about it, we just use one, we just use develop. So what develop, what Maturin develop this command, what this command do is, it will take the code, the rest code that you have written in that file, And then it will get a package, put it into a Python library, and install it in your virtual environment so you can use it and test it out right away. So we're going to do that today. That's why it's very good to have a virtual environment because, you know, you don't want it to be in your other projects. So have a new virtual environment for this development. So what you can do now is to type maturin develop. up, it would take a while to run because it's kind of packaging everything, it doesn't have any cache because it's brand new, and it gets installed in your, it becomes a local library and it gets installed in your virtual environment. So do that, and then after, when it's finished, you can type pip list, you'll see that you you would have this Pio 3.1.1 thing that got installed in your virtual environment. Everybody good? Follow? If you have questions, this is the time to ask. Or if you say it doesn't make sense, it's too fast, then let me know. OK, good. So we should use the develop thing? Yes, we will use develop today. Because, again, other things, for example, we are not going to upload it to PyPI. We are learning, so we are not going to do that. Yeah. All good? OK. Good. Now is the time for you to shine, because now you have to write a Python script to try out this sum of string thing from your package. So I assume everybody is a Python programmer, and then you can do it yourself. Do you want to do it yourself, or do you want me to give you the code? No answer. So I assume you want to do it yourself, so try it out. And I can give you the code after as well, but first try it out. So like, yeah. It's boring when I feed you everything, right, so. Hmm? Yeah, you write your own Python script. Create a, like, I create a try.py, that's what I do. So you can create a try.py and then, you know, write some Python script, import that library. So it's already installed, you know, Maturin already install it, pip install it to your local environment, so you can just import it, use it, try it, see if it works, see if you understand what's going on in the code. Oh good. Thank you. Thank you. Anybody is happy and like already tried it out and happy to move forward? Give me a thumbs up if you're happy to move forward. Okay, cool. Everybody, well done. You're a very good Python programmer. Okay. So if you're like, oh, I'm sorry, like I have some hiccup and I type a little bit slow, then don't worry, like I have the code and this is in the repo. You can just copy and paste after. So don't worry, like it's just a very simple Python script to test it out. So you know, you don't have to learn it, you know, you write better code than me, so forget about that. OK, moved on. So next step, because that's actually the end of the slide deck. There isn't much in the slide deck. Everything is actually in that repo. So now, the most important thing, I don't know if it's big enough, oh, it was too big, OK. Go to this repo, and all the information, exercise, everything I just explained is actually there. So if you have that repo, you can actually, like, for example, if you're watching online, if you have somewhere to go, I don't know, some work email meetings you can actually go to that repo and then like you yeah you know say a lot like bye-bye like you can you can do it yourself but uh for people who are sitting here and committed to the workshop we will do it a little bit like you know we're good together and then we'll see how far it goes if we run out of time you can still do it at home that's the good thing about it that's why i put everything on the github rebel so uh now you can go there and um so where we are is actually now we're just like get ready to start our first exercise which is the hello world exercise so if you see the readme you can scroll to the um you can skip everything and scroll to the hello world that's where we are right now okay so if you want to uh because uh again like all the code all the rust code actually is um you know is in the folders in that repo so it's up to you. You don't have to clone, you can just copy and paste if you want or if you just want to like, you know, if you're super lazy copy and paste is too much work for you, you can clone it and you can just run the script. So up to you. Yeah. So um, yeah, but I suggest you follow along and copy and paste because like, there are some steps that we're going like, so those scripts actually is like the final results at the end of each exercise. So there are some like intermediate step in between which you will miss out if you don't follow along, so yeah. Everything good? Everything got the link? Good? Okay. If not, just shout, no, stop, but I would just assume everybody's good. So right now, we would, so sorry about that, I know it's a bit hard to see, but again, you have the GitHub repo, so if you can't see, go to the GitHub repo. So what we do is we will do the first exercise, so we will go to the source and then the lib.rs, this file, so we are not going to create a new file. Technically, you can, but again, we are learning today, and we are just doing 101, just baby steps. So we're just writing everything in this file to make things much simpler. So now we are starting. So we will start to write our first ever function in Rust that will become a Python function. So what we'll do, we'll just do something very simple. take a name as a string and then we would just output a string saying hello so um again like i'm terrible at typing and you know i don't like live coding because i made a lot of typos and mistakes so i i it's like your cooking show you know like oh instead of like chopping the tomato it's already chopped it's here you know that's what i'm going to do now i'm going to copy and paste what i've done before you can also do this like copy and paste so um or actually i would I would go to, I think what I would recommend you to do is, one sec, bear with me, go to the GitHub. Again, this is super small, but don't worry, this is for me, this is not for you. You have it on your computer, I assume. So just copy this, you know, this is the function that we got to write, and I'm going to paste it there, and then you will see it at the end. So yeah, just paste it here. This is our function. So very simple, you know, take a string. So again, like, everything needs to be typed. So actually, in Rust, there's more than one type of string. But again, just follow my code. We are not going to explain that, because that would be a rabbit hole to dive into. But anyway, you can read the Rust book if you want to learn all of those. Yes?

Speaker 2 [33:58]

There is one question on Slido.

Speaker 1 [34:00]

Yes.

Speaker 2 [34:00]

Yes. Yes. Concerning the last example, are you required to write stub files to get type hints from compiled Rust modules? Some strings seems to be typed as args any, quarks any.

Speaker 1 [34:11]

Oh, so is it for the library that you publish, like you want the type, like for example, in Python, if you write in Python, you type in it, and then it will become a type, the function will be type in. Actually, this will be done for you. Well, you don't have an option. You have to type it because this is Rust. You have to type it. But after that, like when Maturin do the magic and then it becomes, for example, our case, like pi03101 library, inside it will be typed. You can actually change the name of the variable and add more description and all that stuff using another macro, but I will talk a little bit about that in this exercise, but right now don't worry about it. You have to type it anyway, so just type it. And then actually, your Python function will have that type as well. So it will be done for you for now. And then later you will see that we can use another macro to do some other magic, which Like, yeah.

Speaker 3 [35:26]

Yeah. Thanks. Uh, well the thing is, I just tried to clone this repo from GitHub and uh, after trying to do type, uh, uh, how does it called, uh, maturing in it, uh, maturing, maturing develop, uh, I re I, I receive an error, uh, during the import of this model, which is input error with dynamic model doesn't define model export function.

Speaker 1 [35:53]

You are one step ahead of everybody.

Speaker 3 [35:57]

Ah, okay, okay, so I'll just copy and paste it.

Speaker 1 [36:02]

But you're doing the right thing, trust me. I'm going to explain in a bit. Yeah. OK. So yeah, you may be like, copy and paste and try it out. I know you are so excited to try it out, but I'm going to explain it now. So again, don't worry about this format, exclamation mark. It's just like an F string. It's just the Rust version. OK, so everything seems good, right? So now I can do Maturin develop, and it will work. Yay, isn't it? OK, so let's try it out. So save, and then, oh, what have I done? OK, one second. Let me close this. OK, right, so here I do maturi develop, right, right, right. Oops, what's going on? What's going on? OK, so I just, OK, it complains that I'm using a snake case. Well, I'm a hyphen mister, what are you going to do? But anyway, you should be able to see another error saying, like, it's a dead code somewhere. Yeah, function say hello is never used. So this is a dead code warning. It's because the thing is like, yeah, one dead code is on. So to make this warning go away, you can turn it off. So all these are actually warnings. So it doesn't hinder. You still have something that you can use. But now you can't use it, because the sayHello function is not put in the module. Remember, I said that every single function needs to be put in the module. We haven't done it yet. That's why it's good to leave this warning on. You can turn it off, but leave it on for today because we are beginner. We want to learn it, so leave it on. It will remind you that you have to put your function in your module, otherwise it's a dead code. That function can never be used because we haven't put it in the module, so that's why we have that shout at your face. So let's fix it, let's fix it. Let's add it to our module. So what we gotta do is that, again, Okay, follow the example, that's why it's good to have the example here, I love what Maturin have done for us, I just copy and paste and then I change this into say hello, so it should work if I change it, right? So okay, let's try it again. And then, oh, do I do something silly? That's why I hate live coding, yeah, because when I copy and paste, I delete the bracket, That's why it's like doing, yeah, okay. Yeah, there's always hypo and stuff, okay. Right, so now I only have the snake case warning, but again, I don't care about this right now. You may have to worry about that when you are writing a real thing, but again, we're just playing around now. So this warning, I'm just gonna ignore. I'm bad at naming things, I admit it. So what, okay? So now, this thing should work, right? So now I have a Python package that is badly named py03101. And also, why is this capitalized? Maybe that's why it's not happy about it. So now I have py03101, and it's still not happy, OK, whatever. It has a sayhello function in it, and it should work, right? So let's try it out. So I should have a script like this. So yeah, it's saying hello, I used the say hello function, it's saying hello to John. So it should say hello to John, so let's see if it works. So the Python, and then I put it in the directory, you can put it at the same working directory, it doesn't matter. So I quite try the ut, try the py, oh, error, what's going on? You see, that's why I hate live coding, it's like always something's going wrong. Yeah, I put up, that's not the FindModerExport function. Whoa, pyinit. Did I do something wrong with this? Trust me, it works when I was preparing it. Did you get the same error or not? Or is it just me? I get the same error too. You get the same error? Okay, let me quickly... I think it's just some, maybe some name thing and stuff. I think it's because of the module name You're using lowercase in your file code, but it's uppercase, the O in your... Yeah, I don't know why it become uppercase. It's all lowercase. Yeah, but in the RS, this RS is uppercase O. Oh, yeah, why? Okay, I don't know why, like, it may be because... Oh, yeah, because I changed it halfway through and I developed the workshop. Sorry about that, my bad. Okay, it should be fine now. So let's do Maturin develop again. It compiles again and fix all the issues. hopefully now we don't have warning so sometimes the warning is actually good you can't ignore it ignore the warning you have the same consequences as me don't learn from me I'm giving a workshop I don't know what I'm saying okay so try again as if nothing happened now it works they say hello John yeah I'm happy okay do you get it do you get it you get it Yay! Good. Well done. Okay. Well done me. Yeah, it's like, like, again, like, live coding usually becomes a, like, you know, joint effort. People help me to debug, so that's good. Okay, question.

Speaker 2 [41:31]

There is one more question. I know Rust does not use object-oriented programming. It doesn't implement classes, but structs. Is it possible to export these structs to have them as classes in Python with methods?

Speaker 1 [41:47]

Actually, there is a way to do it, but we are not going to cover it today. If you want to learn, you can go to PyO3, the user guide. It will explain how to do it. But again, that's more advanced topic. So today's 101, so that's more for maybe 102. So yeah, sorry, we are not going to cover that, but you can find the information on PyO3 user guide. The link is provided in the end of the README, actually. Right, so I assume everybody is good. So let's move on to the next thing we want to do. Now you have successfully written a function using py03. So you have written Rust. Yay! OK, so it's a big step for a Python mister who only do Python. That's good. OK, so next thing we want to do is to try something else. So we have fixed the problem, the dead code problems. So what we're going to do now is to try to see if we can make a default. Oh, sorry. I jumped ahead. So now we're just saying hello, which is a bit boring. So how about we welcome John to PyCon DE. Shall we do that? Shall we do that? OK, so let's add another thing. So we add another variable to conference. So it's another string, right? OK. So and now I want to like, hello, John, welcome to... Yeah, typo. See, live coding. Welcome, okay. Welcome to is the thing we wanna welcome them to and then I would put it in Python and it will be the conf, right? So okay, I hope it's fine. So let's do it again. No warning, which is good, or no error actually. So what we're going to do now is to, let's say, conf is pycon.de, shall we? It should be a string, so now I kind of lose my skill to code in Python as well, which is bad. Okay, cool. So now let's try this script. So Python script running, okay, oh, welcome to PyCon DE, so yay! What about if the user, the Pythonista user, who is like, nah, I don't know which conf friend it is, can you just give me a default in Python? You can have a default, right, in Python, remember we can just like, oh, conf equals to something something, and then that would be the default. So can we do that in Rust and Pile 3? So to make a default, we can't just do this. This is Python, right? You can't do it here. No, no, no, no, no, no. So what you're going to do is to use the function signature. So the function signature is also a way for you. There's actually another text function signature. Again, I'm not going to go through the detail. It's in the user guide that you can also change. For example, this is name here, but you want the Python one to not be named, maybe like username or something like that, then you can change it there. But today, what we're going to do is to create a default for it. So what I'm going to do is to add one line, add one macro, which you will see here. So this macro actually is defining what the signature for the Python function will be. So in the Python function, we will take a name and a conf, so two variables. And for the conf, the default value is the conference. Not toString right now. ToString is just the new ones that Rust have because of different string type thing in Rust that I have to use toString to make it into the right type. But don't worry about it. Just copy and paste it. But this is how you can change the signature from just name and conf to name and conf. The default value of conf is the conference. So this is how we do it. And now, if we save and develop again, so let me go to terminal and develop, oh, I have something wrong, what have I done, did I miss a bracket, maybe I missed a bracket, ah, yeah, I deleted it by accident, okay, so, do it again, that's why the compiler of Rust is really good, it just tells you that something is wrong before you run the code, because because it's compiled and then like, I like it. Okay, so now it's installed, it's good. So now let's run a test code. So now, what if I don't say conf, will it give me the conference? So let's see. So let's run the code again. Let's go back to this. Oh, welcome to the conference. Oh, why there's a question mark at the end? I don't know what I'm doing, sorry. I should delete the question mark. But anyway, welcome to the conference. Are you welcome? Yes, you are. So yeah. Anyway, so you get the idea, right? Now you have a default. And then you don't have to provide it in your Python function. You have a default value. So that's cool. Now you can do some very Pythonistic stuff with this Rust code. So that's great. So we have tried those, too. Another thing you can try, if you look at the documentation, the README, you will see that, oh, now we can also have a look. You can go to the, you know, open a Python terminal and have a look at what, you know, PyOpher has done for you. You will see that, you know, imported and checked. But because of the time, we have, like, less than an hour right now, and we have two more exercises, so I'm not going to do that. You can try it at home or, you know, try it. If you're very fast, you can try it now. But I'm going to jump into exercise two right now. So if you're following along, we're now at exercise two. We're going to try to read a file and see what happens. So now, again, what we're going to do is to read a file. So it will be a very simple, right? I just copy and paste and add another function. Why is it not here in one second? Okay. Copy it and then paste. Okay. Another function. But remember in Python, right, if we use something that's not here, not in the, you know, not already included for you, you have to include it. So actually we are doing something here. We are using file, something from the file crate, which we haven't included. So we have to include it. So we have to put in some extra lines at the top. So we have to, yeah. So we're using the file and read, the IO read. So that's why we have to include those because we are using some like this one is from that.

Speaker 2 [48:51]

There's two questions in the chat for the last section still. How difficult is it to debug the embedded Rust code inside a PyO3 library?

Speaker 1 [48:59]

How difficult it is to debug, and again, like it depends on how familiar you are with Rust code because like all these error that you saw is actually by the Rust compiler. So I would say that the Rust compiler is quite good at telling people what's wrong and the suggestion of how to fix it. But if you are like, oh, if I am completely new to Rust, it may be a bit scary for you. So again, like I think I mentioned it in the README, like if you want to use this in the in the future for real, learn Rust. I'm not going to lie to you that, oh, you don't have to learn Rust and I can make everything work. No, because you're developing in Rust. Today I'm just going to give you a taster, so if this is something for you, you know where to find more information and learn it.

Speaker 2 [49:47]

Okay, next question. Can I modify sum as string function to accept multiple types other than use size, for example, a floating point number? Yes. Or do I need to write an entirely new function?

Speaker 1 [49:59]

So for following point numbers, for different types, so again, exercise one and two will be just using string. But for exercise three, we'll use different types. So again, you are very smart, and you are very ahead of everybody else. Shh. OK?

Speaker 2 [50:14]

Last question is, is it possible to pass dictionaries or lists to a Rust method? Yes.

Speaker 1 [50:19]

Yes, again, we will cover it in exercise three. Thank you. Very smart. Everybody's so smart. I'm so happy. OK. So again, we're using something that is from file, from read. So that's why we have to include it. Again, all this, I'm not going to tell you everything. If you write in Rust, you have to find this information yourself. They are all online. You can easily find them. Oh, how to use this? File, open. You will find it online. So we have this new function, and then we are taking two strings. So one is the file name, one is the name of the person you want to check whether they are in the registration. And then I would then open a file. So this is a buffer that actually, in Python, you also have that. You open the file, and then you read line from the file or read from the file. So that's what it does. The thing that's different is that you have .expect here. This is how Rust works. This is basically saying that if it doesn't work, so in Rust, you have to be very explicit about how error got handled. In Python, if you don't capture the exception, it would just error and then terminate, right? But in Rust, you have to be very explicit about what happened if something doesn't work at runtime. So for this case, if the file doesn't open, or you will see something will demonstrate something goes wrong, it will just panic and say like, oh, no, no, no, I can't continue. This is actually good. That's why Rust is considered a relatively secure programming language, because you can't access something that doesn't exist. So yeah, so that's a good thing, that if it doesn't exist or something goes wrong, it panics, and it just stops doing anything that may cause damage or security problems. There's a mute here. Again, this is something, again, if you know Rust, you do this, but if you don't know Rust, it just means that it's mutable in Rust, not everything is mutable by default in Python, it is, so you have to say it's mutable in Rust. So, information for you if you're not familiar with Rust. Another thing is like the content, so this is, we created an empty string, so it's a new string, it's empty, that we're going to store the content in it, again, we're going to put the content inside, so it needs to be mutable. So the file, we've got to open it and do something with it. So mutable, and then content is the empty string. But we want to put something inside, so it's mutable. So after that, we would read. So we would put the things in content. So again, why is there an emu here? Again, it will be another whole day workshop to explain this. And also, this emu thing, by the way, is why people think Rust is so hard to write. So I'm not going to explain this. Otherwise, we will be here for the whole day. Just follow along if you're not familiar with Rust. If you want to learn why, again, I have the information for you, the Rust book. The reference is at the end section. You can find the link to study Rust. So you would know why there's the end mute there. But anyway, you have to put end mute here. So after that, the content will have the content of the file. And then it will check if it has the name of the person. Again, there's an end there. Don't ask just because you have to. Put it there. And then if the name is there, just give back, you are registered. Yay, I can check and find your name badge. Good. If not, then it's like, oh, sorry, you haven't registered. You can't come in. Please get a ticket. OK, good. So it all makes sense. So let's do it now. Oh, before that, of course, we have to add it here, right? We have to add it to the PyModule, of course. So let's do it quickly. So that is a checkRack. So let's do that. CheckRack. OK. So right now, it should be doing something. So let's see. Oh, OK. OK. Right. So we have some code. So now we want to run this to see if I can find join in redlist.txt, okay? So let's do that right now. Let's do, okay, size two, ooh, something's going wrong. What's going on? Well, oh, this is not expected because I forgot to develop, sorry. So again, nothing happened. Again, oh, something goes wrong. Like, okay, look at the error message, right? We have a panic exception. It's a file not found. Of course, because, you know, if you look at our working directory, that's not racklist.txt. It doesn't exist. So what happened is that, remember, I said that if, like, in the code, Rust will say, will panic, right? So here, if they can't find the file, it will panic. And it does. And then because it's in Python, so when we run the try code, the try the py is in Python. So in Python, it automatically becomes a panic error. So if you look at the error message again, it's a panic exception. But I'm pretty sure you have done it before. Like in Python, if you read a file, you've got to put the file there. It's not a panic exception. That doesn't exist in Python. It's a file not found error. So this is not good, right? If I'm like shipping this Python package and people like if they do something wrong and get this, like this error message sucks. So how can we handle the error more elegantly, right? So PyO3 already know you will have this problem. So provide a way that you can do it. So what it does is that again, I have to copy some code. So what it does is that you can define, Like, you can put in a new exception for your Python. So in this, a PyOver exception thing, inside, it will have a lot of different Python exceptions that you can use. So for example, file not found error. So if this is just Py, file not found error, because it's Python exception. There are also other, like the value error and other things. But again, you can find that in the user guide. Yeah, you have to include that, and then you can use it. So but how do I add that in here? It kind of needs to rewrite the code a little bit more. So again, I'm copying and pasting because I'm very bad at live coding. Oh. And for people who are not familiar with Rust, this may be a little bit tricky for you. But I can explain it, so don't worry. So you can see that what changes is that instead of .panic, .expect here, which would cause the panic, I would capture the result. So the file open will give me a result back. This is how Rust works. It will give you a result enum. So what I can do is you use something very Rust, which is matching, the pattern matching. If it's OK, that means that if it can find a file, everything is fine, then it will capture the file here. So remember, it was like let new file equals to something something. So instead of doing that, I first capture the result. And then if it's OK, I unpack it here. And then I can do the same thing. So this is the same thing. All this inside and the middle are the same thing. So I do an extra step of checking the result. So this is the happy path. So what about the unhappy path, which is error? So if it's the error, I don't care what error it is, because I assume that, well, this is an example. So it's oversimplified. I just assume that if anything goes wrong, it's because the file is not there. So I just float a pyrofile not found error, and then I can even put in my own kind of warning error message there. So this is how you can change a panic exception into something that is more Pythonistic. So let's try this one. So shall we? So we can. We can do develop, sorry, I did something wrong, oh no. Oh yeah, wait one second, what's going on here? Create a module, what Python, hmm? Use a fun declare, create. Anybody smarter, well, all of you are smarter than me, like maybe some of you can spot it yes but sorry can you be more specific oh yeah this one yeah yeah yeah why is it not there oh I made a mistake probably okay right yes that's what I said everybody's smarter than me that's good Okay, so yeah, I'm missing here. I have to fix this as well. Okay. Yeah, I will fix it. I promise Huh? Oh, yeah the FN Okay So Thank you for the live debugging Okay, cool. So now let's try again. Shall we so let's try again And now we have a file not found error, like this is very Pythonistic. And then I have my own custom error message. Yay, so we did error right. So it's just a little bit oxymoron, that's great. Okay, so yeah, we have just tried the unhappy path, right? That something goes wrong. So what if we have the file? So what you can do is to create a file that doesn't exist and try it. this and try it. I already have it here, actually. But the working directory is not correct. That's why it keeps missing the file. What I can do is go to the directory and do this again. And then it should work because now I have the file and you have John in it and then John is registered and welcome to Python, the conference. Okay. So, yay! So, that's exercise two. So how much time we have? We have around half an hour-ish. So last, we have exercise three, which is the handling different types. Because so far, we just use string, which is not realistic. We have so many things in Python. We have lists. We have dictionary. We have floating points. We have the integer, whatever. Let's try them. Not try them all, but try some of them here in exercise three. So let's go to exercise three. So now we're here at checkpoint. We're here, exercise three. If you're following along, we're here. So yeah, we're talking about the type. So again, the nuance of two strings. So again, you can check the information yourself. Now, we want to, let's say, do some numbers. So let's put in a list and count how many items on the list. So this is what we're going to do here, OK? So here, now we have another function. This time, instead of string, we check in a VAC of string. So a VAC is kind of like this. So if you are at the readme.file, you will see that I have a link to the conversion table that provided by the PyO3 documentation. So if you check there, so actually it's here, user guide here. If you open it, it's a bit small. Let me zoom in so you can see that. Oh, I have a list. So what is a list? A list is actually a VAC. But the difference here is that you can see the T here, which means that in Python, you just have a list and it could have anything in it. It could have different type of things in it. It doesn't work in Rust. Well, there's a way of creating something in Rust that have different things, kind of like a chain kind of thing. But that's not what we are doing here. So for simplicity, we have to have a list that have every single thing in it is the same type. So this t is the type of that thing. For example, if we have a list of attendees name, then it will be a list of strings, right? So then we have a vec of strings. So that's what we are having here. And then I'm going to count how many attendees in the list. So it will become a integer. So integer is, whoa, what's going on here? There's so many integers. Yes, there's so many integers. So in Rust, there are different types of integers. There are i64, u64, u size. So i is, so let me think. i is the plus and minus, so you go positive, negative. So it's signed. u is unsigned. And so the number 6432 is how much memory you use for describing the integer. So the bigger the integer is, the more memories you're going to use. So for example, if you have a number that's more than 255, you probably need something more than 8, right? And what is u-size? u-size is whatever the default in a machine it is. So again, I'm not going to go into the nuance. You can read the Rust book if you want to understand all of this. But again, just have a brief understanding of it. So yeah, I'm using useIce here for simplicity's sake. And then I just, there's actually for the VAC, there's a default method, .length. So it's kind of like Python, which is great. Questions?

Speaker 2 [64:54]

There's one question concerning the types if it's possible to consume uncollected Python intervals in rust What I? guess iterators generators Is it possible to pass them to Russ?

Speaker 1 [65:10]

I haven't thought about that, actually. Let me think. If it's a iterator, an iterator should be a kind of, yeah, iterator is, oh, it doesn't have an equivalent. So, but an iterator actually is a, I forgot the Python term of that, oh, jeez. It's not a function, but like, but it would, geez. Generator? A generator, yeah, like a generator. Is that a generator? Is, there's, these are actually Python native type. You have that, but maybe PyO3 provided that. There's like PyIterator, maybe PyO3 provided, but I haven't have a default on it, so I don't want to answer that question because I don't want to give you wrong information, so sorry. But that's a very good question, I have to try it out myself to give you a correct answer.

Speaker 2 [66:13]

there's two more questions that are very related like when would you go to Rust what are some of the operations that are worth converting to Rust

Speaker 1 [66:23]

It's a very good question. The benefit of using Rust is that because Rust is very good at doing concurrency. Python, we have the and everything is not very multi-friendly. In Rust, there's less friction doing those things. Again, this is something that I can't simply give you a very easy answer for that. But you may have to see what the difference between Rust and Python. What Rust is known for is it's very memory safe because of how they do the borrowing thing, those end thing, the borrowing and things, like ownership, that make the Rust compiler is very robust and very straight on checking those. So that's relatively memory safe. And also, yeah, again, Rust don't have the GIL, and you can easily do concurrency. So that may give you an idea, but again, I don't have a very definite answer for that. OK, sorry. Let's move on. So this is the method provided by VAC. It's kind of very similar to Python. Great, we love it. So let's add this and try. So remember to add it. otherwise you get another warning. So let's do that right now. And let's do that quickly. Okay, so let's have a play. So in exercise three, I should have it in the script here. It's not, oh, I look at the wrong thing, okay. Yeah, so this thing should work and the rest should not. So let me comment it out. This one also doesn't work because I will go back to the original directory, okay. So let's do that. So let's do a develop. So I go back to the original directory, do a develop, and then I do a, I run the try script, so Python. Three, try. I have done something wrong. I have commented it out, I forgot to save, sorry. Let's do it again. Right, so now it counts the list, and there's four, so one, two, three, four, yeah, it kind of worked as expected, so again, it should be quite straightforward, and so let's do one more, one more exercise, we have quite a lot of time, so yeah, good, I'm a bit worried about time, so okay, so now let's read in a dictionary, so a dictionary, what is the the equivalent in Rust. So the equivalent, according to this table, is a hash map. So a hash map is, you can see this, oh, this is a bit weird. Why there's this hash brown, colon, colon? It's because, again, it's not by default there. We have to use it. So it's from a crane called hash brown, so let's use that. So let's include that in the thing. So let me go back to my, let's go back. And then, do-do-do-do-do, yeah, we are exercise free and then we should be, yeah, we have to include the HashMap from the, or there's actually in the collection. So there's also some like standard collections in, it's kind of like our standard library. So there's a standard collection in Rust, so you can use that. There's the HashMap there as well. You can use it. Or you can use the hashbrown library. I don't know the difference. But yeah, again, you can look into the detail if you want. So sorry, that's PyData London. Why is it PyData London? One second. OK, here. Right. So let's do this HashMap thing. So I'm going to put it here. Again, I'm not going to live code because I've made tons of mistakes. I'm just going to copy and paste. So you see, now we want to do a kind of more mathematical, like find average. So it won't be an integer. It will be a floating point. So if you look at, again, I'm not going to show it again, but if you look at the table, you see that, oh, again, my floating point, you also have different size of floating point you can use. If it have more digit after the dot, then you may want, if you want it to be more precise, you may want to use a bigger memory for that. But like for this, I think 32 is more than enough. So F32, I'm using that. So the result, so the HashMap, so HashMap, this is like the key and the value in Python. So key, I am using the name of the person, so that's the string. The value, it will be a folding point number. Because for arithmetic in Rust, I can't mix the different types, so it's not like Python again, everything's so straight. So I'm using floating point for both the count and the sum. So it's here, and then remember it needs to be meal because I'm like adding them in, so here. For loop, get of like what Python is, and then add it in, and then just like do the floating point division here, and it should return the result. It's quite straightforward, this one. So let's go back to my try script, and then try this one. So I have attendees or string, and then this one. Python, I mean, in Rust you have to do this, but like Python, it would automatically convert it for you, so it should work. Yeah, so it's like this. So let's do develop again. And then, oh, yeah, I forgot to edit. That's what you get when you forgot. Yeah. I think after a while, then, it's muscle memory. You would always remember to add that in. Yeah, OK. Right, OK, so let's try again. And now we have, oh, it will calculate the average. So the average is the floating point number. And if you don't trust me, pull out a calculator and then calculate it yourself. It's even correct, but it's correct. So that's all exercise. And now, at the end, I give you the freedom of trying different things. For example, you want to try your iterator, you can try it now. But anyway, so now it's kind of like I always like doing a freestyle session at the end of the workshop so people can, if you are a little bit like, oh, it takes time to catch up, you can now catch up. Again, follow the README. If you're already very smart, I know all of you are very smart, a lot of you are very smart, and you're very fast, you can try something. You can try all of the different types. Go to the table and try different types of things. Come up with the creative. Come up with something to try. Can be very silly. It's a toy anyway. But yeah. And also queue anytime.

Speaker 3 [74:10]

Yeah, I have tried something. I have tried to make a wheel file, but I have not succeeded in doing a wheel file which contains the compiled Rust code as well as the Python code from my package. So I actually want to have a wheel file which contains my Python code and the SO file, so I don't have to ship two wheel files.

Speaker 1 [74:36]

That one I don't know. I don't know how to solve it. Sorry.

Speaker 2 [74:42]

Other questions I Realized that my IDE autocomplete is not working and I just looked that the Interface Python interface files are not created By pi o3 right now. Is there like a workaround for that to get the autocomplete work?

Speaker 1 [75:08]

Have a look at the, because remember we used the signature, another thing called text signature. That may be able to help you. So have a look at the text signature. If that one doesn't, then I don't know.

Speaker 3 [75:22]

Well, you partially already asked the similar question, but this one is going to be a bit different. So usually when people do some Python optimization, Python speedups, they try to speed up a small part of their Python application, which is the crucial one. So I wanted to ask you, maybe you can name some examples or is it usual for Rust to be just used to rewrite small parts of Python code? Just when people want to optimize something in their application and they have this model which is called very frequently and it's very slow and they just want to write that model in Rust. That is like usual part of it or this thing with file 3 is more popular to write leaps and things like that.

Speaker 1 [76:29]

I think, like, what you mentioned, I think there are other tools to do that already, like a lot of JIT compiler that you can use, for example, like NumPy is one that you could, like, maybe speed up some numeric calculations using NumPy, that is a JIT compiler, so it will, like, compile that part of the code, and then, like, that's exactly what you mentioned, so I don't see the reason of, unless you want to, like, publish this as a library and you want to write Rust. I don't see the, like, for most people I think the most direct way is to use those tools to, like, to kind of speed up that part of Python, so, yeah. Yeah, because we had this need and we ended up using migracing when we were seeding it. Yeah. It's important by use case. Yeah. But in your Rust, so I'm just... Yeah, so this is more for, like, people who kind of, like, you know, either have experience in Rust or want to write in Rust because, you know, for me, I think like, oh, I like writing Rust more than C. So, yeah, so that would be, yeah, that would be like the way to go. But again, like, for what you mentioned, I don't see the point of doing something compressional because there's already other tools that are already doing that. You just need to purely written in Python and, you know. Maybe Rust is faster. Then you have to do some profiling and try it yourself. Uh, so my question is, so far we just wrote all the functions in one module, right? So what if we want to modularize our package? Like having multiple modules within that same module that we wrote in Rust, how do we do that? Yes, that's a very good question. So for how to do that, you have, uh, well, in Rust of course you can do that. You can even have different crates and put it in there. For example, in our code, these are different crates that are written by different people. You can do that as well and have different files. In real life situations, that would be what you do. You won't put everything into one file, but again, then you have to understand how to do that in Rust. I would recommend you go to the Rust book. There's actually a chapter talk about how to, like, you know, package different crates and how to put them in. So, like, yeah. I understand the part of Rust. I mean, about the part of the Python module. Oh, this. Yes. Exactly. Yeah, that one. So, yeah. If, for example... So, for example, this is on the other, you just, like, put this in, like, what you do with Rust. Like, if you have this, like, travel average, then you just put it here. But then, of course, you put it somewhere else and then use, and then, you know, yeah.

Speaker 2 [79:32]

Any other questions? There's one more in the chat. This is how to consume C-like data structures like a NumPy array from PyO3.

Speaker 1 [79:43]

Again, NumPy array is already quite optimized because it's written in C. And I don't think you can get a lot of speed advantage of rewritten every single, like, the thing is already well written and proven in NumPy. I don't know how you're going to, well, I mean, you can create a, you know, kind of like pandas and polars. you can create a Rust version of NumPy, but if you want to try to easily get some speed advantage, I don't think that would be much different because both of them are, you know, because NumPy is not written in pure Python, it's already quite optimized, and, you know, you can try. I mean, if you want to go through the endeavor and maybe create a library that is much faster and safer, good, but it's not easy.

Speaker 2 [80:47]

All right, then I think this is it. Let's thank Turk again for the wonderful hands-on tutorial.

Cheuk Ting Ho

After having a career as a Data Scientist and Developer Advocate, Cheuk dedicated her work to the open-source community and working as a community manager at OpenSSF. She has co-founded Humble Data, a beginner Python workshop that has been happening around the world. She has served the EuroPython Society board for two years and is now a fellow and director of the Python Software Foundation.

Social card for talk: PyO3 101 - Writing Python modules in Rust