Running An Open Source Project Like A Start Up

Speaker 1 [00:03]

Thank you so much. It's okay, yeah. Okay, so before I talk about what's going on with my project, I want to know how many of you have written a Python library before? Oh, good. How many of you have written a library that people can download and use? So how many of you want to get more contributors and be more popular? Yes. Okay. So, yeah, today I'm going to talk about how to run an open source project successfully. It's just my point of view. I won't say that my library is very successful, but I think during what I've tried to do, then I learned a lot and I want to give you some of my experience that may be something that you could think about doing and something that maybe you don't want to do. and also it's an open source project that I'm talking about so is it an enterprise? It's not but it's kind of like building a startup but of course you won't get money out of it but if you get really successful you may turn it into a business but that's another talk so yeah I'm Czech or you can call me Cherry so I'm from London so in London I organize meetups like London Python Sprint which encourage people to contribute to open source. So I'll tell you what benefit I get from it. And also I run an AI club for gender minorities, which we try to promote inclusion and diversity. This year, EuroPython, I am also involved in the committee. So I was helping to organize it. Also, I've contributed to other open source projects. Some of them more contributions, some of them like a minor contribution, but I learned a lot from them, especially I would want to give a shout to DateUtil. It was the first project that I contributed to and the maintainer Paul, he's a super nice guy and I learned so much from him and from the project which is great. I created Pick and Mix which is the project that I got to talk about today and I also volunteer for DSC which is also a committee within Non-Focus to promote diversity. So yeah I was a bit crazy. I was a bit crazy. I, you know, I was writing, you know, using Python, write some code, and I work as a data scientist, but I'm always interested in doing some, you know, Python software development. So I give myself a challenge. Can I, you know, build something and then to be an open source maintainer? So, yeah, this is the story of how I become a developer, kind of, marketing person, kind of, project manager. Maybe for free. So, yeah, like all startups, all these projects, I think we need a good idea. So, well, I have an idea, and I think it's a good idea. I don't know whether you agree, but I think it's a good idea. So, you know, if you are, you know, like me, you know, data scientist, or you're into machine learning, then you have maybe known about Scikit-Learn. So Scikit-Learn is a library that provides a lot of machine learning models that you can train and you can use, and it's a very powerful tool, and I love them. So, yeah, so, well, and in machine learning, there's also a technique called stacking. It's kind of helped you to, in some cases, make a better prediction. So I was thinking about, because I did a course a long time ago, and I was trying to build my own stacking and I have to, you know, use all this training data and try to, you know, fit into different ones. And, you know, it's kind of like complicated and not as easy as just using scikit-learn models. And I was thinking, can I make it easier? Because a lot of steps, I can imagine, it could be quite similar, like again and again with similar thing. So yeah, if that's the case, if you're writing code that do similar things, write a library for it. That's what it is. So, yeah, and then I also get inspiration from Keras and TensorFlow because if you have tried deep learning, tried to attempt deep learning with TensorFlow, you see that, oh, it's actually quite difficult to use. You have to make these sessions and then control how these tensors, you know, do all these things and then how to backprop and then calculate the laws. And, ah, it's very complicated. But if you know Keras, it's actually super easy, Especially if you just build a sequential model, you can just call in the API, and then you just need to tell Keras what the architecture of your neural network is, and then it will just build it for you. So, wow, maybe I can have something for scikit-learn that is like Keras for TensorFlow that is super easy. So, yeah. Okay, for those of you who don't know what is stacking, so this is for not attempting to explain stacking. So in machine learning, you have different models. Let's say you have classifiers, and also you have regressors which do regressions, like all these ones here. Let's say in this example, there's three of them, and then you have some training datas. Usually they should be the same ones, right? You use the same set of data to train these three models. And then in this layer one, level zero, let's say, this first layer, that each of them will give you some predictions of some sort. They may be a good prediction. They may be a bad prediction. But it doesn't matter because we are trying to gather all these predictions and then use it to train another classifier to make a very good prediction. So this is what stacking is trying to do. So, yeah, so in some cases it's good, but in some cases it's not so useful. So if you have a lot of data because the data you train on the first layer, you may not want to use them again on the second layer. So you need quite a lot of data. And it takes longer as well because you have more things to train. So it's not always, you know, perfect. So in some cases, it's a good way of doing it. So talking about the tool that I want to build, I want to be, as I said, I want it to be able to pip install. So how? Like, what does things come from when I do pip install? Like, why is, like, magic? Where do things come from? So, actually, it's coming from PyPI. So, it's a repository. So, it stores all the things that if you import, let's say, when you Pymin store requests or Pymin store data, Pymin store pandas, all of them, actually, some people put it on PyPI. And then when you type pip install in your command line interface, then it will just connect to this and it will download the project for you. You can specify the version or you can just download the latest if you don't put in the versions. So this is how it works. And then because now I know how it works because I upload something on it. Actually, it's quite dangerous as well because I'm a nice person. But if I'm a nasty person, I can put something, you know, a malware in it. And then if you pip install it, it will be horrible for your computer. So be careful next time when you pip install. Make sure that you know what you're doing. So that's PyPI, right? So I want to create something. I'm a nice person. I create something that hopefully doesn't break your computer and is safe to use and that you can download it. Also, I want it to follow at least some of the practice in software development. So CICD, Continuous Integration, Continuous Development. So CICD, there is actually a lot of tools that on GitHub. It's a third-party tool. Of course, now GitHub has like a GitHub Action that you can use. But the tradition is like there's this third-party service that's called Traverse CI that you could use to, you know, it will create a lot of the different images. You can choose which one to test. You can choose, like, test on 3.7, test on 2.7. Like, you can choose all of them and then ask Travis whenever you make some changes and put it on GitHub, okay, make some changes. I put the code on GitHub, and it will take your changes and then build a new package, like, new version of your package and then test it with their environment. So how many of you have used Travis before? Good, so I don't have to explain too much. So I'm making myself stupid here. So, yeah, so Travis is amazing. And then so also now a friend of mine, also a contributor, helped me to use a script to do a coverage test as well. It will pop up later, so that's why I mentioned it. And also I want my library to be well-documentated because I really want people to use it. It's not just for my enjoyment. I really want people to use it. So I put, you know, I try to make the documentation nice, try to later I will explain more in the doc string, explain, like, what's the input, what's the output of the function, and also use read the docs. It's a really good tool that you can host your documentation on their site, and then it's automatically built from your RST files in your project. So it's very convenient. and to make it more convenient that we have cookie cutter. Yay! So what is cookie cutter? Who has used a cookie cutter before? Good. So what is a cookie cutter? The name suggests that it's something, it's like a mold that you can make similar things, you know, all the time, right? If you have done a lot of development, you may see that, oh, actually a lot of my projects, their structure is very similar. They all have, like, the app itself and then the test and then other things for the CICD or talks and whatever. So, yeah, so the cookie cutters is a tool that you can download and use it. And you can make your own cookie cutter. But for this case, somebody already made one that's for five packages that provide all the things that I want, like traverse, read the docs. So all you need is to run it Of course you have to download cookie cutter Download cookie cutter first And then run it And then it will ask you some questions What's the name of the project? Who are you? Do you want to use PyTest? And all these things And after that, it will set up all the files for you So I can show you my PicMX This is my project So it's on GitHub And all these things, most of them generated using cookie cutter. So it will give you a setup.py, you know, for a package that you can install with pip, you need the setup.py. You also have talks, which will build your testing environment for you. And you also have readme, that is this one, very nice, generated. Also, it will have the contributing guide, you know, all these things, license, you know, you don't have to find copy and paste from somewhere. And tests, you know, everything, the docs, you know, these are for the read the doc. So you can see that all these RST files, read the doc will use them to build a documentation, which I can also show you. Here, read the docs. This is built from all those RST files. So you can see the most, well, you can have installation, tell people that how can you install them, but the most useful thing will be usage. So all these you have to write yourself, though. You have to write some examples that how people can use your library. But you don't have to build a website from, you know, HTML or whatever, you know. It's just there. Yay! It's good. And they host it for you as well. So it's very, very good. Where were I? I was carried away. Sorry. Yeah. Let's go back. Okay. So, yeah. It's very useful. Powerful tool. Cookie cutter is here if you want to take a look at it. Cookie cutter, pie package. So, yeah. Next slide. Okay. So, the journeys begin. I become a developer for free. I want to be a developer but like I want to get paid but so far I haven't been paid so I am as I said I wasn't you know working as a developer before I was data scientist I am still a data scientist I'm making a career change talk to me later but if you're hiring so yeah I'm trying to do some TDD which is test driven development so So what happens is, like, when you write the code, so, oh, maybe I should also ask the audience who know or use TDD, yeah? Okay, then I don't have to explain too much, yeah. So at least I try. I try my best to write the test first and then don't write the code first. But I'm not, like, I think practice makes perfect. So I was kind of trying to do it that way, but sometimes I was, like, cheating, you know, write the code first. And, yeah, so I would try better next time. But I do create a lot of tests. It's actually, I feel like I spend more time writing the test than writing the code. But yeah, the coverage of the tests now from that coverage library that I use is telling me that 99% of the code are covered. Yay! But there's one case that is a negative case of one method. So if you want to cover that case, there's a ticket open. and you're welcome to contribute. So go look for contributors, please. And then, oh, yeah, I'll write a lot of tests. And also I try to make sure the tests are well-named because you have a lot of them and you want to know what they're doing and when you have a test that fails, you will be able to know where is it failing, why is it failing or what is failing. So, yeah, so make sure that usually I do is like I write the name of what I'm testing, like the method or which class or whatever. And then the case, is it like a case for testing it for, let's say, empty input or let's say it's like input of a negative value or whatever. So, yeah, make sure you label them. Also, documentation is very important. Again, as I said, you want people to use it, right? So make sure that people can use it happily so they will keep using it or tell their friends to use it. Dock string is a must because people think, oh, I can Google documentation. Why do I have to read the dock? Why do I have to write dock strings? Because there are people who code on a plane. Or without internet. And that really helps when they can, you know, you can read the doc string and understand what it does. When you can't Google, oh, please show me. I don't know. So, yeah. Comments, because people are going to contribute to your code, some comments really help, especially you have a lot of funky logic. Because I remember in Days Util, there's, you know, the parser is a very complicated thing. And there's a lot of complicated logic there. And there are some really good comments that really helps me to understand what that check is doing. So, yeah. Yeah, make sure you have more documentation. And then also writing the example is very important. Recently, I added a quick start guide here. Actually, somebody wrote it for me because I got contributors. But, yeah, I tried to put it on the first page as well. So when people just come across your library, they will think, oh, okay, what it does? And then, is it easy to install? Do I have to have some prerequisite before I install? And what is it for? How can I use it? So they have a brief idea of, is this library for me? Is it for what I'm doing? So I try to make things in one page so people who came across just randomly would know it exactly what this is doing. Instead of, you know, oh, maybe, oh, what is this? It's nice, and then I click on the read the doc, and then I read the usage, and it's kind of, you know, people will just forget about it, you know. So make it easy for people. That's very important. Yeah, quick start guide. Some, some. Yeah, besides those, of course, there's coding. Good code is very important. I refactor my code a lot of times because I don't like how things were designed and how, you know, even though it's working, I don't like it to be put in that way because I want it to be very well-structured so people who want to contribute when they pick up the project, it's very easy for them, very straightforward. There's, like, only two classes, and then each class is doing different things, and then you can put them together. I want to make it very easy to understand. Try not to do ugly patching I do it sometimes at work And know when you're sharing the code With everybody on the internet You want it to look nice Because everybody's going to see your code And your future employer may also see your code So, yeah And then follow path 8 At least I tried Yeah, I tried my best There are some tools to help you You can use black Personally, I am still afraid that Black would change my code too much. And so I was, oh, should I use it? Linting and use fake age to check it. My fake age report is horrible because I'm not doing very well. But yeah, but at least like try to follow pet age so everybody can pick it up very easily. Yeah, so fun bit, marketing. So why do I say it's fun? Because you create these things, right? You spend, let's say, your whole weekend Stay in your flat. It's in your basement, you know, like typing away and coding. Oh, my God, it works. It's amazing. You want to tell everybody that you did that. So tell your friends. Use social media. I use Twitter, LinkedIn, even WhatsApp, and then my friends mailing list. Yeah, so tell your friends. Give licensing talk. I gave a licensing talk in PyData London, one of the meetups. And then what happened afterwards, because at that time, I don't have a logo. And I just, you know, give a lightning talk. And very sneakily, like I say, oh, I have this, by the way, I need a logo. So, and then people do come back to me. I got one person, I really have no idea how that came through. But one person's telling me, I have a friend, he or she is a graphic designer. I was like, yeah, that person can help. I was like, whoa, great, really? Yeah, I can't believe myself. Yes. And then I got a logo that is actually designed by this designer called Agus. You can see the credits here. Here. Yeah. So this is the designer. I need to give some credit. So I will show this designer's website. So it's very cool. Professional. So my logo is nice. So if you like it, go check out the website. um so i got the logo and then i'm i made some stickers um so there are lots of companies do stickers i don't want to advertise for them so i don't mention the name but um but yeah get some stickers it's not too expensive it's reasonable amounts of money get reasonable amounts of stickers and i give it to people so i have some in my bag come talk to me afterwards if you want sticker of pick and mix so come talk to me um also because i help organizing london python sprints i can recruit contributors because um we have people who are keen to contribute together regularly and then there'll be people who want to contribute to some already well-known libraries like scikit-learn and pandas but there are some people who are just curious and who want to try out and don't want to work on a super complicated, sophisticated project, then I'll be like, come on, come sit next to me. We can work on my project. So, yeah, but like everybody was having fun. I mean, my project is very, you know, young and very easy to contribute. It's not complicated. So, yeah, so welcome. If you want to contribute, talk to me. And Hacktoberfest, you know what? But recently, I got a lot of my issues, you know, there are people sending pull requests to fix my issues because I put the label Hacktoberfest on it. I can show you. It's amazing. Like, last year, I was, you know, participating in Hacktoberfest. I was contributing to, you know, those, like, Day2Till, Scikit-Learn, like, Keras, and all these, right? But this year, I was like, okay, maybe I should give it a go. So I put some text on it, Hacktoberfest. And then actually, oh, I can show you the closed ones. Actually, there are people who really found my library by the tag, and then they sent a pull request. You see, like, oh, these are closed because they're contributors that I haven't met. They are from the internet. Yeah, that's amazing. And then if you want more stars and folks for your repo, that's a good way, you know. doing it, HackTurbanFest. I'm advertising for them because this is really amazing. And you can win a free t-shirt. Like, yeah, just four pull requests in this month and you got a free t-shirt. Why not? I already done it. I can show you. Yeah. Please load. They're busy. Okay. Yeah. Four pull requests. I have a t-shirt coming soon. Almost there. Yeah. So, yeah. And join that. and they really helped me out, so it's pretty good. So what else I can do? I'm thinking you can also do that as well. Writing a blog post. Recently, I started writing on Medium. I uploaded some workshops there. I think I could maybe create more other posts, like some write-up for my talk maybe, or maybe I can tell people about Picamix. It's an advertisement. At the same time, you know, sometimes you got rewarded when you write a blog. So you can do that as well. Create more tutorials and talks. I did create some tutorials before for like building a chatbot or doing reinforcement learning. So maybe I could create a tutorial for machine learning for people who want to use stacking and, you know, Try to sneaky, like, you know, use pick and mix, and that could do as well. So you're doing something for the community, but you're also getting all these contributors and people who know about your libraries. Yeah, talks, I'm doing it now at Python.de, so I'll put the logo there. So I'm also thinking about now there's only, you can see there's only a handful of contributors, But if you go big enough, maybe you would consider having a community Slack. Like Keras, they have a Slack. Docker, they have a community Slack. So that's very good. Also, when you go big enough, then maybe you want to have more contributors, more maintainers. Because sometimes I feel I'm too busy. I don't want to reply to people on GitHub or I don't want to read the pull requests. Then you need more people to help you if there's a lot every day, right? So maybe you can get people who contribute a lot to become a maintainer if they want. Also, you can create a mentorship program, which I think is really good. You can, you know, it's like a network, right? So you are here, you are the BDFL, and then you have all these, like, maintainers, and then they can have, you know, more senior people that they mentor, and then you can have the beginners here. So it's a pyramid structure, well-sustainable structure. So yeah, at the end, as I said, there's a lot of pull requests and all this stuff. I have to manage the project. Yeah, so again, communication is very important when you manage the project. When you have all these issues that you want people to contribute, besides putting the tag Hacktoberfest there, You also need to write it, you know, in a way that people understand what's the issues and how to fix it. Sometimes it's just a typo, but I would just write it there and I could fix it in, you know, one minute, two minutes. But I would make it into a very easy, very obvious issues, put the tag there, good first issue, hack super fast or, you know, help wanted. So people who never contribute or don't feel confidence making pull requests, they could try that out. and you can get new contributors. And then usually you have to talk to people. If they miss something or have a typo, you have to communicate nicely with them because they work for you for free. Also, you have to prioritize which issues are more important, what you can, like I said, tag it, write it very clearly for people to do, what you have to do it yourself. You have to really prioritize and organize. So, in GitHub, there's a project board. I won't show you at the moment, but you can check that out. Or use other things like Trello board or GoGarkon, things like that. Yeah, I always, you know, encourage and acknowledge people to help me. You will have your name on the authors.rst. Authors.rst, yeah. Maybe it's too time-consuming. Oh, I can show you. Okay. Here, here, here. Yeah. Yeah. All these people help me out. They are lovely people. I love them. Yeah, so acknowledge them and give them a lot of your love. Yeah, I do everything for free, but it's very rewarding because I learn a lot in software development because I don't get a lot of opportunity to practice in my work, so I can do it in open source. I made lots of friends. I do have contributors that, you know, if they're in London, I'll invite them. Oh, shall we have a private sprint? Come to my flat and we will have food and drinks and work. I got inspired by a lot of people and by other projects as well. I'm a BDFL of my own project, so I feel good about myself. So, yeah, Benevolent Dictator for Life. So that's the title for Guido who created Python. But you created something, so you can say I'm a BDFL of my own project. Yeah, so please start yours now. And then I know in this conference, people will join the Telegram, and I have created a Telegram group. So please take a picture, join it, and then we can discuss about, you know, what I talk about, you know, how to start your own project. If you have a young project that you want some help or get some ideas, opinion, And, yeah, we can talk there. I think I want to build a community there. So, yeah, that's all. And, yeah. Thank you very much for the talk. We have a few seconds if somebody has a question. So, is there a question? Yes, please. Lucky one. So, how do you select issues for DevSprints, especially? For the sprints? Yeah. Yeah, you have to, again, prioritize what is important, what you want to get fixed before your next release. And also a mix of difficulties, more very easy first issue because most of the people that are new to your project, maybe only a couple of them are more tricky ones that you've got to have close attention to that person and work together. Yeah. Okay, thank you. I think that's all the time we have. So let's have another round of applause.