Faster Workflow with Testdriven Development

Speaker 1 [00:04]

It's so cool that so many people are here at this ungodly hours, like just one digit times are not the best times. You can find the slides, especially for the people in the back, this might be useful there, and as well the template code, which is code because we don't have much done yet. Welcome to the talk, Faster Workflow with TDD, a bold statement, you know. Just let me get started. Who am I? I'm a Polyglot developer, so I like to choose a lot of languages for the goal I have, and recently it's Python, but you see a lot of more languages. I put Java rust and go in small brackets because I don't like Java and rust and go is very fresh. So, uh, yeah, it's, uh, it's working, but it's not very senior like working. Uh, at the moment I'm working at Magoo as an external consultant for the DB system. Um, I'm like 90% backend so you won't find any front end code today. Um, I can't tell you much about it. I guess most of you might know more, even if you don't do it full-time, about front-end. I have a background in science and insurance and logistics. Right now I'm at as an external contractor and before at the back-end services. And yeah, Maxplug Institute before that. And of course I love Python because it's awesome. all know that. For the new ones, you can find the code there. If someone arrives later, you might give them the link. It is not as hard to remember, I guess. What is the goal of this tutorial? Or my goal for this tutorial? I hope you share it. I want you to have the knowledge to apply TDD. After this tutorial you can start with TDD at your company or your private projects or your university projects or whatever. This is my goal. Another goal is to make you know the benefits of TDD and I guess we could arrive there too. Of course I would like you to get the curiosity to try this in your real world projects and not only just leave today and never touch TDD again, then I would have done a bad job. In the future, once you know how to apply TDD and how to use it, get faster in your workflow and get faster things done and have less anxiety releasing stuff because, you know, you're backed by PyTest. Before I go more into detail, who was yesterday at the PyTest talk? About a third. So you are very well prepared for that. The rest, maybe two. Who used PyTest in the past? This is nice. Awesome. Because I won't go too deep into PyTest itself, because PyTest is awesome, and we can do a lot of stuff there, and when you wanted to do a deep dive into PyTest, you would be there yesterday, because, yeah. Okay. What is the motivation for test-driven development? And one of the main motivations for me personally is that I always thought before that testing or normal testing is boring. Who feels the same yet? Oh, okay. No motivation for you today, sorry. The reasoning for me was because I always checked it out, my script or my code, and I knew it worked, and then I had to write the test, and yeah, that got boring because I already tested it by myself. Why do I have to write the test again? And writing the test first totally, yeah, removes that bad part. Another good thing is that we have some kind of, yeah, pre-detailed project documentation for other working members, colleagues, and maintainers after we leave the project maybe, which will happen better, in my opinion, in test-driven development than in other parts because you try to write a test which makes your code work, and then you write the code itself and the focus on doing this as minimal as possible and as expressive as possible is a very good thing, in my opinion. You have very short cycles of a working product, which is awesome, because I remember in the past sometimes, like, hours of work and then something messed up and, yeah, I had to start bug fixing. With TDD I rarely do bug fixing, not none, it's very like starting a debugger is once a month to a month maybe. Let's see. Maybe we got lucky today and I have to debug. Yeah. Very simple thing which applies to testing after as well. It's easier to test code. This is not good English. But code which is easy Easy to test is also easy to maintain because we apply the same principles and making easy to test code is a basic principle of TDD because we are testing all the time and out of pure laziness we are trying to make it easy testable. And just some points before, onboarding new members is easier because they know when they break something, they know how stuff works, and they can get started a little bit better. And yeah, of course, the feelers deployments and refactorings, I don't know how many of you had old projects where someone, some senior said, don't touch this part, this worked all the time, and when you touch it, you break it, and then we don't know what's doing anymore, And, yeah, this is, in my opinion, a code smell and shouldn't be happening. And TDD will make this far less often. Maybe completely mitigate it. Okay, which project do we do? I thought about an API for an URL shortener backend. It's very minimal. It's very precise and small. And we can start Greenfield, which is easier for TDD to get started with. And at least there's not another to-do app. I don't know about you, but I did a lot of those in various languages and I don't want to see any more to-do app. And I hope you feel at least a little bit the same. Okay. Theory, just a little bit, but I keep it to a minimum, promise. Because it's boring. Okay, with TDD, we always start with a failing test. We write a test, a minimal test, which tests some part of our code we want to write, or a feature, and it has to fail. We test that it fails because a test which we write first and doesn't fail is not very useful. After that, we go to the green stage, where we try to make this test green. Like the name says, it's pretty awesome because once you get the hang out of it, you kind of has the psychological phenomenon that you try to get always everything green again, which is totally motivating. And especially for people like me with ADHD, where we constantly switch to other context, it's easy to always, yeah, get us back to focus again. Yeah. And the last part is refactoring. Because the test might be minimal, and it might be even bad code and the green part might be bad code because we are writing the minimal stuff possible to make it work. We need some kind of buffer to make it work beautifully, and especially with Python, we want to make it Pythonic and readable. So in red and green, it's totally okay to just copy and paste code. It kind of feels strange at first because, yeah, double of code is not the best idea, but with the refactor step, we can make sure that then everything is working because we have the test, we have the code, and as well as beautiful. Yeah. Okay. So the full cycle. So why in this order? I just told you because we have small incremental changes of working steps. And in my opinion, a good analogy is video games with autosave. We have this constant autosave where we can go back to a working product. Of course you should use Git with it. I won't use it today because, yeah, different topic. But in general it's a good idea after one cycle to save it, to commit it to a Git history. And yeah. We don't have new functionality without a failing test. We don't write new functionality without a failing test, which, yeah, is part of the whole idea and then one advantage about it is we are forced to think about what we do we have to know first how our code will work and after that we write the code so we have to think about it which is great sometimes it's more like instinct to driven development in the past especially at University and later We plan it out more. And, of course, the refactoring step, again, it ensures we don't leave technical depth for the last two stages. So we have to do it if it's possible, of course. What I won't and can't cover is design principles like dependency injection and so on. So this won't be here today. It's important. It's good practices. But, yeah, it's only one and a half hours. the best or even okay solution, so we will do it in a test file and a main file and if we have time in the end, we might clean it up a little bit, but I will put it in two files because the focus on TDD and not a beautiful project structure and architecture. We won't do a clean separated architecture like DDD or something like that, what I prefer, so it's easier to see. and keep in mind one and a half hours we will see how far we can come it will be how far we can get and yeah I think this was pretty fast maybe okay set up you can find already the the code zip in the link I showed you but for everyone who is late or just to tell you what is in this package, we have some project directory. We will build ourselves a virtual environment. Who doesn't know what a virtual environment is? Nice, I can skip it. So we activate it. The lower two parts I just Googled because I don't know how Microsoft stuff works anymore since XP also. And you can install it directly for everyone who is new or you can just install the requirements which is not highlighted also. You can install it via the requirements file. This is the part you should do manually because I didn't put the VN file and folder in there. And after you download it, you will have additional files like conf test and the initpy to make it a module and so on. Yeah. I would wait a short time. Yeah. Can you share the link again? Yes just a second. You will find a beautifully designed page because, as I told you, I'm 90% back-end and I see front-end code and I'm, yeah, it's not as beautiful as it could be. Okay. So we have this, we have this, this, this, this, awesome. Okay. We should have some kind of like this structure additionally there will be the requirements file which you can use to install your stuff for your for your virtual environment And of course the VN file folder and yeah Okay, I would wait a short time until everyone's ready who's Not ready yet. Oh who's ready yet? So it's a little bit easier. Okay, like two-thirds. I wait a little bit more While we're waiting, if you don't, because you're listening and you want to write and it might be a little bit too much, in this case just listen because I will upload everything and you can do it again in your own pace. I will even do some rework after that to make it a little bit more fitting to the questions and make it at the steps where we do it beautifully and so on, this will be finished like today, maybe tomorrow. I wanted to do the code challenge, so maybe I don't have time today. Who doesn't want a Nego R2D2? So who's finished now? Finished. Yeah. That looks a lot better. Awesome. Okay. I will jump to the terminal soon, but the first part recap, we have this red part where we are starting, after that we will try to make it green, and then we will refactor it. First part. The first part at all is for me, I thought, yeah, I want to write a URL shortener, and what is a URL shortener? It maps a short URL to a long URL, and we can create short URLs by putting a long URL in there, and I want to do the domain logic, the basic business logic first. And I thought about a class like URL pair, and I put there a method called test model exist, which is very basic, and we have a long URL, which is, yeah, not very compliant, but we'll write a test here, so it's okay, first. And I'm not doing any more until here. And if I run pytest after this, so we are importing it. I called it shell, if anyone is asking what this means, it's just for short URL, and I'm not very creative, I just took the first letters I guess in Berlin I should drop the vocals but okay and of course you can choose shorter names okay everyone should have Python in the virtual environment and you should already have the VN activated, and when you run PyTest in the folder, you get an error, which is what we want. We want to start red. We want it to scream at us, you are doing something wrong. Please fix it. And we are happily going to fix it soon. You can find the slides online on this link. So I will go on to the next part. We want to make it green. So I am creating a class, which can have a long and a short string. I'm using data classes because they're awesome and perfect to store data and I'm not diving too deep into the fast API stuff which is awesome, too, but different topic, different time. And after this, wait, sorry. We have it green. Is it large enough for the people in the back? Yes? No? Oh, sorry. This is why I asked. Better? Okay. In the worst case we just have to take one terminal and we will be fine too. We tested it, now it works and tells us yeah, nice, wait, let me source. We tested it, everything works. We have our first green, which is not as impressive right now because we are doing not even junior developer stuff right now, I know. But after green comes refactor. And we don't have a lot of code here. So there's not a lot to refactor right now. But refactor can mean our workflow and how we work with our stuff. And running PyTest all the time is not very nice, doesn't make fun. It's manual work, and we have computers for that. And for this, there's a PyPI package called PyTestWatch, which will watch our changes, our code and our tests, and run every time we change something, which is awesome. So we can start a terminal, let this run. So minus C says clear the screen after each rerun, so we just see the failed test or not the fail test, and everything is awesome. It's called PTW, which looks like PTW minus C. You see when I, I don't know, make stupid stuff like just save this, it will rerun again and we don't have to run PyTest all the time, which gives us a lot more time. Additionally, You can of course use the features from VS Code and PyCharm and if you're a Vim freak like me, you can just run like a small shortcut which will run it, whatever floats your boat, but in general, make it as easy as possible. We don't want tests to be annoying. Okay, I have it here and what I'm doing now is I will quit this, I will make this a little bit larger and start a new terminal because what I'm in my terminal I have a small bell in the lower part I don't think you will see it in the back but I will tell you if I see a bell and the belt tells me yeah there's something a test failed and so I will switch forth back and forth okay so let's source it again so we have this active also okay yeah who can tell me what the next step could be no worries the next step we start again with green we want to make a new red we want to make a new test which will fail and our test yeah is like this model exists which is not very useful for us but we want to test which might be a a little bit more useful. Like we don't want to give it a short URL, right? Because it should generate it for us. Now our next test will exactly be that. Test model generates short URL. And I have this. And I will create my pair. And I will give it only a long URL. long URL, call it whatever we want. You see I already have some kind of message telling me I'm doing something wrong, which is great, but of course I want to have PyTest doing that for me in this case. It's okay. I should run PyTVC like this. And you see, yeah, it's It's red, awesome. What do we do for that? We go back to our project, and we have a lot of system, yeah, close it, close soon, so I will start with an init, and of course we don't pass, but we won't, so long, I won't do type check types here. I do types all the time, all the way, but in this case it would be too long and it's not really helping the code in this case at least. And so what we can do is now we repeat what we did before, we save it, and I would make it a little bit verbose, I know. And what we want to have is some kind of minimal code to make the test work. And I will get it just a standard default state value which is an empty string. Not very useful, but hey, it's a minimum amount of code to make it work. Just I will switch back soon, but you see it's two passed, awesome, okay. So I don't hear typing anymore. Are you finished or not? Who's not finished? Okay, I wait. Yes, yes, we can go back to the test. As you see, it's still not asserting anything, so it's very, very, very basic. And so we don't have the full test yet, and we can assert it, that we have actually a short URL pair. pair, and we want the short one, and yeah, as the empty string is equal to false in this case, we can, of course, be very verbose and make it a bool, but I'm not a fan. And when we run it, the people in the front see that I have a small bell in the lower part. That sounds strange. Which tells me, yeah, it's not working anymore. What we wanted. So of course we can now cheat a little bit and just put an A as a default value which is of course we are thinking beings so we can sometimes take two steps or more steps and be faster, but to be honest, I would like to make sure that we have at least a short URL which has at least a little length like four characters. Because it's only for us, four characters would be enough, like all characters times to the power of four, so So I will make it like this, it will make it like this. And well, this is a more usable test. I skipped ahead and didn't just do a default value of A because as I said, we are all thinking beings, we can, this really small steps we can just jump ahead and because otherwise it will get annoying if you write it every day like this. Okay, we have this thing here, and now we have to be more creative, and yeah, because I'm still a cheater, I will do it like this, and awesome, I'm having working code, yeah. What we now want to do is force us to do proper code, and what we want is if we have different URLs, it should generate different code, and then we will be forced to use some random character combination. Let's start with a URL. It generates this but we have a new test and you see I will do test model generates random, you don't have to type the whole thing but it's maybe better if you see it and I will rewrite it as first URL and second URL and very long other URL and we want what we want is asserting that first URL short unequal second URL short not sort which will bring me my ball again which tells me not nice you broke something and normally I would do some regex to check if the short part has some combination of letters and doesn't contain like a percent sign or I don't like ones and zeros and O's because you kind of miss them and I will put them away. But in this case, this will be sufficient to implement our next part of the code. We'll step to the next part. What we now have is not this. This is very, very bad. We want to make it green. And what we do is if short is none, we didn't give it some short URL, then we will generate one. And for this, I'm doing some very ugly coding here, you will see it soon, but we are not at the refactoring step yet, so let's do it. We need random because we want to select some random string. We will need string because we are working with strings and nobody has time to type A to Z all the time and it's not nice to do it. Then what we are doing next is we are having, we're using choices, so random.choices. Who doesn't know what choices does? Okay, choices is something from the battery toolkit of Python which gets some iterable thing like a list or everything which is iterable, for example a string, and it will give us n types of elements from there. from there so if I put in string ASCII let's take all letters and with K I can tell it that I want four random letters what I'm getting back is random cars so and the random cars list we can't do much with the list what we do want is save, in short, our, let's do it like this, our random cars, not cars, characters. So I will switch quickly over, yes, so as you see, we did some very, very ugly code, But we have it working, and now we are in the blue part where we can clean it up again and make it more beautiful. Maybe not the most beautiful thing you have ever seen, but okay. Of course, we put our imports to the top, easy. No bell in my lower tab of the PyWatcher, PyTestWatcher, which is awesome. Then of course, having this in here is, yeah, not nice. So what we want is some kind of randomized string. Let's call it like this. Yeah, make it a method here. And I have to fight the urges to put types everywhere. And yeah, just putting the fun stuff here and return short. So now we have this and this, stop. If short is none, then short equals randomize, nope. Randomize string. Ah, I'm missing the self, right? Sometimes I like it more if it's everything in here. So I will give it the value and if the value is none, so it's a little bit nicer. So we really just have an initialization there and not a lot of logic. Yeah, let's do it. So we give this a value and it has to take a value if value is not none. re-return it directly. Why I'm doing it like this? I don't like to have a lot of indented code. It's a thing I don't like, and I think it's a good thing to not to do it. What we are doing now is putting this, it will be a little bit more cleaned up soon. We have to put a short value here. So let's, I didn't break it. I hoped it would break when I'm randomly trying to change stuff out of plan, but, yeah, I didn't break it. Like here. Everything still works. Yeah. Now we have a URL pair which generates a short URL, some kind of. And what I want to do is test that it really works. So I test my test, which is kind of strange, but let's do it. And I break it by doing this. Now it should be a bell there, right? Yes, we have some... Yeah, that's not the same. Awesome, that's what we wanted. And let's change it back. Should I go back to the main file or to the test file? Which is the one... Yes. Thank you. I will stay short at the test file because it should come first. And a reminder, I will upload everything after that, so if you just want to follow, it's okay too. But the best part of tutorial sessions are that you can type along and try other stuff. Okay, we have, once we're finished typing, we have the basic core logic in place, which is extremely complicated, generating just a random string and might be not the most complicated stuff you did in the past few days. But we want to go further. We want, of course, to make some kind of API. So I think for now, our model is working like intended. Maybe we should, of course, we could add validation and so on, and maybe we do it later, but for our minimum viable product, it's enough. And, yeah. So, can I switch to the main file, everything okay with that? Okay. It's not very complicated, not very beautiful, but, yeah, randomize the string, and we have this here. Okay. testing our API. Of course we can try to start our server and then send requests to it and normally this would be the very first thing we do but who has used fast API before? So around 20% I would say. Fast API is a little bit, yeah, it's a very small framework and a very fast framework for pulling up an API and starting an API. You don't have to know much about fast API to get the next step, but keep in mind this won't be the most beautiful fast API code you've ever seen or you will see in the future. It will be just to bring you the concept of TDD closer. Yeah, let's start with that. For fast API, it has an awesome helper test. We can reach an API. Now let's call it our API. And fast API, from fast API, test client, import test client. You see, I will do it again here, even if it's bad practice, because it's faster, and I have the reflecting step later, and so I don't have to worry about doing ugly stuff now. And the test client from a fast API is a class which takes our API object we will create later. And with this, we can send requests to it without starting a server, which is a lot faster. We want fast test. fast testing, otherwise it would hinder us and be a problem in the future because we have to wait for tests until they finish and that's very annoying and we want not our code and our tools to be the slowing factor of our work. So I will start with a client, a test client, and I will insert, you see it's an ASGI app, async. I will put an app here, and the app should be from, let's call it app from, oh, the app will be imported from show. You remember this is the module we created in the very beginning or had in the test folder, and we have the file main there, and we want to import app. And this is already enough to tell us that we messed it up. on purpose of course. And you'll see on the top front line, you'll see that I have the bell again, which tells me, yeah, you broke something which was intended obviously. Um, I will jump over so we see cannot import name. Okay. I will go on of course to the main file. Okay. As I told you, I won't do a lot of splitting of the code in what we normally should do. I will keep it all in one file so we have not too many files to switch back and forth. We want to make a FastAPI. Okay, I will import it manually from FastAPI import FastAPI. So Okay, we created it. Which is very boring right now. It's very just, we're creating the object, and the test client takes this, starts the server, has no information at all, but at least we can do it. I will just jump here. Everything is green, everything is awesome. Shall I keep the code here? Hands up if you think so. No, shall I go back to the testing code? What was the command you ran, do you have it? It's installed with the requirements text file. If you didn't install it, you can do like pip install PyTest watch, which is one of the most used plug-ins for PyTest I'm using. So the minus C is there to clear the screen between each test. So I like it more like this. And if you want to have it really verbose, you can do the minus minus tells PyTW that after that we put the PyTest stuff. So for example, let's do it like verbose. Then it looks like this. And we have each test. I will leave it like this. So it's a little bit nicer. Okay. So we have this, which is not yet working at all. Oh, and it's called a really ugly name. And what we now want to do is get, of course, some URL. Let's call it API. And on the API, maybe we have the endpoint of URL. And this will be our response. And we're doing our assert now, response status code equals 200. Who doesn't know what the HTTP status code means? Especially people from data science might be confused. It's each website you are calling, you get some status code back. And 200 means everything is okay. 404, most people already know is yeah, here, nothing to find here. And we have all kinds of status codes. And for now, I have it ugly. Of course, we could use some speakable names and not some magic number nobody understands. But hey, we are going to have our refactoring stage for that. OK, any questions so far? We can just take a short breath. So we are doing the second part of our software, is building the API which means we are not so far away from our working piece of code or working minimum viable product for our startup to finally bring the first short URL service to the web and being a little bit late for that. Okay. So save it. We have a bell. I will show you. Yeah, we got a 404, and it's of course not the same as 200, which is nice to know. And what we are doing now is we are going back to our app, which is like this. Now it's some fast API specific code, but what we can do is we can write some function, like index function, let's call it index. It's maybe not the best name, but it's working. So what we want to do is give it a decorator and just make it like API URL, right? Yeah, yeah, two blank lines found, okay. And as you see, we don't have a bell. Let me show you, everything works, awesome. I guess everyone had some contact to decorators. who doesn't know what a decorator is? Okay. Uh, the decorator here is this at sign and some function we can give it, which reps our, um, whole, um, function we have below there. And, uh, it's a little bit of magic, which tells, um, fast API that our index function will listen to a get request from API URL and it will return nothing. It will just, uh, pass. So, um, If you don't return anything, we will have, yeah, we will just have a 200 code, for example, when we send stuff. Okay, this is the minimum amount of code. Let's get to the refactoring stage. Who isn't ready yet? Yes, I can. Here. I can. the test would be the part I'm now refactoring of course because this is very ugly and after we refactor it we will get started at this test again because it's not right the complete part of the software okay I will start here of course having an import directly here is not as elegant especially as we don't need it to be there which is very seldom the case that we actually needed in place here. And let's go to the top and import it here. And you see I'm already having some duplications. Save it. You've seen I've saved it, and each IDE can have the possibility to reorder your import statements in a useful way. I like my imports to have the batteries included modules will be in the top then one space Empty line then the external dependencies and then my own project dependencies in this case Those are okay back to the part here we have another thing we might want to improve like having a fixture which gives us a client and might having some kind of method which can send stuff But I'm seeing an issue here, and this is the magic variable 200, which is not very speaking, especially if you have new members of the team who were here and didn't know that this means everything is okay. We should either write some variable, but of course we have this already in the starlet, which is a dependency of fast API Import and you see we have a lot of HTTP stuff Which is very speaking because we have this number we have HTTP we have okay, which is I like it It's a perfect kind of variable, which is very speakable. So and yeah save it and if everything went well we don't have an error nice this is what we want in this stage at least so okay when we want to send an URL to our API who sees a problem here okay when we send something we want to not get it we want to post it so we are sending a post request which is more or less some kind of string in the header of our request we're sending to the server and we're telling it to yeah see I want to post some stuff to you and for this we want to post of course URL and the data I will keep empty because this test is only to to make sure that we can reach our API but yeah after this we should have the bell which tells us yeah you did something wrong and we say thank you because this is what our test is for I will make a short overview soon so we know where exactly in our project we are time-wise and yeah it's actually pretty easy to fix this make it like this look back here and now everything works again okay I'm changing this from get to post is pretty easy but the test is of course a little bit more code let's split it a little bit. In more complicated tests it's nice to have some stage where we prepare our application then we have some action we do and then we have some validation to do. In very short tests I keep it more together because it's not as important to have an overview in just three lines. Who isn't finished yet? Awesome. Okay, let's switch back to here. Okay, we have the skeleton in the beginning. We're having the core logic, and we can access the API. What we don't have is something useful. Not yet, at least. And for having something useful, this means we need to post an URL and get a short URL back. So this would be at least a good start for our API. and for this, I randomly start a file browser, because why not? So, which will be our next part. Actually, we don't expect this to be the thing. We want when somebody posts to our URL endpoint, we want to have some data there. And if there's no data, we want an error. So, So I will fix this up. I will say no, I don't want it like this. It shouldn't be okay to send us no data because why are you using our service? You want a short URL, give us a long URL and we can help you. And of course when I do this, it's failing again, which is what we wanted, we go back. What we need is some kind of short URL. And in fast API, we have different kinds of receiving those. We can either expect form data, old school, like when you have a formula where you send and submit and this is the kind of data we get, this is form data, or we can send and receive a JSON body, which is in my opinion, it's very opinionated, I admit, is easier to work with. In fast API, to work with this JSON body, we have to create an object. And what we want is to receive a short UL. And for this, we will create a class short UL, which will inherit from base model, which we will import here, from fast API, no, from Pydentic, sorry. Check import base model. You see it's super ugly, but we have, again, we have the stage later. And the short URL really just has some field called, wait, it's a long URL we're expecting, sorry. Long short, yeah, whatever, long URL, URL, okay. And the URL, of course, should be some kind of string. So we're expecting a string for our API, and for our API to expect this, we have to put it here and say call it long URL is a long URL object for this I will do typing because fast API uses those types to create a swagger interface who doesn't know what swagger is okay I will show you soon normally we don't need it actually but it's nice to see something what we are doing here okay I created this I will import this and you will see I have this module level import not wait I would not not yet let's go here and see everything is working again awesome let's make a very short detour and I will show you our application right now which is yeah not as impressive as it should or could be but UV corn is some kind of server which can create our fast API application and starts it and give us some interface to work with it and if we do this you see we have this 8000 port and wait a second it's the same port for my presentation so it should not be happy about using the same font okay let's start localhost 8000 detail not found awesome this is of course we have given didn't give it a root but under Docs we can see a full documentation of our API which is very short at the moment but not only that we can see what we are expecting we're expecting a JSON file containing URL with a string in it. And we even cannot try it out, so. And response body null, because we didn't create it yet. But this helps a lot when you work together with front-end teams, for example, and they need, yes? Sorry, again? Can you zoom in? Ah, yes, sorry, yeah. That's the Swagger, right? Yeah, that's the Swagger interface, which uses an auto-generated OpenAPI JSON file, which will generate it from your code, so you don't have to document this part yourself, which is awesome, because I don't like writing documentation, and of course, sometimes it's necessary. In this case, it's not necessary, because the application does it for us. Everything which is boring, is nice. When it's taken out of our hands. Okay. Let's quit this short detour. This is what we are using. The guy who invented this framework, by the way, is on this conference. It's St. Sebastian Ramirez and he's pretty awesome. Yeah. So back to our testing, which which is here. Okay, we have green code, of course, we can now switch back to our blue stage where we refactor everything to be a little bit less annoying, like middle in the code, a statement of importing stuff, and put it at the very top. And what else do we have? Of course, we should normally put our domain logic in an own file and so on, and in my test, we didn't have the time for it. So we have to skip it. Okay, next step. We have to think about the next step because right now it still doesn't do any useful stuff when we are getting here. So what we can do is go back to our test and write a test for the next step we're expecting. What do we expect? We want the first usable part of our application. test, we get a short URL back from API. Call it whatever you want, but I will create again this. Let's just copy and paste, because that's not what we're supposed to do, but we're doing it anyway. We want to send data. We now know that there's a URL field, and we know that we can send it a long, long, long, long, long, a long, long URL and send the data here and assert response status code. So it should be HTTP 200, okay. So this should be pretty easy. Let's quickly look, 4, 2, 2, strange, what could this be? Mm-hmm. Mm, yes. Yep. Wait, it's like this. We can look into the response code and let us tell it what's going on. It's a little bit more real world example where everything doesn't work all the way right in the beginning. Let's put the text here and of course our code runs again and see, we get a message. Value is not a valid dict. Okay, nice. So which value, yes. Sending data, I guess we can send JSON. And after this it should work. Just to show you, I don't see a bell, which always is a good sign. Normally I would have side by side so I directly see it and don't have to switch the tabs each time. We now have this kind of stuff here, but it's of course not yet useful in the test. What we want is actually response. In the response is JSON. We can just use response.json to get the JSON, and from there, I will clean it up soon. No worries, I know it's ugly. I want to get the short URL, for example, or the URL, let's call it short URL, and of course I don't want it to fail after that, but it fails. Just a short, to show you shortly, none type object is not subscriptable, which is, who doesn't know what this means? One type not subscriptable, this part here. Okay, great. In Python 3.10 and 3.11, a lot of the error messages got better, which is nice. So update as soon as possible. I love the new error message feature. Okay, we have again the red part where we have a failing test again. So you see, we always have the cycle, red, green, refactor, red, green, refactor, red, green, refactor, and so on. What we now do is, again, just getting this to run. And for this, we want to create our, we, of course, could, again, start with some random stuff we give back, which is not nice. But we already have our core logic. And, of course, it's OK to use our core logic. We create our UL pair. And we have it here. and we will put in the long URL string and no short URL string, of course. And what we return is, for now at least, short URL should be URL pair short. So, And everything should be green, just a second, like this. Fast API will behind the curtain will start using this dictionary to create a JSON from it and will send it back. And fast API has actually a very nice feature, you've seen the Swagger interface, and for now it will only know that we give it back some dictionary with some data, which is not very useful purely for documentation purposes it's nice to use an object here as well so what we want is like and of course we hope that we don't break it with this so this is not let's call it short URL response base model and call it short URL which is a string of course and what we now can do is just return short URL response with short URL can write it directly like this short of course it's nicer to actually write some kind of code which translates our URL pair to our to our short URL response but in general when you have what did I do on the URL pair see break it all the time and what we now have in the swag interface who would be who think it's nice to look at the swag interface again so we can see the benefit of this part. Nobody? A few. Okay, we can do it. I will show you, because this is something which is purely out of documentation and maybe data validation. And let's go back here. Yes, we have... Go back to the Swagger interface, which is like docs here. And if we click here, we don't see it yet nice good catch myself and what we have to give it as a response model now it's working response so with response model we tell fast API what to use and what to do and for this we have to restart the server okay now Or again, I will go back to the code soon, no worries. So now we see what we get back as a validation error. For example, if we don't give it a string. And what we get back as an answer, and we even can try it out. So if we try it out, and don't put a string here, but a number, for example, directly a number, it should give us some kind of error or not. Awesome, yeah, we can write a test for that, that it should give us an error when we are just giving it a number. It's translating the number to a URL in the backend then. You can either like it or not, but we can change it. Shall we write a test for that or shall we go further and try to make it more useful? Test, hand up, and make it more useful. Yeah. Product managers versus... So, we're here again, let me start again the PyTest watch. Just so you know, we are at 10 hours, 10 o'clock and seven minutes, so we have like at max 10 more minutes, so you have more time for questions. So, back here. Okay, something useful. Back to the test, of course. Not yet, let's see, clean up part is important still. The comma here just means after the comma we can put some error message when this fails which helps us debug our code. So what we are now doing is a little bit of stuff to refactor it which is important. The refactor part is the part which ensures that we don't have problems in the long run. So what we want is a client, of course, because we don't want to create the client all the time. And this is a PyTest fixture, and we are doing it like this. Yeah, we can do it like this. PyTest not defined, wait, not yet. Import PyTest, so. Okay, where are we? Here, okay. And because we had some working code, we can just refactor without being sad about it or being anxious that something might break. Again, I will do the typing here just to give me better type hints when I type, when I write my code. And put the same here and it's a test client. And so refactoring is normally, Everything should be green, nice, didn't mess it up. And yeah, another one we could do is like write a small helper method to post stuff to our URL so we don't have this all, but we only have two tests using it yet, so I don't think we should do it now. We should start using something useful, or test something useful, because we want to make sure here as well that we have multiple requests that we get different short URLs back. This is one thing. And the next thing after that, what we want to test is that when we have restarted our server, that it's still stored, that we can get it back. Or when we ask for it. So let's start with test that. Let's, how do we call it? test that we get unique URLs. We can use our clients now, which is nice. We don't have to write it again. And we want to have data, let's call it data first request. Very speaking I know, but long, long URL. and of course we want to have a soon second request long long different URL and each time we want to have first response client post again API API URL JSON equals data first request. I will do it like this because normally I like when those two types line up. I like it like this. And for this, we can just replace second and data Okay, what we now want to assert is that, of course, both are different. Response first, JSON, now we are getting a long line, sorry for that. short URL should be unequal to it's getting long but yeah for this we have second not seconds and it's not failing so this can happen to that we write in your test and it's already working sometimes this means we did something wrong sometimes not so always start with the test which fails I will make it same which should happen in yeah the math is easy but not in the head but of course yeah it works like this and I want to clean it up for example let's call it JSON first, and put the response first here. This makes it a little bit less annoying. And second, second, and then we can JSON first, this, no, not this, but this. And, of course, the benefit here is that when we change stuff, we still get all the time everything before tested. And let's see if it still works. It works. No bell here. But I show for the people in the back that everything still works and is nice and happy. And now we can get it. What we now want is not only to have some kind of, yeah, sending stuff there and then leave it there and we can't use it anymore what we want is to send stuff there and then use it later to give you for example a short link for the conference or whatever and for this we can implement the final or one of the final features for our MVP where we can then amaze the money givers that our application actually works and at the details after that like storage and so on. I like to keep stuff like storage to the very end because thinking about if you use a NoSQL or SQL database or flat file or whatever is most of the time done in before. And when you're in the project, sometimes you recognize, wait, I should have used a totally different technique for this, and now it's too late. And with test-driven development, we can make it possible that we don't need to decide it yet. We can decide it later. Of course, you should use some more popular architecture patterns like dependency injection and so on, but yeah, stop talking, doing the next thing. What we want, test that we can use a URL, short URL. Let's call it short, so it's hidden, our name is hidden here. So for this, we have to create a file. We have to get our short UL, and our short UL we have to use to request our next part. Short UL is response, JSON, and there we have a short UL field. And then we want to do a new request. So we can edit it later, but short response, I call it short response. can fix it later when we need to better name and we have to be fast now okay we don't want to post we want to get yeah let's post it in this case because we're sending a data and I don't want to make it more so API and we have a new one in this case short URL let's keep it JavaScript II with this ugly camera case and we want to send it data which is of course our short again a JSON data let's call it data short and we have to be really fast because short URL data short So we want to send it this data and we expect our response, short response status code to be okay. And we can directly use HTTP because it's easier. And we should assert it, of course. And we should use this. And finally, we assert that our, not doubly asserting, of course. that long long should be in short response text. So, what I'm doing with this is making sure that my long URL is in there. Yeah, implementing it is pretty basic. I hope we can do it in two minutes or maybe at most four. I will show you it's red which is not because of this but because of this, there's no attribute post. What did I do? Ah, C, we should of course use our test client. And now it's not callable, of course not. Okay, so now it's broken because we don't have this, which we expected. Okay, what we have now is dev. get short URL, let's call it like this, and give it an app post. What we expect here is an API short URL. And what we expect is, in this case, a short URL, which is, I will fix this soon, short URL. We have more or less the same like the long URL. We have some string we want to receive. and return it just return something right now we have right now is four to two so we have an error again sorry when I don't show the same editor pane for a longer time but I hoped that we could finish at least this part and it might not work sorry for that okay what we have here field required okay we are still missing a field here and short URL we expected. Ah, it's called short. So now we have as answer of course an empty text but what we can do now is to prolong the decision which database we use any longer that we put some database here which is an empty dictionary and save it there and as long as our UV corn runs it will use it the same and when we sense something here as an example we can save the URL pair So we can make a database long URL equals URL pair. What we're doing now is we're jumping ahead. We don't have a test really making sure that we have this database and the storage solution. And this is a break of contract with the TDD thing. So after the session or when you have maybe later at home, you can try to bring this to an end. I will put up on the same side the PyCon 22 thing. I will put the slide there and I will put the code there and I will put GitHub link or maybe GitLab. I like it more. Link there so you can re-follow my steps because I'm making it proper then and not only fast. And, yeah, let me start the, I can't edit, run the wrong, okay, let's get started with the final questions. Are there any questions left? Yes. Oh, two people in the back, yes, the one with the glasses. Oh. Thank you. Depends, sorry. It depends a little bit because the first question was, please correct me if I'm wrong, how far the abstraction should go regarding the tests because we already have the trade-off of making our tests simple and don't do much double work and having maybe a little bit of double work in the tests but don't have them simple again. So we have some kind of a coupling of the tests, which is a bad thing but might be useful if it takes away a lot of the pain. And to be honest, if it's pain for me, I try to not use. I try to go the way of the least pain because I'm very lazy and I don't want to do more manual work than possible. And this is the reason why I use Vim and take like 100 hours of configuring to spare me time of like two minutes of typing something again. And, yeah, strange mind. What, let's go here. And jump, not more jump. What, yeah, yeah, yeah, yeah, yeah. Final advices I want to give you is, yeah, this is not nice. but you can even go farther because with test development it's called test so we have all tests done yeah no we don't have we have just the happy part we just have the part which is useful to get our to drive our development but not the important stuff we didn't check for example if the URLs are valid so we still have to do testing and after that we still have to do more testing who hasn't heard of mutation testing okay I would say most of you mutation testing is testing your tests which is obviously a lot of inception but luckily enough we have a tool for that which can do that for us and what it does is it takes our code, it changes stuff like an equal to an unequal a larger than to a lower than and so on and a lot of more stuff and It looks if our test fails, our test suit. And if our test suit fails, we call this muta, this mutant. It creates mutant of our codes. And I want to advise you to at least look at it and don't use it every day, but put it in some pipeline or so and see how well your tests are to catch every error. And of course, with test-driven development, we don't get very far regarding our errors. Use your IDE. I really love all of those IDEs, they are awesome. PyCharm is, in my opinion, the best paid IDE. VS Code is awesome, really. Not for me personally, I like Vim, but it's really great, and I don't say it lightly because I haven't used Windows for quite a long time because I don't actually like most of the Microsoft stuff, but this is great. And all these IDEs have tools for you to rerun tests. VSCode, for example, have extensions where you only run those tests which are affected by your code changes. You don't have to run all your tests again, which is awesome. Try to find as much stuff which makes your work easier and helps you keeping lazy. Which is a good thing, really. And yeah, testing really can save you a lot of pain and your employer's money, and most of the time employers are more like, I don't pay for tests, I pay for features, but in the long run it's far cheaper, right, like really far. Okay, this is the final thing. Question, yes? Yeah. Yeah. You. Yeah, sorry, I have a lazy eye so I look at multiple people. Not your fault. I have a huge project, and if this should start every moment, it can be a big problem. Yes, it will. What I came to know, PyCharm cannot help me, changes affect our thousands of tests. For example, we have 15,000 tests. So this, I like to, for example, PyCharm, you have the feature where you can watch your test and you can just run one test again. Oh, you can't? They cannot help. Changes affect our main test. Ah, okay. So normally I would say try to decouple your code as fast as possible and only run those unit tests which are directly affected. You might miss some features which might be affected by your changes, of course. But you have to make compromises. In TDD, you want to be fast. Running everything all the time will take a lot of time, obviously, or even just a small portion of it all the time. My advice would be just choose one test you're working on right now, run this, and put some pre-commit, for example, loop for some larger part, so a group of tests. on your CI, CICD pipeline, which can run all the tests again, and because, yeah, you can't do anything else, so hopefully you won't find an error then, but you, waiting for this holds you to finish every time would, yeah, make you very slow, and your mind, my mind would I wonder and I would think of all stuff maybe, but not the work because I'm waiting for the test suit and that's neither productive nor fun. And I hope most of us are having fun while programming. Maybe. Yes. Yep, the one with the, just that app, yes, easier. And, oh, okay. Yeah, the metal head. It's easier. I just take some features out of features. Sorry I want to do this, and then spend half an hour in the lab. Do you use TDD for learning? Yeah. It's a little bit more difficult, I admit that. But what you can do is, nobody forces you to do TDD all the time. What I like to do in new frameworks, to learn new stuff, I like to make some kind of exploration phase where I just type stuff. if I finally understand how the framework works and how the modules work, I go back, I delete it. I really delete even working stuff and start then with TDD again to make it proper because in the exploration phase, I do a lot of not so nice stuff and not the best parts and I'm breaking a lot of stuff but I don't have any tests for seeing that I broke it. And so the exploration phase I think is could be an optional fourth path with a new color. I don't know, let's take yellow in this, which might be optionally added to that. But then go back, because we want to, yeah, it's a disciplined coding technique, and yeah. Okay, it's one minute left, and I will finish it here. You can, of course, ask me afterwards, but I don't want to take more time of the next people coming, and this wouldn't be nice. Thank you.