Learnings from migrating a Flask app to FastAPI

Speaker 1 [00:07]

Thank you for choosing this talk amongst all the other interesting talks and to get the blood flowing after all the cake I want to start with a couple of questions. Who in the audience has used Flask? Just a little hand sign. Okay, so almost everyone I think. Who has used FastAPI? I think there's quite a significant overlap in people. And who has migrated a Flask app to FastAPI? Okay. So, I think one person. Okay. So, I think we're in for quite a ride. Who am I? My name is Aurel. I'm, as mentioned before, freelance software and data engineer working for different clients and usually doing data integration between data source and the machine learning. So, I don't do any machine learning usually. And the first reason is, is there even any good the first question is, is there even any good reason to migrate a Flask app to fast API. And I think if you have a working application, the quick answer is no. Because never touch a running system. But there can be reasons. For example, performance. Maybe you need better data validation. You've heard of async and want to get deeper into async. And maybe also developer experience. So, for us, the main goal was not to migrate from flask to fast api this was more of a let's say collateral damage in the project because we wanted a database integration better code quality in our code base and better testability and the migration to fast api kind of promised to help us achieve this before we dive deeper into the rest of the talk short disclaimer here this is like my experience and there's like your mileage may vary because like different projects have different requirements and different teams have different experiences so you might have different experiences and don't agree with with what i have to say here what i want to talk about is basically three things data modeling comparing flask and fast api then have to write a controversial statement as async is overrated. I think after this chapter, we understand what I mean with that. And also, I have like a small collection of problems you will likely encounter depending on how you set up your Flask app. And last but not least, I also have like a short migration strategy that is kind of like a guideline for you. So, you can easily like get started with migrating an app if you want to do that. And at the end, we have a couple minutes for questions and answers. If you want to get the material, like the slides and everything, you can either use the QR code, which leads to my GitHub account, or you just go to the pre-talks page and can download it there, too. Okay. So let's get started with the first topic. The first topic is data modeling. And usually what we have in general is kind of like this situation. So we have a client that makes requests to some API endpoint and expects some kind of response. So what we have is an input and an output to our API. And usually what we are kind of asking ourselves is how can we validate that the data that goes in and out of the API makes sense in terms of our API and the kind of service we provide by that. And ideally, we also have some kind of documentation going along with that because the client usually is interested in understanding how to use the API. And the question here is essentially how do we deal with all the inputs and outputs in a good way that makes sense for us as developers and also for the clients? In Flask, we usually have something like this. We have an endpoint defined by the decorator, then we have a function, and what we usually do is we have some kind of like request object that kind of magically appears in that context. And to get the data from the request, we just do, for example, get JSON to get the JSON body of the request and then use it like a dictionary basically to understand or to get the data from the request into our variables that we can then use in the rest of the code. So that's what we do. We fill a fake database. It's just a dictionary here. And then return something for example a message that has user created successfully and a status code so far so good we can test this with with curl for example and we see it works just fine no problems there the outputs quite similar actually for example you want to retrieve the user information we define just another another endpoint, give it an ID, for example, as a path parameter, and then do a get on the DB to get our user information. We then return the user. This works. The problem here is we get a password in our response because we just returned the user from the database. I know, like, storing clear text information about the password is not, like, not ideal. That's also not the issue here. The issue is that we have no control or, like, very little control over what is happening in our output. So in general, like, to conclude this, the Flask app works with the request object to understand the request, and then we just kind of return whatever we want. And it's part of the response. is not required here. And in my experience, most code in Flask apps has no typing, like, type hints whatsoever. Even though that might be different for your code base. So, coming back to this picture. Do we have any validation? Any documentation? I would say no. We definitely don't have any validation. We just can, like, take the request object, get the body, and then just assign these bodies to the variables. And if we want to do any validation, we have to manually add this, which usually means we either need more boilerplate code, need some third-party libraries, and Flask Marshmallow is probably the way to go here. But even then, you still need to manually kind of, like, work through it to get a validation going. What's the situation like in FastAPI? It's quite easy, actually. You also just define your route. and then what you have is this user as input and the user in is the type of this input which is in FAST API just a Pydantic model so Pydantic basically provides the like all the type information and what the body looks like in this case and this is a very easy like simple Pydantic model but you can also go real crazy and do all kinds of validation during the stuff in there so So that works just fine. If we test this, it also works, just like before. For the outputs, it's similar. We also have a pydantic model, but we specify this as the response model, or alternatively, we just use the return type and for the route handler. In this case, we have a specific information about what we want to return. So our user out return model is just the name and the e-mail, so there's no password in there anymore, or also no other sensitive information, because sometimes we have, like, a config endpoint or something like that, and we don't want any, for example, AWS secrets in there or something like that. Stuff like this has led to, like, data leaks in the past. prominently probably some, yeah, location endpoints, location points for cars. And if we check this, it just works. We don't get any password in there. And the whole thing, like, the pidentic model works basically as a data transfer object. You can, like, specifically say what is in there and then return just this. This has the advantage that there's like full control of what the output is and before in flask we kind of just returned whatever the database gave us and if the database model changes for example we also get different data and the client might not be prepared for that and worst case is the client crashes too so coming back to this fast api inputs and outputs work pretty flawlessly with pedantic models they are just perfectly integrated. I think that's the whole point of the data validation there. And another thing that I didn't show here is that you also get automatic open API docs. So, you just go to slash docs and you get a perfect open API documentation that you can also provide to your clients, for example, which is quite helpful if you're, like, working between different teams, for example, or have a public-facing API. So, in this case, we can do, like green checks on both of these points another good thing about data modeling is that it forces you to kind of think about your application much more than if you're like if you don't have to do it because you have to think about what kind of information is required what goes into your api and also what is returned by it or maybe even more so what shouldn't be returned to it So, coming to learning number one here is that data modeling is something that you have to do. It's not something you can do in fast API because it just makes life so much simpler, and there's really also no way around that. So fast API is really like opinionated here, and I think that's a good thing in this case because it simplifies maintenance and everything like down the road. Okay. So coming to the next point, this is the rather controversial statement that async is overrated. I don't think async is overrated, but I think it's not always like the promise it has doesn't always hold, especially in APIs. Because usually what we see is that async means high performance. But does it really mean high performance? That's a good question. And I would say yes, but not necessarily at all times, because it's very easy to make mistakes. So general problem we have code like this all the time in tutorials, for example And what we have is this async route handler This also works perfectly fine here because all we do is return a message So there's just like one simple thing and there's nothing that can go wrong there Basically, or if something goes wrong there you have deeper problems with your system probably So how does async work? So the async runtime basically provides a single-threaded event loop. And it kind of makes sure that the event loop is busy at all times if there's work to do. And all the blocks here, they represent tasks. And it's making sure that tasks are always working, like something's always done if there's any work to do. nobody uses your API, you don't have any problems anyway. At least not in that technical sense. So the actual problem. What if we have a time.sleep in here? This simulates just some long-running operation. Could be anything from a database to a different API that's being called, file system operations. There are many things that can take a long time. Sometimes it's also just doing some simulation in the background for some computation. What will happen here? This is what we had before. If we call this API as we just saw the code, it looks like this. We have a blocking task in there that means if there's a time.sleep in there, the event loop and the async runtime just doesn't know that there's, like, time to do something else. And that means there's no, like, nothing else can happen. It's single-threaded. It's waiting for this operation to return, and there's nothing else it can do. Like it can't even answer any other response requests. So the whole API is blocked. You get like no performance whatsoever. Because the entire API is unresponsive. So even other users, good luck. They have no idea. So we have like 2.1.0.2 requests per second if we have a 5-minute sleep operation in there. There are two solutions to the whole problem. The first one is use coroutines. So functions that you can await. That tells the event loop basically that you can, like, do something else in the meantime until the operation is finished. And ideally all your blocking functions need to happen fast. Like something trivial or something close to trivial. That just works fine. But again, even if you have a coroutine, somewhere in the coroutine call stack can be another blocking operation, so there's a lot of room to make mistakes quite easily. However, this does not... This is not the only solution. The other solution is just not use async in your functions. So it looks like this. You have just a normal function definition and then the normal blocking operations. And this is part of the fast API magic that it kind of knows what to do there. And what it does internally is that it has a thread pool basically available. It puts it into the thread pool, executes it there, and the thread pool can be awaited. So there's no blockage there. This comes with a, like, small performance penalty. But at least in most APIs, it's probably not, like, too big of a problem. If you're running, like, at the very edge of your performance or your system limits, probably want to either scale vertically or just like invest the time to make async work for your problem although that can be also quite complicated and scaling vertically is probably the easier solution like at least for short-term so learning number two here is async does not mean high performance because it's very easy to do something wrong why is that I think it's mostly because async requires a different mental model for programming although fast api kind of abstracts away a lot of that mental model um but it's still very easy to make it make a mistake especially because all the tutorials and all the like online information usually usually just define these route handlers as async so there's like the the common like kind of commonality between all these tutorials that async is always like kind of a selling point there almost and yep it's easy to make mistakes call blocking functions by accident you don't even have to be aware of it and if that's in a route that's not called very often you might not even notice at the beginning but sometimes a user does something and then the whole api sleeps for i don't know a couple seconds and to conclude this async is just not the silver bullet for performance even though it can help if you're in the, like, extreme last percent of performance. So, the other question is when do we want to use or can use async without, like, any real danger? Only if you're sure that your, like, function is non-blocking. We just said it's not always easy to, like, understand if a function is blocking or non-blocking. And also in all the middleware and dependencies, that's also one, like, part of the fast API magic that you can declare middleware and dependencies as async. And then fast API, again, just kind of knows what to do by looking at the function of the route handler itself. And, of course, all trivial code. So, all the, like, heartbeat information, for example, you can just declare it as async. And there's nothing, like, wrong with that. So, learning number three is that if in doubt, don't use async. Use just regular functions that that will make your life probably easier and gets you better results in a lot of times. Last but not least, I want to talk about a couple of problems that you probably will encounter along the way. And this is a short overview. Like three problems. First, you have, like, global objects and Flask. You need to do something with them when you migrate to fast API. Flask, because it's more geared towards regular web applications and not necessarily REST APIs only, also supports server-side sessions, which we usually don't want, but sometimes that's just what we kind of have to do for checkouts in online stores, for example. And Flask also has a very, very big ecosystem of different plugins for pretty much everything you can imagine. So what are the global objects? So, this is some code that's not untypical in Flask applications, usually somewhere deep inside the call stack. So, this is a, like, core function somewhere to enrich a file metadata. And what we have in there is, like, some request.args.get for something in our request. You can pretty much do that. There's no one keeping you from that. But this is very dangerous, in my opinion, because it is another, like, argument that goes into your function. You don't even know in your route handler, even if you look at it, if that's the only thing in there. And then usually going with that is something like to do this is just a quick fix, fix later. That's usually the part of the code that lives the longest and is never touched again. So this is something that can be quite dangerous there. Okay. So the problem here is that the request context, as it's called in Flask, is basically used in modules somewhere down the call stack and is relatively unrelated to the actual API. So you have basically API-specific code in your core modules, which you don't want to have. I mean, you're not forced to do this in Flask, but this is not uncommon, as I said. And tag, in this case, is a query parameter, which you don't see if you just look at your So there's some undocumented input, basically, that you just find somewhere deep inside your code at the worst case. There's another global state. It's the G object in Flask. This is basically some object that's, like, specific to the application. It usually holds configurations and settings such as connection strings, database sessions, stuff like that. When you migrate, you want to find the use of all these global states. So ideally, you want to have the request and the g context. For request, that's quite simple. You just can search for it, for example, with the grab command. For g, it's more it's not complicated, but, like, if you search for g, like, a lot of stuff comes up. So you have to usually search for import g, for example, because you always have to import that to make it work. But nevertheless, you can do that anywhere in your code. And fast API, the solution for the request is basically to do just good data modeling as we said before. And for the whole G object, the global object in Flask, you can use dependency injection. And dependency injection in fast APIs is very elegant. You just define in your route handler by this annotation from the typing package and the depends object from the fast api package something like a function or a callable at the end of the day that provides something to this function and in this case we are creating a database session and we are giving this to the route handler so we can use it in there and the best thing about that is that you can easily like override this dependency in testing so if you don't want to do any, like, real database stuff in your unit testing, then you can just use something else, for example, an in-memory SQLite or dictionary or whatever kind of operations you support, which usually requires some kind of, like, mocking in the test cases, but you don't really get anything for free there. Another thing are the server-side sessions. Again, this is not something I would advise you to do, like, implement if you can avoid it. Flask sessions provides the whole server-side session functionality in Flask. It supports various back ends there, so it's very flexible and also kind of scalable there, and it acts like a key value of storage inside your application. So this is quite easy to use, and it also automatically sets all the session cookies for you, so you don't have to worry about much there. This is not really supported in fast API, to be honest. So if you want to do that, you need to run this manually, like, implement it manually. Again, I would advise you not to do this, but sometimes you can't work around it. For example, if you are, like, stuck with the authorization library because they have, like, a server-side session-based authorization flow. So that's kind of a downside there. For Flask, we also have, like, plug-ins for everything. You basically install a Flask-specific Python package. It's very easy. Usually you look at the documentation. Often it doesn't provide a whole lot of functionality, but it does, like, do some, like, one thing, one specific thing, and it usually just works. You kind of connect the app and the plug-in, and it just kind of works there. In FastAPI, this is not really the case. You don't have a lot of plugins available besides for like all the observability stacks or the open telemetry stuff, auto instrumentation is at least most of the things are available. But it's not really a problem because a lot of Flask specific plugins also just provide an easy way to deal with connection strings or database sessions, something like that. And you just can use dependency injection as we've seen before to extend the functionality of your API endpoints. So, instead of using, like, specific packages, you just can, like, plug and play normal packages, install them, and use them by injecting this into your routes. So it's very, very easy on this front. So, coming to my last point, the migration strategy, this is kind of like a guideline for everyone who wants to migrate an app. Ideally, you You start with a working Flask app, otherwise you will not have a working FastAPI app if you're migrating. Then you kind of create high-level testing. So not like unit testing with the Flask test client, but more like Postman or curl request that you kind of understand what the current status quo is and then can test against that later on in your FastAPI app. You then create endpoints in your FastAPI app that mirror what your Flask app has. I mean, very obvious, and then from there, you identify the locations where the global state is used in your Flask apps. Depending on how well your code is architected and how, like, disciplined you have been during the Flask development, this is either be very easy or a lot of work. But you just do that, and then basically you shift left. You move it as close to the endpoint function as possibly, and then call the rest of the functions down the call stack by providing the information by, like, arguments. This makes everything much easier. And even if you just stop there and you haven't migrated at all, your code base has significantly benefited from this XFS. If you still want to migrate, you can then identify the input and outputs from the requests and basically create models for that. By doing that, you get free data validation, free in the sense of there's no, like, development overhead anymore. And at the end, you just implement the fast API endpoint, which, if you have, like, separated core modules there, it should be quite simple to, like, do that. So this is it from my content. I'm excited for your questions, and if you want to connect, there's, like, my contact information. Again, the material is on pretalks, and thank you. Thank you, Aurel. Don't forget that you still have slides for your questions. Maybe the first question is, what was the most unexpected use case during migration on your experience? Flask is very flexible in a lot of terms. The whole data validation side has the problem that it's very inflexible. That's kind of the point of the exercise. But when you have non-standard clients calling your API and you're writing an API that basically has to conform to the calls being made, you expect a lot of behavior that doesn't work like that. And in fast API, usually for a file, for example, you name the parameter and it's called file. And then you have to, like, the body that holds your file has to be named file2. That's a problem when your file keys in the body are dynamic, which is not standard, but happens if you work with, like, real life systems. And then you have to, like, do a lot of, like, custom work for that. Thank you. there the situations when you wouldn't advise actually to migrate? Yes, if you have a working app that just does what it does and has no problems in that respect, I wouldn't migrate. It's not a good business case. Sure, it might be fun technically, but there's no reason to do so. Again, in this project, the migration was more collateral damage or helped us to achieve the goals and better testability, better code quality, that kind of stuff, because it enforces a lot of things that are not necessarily enforced by Flask. If you have a disciplined team there and a well-working Flask app, there's no reason to migrate, in my opinion. And maybe the last question. You mentioned that there is different plugins for Flask and also for FastAPI. Is there some plugins that you might recommend? I think for fast API, you just have to look at all the instrumentation, that kind of stuff. These plugins work quite well in my experience. Besides that, there are not too many fast API specific plugins that I have used there because there are just not a lot around. And there is like one server-side sessions package for fast API, but I think it's no longer maintained and because like it's actually quite a lot of work if you want to have a scalable solution there thank you a lot you're welcome thank you all