µDjango, an asynchronous microservices technique.

Speaker 1 [00:06]

Thank you, thank you for all. This talk is only a small part of BGC's area, handled with complexity in huge projects, which are built on different technologies. And today my talk should go about Django and MicroDjango. And before I start, I want to say thank you for my family who support me. I want to say thank you to my wife Elena I want to say thank you to our children Maya, Mark, and I want to say thank you to my animals Kisa and Marcel and also I want to say thank you to my daughter Maya she created all illustrations for all my presentations and thank you Maya Who I am? My name is Maxim Danilov, I work in commercial software development more than 27 years, and it was different languages, last ten years, nine years I worked with Python, with Django, and last six years I worked with different front-end reactive frameworks. And all links to slides from different conferences you can find on Bitbucket. On the end of the presentation you can find a QR code to link to slides. And right now we go to our topic. MicroDjango or Django microframework. What is it? The important idea which lays in these words MicroDjango is a single PI file project. Single. And how it can see in a folder? It can be organized files like gitignore, like dockerfile, like readme, and, for example, you can see test there, but test should not be there. It should be in other folder. And in this case, we have only one mind.pi file. But this file should allow us to have full Django functionality, because we built this project on Django. It should be possible to use different old batteries from Django. It should be runable standalone, runable in monolith, it's important. It means microservices can work standalone, but are able microservices to work in monolith? Interesting question. In my case, it's able also to use in monolith structure, and it should be testable. The story of refining this theory started with Django itself, but the first step they they did Yuliya Elman and Mark Levine in the book Lightweight Jungle. It was the first time this technology was presented. After that, I made this technology in Prototube, which was provided for me for my project Wine Pet. This is a business data lake about alcohol, about wines. And Armin Wolf and Florian Enmosa, they provide me the project also with reduced quantity of files and folders. They simplify standard Django project. The next step I find in repository from Kirill Klenov, he used this technology in benchmarks. Django, if he wants to test Django, he uses lightweight Django technology to compare this framework with other frameworks. The next step, it was already a little bit famous step, because in the Django community It was discussed since 2019, how we can create a single API file project, build it on Django. I can imagine they simply passed the Lightweight Django book, but they created like the same idea which was in Lightweight Django, and a little bit improved this idea because Django not stay in one place, Django has some extent itself, and that's why came some changes. The last step which will be done did Paolo Migliore from Django, he offered the name MicroDjango, I like this name, and he also runs this project like async-await-paradigm. Okay, let's go, how it works. We can create a standard project with command Django admin start project MicroDjango. I don't like this idea from standard Django, because we have here words which go against the dry principle. Don't repeat yourself. We have folder-Django, and in folder-Django we have folder-micro-Django, and in folder-micro-Django we have again folder-micro-Django. It's something wrong. My suggestion for you, we can create with only one command, project, and we can rename the root folder, and in this case we have the project folder, microjungle, and one folder with configs, there I can save all different configs and settings for my project. Okay, but this is still a normal Django project, and if I do the next step, I create the app in this project, we have the new folder in the root folder, and in this folder we have thousands, okay, not thousands, I know, 8 files, 8 .pi files, I don't know why we need it. Okay, file models we need for work with object from database. For example, apps. But why apps? Why not settings? And it's completely forgotten technique about app config. It's powerful thing, but nobody use it. Probably me. I don't know. Next question, why admin is not plural? I cannot answer. And why migration created right now and not before I start to create the schema for database? Many questions. I don't have answers, nobody has answers, as is, use it. And accordingly, what we should do, we should throw away all files from folders which we don't need, and copy it all in root folder. What we have? We have ASCII, WSGI interfaces, we have URLs, we have tests, settings, models, and manage PI like a start endpoint. all in root, delete empty folder, remove unnecessary files, delete migration. After that, next step, we can squash Managed API, ASGI and WSGI interfaces. I simply copy the two strings from ASGI and WSGI in Managed API. Important thing and important idea in microjungle, all what you do in start entry point in this file, all should be lazy. In reality, all what you do in Python should be lazy. But nobody do it. I offer only for you to use lazy technique. What it means? It means if I run, if I call Waze GI, like if I want to create instance, instance will be created not right now, only on the first call. Okay. This is the idea, I squashed the three files in one file, but it's not enough. We have many files there. And right now we can see how it works with models. Models file we don't need. We can declare some there in project model, our model, and this model will be registered in apps registry from Django on import, on the first call of this file. There, our model is declared. In this case, which file calls always on start? URLs.pi. Okay, we can put models in URLs.pi, and the model will be registered in Django pre-existry. In this case, I squash models and urls.pi. It works all examples which you see there is in the repository from this talk. You can do all steps one after the other and check I'm a liar or not. And last frontier. here. URLsPI. We have URLsPI on the previous screen, and right now we should invent something how we can move URLsPI in my API. I rename manage API in my API only because I like this name. I move, I simply copy from URLs in my API my files, and you can see there again my love for lazy objects. Any URL patterns, I again wrap in lazy calculation art, and URL patterns will be calculated, list of all URLs will be calculated only if settings.pi, only after our server starts. In this case we have three files, test, settings, main.pi. How it works? It works. Right now I run the admin panel and you can see on the left side something is wrong. What I forget? In this case, simply I forget to add on the dev stage, I forget to add static files. Of course, below you can see, please don't forget to add static files in URL to work correctly with CSS and so on. It's really interesting. Try to switch off static files for your project, switch off CSS and try to see how it's ugly, how it seems ugly for user if one CSS file not comes. You can test it on Django, on the raw Django, it seems not too pretty. But it doesn't matter, we go further in model Django, in micro Django. Okay, data serialization. For example, I have one idea. We have a database, I want to take elements from the database, serialize it, and send it to the customer. I use Django Serializer framework. Django Serializer framework was previously than appears Django REST framework, and Django Serializer framework works faster. I offer you to try to use it, and in this case, I use this serializer, I convert it in JSON, and I use also generic class-based views to send my JSON to the customer. It works, you can run it, and you can see the framework. In this case, I want to serialize object from table users from standard Django. But somebody can tell me, Max, this is all synchro, and how it goes with asynchro8? I don't see any problem with asynchro8. Django is completely not ready to work with asynchro8 paradigm. Right now this example from Django 5.0.1 or .2, but it's still, Django is still not ready to work with async generic class-based views, and I should override all if I want to work with async-await paradigm. This is example how I can override it. This is async class. I use async-await. I still want to serialize users, also I can set up there which fields should be serialized. I remove all fields, I get only username and email. Here I use also one special technique. In this example, this is the full file MyNPI. I don't have any more in this file. This is full file. And I can run it, and it works. How it works? Because I use here some special technique. I wrap this class, async class, with decorator, which create URL patterns with URL to this view. this view. This is some trick, but this trick, it makes our Django seem a little bit like a Flasky or like a fast API, like a Lightstar. It works, it still works, you can check it, And right now, what about tests, Max? Do you tell me about this technology, but what about tests? Test file is the same test file as before, I use there also, this is example how I can test async views in Django, and I provide you the command line for a command line interface, and it works in asynchronous mode. Thank you for Django developers. It works perfectly with async functions. And you can see I work always with classes. I don't like function-based views. Okay, next step. Some words about settings. You can say to me what settings should be lays in my project. In reality, no, you don't need to have settings in Django project. Why? Because for test stage, you need test settings. For production, you need production settings. For development, you need development settings. In this case, of course, you can collect all settings in one project, but in reality, if you have many developers, your developers should not need to see what happens in production settings. They should have only dev stage or test stage settings. In this case, settings we put usually in other folder. And after that, we can run, start our project with different settings. Django offers us to do it in different manners, for example, different arts. For example, we can put path to settings in environment. We can add a path to settings with attributes like flow. And in this case, in this case, mind.pi, this is a folder with important project. Test.pi lays in test folder. And settings lays down there. I don't need to know about it because I need settings only for development. how I run it. This is common how I can run my Django project. If settings lay sound there completely on the other folder, other hard drive, or on USB stick, it doesn't matter. It runs, and you can use it. Next step. How can I use this technology like a monolith? I have the root folder, and in the root folder I have the folder first service, in this folder I have one file with first service, which can be started like a standalone microservice, but after I have second service in the next folder. And after that, I have the collector, which I create on the left side of this slide. This collector goes through four folders, collects all MyMPI from four folders, and takes all URLs from these services. And in this case, if I run the root MyMPI, I run many multiple services, and it will be run like one monolith, if I want. But sometimes I don't want to do it. And in this case, I can use Docker container, there I can pack only one service, or probably I can pack in Docker container two services from 400 services, or three services, it depends on you, what exactly you want to pack in Docker container, and it can be run at standalone, and also I provide you there techniques how I can run Docker container with different settings, with test settings, with dev settings, production settings, and if you don't know, Docker offers us the new syntaxes to run docker-compose. Please if you don't use the new syntaxes of docker-compose, in this case simply docker-compose up. Previously it was docker-compose up. The second version is more logical for me. And in this case, please take information about how Docker offers you to work with new paradigm. Okay. We have folder settings. We have end there. We have settings there. And we have my API file, only my API file in one folder, and Docker file. And probably somebody from you see the one problem. I never show file requirements.txt there, or I never show there my project.toml file. Right? This is my other idea. I don't use any more requirements file, because all requirements are written in Dockerfile. And for local development, I install, if I don't want to run Docker, I install dependencies from Dockerfile. And it means for me Dockerfile is one source of truth for my requirements. Probably the sole and logical question, Max, why you offer to use it? If we have architecture, like safe and read separated services, for example, we write to database, we find one service and we read from database with other service. In this case, this works perfectly. If, for example, our read microservice is broken, we still can write in the database, and we can independently restart the read server, or other case. Also we have some ideology which names modular monolith architecture. Modular monolith architecture. This is an interesting architecture, I covered it in my other talk, but you can find it in the internet, you can find the articles about it, it was published in 2019, and perfectly clear why modular monolith probably is a solution for your problem with monolith which seems like a mess, probably. Okay, also we have messy monoliths, which we try to deal with them, and for some APIs we can remove APIs in microservices and run it in container or separately, it depends on you. And also, on async-oriented Django projects, if you use it, probably micro-Django offers you the best way to run, to create, to test, and to run your async services. And what I don't cover in my talk right now... Thank you! Migrations. About migrations, especially on this screen on the right side, in this case we have big monoliths which use migrations in different arts, and microservices only use reuse models from core service, and for microservices we don't need to do migrations because it's not responsibility from our microservice. But it's still usable, and, sorry, for this case, we should create folder migration, and we should create migration, different migrations, because I don't want to overwrite the whole part of Django which create migrations for us. Second, how we can follow dry principle in our microservice projects, patterns and principles. How is it possible? Probably it's possible, not in our universe, but I try to give some suggestion how it goes, probably in my next talk, or you can find my talk on the jungle Lettland, on the PyCon Lettland three weeks ago. I also cover this theme. Okay. Repository organising. We have collected, my company collected some ideas how repositories should be organised and how we use templating in our repository to create microservices. Easy organising is not real, But it's possible. Okay. And in reality, we have 412 microservices right now. And how to deal with that without headache? Probably it's possible. If somebody knows the answer, please tell me. And of course, I completely don't cover deployment practices. How we can deal with this bunch of microfolders, how it works, how it works with installation, and so on. But it's easy, fast, furious, and, yeah, probably I lie, but, yeah, it works. It works. I offer you to make microservices, not war, all examples you can get by this QR code. And right now, I am open for any of your questions about these techniques. I'm ready. Thank you. Thank you for the great talk. Maybe you can answer a few questions. One of them is what about other Python web frameworks like FastAPI, Blacksheep, and Starlight? They already don't have so much files as Django has. Exactly. On my previous talk in PyCon Lettland or in Bratislava, I show examples how I use Django views with Starlight, how I use Django with Robin framework, and how I use Django with fast API. I use from fast API routing and the possibility to work with web sockets, and from Django I take Django classes, and in this case I mix Django and fast API. But if you speak about simply one file, in this case, right now I offer you possibility also to use Django with one file. The same quantity of lines. Next question. How do you actually install the requirements from a Docker file? Sorry, one script which takes pip installs lines from docker file and it runs on computer. One script. I use, in this case, I use fabric command line extension. Okay. Last question we can do here is how do you use and configure things like linters without pyproject.toml? Good questions. Right now for every project I have settings for VSCode, but are you right, the question is completely correct, without pyproject.toml settings for lenders should be provided. So thank you for the great talk, I will put the other two questions into the Discord, You can answer in there. Yeah, I can answer also. Or we don't have time. We don't have time. Sorry.