Unlock the Power of Dev Containers: Build a Consistent Python Development Environment in Seconds!

Speaker 1 [00:05]

Thank you very much for this warm welcome and also welcome from my side to the audience here and to all the listeners on the internet. So today we'll speak about dev containers and I want that you get two points for my talk today. So first of all I want to show you the basic concepts of dev containers. It's the sense that you know what's behind this tool and how it does it work. And the second point is that, to my personal point of view, I think that dev containers are at the moment one of the best methods to get a consistent Python development environment. And why does a consistent Python development environment really matter? So I think consistency is the key for reproduction. And I've just made a picture of a car manufacturing, and there's an assembly line where the cars are rolling out, and what do you expect from cars? You expect from cars that they work at the end of the assembly line, that they work as expected. When you go in, you open the door, you start the motor, and you can drive. Now, think yourself, how many times did you fail to reproduce a Python development environment? Because we are quite often reproducing Python development environments, so we are reproducing them. For example, when we come back to our project, we skipped it for three months, we come back so we need to reproduce our environment as we got three months earlier. We have to reproduce it when we want to deploy our code for production, that it's running on the servers in the same way as it's running on a local laptop, or, from my point of view, The most important part is when we want to share code with our colleagues or the community. And I've just been in a talk also today of Artem and he also just pointed out that reproduction is really critical for Python code. So just think about reproduction that it really matters to the Python community I think at some point. And let's see what's the problem and what's the road to reproduction and therefore I've just chosen an example. So, for example, fast API web server. You want to write down a fast API web server. Who knows fast API? First survey. Ah, great. Those who don't know fast API just think of a simple flask server and I think then we get 99% coverage here inside this room. So, when you want to make a fast API web server, you go to a fast API website and go to the docs and copy the example code. It's exactly the example code. And you read further the documentation and see, okay, when I want to start a web server, I go to the console, I do a pip install. You have to install fast API in two packages. And the doc say that afterwards I should start it with the command uvconn main app? That's it. Everything's working. Well, maybe. If it's your very first Python project, yes, for sure, this will work. If it's your second or third one, you most probably get to a point where when you install the packages, you get feedback, oh, I can't install the package. It's just a discrepancy of dependency of another package. And then you read, okay, then I have to do a virtual length. Okay, you start doing virtual length, you're doing the next 10, 12 projects with virtual length, but you come to a point where a package says, no, I don't want to run, because you got the wrong Python version. I need Python 3.11, not 3.8. Okay, and you install the next Python interpreter, and finally it's the year 2024 and you're a data scientist or machine learning expert, and you want to get access to hardware acceleration. And you install some TensorFlow system libraries and so on, and over the months of working your system descends into chaos, as Randall Monroe drawn here on the screen. And I love this picture because he really got the points where Python and one Python development environment descend into chaos and what we need to care about if we want to have a consistent Python development environment. So let's analyze this picture. So doesn't first level, which we have to think about when we want consistent Python development environments, the Python packages. We have to fix the Python versions, the Python package version we are installing. So when we go back to our example, we installed fast API, but if you really want to reproduce the result, we have to care about the version. For example, fast API 0.110.1, they are really at 110, I wonder why, when they reach number 1. But if choosing the colour green, because in the Python ecosystem, we get pretty good solutions for this problem. To fix a package version, and even the building wheel. I just think about poetry, which I love to use, but there are plenty of other good solutions out there. So, okay. Second level is the interpreter level. This is a bit trickier. We don't have so much option to switch our Python interpreter in our Python environment. I'm just thinking about conda or pyenv. We get solutions, but we don't have as many in our ecosystem. So, And then the server level comes, the system level. That's what I mentioned earlier when you become a machine learning expert, a data scientist and you want to use hardware acceleration. And at that point, you have to install some C or C++ libraries, for example, on the system level. And there, to my knowledge, we don't have really good solutions on our PyFu ecosystem. So what may be a solution? There's one golden rule in informatics. Just look around what problems others had, perhaps they were the same problems anywhere, and copy the solution to it. And five years ago, when I started working at a company, I got a lot of colleagues from the Java and Kotlin space, deploying databases, front and back-end services, and I got in touch with containers. And I learned to love containers. So, now comes a very critical point in this talk. And I make a survey. Who knows about containers? Please raise up your hands. Ah, that's pretty much. Who doesn't know? Counter? Test? Two persons? Three persons? Okay, I don't want to lose you, so let's recapitulate. Containers are beautiful, because they solve some problems, yeah, I want to convince you, so I grew up in the 90s using computers, and when I had to install a web server, everything was fine, but when you installed a database next to the web server, you had to keep sure that the system libraries were compatible, and all those administrators out there had the same problems, and they want to get rid of those problems, so they just thought about somehow isolation techniques. So you have a common operating system, for example Linux, you get a kernel, and you put just other operating systems on top, sharing the same kernel, and one quite known solution for this nowadays is Docker. So you install Docker on your laptop and you get access to this container technology. And in fact, you can use containers such that your Python environment lives in splendid isolation. And we'll just look how this works by going back to our web server example. So how do I deploy a web server, for example, fast API web server using Docker in order to derive a method how I can get into development with this container. So let's come back. When you want to dockerize your fast API web server, you go on to the console back again and you do a pip freeze and save the output into a requirements.txt file. This is the simplest method to, well, to somehow fix the package version. You can argue if it's the best one, I wouldn't say it because I'm using poetry, I would say they use Positree, someone else would use something else, so there are a lot of solutions there for fixing it, but it's the shortest one to get on a single slide, such that everybody can read it. And afterwards, we are now going into the Docker world, so we set up a Docker file, and for those of you who don't use Docker, I'm a mathematician, I get some definitions of how this thing works. So, a container is created by an image. And an image is defined by a docker file. And you build from the docker file the image, and then start the container back. So, this you have to keep care about. So, and let's see how we containerise it. So in our Docker file, the very first line, we take an image for, let's say, a repository from Docker Hub, and we take the Python 3.11 bullseye image. And in doing so, we fixed already two things. The system and the interpreter. We've chosen a DBM bullseye system, which is more or less fixed now, and we've chosen the Python interpreter 3.11, great. Then we have to take care about our project, so we copy all our files from our working directory to our image later when it's built, and we run in build time a pip install to fix the packages in our container. And then we just say, okay, when you'll ever be started as a container, do the command uvcon main app in order to start the web server. And now you go back to your console, you make docker build, you say, okay, I build now this image and name it my image, and the context is the dot, so keep care about everything which is lying in my working directory. And afterwards, so what will you see when you really type it in your console? It will download the Docker image, and it will copy all files inside, it will do the pip installs, you see how the package is getting installed, and that's it. It's got saved on your local machine, and you do a Docker run my image, and you see now that the command is executed and the web server is running isolated on your machine. So, we containerised our web server, but that's not the thing we wanted to do. We wanted to do development, really, that's our goal. You could actually go inside the container using your favourite IDE WIM, I hope it is or not, and start programming, and you would have everything inside the container you need to program using WIM, but that's not the way how you like it. You would rather think about I want somehow a sync between my directory in the host system and some directory, let's call it workspace, inside my container. You can do this with stocker. So we go back to our console, we even leave this docker file, because we now want something more interactive, so we say docker run, and now comes to open an interactive session with my container, so open a bash for me, please, and we say, okay, as a basis, we use again And in addition to this, we want our sink. So please mount my local directory, here the dot, of my project into this workspace folder. And make the sink. So on the next slide, on the next command line, after doing so, you get a bash inside your container, and now you install yourself the packages, and start the server, but now congratulations, they are not in sync, now they are in sync. Now congratulations, you get your first dev container, or at least containerised development environment. Because you can now work from inside or even from outside, you get access to a fixed Python environment. And there's not much missing to transfer this solution into a dev container specified solution. And that's one key point I want that you take away from this talk, that when we talk about dev container, it's basically 90% Docker, or something another container solution in future perhaps, and only a 10% service layer which gives us some benefits in terms of convenience in using it. So let's see how we transfer this solution in a dev container context. If you want to use now dev containers, you create a dev container JSON inside your project. And you need now three lines in your JSON. First of all, you give the baby a name. So it's, for example, the PyCon dev container. you say to him okay I use this Python 3.11 bullseye image as before in the docker run and as we did afterwards after upstarting the container as we did the installation of the requirements file so we do pip install minus r requirements that's the dev container chasing and you don't see the thing with the mounting inside the specification because the dev container standard will do it for us. So when your IDE supports dev containers, it's keeping care about the thing. And that's the service layer I mentioned, 10%, which are on top. And who thinks that dev containers are really limited to VS code or something like this? Hands up. Okay, a few. Okay. So I have to admit, the dev container specification came up in the context of GitHub and Microsoft, but it's growing. So more and more IDEs are supporting it, or cloud providers are offering that you can spin off a development environment using dev containers. And I know my colleagues, for example, and you're using a lot of PyCharm. Nowadays, PyCharm is supporting dev containers. You can use Anwim, for example. It supports also dev containers, so it's getting quite common. And the dev container specification is indeed open source, and got 2783 GitHub stars and perhaps get even more stars after this talk. So, and the beautiful thing is, when you get such a dev container JSON inside your repository, you are really just one click away from a working Python development environment. So when I clone a repository with a dev container JSON inside, my, for example, VSCode IDE says, oh, I found a dev container JSON, shall I open the container for you? And I say, yes, do it, please, because I don't want to read the documentation, how to get a Python development environment. And that's the second thing which is really important to know, I think, about dev containers. So when you start using dev containers for your projects in the community or at your company, it's some sort of living documentation. Because you know when at least one developer is working with a dev container, Jason, you have one place you can go to to read how the project is started even if you're not using dev containers at all. And that's really one big benefit compared to documentation which can have just discrepancy. So please, when you use dev containers, put it also to your repository, make a git add. It's not just for development, it's also for documentation in my point of view. So and we can now even do more things with this dev container JSON. So for example, we got a fast API web server, so web servers have always API keys. We can for example say, okay, the API key for our web server is stored in a .end file And we pass it to the container, defining that there are additional arguments for the container at startup, which are shown here, so slash slash and file, and oh, there's missing a dot. Or is it? Yeah, no, there is one. I just don't see it from here. And that's it. So you get now basically access to all features of docker, which is below, it would be the same arguments you would put behind the docker run command, in a convenient manner in the sense that you don't have to take care about mounting your working directory inside. For sure, we have to mention one thing. When you use dev containers alone, it doesn't solve your problems. It just solves your problems when you think about fixing your packages and your interpreters as you did before when you did it in a reproducible way. But in addition to this, with dev containers, you can keep care about what's installed on a system level and reproduce it. I think that's the reason why you should use dev containers. It's really great for reproduction and documentation, as we said. And I've just joined some talks on Monday about security. I think dev containers are really also an option when you think about about untrusted packages. So let me describe the problem for you. So when you install a package from PyPy, I've learned on Monday that it's not so well protected in the sense of are there any vulnerabilities or backdoors inside or is it really secure to run the package? And I know, I read some news that PyPy is really problems assuring it, so they stopped new projects, for example, which are allowed to PyPy for some days, to care about security issues. And you should really think about using dev containers when you have sensitive information on your laptop, because Docker or any other container technique gives you another level of security on your system. For example, Docker is using cgroups and namespaces and kernel level and you get a certain possibilities to isolate the code running inside a container from your host system. So when you install really a package with vulnerabilities, which gets exploited, at least the attacker doesn't have access to your local file system directly. You may have access to this mounted directory somehow, but it doesn't have access to your complete file system where also your SSH keys, your private keys are stored. So I really love this extra level of security. And speaking about security, perhaps also a reminder, when you're using, well, anything you don't trust, perhaps you shouldn't use your FFH key to download it from GitHub, GitLab, et cetera. Perhaps you should use tokens, but that's a completely different story. So and the thing that containers are really great in is remote computing. And that's why I use them most of the time. And they are good in remote computing because that's the origin, I think, of dev containers so far as I know. So when you go on GitHub, you see perhaps the button, please make me a code space. And this code space is in fact a dev container living in the GitHub cloud. So I think dev containers are really an option for the Python community when they want to get more members from the community to contribute to code, because, well, okay, there's some issue with GitHub if really a lot of community projects should be on GitHub, but there are other options, and the thing is, when you get a dev container chasing inside your repository, when there's a new one coming in, you can just spawn development environment inside the cloud and start programming, and he doesn't have an overhead in the sense of setting up an environment, development environment, setting up unit tests himself, when he just wants to do a little contribution, a small pull request, I think this is really something which is attractive to get more community members working on a code. And yeah, and remote computing, not really in the cloud, but on a bare metal server with dedicated GPUs was my point where I got in touch with dev containers. And to the end of last year, there was not so much documentation on how to do it. And that's why I added this to the abstract of the talk. So we'll now pass to the remote computing features of dev containers, especially if you want to use GPUs. I don't know for how many of you is this interesting, just to get feedback. Ah, okay, so using GPUs. One question I got is, do you use these GPUs on a bare metal server or on a virtual environment? So do you go to Paperspace or somebody else and run up a virtual machine with GPUs who does this? Okay, and who doesn't have a bare metal server? Okay, we are not so GPU pure as we mean. So let's dive into this one. So a lot of you raised your hand having a bare metal server. and when you get a bare metal server which is not your laptop because an A100 has not so much place inside this laptop it's somewhere else and you have to connect your laptop to this somewhere else and when it's a bare metal server it's an SSH connection most probably if you're using, well as I told GitHub Codespace for example you just open it in a browser but we are now really looking on this SSH because that's the thing I'm using and what I think you are using too. So, the thing is, when you want to start using dev container with your GPU remote machine, you first have to think about some abstraction layer between you and the dev container. And as I say, GPU, the most probably GPU thing you got hardware is an NVIDIA card. So, you have to take sure, or be sure that on your remote machine, you get two things installed. So, first of all, Docker, because it's the basis of all, and the Docker container toolkit, I don't know, they're changing their names. So when you get these two things installed on the remote system, it's just like coding on your own machine. You can just forget about the rest. And that's really very cool. And that's the idea of this isolation. So dev containers have really one pro. So you don't need to care about the operating system, which gives you access to this kernel. You just put your Docker container on the machine for development. So, and now let's see how we can get access to the GPUs using dev containers. Now, we'll write again a docker file, because it's only two months ago, I think, that Microsoft has really post templates for using GPUs on GitHub, so as I wrote this, I had to do it on my own. And as I did it on my own, I wrote this docker file, and I know from my time before dev containers that I should use an NVIDIA CUDA image, which NVIDIA prepared for us in order to get access to this CUDA using the container toolkit. And that's really a point you have to be, well, careful, because I mentioned the container toolkit. And you have to be sure that container tooltip is compatible with your CUDA version. So the first point. And there are other points which I'll mention which makes it a bit tricky why there are not generic solutions for this. But you get somehow a recipe how you can solve it yourself. And if you can't solve it, just message me. So NVIDIA did some work for us, so they took an Ubuntu system, Ubuntu 22.04, and installed CUDA on it. So what's missing now is, when we think back, so we fixed now the system, it's always the red marks I used before. So we now have to fix the interpreter. And there I'm just don't care about. I just used the interpreter which is given by Ubuntu. And there will be a talk, I will refer to it later or perhaps now, ask Jürgen Mach tomorrow why Ubuntu is providing the version it provides. He will answer it. I asked him. I met him before. So we are now just happy to use the interpreter Ubuntu provides us in the version 22.04. And we now come back to the dev container JSON. So before we have seen that we have used an image. But now as we did some individual stuff, we just use the build command in the dev container specification. So we can even use our Docker file to build a dev container. And here, we specified docker file in our project level and give them also the context which we've seen earlier. This was also the dot after the command. And afterwards, again, the post create command, pip install, and now we come back to the run arcs which we've seen before passing the environment variables. Now we're using this run arcs to pass the information that when the docker container runs up, it should have access to all GPUs on my bare metal server because the other colleagues shouldn't be working with it. I want it all. That's something really you have to take care about when you get a bare metal server and you get colleagues, you have to just speak with them who is using these GPUs. So it's tricky, but giving a coffee or two, you can convince your colleague doing the computations tomorrow, I think. So in addition to this, such that you don't get warnings from PyTorch, for example, you have to augment the SHM size of your container to, well, perhaps one gigabyte, perhaps two gigabytes, you have just to try. So when there are warnings, increment the size. Yeah, sorry, that's how I'm doing it. So everything's fine, almost, because I've skipped something, and something really important. So when I first did it last year, I just thought, oh, my God, I just tested PyTorch. Does it have access to my GPUs? And it said, no, I don't have access. I just asked why. And when you install PyTorch packages, you have to care about one thing. You have to care about that you're using a certain building wheel, which is somehow compatible with your CUDA library. So here it's the CUDA 11.8 library, which is of course compatible with the CUDA 11.7 on top here installed. Don't laugh. I've fixed CUDA for more than three years. And at least when there are version switches, for example, when they switch from 11.6 to 11.7, you have to just keep care about which libraries are compatible. You have to test it. That's sometimes undocumented. But they're really going to be better, I have to say. Yes. So. What's REST? Have you ever used dev containers before in your everyday life as programmer? Who did it, hands up? Okay. Who will give it a try now? Very good. Because I think I wouldn't use dev container if there weren't those colleagues using containers in a completely different domain. And I think that's why this talk is important. I just want to transfer this domain knowledge to the Python community. There exists something which can help us solving problems. And I hope that you are really interested in trying. So I have just put on GitHub a repository. You can start working with it. I also post a lot of information, no, okay, not a lot of, some information we can start further and feel free to connect or ask questions now or even later and I don't have anything to say anymore oh I forgot something yeah because I love to be at Pico and I think we should now applause to all the volunteers and professionals which are doing a great job over the last day and which will do a great job I'm sure for the next day that this conference is such a lovely place that you always want to come back.

Speaker 2 [35:22]

Thank you very much, Thomas, and it was a very nice presentation.

Speaker 1 [35:27]

Oh, thank you.

Speaker 2 [35:28]

mathematician who went through a lot of different passes now in the cyber security so it was quite interesting and we have a lot of questions

Speaker 1 [35:40]

Very good, we've got a lot of time for questions.

Speaker 2 [35:44]

Is how do you set up debugging with dev containers? Okay

Speaker 1 [35:47]

Okay. So, setting up debugging with dev containers, I think, depends on your IDE. So, I got very good experience with VS Code, which is, as you could expect, because they got the idea of using dev containers. So if you use VSCode, you can straightforward debug. You even don't know about the difference. It's just a small bar on the down left side which says, oh, you're now in a dev container living. So there you can just go straightforward. I don't have as much experience in other IDEs, but at least when you're using Anvim, they're using the VS Code extensions very much so you should get through there and I'm for sure that PyCharm has this debugging feature out of the box now as they're stable. They were in beta phase till last month I think and I got just feedback from my colleagues they got problems but they should have been fixed in the last month because they are now not anymore complaining about dev containers.

Speaker 2 [37:00]

Thanks. We have a related question to the IDEs. Is there a need on how to best ensure cross-IDE compatibilities between, let's say, VS and also JetBrains and so on? Okay.

Speaker 1 [37:15]

The support of dev containers is getting better and better, but for sure if you want the most convenient experience of DevCon A, at the moment VS Code is offering the best experience, but the others will also do it, because why I'm so convinced about dev containers, you've We've seen really to the mid of this talk that I've only used docker to reproduce something which is very near to a dev container. And it's really the dev container is 90% docker and 10% a service layer which gives convenience. And I think this is somehow something which makes a tool great when it doesn't reinvent the wheel, but put something on top of an existing solution which is running and stable and used in everyday life.

Speaker 2 [38:14]

Thanks. Every developer typically has a slightly different dev container JSON file. So how do you handle this when adding this to the repository? Okay.

Speaker 1 [38:28]

Everyone has its own, well, customizations, that's how we would call it. So I would really push, or that's how I do it, a minimum dev container JSON to the repository, which gives you access somehow to that development environment is reproducible. And the thing is, when you get customization, I just store them along in another file, which is perhaps my customisation, and you can add this, for example, to gitignore, just to be sure that it won't be added on the git push and you can just switch it, so yes, it's a problem that you cannot expect any other developer using your dev container JSON, especially when there are customisations inside about what extension you should install, but if I'm new to a project, I would love to have at least one dev container chasing which is working and I would just customize it myself and store it alongside where it's not getting pushed.

Speaker 2 [39:43]

So since we have some time to ask more questions Will you recommend using dev containers together with the docker compose? Yes. Yes

Speaker 1 [39:54]

Yes, and that's really great. This question is great because Docker Compose works out of the box with dev containers. So we've seen a really simple setup with fast API web server. But you can write a Docker Compose JSON, a dev container JSON on top of your Docker Compose solution where you also start up a database and a front end such that a new developer coming into the project sees also how the changes are affected on the front and he's doing in the back end or that he gets a database for integration tests for example. So Docker Compose and DevContainer is perfect compatible and there is existing really good documentation for this.

Speaker 2 [40:42]

Thanks and how to run tests on the dev containers when they also require for example some databases?

Speaker 1 [40:49]

Yes, okay, I think this one is really near connected to the setup for Docker Compose. So when you have unit tests which need a database, you can start up a database next to your dev, or next to your dev container, inside your dev container. It will be two containers interconnected. But that's really a possibility how to start up a database next to your development container, and you get access to it, and you can run your units. This is a perfect setting. And I just want to say one more thing, but I don't remember it. What was the question? Was there another, a second part, no? I think it was something. It was.

Speaker 2 [41:29]

Possibly three guys.

Speaker 1 [41:32]

No, parts I remind later.

Speaker 2 [41:36]

There is one more question, which I assume that you mentioned this in the talk. It says that I'm using dev containers, but others in the group not. So how to collaborate? So what's your best, let's say?

Speaker 1 [41:54]

Ignore them. No, I mean, I don't want others to use the tools I think which are working fine for me because that's not the way how it should work. So I think the way it should work is you're using dev containers, so the other colleagues should respect that you're pushing it for some colleagues which might join the project and also convince the dev container are useful to them. But I'm not a fan of telling others only use dev containers because the dev container JSON, as I said before, has its right to exist next to the usage as development container itself but also documentation to somehow document, documentate how, what is a working setup for this project.

Speaker 2 [42:46]

Related to this question, how about using stuff like Poetry Hatch and so on in the containers?

Speaker 1 [42:54]

On this DevContainer 101 repository, I refer to two other projects which are using Poetry to fix the packages. So when you decided to use Poetry, you will add, of course, a Poetry toml, a PyProject toml, and a Poetry log file to your DevContainer, and afterwards, it's Poetry, and you post create command. And perhaps when we come to this post create command, we have to mention one thing. Who knows the project Ludwig, Uber Ludwig, and machine learning, okay? Who knows Ray, Ray GPU, okay, more. When you think that it's a good idea to use the ray image to get GPU support. So when you're not want to use this Nvidia CUDA file, I showed you before, but you think that you can take the ray thing to get a basis. But there's one thing you really have to keep care. It's about the mounting of the directory. So there's some magic going on behind the scenes in order to get a convenient matching of the workspace folder to your my project folder. So you define, for example, that you're on the remote system, the user, my user, and on your local system, you are somebody else. Joey, John Doe, or Jane Doe, to be. And the magic is when the dev container is spun off, they are doing a matching of permissions and UIDs, who owns the file, et cetera, et cetera. And the ray image got one thing. They installed before Python packages on the user folder. And all these Python packages, when you know Python packages with GPU support can have six gigs. And they're just doing a schmuck. And this takes half an hour to spin off. And that's why when you're really installing the GPU packages with CUDA inside, do it as post create command, otherwise you will wait a half an hour for your dev container to spin off. This is one thing you should keep care of. when you install packages in a dev container, I recommend you to use the post create command.

Speaker 2 [45:40]

Thank you, Thomas. You're out of time now, but there are a lot of more questions. Thomas probably will be around here, so maybe you can approach.

Speaker 1 [45:48]

Drink a coffee with me.

Speaker 2 [45:48]

Drink a coffee with me. Thank you for being here and enjoy your coffee break.

Speaker 1 [45:49]

Thank you.