Introduction to Docker for Pythonistas

Speaker 1 [00:03]

Yeah, thank you. I'm happy to be here. It's my first talk at a conference ever, so I hope I don't mess up. I'm really surprised how many people showed up. I'm a bit scared of it, to be honest, but let's see where this goes. So, just a short introduction about myself. So, my name is Jan Wagner. I've been working in the broad area of IT now for something 10 plus years, but still currently a data science student in a master program at the University of Mannheim and just recently started to work for the Axantec Consulting Group in the area of BI and data engineering. So shameless advertisement if you're looking for a new job. They are hiring all across Germany, so check them out. Social stuff, of course, you can find me on Github. At the moment there's only one repository containing the slides and some demo files I will show you later. Also, if you want to connect on Xing, just add me there, or on Twitter. So, what I will talk about... I will briefly introduce what Dogger is and what the difference is between container and classic virtualization. If you have Dogger installed, I will give you a really small hint if you're a Windows user. I will explain the key elements, so the registry or build an image, then what the image is and what containers are. I will show you the Dogger hub. That's the Dogger standard registry. where you usually pull all your images from. I will give you an introduction to the doggr file. So if you want to build your own images, what you have to do. I will give some examples and a short demo, which I really hope it works. And of course then, okay, what doggr and Python, how you can combine it and why it gets along really good. then some last words of warning um not yeah there's no free lunch as as there's the saying so dagger comes with a few downsides um and a short outlook so what you can do after this talk um which brings me to my goal i will give a a more high level uh introduction to what it is But some details as well, and I want to motivate you to just start playing around with it a Really short disclaimer views are all my own. I have no affiliates with dogger whatsoever I really just think it's a nice any piece of software and what I show you is not necessarily best practice Just demos, but it's all to my personal best knowledge and intention to bring you dogger What is doggr? If you ask Wikipedia, doggr is a computer program that performs a rating system level virtualization, also known as containerizing or containerization. Okay, what does that mean? Maybe you've come across a similar picture. So on the right hand side, you see a classic visualization you have somewhere metal server cloud instance whatsoever and you have a guest operating system on it or hypervisor something like that and then you start to visualize other machines and every machine has its own operating system on board you have to plug in you have to update the system you have to maintain it if you have a I don't know a domain infrastructure you have to migrate it there and it's complicated and you have a lot of overload so what Duggar is or what Duggar does content container does is you save actually the the guest operating system Duggar or similar container virtualization programs just sit directly onto your operating system and they get a block or they locate resources from that system for your container that you will run so everything is separate for themselves but it only brings the software that you really need there's no overhead this is This is a graphic from a German printed magazine that I really found good. So the idea of containers is not new. Does someone of you use FreeBSD? Okay, no one. One, excellent. So FreeBSD has a system of jails. A long time ago they introduced that. and jails is basically like a container so maybe dogger got popular because the marketing is better a container sounds better than a jail maybe i don't know and so this is just a really rough overview and you can see there are rkt or lxd as similar systems like dogger yeah get dogger it's really easy go to dogger.com choose your platform of choice download and start the only two differences is there's a dogger ce that's a community edition that's for free that's you can download and the dogger ee the enterprise edition that comes in with a different update life cycle with commercial support and so on and so forth but usually for you see is more than enough so if you choose to use dogger on Windows which you can here's this is a hint that I had to find out myself at some point if you use it on Ubuntu like I do now the dog a demon just takes what it needs from your system hardware wise if you use it on Windows you have to tell him which what the demon or the service allowed to take so you can see it on a screenshot I have to say okay you can have two CPUs you can have one gigabyte of RAM and so on the downside of that system is as soon as you start a service it really will allocate the memory from a system even if you have nothing no container run at all so that's just to know okay so now it's installed what now um on your machine is now a fine little service a docker service or docker daemon and that service is usually per default accessible within from within your system so you can um yeah talk to the to assist the daemon with your command line or with other apps from your machine. It's not accessible from the outside on default, but you can activate that if you want to for some reasons. Okay, the key elements. Basically it's all about the image. In the middle you have an image that is for some piece of software that you want to use, or that you just build. So where does the image come from? Two options, from the Dogger registry, which I will explain a little bit more soon. For example, the Dogger hub, or you can build it on your own with a Dogger file. I will also show that. And then when you have an image, you can run, you can spin up containers. You can spin a one to so much what your hardware can handle. One note here, the image is, yeah, let's say static. Static in the sense of it's there. And in this exact state, you will start a container. So now you can operate on the container. You can do whatever you want to do with it. and the data is in the container you can stop a container and a date and start it again and the data is still there usually but of course if you delete a container your data is gone so there are ways to have a persistent data storage but that's another topic another another topic but another another option so the docker registry and the docker registry you can maybe compare it to git so it's just a system where you can pull and push images from and to where you can store it where you can versionize it tag them with different version numbers and so on so the standard registry is called docker hub which comes from the docker ink themselves and this one is the standard configuration for your dogger service where it will search and pull images from and they have something 100 plus images from all kind of companies so official images but also from users like you or me we can pull and push images up there for everyone to use if you want to and some companies like to have their own registries so if you want to use something from the elastic stack um yeah they have they have their own registry which you can just add to your to your service to your docker service um yeah just a small uh yeah small look at the dogger hub that's um from the top i don't know 100 images or so i think you can see a lot of familiar logos so um like redis ubuntu mongodb python of course wordpress jenkins uh dogger registry so yeah you can build a dogger registry with a dogger image of course you can also run dogger inside the dogger container but that's weird um so what to do you can just use the search on the dogger hub website or you can do a search from your command line with the command dogger search and then here for example pi pi Okay, so the doggar file. You wrote a piece of software yourself, or you don't find the image in a way that you want it to use, you can start writing your doggar file. The doggar file is really just a plain text file. You usually put it inside your project folder, on the root of your project folder that you want to use, where your data is it is called dogger file with no extensions and this is a standard name that you should use and it contains yeah building instructions so some keywords are from copy run expose and a lot more i will show some of them in in a demo but i will not necessarily explain everything to you dogger comes with a really good documentation you can spend days reading the documentation and it is really good and it's really useful so I highly highly encourage you should read it it's really good so then containers of course if you have an image on your system with a specific version tag you simply can start your container using your command line if you want to or you can use a graphical user interface there at the moment a lot a lot of like kinetic which came which comes directly from dogger it's not so good but it does the job somehow. A portainer, which comes in a doggr file, in a doggr image, which starts some kind of web GUI. It's really comfortable. And to run, you just have to put in, yeah, doggr run minus dash dash name, then just some name for your container, and then the image that you would like to start as a container. Then if the container is up and it doesn't fail you can do a container name of your container start stop restart I will show you a little bit Okay, so for those of you who don't know, this is Visual Studio Code, just an IDE to program in many different languages they also have a dogger plug-in then you get this final way here on the left I can also see your images and containers here and you can use it as good to start them up or stop them so I just put some some commands here and they all start with dogger so if you just put in dogger you get a lot of information which commands you should use or should not use you can go with dogger version maybe I should better all right so dogger version shows you which version you have what you can do and so on and so forth good that's rude the info shows you a lot more information so now which operating system I have on my laptop what kind of hardware it finds and can use and so on just in my cheat sheet here so let's say you want mysql on your system because you needed to test your app or something you just go dogger pull mysql I just know that mysql is actually the image name of the software because I looked it up on the doggahub before and you can just say okay, pull it if you don't do, if you don't give it a label for a specific version, it will always assume to pull the latest one pull the latest one I will stop that because it will take too long with the internet connection at the moment um nevertheless there are more commands there's docker images so with an s at the end which will simply show you what images you already have so i'm doing my preparation i already downloaded the python version 3 image and the latest mongodb image rabbit mq and alpine 3.8 for those of you who don't know Alpine is a Linux distribution a really really really small one so you can see it on the right side it only has 4 megabytes so it's really small and it's often used as a so called base image which I will show you in a second but that's a little tricky you can do also docker image so no s and list that's the same so you can either use images or image list what you also can do is dig dogger image inspect to see what the image you you you downloaded has as brought to you to your system so let's take the Alpine one and there's just really a lot of information, what it uses, how it is configured, how it will start. As I said, it's really good documented in the docker documentation. And maybe a more interesting command is the docker build command, which usually your command line should be in the folder where the dogger file is and if it's called dogger file which it will look for then you can just go with dogger build minus t for tag to give it a tag something like this and you have to put a dot at the end this dot is important this dot says basically um sorry basically the docker file and everything that you need to build is right here where the command line is at the moment so that dot is important if you don't use it you you will get an error message, so just use it. I prepared a little bit. So this is a really small docker file. As already said, the first line is usually from, you need a base image like Alpine as Linux distribution or something else. And there are the possibilities to start directly from scratch, but usually you don't do that because you just use Ubuntu or Alpine or something else as base image. You can give it labels, like the maintainer, the version, which will show up on the docker inspect command, and then you can say what it should do after after you build it um there are difference maybe some of you have seen the talk right before mine he already showed it there's also the entry point um this command is executed when the images build and then there's like i have the command command cmd this command is always executed when a new container is started from your image okay so yes so let's just talk about build I name it first schema version 1 and file is right here where I am at the moment so okay so you can see it counts the steps so from Alpine I already pulled it on my laptop tomorrow so that's already there it reuses images that are already there so it will not download anything again. It put in my labels and the command what should it do when I start it as a container. So I can now go doggor images and here's the image I just created and now I want doggor run container name it first container and I want to use the first demo version one image so just again it's dogger run for your container dash dash name to give the container a name and then you say which image you want to use and in which version it run it says hey it's gone that was a container just to show you this yeah sure sorry is that bit better all right so do a container list and it shows nothing why well my container started it made his command so say and it yeah it stopped again so this is why the container is not listed here but you can say dash a for all and it will show you every container and not just the ones that are running so for time reasons I will not execute this one but I will tell you about it so rabbit MQ is a message message broker maybe some of you have heard it or not um so basically is a hub where you can create channels and subscribe and push messages to that channel to just go through there um yeah and you can of course use it with dogger this command would run a container called pycon d more rabbit mq and then we have a dash p for port which maps the local part of your machine to the part of your container. This is most of the time important if you really want to access your container. If you don't specify it, then there's just no way to communicate with your container. Then MongoDB. The image name is just Mongo. Mongo. I don't know why they have to choose this instead of MongoDB, but it's, again, that's the fourth one. So, it's, again, the same command. So, it's doggorun dash dash name pycon mongodb and just call it from the image model in that case the image was already built to expose their standard port and for some reason you don't have to to define it additionally so now you see this is a dog Mongo container that runs and it shows you everything from the log file here directly in your terminal but this also means you can't work with the terminal anymore and if you go with ctrl C it will shut down the container that's maybe stupid so what you can do is just put in a minus D for detach you should do it on the right so what's wrong I just run the container and now I want to create a container with the same name again and that does not work as you see so let's call it one so now the container runs somewhere in the background and you can still use your terminal like you can go with Python call it Mongo PI of course I use Python 3 and you can just see it can connect that's all what the file does I also prepared a example with flask but I already seen the red cards so I'm running out of time it is on my github you can check it out I will just finish with my presentation. Okay, so Docker and Python, just another additional info. You can pip install Docker. That means you're installing the SDK to control the daemon with Python. So the Docker daemon with Python. If you use images in container, this can be a good alternative instead of using virtual environments, which usually mess up your system pretty hard. And you can find a lot of images on the Dogger Hub for PyPy, Jupyter, Django, and so on. Just two downsides. The first is the same like apt-pip, npm, and so on. Think before you pull and install. Just watch what the image is about that you're going to pull. Read the documents about it, and don't blindly use anything that you find. The second one is, depending on your use cases, especially in the enterprise area, Docker can put an additional layer of complexity to your project. So if you don't need it, don't necessarily use it. It is awesome to ship your application, but the application should still always be shippable and installable without Docker, best case. So where to go from here? Docker documentations. There are a lot of free video courses available. Same here if you want to go further. Of course, Docker Swarm, Docker Kubernetes, doggocompose, everything really good to follow and to add complexity. So, sorry that I had to rush a little bit, but still I hope you have a basic idea of what doggr is, what containers are, how and where to search for images, to search for help, and I hope you want to start playing around with it and so on, happy sailing. Thank you. so maybe time for one or two questions i know you didn't get into it too much but um when you're building your own uh containers have you managed to find a way to clone um a non-public repository That is something I think that you have to authenticate. Yeah, I haven't had that use case yet. So, sorry, no idea. well basically I switched to Ubuntu yeah sorry it does run on Windows it's getting better there from update to update but it should be stable enough for you to develop on your system on a Windows system but if you would use it on a server production environment you would use some Linux distribution as your system So yeah, sorry. from Docker together with standard virtual machines, but I decided to execute on the virtual works and effectively use Docker under Linux. Don't use, don't use, don't use standard over Windows. Yeah, I can confirm that if you use VirtualBox and Dogga on a Windows both, you have to switch on or off the Hyper-V extension somewhere in the system settings, but Dogga will give you a nice little error message with a cute wail being like, sorry, you have to deactivate it. You know what's happening at least.