Size matters: Inspecting Docker images for Efficiency and Security

  1. Introduction

    • We start with an example Dockerfile for a Python-based image.
    • We will explore the role of OverlayFS, Docker’s file system for combining layers, to understand how layers stack and how data (or even secrets) can be retrieved from individual layers.
  2. Layer Analysis

    • To gain better understanding of layering, we use simple command-line tools like docker history and docker inspect to examine image layers.
    • We introduce dive, a tool for exploring the contents of each layer.
    • We apply these insights to optimize the image by implementing multi-stage builds to create a smaller image with fewer layers, improving storage efficiency, build speed, and security.
    • We discuss the benefits of Docker’s caching mechanism in reducing build times.
  3. Security Enhancements

    • Given our example image, we will use trivy, a comprehensive security scanner, to scan the example image for vulnerabilities and demonstrate how to address common issues.
    • Finally, we introduce hadolint, an open-source linter for Dockerfiles.

To get the most out of this session, participants are encouraged to clone the session's repository.

This session took place in track MLOps & DevOps and was classified suitable for intermediate domain / novice python by the speaker.

Transcript (auto)

Auto-generated from the recording utilizing Open-Source AI. Speaker labels (Speaker 1, Speaker 2) reflect diarization, not identity. Timestamps refer to the recording.

Speaker 1 [00:08]

Good afternoon, everyone. Thank you for joining in these huge numbers. It's great to see that you're interested all in Docker images. So today we'll be talking about Docker images for efficiency and security and with a special focus on Python. Right. So a short introduction about myself. Irena Grgich is my name. I'm a lead DevOps engineer at Carl Zeiss. If you don't know Carl Zeiss, optics company, what I do is I maintain a computer vision platform at Zeiss, and that platform runs on Kubernetes. If you want to learn more about my job or Zeiss in general, feel free to join the Zeiss booth, which is downstairs. Now, as most conference talks, this conference talk also has a repository where you can find all information that you will see today, including the examples. I will also upload the slides, so feel free to check it out. It's on my GitHub profile called Python Monty. It's a public repo, and it's called Inspect Docker Images. Now, I will share this slide at the end during the Q&A, so don't worry. We'll see it again. And what the repository is, it's a simple Flask app that sends requests to the cat API and obtains a cat image, and then it displays it on a simple HTML page. So this is how the app looks like when you run it on your local machine. And this is the repo structure. So the repo has a cat app directory where the Flask app code is stored. So it's a Python module. Then the next directory is the Docker directory where you have all the Docker file examples that you will see today. Then what you can see here, pyproject.oml and poetry.log files already give you an insight that this project is maintained by Poetry. So Poetry is a Python dependency management tool that lets you also manage Python virtual environments, and this repository is managed by Poetry. Now let's look into our first original Docker image, starting, of course, with a Docker file. So this is how the Docker file looks like. As you can see, it already, the base image is the Python 3.12 image, and the image has a build argument called token. Then we are setting some environment variables, including the value that we passed via the build argument token. Then we are installing some Linux packages. We are setting the working directory to be app. We are copying the entire content of the entire repository into the image. We are installing poetry with a specific version, and we're using poetry to install Python packages. And of course, in the end, we're starting the application. So this is our original Docker file that we will dig deep into. Now how you build the image, you build it like this, bypassing the build argument token via a specific value, which is in your virtual environment, wherever you run this command. So if you're wondering what this path is actually about, I purposefully made this example to install one private Python package from a Azure DevOps private artifacts registry feed. So this is an example that you quite often see in the industry, right? If you're working in companies, you quite often have private Python packages that you need to install, which means that you also have to handle the authentication to whatever this private registry is. And this is what we'll also look deeply into. So first command that we will look into when inspecting our image is Docker history. simple command you run it like this so zero minus original is our image that we already built and this is how the output looks like so what you can first notice here is that the commands that you see in your docker file at the bottom are actually at the top here so if you remember in our docker file the command one was the last docker instruction here you see it at the top so it's reversed what you can also see is that only the run and copy instructions contribute to the image size, a.k.a. they will generate a layer. We will talk about layers in a bit. And it's quite important to remember this, that only these three Docker instructions will contribute to your image size or generate a layer, so run, copy, add, remember that. And before we talk about layers or understand what layers actually are, let's talk about how Docker actually unifies the image layers via the file systems, and the file system is called OverlayFS. So what happens when you run your container? When you run your container, you will have multiple image layers that form a read-only base. So they're read-only, and they're called lower dir. On top, you will have one container layer, which is writable, and that is called upper dir. And so what Docker does, it unifies these image layers that actually each image layer has its own file system. So it unifies these different file systems into one. And that is called the merged view or the merged directory. And it uses that by utilizing the storage driver called overlay2. So now that we know that, this is an image that depicts it. And quite an interesting thing to know is, for example, file2, if you have it in one of the image layers and in the container layer, what you see is actually the file2 from the container layer because the container layer takes precedence. What does this mean? Well, think about it. We said each image layer has its own file system. That means that I can access via accessing the overlay to file system, I can actually access the file systems of all these different layers that are maybe even hidden because what we see is the merged view, right? We don't see each image layer. But by accessing the overlay to file system, I will have access to the separate file systems of the image layers. So on Linux, how you do it, it's quite simple, right? You just go to the varlib docker overlay 2 directory, but since most of us have Macs, it's a bit different. So on the Mac, it's a bit different because just how Docker runs on the Mac. Now how typically Docker runs, you will have some sort of a Docker client that starts a lightweight Linux VM on your Mac. Now when you type Docker commands on your, for example, on your terminal, that will talk to the Docker client, and the Docker client will forward that to the Docker daemon, which which runs in the Linux VM. So what we're doing here is we are starting a privileged Debian container with root access to the whole system. We are entering the namespaces of the process ID1. You see this by this minus T1. And with that, we gain access to the lightweight Linux VM, which is the host of all the containers. Sounds like magic, but it isn't. And inside of this container, we will have access to the same directory that you would have on Linux as well. So if you, you know, list all directories that are there, you will see some cryptic directory names that make no sense. But let's make sense out of that. With docker inspect. So docker inspect, again, quite simple terminal command that lets you inspect your images. Output looks like this. It's huge. I know. Let's look into different blocks. Firstly, environment variables, right? The one block tells you which environment variables I'm actually setting in my container. neat, right? Next one, the layers digest block tells you, so first of all, you can see this image has 16 layers. You can count or you can believe me. And these are the unique digests of each and every layer. So each and every layer does not get an image ID, but it gets a unique digest. And the most important one is this graph driver part. Now, you can see at the bottom the name is overlay two, right? And you can see this, that you have the lower And it tells you exactly for which layer, where is the file system, right? So the exact path to the file system of each layer. Still, I'm, you know, now we know, okay, where do we need to look? But we don't know what to look for. And there we can use dive. So dive is also a great command line tool for exploring Docker images. It looks like this. On the left-hand side, you will see the layers and the instructions that generated these layers. Quite neat. And then on the right-hand side, even better, you can check the layer contents of each and every layer. So you can check which files were added to this layer, which files were removed, which files were modified, and so on. And at the bottom, of course, you see some information about the image, for example, the size. Now, image details, the size is 1.5 GB, has 16 layers, build time was 188 seconds. Because this is our baseline image. Now let's maybe combine dive with what we have seen in the Docker inspect with the image layer paths to try to figure out how to use both of them together. Now the idea is to use dive to figure out which layer I'm interested in, right? For example, I want to see which layer has added a file that I want to look deeper into. Then we will use docker inspect to find the directory paths, and we will use then overlay two to access the files. So for example, I've seen here, okay, there is a copy dot dot command, and this is the 14th layer, aka the 14th lower dir. And I see on the right-hand side, okay, there was a file app dot pi. I want to see the contents of that file. What does this file contain? So we do some JQ magic on docker inspect, but what we're doing, it's not really magic. We're just reordering the output of docker inspect that it matches the same order that it appears in dive, right? Nothing else because the order inspect is reversed, so we just reversed the order. And remember we said it was the 14th layer? So this is the output that we get now, and we count from top to bottom, 14th layer is this one. So that means, okay, I need to, this is where I need to look to figure out the contents of that layer. Now again, we enter our privileged container. We go to this directory, and we can see that there is a directory called app. Now you remember, this was our working directory that we were setting in our Dockerfile. And we look into it. There is a cat app directory and app.py. And with this, you see we can view the contents, whatever the contents are of app.py. Now let that sink in for a little bit and think about how actually powerful this is. So imagine in one layer you're adding a file that, for example, contains a secret. And in the next layer, you're removing that file because you know you're removing it, you removed it. In the container, you will think it's fine, right, because you don't see the file, you don't see the file anymore. However, you can still access it by specifically targeting that file system of the particular layer where you added the file. So good thing to know. Right, now let's start with improving the image. First, every image starts with a base image. In our case, it's a Python-based image, so first things first, you should always pick an image that comes from a validated source. For example, in our case, let's, we're picking a Docker official image for Python, which is the first one that you see here, and there are multiple options for Linux. The first one is that we used in our original Docker file, however, is that a correct image for runtime. Now, you can already see it's 1 GB. It's a huge image. It contains the entire Debian Bookworm operating system, including a lot of Python tools, build tools, utilities, et cetera. So it's really a full-blown operating system image. There are very rare cases when you're running microservices or containerized applications in runtime where you will actually need such a huge image. Very little. So bear that in mind. The image that I personally most use is the Python 3.12 Slim. It's as you can see quite small, 43MB. And it actually defaults to the Slim Bookworm. So it's just a general Python image. You will have actually for runtime everything that you need, it will be installed in this image most of the times. Right. So what we did here is or what we're doing, we are focusing on this particular part of the image. And we are switching from Python 3.12 to Python 3.12 Slim. Not even that, we are, furthermore, we are going to pin the version to a specific digest of that image. Why are we doing that? Well, as always when you pin versions in software development, for consistency purposes, because the publisher actually of this image can, you know, tomorrow decide, okay, let's overwrite the Python 3.12 slim, and then you might get some unexpected changes when you rebuild your image. So that's also a nice thing to have. So this is the part that we changed. We exchange the base image, and by doing that, of course, we reduced the size greatly, because, you know, 43 MB instead of 1 GB. At base, we reduced also the number of layers, but the build time is longer. Now, the build time is longer. If you think about it, it's a slim image, which means you have less stuff preinstalled, so it can be that the apt-get update and install commands will take a bit longer. That's why. will tackle that in the next section, which is removing unnecessary packages. Now, what I see quite often is that people copy-paste Docker files, which is per se not bad, but it's bad if you copy-paste them and not think about what Linux packages you actually need for runtime. So, in this case, you can already see that we are actually installing peep twice. So, once with Python 3 peep and once with curl. And this is, by the way, an image from the industry. I literally really copy-pasted it, so this existed somewhere. And the fun fact is we don't even need pip because it comes with a slim image pre-installed. So second block is even worse. It's some OpenCV images, sorry, packages that you usually need for computer vision. But for this, you know, like just for sending requests to an API, it's really not needed. The only package that we need is libpq, which is a package that you usually need when you're communicating with PostgreSQL databases. And you can also notice that we are again pinning the version. So this is what changed. And as you can expect, the size reduced quite a bit, number of layers, and also the build time got a lot shorter. Right. One of the most important sections of today's talk is about layering and how to properly layer your Docker instructions. And my favorite topic, by the way. So why should you care about your Docker layers? Well, more layers essentially means your image will be bigger. It will be more complex. More complex means bigger attack surface. Your image pool will be very slow and your container start time will be very slow. And believe me, when you're running containerized applications, for example, on a Kubernetes cluster, the last two will matter a lot. Now how should we order actually when thinking about ordering the instructions, how should we order them? Well, it's great to know that Docker actually caches your layers, which means that a layer only needs to be rebuilt if it actually changed. And also a good thing to know is all other layers need to be rebuilt after the layer that changed. So you should really think about how to order your instructions properly. A general rule is that least volatile instructions go to the top and the longest running ones go to the top. Usually, very often, the least volatile ones are also the longest-running ones. So let's look into the blocks that we will change, so pretty much everything. First things first, this is not really related to layering, but just a general good practice. Do not set Debian Frontend non-interactive into the environment. So what Debian Frontend non-interactive does, it changes the behavior of the install and update instructions of, for example, apt-get, so that there's no interaction needed. And if you set this as an environment, it will persist on your container, which means when you run the container somewhere, it will, you know, change the behavior for the user, whoever the user is. So it's essentially something that's needed during Docker build. So you can either set it as an argument, an argument persists only during the build, or you can set it, like, for a particular run instruction, as you see at the bottom. Next, this is a nice thing to have, right, chaining multiple end instructions into one. The end instruction does not generate a layer per se. It generates an intermediate layer, but this is, you know, for readability purposes and it's just a general best practice to kind of chain them into one instruction. This would be obvious to everyone, but literally I see this quite often. I mean, running echo commands in a Docker build is senseless anyway. Echoing secret is even more senseless. So please don't do that. Next really important one, always, always, always chain your update and install instructions into one run instruction. So why should we do that? So we're talking about layers, right? One run instruction is one layer. Now if I do an apt-get update in one layer, the package registry that I'm, you know, pulling to get updated will be updated for that layer, aka that file system. But it won't be available in the next layer, which is a completely different file system. So if I want the updated package registry, I have to combine them into one run instruction, into one layer, and in the end, the nice thing to have is just after you install everything to remove the updated package registry just to save up some space. Next question, poetry, right? How should we handle this? So what we're doing here is should we have, you know, the poetry installation and the Python dependency installation as one layer or as two separate layers? So I prefer two separate layers, but, again, it greatly depends how you want to utilize Docker cache. So why do I prefer two layers? Well, most of the times, you will update your Python dependencies more often than you actually update your poetry version or poetry installation. So that's why most of the times it makes sense to keep them as separate layers, even though you will have one layer more. But again, it's all the question of how do I want to utilize Docker cache. Very, very important one, always, always, always first install your Python packages and then copy the application code, right? We talked about volatility of instructions. Your code is, in general, bound to be more volatile than your Python packages because, you know, you change the versions less frequently most of the times. So, you see on the right-hand side what we're doing is we're first installing Poetry. Then we're copying just the files needed by Poetry. Then we are running a Poetry install dependencies. And in the end, we are targeting the copy only on the cat app directory. So, on the left-hand side, we did copy.dot, right? We copied everything. On the right-hand side, we're doing selective copying. Now, copy.dot in general is not bad, right? It's not a bad practice. However, if you're doing it, make sure really, and this is really important, to maintain a Docker ignore file that is well kept and well up-to-date. Because the Docker ignore file tells Docker, ignore all of these directories and files, And then you can copy the dot, you know, selectively copy whatever you need for your image. Great. So this is how our image with improved layering looks like now. Setting as argument, as we said, right? Chaining and instructions. Chaining update and install into one. Installing poetry. Copying just the poetry files. And installing the Python dependencies. And copying the application code. Next one. This is also quite important if you're using poetry. So poetry in the meantime became, I think, quite popular. I wonder if there are some poetry talks. We'll see. But anyway, poetry is quite popular in the meantime. And there is literally very little documentation how poetry should be handled in a Docker scenario. So that's why this chapter. And this is the parts that we are specifically looking at. So first things first, it is per Poetry documentation in automated scenarios, which is CICD and as well as Docker builds, you should use Poetry to, you should use pip to install Poetry. So there are multiple ways to install Poetry. Pip is the suggested way for automated scenarios. We're already doing this. However, it's also recommended to delete the pip cache directory or disable it. So that's what we're doing. And that will save you some, in this case, I think it was like 30 MB. Next one, very important one, you should install Poetry in its own dedicated virtual environment. So this keeps Poetry separated from the rest of the system. Which is great, because, you know, it could happen, if it's not separated from the system, that Poetry's dependencies maybe get updated accidentally, maybe they get deleted. And this keeps everything nice, clean, and separate. So what we're doing here is you can see we're installing Poetry in the Ept slash Poetry directory. Next one, Poetry gives you a great option to separate your dependencies into dev dependencies, test dependencies, runtime dependencies, and so on. So dependency groups. And of course, for runtime, you should only install the main dependencies. So the dependency is actually needed for runtime, not like test dependencies or dev dependencies. Next, a very also important one, install your Python dependencies in a dedicated virtual environment. So this is not the same virtual environment that we are installing poetry into. It's a separate one, right? And we are handling this with the tool you see poetry-virtual-envs-create is now set to true, and poetry-virtual-envs-in-project is set to true. With these two, we are telling poetry create the .vn directory in the current working directory, which is app, and install the dependencies there. So that's why we're actually, if you see the path, right, we are adding app vn bin to the path so that we know that when I run the Python executable, it will be taken from app vn bin. And fun fact, when you do, if you're working with VNF, if you do VNF bin activate, that's exactly what VNF bin activate does. It just appends the path to the variable path. Great. Another one, nice to have. Same as we deleted the pip cache, you can also delete the poetry cache, saves up some space. There is a way to mount it, which, of course, unfortunately, we don't have time to do today, but it's a bit complex. Yeah, you can just delete it. Great. So, this is what we changed. As you can see here, we're installing, as we said, poetry in the separate virtual environment. We're installing Python packages in a different separate virtual environment. And deleting poetry cache. Right. Next one. Very important one. So, whoever is sleeping, wake up now. Secrets. What we're doing here is, I said it a little bit at the beginning, we are passing the secret that we need to authenticate to a private package registry as a build argument. So this is really, really bad practice, terrible practice. Why? Well, because if you set it as a build argument, even if you don't set the secret into the environment, it will still persist, you know, it will still be visible in even simpler commands like Docker history, right? We saw it in Docker history that the argument value was not masked, so it was visible and it's really easy to read the secrets. Setting it into the environment is also the same, like, same level of bad. You can see it in the container. It's also unmasked. There is a really, you can see, yeah, docker inspect, you can see the value of the secret. There is a really simple way to solve this, and this is by using docker build secrets. So what you're doing here is you are mounting the secrets to a particular run instruction. You can see here, right, what we're doing here is I specify, okay, secret with specific ID, and this particular run instruction will utilize the secret. The secret won't be persisted in the final image or the container. So this is the part that we changed. And how do you build this image? Well also quite simple by using the secret option when you're building the image and you have the option of you know You have to specify the same ID that you're specifying the docker file And you can either pass the value from the environment or from a file Right Non root user another important one So following the principle of least privilege that we all Know so well your container should not run as root user. Why is that? Well, if you have an attack on your application, an attacker Can break out of the application and, you know, land into the Container, and he or she will have root access to your host Machine, which is really bad. So, again, a very simple way to Solve this. What you practically do is you Create a separate user. You can give it a certain name certain name, like app user, or assign it a specific user ID and group ID, you assign it to a particular group, and you execute some root-level instructions, you know, like installing packages, and in the end, when you don't need any root instructions anymore, you switch to the onRootUser. So this is how it looks like in our changed Docker file. You see? We are creating the user at the top, right? Why at the top? Because it's least volatile. It's a layer that doesn't practically change, so we put it at the top, and then we execute the whole root level instructions, and in the end we switch to the non-root user. And now your container will start as app user. Quite neat. Next section is bind mounts. It's kind of nice to know that it exists, I would say. So what we're doing here is, if you think about it, we are copying the two poetry files into the image. But actually, we just need them for the build when we're installing Python packages. We don't need them to persist in the image. So what you can do for that is so-called bind mounts, which you, similar as mounting secrets, you mount files that are needed for a particular run instruction. And they don't persist in the final image. So instead of copying the files and running then the installation, what we're doing here is we are mounting the files to this particular run instruction that does the installation, and that's pretty much it. So, we saved up on one layer, so, because we removed the copy. And this is how you, exactly, so, this is how it looks like, you see? And this is how our image looks like now. And this is our last section of improving the Docker image for today. Super cool one. one of my favorites, multistage builds. So, why are multistage builds cool? Well, they produce smaller images. They offer a cleaner separation of your build stages and your final output. And they can create reusable stages that you can use across different multiple images. They offer also improved security because, you know, smaller text surface and the builds are are really, really fast. So this is the image that, you know, we are trying to improve. And the idea is that we will create three stages. So the first stage will be the base stage, which will install Linux packages that we need for runtime. The next, so this is how the base stage looks like. You've seen this, you know, already multiple times, just installing this one Linux package that we need. Next stage will be the installer stage, where we, you know, we are installing packages that are needed for the build, but are not needed for runtime. In our case, this is poetry. And we will also, strangely enough, install Python packages that are needed for runtime. We will see why. So this is how this stage looks like. As you would expect, we're, you know, you see this, you saw this multiple times, we're installing poetry and we are installing the Python dependencies into a build slash VN directory the last stage will be of course the runner stage which takes the base stage as base image right because we said we needed those packages for the runtime and now you see we will copy Python packages that we installed in installer stage into our runner stage and in the end we will set the non-root user so this is how the runner stage looks like you can see we are taking from the base stage we are creating the non-root user we are copying from the installer stage the entire virtual directory path where the python packages live into a new working directory like into our working directory into the v-env path, and we are, of course, appending that path to the path variable. Cool, right? And in the end, we are switching to the non-root user. So this is how the entire file looks like, the entire Docker file. You see that the stages occur one after the other. And the cool thing is you can targetly, we won't see it, unfortunately, but you can targetly build the stages separately. So I can tell Docker, okay, build just the stage installer, build just the stage base. And the cool thing about that is that Docker cache is utilized super efficiently because you can actually cache the entire stage. So if the entire stage didn't change, you cached it as one layer, so to say, and that enables you to not only save up on the image space, as you can see here, but also your builds will be very fast. And last but not least, I'll tell you about two security tools that are super useful, and yeah, that will be then the last chapter of today's talk. So first tool that I want to mention is Hadolint. Hadolint is an open source Docker linter that not only lints your Docker files, but it can check for some building best practices in your Docker files. The cool thing is it's easily integratable with CI CD pipelines, but the coolest part, in my opinion, is that it's available as a pre-commit hook. So when you're doing your commits, the hook will run on your Docker files and automatically lint it and check for building best practices, which is really cool. You can also, you know, specifically run it on particular Docker files, what we're doing here. And you see, it tells you already in our original Docker file multiple consecutive run instructions. It tells you use pip install no cache dir. So it's quite a cool tool. Now a cooler tool is Trivy. Trivy is quite powerful. And Trivy is also an open source tool and it's a security scanner for your Docker images. So it scans for vulnerabilities, not only in operating system packages, but also in your Python application dependencies. And it also scans for misconfigurations in images. So, basic configurations similar to Hadolint scans for best practices, right? And it's really easily integratable with CI-CD pipelines, especially GitHub Actions. So, GitHub Actions, if you check the repository, you will actually, you can also access the repository and you go to the security tab, the code scanning, and you can see that it tells you already, okay, secrets passed is via build arguments. So, it's quite cool. And also the last one, for example, it tells that there is a critical vulnerability for this zlib package that we're installing. So these are just some high vulnerability dependencies, vulnerabilities that are mentioned. Image user should not be root and stuff like that. So most of the things that we mentioned today Trivi can detect, which is quite nice. Right. Now we move to Q&A. And as promised, I will share this again.

Speaker 2 [33:25]

Thanks a lot. I think as you can see from the smartphones, there's a lot of interest in this room and we can also see it in the Q&A section. Yes, and I'm going to start with the first one. And it's what about security updates when you pin the digest of the base image?

Speaker 1 [33:42]

Yeah, I love this question. Thank you. Yeah, very important one that I didn't mention. You should always, I mean, regarding all dependencies, right, pinning image versions, pinning Python package versions, it's best practice to really rebuild your images very often. So make sure that you just don't pin the version and never update it. But the purpose of pinning is not, oh, I will never update it. The purpose of pinning is to prevent unexpected issues from happening and to have controlled updates. Whenever I want to update, whenever we say we should update in regular intervals, we do it, not accidentally. So that's the point of pinning versions.

Speaker 2 [34:25]

Thank you second question thoughts about using poetry versus UV and docker files

Speaker 1 [34:33]

Yes, good question. I've seen a lot of UV examples, like on GitHub forums, et cetera, that they use. I would say using UV in Docker images is the section that we talked about in poetry, UV would be quite similar usage, right? So you would also, as with poetry, install it in dedicated virtual environment and so on. So, yeah, I think it's a good tool, if that was the question.

Speaker 2 [35:04]

I think so. If not, please feel free to ask another more detailed question in Slido. Next question, and I think you kind of talked about it on the last slides, but we go again. What are the most useful low-effort measures? Can you recommend references like the OWASP Docker cheat sheet or similar guidelines?

Speaker 1 [35:27]

Yes, so I think it's in one of the slides, footnote links.

Speaker 2 [35:28]

Yes.

Speaker 1 [35:32]

There is on the Docker documentation, Docker best practices. So if you just Google Docker best practices, many things that I mentioned today are also in that documentation, and some more. And also, Hadolint implements and scans for all of these best practices mentioned in the Docker documentation. So I would suggest that as a starting point.

Speaker 2 [35:53]

Next one. To what extent do the presented concepts apply to other container build tools like Builder, I guess? I'm not quite sure. Maybe you can have a look.

Speaker 1 [36:04]

Have a look. Where is the question? The last words here. To what extent do the presented concepts apply to the other container build tools like BuildDuck? No clue. I just use Docker. No clue. Okay. No clue. Okay. Okay.

Speaker 2 [36:18]

Did you try other package managers before you landed on Poetry?

Speaker 1 [36:18]

Okay. Okay. Okay. Okay.

Speaker 2 [36:21]

If so, why was Poetry the chosen tool?

Speaker 1 [36:22]

Okay. Okay. Okay. The chosen one. Yeah Okay. Okay. Yeah, so Okay. Okay. Yes, I Okay. Okay. Would say Okay. What I see a lot in the teams that I worked in a lot of people use it just bare-bone pip Okay. especially in docker scenarios Which is I mean, it's bare-bone pip, right? It doesn't really offer this this powerful, you know, for example dependency separation that poetry does Because what Portree has, what Portree also has is hashing the dependency versions, which is really great for, you know, reproducibility purposes. I personally like it because, A, it's super fast. It's a lot faster than, for example, peep, I mean, conda, no comment. It's even slower. I like it because it has, you know, this clean separation of dependencies, but within one virtual environment, which is also quite nice. And it's easy to install. It's, you know, you can see it's easy to use. So, yeah, that would be it.

Speaker 2 [37:26]

Does all this work also with Docker Compose?

Speaker 1 [37:30]

Yes, it does. I can confirm. We use it.

Speaker 2 [37:36]

Best approach for installing python packages in a docker container using poetry when some packages are pre-installed via pip in required versions

Speaker 1 [37:52]

Which one is it? The first one.

Speaker 2 [37:53]

The first one. The first one.

Speaker 1 [37:55]

But if you have a you know if you if you completely separate your your poetry installation and you completely separate your Python dependencies What we're you know what we shown then I'm not sure how this would relate if you if there are some system packages in the system peep installed Whoever asked this feel free to come and discuss later

Speaker 2 [38:20]

Can you elaborate on the advantage and overhead of using virtual environments in a Docker container, which is also quite isolated by definition? And maybe I expand this question because the next one kind of goes in the same direction. Could maintainability of the Docker image after such improvements be an issue?

Speaker 1 [38:45]

Why?

Speaker 2 [38:47]

Many engineers could be confused as a lot of those practices are not well known.

Speaker 1 [38:53]

Actually, they're all documented on the Docker documentation. No, really, really. Like official Docker documentation, they're all documented. And as you can see, there are official tools that even let you scan for all of them. So I'm not sure how... I wouldn't suggest writing your own Docker documentation within your team and documenting the best practices that are already documented. So don't do that. Maybe just link to the official documentation that already exists. Thanks for watching!

Speaker 2 [39:24]

Maybe that's a good last comment, and thank you again for this talk.

Irena Grgic

About — in the speaker's own words

As a clean code enthusiast, Women in Tech advocate, DevOps engineer, and mathematician, I have worked in multiple tech fields. My journey has taken me from roles as a data scientist and machine learning engineer to MLOps, culminating in my current position as the lead DevOps engineer of a computer vision platform with hundreds of active users. I possess a broad range of experience in multiple programming languages, creating fast and structured CI/CD pipelines, deploying entire platforms to Kubernetes, and working with various cloud providers. I am passionate about efficient, well-readable, and easily maintainable code and strongly believe that machine learning products should be developed with the same standards as good software.

Social card for talk: Size matters: Inspecting Docker images for Efficiency and Security