How to Find Your Way Through a Million Lines of Code

Speaker 1 [00:04]

Hello and welcome to my talk how to find your way through a million lines of code. My name is Jürgen Gmach and I'm a software engineer at Canonical. You can always reach out to me on Twitter. As a software engineer from time to time we have to pick up a new code base. Maybe you want to contribute to an open source project or your team takes over a project from another team or maybe you change employers. This is what I did. And this might be a scary thing, right? But there is no need to worry, because picking up a new code base is a skill just as many other skills you need to learn as a software engineer, and I will help you with that. Over the course of the last 15 years as a professional software developer, I worked on many code bases and I picked up many new code bases and I developed a process to look at those projects and detect patterns like the directory structure like have a look at the tests or have a look at configuration files and this worked all great until I joined Canonical to work on the Launchpad project launchpad is basically a code hosting platform but not only that it's also there for building packages for Ubuntu creating translations when hosting translations for open source projects and maybe you heard of them the famous PPA the personal package archives are also managed by this and a couple of facts about Launchpad. Launchpad is 18 years old so in Germany it would be allowed to drive a car or go voting. At some point in time up to 30ish engineers worked on the project concurrently. Currently we are four and soon five and as you can see on the picture here there is more than one million lines of code only in the core library. and another fact the test suite takes six and a half hours to run on my laptop this is something we have to talk about later on where would you start here right this is huge and well i can tell you what i did as this is an application and no library i just used the application i logged on at launchpad.net and use the application and even more my back then manager christian gave me little tasks like create a project set it to private invite more colleagues create a translation and all that and this is super important that you learn about the domain of the application. Most of the time the domain will reflect in the code base. There will be components like one component for the translation section, one for the issue section, one for the other section. So you already have a good view of the code base in your head. The other thing is that you learn the terminology and this is super important on the one time on the one hand that you can use the terms you found in the application to search later on in the code base and the other thing is that you can speak with your colleagues and the other stakeholders in the same language maybe a builder is not the same thing for an external one and somebody working on the code base here and additionally you have a fresh look on the code base or on the application and maybe you find some rough edges maybe you find even bugs because you use something in the software which wasn't supposed to do to be used that way and so you can report issues and feature requests and so you can basically contribute to the big big project from day one And after you're a little bit familiar with the application, there are different ways to go ahead. You could sit down and read the documentation for a couple of hours, days or even weeks. You can look for a mentor, maybe the company offers that. You could try to get a colleague to pair program, so all good ways. But I tell you what I did and what I always do when I work with a new development environment or a new code base. I set up the development environment. And as you can see, there are two heads and the one thing is running the tests and the other thing is running the application locally. Because when you do changes, you don't know yet whether you can trust the test suite. So you want to try out it locally. And before we do this, let's have a quick look at the codebase. Things happen when you work with huge codebases, even when prompt timed out, which was easy to fix as you just had to increase the timeout. Now let's have a look at the directory structure. And there is a lot of things going on. And actually not all of that important. A few of them really stick out. And you will get when you work with a couple of Python projects, you will want this to have this view on this directory structure. And have a look here. We have a doc directory, we have a make file, we have a pre-commit config yaml, we have the readme, we have requirements files, setup gg, setup py and toxini. Let's have a look at some of them in detail. But before we have an even closer look at those, we have a look at one which wasn't there. This is a Docker or Docker Compose configuration file. Actually, this is something I always look at when I start with a new project because with that you can spin up a container in no time and here in this configuration file you clearly see the architecture of the application. You see a Postgres database is used, the web component is a Django application and you can already have a good picture of the application before even working with the code. Now let's back to those who are really there. We have a pre-commit config YAML. This is the configuration file for a pre-commit, which is a linter runner. And this is a great sign, not only that you know that your colleagues care about code quality, but also this basically codifies the conventions of the project and you don't have to be scared to create your first pull request when the colleagues won't nitpick about an empty line, spacing here and there, line breaks and something like that. So this is a huge plus having such a configuration file. The next one is read-a-docs.yml. This is the configuration file for the awesome reader docs documentation hosting service and this basically means there is documentation for the project which is a good thing and also the documentation is checked in to git which is a great thing and this is something I will tell you more about later on. The next one we have a toxini or makefile and this is also a great sign. Toxini and make files are usually used as an abstraction layer to create and manage virtual amps and build the project and maybe run the tests in this case it's used to create the documentation locally which is a great thing because you can check whether it builds or you have some mock-up errors and when you have a look at the last line you really don't want to remember this command right and the next group of files are the requirements txt setup gfg setup py py project toml and that are all configuration files for the build and installation steps and they contain more but the most important things are the dependencies and listed in those projects and before we have a look at some Python dependencies, I want to speak quickly about the different types of dependencies. We have Python dependencies, which you usually install from PyPI or a private project hosting service. We have the system-level dependencies like Apache, Nginx, HAProxy or whatever your application needs postgres and you can have an external API like the integration of a prime payment service like stripe or something similar let's quickly talk about Python dependencies and when you have a look here this is a huge huge list right and Async going on. We have Boto3 which is a SDK for AWS for the Amazon Web Service. So maybe there is some connection with the AWS cloud. We have Gunnicorn which is a WSKI server. So you basically know this application is a WSKI application. We have Laser SSH server which indicates that you can SSH into this application. We have LaserRestful, which maybe nobody heard of before, but this also indicates that maybe this application offers a RESTful API. We have OLAVIP, which is an authentication library. We have PSYCOP2, which is a library to help access the Postgres database. This is quite important. We have PyMemcache, which is a Python library to access the Memcached key-value store, so you can think that maybe there are some cache involved. We have Storm, which is an ORM, Object Relational Mapper, to access the database just with Python and no raw SQL statements. And we have Twisted, which is not really a lightweight library, and this is used for event-driven programming. And when you have a close look at the bottom, there is a ton of SOAP packages listed there. So basically we know this application is built upon the SOAP stack, which is a web application server. So now let's finally set up our development environment. We already saw there is no docker configuration there, right? And there was also no automation like Ansible or Batuu or something like that. But we have a comprehensive documentation. Unfortunately this documentation was outdated and the steps didn't work. And things like that can happen, especially when there is low staff turnover and the other engineers are working on the project already for years, who will have a look at the setup documentation. And no worries. This is a great chance to contribute again to the project already on day one or day two. And I tried to fix the issues and set up the development environment. And this is what I always do when I work with a new project. I try to improve the situation there. But actually, I tried. I tried too long. And usually one thing, one rule I follow is don't spend too much time on a problem, especially when you work with a new code base. Maybe 50 minutes or 20 minutes is okay. But then get unblocked. Talk with your colleagues. Ask them questions. And I really hope you work in an environment which encourages and even expects that you ask questions and this is really really important and if you take one thing from this presentation do this one take this one now that we have the development environment up and running we need to be able to run the tests and as i already mentioned ideally there is a tox configuration so you can just enter tox-e and pi37 or 39 or whatever and run the tests or you have to look in the documentation and be aware that while we all agree pytest is the greatest test runner in the Python ecosystem and in basically maybe in all ecosystems it's not used everywhere so you really have to figure out a couple of commands with the used test runner and especially the most important one I think is to select a subset of tests because when you work with such a huge code base and I already mentioned running the complete test suite takes six and a half hours so you can't just quickly fix something and wait six and a half hours to check whether this has worked. So you need to figure out how to select a subset. Usually this is done with pattern matching, pattern globbing or with a regex and sometimes it's pretty hard to select the right test because there are many similarly named tests or even a module named like the test and just shorten it and it's sometimes hard and i have one very obvious trick to work around this i just temporarily rename the test which i need to run i just append a couple of a's or x or something like this so i can really select simply select this one. Maybe you can even create a better name and create the pull request. And as you can see on the picture it might be a great idea to figure out how to run the tests in parallel so they are running faster. Now you're ready for your first task and this should be a small bug, a really tiny bug and ideally chosen by your manager, your colleague or the maintainer or when you contribute to an open source project maybe there is some bug labeled as good first issue and this should be something like a wrong label a wrong error message or something really tiny right but still now you have this one tiny bug and a huge huge huge code base where would you even start here, right? And the first thing I always do is have a real close look at the bug description. We need to find a good term which we can use to search in our code base. Look for strings. Maybe there is an error message presented there. If not try to reproduce the bug in your browser or in the API and have a look for good search terms on the UI and when you don't find some you can also use the code inspector of Chrome or Firefox and have a look maybe at closer look at the names of the fields in a form to have something to search for and yeah when we want to search now there are different ways to search you can use grab but grab has a couple of downsides it's pretty slow and it even searches in directories with where you don't want to search so a better way is git grab you can search only in the tracked files and it's much much faster and you can search even in past revisions. You can also use ripgrep which is a new implementation of grep in Rust which is blazingly fast or what I really like to do is use Visual Studio Code and I have to admit previously I never used any of the filters because it was not necessary. I worked on normal size project. But this time I really leveraged all filters. You have an include filter, an exclude filter. You can search for occurrences only in Python files, only in Python test files, only in Python files which are no tests. And as a bonus use case-incentive search. I never used this before but this could cut down your result set in half and this is already great and while you're there make yourself familiar how to do a search and replace across the code base maybe there are thousands of comments or whatever or something which you want to remove and you don't want to do this manually right so now we have found the code we want to change and now we need to write our first test right usually i practice tdd but this is especially hard when you work on a new code base maybe you don't even know where to place your test and you don't know how to set up a test in this new environment so what i usually do is i look for similar tests but wait there are 10,000 of tests, how would you find a similar test to what you need to write? And there are two ways. The first one is set a breakpoint at the production code you want to change and then run the test suite. And when the test suite stops, you know a test has covered this line. In the debugger, or depending on the debugger, you just have to go up the stack and you see the test name. this is great or you can just break the code deliberately run the test suite and maybe 10 tests fail and you know this test cover these lines and this usually works great but in this huge code base and when the test suite takes six and a half hours no way this does not work and there is even a better way and I think you run tests right because you should and you run coverage because you should And when you have a look at the picture here, this is the typical view of a coverage report. On the left side, we have the gutter, which shows which lines are covered. But how would you know which test has passed this certain line? And since version 5 of coverage, there is a new feature, which is called context. Next, when you have a look at the right side here, for every line of code, the number of tests which passed this very line are listed and you can click on the little triangle and then you have three, like here in this example, three tests which passed this single line. Then you can go to the tests, update them or maybe copy them, use them as a template and create a new test. Sometimes this does not work, because you need to set up something new. And especially in such a huge code base, setting up a new test can be pretty pretty hard, because you need maybe a complicated data structure, or maybe you have set up a complicated environment like a company, a delivery node, customers and all those things. And factories help here. Codes are basically functions which create fixtures or the setup for a test. Usually with sane default values which you just have to update a little bit to make your test scenario happen. Look for them. A quick interlude here about the tooling support while you're in the new codebase. Make sure your IDE or your editor has set up the tool support. And in just this example, when you hover over a symbol, you get a pop-up with both the signature of the function, the doc string, and you can click on the symbol and go there and go to the callers and all that. And this is awesome and especially in a huge codebase because you don't want to have 20 or even more tabs open. You can navigate the codebase with modern editor. So now it's time for your first commit. And as we already said, we have pre-commit in place, which is a great thing because your code is linted. You don't have to worry about obeying to the project conventions, they are codified. Before you create the commit, review your changes yourself. Maybe you have left a comment in there, a commented out code. Maybe you have left even a breakpoint or something like this. Have a look at this yourself. Then look at the commit history. What are the conventions for writing a commit message? surprise your colleagues by writing a really good commit title and maybe they use the link to the bug report or something like this. Use a succinct title, a good message which describes the what and the why. And similar things apply also to your first merge proposal or pull request. Look at other MPs, how your colleagues did this. And then Then you need to figure out who will review your Merch Proposal and how do you get them to review it. And once they approved it, who will merge them. And as always, please take notes of everything you learn. And at this place, at this point, I think you already had many, many, many questions. And I hope you took notes of everything. about undocumented things, about strange things. And a quick word of warning, chat is not documentation, pull requests are not documentation and emails are not documentation. When you're at a new company or a new co-place for half a year or whatever, you can't search all of them and you won't find your stuff again. take notes and there are different ways to take notes you could create a developer journal and I really recommend that that's an awesome thing because you can take down your accomplishments you can use the developer journal for your stand-ups for retros for maybe for your CV when you want to apply at another the company again but that's not the place for documentation about the project also your local nodes are okay but nobody else can profit from them and i really hope there is some kind of official documentation especially i hope it's something like reader docs so you can contribute in with a pull request and the good thing about this is you will think about it again when you create an official pull request and then somebody refuse your changes and maybe you understood something wrong and took the notes of it and so you get corrected and this is awesome and when you really want to take off with documentation i would recommend to have a look at dear taxes framework by my colleague daniele proceder and basically this is a framework which groups all documentation in different areas and I think tutorials are pretty self-explaining that you learn something how to learn to use a product how-to guides are bullet points no stories it's like how to deploy check out the code create a tag push it over here and push the button or how to do a database migration. Explanation is something more like an architecture guide or technical decisions in depth explained and reference is something like API reference or a style guide. There are great talks out there you can have a look on YouTube. And now we are about to deploy and ideally this is done by the developers right and not by the sysadmins and when we are able to deploy ourselves usually there's some kind of configuration management or orchestration tool like ansible batuu or something similar and you can have a look at those configuration and that's a great source to learn more about your application because all the all the complete infrastructure is codified there and and where it gets deployed and all those things. So this is basically the way I learned to get up to speed with the new code base with Launchpad. And there are other ways to learn. You can have a look at log files, so you can learn more about your application. You can certainly, as I already mentioned, pair program with somebody. You can look at what the colleagues did. Have a look at old Jira issues, old merge proposals, old commit messages. When you have to implement a new API endpoint have a look how the colleague did the other API endpoint. You can read tests, maybe there are approval tests, acceptance tests and doc tests. They tell a story so you can learn more about the application. And you could start your application and follow through the whole stack from the entry point. And to wrap up here and have a quick recap the key concepts I saw picking up a new code base is look for familiar patterns, do never hesitate to ask questions, take down notes of everything, be aware of what works and what does not work and no pressure at all, but I think you can add value from the one, improve documentation or create bug reports. So thank you, thank you, thank you for all the pictures I was able to use for this presentation. Thank you for the slide generator and thank you, dear audience, for joining my talk. And now it's time to pick up a code base of yourself. It's really fun. And if you don't have one, we are hiring. You can contact me on Twitter or any other channel. So thank you so much and have a great, great PyCon DE in Berlin. Bye.