In Praise of Documentation: Tools, Tips & Techniques for Literate Programming in the AI Age

Speaker 1 [00:06]

Okay, please take a seat if you haven't already. It's my pleasure to introduce Stephen, a data scientist from Liverpool, but living between Berlin and London, right? So today's talk is in praise of documentation. He might make us feel a bit sorry for documentation practices, but he will also give us hope for a better world. The floor is yours.

Speaker 2 [00:31]

Hello, Guten Morgen, Darmstadt.

Speaker 1 [00:31]

Thanks.

Speaker 2 [00:40]

I feel a bit like Peter Gabriel with this microphone on my face. My talk is in praise of documentation, and it has one simple message. Please document your code. And that's the too-long-didn't-read. By the end, I hope to have convinced you that documentation is a good thing. And if you want your code to be used, you should document it. I'll explore tools for literate programming and also share some tips on writing well. Writing good documentation is becoming more important in the age of AI, so-called. And I'm essentially arguing for better scientific research practice in data science in industry settings. And also I'm really saying that narrative is extremely important. So I think my talk follows on really well from the keynote this morning. So how I learned to love documentation. Can you put your hand up if you think documentation is important? Oh, lots of people. And keep your hand up if you enjoy writing documentation. A very small percentage of the people think it's important. It's an interesting cognitive dissonance, isn't it? If someone could turn the lights down, has anyone got control of the lights? It's blinding me a little bit. I don't know if anyone does. Maybe they don't. Okay. Anyway, I first discovered the pleasures of technical documentation 40 years ago as an 8-year-old boy using my BBC microcomputer. And that's it there. It's part of the personal computing age that Aaron talked about before. And this computer was literally my best friend. I was an only child. And I created a real attachment bond to this machine. a bit like the attachment bonds we have to our phones now. I was always trying to communicate with the machine. The BBC Micro was an 8-bit computer, that's 8-bit, funded by the actual British Broadcasting Corporation, the BBC. And it was released in 1981 as the centrepiece of their computer literacy project. And that project was designed to make the British public more computer literate by making computers and programming more widely accessible to the masses, mainly by making the BBC Micro available for use in schools. It was quite an expensive computer. I got a second-hand one from a rich uncle one birthday. The machine was designed and built by Acorn Computers in Cambridge in England, who also built my first computer. So this wasn't my first one. My first one was called an Acorn Electron. And Acorn built, as well as the Acorn, the ARM processor, which is now sitting in all of our phones. And one of the first distinctive features of the BBC Micro when you switch it on and you plug it into your TV set, it will make a beep, beep sound, and you would immediately find yourself on what we now call a command line or a terminal. And it looked like something like this with a blinking cursor. I was faced as a young boy with an immediate practical problem. How do I use this thing and what is it? I have no mouse. There's only a keyboard as my input device. There's certainly no internet. and at this point at least 10 years later I got my first modem but it wasn't plugged into the BBC and there's also no graphical user interface there's no mouse like I'm using now but thankfully the computer came with a user guide a very well written user guide and it explained how to type BBC basic commands to run your programs using the keyboard. So that blinking cursor, you were straight away in BASIC. So it's a bit like being in a Python interpreter, straight away. And the programs, you would type the programs out from the user guide and press enter, and it wouldn't work. Type it again, keep pressing it. And then if they do run, they run from cassette tapes or five and a quarter inch floppy disks, which would make a noise. There was also a magazine, BBC Microuser, which I avidly collected, and I learnt to program in BBC Basic by manually typing out the type-in programs, like that cat GIF that you see, and mostly of computer games. In these early days of popular computing in Britain, learning the technical task of computer programming was intimately tied to reading documentation literature, and they're very closely tied together, I would argue. So years later, I learned my early enjoyment for reading what we came to call docs was matched by an enjoyment in writing technical docs when I was at work. But I quickly realized that most other engineers and data scientists often didn't share my interest or indeed passion for the docs at all. I often found people either wouldn't document their code at all, or they would document it poorly and often under duress by their manager telling them they had to do it. But I want to say that bad things can happen in companies without good documentation, and it can actually be catastrophic in some cases that I've seen. Documentation is the institutional memory of a company, and without good documentation, it becomes very difficult, almost impossible to understand the code base that's running the software and the products. New colleagues can feel alienated. They can struggle to quickly learn the new and complex code base, especially of legacy systems, and be unable to fix or update it. It might even take a whole team of PhD-level developers months, even years, to reverse-engineer a large legacy code base to be in a position to rebuild or replace it. And needless to say, this is a very expensive and time-consuming task. In her recent book, which is excellent, actually, by Katherine Nelson. It's called Software Engineering for Data Scientists. She argues for the importance of documentation, and she has a whole chapter on it. And she says, Documentation is an often overlooked aspect of data science. It's commonly left until the end of a project when you're excited to move on to a new project. The documentation is rushed or omitted completely. However, documentation is a crucial part of making your code reproducible. If you want other people to use your code, or if you want to come back to your code in the future, it needs good documentation. It's impossible to remember all your thoughts from when you originally wrote the code or initially carried out the experiment, so they need to be recorded. I also would say that all of us are writing what will eventually become the legacy code of software of the future and I think by writing documentation we can be a little bit more responsible in our stewardship of these technical systems to allow their future maintenance, particularly in times of crisis when those systems break, which I would argue is going to only become more and more likely now we're shifting more to probabilistic, non-deterministic systems, like agentic systems. So Katherine Nelson has a list of different types of documentation. I won't go into detail on these, because you could find them in her book, and they're kind of obvious or basic, like having names, good names for your variables, having comments in your code, using doc strings, which are the little quotes that could start functions or modules or classes, having good readmes, making tracking reports of your experiments and logs. The bit I'm going to focus on is Jupyter notebooks. Has anyone used Jupyter notebooks? Pop your hand up. Quite a lot of people. These are probably the most common format that data scientists use in industry settings, and she said that notebooks will be much easier to read if you give them good names, give them a structure, and intersperse text and code. So who has heard of literate programming? Can you pop your hand up? So lots of people use Jupyter notebooks, but they've never heard of literate programming. This is essentially the genre of programming that you're doing or aspiring to do when you're using Jupyter Notebooks, and I think it's a helpful concept and history to know about. The concept was invented by Donald Knuth, I'll say Knuth with a silent K, I don't quite know how to pronounce it, in 1984, and the basic idea is that it's a combination of a a document formatting language, for example, Markdown, Markdown cells, interspersed with a programming language. Now, you may use Python, but it could be any other higher level language especially. He coined that term and suggested that literate programming is inherently bilingual because it combines two different genres of writing within the same computer program. the document formatting, and the programming language. And he proposed it as a new motto for software development, actually. It's almost like a paradigm he was proposing for software engineering, building on the foundations of what was then structured programming. In Neuth's work, he actually was using tech, which is like LaTeX, and he was using Pascal as his compiled language in his web system. Yeah, so he developed what I would say a new literary form or a proposal for a new literary form of literate programming where the formal computer code is not primary as it was in structured programming but instead becomes secondary because the code and the prose are interspersed with each other. I got interested in when literate programming became a thing so I used Google Books' Ngram viewer to search for the term and it shows the frequency of the published books which increases from the mid-1980s onwards. I'm using Plot9, by the way, which is a Python port of the ggplot2 plotting library in R. And then this is a table depicting that. Those do move, those little nanoplots. This is using a package called great tables in Python which is a port of the GT package in R. On my screen, they actually animate when you hover over those little nano plots. For some reason, it doesn't on that version. And then I couldn't help. I made this presentation in Quarto. Does anyone use Quarto? Only a small number of people. I would check it out, because it's quite cool. You can embed live maps inside the talk. So the talk and the web page is all rendered in Quarto. And I just wanted to have fun making a map with iPie Leaflet to show Cambridge in the UK, where that they made my BBC computer. Back to full screen, please. Okay. So, what was funny about looking at Nooth's work and what he said back in the 80s, so it's like 40, over 40 years ago now, was that he designed his web system as a tool for systems programmers, not for high school students or hobbyists, because the programmer needs to be comfortable dealing with multiple languages simultaneously. It's really interesting to think that 40 years later, this literate programming language is actually most commonly adopted, not by systems programmers, but instead by scientific programmers and data scientists, who tend to use high-level scripting languages like R, Python, Julia. Also, the tools of literate programming, like Jupyter Notebooks, are considered particularly good for teaching and learning programming, especially in cloud environments where the language interpreter is included. But they're mostly used as tools designed to support the practice of scientific reproducibility. And so Cloiva, there's a few research studies on the use of Jupyter Notebooks, and they say that they're designed to be readily read and replicated. And so they're kind of part of this open science trend where you try to make your results replicable by interspersing the prose text with the code. And the Jupiter notebook was introduced firstly as an IPython notebook in 2011 as this open science reproducible form. But what's interesting, and that's where the lamentation comes in, a lament is a poetic discourse expressing sadness or grieving over losing something. And that's part of the reason I thought of that term, because a few years later, Pimentel et al. analysed almost one and a half million notebooks on GitHub and found that out of the almost a million they attempted to execute, only 24% could be executed without errors, and only 4% produced the same result. and it's a real shocking indictment actually of the project of open science and the reproducible nature of Jupyter Notebooks and I very much like this evidence-based way of evaluating the success of data science and in the paper which you can read I've got the references at the end they make a number of improvements for improving Jupyter Notebooks. The criticisms are that there are hidden states inside the notebooks, unexpected execution order with fragmented code, and bad practices of naming, versioning, testing, and modularizing code. And also the notebook format does not encode its library dependencies like the Python dependencies, for example, and the versions which make it hard or impossible to reproduce the notebooks. So ironically, as many engineers point out, they tend to be associated with poor software engineering practices that actually undermine their reproducibility. And there's several solutions that have been put out there. I'm not going to go into a lot of detail on these because you can follow them up yourself. I've used Jupytext, which allows you to sync a Python script with the notebook. Incidentally, that makes it easier to use these language agents with notebooks because they're JSON files that are a bit tricky to parse, but Jupytext helps with those. Also, Merimo notebooks are reactive notebooks explicitly designed to address the problems with Jupyter. I've not used them, so I can't really comment on those. The usual practice that Nelson endorses, I would say, is the case that you can use Jupyter Notebooks for development work, but then you would refactor those notebooks into modularized Python scripts with their accompanying tests. I think that's quite common in industry settings. I've kind of veered between using Jupyter and Quarto docs. Quarto actually comes out of the R world, which most good things in data science come out of R originally, and then they get ported into Python. And Quarto is a good example of that. And they're essentially markdown format files where you do the same thing. You intersperse text with code, but you actually create the cells yourself using three backticks for each cell. And they all start off with a YAML at the top, which sets some of the formats of the files. And the advantages of those is that they're text files, so they're easy to version control. The thing I want to just finish on is this broader idea. I don't really mind how people document their code. There's all different ways of doing it. But it's more to think about Noos' ideas about computer programs as literature. He makes several arguments that really shocked me when I read them for the first time. He said, I believe the time is ripe for significantly better documentation of programs and that we can best achieve this by considering programs to be works of literature. Let us change our traditional attitude to the construction of programs. Instead of imagining our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do. And this one I was very struck by, The practitioner of literate programming can be regarded as an essayist whose main concern is with exposition and excellence of style. Such an author with thesaurus in hand chooses the names of variables carefully and explains what each variable means. He or she strives for a program that is comprehensible because its concepts have been introduced in an order that is best for human understanding. using a mixture of formal and informal methods that reinforce each other. It sounds so old-fashioned now, but also unusual for a computer scientist to write such a thing. I never heard a computer scientist say this when I was studying computer science. And also that a systems programmer would write in this way. It's very unusual. And it's very influential, his work on algorithms. He has a book as well called The Art of Programming, which is also very influential. So just to finish, I thought I would run with Knuth's ideas and get some inspiration about writing literate programming from George Orwell, one of the greatest writers of the 20th century. In his 1945 essay, Politics and the English Language, George Orwell presented rules for how to write well. While he was writing 80 years ago about political language, which his rules are relevant to writing technical documentation now. For example, good writing should be active, precise, and simple. Write actively rather than passively. Be precise rather than vague. And use simple words and verbs, especially, rather than complex, longer words or phrases. Especially these very long noun words, which can be very confusing. if we can drop a word from a sentence without losing the meaning, we should drop it. Orwell called the use of language a habit, and he made a close relationship between language and thought. By learning to write well, we can develop good ways of thinking. By writing clearly, we think clearly, and thereby communicate our thoughts more clearly to others. For Orwell, the thoughts are in the words and our thinking is displayed in our writing so we'd better choose our words wisely in writing well we can feel more like a human being rather than a machine which is very relevant to our age of AI so called so I'll finish there, I won't talk any more about AI because I'm bored of hearing about AI so I'll just say I hope to have impressed upon you the importance of writing docs and the pleasure that can come from writing docs, such that you might also be in for praise of documentation. When we build data systems or we do data analysis, we are writing text. And when we code, arguably, we're also communicating. We're writing, whether to a human or to an AI agent. And when we write technical documentation, of course, we're also writing. The code we write now will be read by our future selves, future readers of our reports, and future users of our software systems and tools. So in writing our code today, we're writing the legacy code of tomorrow. And documentation is the user guide for the data infrastructures of that future. While writing documentation can be technically fun, the solution to the lack of documentation is not so much technical as cultural it's a matter of taking time to care and pay attention so take care of software infrastructures and write good documentation please, thank you

Speaker 1 [24:38]

Thank you so much.

Speaker 2 [24:39]

That's all right.

Speaker 1 [24:41]

So we have a few questions and you left us plenty of time to answer them. So thank you. Okay

Speaker 2 [24:46]

Okay, great. Long questions. Oh.

Speaker 1 [24:50]

Nowadays, I basically instruct load code to document everything twice, user documentation and developer documentation. It is also instructed to keep the documentation up to date. This has worked out quite well so far. Do you have any practical tips I could still benefit from?

Speaker 2 [25:06]

Yeah, it's a really good question. I think what's happening at the moment mostly is that developers are outsourcing their thinking to these agents most of the time. There are exceptions, both in terms of code and documentation. I think there's a bigger risk that engineers outsource the docs to the agents because they don't like writing docs. And they don't have enjoyment doing that. it's quite difficult to write, just as it's difficult to write code. It's a really tricky one, because I think we do... My only practical advice would be do check what it's writing. And it's an obvious one, but I know a lot of people aren't, and I think it's very easy now to just let it do its thing and not check. So just as ideally we would check the code that Claude Code or any other agent is writing, we would also check the docs.

Speaker 1 [26:13]

Next one. We love the layout of your presentation. Could you tell us which tool you used for creating it? Obviously, it wasn't PP.

Speaker 2 [26:22]

PP. Oh, PowerPoint. Yeah, yeah, yeah. Yeah, so I used Quarto. Sorry, I did say I was going to show a link to that. So, Quarto, Q-U-A-R-T-O. If you search for Quarto online, it's a scientific publishing system. And I used that to create the talk, also my website. I would say it's one of the easiest tools you can create a whole portfolio with, actually, and deploy it through the web if you're not a web developer. So Quarto, yeah, I use that one.

Speaker 1 [27:02]

What things would you suggest to commit in code and what things in external systems like wiki user stories documents?

Speaker 2 [27:09]

Yeah, that's a really good question. I didn't have time at all. I only had 20 minutes, so there was a lot I had to cut from my talk. There is a longer version of my talk at my website, which is carecodeconnect.io, carecodeconnect.io, and there's a longer version on there. Also, you can just come up to me and speak to me like a human being, and I'll tell you this stuff again. I'll take my lunch in a minute. It's a really good question. The big risk is, like, you repeat yourself in different places, and then when the system changes, the documentation doesn't update. So you get out-of-date documentation. That's even the case, ironically, with this. I did docs for the talk, and then I changed the talk, and I didn't change the docs because I ran out of time. The best advice I can give is what they do in Rust, where they document as close to the actual code as possible. That's kind of what you're supposed to do in Jupyter Notebooks, but it's often the case that you might also need docs elsewhere, like in Wikis or internal notes. if you can try and have it in one place and as close to the code as possible.

Speaker 1 [28:27]

Okay, one question you would probably need 20 minutes to answer, but you have two minutes.

Speaker 2 [28:31]

Oh, God.

Speaker 1 [28:31]

What makes documentation actually good? By good, I mean useful readers of the code, including forgetful future users.

Speaker 2 [28:38]

future? Oh, that's a brilliant question. Yeah, thank you for that. I said good because, you know, I think we do have an aesthetic sense, however technical we are in our work. I think even if we're writing code, I think we even will have a sense of the aesthetics of that code and what we like and what we don't like. I think the big advantage of an engineering mindset is that engineers tend to be very concise, sometimes too concise, and so concise that you're not sure what the meaning is of what's being said. The big problem of the arts, humanities, and social sciences, we go the other way. We talk and talk and talk and talk. I could talk for hours about what's good writing. So I think, like Orwell said, I think concise is really good be clear that the meaning is in the words themselves the meaning is not in your head so when you're writing don't expect people to mind read the meaning is in what you leave behind whether it's the code or the text so make sure the meaning is in the text yeah

Speaker 1 [29:48]

There are a few other questions, but maybe you can take them now, or maybe this card, whatever.

Speaker 2 [29:52]

whatever I'm not on discord so that that's

Speaker 1 [29:54]

That's on you. So, yeah, let's thank our speakers. That was wonderful. In 10 minutes, we have Joshua Gerner, the seventh-century lesson from the Pidentica AI system.