Why Exceptions Are Just Sophisticated Gotos - and How to Move Beyond

Speaker 1 [00:08]

Yeah, so my name is Florian. I'm head of data science at InnoVEX. I have a mathematical background. Yeah, did some projects in data science, more like classical data science. Always a huge fan of the PyData stack and also contributed a few open source packages. Since I mentioned my employer InnoVEX, we're an IT project center. Yeah, focusing on innovation excellence, you find me and like 19 of my colleagues also here at the conference. Talk to them. And we have a total of six talks and one special workshop for kids. So check them out. And if you're interested, then talk to me or talk to us after the talk. So regarding the agenda, I'm going to tell you a little bit about the history of GoTo, which I find really fascinating, how it all evolved away there. And then we will go to exceptions, what exceptions are, why they exist, why we make use of them, and why they were some cool new feature. And also the evolution towards kind of successor of exceptions towards result types. And then, of course, we want to see how we can use result types in Python. And in the end, I'm going to wrap everything up with a small little conclusion. So let's get started with the history of goto. So first of all, what is a goto? It's just kind of jump to a certain label. So if you have your control flow in your program, you're just transferring the control flow to another line. So how does that look in practice? So this is some really old, like, I think version 1 from 57 Fortran code where we would have, like, where we can have labels in the first column. And what we see here is just a really simple loop where we square all the integers from to 100 or 99 even and yeah what you see is we just here in line 5 we jump to 2 until a certain condition is met and then we jump to line 6 and jump out of the loop so it's like really really old cool code but just to get you an idea yeah how goto was used back then but even nowadays in more like modern code in a way, I mean still a lot of stuff is programmed in C, here we have the classical example of two nested loops and if a certain condition is met we jump out of the loop, out of both loops by going to a labeled, yeah to some label end of nested loop and we jump out of it and we're going to come back to this example quite many times. So, as I mentioned, Fortran was the first language to introduce goto and the if statement in 1957, so quite some time ago, if you think that people started to program back then. And typical applications of a goto are, of course, to skip code, to loop over code, as we've just seen, to break out of loops this is something we still do today in many languages and also for error handling so instance if you look at the Linux source code even in newer versions like 6.12 I check that it's some crapping on go to there's more than 200,000 go to statements in the Linux kernel and this is not a sign of like bad programming it's just their style so you jump somewhere you clean up resources if something went wrong. So go to, even though we all learned in computer science class that you should not use it, it's still around and alive. And after Fortran it was also used in many other languages. I just only I'm showing the popular ones like ALGOL or some of you may know Pascal back from school or see if I just mentioned and yeah so as the picture already kind of indicates with go to it's really hard to to program and it also end up in and what we call spaghetti code I mean we still use this term quite often and this was actually coined in 1977 by a guy called yeah guy Lewis Steele and he had paper it says like hey macaroni is better than spaghetti and for him was macaroni structured programming and he showed that yeah he pointed out that go to is a really bad statement and you should not use it because it's hard to understand it's hard to follow the control flow it's like yeah you're jumping around a bit like the ends of spaghetti on your on your plate and it's extremely hard to debug and he pointed out that with structured programming which was the cool thing at that time so in the in the 60s introduced with Algol 60 that you should use structured programming to improve the clarity the quality of your code and of course to lessen the development time so what is structured programming it's basically things that we use just as common code blocks like in almost any modern programming language it's just structured control flow like loops things like indentations and code blocks using functions to modularize your code but back then in the 60s this was like a new idea right and with every new idea you first of all have to prove that you can write any program now with structured program that you could do before using goto. And this is what then Böhm and Jacobini did in 66, that they really mathematically proved that every computable function, everything you can do with goto, anything computable, you can also do just by using structured programming, so not using a goto. And then some quite famous person, Dijkstra, said then in a paper he also pointed out that goto is completely harmful and now that we know that you don't really need it, you should completely abandon it. And Dijkstra, if you know him, also touring award winner maybe from the Dijkstra algorithm, this is the shortest path algorithm, so this was kind of like a huge voice at the time when he said like, hey, get rid of goto. And then someone else, also a Turing Award winner, Donald Knuth, pointed out that, well, you might be correct, but in some cases, GoTo is actually still useful, right? So, your kind of abstraction you're putting on top of GoTo is maybe a little bit leaky. And in some paper, he then showed that, yeah, if you eliminate it completely in certain cases, it makes your code harder to read. And it might even make your code less efficient. It was like no mathematical proof, but more like an empirical proof. And what's fascinating is that also in Java language that was, yeah, coming up way later, one got rid of goto. So there is a go-to statement, but it's not used in Java. But still we have a break statement. So just for the example I showed before, you can say break search. And with this, like, label on top of the outer loop, you still have a way to kind of jump out. And this kind of goes back to what Knut said, that in certain situations kind of jumping somewhere, it might help. And yeah, if you ask your question now, so why did we actually start with go to? I mean, who came up with the idea of go to? That's what the machine actually does. So in the end, everything loops and so on, if statements. It's just all a lot of go to statements in your CPU. The assembly language is just putting, like, more like, yeah, putting, giving names to some binary codes. And then we as humans, we start to build up abstraction after abstraction on top of this because we think an abstraction is better. But goto is kind of the natural way because this is what a CPU can do. And now we are moving further and further away. Also a little fun fact, in 2007 in a pep someone tried to introduce what we just saw in Java, so-called labeled break and continue statement in Python 3.1. And as always, there's a nice discussion in this pep around it. And in the end, Guido van Rossum said himself, like, no, there's not going to be a go-to or kind of go to which this break statement is with the label, there's not going to be something like this in Python. So the natural question comes up. We just saw the nice readable Java code. So how would you now solve this problem in Python? So you could use some kind of state, right? So here we have a nested loop again, and we are trying to find out the indices. So i and j, where the arrow equals some kind of target value. And normally, we would kind of break out here. But since there's no labeled break in Python, how can we do this? we can for instance say we use the state within found i is no longer none and then we break two loops but does this really look readable not really and there is a huge huge stack overflow discussion about how to solve this problem if you're interested this is just some examples how you can do it another one is to use the l statement so as a small question so how many of you know that there's an l statement in a for loop i would say it's like 50 50 60. wow so this is also a way to do it i think quite tricky so the l statement is only executed if the for loop did not encounter a break. So whenever it's not found, you continue also the outer loop. And if it was found, you break the outer loop. So please don't tell me you find this readable, because it's definitely not. It's a way to solve the problem, but it's kind of nice to see it once, but then don't use this in any code, right? Then there's the thing which already gets me like to the bridge to exceptions. You could, of course, also just say, yes, I found it. I raised an exception. I catch the exception. And with this, you could also do it. So, of course, don't do it. But it's a way. And it shows you you are jumping there. And what's the Pythonic approach, actually? So you would use a function. And this is what even what Guido van Rossum, he kind of defines what Pythonic is, right? So as the inventor of Python. And he said, like, well, just put it in a function, call it search, return it, and then you have your kind of labeled break. And this is good to keep in mind. So whenever you're kind of feeling I would like to jump there, you are not using enough functions. Yeah. And this already gets me to the exceptions. And we have seen already some similarities. But just a small reminder, recap, what's an exception? It's some kind of event that breaks the normal program flow. Typically, it's some kind of error, special case. Yeah, just to get you an idea, it's not a really good example. But let's say you want to read a positive number from a user. And what if the user supplies a negative number? You check for it. You raise a value error. And then you somehow handle it. some error or maybe ask the user again to correct his or her mistake, yeah, that's an exception. I think we all know exceptions. So, why are exceptions useful and why were they invented? So, now we can do something like in this example. So, let's say we start here, we read the data, this is just some dummy code, we read data, we process the data, read data calls process data, we square it, we display the results and then up here in certain cases an error is raised but we don't want to handle the error directly here, right? We want to somehow handle the error here and what happens is that we built a call stack, right? So our module called read data called process data called display results here the the exception is raised it climbs up the stack and the handler can start reacting to it and this is basically the the idea behind exceptions and why are we doing this because we can now separate our like normal logic our business logic from the error handling, right? Because most cases we assume everything's gonna be alright. And so this was the idea we want to somehow separate it and without jumping around. Yeah and this was the idea of exceptions. The first like popular language that used exceptions, there were others before, but Ada might be a name you know, that this was a programming language in the 80s. Then it was also introduced as a huge feature in C++ besides the classes, that C++ has exceptions while C has none. It's of course also a huge feature in Java, there they even improved the possibilities even more of exceptions. You can have enforced exceptions that you say my method throws an exception and someone has to, so you can enforce someone is really handling the exceptions and so on. So a lot of, yeah, of experience with exceptions went into this. And of course we also know exceptions in Python. So what are the problems with exceptions? So some we have already seen. We already have, we have again some kind of invisible control flow, right? So we saw it in the example. So you are jumping somewhere. It's not really clear. It might be not clear when you're reading the code. So what happens now in this case? Because you have to look somewhere completely different. It might for production code it might be quite surprising if you use a lot of external dependencies and maybe you haven't checked the documentation and at a certain point there's an exception raised you might have never seen before and then your program terminates. So you have to be really hard figuring out what kind of exceptions can be thrown if you're just using a library. You have, again, the debugging complexity. Since exceptions can be thrown at various points, then concurrency and parallelism is always a hard thing to do, right? But it gets even harder if you're dealing with a lot of exceptions in async code. It's kind of hard to figure out where the exception is then really caught, at what point parallelism, if you're dealing with threads, processes, one thread is dying because of an uncaught exception and then you get in a dead log and then how to figure out what was the real reason you can also not kind of transfer exception from one process or one thread to another and so this makes it even more complicated then that's also performance so if you are looking for exceptional performance then And you should not use exceptions. For instance, in C++, if you're looking at the Google C++ guidelines and best practices, they kind of say, well, don't even use exceptions, because, yeah, under the hood, there's a lot going on of bookkeeping where exceptions can be thrown, so I'm definitely no compiler expert. But it's actually not that easy to implement something like this in a programming language. So to sum this up a little bit, we have seen that a goto is just you jump to some other line and exceptions, they always go up the stack. But you're still kind of, yeah, changing the control flow because you go maybe to a completely different point in your code. So people are, of course, aware of this, of these problems. And so there was a certain kind of evolution, I would say, more towards result types. So what is a result type? It's a really, really simple idea. It's the idea that you have a kind of box. And if your function returns a value or some kind of error, then you put either the value or the error state in a box. And then you handle the box around to the person calling you. And this person has to open the box. And while opening the box, you can enforce if there's an error in it, then you can also enforce handling the error or someone opens the box and there's the value in it and everything is fine. Just as a little side note, because this then gets really complex and I'm no expert in this, but this is then what's called an algebraic data type. If you have some certain kind of mathematical laws on it, it's called a monad. And this is an important aspect of functional programming. And if you read it up, then it's kind of, whoa, a lot of math. But in the end, it's just a blank. It is not so difficult. So I leave out all this. But just the connection, if you hear about monads, this is kind of the idea. So just before we go to Python, how do other programming languages solve this, right? It's always good to look beyond the own domain. What do other programming languages do? So let's say go. We have some safe divide function where we want to, in case the user says divide by zero, kind of raise an error. And what Go does is that in those cases, so Go has no exceptions, you return a table. And this table is the actual value and the error. So in case user provided zero, we return some dummy value zero and an error. In all other cases, we just divide x by zero and, yeah, return a kind of none, a nil. And then we check if there was an error. And we can access, we handle the error or we handle the result. So this would be the way to do it in Go or like the easiest way. What Go doesn't do is that there's like no unpacking. So Go does not really enforce that you handle the error. Of course, you should. Like newer programming languages, Rust, almost like a sister language to Python, right? A lot of things are written in Rust nowadays. And how does it look like here, right? We also have a safe divide. And this time we have a result type that is either the float or some kind of error string. So internally it's kind of an enum. And we have two variants of it. So the error and okay. And if there was an error, we kind of this is where we say here it's a packed error. And it's a success. This is okay with the actual float. And now something interesting happens here. We match now the result of safe divide. And we match the two variants, okay and error. And by matching this, this is the first time we can really access, this is the point where we kind of unpackage, like Schrödinger's cat, and take the result and can make use of the result or handle the error. And this is something that also the compiler can easily check and tell you, well, you forgot to handle the error case and this makes this makes rust so robust against those kinds of errors so you might think so rust really new 2015 right it's like really really young and not even a teenager right in in programming language so or just becoming one it's really new language but actually this has been here for many many years like in a quite old language I had to learn at university when when doing some computer science classes who knows Haskell or have heard of oh yeah so here we also have some kind of type constructor with a string and a double and then there's some some data constructor where we say okay we match so I agree the syntax is a little bit strange at first in Haskell so we match the zero and then we say left division by zero or right X divided by Y and here it's actually the same as in Rust instead of match we say hey we have two cases left error or right value and now we have unpacked packaged it and can actually make use of the error or the value so basically it's similar it's really similar to a rust and rust borrowed from Haskell I assume at that point so yeah what at that point have we gotten from result types right so we have no more hidden control flow we need to directly handle everything. It's extremely explicit. And, I mean, this is what we always say in Python, right? Explicit over implicit. In, like, fourth-generation languages like Rust and Haskell, we can even enforce that someone is handling successes and failure. The code becomes also much more easier to reason about and so to show this in a meme because it's so cool and before we were like well no failure path only looking at the success path and now if you use result types you can basically start over engineering in Python and this already says it in a way that you should also be careful with this, right? Yeah. So, how can we make use of this now in Python, right? There are many libraries. So, I found four. Surely there are some more. There's the returns result O slash expression library. Important is always to check if they are still maintained. So, this kind of of, yeah, eliminates already, too. And returns really is the one I found that is most like Haskell and functional programming inspired. It has a lot of features. It's quite Pythonic. And this is why I opted for my small little experiments with, yeah, I opted for returns. It's also expression which is more like OCaml, F sharp inspired, more simplistic. But let's look how our example would look like in Python. So it's quite similar to Rust and Haskell. So we have a result and some two variants, failure and success. So we have a result of a float and a string, the result type. And again, we do it as usual. We check for zero. We return a failure with a string and a success. And this is just to check. So as expected, this function returns no longer just a value, but a kind of packaged value or result. in python we have a lot of exceptions right and if you are now using something like this you might wonder okay how can i integrate code which does not make use of result types and therefore returns as this safe decorator which basically just catches all exceptions and puts them as yeah as exceptions in this error state. And with this, you can integrate also other code. How can we now unpackage this? So, this is also really Rust-like. And in newer Python versions, I think it was the 3.10, we have this match case statement. And the maintainers of returns thought okay why not make use of this so you can really say match the result of simple division and as a case success value and then you can access the actual value and the error so it's it's not enforced by the Python interpreter but you can make this and then it's quite analog to Haskell to rust and so on and And with this, we already would have everything we need, but in the kind of a way we now do a full circle, right? So, instead of having exceptions where we jump somewhere, we now have, we're now returning always an error. And it's almost like C again, because now every function needs to handle the error. And it's a lot of boilerplate code, if you would assume you would program like this. And in every example, you're like, K is in failure. I cannot handle the error. I return the error again, package it, and so on. So what is the solution to this? It's called railway-oriented programming. And the idea really is a railway and switches. So you always have a success and a failure path, a happy or error path. And you think in those like back when you were a small child and you would put together those railroad components and with this simple basic blocks you now start designing your program like for instance yeah you have some kind of function that either goes right or produces some kind of failure. And then you could say, okay, I apply an operation only in case there wasn't failure. And I might recover from this failure. Then we are here. Or I apply a function on my actual value. And this function might also fail. So, there's another switch to the error path. And with this, you start, yeah, composing everything. So, there's basically only four different components. And one is map for success path. If you apply something that cannot go wrong, like adding one to a number, so it's just success two, Then there's alt and we are just putting an exclamation mark after the error. So, this is the easiest part. The other things you have seen is so-called bind and lash. Bind only works on the value if there wasn't success. So, we call the divide function, which might fail or work. If you divide by two, that's fine. If you not, then it's zero. zero it's just completely ignored and so that's the magic behind it if you provide a failure because this that's then the logic would bind us for you and it's analog to kind of lash right where you could recover or not recover so I'm just checking if there's an R in error there is one or not in no and success It's also like a no-op. Lash is a no-op on success, so nothing happens. So this function is never even called. And yeah, with those four components, you could do now method chaining and do bind here, lash here, and alt here, and so on, and map here. And you could already design everything you need way more concise without a lot of boilerplate code. Returns has a lot of additional functions like pipe where, yeah, since the time is quite short, just I don't want to go into details. Let's say you have your functions here and you just want to kind of pipe through a value and use the result type again as an argument to the next one. And then you can kind of compose now those different functions really easily and pipe takes care of the fact if the result was an error or was not an error. And with this, you can design a program without using exceptions also for more complex things. because you can then at a completely different point handle your error as you would have done before with an exception. So, to wrap it up or the last few slides, what else can be done with containers? So, we haven't talked yet about IO and futures, also a huge topic. We have not talked about our beloved with statements or context managers also those need to be thought of in a different way and therefore that's the managed we have way more ways to do compositions with a pipe statement then also this is more like the functional part of it pipe works really well if you have only array functions so functions taking only one parameter but what if not then you need to do curry and curry partial applications and so on and another nice thing that returns provides among many many more things is if you want to use recursion it's always a little bit tricky in Python right if it's too deep so more than 1,000 calls I think then you have and I think yeah it's a recursion error or something like this. And with trampolines you can do a tail call optimization so that you don't have this problem anymore. So to conclude, I think the most important part is even if you're not using result types, also consider the failure path if you wanted to program robust code, right? It's not just a happy path. Be aware of the fact there might be an exception and there might be an error. We have seen algebraic data types like result, how they work conceptually. We have seen railway-oriented programming as a concept that replaces traditional exception handling, for instance, in Rust. And, yeah, we have seen that fourth generation languages like Rust and And Haskell can even enforce this on result types. So now the big question, should you apply this in your next Python project? So this is just my take. I think it's fascinating, and I was playing around a little bit with it. But since Python is not inherently functional, right? It has a lot of, like, dictionaries, unmutable, and it's kind of sometimes a little bit tricky with all the state and so on. So it's not really a functional programming language, but you're kind of wrapping now something completely functional over it. You have to be really, really careful. And it might make your code less readable, less idiomatic. And yeah, it might be the right tool for certain use cases. But if you are, yeah, you have to be careful if your team thinks functional. I hope it is functional. But it should be thinking functional. And only in those rare cases I would make use of this. So please don't go out and tell that I told you to use this now in every project. But it's nice to play around with it, definitely. And with this, I want to say thank you.