Write your Own Decorators

Speaker 1 [00:03]

Thank you very much. Good afternoon, everybody. First day of PyCon D 2019, and probably the second tutorial of the day, and here's the time slot. My name is Mike Müller, and I will be the presenter of this tutorial. I would like to introduce myself a little bit, a few words. So I've been a Python user since 1999. I started with version 1.5, and I lived through all these Python versions up recently. I started doing Python training in 2004, and that's actually what I spent most of my time on. I teach Python and I organize Python courses, and therefore also I'm teaching this tutorial. Maybe a few questions about your background. Please raise your hand so they have a basic understanding of how much Python experience you have. Who prongs programs with python less than one year please raise your hand very few people between one and two years also a few between two and five okay that seems like majority more than five years also a few people so we have a good mix of of people but this tutorial is designed to give you an understanding of decorators you need to understand how python functions work and basic Python knowledge, more or less intermediate Python, and we want to dive into decorators, and you should be able to understand if you know how function works, and if you know that everything in Python is an object, you should be all set. We don't need to know much about object programming in Python, though that's not the real topic of this course. Please download the zip file. I will upload this again. In there, you'll find a PDF with a documentation. I have it here just to follow along. And I will work in a Jupyter Notebook. If you know what it is, you're welcome to use one, but you don't have to. You can use any tool you prefer. So you're totally free to use a tool. It's not about the tool here. It's about decorators. And I would like to show you how to write your own decorator. Using decorators, obviously, it's very simple. So we will see something, how this works. So that's the download URL. So I would recommend to you to have a new directory. That's what I did. I created a new directory, and I'm working with environments. I have an environment here, and I use JupyterLab here to work with. If you know how Jupyter works, you're welcome to use Jupyter. If you don't know, I have to say something about the port. Somehow my setup is mixed up, so I have to come up with a port number. Otherwise, it's not going to work. So, JupyterLab is just a tool that helps you to work with Python. And since we have the PyData here included, probably a lot of you will know how this works. And I will make a new notebook, a Python 3 notebook. I would like to rename it first and call it decorators. So, I'm pretty much following if you download, you will also find the PDF. I'm pretty much following the PDF and I don't want to be too fast, that's why I pretty much retype everything here. So, when you download something, you should see that's a notebook, you should see the PDF and that should be a directory decorators with just a few example scripts I gave you because a big part of the training of this tutorial will be about examples. So what you can do with decorators on top of how to write your own decorator. So, first I would like to give you a motivation. Why do we need decorators? So actually, the principle of how a decorator works is much older than the decorator syntax. The decorator syntax is this add sign you might have seen. So this is rather new compared to the principle. and i show you an example why a decorator can be useful so let's assume you have a class c so in the documentation you still inherit from object which you have to do when you work with python hopefully python 2 hopefully nobody of you has to work with python 2 so if you work with python 2 you need inherit from object in python 3 you don't have to do this anymore and you would like to have what's called the static method so static methods in python are nothing i'm actually not needed you can always write a standalone function but sometimes you want to have like a standalone function still be inside the class and this would be just a normal function here which does something very useful and you see there's no self when you write a normal method you need to put a self and to make this one a static method you have to do something like this that's the way you did it before you had the syntax now you can make an instance of with class and now you can call this function on the instance and you can call this one on the class with the same syntax this wouldn't work with a normal class with a self and it only works because you put static method there. So obviously it's very important to know that func is a static method not a normal method. But you hide this very important fact at the end because you need to have this function object here defined to be able to actually use it. And imagine if this method is a little longer not only one line but a few lines then this might go out of the screen you don't see this important fact and you think the person forgot to self yeah so and this is was one of the motivations why we have this add decorator so you can achieve the same thing i just copied a cell so i can get rid of this whole line here i don't need this line anymore and now the decorator this one is not exactly the same so the cell i copy the cell again there's two cells both cells i see here do exactly the same and that's the main gist of what the decorator is doing so the decorator takes your function and returns a modified version of your function or method in this case so function method are technically exactly the same only that methods live inside the class so if you say functional method it should be the same thing and this is what the decorator is doing so this decorator syntax with the add sign is doing exactly the same and you can see I can use the same static method the static method is exactly the same thing only it turns a different color because JupyterLab recognized it as a special syntax as a decorator but static method is just a function that takes a function and returns a new function new version of the function with the same name so if you understand this one you know already how a decorator works and you can always use decorator this way or this way this way most of the time you use the add sign but sometimes you also use this one because the add sign you have to actually literally type the add sign in your code here since you have a function you could use different method because static method is a function and the function is an object you can put this static method into a list or dictionary or anything else and use it programmatically if you like. That would be sometimes useful. This is the motivation for writing a decorator, but now we want to write our own decorator. How to do this? We're going to look at how to write your own decorator. We'll do it naively first. So we'll write a function that's called hello that takes a function and prints hello so you do something like this so and now I have another function I'd say I use hello as a decorator and I use to find a very complicated function that adds x plus y return x plus y so we will redefine this function multiple times so it's not about the function but it's about the decorator and as you can see here the minute I define this function the decorator is executed so a decorator is executed at some people say compile time or say import time so if you put this in a script and you import a script if this is on the left side of the script it will be executed immediately of course what happens so this is equivalent to equivalent to what so if you go with this one so you say at equals hello at that's what happens so both cells are equivalent and this hello will be printed out the minute I define this function there are some disadvantages of this approach what I do here because if I try to call my function it doesn't work and it gives me this very common error message non-type object is not callable, because what did I do? The hello doesn't return anything, and if I don't write return, it essentially puts return none there. So this decorator, while I can apply it, doesn't do anything useful. So you can write a decorator like this, but it's typically not what you want. You have to improve your decorator. So, let's write a decorator that looks more like something that you would use. So for this one, we have to make it a bit more complicated. So I use hello, and I get my function. And most of the time in the decorator, or the most common case, is you define a new function inside a function. So in Python, there's no limit how deep you nest this. So you can write a function inside a function inside a function inside a function. As long as you understand what you're doing, there's no problem with it. And then you have this internal function. The name of this function doesn't matter. You can call it whatever you like. There are different approaches. We will see some of these naming approaches. I have this one. And then I use args and keyword args. Who does not know what the star and double star means? Please raise your hand if you don't know what it means. A few people? So let's explore this very quickly. so if you write a general function that has an args argument and I just print it now I can call this function without an argument and I do get an empty tuple I can call this function with one argument and I get a tuple with one element you see this comma is necessary for one element tuple because just a normal parenthesis which just means grouping and the one would be grouped as itself. So you would need this comma there. And you can call it this two arguments and you get the tuple this two and this three, this four, this five, this six. So this is star. The star is kind of harvesting all the arguments you supply or positional arguments you supply. If you don't supply any then there will be an empty tuple otherwise it will be put in the tuple so i can now extend this one and use a double star so the name kw args is not required but it's a pretty strong convention i would say people do this all the time and now i have a different function and this function if i call it gives me back an empty dictionary and if it called this one keyword argument it gives me back a dictionary with one element or one item I would say and if I put this two call this two I get two back so you can combine this and this is what you can do here so I say keyword args and I just print both of them yeah and if i call the function without anything i do get back an empty tuple and an empty dictionary and if i call this with positional arguments and keyword arguments something like this then i get back a tuple in this one so i can write a very very generic function that's what we want here because you see our attacker will take a function and return a function as we can we'll see in a minute and this should be very generic because we don't care actually what the original function looks like what the signatures what kind of arguments it takes we don't care we want to write it for any kind of these functions and that's where we need it so this is one way of using this star double star and the other one is when you actually call a function. When you call a function, it does exactly the opposite. So, print. If I print something, I don't know if it's A, but I have something to print. So, I print something and it prints it. But if you look at print, this gives me a help, then you see print takes optional arguments and these can be keyword arguments. Or, value takes value, value, value and I can put in multiple arguments so I can say print 1,2,3 that would work if I have a tuple with 1,2,3 I can print my tuple which prints the tuple as it represented but now I can use my star again and here it takes the tuple and unpacks the tuple and this would be equivalent to writing print t 0 t 0 I tend to change keyboards sometimes on the PC keyboards are a bit different so this would be the same thing, that's actually what it's doing so it's not very convenient to write this one, but you see that what happens, so instead of writing something like this, I can use a star and unpack it, the same happens for the double star and unpacks it and would use a dictionary for this okay once we know all this how this works now we do have the opportunity to redefine our hello so we write this wrapper function and this wrapper function is adding functionality so with a decorator you can add functionality to function you cannot really take away functionality because the function called those original functions kind of atomic from this perspective and you cannot take any functionality away but you can add something and you can say print hello here and then I can return what the original function actually produced. This is very common. You don't have to do something like this. Sometimes you have use cases where you don't do this but in most cases you want this and now we do something you might have seen before you return the function so the function hello produces a new function on the fly so now i'm so far away from this one i just copy the cell down here just so you can see it so and now we can use our function hello and decorate our function add which i need to redefine if you do it this way here x y yeah and we return x plus y so now nothing happens at definition time at import time if you want so there's no print of hello but now when i say add i do get my result three and four makes seven and at the same time i have additional functionality in this case it prints hello why it's like this if you look at add then you see actually add is not add anymore but is this our rep function so this function has been produced the minute i defined add the add hello the add sign hello call this one and produce this function again this cell is equivalent to saying add if i define add without the decorator so i promised you that we implement this function multiple times and i say add equals hello add that what happens so this is very important that you understand this one that's what happens and you see hello returns a new function which replaces the original function so you have this name here at actually normally when you define a function the function name is the same as the string you used here but here now name is different because you didn't do it properly yet and now the name is wrapped and as you can see here that's wrapped that lives inside the other function it has something with a closure and so but it doesn't really matter now it's a function that has to be produced at run time so in Python everything is an object including functions that's why there is no problem whatsoever to return a new function from a function and that's the main concept of decorators and that's producing a function from inside a function worked before the syntax with an add but you had to do something like this so you could modify a function at runtime with this method but the new one is using this add decorator ok that's pretty much what you need to know about decorators any questions so far okay now the rest everything else we cover now it will be details but some details can be interesting and also we're going to cover a few best practices actually how to do it in practice and i'll show you some some examples good so there's two things one of them is the name is not really nice here so the name you're using is not really nice we will fix this later on and also I hardwired the hello so if I would need to write a goodbye decorator I would need to implement pretty much the same thing the only thing I need to change is a string so it functionally is nearly the same there's only one small difference whenever you have this case that you do nearly the same and there's a small difference hopefully you can parameterize it so use this string as a parameter to make this more generic and you can do this in Python by adding yet another layer so here I add another layer of functions here and I make it more general so I say now this is a function say and I say a text and now i take exactly this and i need to just change it now there are different ways of doing this very often um yeah you use underscore so that's one way of naming these things because these internal names you won't see you don't want to see them again they're just helper names so you have this strange underscore name here on this wrapped i call to underscore say this is one pattern to do this and now you see here i don't return i don't print hello text or the greeting if you like but text is more general and of course now i cannot return wrapped here but i return say and i do another return underscore say so this is a common pattern if you want to parameterize your decorator, looks a bit scary but actually it's pretty easy, so I have three deaths and three returns so in most cases you want to match it you can have functions the inner function could work without a return but most of the time you want to have original functionality, that's why you call func and return the value though therefore you have three deaths and three returns, very easy and now I can redefine my function say hello and as you can see I can reimplement my function you may minimize this space and we return x plus y and now i can call my function it still works and it prints hello and i didn't hardwire hello inside the decorator but i can supply it as a parameter from outside now we have the three level decorator which allows me to parameterize it with in this case a text i print out and of course you could redefine this function, that's what I'm doing now and use a different string shouldn't we come to a surprise so it's a different string and then you can say goodbye and then if I add something with my function now it says goodbye not hello in fact you can stack decorators you might have seen somewhere so there's no technical limit as far as i know how deep you can stack them and then if i have my add function now it still works but now it says hello goodbye so now i add two different features to my function it prints prints two different strings which i can supply as a and parameter. So, this is pretty common if you do web programming. A lot of those web frameworks offer something like permissions or something business decorators. Question? Yeah. Yeah. So, that is an interesting thing. So, let's do this for this one first. So, this is exactly the same. So, if you take this here and you want to have the same let's do this with the one for CERN now. You define the add first and then the say later. So you have this one. And then you go with the same procedure. Then you say add equals. This is the first call of say, which does the first return, the outer return. And then I call this again and I give it the function add. yeah that's easy one now if you want to do it with two so i have to think myself to put it in the right place so then you here you have two these you call it say this hello first and then you call it this goodbye so you do it twice and that's not that easy uh see if i get it right i call say on this one so say returns something this goodbye and of course then you need to put parentheses in here hopefully this is correct yeah so it gets a bit hairy here and I say goodbye twice because I put goodbye twice in there which makes sense so you have to put the hello first in here and that's what it's doing so a lot of parentheses the decorator one is much easier to read but that's actually what happens so if it's difficult to read you might try to indent it a little bit because you can always something like this it's up to you if you prefer anything like this. Whatever way that looks good to you. I'm not sure if it looks better. It doesn't really help. But you could try to do something like this. This application is still useful because now I can have two versions. Here I use the same name, but I can use a different name. So if you want to have the original version and the decorated version at the same time, this would be a way to go. We will see if you do it right, we can still have the original version. The original version can still be there if you use what I call best practice. So, obviously, this is easier to understand than the second version. But that's what happens. So, you can parameterize them. You can chain them. So, you can put as many decorators as you like. And sometimes you can do this. Good. So when you talk about decorators, most of the time you talk about function decorators. So function decorators have been introduced, I think, 2.4 or something like this quite a while ago. But since 2.6, also not that new, it was also more than 10 years ago, we have what's called class decorators. Class decorators. The good news is class decorators are simpler than function decorators. you can use the same principle for a class so if you like you can define a function you gave the great name mark here and I have a class and the first thing you need to do you need to return the class otherwise it's going to work and this one doesn't do anything but now I can do something to my class I can read but I can also add something and now I can use my decorator and I can find a class which doesn't do anything so I put pass here otherwise I get a syntax error and now if I make instance of my class actually I don't have to make instance I can do it without the instance of course I add it to my class then you see now it has attribute new which it didn't have before so very easy so it's doing the same thing as if I would have my class B that doesn't do anything and then I say B capital S B equals mark B then B has this new that's what happens exactly the same principle so mark is just a help of function which takes a class and returns a class and you're free to do whatever you like in this case I add an attribute maybe not a good idea in real world but maybe sometimes you can use it but you can also inspect the class so you can go inside the class and can check if the class what method it has how long the method name Before class decorators, you would have to use a metaclass for this. Metaclasses are way more complex than decorators. This is pretty easy to understand. Compare the metaclass and metaclass. I need to explain an hour to get to this explanation how metaclass actually works. Metaclasses, most of the time you don't need them. Very few people need metaclasses. Class decorator is much simpler and you can apply a class decorator to a meta class there's one difference when you use inheritance so once you use a meta class all child classes inherit will get it here you have to apply the decorator to each class so inheritance it doesn't propagate through inheritance that's a difference but otherwise you get the same functionality so class decorators are actually conceptually simpler because you don't have all this def def def def thing you get a class and you return a new class and you're totally free to do whatever you like because a class has a lot of properties so this is also a use case to look inside the dict so the dict is a mapping proxy that's essentially a read only dictionary if you want which behaves mainly like a dictionary you can look inside and you see all these attributes, you use one, you add it and a bunch of this double underscore leading and trailing double underscore attributes the doc string and a bunch of other things that are always included the module name and so on and then you could inspect it if you have methods and you could check for whatever things you want we do an example we have an exercise with it so now that's the basics now the best practice best practice see practice practice when we go back now I go back to the simplest example that works and which would be our add or our hello without anything yeah so because this has nothing to do with parametrization and this nesting all those kind of things so if i do this i have this one and i add my hello to my add function so we did this before i guess so and now if i look at the name the name is gone and also if i which i always should do i add a doc string yeah so i add a doc string and then i look at my ad and there's no doc string i lost my doc string that's bad what's the solution you should always use functools reps so from functools so functools is a module of the standard library you should import reps so there's nothing to eat but a helper function that helps you to make a nicer decorator so if i go to my decorator then you have to specify here and ironically enough it uses a decorator to make a nicer decorator so you apply the decorator reps and you tell them to use your function funk yeah so now person knows okay this rep is going to rep the function funk so this is the thing you always should do and now if i do the same thing again i define my function yes if I find my function here now and I look at my add now it keeps the signature and keeps the doc string so this attributes are preserved because reps if you look at implementation of reps it's doing not much more than going into the original function and getting the attributes and adds it as a new function so the new function it's a different object but for most practical purposes it looks like the original one with added functionality the name is ok the doc string is ok and so on so if I say add.name then it keeps the name and there is also I forgot the name the original the wrapped yeah so you also have access to the wrapped function so if you want to call the the original wrapped function without the functionality you still have it yeah now it still does x plus y but without this printing of the hello the original better decorated one puts this hello there so you still have access to original function just in case you need it maybe you want to test something without a decorator, you could test wrapped. These standard names are special internal names. So there are some considerations. If you write tests, you might access them. If you, yeah, some of them can be accessed. So read access is fine, write access. So these ones, you have to be very careful. You have to know what you're doing. Yeah, this depends. But wrapped is explicitly there. And of course, you could say original function equals add.wrapped. and then you have a better name to the original function, which can be useful. Good. There are a few things you have to consider if you use recursion. So recursion is supported in Python, but it's for many cases not the right tool. So Python 3 now has a recursion limit of 3000 as a function calls itself 3000 times to get a recursion error. So recursion can be used if your problem is really recursive and you don't have to recursive so deeply that's fine, but don't replace loops with recursion. That's typically not the way to work in Python. It can make your program very slow. So if you decorate a function, the function calls itself, it calls the decorated version. That's something you have to be aware of. If you have recursion, there's just one example here that when you recurse something, you will get the decorated version. Good. That's the theory of decorators. So, everything is a function. Python, you can define functions inside of functions. You can return functions. And then, the decorator is just a nicer syntax to replace an existing function with a newer function. Plus, we also have class decorators, which do exactly the same, but with classes. Any questions so far? Good. So, the rest is use cases. So, I would like to show you some use cases. And then they also use some other principles here. So if you downloaded the zip file, then there's a directory decorators with a bunch of these examples. And we go through one by one. And this might give you some inspiration what you can do with these decorators. So the prime usage for decorators is what's called cross-cutting concern. So you want to do something in many places in the application that has nothing to do with your application logic. And I have a few examples here. You see logging is one, caching is one, and argument checking could also be one. And let's have a look at these. So I load my arc check here. And this is a check function arguments for given type. So Python is dynamically typed. You might know that Python supports type hints, which are just used by tools before you start your programming. And there's no runtime type checking. Sometimes it might be useful to implement something. And a decorator could be one way to do type checkings. Like if you need to talk to the outside world and they need special types, then you can just use any type. Then you can check the type here. That would be maybe one useful example. So we use fun tools as always. and here i use a three def so if you have three deaths i have three returns remember and i use outside i use check which is my decorator and here use a star syntax remember star syntax means it takes an unspecified number of positional arguments and these will be the types so i specify the types i want then the next one is a funk and then the internal one is the one that's wrapped with FuncTools wraps. And this is the body of the whole thing. You have to return, return, return down here. And here I check two things. First I check that the number of arguments match the number of types I've specified. So arguments is a tuple, has a length, and archetypes is also a tuple, has a length, and if this length doesn't match, I raise an exception and say okay, type error, and I assemble a message and say okay, I expected so many and I got so many. If the first thing is fine, so the length matches, I do the same thing. The next thing I do is zip. So I zip together the arguments I actually get and the types I expect. This is my argument and this is my type. And if this argument is not an instance of the type, I raise an exception. So when you do type checking, it's highly recommended to use is instance, not type. So type would check for the specific type. this instance would include the parent class so it would account for potential inheritance so in most cases you want to use this instance and that's why I use this instance and if it doesn't match again now I assemble an error message and I raise type error and I use here the type I get and the type I expect I just do this one and I just create an error message with as much context information sorry so now we have I executed the cell, now we have a check and I can now say check and I can specify types and I say now I want to have two floats for my add function and once I do this it still looks the same and if I try to call my add function these two integers doesn't like me anymore because it says now expect the class float float but class int int so if i just add a dot here which makes it which makes it an integer and float now it works so I can now with this decorator force this function to take only two floats and nothing else so this would be the type checking you can supply here I don't recommend using type checking all over the place but there might be a few cases where this is very useful especially if you When I save something in a file somewhere or in a binary format, and it has to be a special data type, before you do it, you might be able to check this here. Good. This is a principle. So I write this small helper. Yeah. And I'd supply the types, and then I check the types of the function. Likewise, if I call my function here, the normal function would have called this already. But if I call on this one argument, I do get a number of arguments error first before I go to get to the types. This is normal if you don't specify a default parameter, you will get the same, pretty much very similar message from Python. But even if you have a default one, you would still have two, which is kind of a little bit contradictory, I guess. So you need to design this one in a way that makes sense. but it's just as an example what we have here so argument checking is one example and now this looks pretty scary but it's you will see this will be the same pattern again and again and again it's just what you do inside it's a bit different therefore the next i would like to load is caching so i call it cached because caching i think it's a it's not library module here i use caching means when you have a function that's deterministic so it produces exactly the same output for the same input you might be able to cache the result and there are many different ways of caching it here, I use pickle to generate some cache which assumes that the arguments in my function are pickleable most of the objects in Python you can pickle, some of them you cannot you cannot pickle functions you cannot pickle file handles you cannot pickle lambdas you cannot pickle sockets or something like this so there are some exceptions but let's assume everything is pickleable and now I use only a two layer approach and I have a very simple cache you don't have to implement the cache yourself there is an LRU cache in Python already but there is an example and you might even use LRU cache in here to modify the behavior with the decorator LRU cache also works with the decorator by the way so what i do i keep a cache inside this function which is maybe not a good idea because it's very difficult to access it you could but it's not typically the way to do it and then i have my functual reps here and i have this one and this is pretty easy i just generate a key by using dumps so i take the arguments and the keyboard arguments in the tuple which assumes they are pickable generate a pickle object which i use as a key in a dictionary and I just put this key in the dictionary. If the key is not there yet, I call the function and caching assumes that the function call is expensive. So for our example, x plus y, the caching probably takes more time than calculating x plus y. This is an artificial example here to show how it works. In a real world, this function should be something at least a few milliseconds or seconds. So if something is a bit longer, you want to shortcut it and don't want to produce the same result again and again, then caching can be useful. And I put in a dictionary. If it's there already, it just goes to the dictionary and returns the result. So this assumes if you have the same combination of arguments, I produce the same result. If the function doesn't fulfill this criteria, this caching doesn't work. Good. So you can make your own very simple cache. And now I can use my cached for my add function and here I make it a bit nicer I say print do you see something happens and I return x plus y and if I call it now you can see if I call this 4 and 5 it prints adding if I call it again this 4 and 5 it doesn't do anything. Yeah? So it just retrieves the pre-calculated 9 from the dictionary because it uses a key. The key calculation is probably more expensive than the adding here of course, so it doesn't make a lot of sense to do anything like this. If I call this function like this which is totally possible with a keyword argument it recalculates it again because it's not smart enough to recognize that the 5 y equals 5 is the same as the position of 5 before it's a different case but there are not so many combinations that's probably easier maybe you can figure this out but it takes maybe so much work then just recalculate the result next time it will be there so now my cache is not really there it should be hopefully in the closure and you can see there's two objects in the closure the closure is the thing that's preserved that's actually what's used to live in the outer function and now I still need it in my decorator and one is the function that I'm wrapping and the other one is a dictionary which is in the first cell so if I have add which is closure which is a tuple and I access the first argument. It's a dictionary and then I can say give me the keys and then it shows you the keys. Oops. No, it doesn't have the keys. It's not a dictionary. What I'm doing wrong? It's a dict. And dict doesn't have keys. strange yeah it's still a cell so it's cell 0 and inside the cell there's a so I'm in the cell and inside the cell there's a cell yeah cell contents sorry cell contents which is a dictionary and you see that's a dictionary now and of course it's just the pickle. It's a binary format with a special pickle protocol the standard pickle protocol and you see the nine is there twice but once with two positional arguments and one positional and there's a keyword argument. So this is not a very convenient way to access a dictionary so you would need to put it somewhere else potentially to make it more useful but that's the principle of caching. so you can write your own cache this is pretty much one of the simplest cache you can write but of course you could instead of pickle it to a dictionary you can use shelf and persistently pickle it for the next time or anything like this yeah you could also have cases and say a few things that you don't want to cache because they don't produce reliable results you only want to cache certain types of or the result only for certain types of arguments so if we need a cache that's a bit more sophisticated, then you can use it. LRU cache in Python does most of it already, and you could actually use it, but you could write something around it to make it more specific to your needs. And this would be a very prime example for decorator. Good. So another example, the next one is logging. So a very common way to write a decorator would be logging. Logged. So, and here I make it a little bit different, so from future importance, just for Python 2, I probably can take it out pretty soon, functools, and I have a constant, login, false, it's like a global switch, I can turn something on and off, and this is again a two level decorator with two deaths and two returns, looks exactly the same, we take the function, have the wrapper with the args and keyword args and then I'm using logging so if logging is on I do something otherwise I just return the result of the function anyway so if I do this and I use this locked decorator locked in front of my create add function and I call my function nothing happens except the production here but if I have this global so it's a bit strange I'll use a constant I change the constant I don't know if it's a good design decision but I do it anyway right here just for the purpose of showing you and now if I change this constant at runtime and I call add again now the logging is on so instead having if in all these functions that are to be logged or not, I just change one global switch and all the functions are decorated with do logging. Of course, in reality, you wouldn't just print log, you would do some logging. Now you could use Python's logging module or some other solution to do some actual logging, and you can globally turn on and off logging. Of course, you could make it more sophisticated, introduce turning on and off for different log levels and so on, if you like. But this would be something you can do with a decorator pretty easily. And in the real world, that would be maybe a bit more complex than this. But this would be the logic. Again, cross-cutting concerned is a main application of a decorator because logging has nothing to do with your business logic, but it's something you can use potentially for many different libraries because you want to do logging. But actually, what the logging is doing, you could have another layer and supply the logging function as an argument and then call the logging function. Have another layer, three devs, and outside you introduce the logging function. You can inject whatever logging function you like with a decorator. This would also be possible. Good. That's logging. Another thing. And here we have this global switch. one more example with functions and then I do have one example with classes and then we do have exercises any questions so far? any questions? yeah so it's calling of functions in Python is comparatively slow because you always have to look up the name and do things so it depends what you compare it with yeah so if you do a call to a database then Python decorators are probably much faster so if you call a function in the loop again and again and often you might be if you have a lot of extra calls you might slow it down yeah why does it have an overhead because I saw that the decorator is only called once at the import time yeah the decorator but if I but I replace let's look at this example here I replace the original function with a function so I call this function and then I call this function this is not a good example because it's caching here I return a new function so this function checked will be called so you're right, this one is only called the outermost one is only called at import time but then I replace the original function with a double function so for each normal function call i have two function calls because i call the outer function the wrapper and this one in turn calls the function here that's the second call so i always have a second call if i have this most decorators are like this that i want to change behavior when i call a function if i don't want to do this and i use the decorator as our very first hello our very first hello that didn't work it would have worked if it just returned the function so remember the very first one the very first one the very first one yeah this one so if this one didn't work because i didn't return anything but if i return the function unchanged then there wouldn't be any new runtime overhead but the decorator would be executed only at import time sometimes actually our next example is exactly sometimes you want this more use cases are typically you replace the built-in function that means you call the wrapper, and the wrapper calls the original function because you want to have the functionality of the original function. Unless you want to turn it off totally, that's also possible. For some testing, you want this function to be called, but you don't want the function to do anything, then you could turn it off. That would be another application for decorator. Yeah, but the way we do it here, for most cases, you would double the number of function calls, which adds overhead. Yeah? For the check decorator, is it possible you can remove one of the wrapper functions by appending on timestamp of the function? Probably you could, just a minute. You can add the question, you could add attributes to functions. That would be something you could do. So, let's look at the check. Argument check. So, now we make it a parameterized run with three def, def, def. and three returns and the outermost layer gives you the arguments you could also somehow get the arguments in a different way yeah and you can add attributes to a function but only after the function is defined i'm not sure i don't think you can function itself maybe you can do it in a function you could say function name dot argument equals something that's possible but then you would need to modify the function itself and if you do it from outside you would need the function first and then you can do it and then you're out of the nice decorate because the decorator you can do everything on top and you see it that's a syntactic sugar the decorator gives you and you would lose it certainly you can achieve the same functionality with different ones or different methods and maybe get rid of one of these layers but the thing is that decorator is a bit more complex but you write it only once and then you can forget to just use it So you have this complex thing written once, and you don't have to bother on your source code. If you have a different solution, you have to tell the user you have to do three steps. Then you offload more complexity to the user. Here you put the complexity in the library, the decorator as a library. And then when you use it, it's easy. That's probably the better solution. But certainly there are different ways of doing it. But once you have a decorator anyway, you can as well make it a parameterized decorator. which makes this much easier to understand because people understand parameterized decorators. And using it, you don't have to understand how it actually works internally. You just use it like this as a recipe and you don't know what it's actually doing. It's still okay. Of course, it's always better to understand why things work, but you don't have to necessarily. Any more questions? Good. So let's go to our last example for registering any kind of a registry so it's not necessarily the windows registry here registering and this is a concept you want to do something beginning and like if you have a framework who knows the difference between library and a framework so typically you import the library and call functions of a library a framework turns it around the framework you write code and the framework is calling you that's more or less the definition of a framework something like this and if you want to give the framework something to call you can register it so if you want the user to extend the functionality with some kind of plugging this would be a very poor man's way of doing this there are very good libraries that help you to write plugins but just for the understanding i do have a registry which is again a dictionary and i want to do something and i have two different register ones. So, register at call and register import time, which also has an implication to this overhead. So, register, let's look at the registry at call first. It's a three name, a three level, because I want to have different registries with names, so I can supply a name. And then we have our def, def, def, reps, args, keyword args. Again, the same pattern. So, you can take this thing as a scaffold for your own decorator and just replace it with different names and different arguments and then i use a set default which i hopefully notice means i go to the dictionary so if the name is there already i append to an existing list if not i put a new list and append to it immediately so for this one i don't have to check if there's something already append to it otherwise add a new list set default is doing this for me has nothing to do with the decorator it's just a normal dictionary method. So I do this and then I just return this from the function. I do my standard protocol. So let's have a look. So if I look at the registry, which is an empty dictionary, then I use my decawriter, register, add call, and I give it a name. simple and again as i promised we define our function so if i do this and if i look in the registry what's written in the registry anybody what's in there nothing because i haven't called my function yet so the name if I call my function I get my result and if I look at my registry again then you see now the function ends up in the registry so now when somebody calls something happens to it so then I know this function has been called if I look at the registry I know this function has been called in this case I just add the function object but you could also do something else whatever you would like to do there good this is the first one the second one is registered at dev looks very different so there is one level less even though I have a name and I do the same thing here but then I return the function unchanged and this wouldn't do any overhead because I just have this at import time once I call this register at dev once and then I return the function unchanged this is not that common but it's possible to return the function unchanged because i'm not i don't want to change the functionality of the function i just want to have a side effect when the function is defined and i know somebody defined a function so let's do this uh clean our registry so registry is um a dictionary that's why i can say clear which clears my registry yeah and now I use my decorator register at def and I define my function and I did a mistake I have to make it give it some kind of string because I decided to do something like this and I return what I'm doing x and y x plus y so and now if I look at my registry my registry changed the minute I defined the function and when I call my function nothing happens you can call the function as often as you like it doesn't change the registry again so the registry is still the same so these are two ways to some kind of whatever you want to call it. Sometimes you want this. Now I know my user defined this function and marked it in a special way, and I can find it. The other one is doing something that the function is called. Now I know this function is because it's kind of a special function, and I can see if the function is called or not. These are two applications of whatever you want to sub-sum under this concept of a registry. you have some kind of thing that happened and you have a feeling what happened one more example for a class decorator so far all the decorators are for functions and very often you will use decorators for functions but you can also use a class decorator so I define a function it's called assert fluid yeah and I give it a class and when you do this the first thing you should do return class because I tend to forget this one and now you write the body whatever you want to do with the class so I use assert so assert means make sure that this what's on the right hand side is true if it's not true you raise an exception you have to be careful it's not exactly the same as doing exception handling because if you call python with dash c at the command line for your dash o sorry for optimized then assorts will be turned off so you can turn off assorts at runtime they're just for something for debugging but i use it here just but be aware so if i check that i assume that my class has a has an attribute temperature and this temperature is supposed to be in this range. You see, it's degrees Celsius and we assume 0 and 100. So some physicists will say, okay, if you have a different pressure and you have some salt inside and it is not true, but we assume that these are the temperatures to make water solid and water as a gas. so you have it's not really water anymore so if I have a class water here which I decorate with my decorator so 25 degrees nice and warm as in the laboratory then it works if I do the same thing and let's call it not water but ice and ice has negative temperatures in Celsius so it's minus 25 and I try to define the class I cannot define the class the minute I define the class so at import time or byte compilation time if you want I get this assertion error and of course I can also raise the string here temperature temperature must be between 0 and 100 degrees so it doesn't fit here anymore on my line because I have so big a resolution here or zoomed in but just write something like this and now if you do this then you get this message whatever you write on the right hand side so of course if you want to make it more sophisticated you can also raise an exception raise an exception but do it in a traditional way then it will also work and now you can if you put the decorator in front of a class you can only define the class that corresponds to your requirements for the class but you can check it at the definition time that can be interesting yeah good thing so you you can you can always use a class with a done the call method that's what you mean that works like a callable because actually you don't have a function Python has a notion of a callable which can be a function a method or instance of a class that has a standard call and if you're familiar with this one then you can write because you wouldn't have this many you can save one level typically I use this method when I have a function that has a state so normally functions don't have a state so maybe an example, just nothing with decorators yet, but you could do something like this. You could write a class that's called a function and this is my prime example for this kind of approach. Yeah, and then you have this call method and this would be actually the function that you do. So you have the self and you would have let's say x and y you can make it more general just hand the function in but do something like this so i would say self dot counter plus equals one and then return x plus y so now i can make an instance of the function that's instance of a class but it behaves just like a function and now i have an attribute and if i call it with something it does my add and if i look at the counter the counter increase i call the function so the function knows how many times it's been called you can do this as a function to function can have an attribute but when you have a function I would strive for a kind of a pure function. There's no such a thing as a guarantee for a pure function in Python. So a function without a side effect because incrementing the counter is kind of a side effect and therefore I like this approach. And of course you can use this approach to write a decorator because we have a function. So it depends but this assumes that you know how this works and then you can use it because in the end is just callable, and you could use the init to parameterize it, and you would achieve the same thing. It would reduce one level of indentation. No, that would be the same. This is just a call. I don't think there's any difference in performance. I wouldn't say there is, but I'm pretty sure there's no difference. The overhead would be still there, because you still call the down-to-call, which calls which acts as a wrapper for the other function. So overhead-wise, it would be pretty much the same, I think. So this is a totally different concept, but if you know this, you can apply it instead of a normal function. You can use a core, but it would be exactly the same thing. Do you have any more questions? so you still have a few minutes I think until when do we have one minute left oh I thought it would be one half hour no okay then then you have to have a homework so write a decorator and here the decorator the task is I just show you it's a time measurement and one way would be just the last thing import timeit is a module of timeit and timeit gives you what's called a default timer it gives you a timestamp so what you can do you can make a timestamp before and then you later on after your function call you do a timeit default timer again and say minus start which gives you it tells me It took me 11.6 seconds to do all this. So this one you can incorporate in the decorator. And this decorator can measure how long it takes to call a function. For instance, cross-cutting concern time measurement. So our cross-cutting concern now is break. So thank you very much for your attention. Hope you learned something.