See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/272833859

Advanced Python Techniques: Decorators

Conference Paper · February 2015

CITATIONS READS

0 3,807

1 author:

Tomo Popovic
University of Donja Gorica
132 PUBLICATIONS   657 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

VIRAL - Vitalising ICT Relevance in Agricultural Learning (Erasmus+) View project

RTEPMS - Real-Time Environmental Parameters Monitoring System (National) View project

All content following this page was uploaded by Tomo Popovic on 28 February 2015.

The user has requested enhancement of the downloaded file.

 Informacione tehnologije IT'15

NAPREDNE TEHNIKE U PYTHON-U: DEKORATORI

ADVANCED PYTHON TECHNIQUES: DECORATORS
Tomo Popović, Electrical and Computer Engineering, Texas A&M University

Sadržaj: Programski jezik Python je postao veoma popularan zadnjih godina. Python je sve više
prisutan i u edukaciji zbog veoma čitljive sintakse koja omogućava brzo učenje programiranja i
principa računarskih nauka. Ovaj rad istražuje dekoratore u Python-u, koji se smatraju
naprednom tehnikom čak i od strane iskusnih Python programera. U radu je dat opis mehanizma
dekoratora kao i njihova implementaicija korišćenjem klasa i funkcija. Opcije za implementaciju
su objašnjene korišćenjem ilustrativnih primera i njihove vizualizacije.

Abstract: In recent years, Python had become an extremely popular programming language.
Python is becoming more and more present in education as its syntax gets out of the way and
makes it easy to learn and use to introduce computer science concepts. This paper explores the
inner workings of Python decorators, which is considered an advanced topic even for experienced
programmers. The paper provides an insight into decorators mechanism, as well as their
implementation using classes and functions. Implementation options are discussed using examples
and visualization.

1. INTRODUCTION and even replaces it with a completely different behavior.

The use of decorators assumes a line of code placed above
In recent years, the Python programming language has a function or method starting with character @, followed by
become extremely popular. Python is very flexible, versatile, the decorator's name and arguments. Decorators are used in
fun, easy to learn, and it enables programmers to be more various frameworks, such as popular Nose testing framework
productive. In Python, one can write scripts that run in OS [5], and they provide an easy syntax to use. For example, the
command line, cross-platform GUI applications, web Nose testing tools module provides the function timed,
applications, financial applications, as well as scientific code which passes if the decorated function, a function being
similar to Matlab [1,2]. Getting started with Python and its tested, finishes within a specified time limit (Fig. 1).
ecosystem is easy. The syntax gets out of the way and the
code almost resembles a pseudo-code. In addition, Python is @timed(.1)
open source and it is supported by great and extremely active def test_that_fails():
community. time.sleep(.2)
Advanced Python techniques may require more Fig. 1. Using the timed test function from Nose
exploration to be understood and used properly. Even
competent and experienced programmers often do not utilize A decorator is typically a function that takes another
all of the idioms and advanced concepts available in the function as an argument, and then returns a new function [6].
language. Python decorators would fit into that category. The returned function is used to replace the functionality of
While decorators are fairly straightforward to use, their the original, also called decorated, function. The use of
implementation may not be as easy to grasp. Even some decorators sintactically looks similar to annotations in Java
definitions of Python decorators seem ambiguous since they and C#. However, a better comparison would be to compare
mix the description of how to use them vs. how to implement the decorators with macros in C/C++, as explained in [4].
them [3,4]. Sometimes, decorators in Python get mixed with decorator
This paper is an attempt to introduce and explore internals design pattern [4,7]. Although the decorator pattern can be
of Python decorators, their implementation, and use in our implemented using decorators, they can also be used to
own code or libraries. Section 2 provides background on implement other design patterns [7].
Python objects, functions, introduction to Python decorators, Decorators can be used to implement loggers, timers,
as well as a visualization tool called Lumpy. Section 3 format conversions, frameworks, etc. They offer a powerful
illustrates implementation of decorators using classes and mechanism and easy decoration syntax to provide for
functions, while Section 4 provides discussions and modified functionality of decorated functions.
conclusions. References are given at the end.
2.2. Everything is an Object
2. BACKGROUND
Everything is an object in Python [8]. All the values,
2.1. Decorators in Python classes, functions, object instances, and other language
constructs are objects, which means they can have attributes,
Decorators in Python are applied to functions and have a member functions, they can be passed as arguments, etc. This
purpose to augment their behavior. A decorated function is concept is particularly obvious when when considering how
typically wrapped by another function, which is called instead Python handles assignments. Even integers can be seen as
the original one. The idea is that a decorator adds some new objects in Python (please refer to example in Fig. 2).
attributes to the original function, adds new functionality to it,
 Informacione tehnologije IT'15

a = 2
b = [1,2,3]
c = b
print a.__add__(2)
print (2).__add__(3)
Fig. 2. Everything in object in Python
2.3. Functions in Python

Functions are code constructs that are supposed to do

something, typically generate an output, based on their
arguments. In Python, functions can have a “procedural role”
without returning a value (they return a None). There are also
functions without arguments. In general, functions in Python
are fairly straightforward and have a very clear syntax [1,8].
What may not be so obvious is that functions in Python
Fig. 4. Visualization of Python objects using Lumpy
are objects like everything else that is user-defined (Fig. 3).
Functions have attributes, such as __name__, and they can be We will here start with a decorator implementation example
assigned to a variable. In this case the func1 function object using classes, which makes it easier to relate decorators to
is assigned to variable y and then called using that new name. object instances of that class. In Fig. 5, a class called
Instead of variables that contain values, Python language MyDecorator is implemented. The constructor method
deals with names and bindings, which in this case means that __init__() defines an attribute and binds it to the object
the variable y is bound to the same object as name func1. representing the decorated function f. The class also needs to
Functions are first class objects, which means that they be callable so the method __call__() is defined in which
can be assigned to variables, placed in lists, stored in the decorated function is called and executed. The purpose of
dictionaries, passed as arguments, and returned from other decorators in Python is to wrap the original function they
functions as a result [6,8]. Another very important concept to decorate and to provide for some modified or additional
introduce here are closures. A closure is a first-class function, behavior. In this simple example, we added printing before
a function that can be stored as a variable, and also has an and after the call of the original function object that is being
ability to access other variables local to the scope in which it decorated.
was created. The implementation of decorators rely on this
very concept, which will be explained later. class MyDecorator():
def __init__(self, f):
def func1(): self.f = f
print "Inside func1" def __call__(self, *args, **kwargs):
print("Entering", self.f.__name__)
func1() self.f(*args, **kwargs)
y = func1 print("Exited", self.f.__name__)
y()
print y.__name__ # @MyDecorator
def func1():
Fig. 3. Functions are objects as well print("inside func1()")

2.4. Visualization Using Lumpy @MyDecorator

def func2():
Lympy is a Python module that generates UML object and print("inside func2()")
class diagrams [8]. Lumpy is used from a running Python …
program and it can be a very useful debugging and Fig. 5. Class implementation of a decorator
educational tool since it generates a high-level visualization
that are somewhat in line with UML standard. In this paper, As shown in is shown in Fig. 6, the module object
Lumpy will be used to illustrate what happens when Python contains attribute MyDecorator referencing a class object
decorators are used. An example object diagram for with name MyDecorator. Since the decorator is commented
previously shown code (Fig. 2 and 3) is created using Lumpy for the first function, its name func1 is bound directly to a
and depicted in Fig. 4. The object state consists of a module function object named func1. Finally, the decorated function
that contains main references, where variable a is bound to an is bound to a MyDecorator that contains an attribute f
integer value 2, b and c are bound to a list object, and func1 bound to the original function func2. By calling func2, we
and y are bound to the same function object named func1. actually call an instance of MyDecorator that exhibits
modified behavior in its __call__() method.
3. IMPLEMENTING DECORATORS

3.1. Implementation Using Classes

Decorators in Python are often introduced through

functions. That can be confusing, especially if the audience is
not used to the concept of functions being objects.
Fig. 6. Decorated function references a callable object
 Informacione tehnologije IT'15

3.2. Implementation Using Functions

As mentioned before, decorators need to be implemented

as callable objects so that that they can be used to replace the
decorated functions. In the previous example, the decorated
function was replaced by an instance of a class that has a
__call__(). More commonly, decorators in Python are Fig. 10. Adjusting the names of decorated functions
introduced by implementing functions as decorators. Since
functions in Python are callable objects, they can be used to 3.3. Decorators with Arguments
implement decorators as illustrated in Fig. 7.
When implementing decorators with arguments, there is
def my_decorator(f): different behavior that needs to be noted. If the decorator has
def wrapped_f(*args, **kwargs): arguments, the decorated function is not passed to the
print("Entering", f.__name__) __init__() constructor, but to the __call__()method.
f(*args, **kwargs)
print("Exited", f.__name__) The __call__() method is only invoked once, during the
return wrapped_f decoration process. It has one argument, which is the
decorated function. In this situation, an internal wrapper
@my_decorator function is needed (wrapped_f in Fig. 11). The arguments of
def func1():
print("inside func1()") the decorated function are to be passed to the wrapper, which
can internally access both the arguments of the decorator and
@my_decorator decorated function.
def func2(x):
print("inside func2() %d" % x) class MyDecorator():
… def __init__(self, arg1, arg2):
print(func1.__name__) print("Inside __init__()"
print(func2.__name__) self.arg1 = arg1
self.arg2 = arg2
Fig. 7. Function implementation of a decorator def __call__(self, f):
print("Inside __call__()")
The Lympy object diagram for this implementation is def wrapped_f(*args, **kwargs):
shown in Fig. 8. It can be observed that both functions func1 print("Inside wrapped_f()")
print("arg1", self.arg1)
and func2 are now replaced with instances of new function f(*args, **kwargs)
wrapped_f. However, it is very important to note that, print("After decorated f()")
although they have the same name, these two instances are return wrapped_f
two different objects in memory and they internally reference @MyDecorator("test", 1, 2)
instances of the original functions func1 and func2. Another def func1(a,b):
important thing to note here is that wrapped_f() is a print("inside func1()")
closure, because it captures, closes over, the actual value of f. return a+b
…
Fig. 11. Class implementation of a decorator with arguments

A template for the decorator function implementation with

arguments is provided in Fig. 12. Although this
implementation is more concise it needs some exploring to
understand. At the time of decoration the decorator arguments
Fig. 8. Decorated functions are replaced with their closures are passed to my_decorator and the inside wrapper
function is only called once. A reference to the decorated
Printing names at the end of the program in Fig. 7 reveals function is passed as a parameter to the wrapper, which
that both names are the same even though internally the two defines an inner function wrapped_f that will replace the
reference two different function objects. If this behavior is not decorated function. Later in the code, whenever the decorated
desirable, the names of decorated functions can be changed function is called, the corresponding instance of wrapped_f
dynamically before it is being returned (Fig. 9 and 10). gets called instead. Please note that each instance of
Python is very powerful when it come to types of information wrapped_f internally has access to the arguments originally
that can dynamically be obtained and modified about passed to the decorator, as well as to an instance of the
functions. Fig. 10 shows that decorated functions func1 and original decorated function. This inner function creates a
func2 now reference wrapper functions with different names. closure for the decorated function f, but also for the
arguments passed to the decorator function at the time of
def my_decorator(f): decoration. On the “outside” the object diagram looks exactly
def wrapped_f(*args, **kwargs): the same as in the previous example shown in Fig. 8. Each
print("Entering", f.__name__) decorated function will be given a new instance of a
f(*args, **kwargs) wrapped_f that will close over the decorators arguments and
print("Exited", f.__name__)
wrapped_f.__name__ = "wrapped_" + f.__name__ the decorated function. If printed, the values of func1 and
return wrapped_f func2 reveal that they in fact reference two different
Fig. 9. Function as a decorator with the name reference instances of wrapped_f:
<function wrapped_f at 0x7f8d1f407230>
<function wrapped_f at 0x7f8d1f407320>
 Informacione tehnologije IT'15

def my_decorator(arg1, arg2, arg3): ('Entering', 'pi_func1')

def wrapper(f): Exited pi_func1, execution time: 0:00:29.527301
print("Inside wrapper") 3.14159266359
def wrapped_f(*args, **kwargs): ('Entering', 'pi_func2')
print("Inside wrapped_f()") Exited pi_func2, execution time: 0:00:00.000012
print("arg1", arg1) 3.14159264621
f(*args, **kwargs) ('Entering', 'pi_func3')
print("After decorated f()") Exited pi_func3, execution time: 0:00:00.000003
return wrapped_f 3.14159265359
return wrapper
Fig. 14. The output of the execution timer example
@my_decorator("test", 1, 2)
def func1(a,b): 4. DISCUSSION AND CONCLUSIONS
print("inside func1()")
return a+b A decorator is a callable object (typically a function) that
accepts one argument, which is a reference bound to the
@my_decorator("test", 1, 2)
def func2(x):
function being decorated. This object is often called a
print("inside func2() %d" % x) wrapper object. A decorator can be implemented as a function
… or class and it has to be called with a single argument. For the
Fig. 12. Decorator with arguments using functions class implementation, the __call__() method needs to be
defined to implement the new behavior. During the decoration
There are three levels of functions in the implementation process, a callable object instance of that class is created.
and the most inner one is the actual replacement function. When calling the decorated function, the wrapper object is
Even though the functional implementation appears more called, a function or a callable object depending on the
concise and clear, it may not be as easy to understand as the implementation.
implementation version using classes (Fig. 11). Due to the mechanism of binding names to functions as
first-class objects that can be stored as variables, decorators
3.4. An Example: Execution Timer can also be nested. This means that multiple decorators can
be applied to a single function. Nesting will be resolved from
An example of the use of decorators is given in Fig. 13. the decorated function up. Nesting of decorators can be very
The decorator function is designed to measure function powerful allowing the simultaneous use of decorators from
execution time. The assumption is that the decorated different libraries. More good examples of decorators are
functions (pi_func1, pi_func2, pi_func3) calculate given in [6], [9], and [10].
number Pi. Adding the decorator to each of these functions This paper provides a systematic and illustrative
means that an instance of the ExecTimer will be called and introduction to Python decorators. The implementation of
the original functions will be wrapped with code to measure decorators in Python is given using conceptual examples that
execution time. The example output is given in Fig. 14. are related to Python decorator mechanism. The examples are
accompanied with object diagrams using Lumpy, which can
from math import pi, fabs, sqrt be a useful learning tool. The paper covers templates for
from datetime import datetime
decorator implementation using both classes and functions.
class ExecTimer(): The paper addresses the peculiarities of the implementation of
def __init__(self, f): decorators with arguments. Hopefully, the text will motivate
self.f = f readers to use Python for learning computer science concepts,
def __call__(self, *args, **kwargs):
print("Entering", self.f.__name__)
as well as to explore decorators and other advanced Python
t1 = datetime.now() techniques.
result = self.f(*args, **kwargs)
t2 = datetime.now() REFERENCES
print("Exited", self.f.__name__)
print("Execution time: %s" % (t2-t1)) [1] Python Programming Language, http://www.python.org
return result
[2] IPython Interactive Computing, http://ipython.org
@ExecTimer [3] P. Eby, "Python 2.4 Decorators," Dr. Dobb's, No. 372,
def pi_func1(eps): pp. 54-57, May 2005.
"""Gregory-Leibniz series"""
… [4] B. Eckel, Python 3 Patterns, Recipes and Idioms,
http://www.mindviewinc.com/Books/Python3Patterns
@ExecTimer
def pi_func2(eps):
[5] Nose documentation, http://nose.readthedocs.org
""" Gauss-Legandre formula""" [6] D. Beazley, B. K. Jones, Python Cookbook, 3rd ed,
… O'Reilly Media, May 2013.
@ExecTimer [7] E. Gamma, R. Helm, R. Johnson, J. Vlissides, Design
def pi_func3(): Patterns, Addision-Wesley, Nov 1994.
return pi [8] A. B. Downey, Think Python, O'Reilly Media, Aug 2012.
eps = 0.00000001 [9] M. Harrison, Guide To: Learning Python Decorators,
print pi_func1(eps) CreateSpace, Sep 2013.
print pi_func2(eps) [10] The ASPN online Python Cookbook,
print pi_func3()
http://aspn.activestate.com/ASPN/Cokbook/Python
Fig. 13. Checking execution time using decorators

View publication stats