In the previous episodes

basic stuff, syntax
functions (wrote our cool min function) + functional stuff
scopes
PEP-8
strings
bytes
collections

🏛 Classes basics

👇 Class

>>> class Counter:
        """I count. That is all."""
        def __init__(self, initial=0): # constructor
            self.value = initial
 
        def increment(self):
            self.value += 1
 
        def get(self):
            return self.value 
>>> c = Counter(42)
>>> c.increment()
>>> c.get()
43

Welcome, self keyword

Attributes: object and class (like in other languages)

>>> c.some_attribute = value
>>> class Counter:
        all_counters = []
 
        def __init__(self, initial=0):
            Counter.all_counters.append(self)
            # ...
 
>>> Counter.some_other_attribute = 42

㊙ Private attributes

>>> class Noop:
        some_attribute = 42
        _internal_attribute = []
>>> class Noop:
        __very_internal_attribute = []
 
>>> Noop.__very_internal_attribute
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: type object 'Noop' has no attribute []
>>> Noop._Noop__very_internal_attribute

class MemorizingDict(dict):
    history = deque(maxlen=10)
    
    def set(self, key, value):
        self.history.append(key)
        self[key] = value
    
    def get_history(self):
        return self.history
 
d = MemorizingDict({"foo": 42})
d.set("baz", 100500)
d = MemorizingDict()
d.set("boo", 500100)
print(d.get_history()) # ==> ?

>>> class Noop:
    """I do nothing at all."""
>>> Noop.__doc__
'I do nothing at all.'
>>> Noop.__name__
'Noop'
>>> Noop.__module__
'__main__'
>>> Noop.__bases__
(<class object>,)
>>> noop = Noop()
>>> noop.__dict__
{}

🔖 dict attributes

>>> noop.some_attribute = 42
>>> noop.__dict__
{'some_attribute': 42}
>>> noop.__dict__["some_other_attribute"] = 100500
>>> noop.some_other_attribute
100500
>>> del noop.some_other_attribute
>>> vars(noop)
{'some_attribute': 42}

👀 slots

Reduce memory usage!

>>> class Noop:
        __slots__ = ["some_attribute"]
 
>>> noop = Noop()
>>> noop.some_attribute = 42
>>> noop.some_attribute
42
>>> noop.some_other_attribute = 100500
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Noop' object has no attribute []

🔗 Bounded and unbounded methods

>>> class SomeClass:
        def do_something(self):
            print("Doing something.")
 
>>> SomeClass().do_something
<bound method SomeClass.do_something of []>
>>> SomeClass().do_something()
Doing something.
>>> SomeClass.do_something
<function SomeClass.do_something at 0x105466a60>
>>> instance = SomeClass()
>>> SomeClass.do_something(instance)
Doing something.

@property

>>> class Path:
        def __init__(self, current):
            self.current = current
 
        def __repr__(self):
            return "Path({})".format(self.current)
 
        @property
        def parent(self):
            return Path(dirname(self.current))
 
>>> p = Path("./examples/some_file.txt")
>>> p.parent
Path('./examples')

🍩 more @property

>>> class BigDataModel:
        _params = []
 
        @property
        def params(self):
            return self._params
 
        @params.setter
        def params(self, new_params):
            assert all(map(lambda p: p > 0, new_params))
            self._params = new_params
 
        @params.deleter
        def params(self):
            del self._params

👩‍👧‍👧 Inheritance

>>> class Counter:
        def __init__(self, initial=0):
            self.value = initial
 
>>> class OtherCounter(Counter):
        def get(self):
            return self.value

Search for values/methods:

object ➡️ class ➡️ base classes

👨‍👦‍👦 Inheritance

>>> class Counter:
        all_counters = []
 
        def __init__(self, initial=0):
            self.__class__.all_counters.append(self)
            self.value = initial
 
>>> class OtherCounter(Counter):
        def __init__(self, initial=0):
            self.initial = initial
            super().__init__(initial)
 
>>> oc = OtherCounter()
>>> vars(oc)
{'initial': 0, 'value': 0}

☑️ isinstance() predicate

>>> class A:
        pass
>>> class B(A):
        pass
>>> isinstance(B(), A)
True
>>> class C:
        pass
>>> isinstance(B(), (A, C))
True
>>> isinstance(B(), A) or isinstance(B(), C)
True

☑️ issubclass() predicate

>>> class A:
        pass
>>> class B(A):
        pass
>>> issubclass(B, A)
True
>>> class C:
        pass
>>> issubclass(B, (A, C))
True
>>> issubclass(B, A) or issubclass(B, C)
True

🎭 Do not overcomplicate with inheritance

In case you are wondering how search for methods in subclasses works:

https://code.activestate.com/recipes/577748-calculate-the-mro-of-a-class/

Mixin

>>> class ThreadSafeMixin:
        get_lock = 
 
        def increment(self):
            with self.get_lock():
                super().increment()
 
        def get(self):
            with self.get_lock():
                return super().get()
 
>>> class ThreadSafeCounter(ThreadSafeMixin,
    Counter):
        pass

But we have decorators!

🏛 Classes basics wrap-up

all attributes are stored in dictionaries
properties are functions that can be called like attributes
do not overcomplexify the inheritance
magic names like __dict__, but later about it – https://docs.python.org/3/reference/datamodel.html#special-method-names
decorators can be used too, but later about it

🏵 Decorators

Decorator is a function that gets a function and returns smth

>>> @trace
    def foo(x):
    return 42

>>> def foo(x):
        return 42
 
>>> foo = trace(foo)

🚲 Let’s write a decorator

>>> def trace(func):
        def inner(*args, **kwargs):
            print(func.__name__, args, kwargs)
            return func(*args, **kwargs)
        return inner
 
>>> @trace
    def identity(x):
        "I do nothing useful."
        return x
 
>>> identity(42)
identity (42, ) {}
42

Problem – we forgot old metadata

>>> help(identity)
Help on function inner in module __main__:
inner(*args, **kwargs)
>>> def identity(x):
        "I do nothing useful."
        return x
 
>>> identity.__name__, identity.__doc__
('identity', 'I do nothing useful as well.')
>>> identity = trace(identity)
>>> identity.__name__, identity.__doc__
('inner, None)

Let’s fix it!

>>> def trace(func):
        def inner(*args, **kwargs):
            print(func.__name__, args, kwargs)
            return func(*args, **kwargs)
        inner.__module__ = func.__module__
        inner.__name__ = func.__name__
        inner.__doc__ = func.__doc__
        return inner
>>> @trace
    def identity(x):
        "I do nothing useful."
        return x
 
>>> identity.__name__, identity.__doc__
('identity', 'I do nothing useful as well.')

This could be done simpler

>>> import functools
>>> def trace(func):
        def inner(*args, **kwargs):
            print(func.__name__, args, kwargs)
            return func(*args, **kwargs)
        functools.update_wrapper(inner, func)
        return inner

Or even more simple

>>> def trace(func):
        @functools.wraps(func)
        def inner(*args, **kwargs):
            print(func.__name__, args, kwargs)
            return func(*args, **kwargs)
        return inner

🚲 Let’s add some config flag

>>> trace_enabled = False
>>> def trace(func):
        @functools.wraps(func)
        def inner(*args, **kwargs):
            print(func.__name__, args, kwargs)
            return func(*args, **kwargs)
        return inner if trace_enabled else func

Important reminder

>>> @trace
    def identity(x):
        return x

>>> def identity(x):
        return x
 
>>> identity = trace(identity)

>>> @trace(sys.stderr)
    def identity(x):
        return x

>>> def identity(x):
        return x
 
>>> deco = trace(sys.stderr)
>>> identity = deco(identity)

👀 We need to go deeper

decorator with arguments

>>> def trace(handle):
        def decorator(func):
            @functools.wraps(func)
            def inner(*args, **kwargs):
                print(func.__name__, args, kwargs,
                      file=handle)
                return func(*args, **kwargs)
            return inner
        return decorator

and we can make another decorator

>>> def with_arguments(deco):
        @functools.wraps(deco)
        def wrapper(*dargs, **dkwargs):
            def decorator(func):
                result = deco(func, *dargs, **dkwargs)
                functools.update_wrapper(result, func)
                return result
            return decorator
        return wrapper

1.`with_argument` gets decorator `deco`
2. wraps it to `wrapper`, `deco` is a decorator with args, 
then wraps it into `decorator`
3. `decorator` copies new decorator with `deco` and copies 
inside it internal attributes of function `func`

🚲 We did it again

>>> @with_arguments
    def trace(func, handle):
        def inner(*args, **kwargs):
            print(func.__name__, args, kwargs, file=handle)
            return func(*args, **kwargs)
        return inner
 
>>> @trace(sys.stderr)
    def identity(x):
        return x
>>> identity(42)
identity (42,) {}
42

Questions?

How about default variable?

>>> @with_arguments
    def trace(func, handle=sys.stdout):
        @functools.wraps(func)
        def inner(*args, **kwargs):
            print(func.__name__, args, kwargs,
            file=handle)
            return func(*args, **kwargs)
        return inner
>>> @trace
    def identity(x):
        return x
 
>>> identity(42)
<function trace.<locals>.inner at 0x10b3969d8>

The solution

>>> @trace()
    def identity(x):
        return x
 
>>> identity(42)
identity (42,) {}
42

Same, but simpler

>>> def trace(func=None, *, handle=sys.stdout):
        if func is None:
            return lambda func: trace(func, handle=handle)
 
        @functools.wraps(func)
        def inner(*args, **kwargs):
            print(func.__name__, args, kwargs)
            return func(*args, **kwargs)
        return inner

What’s * for?

😎 Practical decorators

@timethis

>>> def timethis(func=None, *, n_iter=100):
        if func is None:
            return lambda func: timethis(func, n_iter=n_iter)
 
        @functools.wraps(func)
        def inner(*args, **kwargs):
            print(func.__name__, end="     ")
            acc = float("inf")
            for i in range(n_iter):
                tick = time.perf_counter()
                result = func(*args, **kwargs)
                acc = min(acc, time.perf_counter() - tick)
            print(acc)
            return result
        return inner
 
>>> result = timethis(sum)(range(10 ** 6))
sum     0.026534789009019732

@once

>>> def once(func):
        @functools.wraps(func)
        def inner(*args, **kwargs):
            if not inner.called:
                func(*args, **kwargs)
                inner.called = True
        inner.called = False
        return inner
 
>>> @once
    def initialize_settings():
        print("Settings initialized.")
 
>>> initialize_settings()
Settings initialized.
>>> initialize_settings()

@memoized

>>> def memoized(func):
        cache = {}
 
        @functools.wraps(func)
        def inner(*args, **kwargs):
            key = args, kwargs
            if key not in cache:
                cache[key] = func(*args, **kwargs)
            return cache[key]
        return inner

Ooops…

>>> @memoized
    def ackermann(m, n):
        if not m:
            return n + 1
        elif not n:
            return ackermann(m - 1, 1)
        else:
            return ackermann(m - 1, ackerman(m, n - 1))
 
>>> ackermann(3, 4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in inner
TypeError: unhashable type: 'dict'

The solution

>>> def memoized(func):
        cache = {}
 
        @functools.wraps(func)
        def inner(*args, **kwargs):
            key = args + tuple(sorted(kwargs.items()))
            if key not in cache:
                cache[key] = func(*args, **kwargs)
            return cache[key]
        return inner

@deprecated

>>> def deprecated(func):
        code = func.__code__
        warnings.warn_explicit(
            func.__name__ + " is deprecated.",
            category=DeprecationWarning,
            filename=code.co_filename,
            lineno=code.co_firstlineno + 1)
        return func
 
>>> @deprecated
    def identity(x):
        return x
 
<stdin>:2: DeprecationWarning: identity is deprecated.

@pre

>>> def pre(cond, message):
        def wrapper(func):
            @functools.wraps(func)
            def inner(*args, **kwargs):
                assert cond(*args, **kwargs), message
                return func(*args, **kwargs)
            return inner
        return wrapper
 
>>> @pre(lambda x: r >= 0, "negative argument")
    def checked_log(x):
        return math.log(x)
 
>>> checked_log(-42)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in inner
AssertionError: negative argument

@post

>>> def post(cond, message):
        def wrapper(func):
            @functools.wraps(func)
            def inner(*args, **kwargs):
                result = func(*args, **kwargs)
                assert cond(result), message
                return result
            return inner
        return wrapper
 
>>> @post(lambda x: not math.isnan(x), "not a number")
    def something_useful():
        return float("nan")
 
>>> something_useful()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in inner
AssertionError: not a number

You can apply multiple decorators, but the order matters

>>> @square
    @addsome
    def identity(x):
        return x
>>> identity(2)
46
>>> @addsome
    @square
    def identity(x):
        return x
>>> identity(2)
1936

🏵 Decorators wrap-up

they are useful if you want to simplify your code
they can be difficult sometimes
use default solutions

More decorator examples – https://wiki.python.org/moin/PythonDecoratorLibrary

🛠 functools

🤙 @lru_cache

>>> @functools.lru_cache(maxsize=64)
    def ackermann(m, n):
        # ...
 
>>> ackermann(3, 4)
125
>>> ackermann.cache_info()
CacheInfo(hits=65, misses=315, maxsize=64, currsize=64)

🤘 partial

>>> f = functools.partial(sorted, key=lambda p: p[1])
>>> f([("a", 4), ("b", 2)])
[('b', 2), ('a', 4)]
>>> g = functools.partial(sorted, [2, 3, 1, 4])
>>> g()
[1, 2, 3, 4]

🤌 singledispatch

>>> @functools.singledispatch
... def pack(obj):
...     type_name = type(obj).__name__
...     assert False, "Unsupported type: " + type_name
 
>>> @pack.register(int)
... def _(obj):
...     return b"I" + hex(obj).encode("ascii")
...
>>> @pack.register(list)
... def _(obj):
...     return b"L" + b",".join(map(pack, obj))

🤏 reduce

motivation

>>> sum([1, 2, 3, 4], start=0)
10
>>> (((0 + 1) + 2) + 3) + 4
10
>>> ((1 * 2) * 3) * 4
24

…with the help of reduce we can

>>> functools.reduce(lambda acc, x: acc * x,
... [1, 2, 3, 4])
24

By the way, not so popular in Python!

👨🏻‍💻 Viktor's notes

Explorer

3 · Classes basics · decorators · functools

In the previous episodes

🏛 Classes basics

Attributes: object and class (like in other languages)

㊙ Private attributes

🔖 dict attributes

👀 slots

🔗 Bounded and unbounded methods

@property

🍩 more @property

👩‍👧‍👧 Inheritance

👨‍👦‍👦 Inheritance

☑️ isinstance() predicate

☑️ issubclass() predicate

🎭 Do not overcomplicate with inheritance

Mixin

🏛 Classes basics wrap-up

🏵 Decorators

🚲 Let’s add some config flag

Important reminder

👀 We need to go deeper

🚲 We did it again

How about default variable?

The solution

Same, but simpler

😎 Practical decorators

@timethis

@once

@memoized

@deprecated

@pre

@post

🏵 Decorators wrap-up

🛠 functools

🤙 @lru_cache

🤘 partial

🤌 singledispatch

🤏 reduce

🏁 Any questions?

Graph View

Table of Contents

Backlinks