Python decorators, SRP, and testability

On the SRP:
For those unfamiliar with the Single Responsibility Principle (SRP), it states that there should never be more than one reason for a class to change.

That is to say: do one thing, do it well.

Decorators (not to be confused with the decorator pattern) can add behaviors or side-effects to a method, and this can be dangerous. It seems harmless, because by adding a decorator, you’re likely taking boilerplate code and shuffling it elsewhere. However, they often encourage badness because of how easy it is to add these behaviors.

On testing decorated methods:

Adding an @decorator to a python object unarguably makes the undecorated code difficult to test in isolation. Decorators are applied at compile-time, and cannot be mocked or made to not-execute without some pain.

There are certainly a few common tricks that can help test a decorated method with minimal side-effects, but they require changes to the decorator itself. There’s just plain and simple no way to un-decorate a method for isolated testing.

Let’s look at a few common examples:

@expose: register a URL for a view function in a web framework

class BlogPostController(object):
    @expose("/blog/{post_id}")
    def index(self, request, post_id=None):
        """ show a blog post """
        post = adapter.get_post(post_id)
        return render("show_entry.html", dict(post=post))

Without the `@expose`, your `show_entry` knows how to get a given post and render it in the proper template. With the decorator, it also now knows which URL corresponds to that. You now have multiple reasons for this block of code to change, including pointing to a different URL or using a different template. It’s preferred to have a separate module for managing which urls point to which views.

Harm factor: low. Ick factor: medium – high.

@cache: try to get the result from memcache, otherwise, execute the function and stick it in cache for next time

class PostAdapter(DataAdapter):
    @cache
    def get_post(self, post_id)
        """ grab a blog post from the database """
        return self.query(Post).filter(id=post_id).one() # SQLAlchemy folks need to talk to Mr Demeter...

OK this seems cool right? You only hit the database when you have to, otherwise we get it even quicker by looking it up in the cache.

What happens if you want to disable caching? A separate cache abstraction layer would reduce volatility in your data adapter.

And what does the unit test look like?

class TestGettingAPost(object):
    def setup(self):
        self.query = Mock() #don't hit the production database!
        self.post_adapter = PostAdapter(query=self.query)
    def test_getting_a_post(self):
        assert self.post_adapter.get_post(123)

Damn, there’s no way to mock out the @cache decorator so it doesn’t run. Try to, I dare you. You’re likely going to actually get post 123 from your production memcache. Crappy. The only thing you can do is make the @cache grab the cache implementation from the PostAdapter instance (and mock that out in you test), or find some other sneaky way of disabling caching for test runs. But the @cache decorator isn’t all self-contained and fun anymore.

Harm factor: medium – high.

@validate: Make sure the request matches the specified schema, otherwise hand-off to error handler

class EditPostController(object):
    def _save_error(self, request, errors):
        """ @validate decorator kicked flow here, redisplay edit page with errors """
        ...
    @validate(schema, error_handler=_save_error)
    def save(self, request)
        post = self.schema.to_python(request.params)
        self.post_adapter.save_post(post)
        return redirect("/blog/{0}".format(post_id))

Without this @validate, there would be a lot of boilerplate code inside the `save` method. With it, any unit test for the save method will be likely linked to your schema. You’d have to make an actual valid request in order to test this method. That’s outside of any tests for your schema directly. That means double-coverage but 2 tests to update when requirements shift.

Harm factor: low-medium

Conclusions (or tl;dr):

As easy as it is to become infatuated with Python decorators, they definitely encourage you to violate the SRP. This can create a myriad of problems:

• Difficultly in isolating system under test
• Added complexity to enable testing
• Redundant redundant unit tests
• Making a code module more volatile than it ought to

They still have some valid use cases and can lead to cleaner code, however, as Master Yoda once said, “when you look at the dark side, careful you must be…”