Testing

We try to keep our quality standards high. So, we use different tools to make this possible.

We use mypy for optional static typing. We run tests with pytest framework.

pytest

pytest is the main tool for test discovery, collection, and execution. It is configured inside setup.cfg file.

We use a lot of pytest plugins that enhance our development experience. List of these plugins is available inside pyproject.toml file.

Running:

pytest

We also have some options that are set on each run via --addopts inside the setup.cfg file.

Plugins

We use different pytest plugins to make our testing process better. Here’s the full list of things we use:

• pytest-django - plugin that introduce a lot of django specific helpers, fixtures, and configuration

• django-test-migrations - plugin to test Django migrations and their order

• pytest-cov - plugin to measure test coverage

• covdefaults - plugin for coverage to smartly ignore more meaningless lines.

• pytest-randomly - plugin to execute tests in random order and also set predictable random seed, so you can easily debug what went wrong for tests that rely on random behavior

• pytest-timeout - plugin to raise errors for tests that take too long to finish, this way you can control test execution speed

Tweaking tests performance

There are several options you can provide or remove to make your tests faster:

• You can use pytest-xdist together with -n auto to schedule several numbers of workers, sometimes when there are a lot of tests it may increase the testing speed. But on a small project with a small amount of test it just gives you an overhead, so removing it (together with --boxed) will boost your testing performance

• If there are a lot of tests with database access it may be wise to add –reuse-db option, so django won’t recreate database on each test

• If there are a lot of migrations to perform you may also add –nomigrations option, so django won’t run all the migrations and instead will inspect and create models directly

• Removing coverage. Sometimes that an option. When running tests in TDD style why would you need such a feature? So, coverage will be calculated when you will ask for it. That’s a huge speed up

• Removing linters. Sometimes you may want to split linting and testing phases. This might be useful when you have a lot of tests, and you want to run linters before, so it won’t fail your complex testing pyramid with a simple whitespace violation

mypy

Running mypy is required before any commit:

mypy server tests/**/*.py

This will eliminate a lot of possible TypeError and other issues in both server/ and tests/ directories. We use tests/**/*.py because tests/ is not a python package, so it is not importable.

However, this will not make code 100% safe from errors. So, both the testing and review process are still required.

mypy is configured via setup.cfg. Read the docs for more information.

We also use django-stubs to type django internals. This package is optional and can be removed, if you don’t want to type your django for some reason.