Complete python toolbox for modern developers

Complete Python toolbox for
modern developers
Jan Giacomelli
Python Web Conference 2021

About me
● Tech lead and co-founder typless.com
● Author at testdriven.io
● Twitter: @jangiacomelli

Python development
● creating and re-creating virtual environments
● installing, uninstalling, and updating dependencies
● type checking
● writing and running tests
● following consistent code style
● avoiding security vulnerabilities
● documenting code

Modern Python Environments - pyenv
Lets you easily switch between multiple versions of Python.
● Install specific Python version: $ pyenv install 3.8.5
● List available Python versions: $ pyenv versions

Modern Python Environments - pyenv
● Select a specific version as a global default: $ pyenv global 3.8.5
● Select a specific version for the current project: $ pyenv local 3.9.0

Modern Python Environments - pip + venv
Fairly simple to use. Pre-installed with most versions of Python.
● Create virtual environment: $ python -m venv my_venv
● Activate virtual environment: $ source my_venv/bin/activate
● Install a dependency: (my_venv)$ python -m pip install <package-name>
● Create list of dependencies: (my_venv)$ python -m pip freeze > requirements.txt

Modern Python Environments - pip + venv

Modern Python Environments - Poetry
Powerful CLI used for creating and managing Python projects.
● Create a new project: $ poetry new <project-name>
● Install a dependency: $ poetry add [--dev] <package name>
● Run a command inside the virtual environment: $ poetry run python -m pytest
● Dependencies are managed inside pyproject.toml

Modern Python Environments - Poetry

Modern Python Environments - pipenv
Managing dependencies and virtual environments.
● Create virtual environment: $ pipenv --python 3.8
● Install a dependency: $ pipenv install [--dev] <package name>
● Run a command inside the virtual environment: $ pipenv run python -m pytest
● Dependencies are managed inside Pipfile

Modern Python Environments - pipenv

Which one to choose?
● pip + venv
● poetry
● pipenv

Python testing - pytest
Go-to testing framework for testing Python code.
Compared to unittest:
● less boilerplate code -> more readable test suites
● uses built-in assert statement -> more readable and easier to remember
● updated more frequently (not part of the Python standard library)
● simpler setting up and tearing down test state with its fixture system
● uses a functional approach

Python testing - pytest
Plugins:
● pytest-django - set of tools made specifically for testing Django applications
● pytest-xdist - run tests in parallel
● pytest-cov - adds code coverage support
● pytest-instafail - shows failures and errors immediately instead of waiting until the end of a run

Python testing - Mocking
Practice of replacing real objects with
mocked ones, which mimic their
behavior, at runtime.

Python testing - Mocking
● pytest’s monkeypatch
● Python’s built-in untttest.mock library
○ Mock
○ MagicMock
○ create_autospec

Python testing - Code coverage
A metric that tells you the ratio between the
number of lines executed during test runs and
the total number of all lines in your code base.
pytest-cov - adds code coverage support

Code quality
Code is considered to be of high quality when:
● It serves its purpose
● Its behavior can be tested
● It follows a consistent style
● It's understandable
● It doesn't contain security vulnerabilities
● It's documented well
● It's easy to maintain

Code quality - linters
Linters flag programming errors, bugs, stylistic errors, and suspicious constructs through source code
analysis.

Code quality - flake8
Wrapper around Pyflakes, pycodestyle, and McCabe.

Code quality - formatters
Reformat your code based on a set of standards.
Code formatting is a dull job that should be performed by a computer.
● reduce merge conflicts
● easier to read code
● easier to find bugs
● easier to onboard new developers

Code quality - isort
Automatically separates imports into groups standard library, third-party, local, and alphabetically
orders them.

Code quality - black
Code formatter that's used to reformat your code based on the Black's code style guide.

Code quality - Security Vulnerability Scanners
Your code is only as secure as its weakest link.

Code quality - bandit
Tool designed to find common security issues in Python code.
● hardcoded password strings
● deserializing untrusted code
● using pass in except blocks

Code quality - safety
Used to check your installed dependencies for known security vulnerabilities against Safety database.

Type checking - type hints
Type hints allow developers to annotate expected types for variables, function parameters, and
function returns inside Python code.
● not enforced by the Python interpreter
● better express the intent
● autocomplete
● less bugs

Type checking - mypy
A tool for type checking at compile-time.

Type checking - pydantic
Uses type hints to validate data on runtime.
● easy to use
● uses type casting
● use it when dealing with external data

Type checking - Marshmallow
Helps to validate complex data and load/dump data from/to native Python types.
● doesn't use type casting
● schema and class defined separately

Type checking - Typeguard
It enforces types while your program is running.
● typechecked decorator
● comes with pytest plugin
● for classes and functions

Documenting Python Code
Without proper documentation, it can be very difficult or impossible for internal and external
stakeholders to use and/or maintain your code.
● standalone resource
● should always be present
● how and when to use something

Documenting Python Code - Docstrings
A special "string literal that occurs as the first
statement in a module, function, class, or method
definition".
● __doc__ attribute of module/class/function
● multiline and single line
● different formats (Google, NumPy,
reStructuredText, Epytext)
● code examples (doctest)

Documenting Python Code - Sphinx
Converts your project's docstrings to HTML and CSS.

Documenting Python Code - Sphinx

Documenting Python Code - OpenAPI
Standard format for describing, producing,
consuming, and visualizing RESTful APIs.
● used for SwaggerUI and ReDoc
● can be imported to Postman
● can be used to generate SDKs
● can be auto-generated

Documenting Python Code - OpenAPI

Conclusion
● creating and re-creating virtual environments - pyenv, pip + venv, poetry, pipenv
● installing, uninstalling, and updating dependencies - pip + venv, poetry, pipenv
● writing and running tests - pytest, pytest-cov, unittest.mock
● type checking - mypy, pydantic, marshmallow, typeguard
● following consistent code style - black, flake8, isort
● avoiding security vulnerabilities - bandit, security
● documenting code - Sphinx, OpenAPI
You can learn more about these tools in Complete Python Guide

Complete python toolbox for modern developers

More Related Content

What's hot

Similar to Complete python toolbox for modern developers

Recently uploaded

Complete python toolbox for modern developers

Editor's Notes