Python

Builds and packaging tools

XKCD 1987

XKCD #1987: Python environment

Overview

In the Python Packaging User Guide:

Hatch

Hatch is maintained by PyPA (Python Packing Authority), same as other standard tools like pip, setuptools, virtualenv, and twine.

Installing

You are probably going to want to install Hatch from PyPI rather than your distros package manager, since Hatch is under active development and new versions are released somewhat freqnely.

$ python3 -m pip install --user pipx
$ pipx install hatch

You may of course also install it directly with pip install, though using pipx is a much better approach in general.

Switching project to Hatch

Moving a project from poetry to hatch is easy, but not trivial.1

Poetry

Poetry is mainly used for managing an application and its dependencies whereas Hatch is more agnostic to the project type and offers plugin-based functionality for the entire workflow (versioning, tox-like environments, publishing) so you can easily build things other than wheel/sdist, test in a Docker container, etc.

— Hacker News comment by ofekt on "Show HN: Hatch 1.0.0 - Modern extensible Python project management"

Versioning

Use poetry-bumpversion with Poetry to make managing the project's version easier. Its a plugin for Poetry itself, and thus not tied to the project (which would have been nice) or it's pyproject.toml file.

$ poetry self add poetry-bumpversion

And in your pyproject.toml, configure as needed:

[tool.poetry_bumpversion.file."${module_name}/__init__.py"]
[tool.poetry_bumpversion.file."tests/test_version.py"]

With this example it will update __version__ in ${module_name}/__init__.py to the version in pyproject.toml, and also keep the version number in tests/test_version.py up to date.

Python's is operator and None

Info

This text was originally written for a co-worker who was mainly familiar with other programming languages

The is operator in Python tests if two variables point to the same object, not if they have the same value like the == does.

The operators is and is not test for an object’s identity: x is y is true if and only if x and y are the same object.

The mechanics of objects, object instances and their identities

In Python, names are basically just labels referencing a value.

Python provides the built-in id() function to get the identity of any variable or object instance:

>>> id(True)
4381380416
>>> id(False)
4381379768

These are basically just memory addresses (or a pointer is C) stored as integers:

>>> hex(id(True))
'0x105268f40'

If you make a class and then create an instance of it and print(), repr() or str() the instance, the default string representation looks something like <__main__.Thing object at 0xdeadfood>, that is the identity of the class instance.

>>> class Test:
...   def __init__(self, a):
...     self.a = a
...
>>> t0 = Test("foo")
>>> t0
<__main__.Test object at 0x104e31db0>
>>> hex(id(t0))
'0x104e31db0'

So we got an instance of the Test class, called t0 and stored with identity (in C; the "pointer" references) 0x104e31db0.

How it works in practice

Lets create two strings (x and y) and assign the same string as a value for them:

>>> x = "these strings are the same"
>>> y = "these strings are the same"

Since this is the same string, each one should equal (==) each other:

>>> x == y
True

But internally they are stored as separate "object instances" (or, in rough terms, in separate chunks of ram). So that means that they don't have the same idenity. Thats what is tests for:

>>> x is y
False

That evaluates to False because they have different identities:

>>> id(x)
4485761520
>>> id(y)
4485761808

There is a lot of special cases

For a lot of reasons, for example because Python is a interpreted language, and because it does a lot of optimizations behind the scenes and because modern memory management is very complex, this was a very rough way of explaining this (and I am far from being an expert on how Python's internals and memory management actually works), theres a lot of "special cases".

For example, python "pre-loads" and stores single characters, very short strings, integers up to some certain value.

>>> x = "A"
>>> y = "A"

Since x and y both have the value "A", comparing them with == does as expected and returns True:

>>> x == y
True

But because of the "pre-loading", they are actually pointing at the same memory address/pre-loaded instance of str/identity:

>>> x is y
True

Because they have the same identity, and "A" itself also has the same identity:

>>> id(x)
4376398832
>>> id(y)
4376398832
>>> id("A")
4376398832

Which means that this also works:

>>> x is "A"
<stdin>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
True

Python helpfully prints a SyntaxWarning because this would be very unusual to write since "A" is a literal string. Pretty much the only reason to do this is to demonstrate exactly this.

But you can out-manouvre python's pre-loading, for example by creating one of the strings and then modify it:

>>> x = "A"
>>> y = "ABC"
>>> x += "BC"
>>> x
'ABC'
>>> y
'ABC'
>>> x == y
True
>>> x is y
False

But "ABC" is still "pre-loaded" and if you compare them directly, they will be pointing to the same identity:

>>> "ABC" is "ABC"
<stdin>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
True

How this connects to testing for None

But that is how you should be testing for None. Since there is only one instance of None, that gets pre-loaded every time the python interpreter runs, the correct way to evaluate if a variable is None is to check if it is pointing to that identity or not.

>>> id(None)
4381383120

The exact identity of None varies every time that the python interpreter starts but it will be the same for the whole time that it runs, and there will only ever exist one instance of None.

>>> def foobar():
...   return None
...
>>> a = foobar()
>>>

Since foobar() is very simple (and pointless), its fairly obvious that a was assigned None.

>>> a is None
True
>>> a is not None
False

You can still use == and compare the values

But using == still works:

>>> a == None
True

Because even though there only exists a single None, it still has the same value. And since you are comparing the same thing with itself (if both sides of == are None), it will evaluate as True.

This is just less precise, though in practice is works fine. It is also not guaranteed that it will always work in future Python versions and etc.

Decorators

Creating a decorator that accepts a named argument foo and passes it to the decorated function:

def bar_decorator(foo):
    def inner(f):
        def wrapper(*args, **kwargs):
            return f(*args, foo=foo, **kwargs)
        return wrapper
    return inner

@bar_decorator("bar")
def foobar(foo):
    return foo

Since "bar" was passed to the foobar function as foo:

>>> foobar()
'bar'

Decorators that without arguments can also be created:

def decorator(f):
    def wrapper(*args, **kwargs):
        return f(*args, bar="baz", **kwargs)
    return wrapper

@decorator
def foobar(bar):
    return bar

Now the foobar function always gets passed "baz" for its bar parameter, it is "hardcoded" in the decorator since it does not accept paramaters:

>>> foobar()
'baz'

The functools library provides update_wrapper along with the convenience function wraps for invoking it.

from functools import update_wrapper, wraps

def some_decorator(foo):
    def inner(f):
        def wrapper(*args, **kwargs):
            return f(*args, foo=foo, **kwargs)
        return update_wrapper(wrapper, f)
    return inner

def another_decorator(foo):
    def inner(f):
        @wraps(f)
        def wrapper(*args, **kwargs):
            return f(*args, bar="bar", **kwargs)
        return wrapper
    return inner

Both of these decorators work the same way as the first bar_decorator.

References