October 21, 2020

Django allows for nested transactions, with each inner transaction backed by a database's "savepoint" feature. This is a method used to partially roll back a transaction, or commit a new series of steps to a transaction. Transactions within transactions.

For example:

with transaction.atomic():  # Outer transaction
    with transaction.atomic():  # Inner transaction/savepoint
        with connection.cursor() as cursor:
            cursor.execute('...')

MySQL/MariaDB has a behavioral quirk where certain operations performed on a database within a transaction (such as a CREATE TABLE or DROP TABLE) will auto-commit the current transaction.

If this is done in an inner transaction, it will fail with an error about a missing savepoint:

django.db.utils.OperationalError: (1305, 'SAVEPOINT s4463404480_x2 does not exist')

While not a problem for the vast majority of Django users, this can be an issue if you're working with something more specialized (we hit it in Django Evolution).

The reason this fails is that, upon issuing a CREATE TABLE or similar, the current transaction will be committed and any savepoints will be cleared. Django doesn't know this, though, and works under the assumption that the savepoints still remain.

When the inner transaction completes successfully, atomic() will attempt to commit the savepoint to the database. This will fail, since the savepoint has been cleared away. Seeing that, it will then try to roll back the inner transaction and mark an error on the outer transaction. The rollback will, predictably, fail, since the changes have already been committed. Giving up, it just raises the above error.

Deeply confusing issue, but hopefully this sheds some light on what's happening behind the scenes.