pip install fails with 'externally-managed-environment' on Raspberry Pi OS Bookworm

Symptom

$ pip install paho-mqtt
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install python3-xyz ...

Tutorials written for Bullseye or older say sudo pip install ..., which no longer works.

Why

From Raspberry Pi OS Bookworm onwards, pip may only install into a virtual environment. System Python belongs to apt. This follows PEP 668 from the Python community; it is not a Raspberry Pi-specific restriction.

Fix: pick one per project

A. apt, for libraries packaged by the OS

sudo apt install python3-paho-mqtt python3-smbus2

Pre-built and upgraded with the system. Not every library or version is available.

B. A venv per project (recommended for your own services)

cd /opt/sensor-gateway
python3 -m venv --system-site-packages .venv   # keeps apt packages such as gpiozero visible
.venv/bin/pip install -r requirements.txt
.venv/bin/python gateway.py

--system-site-packages matters on a Pi: hardware libraries installed with the OS (gpiozero and its pin libraries) stay importable inside the venv, and pip adds the rest.

Don't

• Don't pass --break-system-packages to get old instructions working. It can break OS tools that depend on system Python.
• Don't sudo pip install at all; pip installs belong in a venv.
• In services, call the venv's interpreter directly (/opt/app/.venv/bin/python), no activate needed. See the systemd item.