EuroPython 2015 - Less Known Packaging Features and Tricks (Slides)

This is a nice presentation because it presents strong opinions about how to do packaging in Python. In a way, what those opinions are do not matter. What matters is that someone has thought about the tradeoffs involved and come to a decision, so that you don’t have to spend more of your own time thinking about those tradeoffs.

Ionel Mărieș has also done a series of blog posts on packaging in Python, including The problem with packaging in Python:

In a way, packaging in Python is a victim of bad habits — complex conventions, and feature bloat. It’s hard to simplify things, because all the historical baggage people want to carry around. […] Concretely what I want is […] one way to do it. Not one clear way, cause we document the hell out of it, but one, and only one, way to do it.

Preach it.

Packaging is a hard problem in any language and made more so because all modern languages are expected to interface with C somehow or another and C has terrible build systems that always break unless you babysit them. That said, Python does a worse than average job at it:

• Python itself is highly portable, but because so many Python packages rely on C extensions, in practice Python is as difficult to get installed as C due to DLL hell.
• There are really only three places one would like to install a Python package: globally, user locally, or in a specific folder. Pip, however, is hostile to all but the choice of global installation, so users have to use virtual environments to get around this limitation.
• Installation uses both files of pure metadata (MANIFEST.in) and files of arbitrary code (setup.py), giving one the disadvantages of both.
• One wants soft requirements for libraries and hard requirements for apps, so there are two incompatible means of specifying requirements (requirements.txt and setup.py), but the hard requirements file uses the soft requirements list unless you remember to use --no-deps when running it.
• &c. &c. &c.