Distributing a Python Package

Posted by Marc Julian Schwarz on Friday, February 11, 2022

Distributing a Python Package

Yesterday I published my Python package watchlib [1] to PyPi, the Python Package Index. In this post I will show you how I did it and add some tips, tricks and resources that helped me a lot in the process.

Package structure

To make a package ready for distribution you will have to create a folder that will look something like this:

some_folder/
├── LICENSE
├── pyproject.toml
├── README.md
├── setup.cfg
├── src/
│   └── your_package/
│       ├── __init__.py
│       └── submodule_of_your_package.py
└── tests/

If you are using GitHub to manage your package you will most likely know what a README.md and LICENSE file is. However if you need some help with choosing an appropriate license or creating an appealing readme, here are some resources that you can check out: - choosealicense.com [2] - Writing an Formatting on GitHub [3] - GitHub flavored markdown [4]

The src folder will contain all your sources and the top level folder in there will determine the name of your package. Adding an empty __init__.py file ensures that the folder can be imported later on.

Build configuration - pyproject.toml

In the pyproject.toml file you will have to enter configuration details that will be used while building the package. To build my project I was using setuptools [5] which greatly simplifies the building process. If you want to use setuptools too, make sure to require it and define the build-backend (in the pyproject.toml file) like this:

[build-system]
requires = [
 "setuptools>=42",
 "wheel"
]
build-backend = "setuptools.build_meta"

Metadata - setup.cfg

The setup.cfg file is the configuration file for setuptools itself. All of the metadata of your package will be located here. The resource that helped me the most was the userguide from setuptools [6]. It's great.

For some inspiration, this is how my setup.cfg file looks like:

[metadata]
name = watchlib
version = 0.0.1a1
author = Marc Julian Schwarz
author_email = my@email.de
description = watchlib is a Python package providing tools for loading, visualizing and analyzing Apple Watch health data.
long_description = file: README.md
long_description_content_type = text/markdown
url = https://github.com/marcjulianschwarz/watchlib

project_urls =
	Bug Tracker = https://github.com/marcjulianschwarz/watchlib/issues
	Docs = https://github.com/marcjulianschwarz/watchlib/wiki

classifiers =
	Programming Language :: Python :: 3
	License :: OSI Approved :: MIT License
	Operating System :: OS Independent

[options]
package_dir =
	= src
packages = find:
python_requires = >=3.6
install_requires =
	pandas
	numpy
	matplotlib
	seaborn
	annoy
	sklearn

[options.packages.find]

where = src

As you can see I used the install_requires field to list all dependencies of my package. These dependencies will be installed when someone installs the package via PyPi. One last note on the version field. PEP 440 [7] goes into great detail on different version schemes, what's allowed and what not and lists some interesting version timeline examples.

Building the package

Now comes the exciting part. To build the package navigate to the folder where the pyproject.toml file is located and run this command:

python3 -m build

When building has finished a new folder, called dist, will have been added on the same level as the src folder. It contains a source archive and a built distribution.

Uploading distribution files with Twine

Before uploading the files you will have to register an account on PyPi [8].

Important side note! I would highly recommend to first register on Test PyPi [9] which is a second Python package index just for testing. Everything works exactly the same but you can treat it as a playground and try out different things.

When you have an account you will need to create an API token [10] and store it somewhere safe.

Now that everything is setup you can start uploading your package using twine like this:

twine upload dist/*

To upload to Test PyPi [9] use this command instead:

python3 -m twine upload --repository testpypi dist/*

Twine will ask you for a username and password. Enter __token__ for the username and paste the previously saved API token into the password field.

And that's it! You just distributed a Python package.

To install your package run the familiar pip command:

pip install your_package

If you used Test PyPi [9] you can install your package like this:

python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps your_package

Need help or want to chat?

I hope this guide made it a bit easier for you to distribute your own package. If there are still any questions or you just want to chat with some like-minded people, feel free to join my Discord Server [11].