Creating and Publishing Your Own Python Package for Absolute Beginners

Create, build an publish a Python Package in 5 minutes

Mike Huls

Sep 23, 2023 — 6 min read

Python packages are collections of reusable code that can be easily shared and implemented across projects. We can write our code once and use it many times in many places. Packages allow us to share our code with colleagues or even the global developer community. As a data scientist, instead of sharing Jupyter notebooks, you can share packages to ensure easy updates, reusability and version control.

In this article we’ll detail the modern approach for creating, building, and publishing your own package to the Python Package Index (PyPI; where you pip install from). We’ll create a real package called “mikes-toolbox2” and deploy it to PyPI so that we can pip install mikes-toolbox2 it. Let’s code!

Before we begin…

This article details how to publish packages to a public Python Package Index. This means that, once published, your package is available to anyone. I’m in the process of writing an article on how to set up your own, private PyPI so make sure to follow me if you’re interested.

If you get lost in the code examples in this article: check out the source code here.

1. Set up a Python Package project

In this section we’ll prepare our project by making a folder and installing our virtual environment and packages.

We’ll start by creating a folder at c:/my_packages/new_package and opening this folder in a code editor. Next we’ll need to set up our virtual environment. See the article below for a deep dive on how to do this. In short: you can let PyCharm handle it or use python -m venv venv.

Lastly we’ll need to install Poetry. This package makes dependency management and packaging very easy. Install with pip install poetry.

2. Package requirements

A Python package needs certain files and folders to be present in order to be a valid package so let’s create these. Make sure your folder looks like this:

c/ 
└── my_packages/ 
    └── new_package/                <-- Project dir 
        ├── src/                    <-- source dir for our package 
        │   ├── mikes_toolbox2/     <-- the package we're building 
        │   │   └── __init__.py     <-- empty file 
        │   └── __init__.py         <-- empty file 
        ├── tests/                  <-- tests go here 
        │  └── __init__.py          <-- empty file 
        └── README.md               <-- required readme for our project

Technically we don’t need the src folder between new_package and mikes_toolbox2 but I like to do so anyway because it isolates the package-code a bit more from the source directory. I think this structure is more explicit, cleaner and easier to manage but it’s optional; you can leave out the src folder.

After creating our files and folders we need a pyproject.toml. This is universal configuration file for Python projects that is designed to be used by all kinds of tools like build systems, package managers and linters. Poetry uses pyproject.toml to keep track of dependencies.

Creating a pyproject.toml is easy since Poetry helpys us generate one with poetry init . Just answer the prompts and the pyproject is generated:

[poetry init] promps us for details about our Package (image by author)

The pyproject.toml should be located in the root of your project directory. So in my case it’s located at c:/my_packages/new_pacakge/pyproject.toml.

Maybe our project or package needs dependencies; how do we go about installing these? Poetry uses pyproject.toml to keep track of dependencies (among other things). I’ll demonstrate by installing two packages: requests and black. We use these packages a bit differently:

request is used by our package itself
black is used by us, the developer, to lint the package

The difference is that black is a dev dependency, we only need it while developing the dependency itself. So when someone pip installs our package, requests needs to be installed as well, but black doesn’t.

Here’s how to install the packages in the different ways:

poetry add requests             <-- will install regularly 
poetry add black --group dev    <-- will install as a dev dependency

In the same way I will install pytest as a dev dependency. At the end our pyproject.toml will contain the following groups of dependencies:

package dependencies and dev dependencies in pyproject.toml (image by author)

3 Package content and testing

Let’s package our functions! I’ve added the mess_up_casing function below in c:/my_pacakges/new_pacakge/src/mikes_toolbox2/my_toolbox.py:

import random 
 
 
def mess_up_casing(my_string: str) -> str: 
    """Messes up the casing of a string completely""" 
    return "".join( 
        [l.upper() if (round(random.random()) == 1) else l.lower() for l in my_string] 
    )

Next we can test this piece of code. In order to do so I’ve written a small unit test in the tests folder. Just call pytest and check if all tests succeed. Lastly we can even format all of our code with black src.

4. Building and publishing our package

Building our package has become pretty easy now that poetry is set up correctly: poetry build.

After your build succeeds you’ll see a new dist folder with a .tar.gz and a .whl file. These are the files we want to upload to pypi.

To do so, first you’ll need to register at pypi.org. After logging in you can go to your account settings and create an API token. You’ll need this token to authenticate when publishing a package. Make sure to save the token somewhere since you cannot see it again after you’ve closed the window.

Next we’ll need to configure Poetry since we’re going to use it for publishing our package; Poetry must know how to authenticate to your pypi.org account. We use the previously created API token for this:
poetry config pypi-token.pypi THETOKEN.

Publishing our package is easy now that we have a correct project structure and we’ve configured Poetry. We can just call: poetry publish and we’re done! After this step you can pip install mikes-toolbox2.

Conclusion

In this article we’ve seen that Poetry makes it very easy to package and publish code to PyPI.org. Are you interested in packaging your code but don’t want to publish it to a public PyPI? It’s also possible to host your own Python Package Index, follow me to stay tuned for a future article on this.

I hope this article was as clear as I hope it to be but if this is not the case please let me know what I can do to clarify further. In the meantime, check out my other articles on all kinds of programming-related topics like these:

Happy coding!

— Mike

P.S: like what I’m doing? Follow me!