Click here to Skip to main content
15,910,797 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
1. What are best practices for what should be included in Python source distributions and what gets included by default?

2. What are best practices for what should be included in Python binary (wheel) distributions and what gets included by default?

(eg. source files, configuration files, test files, test data, README.md, pyproject.toml, setup.cfg, LICENSE, requirements.txt, test-requirements.txt, documentation files)

What I have tried:

I tried asking on StackOverflow, but best practices questions are not allowed there. I hope they are here.
Posted
Updated 30-Jan-22 11:26am
Comments
Richard MacCutchan 26-Jan-22 4:27am    
As with any distribution you need all the code that makes up the application. You also need to identify which other libraries are required to run it. Se also python application installer - Google Search[^].

1 solution

From here:

"I like to imagine sdists and wheel as different optimisation stages between the code that you write in a VCS repository and the files that ended up installed in the user machine, more or less like the diagram bellow:

(A) VCS checkout ---> (B) sdist ---> (C) wheel ---> (D) package installed in the final location
In general the stages (A) and (B) are mostly identical (with the exception of metadata files), so for 90% of the cases sdists will be only another way of distributing your source code. However in some cases you might want to optimise a little bit more.

The classical examples of this sdist optimisation are packages using Cython: instead of adding the .pyx files to the sdist, some developers might want to pre-compile them to C code and add only the .c files. Another example would be people writing code in Hy or coconut that like to precompile their source code to Python files and include those in the sdist.

With that in mind, I would say that the main purpose of an sdist is to distribute packages in a platform/OS agnostic way. As long as that holds, it is up to the developers to decide if they want to publish exactly the code they wrote or if they want to pre-process/pre-compile it first.

On the other hand wheels are meant to be very platform dependent. In theory you can have a package installed by unzipping the wheel archive and simply moving the files to the right directories, no compilation required.

With this introduction out of the way, my personal approach is:

sdist - Add everything I would normally have in my repository to the sdist. I do like to include my test files and docs file together with a tox.ini (to orchestrate the tests and document generation). Including LICENSE, README, pyproject.toml, setup.cfg and all the files related to the project is pretty much a MUST. If you take this approach, I highly recommend setuptools-scm.

(You can decide to leave out CI configuration files, text editor/IDE configuration files, or anything that is not strictly necessary to the package).

By default setuptools will include all .py files in the package directories, LICENSE, README, .c files required by extensions and tests/test*.py.

wheel - Only the files that are required for your package to work. This files should be IMMUTABLE / READ ONLY and placed inside the package directory (this guarantees you can use importlib.resources or its backport to find these files during runtime).

I don't recommend adding any file that can be configured by the users to the wheel, or data bases that can be written. Instead, if needed, I would write those files the first time my program launches to the user_data_dir or user_config_dir.

By default setuptools will exclude everything outside the package directory from the wheel (e.g. docs, tests, examples). If you specify include_package_data = True any kind of file inside the package directory in the sdist is added to the wheel, otherwise setuptools will only include files that can be executed by Python....

I was wrong, it is complicated what is added by default, but it should include at least these:

- README, README.txt, README.rst, README.md
- setup.py, setup.cfg
- test/test*.py
- all pure Python modules in the `py_modules` or `packages` configuration
- all files pointed by the `package_data` configuration (inside package directories)
- all C sources listed as part of extensions or C libraries in the setup script (doesn't catch C headers!)"
 
Share this answer
 
v2

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900