Python Development¶
Important Packages¶
# TODO
Installing Python Tools¶
Check out pipx.
To install a locally developed tool, just pipx install the path. Anything in the scripts section of the poetry project file will be available to run on the command line using the specified name.
pipx install ~/Development/cool-tool
Changes to the project will not be available to the installed version. Running install again normally does not work, but you can force reinstallation:
pipx install ~/Development/cool-tool --force
It can also install packages from pypi, and keeps them in isolated environments, if there are any external ones that might be useful.
Managing Python Versions¶
Instead of using the system Python installations, use versions managed by pyenv. There is an installer for pyenv here.
Python Pre-reqs¶
When building Python 3.7.7, bz2, readline and sqlite3 failed. Include the dev libraries to avoid this.
sudo apt-get install -y build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev python-openssl git
Installing New Interpreters¶
To see what’s available:
pyenv install --list
Install a selected version:
pyenv install 3.7.7
Switching Between Interpreters¶
Associate a python verion with the current directory (this writes the version to a file called .python-version)
pyenv local 3.7.7
To overide in the current shell:
pyenv shell 3.6.8
Not sure if global is a good idea to use tbh…
Pyenv determines which interpreter to use in the order shell > local > global.
To unset one of those, use the --unset switch. The set it to the system python, use the special version identifier system.
Dependencies and Packaging¶
Tools for managing dependencies during development and deployment, and packaging for distribution via PyPI.
Pipenv Alternatives to Investigate¶
Poetry - this shit look good. Manages packaging as well.
Poetry¶
Poetry is a package manager that is apparently somewhat ahead of pipenv in terms of functionality, and also handles distribution.
Installation¶
Auto setup script, installs apart from system python, is described here.
To update, if installed by the above method:
poetry self update
Install completions:
poetry completions bash > /etc/bash_completion.d/poetry.bash-completion
Configuration¶
Create New Project¶
This will create a directory for a basic package project using the current python interpreter:
poetry new <project-name>
README format is currently fixed as rst, unfortunately. Code for that is here.
To add a pyproject.toml to an existing project, run the following in the project directory:
poetry init
Run Commands¶
poetry run <command>
poetry shell
You can define scripts in the tool.poetry.scripts section of the pyproject.toml and run them by name:
poetry run my-script
Manage Packages¶
Many ways to reference dependencies…
poetry add pendulum
poetry add "pendulum>=2.0.5"
poetry add git+https://github.com/sdispater/pendulum.git
poetry add ./my-package/
poetry add ../my-package/dist/my_package-0.1.0.whl
To make a dependency optional, use --optional
Remove a dependency:
poetry remove pendulum
Use --dev to specify development-only dependencies.
To lock all current dependencies:
poetry lock
Install all dependencies:
poetry install
Development dependencies must be explicity excluded with --no-dev. To avoid installing the project itself, use --no-root. Don’t really understand this honestly.
Use --extras or -E to install “extras”.
Update all dependencies:
poetry update
Extras¶
If packages are marked as optional then they must also be added to a special section of the pyproject.toml in order to be installable. This section is called [tool.poetry.extras].
The entries in this section create names that can be used to install the sets of packages listed using --extras or -E.
[tool.poetry.extras]
markdown = ["markdown", "pygments"]
Docs of this feature, not linked properly in their own documentation!
Export to requirements.txt¶
poetry export -f requirements.txt > requirements.txt
Building and Distributing¶
poetry build
poetry publish
Use --repository or -r to use a repo other that the main PyPI (e.g. the test repo). -u and -p for username and password if necessary.
Using pipenv¶
Generally, keep both Pipfile and Pipfile.lock in version control.
Virtualenv creation should be automatic whenever a pipenv command is run, but I believe this command is the best way to explicitly create one with a Python 3 interpreter:
pipenv --three
Activate the virtualenv:
pipenv shell
Install packages:
pipenv install pelican
Or for an existing project, install all packages from the Pipfile:
pipenv install
Install from requirements.txt:
pipenv install -r path/to/requirements.txt
Install dev dependencies (not required for deploy/distribution):
pipenv install --dev pytest
Install editable package from a path (for development):
pipenv install --dev -e .
Lock the dependencies:
pipenv lock
Convert lockfile to requirements.txt:
pipenv lock -r
Check for newer versions of packages:
pipenv update --outdated
Update all packages (or include the package name to update individually):
pipenv update
Creating Packages on PyPI¶
Make sure to do the test uploads first, because uploads to the live package index cannot be undone!
Setuptools¶
Alternatives¶
shiv - for packaging standalone “binaries”.
The pipenv alternatives above may cover PyPI publishing as well.