Marie Kondo Your Python Dev Environment ✨

They don’t think it do be like it is but it do

You can also find this article in 繁體中文 here (still a WIP 🚧).

The year is 2020. Marie Kondo holds up your scattered python dependencies to the crowd. “Do these packages spark joy?” The crowd jeers, “No they do not!” She nods silently and throws them into the pit.

Why use a virtual environment?

First, let’s start with the two rules of Python development.

  1. You do not tamper with System Python
  2. You DO NOT tamper with System Python

The TL;DR reason is when you install a package into your system Python, you usually have to sudo pip install globally. This pollutes your global environment. When someone needs a slightly older version of said package, you’ll run into conflicts and suddenly everything grinds to a halt.

Using virtual environment helps you manage separate package installations for different projects and avoids breaking your system. They have their own installation directories and don’t share libraries with other virtual environments. To start, you have two options:

  1. venv
    This feature is actually built-in in Python 3.3 and later and installs pip and setuptools into created virtual environments in Python 3.4 and later.
  2. virtualenv
    This needs to be installed separately and supports Python 2.7+ and Python 3.3+, and pip, setuptools, and wheel are installed into created virtual environments by default regardless of Python version.
# using venv
$ python3 -m venv env
$ source <DIR>/bin/activate
# using virtualenv
$ virtualenv <DIR>
# source <DIR>/bin/activate

For the Python purists, one of these tools will suffice. You can read more about it here.

However, I am going to write about an opinionated setup that fits my workflow the best. If you’re indecisive like me and don’t want to weight your options, skip straight to the setup.

But there are so many tools!

Theses 8 tangled tools obviously violate PEP20 — The Zen of Python

“There should be one — and preferably only one — obvious way to do it.”

So why are we in this dilemma? Because everyone thinks that all problems in computer science can be solved by another level of indirection.

Except for the problem of too many layers of indirection. We are now left with tools that are work-arounds for work-arounds for a work-around for the design flaws in the original tool. So which one should I choose?

Do it right from the start

I only use two tools.

  1. pyenv
    pyenv is used to manage multiple Python versions so you can try out new language features or work on a different projects with a different Python version.
  2. pyenv-virtualenvwrapper
    pyenv-virtualenvwrapper provides a set of commands for you to create, delete, and jump around multiple virtualenv directories.

Setup

First, install your desired versions of Python with pyenv.

$ brew install pyenv
$ pyenv install 3.7.2
$ pyenv insall 2.7.16
$ pyenv global 3.7.2 2.7.16

The last command sets the priorities of which Python versions to use. Here I have 3.7.2 as my primary Python and the fallback is 2.7.16 .

You should also add the following lines in your .bash_profile or .zshrc .

export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

Next, install pyenv-virtualenvwrapper

$ brew install pyenv-virtualenvwrapper

and add the following to your .bash_profile or .zshrc .

export WORKON_HOME=~/.virtualenv
export PROJECT_HOME=~/workspace
pyenv virtualenvwrapper_lazy

How do I use it?

To make a new virtual env

$  mkvirtualenv [-a project_path] [-p python_version] <ENVNAME>

To list your virtualenvs

$ lsvirtualenv

To work in a virtualenv

$ workon <ENVNAME>

To remove a virtualenv

$ rmvirtualenv <ENVNAME>

That’s it. Now you can pip install -r requirements.txt with no consequences 🙌.

Hi I’m Frank 👋 I write about dev experiences related to data engineering and data science in English and Chinese. 你好,我是Frank,我會用中英文在這裡寫些關於數據科學和軟體工程有關的開發經驗。

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store