7. Virtual Environments

7. Virtual Environments#

Why Virtual Environments?#

Installing externally managed packages directly into the system Python can break core operating system tools that rely on specific Python configurations. To protect against this, Python — an open-source language governed through a community-driven process called PEPs (Python Enhancement Proposals) — adopted PEP 668. Introduced in Python 3.11 and supported by modern package managers like pip, PEP 668 allows the operating system (such as Ubuntu) to mark its Python installation as externally managed. This designation prevents pip from installing packages directly into the system Python environment, thereby encouraging the use of virtual environments for package management. There are two primary types of virtual environments: Conda environments (e.g., base, envs/) and native Python virtual environments created with venv (commonly stored in a .venv directory).

Note: PEP 668’s use of the term “base environment” refers to the system Python (e.g., /usr/bin/python3), not the base environment of Conda, which is a separate self-managed Python installation.

System Python#

Lets start with an overview of what we have done. When we installed Ubuntu via WSL (Windows Subsystem Linux) we installed the system Python interpreter, and you can find where it is located by opening up the terminal (without activating any environment) and typing the following, which python3. To find the version of python type python3 --version. This is demonstrated with the following terminal session:

rebelford@DESKTOP-LA8TH4L:~$ which python3
/usr/bin/python3
rebelford@DESKTOP-LA8TH4L:~$ python3 --version
Python 3.10.12

So the system is using Python 3.10.12, which was installed when I installed Ubuntu via WSL.

Miniconda Base Python#

When we installed miniconda3 it installed its own Python Interpreter. This is what we are calling the base conda environment and we must activate the base environment with conda activate to access its interpeter. It is best not to install packages in this base environment, although it is common to install Jupyter Lab in the base environment. Instead we make virtual environments for our projects and connect the Jupyter Lab to these via the kernel.

The following terminal session activates the base environments, shows where the interpreter is and what its version is:

rebelford@DESKTOP-LA8TH4L:~$ conda activate
(base) rebelford@DESKTOP-LA8TH4L:~$ which python3
/home/rebelford/miniconda3/bin/python3
(base) rebelford@DESKTOP-LA8TH4L:~$ python3 --version
Python 3.12.1

So miniconda3 installed python 3.12.1 in the bin directory of miniconda3.

Conda Env Environments#

Conda environments are in the miniconda3/envs folder and are not part of the project direcotry. When creating a conda environment it is best not to be in any environment, including the base. This can avoide confusion between the system and the base shells.

Creating Conda Environments#

  1. Create env with a name and Python version conda create -n env_name python=X.Y

  2. Activate the environment conda activate env_name

  3. Install kernel conda install ipykernel

  4. Install other packages (give priority to conda and not pip)

  5. Register kernel python -m ipykernel install --user --name=env_name --display-name="Python_my_env(py.version)"

The following terminal session shows my active conda environments

rebelford@DESKTOP-LA8TH4L:~$ conda info --envs

# conda environments:
#
base                   /home/rebelford/miniconda3
jplab                  /home/rebelford/miniconda3/envs/jplab
mordred_env            /home/rebelford/miniconda3/envs/mordred_env
py4sci                 /home/rebelford/miniconda3/envs/py4sci
qspr                   /home/rebelford/miniconda3/envs/qspr

So you can see that the base is in the miniconda3 directory, and the four environments are in the envs subdirectories

If we activate one of the environments we can see what version of python it is running.

rebelford@DESKTOP-LA8TH4L:~$ conda activate mordred_env
(mordred_env) rebelford@DESKTOP-LA8TH4L:~$ python3 --version
Python 3.7.12

And we can see that the mordred_env environment is using python 3.7.12.

Venv Environments#

Comparing venv and conda environments#

In this class we will predominantly be using conda environments, and will try and avoid using pip commands to install packages.

🔄 Comparison Summary#

Step

Conda

venv

Create

conda create -n name python=X.Y

python -m venv .venv

Activate

conda activate name

source .venv/bin/activate

Install kernel

conda install ipykernel

pip install ipykernel

Packages

conda install ... / pip install ...

pip install ...

Kernel reg (preview)

python -m ipykernel install --user --name=name

Same

Location

Global (miniconda3/envs/)

Local (project/.venv/)

conda: Cross-language Package and Environment Manager#

General-purpose virtual environment and package manager that uses precompiled libraries (“conda packages”) to make it easy to install complex tools. This libraries can be obtained through various channels like Conda-forge. The Conda base environment is a virtual environment from the perspective of the system python and has its own python interpreter, which can be different than the system python. Each environment is stored in a special env directory that contains its own version of python. When you activate a conda environment (base or env) it’s name appears in parenthesis to the left of the command prompt.

venv: Built-in Python Tool#

Introduced in Python 3.3 it only isolates Python packages in a project directory. It cannot manage non-Python libraries of binary, and is always tied to the version of Python that it was created with. It is specific to Python.

Contents