Deploying your own Conda python environment

There is a software carpentry lesson currently being developed that might be useful in understanding how Conda Environments work.  See Introduction to Conda for (Data) Scientists help on understanding and using Conda.

There are a number of things to consider, before using this approach.

• You should only deploy a full conda environment on the HPC system if you have hard or complex requirements.  The use and deployment of many “conda” environments can very quickly create 1000’s – 1,000,000’s of files and actually cause HPC filesystem performance issues.  Whilst there are no issues if you need to use a conda environment, it is recommended that not everyone creates one if the preinstalled python software modules or locally installed python modules options is not effective first in the first instance.

To deploy your own conda environment, you can use these instructions

Load a Miniconda software module

(in the example below, the latest version is used)

You need to load a conda software module to be able to create or activate a conda environment.

module load Miniconda3/24.7.1-0

Creating a conda environment

It is good practice to give your environment a meaningful name to help yourself remember the purpose of the environment. While naming things can be difficult, $PROJECT_NAME-env is a good convention to follow. You may also consider using the specific version of a package or why you had to create a new environment.

conda create --name python39-env python=3.9

# # To activate this environment, use # #

conda activate python36-env

# # To deactivate an active environment, use # #

conda deactivate

Activating a conda environment

To activate a conda environment, @ada $ conda activate python36-env (python36-env) bash-4.2$

Deactivate the current environment

To deactivate the environment

conda deactivate python36-env

To install additional python packages within the conda environment

Note you need to be in activated conda environment to run these commands

python36-env) $ conda install matplotlib

Collecting package metadata (current_repodata.json): done

Solving environment: done