Deploying your own Conda python environment
There is a software carpentry lesson currently being developed that might be useful in understanding how Conda Environments work. See Introduction to Conda for (Data) Scientists help on understanding and using Conda.
There are a number of things to consider, before using this approach.
- You should only deploy a full conda environment on the HPC system if you have hard or complex requirements. The use and deployment of many “conda” environments can very quickly create 1000’s – 1,000,000’s of files and actually cause HPC filesystem performance issues. Whilst there are no issues if you need to use a conda environment, it is recommended that not everyone creates one if the preinstalled python software modules or locally installed python modules options is not effective first in the first instance.
To deploy your own conda environment, you can use these instructions
Load a Miniconda software module
(in the example below, the latest version is used)
You need to load a conda software module to be able to create or activate a conda environment.
module load Miniconda3/24.7.1-0
Creating a conda environment
It is good practice to give your environment a meaningful name to help yourself remember the purpose of the environment. While naming things can be difficult, $PROJECT_NAME-env
is a good convention to follow. You may also consider using the specific version of a package or why you had to create a new environment.
conda create --name python39-env python=3.9
#
# To activate this environment, use
#
#
conda activate python36-env
#
# To deactivate an active environment, use
#
#
conda deactivate
Activating a conda environment
To activate a conda environment, @ada
$ conda activate python36-env
(python36-env) bash-4.2$
Deactivate the current environment
To deactivate the environment
conda deactivate python36-env
To install additional python packages within the conda environment
Note you need to be in activated conda environment to run these commands
python36-env) $ conda install matplotlib
Collecting package metadata (current_repodata.json): done
Solving environment: done