Skip to content

Python Troubleshooting Checklist

On this page, you'll find a series of questions and answers that will help you resolve some of the more common Python and conda environment problems.

Installing Python Packages

Are you seeing a permissions error during the pip install?

Yes: Install the package in your home directory:

pip install --user <package_name>

By default Python might try to install your package system-wide or in our anaconda module. Take a look at our page on installing Python packages for further instructions.

Are you seeing a "Network is unreachable" error during the pip install?

Yes: Install the package from the SuperCloud login node.

Compute nodes on SuperCloud aren't connected to the internet. You will need to be on a login node to install new packages.

Are you seeing a "Disk Quota" error during the pip install?

Yes: Use $TMPDIR to tell pip to put its temporary files elsewhere:

export TMPDIR=/state/partition1/user/$USER
mkdir $TMPDIR

If it still fails, try including the --no-cache-dir flag:

pip install --user --no-cache-dir <package_name>

After the install is complete, you can delete the temporary directory:

rm -rf $TMPDIR

Pip by default uses the /tmp directory for temporary files. This space is very small on SuperCloud so we have a pretty restrictive quota on /tmp. Telling pip to use a different location as described above will therefore solve this problem.

Conda Environments

Do you need to have your own conda environment?

Have you tried using our conda environment?

To see if the package you need is already installed in one of our anaconda modules:

module load anaconda/version
conda list packageName

Yes: See the section Installing Your Package in Your Own Conda Environment.

You can't see your environment in a Jupyter Notebook?

Did you add the jupyter package to your environment?

No: Add the jupyter package to your environment. See the section Installing Your Package in Your Own Conda Environment.

Did you install ipykernel in your conda environment?

No: Install the ipykernel package to your conda environment, then run ipykernel to "register" your conda environment.

Did you call conda init bash?

Check your $HOME/.bashrc file. If the following lines or similar appear somewhere in the file, then at some point you ran conda init bash:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/state/partition1/llgrid/pkg/anaconda/anaconda3-2019b/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/state/partition1/llgrid/pkg/anaconda/anaconda3-2022b/etc/profile.d/conda.sh" ]; then
        . "/state/partition1/llgrid/pkg/anaconda/anaconda3-2022b/etc/profile.d/conda.sh"
    else
        export PATH="/state/partition1/llgrid/pkg/anaconda/anaconda3-2022b/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

Yes: Run the conda config command below at the terminal, and then log out and log back in. You only need to do this once.

conda config --set auto_activate_base false

The conda init bash command makes a change to your .bashrc file, which gets run at startup. These lines configure your environment to use the conda command, and also auto-activate the base environment whenever you log in or start a job. The base environment is the one associated with the anaconda module you had loaded when you ran conda init bash, and can interfere with your desired environment. If running the above command and logging out and back in doesn't fix your issue, you can try to comment out these lines in your .bashrc.

Do you install consistent/compatible versions of packages?

Some versions of packages conflict with versions of other packages. Make sure you have consistent and compatible versions of your packages.

Are you running the expected version of Python?

Put which python in your submission script right before you call your python script. This will print out the full path to the python executable that is being used when you run the command python and can help verify whether you are using the expected conda environment or anaconda module. If the Python it is using is located in the expected environment, then that usually means the environment is loaded properly.

Is Python searching in the right place for your packages?

Put the lines

import sys
sys.path

at the top of your Python script. This prints out the list of directories that Python looks in when it searches for packages.

You tried everything here and you're still having problems with Python packages or conda environments?

If you've tried/confirmed all of the above items and still have problems with your Python packages or conda environments, please let us know what you've tried, and also copy, paste and send any errors that you see to supercloud@mit.edu.

References

Here is a list of webpages that were mentioned here, plus some others that might be helpful: