How to Interpret Conda Package Conflicts

How to interpret conda package conflicts?

Some Practical Advice

@Quantum7's answer gives a fine literal interpretation of Conda's conflict reporting. However, I wanted to offer a more practical take, which is that this "feature" from Conda is too non-specific to be useful in most non-trivial environments. And sometimes it won't even include the underlying conflict. Don't waste your time with it!

Conda's Conflict Reporting is Often Not Helpful

On the face of it, Conda attempts to report all possible sources of conflict. That is, all sets of paths in the dependency graph that begin from the explicit specifications and end in the same package. This amounts to most of what is reported being innocuous and frankly distracting. For example, the zlib "conflicts":

Package zlib conflicts for:
samtools -> zlib[version='1.2.11.*|>=1.2.11,<1.3.0a0|1.2.8.*|1.2.8']
samtools -> curl[version='>=7.59.0,<8.0a0'] -> zlib[version='1.2.*|1.2.11']

Since samtools depends on zlib both directly and indirectly (mediated through curl), this comes up as two alternate paths that lead to constraints. The problem is that the intersection of the final constraints are not empty, such that there is nothing incompatible here.

Furthermore, there are cases where none of what is reported is in conflict (e.g., this question or this one), which means parsing through the output could be a complete waste of time.

Try Mamba

Instead, if one is actually concerned with resolving conflicts, I find Mamba to be more effective to work with, both in speed and precision.

# install mamba
conda install -n base conda-forge::mamba

# use 'mamba' just like 'conda'
mamba create -n foo instrain awscli samtools python=3.8

Unfortunately, this example simply works now. However, there are other questions where Conda and Mamba unsatisfiability reporting is compared, e.g., this question.

Dependencies and packages conflicts in Anaconda?

Alright by searching around I was able to have everything up and running and it doesn't seem to be in conflict anymore, I had to uninstall Anaconda, reboot my computer and then reinstall it after my installation was broken though.

As long as packages and dependencies weren't messing around with each other, I was able to install lightgbm, folium and catboost in the regular (base) environment and use them. Those were installed straight with conda install -c conda-forge packagename, except for catboost which I linked. Do not forget to check for the different versions of conda, python and pip (wheel) which might affect your system.

Also, conda install nb_conda was installed to be able to select different environments in the Jupyter notebook straight away.

I got this from this helpful post and a mix of the answers below.

Then, when I wanted to install Tensorflow, Keras, theano what worked for me are the instructions in the second top comment in this thread though you should not forget to install jupyter again in the activated new environment you created.

After that, close everything, re-launch everything and in the top right corner of the Jupyter you should be able to pick the different environments and work from there.

I hope this will help someone else in the same predicament.

Found conflict while installing packages with conda

You have previously specified

python=3.10

for this environment. Most likely while creating it. This has created a pin for this python version. Now when you try to install trim-galore, the issue is that it depends, among others, on cutadapt, for which no version that supports python 3.10 exists:

conda search -c bioconda --info cutadapt=3.7
Loading channels: done
cutadapt 3.7 py37h8902056_0
---------------------------
file name   : cutadapt-3.7-py37h8902056_0.tar.bz2
name        : cutadapt
version     : 3.7
build       : py37h8902056_0
build number: 0
size        : 197 KB
license     : MIT
subdir      : linux-64
url         : https://conda.anaconda.org/bioconda/linux-64/cutadapt-3.7-py37h8902056_0.tar.bz2
md5         : 372ffd8db0a584443d817f2f91b95cdf
timestamp   : 2022-02-23 18:06:03 UTC
dependencies:
  - dnaio >=0.7.0
  - libgcc-ng >=10.3.0
  - python >=3.7,<3.8.0a0
  - python_abi 3.7.* *_cp37m
  - xopen >=1.2.0

cutadapt 3.7 py37h8902056_1
---------------------------
file name   : cutadapt-3.7-py37h8902056_1.tar.bz2
name        : cutadapt
version     : 3.7
build       : py37h8902056_1
build number: 1
size        : 197 KB
license     : MIT
subdir      : linux-64
url         : https://conda.anaconda.org/bioconda/linux-64/cutadapt-3.7-py37h8902056_1.tar.bz2
md5         : bb181439876574c985916d94dce49979
timestamp   : 2022-02-25 22:48:07 UTC
dependencies:
  - dnaio >=0.7.0
  - libgcc-ng >=10.3.0
  - python >=3.7,<3.8.0a0
  - python_abi 3.7.* *_cp37m
  - xopen >=1.2.0

cutadapt 3.7 py38hbff2b2d_0
---------------------------
file name   : cutadapt-3.7-py38hbff2b2d_0.tar.bz2
name        : cutadapt
version     : 3.7
build       : py38hbff2b2d_0
build number: 0
size        : 201 KB
license     : MIT
subdir      : linux-64
url         : https://conda.anaconda.org/bioconda/linux-64/cutadapt-3.7-py38hbff2b2d_0.tar.bz2
md5         : e38630777b8d17687134aa015ca88385
timestamp   : 2022-02-23 18:01:48 UTC
dependencies:
  - dnaio >=0.7.0
  - libgcc-ng >=10.3.0
  - python >=3.8,<3.9.0a0
  - python_abi 3.8.* *_cp38
  - xopen >=1.2.0

cutadapt 3.7 py38hbff2b2d_1
---------------------------
file name   : cutadapt-3.7-py38hbff2b2d_1.tar.bz2
name        : cutadapt
version     : 3.7
build       : py38hbff2b2d_1
build number: 1
size        : 201 KB
license     : MIT
subdir      : linux-64
url         : https://conda.anaconda.org/bioconda/linux-64/cutadapt-3.7-py38hbff2b2d_1.tar.bz2
md5         : c8101ce2f3ffcbca9b3bb2df93a9730f
timestamp   : 2022-02-25 22:44:37 UTC
dependencies:
  - dnaio >=0.7.0
  - libgcc-ng >=10.3.0
  - python >=3.8,<3.9.0a0
  - python_abi 3.8.* *_cp38
  - xopen >=1.2.0

cutadapt 3.7 py39hbf8eff0_0
---------------------------
file name   : cutadapt-3.7-py39hbf8eff0_0.tar.bz2
name        : cutadapt
version     : 3.7
build       : py39hbf8eff0_0
build number: 0
size        : 200 KB
license     : MIT
subdir      : linux-64
url         : https://conda.anaconda.org/bioconda/linux-64/cutadapt-3.7-py39hbf8eff0_0.tar.bz2
md5         : 5aaca511ac0bbc132a07a4ee9b1a7a64
timestamp   : 2022-02-23 17:57:41 UTC
dependencies:
  - dnaio >=0.7.0
  - libgcc-ng >=10.3.0
  - python >=3.9,<3.10.0a0
  - python_abi 3.9.* *_cp39
  - xopen >=1.2.0

cutadapt 3.7 py39hbf8eff0_1
---------------------------
file name   : cutadapt-3.7-py39hbf8eff0_1.tar.bz2
name        : cutadapt
version     : 3.7
build       : py39hbf8eff0_1
build number: 1
size        : 200 KB
license     : MIT
subdir      : linux-64
url         : https://conda.anaconda.org/bioconda/linux-64/cutadapt-3.7-py39hbf8eff0_1.tar.bz2
md5         : cde0658556fc58ac82a54986971020ad
timestamp   : 2022-02-25 22:41:16 UTC
dependencies:
  - dnaio >=0.7.0
  - libgcc-ng >=10.3.0
  - python >=3.9,<3.10.0a0
  - python_abi 3.9.* *_cp39
  - xopen >=1.2.0

You can also already tell in the error message the list of python versions that conda has found to be compatible

python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.5,<3.6.0a0|3.4.*']

With the pin python=3.10 in place, conda cannot downgrade your python version, but there are two simple ways around this:

• specify python as an installation target: conda install -c conda-forge -c bioconda trim-galore python to allow conda to figure out which python version to change your env to
• Create a new environment specifying what you need from the beginning: conda create -n tg -c conda-forge -c bioconda python trim-galore

How to solve package conflict on conda

Use conda-forge which has a strong dependency resolution implementation.

Newer conda versions (>=4.6) introduced a strict channel priority feature.
Type

conda config --describe channel_priority

for more information.

The solution is to add the conda-forge channel on top of defaults in your .condarc file when using conda-forge packages and activate the strict channel priority with:

$ conda config --set channel_priority strict

This will ensure that all the dependencies will come from the conda-forge channel unless they exist only on defaults.

You could also use Pipenv, and the Pipfile feature it comes with.
Pipenv will attempt to install sub-dependencies that satisfy all the requirements from your core dependencies.

see more:

https://realpython.com/pipenv-guide/

Why version conflict when building conda environment

I solved this by removing backports.zoneinfo (which isn't required in python3.9 and above because its baked into the standard library) and google-cloud-bigquery from the conda installs. google-cloud-bigquery was being being installed by pip anyway (as you can see in my original post on this thread) because conda was installing an earlier version to the one we desired (I don't know why).

I also removed tqdm because we are not using it any more. The resulting Dockerfile looks like this:

FROM debian:buster-slim

RUN apt-get update && apt-get install curl gnupg -y && rm -rf /var/lib/apt/lists/*

SHELL ["/bin/bash", "-o", "pipefail", "-c"]

RUN curl https://repo.anaconda.com/pkgs/misc/gpgkeys/anaconda.asc | gpg --dearmor > conda.gpg && \
        install -o root -g root -m 644 conda.gpg /usr/share/keyrings/conda-archive-keyring.gpg && \
        gpg --keyring /usr/share/keyrings/conda-archive-keyring.gpg --no-default-keyring \
        --fingerprint 34161F5BF5EB1D4BFBBB8F0A8AEB4F8B29D82806 && \
        echo "deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" \
        > /etc/apt/sources.list.d/conda.list

WORKDIR /tmp
RUN MINICONDA_VERSION=4.10.3 && \
        MINICONDA_SCRIPT="Miniconda3-py38_${MINICONDA_VERSION}-Linux-x86_64.sh" && \
        CONDA_VERSION='4.10.3' && \
        CONDA_DIR=/opt/conda && \
        curl -O https://repo.anaconda.com/miniconda/${MINICONDA_SCRIPT} && \
        /bin/bash ${MINICONDA_SCRIPT} -f -b -p $CONDA_DIR && \
        rm ${MINICONDA_SCRIPT} && \
        $CONDA_DIR/bin/conda config --system --set auto_update_conda false && \
        $CONDA_DIR/bin/conda config --system --set show_channel_urls true && \
        $CONDA_DIR/bin/conda config --system --remove channels defaults && \
        $CONDA_DIR/bin/conda config --system --add channels main && \
        $CONDA_DIR/bin/conda config --system --add channels conda-forge  && \
        $CONDA_DIR/bin/conda config --system --set env_prompt '({name}) ' && \
        $CONDA_DIR/bin/conda config --system --append envs_dirs /opt/conda/envs/ && \
        $CONDA_DIR/bin/conda config --system --append pkgs_dirs /opt/conda/pkgs/ && \
        $CONDA_DIR/bin/conda config --system --append aggressive_update_packages conda-forge::google-* && \
        $CONDA_DIR/bin/conda config --system --append aggressive_update_packages ca-certificates && \
        $CONDA_DIR/bin/conda config --system --append aggressive_update_packages certifi && \
        $CONDA_DIR/bin/conda config --system --append aggressive_update_packages openssl && \
        $CONDA_DIR/bin/conda update --quiet --yes --all conda="${CONDA_VERSION}" && \
        $CONDA_DIR/bin/conda create -n py3 python=3.10

RUN --mount=type=secret,id=pipconfig,dst=/etc/pip.conf \
     bash -c "source /opt/conda/bin/activate /opt/conda/envs/py3 && conda install \
                avro-python3 \
                behave \
                black \
                conda-tree \
                dirty-equals \
                flake8 \
                google-api-python-client \
                google-auth-oauthlib \
                google-cloud-bigtable \
                google-cloud-datacatalog \
                google-cloud-firestore \
                google-cloud-logging \
                google-cloud-monitoring \
                google-cloud-pubsub \
                google-cloud-secret-manager \
                google-cloud-storage \
                google-cloud-workflows \
                invoke \
                ipdb \
                jinja2 \
                jsonpath-ng \
                oauth2client \
                pandas \
                pipdeptree \
                pydantic \
                pylint \
                pytest \
                pytest-xdist \
                python-dateutil \
                responses \
                rich \
                rope \
                semantic_version \
                sh \
                sqlfluff \
                tenacity \
                zenpy \
    && pip install \
                functions-framework \
                google-cloud-bigquery \
                google-cloud-scheduler \
                object-config-models"

Interpreting package requests conflicts for a failed conda install

Just running :

conda install \
-c conda-forge \
-y \
-q \
dask-yarn>=0.7.0 \
pyarrow \
s3fs \
conda-pack \
tornado=5 \
python=3.7 \
bokeh \
fastparquet \
python-snappy \
snappy \
rapids=0.14 cudatoolkit=10.2

without updating conda config works for me in my system. All the required libraries are installed without any issues or errors. Why are you updating conda config?

Package conflict with anaconda

TLDR

conda create -n beast -c conda-forge -c bioconda python=3.7 beast2

Longer version

By telling conda that it can visit conda-forge, it has some extra options to resolve the dependency conflicts. However, that doesn't really explain why this happens:

$ conda create -n beast python=3.7 libgcc-ng=7.3 libstdcxx-ng=7.3
$ activate beast
$ conda install -c bioconda beast2
Package libstdcxx-ng conflicts for:
python=3.7 -> libstdcxx-ng[version='>=7.2.0|>=7.3.0']
beast2 -> beagle-lib -> libstdcxx-ng[version='>=7.3.0|>=7.5.0']

It isn't clear why these dependencies aren't satisfied, since we already installed libstdcxx-ng version 7.3.

Alternative

If you instead use mamba to make the environment, you get slightly more helpful feedback:

$ conda install mamba
$ mamba create -n beast-mamba python=3.7 beast2
...
Problem: nothing provides requested beast2
...
$ mamba create -n beast-mamba -c bioconda python=3.7 beast2
...
Encountered problems while solving.
Problem: nothing provides openjdk 8.0* zulu8* needed by beast2-2.4.5-0
...
$ mamba create -n beast-mamba -c bioconda -c conda-forge python=3.7 beast2
...
Success

---

Related Topics