Weird conflict errors #22
Comments
|
Yes, your installer needs Python 3.7, as mentioned in the comments. |
|
@jaimergp I tried that, too, it did not work. |
|
I'm having the same problems |
|
+1 same set of problems even with 3.7 |
|
Can you link a notebook @vibhoothi? |
|
@jaimergp Thanks, I was using this, So insdie the dependencies part I just added Debug logs: Package Version Build Channel Size
────────────────────────────────────────────────────────────────────────────────
Install:
────────────────────────────────────────────────────────────────────────────────
conda-standalone 4.12.0 ha770c72_0 conda-forge/linux-64 11 MB
constructor 3.3.1 py37h89c1867_0 conda-forge/linux-64 164 KB
Upgrade:
────────────────────────────────────────────────────────────────────────────────
ca-certificates 2020.12.5 ha878542_0 installed
ca-certificates 2022.6.15 ha878542_0 conda-forge/linux-64 149 KB
certifi 2020.12.5 py37h89c1867_1 installed
certifi 2022.6.15 py37h89c1867_0 conda-forge/linux-64 155 KB
libgcc-ng 9.3.0 h2828fa1_18 installed
libgcc-ng 12.1.0 h8d9b700_16 conda-forge/linux-64 940 KB
libgomp 9.3.0 h2828fa1_18 installed
libgomp 12.1.0 h8d9b700_16 conda-forge/linux-64 459 KB
openssl 1.1.1j h7f98852_0 installed
openssl 1.1.1q h166bdaf_0 conda-forge/linux-64 2 MB
Summary:
Install: 2 packages
Upgrade: 5 packages
Total download: 15 MB
────────────────────────────────────────────────────────────────────────────────
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abortfailed
Traceback (most recent call last):
File "/usr/local/bin/constructor", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/constructor/main.py", line 246, in main
dry_run=args.dry_run, conda_exe=conda_exe)
File "/usr/local/lib/python3.7/site-packages/constructor/main.py", line 112, in main_build
fcp_main(info, verbose=verbose, dry_run=dry_run, conda_exe=conda_exe)
File "/usr/local/lib/python3.7/site-packages/constructor/fcp.py", line 403, in main
verbose, dry_run, conda_exe, transmute_file_type
File "/usr/local/lib/python3.7/site-packages/constructor/fcp.py", line 295, in _main
precs = list(solver.solve_final_state())
File "/usr/local/lib/python3.7/site-packages/conda/core/solve.py", line 281, in solve_final_state
ssc = self._run_sat(ssc)
File "/usr/local/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/usr/local/lib/python3.7/site-packages/conda/core/solve.py", line 818, in _run_sat
should_retry_solve=ssc.should_retry_solve
File "/usr/local/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
return f(*args, **kwds)
File "/usr/local/lib/python3.7/site-packages/conda/resolve.py", line 1322, in solve
self.find_conflicts(specs, specs_to_add, history_specs)
File "/usr/local/lib/python3.7/site-packages/conda/resolve.py", line 352, in find_conflicts
raise UnsatisfiableError(bad_deps, strict=strict_channel_priority)
conda.exceptions.UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versions
Package python conflicts for:
mamba -> python[version='>=2.7,<2.8.0a0|>=3.10,<3.11.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.6,<3.7.0a0']
conda -> python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.10,<3.11.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0|3.4.*']
pip -> python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.8,<3.9.0a0|>=3|>=3.6|>=3.7|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0|3.4.*']
tensorflow==2.8 -> python_abi=3.8[build=*_cp38] -> python[version='3.10.*|3.7.*|3.8.*|3.9.*']
mamba -> python_abi=3.10[build=*_cp310] -> python[version='3.10.*|3.8.*|3.9.*|3.7.*|3.6.*']
conda -> pyopenssl[version='>=16.2.0'] -> python[version='3.10.*|>=2.7|>=3.6|>=3.6,<4.0|>=3.5|3.8.*|3.9.*|3.7.*']
tensorflow==2.8 -> python[version='>=3.10,<3.11.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0']
pip -> setuptools -> python[version='!=3.0,!=3.1,!=3.2,!=3.3,!=3.4|>=3.10,<3.11.0a0|>=3.9,<3.10.0a0|2.7.*|>=3.6']
Package pypy3.6 conflicts for:
pip -> setuptools -> pypy3.6[version='7.3.0.*|7.3.1.*|7.3.2.*|7.3.3.*|>=7.3.1|>=7.3.2|>=7.3.3']
mamba -> python[version='>=3.6,<3.7.0a0'] -> pypy3.6[version='7.3.*|7.3.0.*|7.3.1.*|7.3.2.*|7.3.3.*']
mamba -> pypy3.6[version='>=7.3.1|>=7.3.2|>=7.3.3']
conda -> pypy3.6[version='>=7.3.1|>=7.3.2|>=7.3.3']
conda -> python[version='>=3.6,<3.7.0a0'] -> pypy3.6[version='7.3.*|7.3.0.*|7.3.1.*|7.3.2.*|7.3.3.*']
Package pypy3.7 conflicts for:
conda -> python[version='>=3.7,<3.8.0a0'] -> pypy3.7[version='7.3.*|7.3.3.*|7.3.4.*|7.3.5.*|7.3.7.*']
tensorflow==2.8 -> python[version='>=3.7,<3.8.0a0'] -> pypy3.7[version='7.3.3.*|7.3.4.*|7.3.5.*|7.3.7.*']
mamba -> python[version='>=3.7,<3.8.0a0'] -> pypy3.7[version='7.3.*|7.3.3.*|7.3.4.*|7.3.5.*|7.3.7.*']
pip -> python[version='>=3.7'] -> pypy3.7[version='7.3.3.*|7.3.4.*|7.3.5.*|7.3.7.*|>=7.3.7|>=7.3.5|>=7.3.3']
conda -> pypy3.7[version='>=7.3.3|>=7.3.4|>=7.3.5|>=7.3.7']
mamba -> pypy3.7[version='>=7.3.3|>=7.3.4|>=7.3.5|>=7.3.7']
Package setuptools conflicts for:
mamba -> conda[version='>=4.8'] -> setuptools[version='>=31.0.1']
python==3.7 -> pip -> setuptools
conda -> ruamel_yaml -> setuptools
pip -> setuptools
conda -> setuptools[version='>=31.0.1']
Package _openmp_mutex conflicts for:
mamba -> libgcc-ng[version='>=9.4.0'] -> _openmp_mutex[version='>=4.5']
python==3.7 -> libgcc-ng[version='>=7.3.0'] -> _openmp_mutex[version='>=4.5']
Package wheel conflicts for:
python==3.7 -> pip -> wheel
pip -> wheel
Package python_abi conflicts for:
tensorflow==2.8 -> python_abi[version='3.10.*|3.7.*|3.8.*|3.9.*',build='*_cp38|*_cp39|*_cp37m|*_cp310']
tensorflow==2.8 -> python[version='>=3.8,<3.9.0a0'] -> python_abi[version='3.7|3.8|3.9',build='*_pypy38_pp73|*_pypy39_pp73|*_pypy37_pp73']
Package six conflicts for:
conda -> conda-package-handling[version='>=1.3.0'] -> six[version='>=1.13.0,<2.0.0|>=1.5.2']
tensorflow==2.8 -> tensorflow-base==2.8.0=cuda102py38hcbbd5f6_0 -> six[version='>=1.12']
Package certifi conflicts for:
conda -> requests[version='>=2.20.1,<3'] -> certifi[version='>=2016.09|>=2016.9.26|>=2017.4.17']
pip -> setuptools -> certifi[version='>=2016.09|>=2016.9.26']
Package libcurl conflicts for:
mamba -> libcurl[version='>=7.69.1,<8.0a0|>=7.71.0,<8.0a0|>=7.71.1,<8.0a0|>=7.75.0,<8.0a0|>=7.76.0,<8.0a0|>=7.76.1,<8.0a0|>=7.77.0,<8.0a0|>=7.78.0,<8.0a0|>=7.79.1,<8.0a0']
tensorflow==2.8 -> tensorflow-base==2.8.0=cuda102py38hcbbd5f6_0 -> libcurl[version='>=7.83.1,<8.0a0']
Package conda conflicts for:
mamba -> conda[version='4.6.*,<4.13.0|4.6.*|4.7.*,<4.13.0|>=4.7.12,<4.13.0|>=4.8|>=4.8,<4.13.0|>=4.7.12,<4.8']
condaThe following specifications were found to be incompatible with your system:
- feature:/linux-64::__glibc==2.27=0
- feature:|@/linux-64::__glibc==2.27=0
- tensorflow==2.8 -> __cuda
- tensorflow==2.8 -> tensorflow-base==2.8.0=cuda110py39h3c9bc52_0 -> __glibc[version='>=2.17']
Your installed version is: 2.27 |
|
I think it has to do with the way we are specifying the versions. Instead of using This name: condacolab # you can edit this if you want
version: 0.1 # increment if you change the specs, for reproducibility!
channels:
- conda-forge
specs:
- python =3.7 # Python MUST be version 3.7
- pip
- conda
- mamba # mamba is not needed but recommended
# If any of your packages pulls in cudatoolkit:
# uncomment the line below to make sure it matches
# the output of `!echo $CUDA_VERSION` on colab
# and take only the major.minor components
# Right now, Colab uses CUDA 11.0
# - cudatoolkit =11.1
# Add your dependencies below this line
# -------------------------------------
- tensorflow =2.8
# Pip dependencies are NOT recommended. If you do need them
# uncomment the line below and edit `pip-dependencies.sh`.
# post_install: pip-dependencies.sh
# do not edit below this line
# ---------------------------
installer_type: shI'll update the example in this repo. |
|
@NightMachinery @shism2 - I suspect you had the same problem. |
|
cool that works for installation, I wonder why the tensorflow version still points to 2.9.1 (the current colab one instead of conda version) |
|
How are you deploying the installer? With I can't reproduce your case but found other issues. We are planning the implementation of a new approach that does not rely on the underlying Google Colab setup at all, which should be more robust (but less reusable). |
|
ah right, I tried that way, I uploaded to a server after creating the installer (2.2gigs) and tried to do install-from_url, now have almost all pacakge conflict when Id o Edit: Earlier I wasn't trying to install it, it was my bad, I presumed it was going to be installed. Now it tries to install. |
|
Does it work if you install it somewhere else, as in not Instead of using and let's see if that works. |
|
Yes, it do crash as expected, attached logs here, Edit: just to be clear on how I made the *.sh file is, I just added tensorflow 2.8 as a conda dep, and it generated .sh file,, and trying to do the bash invocation for the file as you advised. |
|
Thanks for the extra info. I'll try to look into this but my time is limited lately. We are working on a different approach too, as I said earlier, so there's a chance that the new attempt is more robust and we skip this problem altogether. I am also working on |
|
Interesting, thanks for update, Another update I have is, If I do a simple condacolab setup and install tensorflow on it with mamba after runtime restart, I am able to get it crashing with glibc mismatch error :), So that means, on plain conda + manual installation of TF with After we solve the above conflict error, I guess it will boil down to this problem, Also, I tried installing relevant glibc with mamba ie. with ---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
[/usr/local/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow.py](https://localhost:8080/#) in <module>
63 try:
---> 64 from tensorflow.python._pywrap_tensorflow_internal import *
65 # This try catch logic is because there is no bazel equivalent for py_extension.
ImportError: /usr/local/lib/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /usr/local/lib/python3.7/site-packages/tensorflow/python/../../../../libgrpc.so.21)
During handling of the above exception, another exception occurred:
ImportError Traceback (most recent call last)
3 frames
[<ipython-input-13-d6579f534729>](https://localhost:8080/#) in <module>
----> 1 import tensorflow
[/usr/local/lib/python3.7/site-packages/tensorflow/__init__.py](https://localhost:8080/#) in <module>
39 import sys as _sys
40
---> 41 from tensorflow.python.tools import module_util as _module_util
42 from tensorflow.python.util.lazy_loader import LazyLoader as _LazyLoader
43
[/usr/local/lib/python3.7/site-packages/tensorflow/python/__init__.py](https://localhost:8080/#) in <module>
38 # pylint: disable=wildcard-import,g-bad-import-order,g-import-not-at-top
39
---> 40 from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow
41 from tensorflow.python.eager import context
42
[/usr/local/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow.py](https://localhost:8080/#) in <module>
78 except ImportError:
79 raise ImportError(
---> 80 f'{traceback.format_exc()}'
81 f'\n\nFailed to load the native TensorFlow runtime.\n'
82 f'See https://www.tensorflow.org/install/errors '
ImportError: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
from tensorflow.python._pywrap_tensorflow_internal import *
ImportError: /usr/local/lib/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /usr/local/lib/python3.7/site-packages/tensorflow/python/../../../../libgrpc.so.21)
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors for some common causes and solutions.
If you need help, create an issue at https://github.com/tensorflow/tensorflow/issues and include the entire stack trace above this error message.
---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.
To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------Basically what I am trying to do is, creating a developer environment with pinned version of TF and some other libraries within colab so even if Google arbitrarily updates library and break things, we will still have working env because it is manual pinned versions. |
|
Yes, I ran into that too. It's not about what runtimes installed, but how the packages that depend on them are compiled. I am assuming our overwriting of The cleaner approach we are working on consists of:
Also, I might need to rebuild the installers |
|
Yes that sounds like a great plan and would love some setup like that. If we manage to do things then just zipping /opt/conda and then a simple script to activate that (4) would be ideal scenario for an end-user. But we need to make sure the libs we install are statically linked (or alteast within /opt/conda alone) which might be bit annoying thing to do. Also, after doing some stuff like i said above, restart of runtime manged to kick in old-version of Tensorflow, currently trying to see if we can have it within construct.yaml file instead of doing runtime deps which i am not sure. |
|
Okay an update now, with these packages, I am able to get tensorflow 2.8.2 working within curent running runtime and Did not require to hack with librareis/and other things to get this working. So that is a good sign:) |
|
I am now wondering whether the packaging problem is how the |
|
Yes, So yes, let's stick to
I am curious why you need this at runtime. Creating the Alternatively, you can use |
I found about condaclab from https://stackoverflow.com/questions/64464195/how-to-install-conda-on-google-drive-for-google-colab |
|
@ssurbhi560 is working on the fully isolated approach so hopefully we won't need to deal with all those workarounds. |
|
Hi @vibhoothi, #31 is merged. Check the prerelease with |
I adapted the example notebook: https://colab.research.google.com/drive/1HjikV9AS7X4eklbPtauTG_N6XNGIwOHG?usp=sharing
But I get the following errors:
I can install the packages specified in
construct.yamlmanually with no problems, so I don't think there is any conflict here at all:The text was updated successfully, but these errors were encountered: