[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scanorama causing segfault on CI (Python 3.9) #2866

Closed
3 tasks done
Tracked by #3124
ivirshup opened this issue Feb 19, 2024 · 2 comments · Fixed by #3125
Closed
3 tasks done
Tracked by #3124

Scanorama causing segfault on CI (Python 3.9) #2866

ivirshup opened this issue Feb 19, 2024 · 2 comments · Fixed by #3125
Labels
Bug 🐛 Development Process 🚀

Comments

@ivirshup
Copy link
Member

Please make sure these conditions are met

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of scanpy.
  • (optional) I have confirmed this bug exists on the master branch of scanpy.

What happened?

The test suite keeps failing with a segfault on the python=3.9 build. I haven't been able to reproduce locally.

Interestingly, I haven't seen it error when I rerun the check.

It looks like this always happens during the call to nn_approx

Minimal code sample

NA

Error output

platform linux -- Python 3.9.18, pytest-8.0.1, pluggy-1.4.0 -- /opt/hostedtoolcache/Python/3.9.18/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/vsts/work/1/s
configfile: pyproject.toml
testpaths: scanpy
plugins: nunit-1.0.6, mock-3.12.0
�[1mcollecting ... �[0mcollected 1474 items

scanpy/_utils/compute/is_constant.py::scanpy._utils.compute.is_constant.is_constant �[32mPASSED�[0m�[32m [  0%]�[0m
scanpy/datasets/_ebi_expression_atlas.py::scanpy.datasets._ebi_expression_atlas.ebi_expression_atlas �[32mPASSED�[0m�[32m [  0%]�[0m
scanpy/external/pl.py::scanpy.external.pl.phate �[33mSKIPPED�[0m (needs modul...)�[32m [  0%]�[0m
scanpy/external/pp/_bbknn.py::scanpy.external.pp._bbknn.bbknn �[33mSKIPPED�[0m�[32m    [  0%]�[0m
scanpy/external/pp/_harmony_integrate.py::scanpy.external.pp._harmony_integrate.harmony_integrate �[32mPASSED�[0m�[32m [  0%]�[0m
scanpy/external/pp/_hashsolo.py::scanpy.external.pp._hashsolo.hashsolo �[33mSKIPPED�[0m�[32m [  0%]�[0m
scanpy/external/pp/_magic.py::scanpy.external.pp._magic.magic �[32mPASSED�[0m�[32m     [  0%]�[0m
scanpy/external/pp/_scanorama_integrate.py::scanpy.external.pp._scanorama_integrate.scanorama_integrate Fatal Python error: Illegal instruction

Thread 0x00007f00347c4640 (most recent call first):
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/threading.py", line 316 in wait
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/threading.py", line 581 in wait
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/tqdm/_monitor.py", line 60 in run
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/threading.py", line 937 in _bootstrap

Current thread 0x00007f004e08eb80 (most recent call first):
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/scanorama/scanorama.py", line 522 in nn_approx
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/scanorama/scanorama.py", line 590 in fill_table
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/scanorama/scanorama.py", line 631 in find_alignments_table
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/scanorama/scanorama.py", line 666 in find_alignments
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/scanorama/scanorama.py", line 793 in assemble

  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/legacy_api_wrap/__init__.py", line 80 in fn_compatible
  File "<doctest scanpy.external.pp._scanorama_integrate.scanorama_integrate[6]>", line 1 in <module>
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/doctest.py", line 1334 in __run
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/doctest.py", line 1481 in run
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/doctest.py", line 1842 in run
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/doctest.py", line 304 in runtest
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/runner.py", line 174 in pytest_runtest_call
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_callers.py", line 102 in _multicall
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_manager.py", line 119 in _hookexec
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_hooks.py", line 501 in __call__
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/runner.py", line 267 in <lambda>
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/runner.py", line 346 in from_call
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/runner.py", line 266 in call_runtest_hook
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/runner.py", line 227 in call_and_report
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/runner.py", line 134 in runtestprotocol
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/runner.py", line 115 in pytest_runtest_protocol
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_callers.py", line 102 in _multicall
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_manager.py", line 119 in _hookexec
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_hooks.py", line 501 in __call__
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/main.py", line 352 in pytest_runtestloop
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_callers.py", line 102 in _multicall
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_manager.py", line 119 in _hookexec
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_hooks.py", line 501 in __call__
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/main.py", line 327 in _main
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/main.py", line 273 in wrap_session
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/main.py", line 320 in pytest_cmdline_main
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_callers.py", line 102 in _multicall
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_manager.py", line 119 in _hookexec
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/pluggy/_hooks.py", line 501 in __call__
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/config/__init__.py", line 175 in main
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/_pytest/config/__init__.py", line 198 in console_main
  File "/opt/hostedtoolcache/Python/3.9.18/x64/bin/pytest", line 8 in <module>
/home/vsts/work/_temp/1dc6f140-196e-4393-a84a-ebdaa5dcda61.sh: line 1:  1811 Illegal instruction     (core dumped) pytest

##[error]Bash exited with code '132'.
##[section]Finishing: PyTest

Versions

anndata             0.10.5.post1
annoy               1.17.3
array_api_compat    1.4.1
asciitree           0.3.3
attrs               23.2.0
cfgv                3.4.0
click               8.1.7
cloudpickle         3.0.0
contourpy           1.2.0
coverage            7.4.1
cycler              0.12.1
dask                2024.2.0
dask-glm            0.3.2
dask-ml             2023.3.24
decorator           5.1.1
Deprecated          1.2.14
distlib             0.3.8
distributed         2024.2.0
exceptiongroup      1.2.0
fasteners           0.19
fbpca               1.0
filelock            3.13.1
fonttools           4.49.0
fsspec              2024.2.0
future              0.18.3
geosketch           1.2
get-annotations     0.1.2
graphtools          1.5.3
h5py                3.10.0
harmonypy           0.0.9
identify            2.5.35
igraph              0.11.4
imageio             2.34.0
importlib-metadata  7.0.1
importlib-resources 6.1.1
iniconfig           2.0.0
intervaltree        3.1.0
Jinja2              3.1.3
joblib              1.3.2
kiwisolver          1.4.5
lazy_loader         0.3
legacy-api-wrap     1.4
leidenalg           0.10.2
llvmlite            0.42.0
locket              1.0.0
magic-impute        3.0.0
MarkupSafe          2.1.5
matplotlib          3.8.3
msgpack             1.0.7
multipledispatch    1.0.0
natsort             8.4.0
networkx            3.2.1
nodeenv             1.8.0
numba               0.59.0
numcodecs           0.12.1
numpy               1.26.4
packaging           23.2
pandas              2.0.3
partd               1.4.1
patsy               0.5.6
pbr                 6.0.0
pillow              10.2.0
pip                 24.0
platformdirs        4.2.0
pluggy              1.4.0
pre-commit          3.6.2
profimp             0.1.0
psutil              5.9.8
PyGSP               0.5.1
pynndescent         0.5.11
pyparsing           3.1.1
pytest              8.0.1
pytest-mock         3.12.0
pytest-nunit        1.0.6
python-dateutil     2.8.2
pytz                2024.1
PyYAML              6.0.1
scanorama           1.7.4
scanpy              1.10.0.dev220+g534145f6
scikit-image        0.22.0
scikit-learn        1.4.1.post1
scikit-misc         0.3.1
scipy               1.12.0
scprep              1.2.3
seaborn             0.13.2
session-info        1.0.0
setuptools          58.1.0
setuptools-scm      8.0.4
six                 1.16.0
sortedcontainers    2.4.0
sparse              0.15.1
statsmodels         0.14.1
stdlib-list         0.10.0
tasklogger          1.2.0
tblib               3.0.0
texttable           1.7.0
threadpoolctl       3.3.0
tifffile            2024.2.12
tomli               2.0.1
toolz               0.12.1
tornado             6.4
tqdm                4.66.2
typing_extensions   4.9.0
tzdata              2024.1
umap-learn          0.5.5
urllib3             2.2.1
virtualenv          20.25.0
wheel               0.42.0
wrapt               1.16.0
zarr                2.17.0
zict                3.0.0
zipp                3.17.0
@ktpolanski
Copy link
Contributor

Scanorama's nn_approx uses annoy, which is a package that has caused many a headache due to its instability. From my experience, these segfaults started showing up since 1.17.x got released, and tend to begin happening more consistently if anything is installed into the environment after annoy itself somehow. Downgrading to 1.16.3 via pip tends to make them go away.

It would be neat if there was some sort of more reliable workaround, keeping annoy stable and not bothering the user.

@flying-sheep flying-sheep mentioned this issue Jun 25, 2024
Open
4 tasks
@flying-sheep flying-sheep changed the title Scanorama causing segfault on CI Scanorama causing segfault on CI (Python 3.9) Jun 25, 2024
@flying-sheep flying-sheep mentioned this issue Jun 25, 2024
Merged
3 tasks
@flying-sheep
Copy link
Member

So we can basically just skip it, thank you for the info!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug 🐛 Development Process 🚀
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Skip scanorama on Python 3.9 scverse/scanpy
3 participants
@flying-sheep @ivirshup @ktpolanski