[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Highly variable genes for sparse dataset in backed mode #2764

Closed
3 tasks done
siberianisaev opened this issue Nov 24, 2023 · 1 comment · Fixed by #2856
Closed
3 tasks done

Highly variable genes for sparse dataset in backed mode #2764

siberianisaev opened this issue Nov 24, 2023 · 1 comment · Fixed by #2856
Labels
Bug 🐛

Comments

@siberianisaev
Copy link
siberianisaev commented Nov 24, 2023 •
edited
Loading

Please make sure these conditions are met

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of scanpy.
  • (optional) I have confirmed this bug exists on the master branch of scanpy.

What happened?

The exception happened when try to run scanpy highly_variable_genes with sparse dataset loaded in backed mode

Minimal code sample

# read backed
adata = anndata.read_h5ad(file_path, backed='r')
X = adata.raw.X if adata.raw is not None else adata.X
# dataset must be sparse there
print(issparse(X[0]))
# calculate dispersions
sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, inplace=False)

True



### Error output

```pytb
loop of ufunc does not support argument 0 of type SparseDataset which has no callable expm1 method!

goes from

scanpy/scanpy/preprocessing/_highly_variable_genes.py

Line 206 in bc349b9

X = np.expm1(X)

Versions

anndata     0.8.0
scanpy      1.9.1
-----
PIL                 9.3.0
appnope             0.1.3
asttokens           NA
backcall            0.2.0
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.3
decorator           5.1.1
entrypoints         0.4
executing           1.2.0
google              NA
h5py                3.7.0
igraph              0.10.2
ipykernel           6.17.1
jedi                0.18.2
joblib              1.2.0
kiwisolver          1.4.4
leidenalg           0.9.0
llvmlite            0.39.1
louvain             0.8.0
matplotlib          3.6.2
mpl_toolkits        NA
natsort             8.2.0
numba               0.56.4
numpy               1.23.5
packaging           21.3
pandas              1.2.1
parso               0.8.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        2.5.4
plotly              5.11.0
prompt_toolkit      3.0.33
psutil              5.9.4
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.8.0
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.13.0
pyparsing           3.0.9
pytz                2022.6
scipy               1.9.3
session_info        1.0.0
setuptools          62.3.2
sitecustomize       NA
six                 1.16.0
sklearn             1.1.3
stack_data          0.6.1
texttable           1.6.6
threadpoolctl       3.1.0
tornado             6.2
traitlets           5.5.0
typing_extensions   NA
wcwidth             0.2.5
yaml                6.0
zipp                NA
zmq                 24.0.1
-----
IPython             8.6.0
jupyter_client      7.4.7
jupyter_core        5.0.0
-----
Python 3.9.13 (main, May 24 2022, 21:28:31) [Clang 13.1.6 (clang-1316.0.21.2)]
macOS-13.4-x86_64-i386-64bit
-----
@ivirshup
Copy link
Member

I don't think we're going to get this implemented for sparse dataset per-se, but we have implemented this for dask arrays wrapping the sparse dataset in

@ivirshup ivirshup mentioned this issue Mar 21, 2024
Merged
3 tasks
@ivirshup ivirshup linked a pull request Mar 21, 2024 that will close this issue
(feat): pre-processing functions for dask with sparse chunks #2856
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug 🐛
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

(feat): pre-processing functions for dask with sparse chunks scverse/scanpy
2 participants
@siberianisaev @ivirshup