static plotting for spatial data (#437)

* Fix patching ForwardRef when generating docs * Require numba>=0.52.0 (#420) * [auto][ci skip] Generate news fragment (#420) * Fix CI cron job, remove cond in newsgen (#424) * Fix CI cron job, remove cond in newsgen * Less flaky dataset download test, fix linting * [auto][ci skip] Update dev release notes (#424) * Fix None in ligrec source/target (#434) * Drop NAs in interactions * Regenerate failing figures * [auto][ci skip] Generate news fragment (#434) * [auto][ci skip] Update dev release notes (#434) * allow Key.uns.library_id to return list * fix logic for Kye.uns.library_id * add Key.uns.image_id * fix logic Key.uns.image_id * fix logic and add scalefactors_id method * return mapping if library_id is None * rename scalefactors to scalefactor * fix logic for return type * fix haystack * init spatial pl * add docstring for _get_unique-map * Fix an import for python 3.10 (#438) * Fix an import for python 3.10 * Fix linting * [auto][ci skip] Update dev release notes (#438) * change mypy * clean spatial_attrs and add sanitize_anndata * add utils to check val in adata.obs * init def spatial * add axis * add axis * get values and color vector * fix tests * fix more tests * set titles and axis * add scatter and circles and polygons * add label * add size and handle case with no image * fix _get_coords * add img and crops * minor fix * fixes for multiple library plotting * add colormap, title and axis labels * fix cmap for shaped_collection * add scalebar * fix scalebar kwargs * fix outline * fixes * fixes for scalebar * fix logic for non-visium data * more comments * more comments * add edges * change import * add edges_kwargs * change type of crop_coord * some docs for review * add mode enum for shapes, handle partial and use spatial_conn from package constatns * handle polygons with array and fix types * more comments * more comments and fix palette handling * fix palette handling * remove print * handle cmap args better * more comments * fix _get_palette for tests * handle crops better * more comments * more comments * more comments * fix axis label * fix axis label * add scalebar to requirements and minor fixes * add scalebar to requirements and minor fixes and handle better na in categorical * remove non from na_color type * make helper function for spatial non image * minor fix * fix types * more fix types * more comments and chekc dimensions * fix check dimensions * remove 3d projection and fixes * make ref count * minor fix on _get_list * another fix on _get_list * minor fixes * minor fixes in pkc_constants * minor fix container * fix types * fix more types and change img_alpha * fix pkg constants * edges -> edge * edge -> edges * enable panel order by library or color * fix iter * make decorate axes and add img channel * add _spatial_utils module * add imports of types * minor fix on pkg_constants * add additional def * [ci skip] Update CONTRIBUTING.rst to mention dev * add type * add segmentation * add api and minor fixes * update * import crops in coord * Update _utils.py (#454) * Fixes napari slider and contrast limits (#453) * Fixes napari slider and contrast limits * Fix numpy/scipy intersphinx mapping * Use better scipy link * Try disable -n auto for 3.7 CI * [ci skip] Use Literal in ligrec * major refactor * cleanup * remove axs * Update squidpy/pl/_spatial.py Co-authored-by: Isaac Virshup <ivirshup@gmail.com> * Update squidpy/pl/_spatial_utils.py Co-authored-by: Isaac Virshup <ivirshup@gmail.com> * isaac comments * add logging error to palette * more comments * more comments and init NamedTuple * more NamedTuple * more comments * fix typing Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> * more comments * more comments * fix for annoying warning * init docs * continue docs * try delete params * continue docs * fix docs * modi gitignore * fix typing * add _color_utils * fix value_to_lot=None * change modenum to inherit str * improve docs * more improvements * fix spellcheck * fix library_id handling * add img_res_key condition for scalefactor logic * change arguments and unify spatial scatter * init cached datasets and static spatial tests * fix fixtures * fix cached tests * fix cached tests * add first images * add first images * add fixture session * add CI cache for datasets download * add CI cache for datasets download * add image * fix CI * fix CI on macos * fix docs * raise valueError when multiple library_id found but no library_key * better error message for wrong list * more tests * fix seg boundaries * fix logic for segmentation masks and boundaries * install squidpy for download data step * add library_id tests and more segment plots * add more tests * fix __init__.py * remove print statemtns * fix title * fix docs * re add Union * add spellcheck * add another test * change dpi * change figure * Fix square scatter * fixes CI * fixes CI * fix notimplemented error and docs * type test * more comments * remove columns redundandcy * fix img_res_key * fix tests * more comments * more comments * more comments * more comments types * fix tests * more comments * more comments * more comments * fix tests * fix lint * more comments * fix size from mike comments * use grayscale * check for valid types in lists * cehck for empty color list * add types from scanpy * remove edges and add conn_key * fix spelling * Fix frameon=True * Fix cmap being reset * Fix warning in groups * Fix NA color when groups!=None * Fix pet peeve - condition * Be strict with palettes * remove empty space between plots * revert back space in grid * more fixes and handle outline when na_color is present * fix docs * fix tests and cleanup * add more tests * add wrapper for func signature * modify wrapper * more comments * fix spelling * Fix not calling decorator function * Clean docs a bit * Fix cropping bug, _get_list bug, pet peeves * test and add test * add params * [ci skip] Fix linting * [ci skip] Remove dead interpolation code, docs * [ci skip] Don't use future annotations * [ci skip] Update defaults * Fix pet peeve in docs * Minor doc improvements * Add seealso's * Add option to return axes * Add test timeout * Readd autoscale_view * Fix undefined variable, pet peeves * Fix pet peeve v2 * Remove macOS 3.7 job * Add initial dask support * Regenerate figures * Fix grayscale for dask * Skip 2 tests on mac * Fix truthiness when img/seg are arrays * Fix truthiness check Co-authored-by: michalk8 <michalk8@users.noreply.github.com>
scverse · Apr 14, 2022 · e83d072 · e83d072
1 parent a249707
commit e83d072
Show file tree

Hide file tree

Showing 40 changed files with 2,239 additions and 93 deletions.
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -44,11 +44,11 @@ jobs:
         strategy:
             fail-fast: false
             matrix:
-                python: [3.7, 3.8]
-                os: [ubuntu-latest, macos-latest]
+                python: [3.7, 3.8, 3.9]
+                os: [ubuntu-latest]
                 include:
-                -   python: 3.9
-                    os: ubuntu-latest
+                -   python: 3.8
+                    os: macos-latest
         env:
             OS: ${{ matrix.os }}
             PYTHON: ${{ matrix.python }}
@@ -82,6 +82,19 @@ jobs:
                 python -m pip install --upgrade pip
                 pip install tox tox-gh-actions codecov
 
+        -   name: Restore data cache
+            id: data-cache
+            uses: actions/cache@v2
+            with:
+                path: |
+                    ~/.cache/squidpy/*.h5ad
+                key: data-${{ hashFiles('**/download_data.py') }}
+
+        -   name: Download datasets
+            if: steps.data-cache.outputs.cache-hit != 'true'
+            run: |
+                tox -e download-data
+
         # caching .tox is not encouraged, but since we're private and this shaves off ~1min from the step
         # if any problems occur and/or once the package is public, this can be removed
         -   name: Restore tox cache
@@ -91,6 +104,7 @@ jobs:
                 key: tox-${{ runner.os }}-${{ env.pythonLocation }}-${{ hashFiles('**/requirements.txt', '**/tox.ini') }}
 
         -   name: Test
+            timeout-minutes: 30
             env:
                 MPLBACKEND: agg
                 PLATFORM: ${{ matrix.os }}

diff --git a/.gitignore b/.gitignore
@@ -152,3 +152,6 @@ docs/source/external_tutorials
 
 # Vim .swp
 *.swp
+
+# vscode
+.vscode
diff --git a/.mypy.ini b/.mypy.ini
@@ -3,6 +3,7 @@ mypy_path = squidpy
 python_version = 3.8
 plugins = numpy.typing.mypy_plugin
 
+ignore_errors = False
 warn_redundant_casts = True
 warn_unused_configs = True
 warn_unused_ignores = True

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -11,7 +11,7 @@ repos:
     hooks:
     -   id: mypy
         additional_dependencies: [numpy==1.21.0, scipy, pandas, types-requests]
-        exclude: squidpy/datasets/_(dataset|image).py   # See https://github.com/pre-commit/mirrors-mypy/issues/33
+        exclude: .scripts/ci/download_data.py|squidpy/datasets/_(dataset|image).py   # See https://github.com/pre-commit/mirrors-mypy/issues/33
 -   repo: https://github.com/psf/black
     rev: 22.3.0
     hooks:

diff --git a/.scripts/ci/download_data.py b/.scripts/ci/download_data.py
@@ -0,0 +1,62 @@
+#!/usr/bin/env python3
+from typing import Any
+from pathlib import Path
+import argparse
+
+_CNT = 0  # increment this when you want to rebuild the CI cache
+_ROOT = Path.home() / ".cache" / "squidpy"
+
+
+def _print_message(func_name: str, path: Path, *, dry_run: bool = False) -> None:
+    prefix = "[DRY RUN]" if dry_run else ""
+    if path.is_file():
+        print(f"{prefix}[Loading]     {func_name:>25} <- {str(path):>25}")
+    else:
+        print(f"{prefix}[Downloading] {func_name:>25} -> {str(path):>25}")
+
+
+def _maybe_download_data(func_name: str, path: Path) -> Any:
+    import squidpy as sq
+
+    try:
+        return getattr(sq.datasets, func_name)(path=path)
+    except Exception as e:
+        print(f"File {str(path):>25} seems to be corrupted: {e}. Removing and retrying")
+        path.unlink()
+
+        return getattr(sq.datasets, func_name)(path=path)
+
+
+def main(args: argparse.Namespace) -> None:
+    from anndata import AnnData
+
+    import squidpy as sq
+
+    all_datasets = sq.datasets._dataset.__all__ + sq.datasets._image.__all__
+    all_extensions = ["h5ad"] * len(sq.datasets._dataset.__all__) + ["tiff"] * len(sq.datasets._image.__all__)
+
+    if args.dry_run:
+        for func_name, ext in zip(all_datasets, all_extensions):
+            path = _ROOT / f"{func_name}.{ext}"
+            _print_message(func_name, path, dry_run=True)
+        return
+
+    # could be parallelized, but on CI it largely does not matter (usually limited to 2 cores + bandwidth limit)
+    for func_name, ext in zip(all_datasets, all_extensions):
+        path = _ROOT / f"{func_name}.{ext}"
+
+        _print_message(func_name, path)
+        obj = _maybe_download_data(func_name, path)
+
+        # we could do without the AnnData check as well (1 less req. in tox.ini), but it's better to be safe
+        assert isinstance(obj, (AnnData, sq.im.ImageContainer)), type(obj)
+        assert path.is_file(), path
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Download data used for tutorials/examples.")
+    parser.add_argument(
+        "--dry-run", action="store_true", help="Do not download any data, just print what would be downloaded."
+    )
+
+    main(parser.parse_args())
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
@@ -20,6 +20,7 @@ Clone Squidpy from source as::
 
     git clone https://github.com/theislab/squidpy
     cd squidpy
+    git checkout dev
 
 Install the test and development mode::
 

diff --git a/docs/source/api.rst b/docs/source/api.rst
@@ -46,6 +46,8 @@ Plotting
 .. autosummary::
     :toctree: api
 
+    pl.spatial_scatter
+    pl.spatial_segment
     pl.nhood_enrichment
     pl.centrality_scores
     pl.interaction_matrix

diff --git a/docs/source/spelling_wordlist.txt b/docs/source/spelling_wordlist.txt
@@ -12,6 +12,7 @@ centroid
 centroids
 colorectal
 colormap
+colorbar
 Colormap
 cond
 conda
@@ -76,12 +77,14 @@ Rescale
 ResNet
 RGB
 sagittal
+semibold
 sc
 scalability
 Scanpy
 segmentations
 seqFISH
 seqV
+seg
 Squidpy
 StarDist
 stromal
@@ -96,3 +99,5 @@ uncommenting
 Visium
 Vizgen
 zscore
+scalebar
+kwargs
diff --git a/requirements.txt b/requirements.txt
@@ -5,6 +5,7 @@ dask[array]>=2021.02.0
 docrep>=0.3.1
 fsspec>=2021.11.0
 leidenalg>=0.8.2
+matplotlib-scalebar>=0.8.0
 networkx>=2.6.0
 numba>=0.52.0
 numpy>=1.18.0

diff --git a/squidpy/_constants/_constants.py b/squidpy/_constants/_constants.py
@@ -107,6 +107,13 @@ class RipleyStat(ModeEnum):
     L = "L"
 
 
+@unique
+class ScatterShape(str, ModeEnum):
+    CIRCLE = "circle"
+    SQUARE = "square"
+    HEX = "hex"
+
+
 @unique
 class TenxVersions(str, ModeEnum):
     # Version numbers as class objects of TenxVersions

diff --git a/squidpy/_constants/_pkg_constants.py b/squidpy/_constants/_pkg_constants.py
@@ -1,5 +1,7 @@
 """Internal constants not exposed to the user."""
-from typing import Any, Union, Callable, Optional
+from __future__ import annotations
+
+from typing import Any, Union, Mapping, Callable, Optional, Sequence
 
 from anndata import AnnData
 
@@ -72,6 +74,22 @@ def spatial(cls) -> str:
         def image_key(cls) -> str:
             return "images"
 
+        @cprop
+        def image_res_key(cls) -> str:
+            return "hires"
+
+        @cprop
+        def image_seg_key(cls) -> str:
+            return "segmentation"
+
+        @cprop
+        def scalefactor_key(cls) -> str:
+            return "scalefactors"
+
+        @cprop
+        def size_key(cls) -> str:
+            return "spot_diameter_fullres"
+
         @classmethod
         def spatial_neighs(cls, value: Optional[str] = None) -> str:
             return f"{Key.obsm.spatial}_neighbors" if value is None else f"{value}_neighbors"
@@ -104,24 +122,6 @@ def ripley(cls, cluster: str, mode: str) -> str:
         def colors(cls, cluster: str) -> str:
             return f"{cluster}_colors"
 
-        @classmethod
-        def library_id(cls, adata: AnnData, spatial_key: str, library_id: Optional[str] = None) -> str:
-            if spatial_key not in adata.uns:
-                raise KeyError(f"Spatial key `{spatial_key}` not found in `adata.uns`.")
-            haystack = list(adata.uns[spatial_key].keys())
-            if library_id is None:
-                if len(haystack) > 1:
-                    raise ValueError(
-                        f"Unable to determine which library id to use. "
-                        f"Please specify one from: `{sorted(haystack)}`."
-                    )
-                library_id = haystack[0]
-
-            if library_id not in haystack:
-                raise KeyError(f"Library id `{library_id}` not found in `{sorted(haystack)}`.")
-
-            return library_id
-
         @classmethod
         def spot_diameter(
             cls,
@@ -138,6 +138,61 @@ def spot_diameter(
                     f"`adata.uns[{spatial_key!r}][{library_id!r}]['scalefactors'][{spot_diameter_key!r}]]`"
                 ) from None
 
+        @classmethod
+        def library_id(
+            cls,
+            adata: AnnData,
+            spatial_key: str,
+            library_id: Sequence[str] | str | None = None,
+            return_all: bool = False,
+        ) -> Sequence[str] | str | None:
+            library_id = cls._sort_haystack(adata, spatial_key, library_id, sub_key=None)
+            if return_all or library_id is None:
+                return library_id
+            if len(library_id) != 1:
+                raise ValueError(
+                    f"Unable to determine which library id to use. Please specify one from: `{sorted(library_id)}`."
+                )
+            return library_id[0]
+
+        @classmethod
+        def library_mapping(
+            cls,
+            adata: AnnData,
+            spatial_key: str,
+            sub_key: str,
+            library_id: Sequence[str] | str | None = None,
+        ) -> Mapping[str, Sequence[str]]:
+            library_id = cls._sort_haystack(adata, spatial_key, library_id, sub_key)
+            if library_id is None:
+                raise ValueError("Invalid `library_id=None`")
+            return {i: list(adata.uns[spatial_key][i][sub_key]) for i in library_id}
+
+        @classmethod
+        def _sort_haystack(
+            cls,
+            adata: AnnData,
+            spatial_key: str,
+            library_id: Sequence[str] | str | None = None,
+            sub_key: Optional[str] = None,
+        ) -> Sequence[str] | None:
+            if spatial_key not in adata.uns:
+                raise KeyError(f"Spatial key {spatial_key!r} not found in `adata.uns`.")
+            haystack = list(adata.uns[spatial_key])
+            if library_id is not None:
+                if isinstance(library_id, str):
+                    library_id = [library_id]
+                if not any(i in library_id for i in haystack):
+                    raise KeyError(f"`library_id`: {library_id}` not found in `{sorted(haystack)}`.")
+                if sub_key is not None:
+                    if not all(sub_key in lib for lib in [adata.uns[spatial_key][lib] for lib in library_id]):
+                        raise KeyError(
+                            f"`{sub_key}` not found in `adata.uns[{spatial_key!r}]['library_id'])` "
+                            f"with following `library_id`: {library_id}."
+                        )
+                return library_id
+            return haystack
+
     class obsp:
         @classmethod
         def spatial_dist(cls, value: Optional[str] = None) -> str: