Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor data loader #299

Merged
merged 47 commits into from
Dec 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
45ab2df
fix docs
giovp Jun 15, 2023
0a7eecb
update
giovp Jun 15, 2023
6cff536
get tile centroid and eextent outside of function
giovp Jun 16, 2023
638dd33
add return type
giovp Jun 17, 2023
1736377
move return table out of init
giovp Jun 17, 2023
1ee6abb
add comments
giovp Jun 17, 2023
288e28a
merge branch 'main' into feat/dataloader
giovp Jun 20, 2023
9eb6854
Merge branch 'main' into feat/dataloader
giovp Jun 20, 2023
9079490
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 20, 2023
38cfcff
update precommit
giovp Jun 20, 2023
5dbd876
Merge branch 'main' into feat/dataloader
giovp Jun 21, 2023
b9a5b9e
simplify return
giovp Jun 21, 2023
1d2f3cc
update tests and simplify
giovp Jun 21, 2023
dd03cdf
add tests
giovp Jun 22, 2023
f846caf
fix tests for pandas
giovp Jun 22, 2023
58b15c8
fix import and docs
giovp Jun 22, 2023
43065a9
update api
giovp Jun 22, 2023
556b87b
update import
giovp Jun 22, 2023
ffe73ec
fix imports
giovp Jun 23, 2023
05f5db7
update api
giovp Jun 23, 2023
bba86ef
fix api
giovp Jun 23, 2023
b37f40c
try fix docs
giovp Jun 23, 2023
a96b6fd
try fix docs
giovp Jun 23, 2023
ceb6f5b
add optional import of dataloader
giovp Jun 23, 2023
d290845
minor fixes
giovp Jun 23, 2023
7878379
fix test
giovp Jun 27, 2023
2f0b5e5
Merge branch 'main' into feat/dataloader
giovp Jun 29, 2023
9b84084
fixed typos
LucaMarconato Jun 29, 2023
13da142
Update src/spatialdata/dataloader/datasets.py
giovp Jul 13, 2023
13dc9be
update
giovp Jul 13, 2023
6ef34a5
update with more comments
giovp Jul 14, 2023
a65a855
fix precommit
giovp Jul 14, 2023
8ef16eb
modified docstring for transform in ImageTilesDataset
LucaMarconato Jul 14, 2023
d24bd7c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 14, 2023
61f347b
Merge branch 'main' into feat/dataloader
giovp Jul 19, 2023
f32d472
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 19, 2023
fc0528a
Merge branch 'main' into feat/dataloader
giovp Jul 26, 2023
854360d
Merge branch 'main' into feat/dataloader
giovp Nov 2, 2023
de79554
Merge branch 'main' into feat/dataloader
LucaMarconato Nov 2, 2023
77a590f
Merge branch 'main' into feat/dataloader
giovp Nov 27, 2023
95b5b49
tryf ixing docs
giovp Nov 27, 2023
16977e0
fix tests again
giovp Nov 27, 2023
4923ee6
fix tests
giovp Nov 27, 2023
63066ea
update
giovp Nov 27, 2023
000f83b
fix docs
giovp Nov 27, 2023
fa539f5
update
giovp Nov 27, 2023
63e761d
fix tests and docs and remove
giovp Nov 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ build:
python: "3.10"
sphinx:
configuration: docs/conf.py
fail_on_warning: false
fail_on_warning: true
python:
install:
- method: pip
Expand Down
8 changes: 0 additions & 8 deletions docs/_templates/autosummary/class.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,8 @@ Attributes table
~~~~~~~~~~~~~~~~~~

.. autosummary::

{% for item in attributes %}

~{{ fullname }}.{{ item }}

{%- endfor %}
{% endif %}
{% endblock %}
Expand All @@ -27,13 +24,10 @@ Methods table
~~~~~~~~~~~~~

.. autosummary::

{% for item in methods %}

{%- if item != '__init__' %}
~{{ fullname }}.{{ item }}
{%- endif -%}

{%- endfor %}
{% endif %}
{% endblock %}
Expand All @@ -46,7 +40,6 @@ Attributes
{% for item in attributes %}

.. autoattribute:: {{ [objname, item] | join(".") }}

{%- endfor %}

{% endif %}
Expand All @@ -61,7 +54,6 @@ Methods
{%- if item != '__init__' %}

.. automethod:: {{ [objname, item] | join(".") }}

{%- endif -%}
{%- endfor %}

Expand Down
14 changes: 9 additions & 5 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,10 @@ Operations on `SpatialData` objects.
match_table_to_element
concatenate
rasterize
transform
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

restoring this, since it's imported from spatialdata._core.operations

aggregate
```

### Utilities
### Operations Utilities

```{eval-rst}
.. autosummary::
Expand All @@ -49,6 +48,7 @@ The elements (building-blocks) that consitute `SpatialData`.

```{eval-rst}
.. currentmodule:: spatialdata.models

.. autosummary::
:toctree: generated

Expand All @@ -61,9 +61,11 @@ The elements (building-blocks) that consitute `SpatialData`.
TableModel
```

### Utilities
### Models Utilities

```{eval-rst}
.. currentmodule:: spatialdata.models

.. autosummary::
:toctree: generated

Expand Down Expand Up @@ -94,9 +96,11 @@ The transformations that can be defined between elements and coordinate systems
Sequence
```

### Utilities
### Transformations Utilities

```{eval-rst}
.. currentmodule:: spatialdata.transformations

.. autosummary::
:toctree: generated

Expand All @@ -119,7 +123,7 @@ The transformations that can be defined between elements and coordinate systems
ImageTilesDataset
```

## Input/output
## Input/Output

```{eval-rst}
.. currentmodule:: spatialdata
Expand Down
1 change: 0 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,6 @@ dev = [
docs = [
"sphinx>=4.5",
"sphinx-book-theme>=1.0.0",
"sphinx_rtd_theme",
"myst-nb",
"sphinxcontrib-bibtex>=1.0.0",
"sphinx-autodoc-typehints",
Expand Down
8 changes: 2 additions & 6 deletions src/spatialdata/_core/concatenate.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,12 @@

from copy import copy # Should probably go up at the top
from itertools import chain
from typing import TYPE_CHECKING, Any
from typing import Any

import numpy as np
from anndata import AnnData

if TYPE_CHECKING:
from spatialdata._core.spatialdata import SpatialData

from spatialdata._core.spatialdata import SpatialData
from spatialdata.models import TableModel

__all__ = [
Expand Down Expand Up @@ -94,8 +92,6 @@ def concatenate(
-------
The concatenated :class:`spatialdata.SpatialData` object.
"""
from spatialdata import SpatialData

merged_images = {**{k: v for sdata in sdatas for k, v in sdata.images.items()}}
if len(merged_images) != np.sum([len(sdata.images) for sdata in sdatas]):
raise KeyError("Images must have unique names across the SpatialData objects to concatenate")
Expand Down
64 changes: 40 additions & 24 deletions src/spatialdata/_core/data_extent.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,9 +115,9 @@ def get_extent(
has_labels: bool = True,
has_points: bool = True,
has_shapes: bool = True,
# python 3.9 tests fail if we don't use Union here, see
# https://github.com/scverse/spatialdata/pull/318#issuecomment-1755714287
elements: Union[list[str], None] = None, # noqa: UP007
elements: Union[ # noqa: UP007 # https://github.com/scverse/spatialdata/pull/318#issuecomment-1755714287
list[str], None
] = None,
) -> BoundingBoxDescription:
"""
Get the extent (bounding box) of a SpatialData object or a SpatialElement.
Expand All @@ -129,43 +129,50 @@ def get_extent(

Returns
-------
The bounding box description.

min_coordinate
The minimum coordinate of the bounding box.
max_coordinate
The maximum coordinate of the bounding box.
axes
The names of the dimensions of the bounding box
The names of the dimensions of the bounding box.
exact
If True, the extent is computed exactly. If False, an approximation faster to compute is given. The
approximation is guaranteed to contain all the data, see notes for details.
Whether the extent is computed exactly or not.

- If `True`, the extent is computed exactly.
- If `False`, an approximation faster to compute is given.

The approximation is guaranteed to contain all the data, see notes for details.
has_images
If True, images are included in the computation of the extent.
If `True`, images are included in the computation of the extent.
has_labels
If True, labels are included in the computation of the extent.
If `True`, labels are included in the computation of the extent.
has_points
If True, points are included in the computation of the extent.
If `True`, points are included in the computation of the extent.
has_shapes
If True, shapes are included in the computation of the extent.
If `True`, shapes are included in the computation of the extent.
elements
If not None, only the elements with the given names are included in the computation of the extent.
If not `None`, only the elements with the given names are included in the computation of the extent.

Notes
-----
The extent of a SpatialData object is the extent of the union of the extents of all its elements. The extent of a
SpatialElement is the extent of the element in the coordinate system specified by the argument `coordinate_system`.
The extent of a `SpatialData` object is the extent of the union of the extents of all its elements.
The extent of a `SpatialElement` is the extent of the element in the coordinate system
specified by the argument `coordinate_system`.

If `exact` is False, first the extent of the SpatialElement before any transformation is computed. Then, the extent
is transformed to the target coordinate system. This is faster than computing the extent after the transformation,
since the transformation is applied to extent of the untransformed data, as opposed to transforming the data and
then computing the extent.
If `exact` is `False`, first the extent of the `SpatialElement` before any transformation is computed.
Then, the extent is transformed to the target coordinate system. This is faster than computing the extent
after the transformation, since the transformation is applied to extent of the untransformed data,
as opposed to transforming the data and then computing the extent.

The exact and approximate extent are the same if the transformation doesn't contain any rotation or shear, or in the
case in which the transformation is affine but all the corners of the extent of the untransformed data
The exact and approximate extent are the same if the transformation does not contain any rotation or shear, or in
the case in which the transformation is affine but all the corners of the extent of the untransformed data
(bounding box corners) are part of the dataset itself. Note that this is always the case for raster data.

An extreme case is a dataset composed of the two points (0, 0) and (1, 1), rotated anticlockwise by 45 degrees. The
exact extent is the bounding box [minx, miny, maxx, maxy] = [0, 0, 0, 1.414], while the approximate extent is the
box [minx, miny, maxx, maxy] = [-0.707, 0, 0.707, 1.414].
An extreme case is a dataset composed of the two points `(0, 0)` and `(1, 1)`, rotated anticlockwise by 45 degrees.
The exact extent is the bounding box `[minx, miny, maxx, maxy] = [0, 0, 0, 1.414]`, while the approximate extent is
the box `[minx, miny, maxx, maxy] = [-0.707, 0, 0.707, 1.414]`.
"""
raise ValueError("The object type is not supported.")

Expand All @@ -184,7 +191,9 @@ def _(
elements: Union[list[str], None] = None, # noqa: UP007
) -> BoundingBoxDescription:
"""
Get the extent (bounding box) of a SpatialData object: the extent of the union of the extents of all its elements.
Get the extent (bounding box) of a SpatialData object.

The resulting extent is the union of the extents of all its elements.

Parameters
----------
Expand Down Expand Up @@ -259,7 +268,14 @@ def _get_extent_of_shapes(e: GeoDataFrame) -> BoundingBoxDescription:
@get_extent.register
def _(e: GeoDataFrame, coordinate_system: str = "global", exact: bool = True) -> BoundingBoxDescription:
"""
Compute the extent (bounding box) of a set of shapes.
Get the extent (bounding box) of a SpatialData object.

The resulting extent is the union of the extents of all its elements.

Parameters
----------
e
The SpatialData object.

Returns
-------
Expand Down
7 changes: 2 additions & 5 deletions src/spatialdata/_core/operations/aggregate.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from __future__ import annotations

import warnings
from typing import TYPE_CHECKING, Any
from typing import Any

import anndata as ad
import dask as da
Expand All @@ -20,6 +20,7 @@
from spatialdata._core.operations.transform import transform
from spatialdata._core.query._utils import circles_to_polygons
from spatialdata._core.query.relational_query import get_values
from spatialdata._core.spatialdata import SpatialData
from spatialdata._types import ArrayLike
from spatialdata.models import (
Image2DModel,
Expand All @@ -32,9 +33,6 @@
)
from spatialdata.transformations import BaseTransformation, Identity, get_transformation

if TYPE_CHECKING:
from spatialdata import SpatialData

__all__ = ["aggregate"]


Expand Down Expand Up @@ -236,7 +234,6 @@ def _create_sdata_from_table_and_shapes(
instance_key: str,
deepcopy: bool,
) -> SpatialData:
from spatialdata import SpatialData
from spatialdata._utils import _deepcopy_geodataframe

table.obs[instance_key] = table.obs_names.copy()
Expand Down
2 changes: 0 additions & 2 deletions src/spatialdata/_core/operations/rasterize.py
Original file line number Diff line number Diff line change
Expand Up @@ -207,8 +207,6 @@ def _(
target_height: Optional[float] = None,
target_depth: Optional[float] = None,
) -> SpatialData:
from spatialdata import SpatialData

min_coordinate = _parse_list_into_array(min_coordinate)
max_coordinate = _parse_list_into_array(max_coordinate)

Expand Down
6 changes: 2 additions & 4 deletions src/spatialdata/_core/query/relational_query.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from __future__ import annotations

from dataclasses import dataclass
from typing import TYPE_CHECKING, Any
from typing import Any

import dask.array as da
import numpy as np
Expand All @@ -11,6 +11,7 @@
from multiscale_spatial_image import MultiscaleSpatialImage
from spatial_image import SpatialImage

from spatialdata._core.spatialdata import SpatialData
from spatialdata._utils import _inplace_fix_subset_categorical_obs
from spatialdata.models import (
Labels2DModel,
Expand All @@ -22,9 +23,6 @@
get_model,
)

if TYPE_CHECKING:
from spatialdata import SpatialData


def _filter_table_by_coordinate_system(table: AnnData | None, coordinate_system: str | list[str]) -> AnnData | None:
"""
Expand Down
1 change: 0 additions & 1 deletion src/spatialdata/_core/query/spatial_query.py
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,6 @@ def _(
target_coordinate_system: str,
filter_table: bool = True,
) -> SpatialData:
from spatialdata import SpatialData
from spatialdata._core.query.relational_query import _filter_table_by_elements

min_coordinate = _parse_list_into_array(min_coordinate)
Expand Down
Loading
Loading