Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow None outputs in FeatureStore #9102

Merged
merged 2 commits into from
Mar 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).

### Changed

- Allow `None` outputs in `FeatureStore.get_tensor()` - `KeyError` should now be raised based on the implementation in `FeatureStore._get_tensor()` ([#9102](https://github.com/pyg-team/pytorch_geometric/pull/9102))
- Allow mini-batching of uncoalesced sparse matrices ([#9099](https://github.com/pyg-team/pytorch_geometric/pull/9099))
- Improvements to multi-node `ogbn-papers100m` default hyperparameters and adding evaluation on all ranks ([#8823](https://github.com/pyg-team/pytorch_geometric/pull/8823))
- Changed distributed sampler and loader tests to correctly report failures in subprocesses to `pytest` ([#8978](https://github.com/pyg-team/pytorch_geometric/pull/8978))
Expand Down
32 changes: 12 additions & 20 deletions torch_geometric/data/feature_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@

import numpy as np
import torch
from torch import Tensor

from torch_geometric.typing import FeatureTensorType, NodeType
from torch_geometric.utils.mixin import CastMixin
Expand Down Expand Up @@ -329,8 +330,6 @@
Raises:
ValueError: If the input :class:`TensorAttr` is not fully
specified.
KeyError: If the tensor corresponding to the input
:class:`TensorAttr` was not found.
"""
attr = self._tensor_attr_cls.cast(*args, **kwargs)
if not attr.is_fully_specified():
Expand All @@ -339,9 +338,9 @@
f"specifying all 'UNSET' fields.")

tensor = self._get_tensor(attr)
if tensor is None:
raise KeyError(f"A tensor corresponding to '{attr}' was not found")
return self._to_type(attr, tensor) if convert_type else tensor
if convert_type:
tensor = self._to_type(attr, tensor)

Check warning on line 342 in torch_geometric/data/feature_store.py

View check run for this annotation

Codecov / codecov/patch

torch_geometric/data/feature_store.py#L342

Added line #L342 was not covered by tests
return tensor

def _multi_get_tensor(
self,
Expand Down Expand Up @@ -375,8 +374,6 @@
Raises:
ValueError: If any input :class:`TensorAttr` is not fully
specified.
KeyError: If any of the tensors corresponding to the input
:class:`TensorAttr` was not found.
"""
attrs = [self._tensor_attr_cls.cast(attr) for attr in attrs]
bad_attrs = [attr for attr in attrs if not attr.is_fully_specified()]
Expand All @@ -387,15 +384,12 @@
f"'UNSET' fields")

tensors = self._multi_get_tensor(attrs)
if any(v is None for v in tensors):
bad_attrs = [attrs[i] for i, v in enumerate(tensors) if v is None]
raise KeyError(f"Tensors corresponding to attributes "
f"'{bad_attrs}' were not found")

return [
self._to_type(attr, tensor) if convert_type else tensor
for attr, tensor in zip(attrs, tensors)
]
if convert_type:
tensors = [

Check warning on line 388 in torch_geometric/data/feature_store.py

View check run for this annotation

Codecov / codecov/patch

torch_geometric/data/feature_store.py#L388

Added line #L388 was not covered by tests
self._to_type(attr, tensor)
for attr, tensor in zip(attrs, tensors)
]
return tensors

@abstractmethod
def _remove_tensor(self, attr: TensorAttr) -> bool:
Expand Down Expand Up @@ -476,11 +470,9 @@
attr: TensorAttr,
tensor: FeatureTensorType,
) -> FeatureTensorType:
if (isinstance(attr.index, torch.Tensor)
and isinstance(tensor, np.ndarray)):
if isinstance(attr.index, Tensor) and isinstance(tensor, np.ndarray):

Check warning on line 473 in torch_geometric/data/feature_store.py

View check run for this annotation

Codecov / codecov/patch

torch_geometric/data/feature_store.py#L473

Added line #L473 was not covered by tests
return torch.from_numpy(tensor)
if (isinstance(attr.index, np.ndarray)
and isinstance(tensor, torch.Tensor)):
if isinstance(attr.index, np.ndarray) and isinstance(tensor, Tensor):

Check warning on line 475 in torch_geometric/data/feature_store.py

View check run for this annotation

Codecov / codecov/patch

torch_geometric/data/feature_store.py#L475

Added line #L475 was not covered by tests
return tensor.detach().cpu().numpy()
return tensor

Expand Down
2 changes: 1 addition & 1 deletion torch_geometric/testing/feature_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ def _get_tensor(self, attr: TensorAttr) -> Optional[Tensor]:
index, tensor = self.store.get(self.key(attr), (None, None))

if tensor is None:
return None
raise KeyError(f"Could not find tensor for '{attr}'")

assert isinstance(tensor, Tensor)

Expand Down
Loading