Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

6834 Add MetaProterties and update load API #6835

Merged
merged 70 commits into from
Aug 30, 2023
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
faa8dc6
add `AutoBundle` and `from_bundle`
KumoLiu Aug 8, 2023
efcb485
Merge remote-tracking branch 'origin/dev' into from-bundle
KumoLiu Aug 8, 2023
be7aa45
minor fix
KumoLiu Aug 8, 2023
e83478a
Merge branch 'dev' into from-bundle
KumoLiu Aug 8, 2023
7c26f75
Merge branch 'dev' into from-bundle
KumoLiu Aug 10, 2023
a7bc4e8
rename to `BundleManager`
KumoLiu Aug 10, 2023
5dc2f4e
Merge branch 'from-bundle' of https://github.com/KumoLiu/MONAI into f…
KumoLiu Aug 10, 2023
b8bc55a
support multi configs
KumoLiu Aug 10, 2023
eb6a9d4
add docstring
KumoLiu Aug 10, 2023
fc677b5
change `bundle_name_or_path`
KumoLiu Aug 11, 2023
e312186
Merge branch 'dev' into from-bundle
KumoLiu Aug 11, 2023
8fb5076
support flexible postfix
KumoLiu Aug 11, 2023
e4d061f
minor fix
KumoLiu Aug 11, 2023
d49f3d0
minor fix
KumoLiu Aug 11, 2023
dc8858b
update based on comments
KumoLiu Aug 11, 2023
cc5c2c7
fix flake8
KumoLiu Aug 11, 2023
52c9f52
update docstring
KumoLiu Aug 11, 2023
5eec914
update based on comments
KumoLiu Aug 13, 2023
42bac3d
fix flake8
KumoLiu Aug 13, 2023
1623a6d
Merge branch 'dev' of https://github.com/Project-MONAI/MONAI into fro…
KumoLiu Aug 14, 2023
0903997
rename `bundle_name` to `name`
KumoLiu Aug 14, 2023
6576c15
update docstring
KumoLiu Aug 14, 2023
0409cac
add `MetaProterties`
KumoLiu Aug 15, 2023
7057fe2
support getting meta property
KumoLiu Aug 23, 2023
5b484ec
Merge branch 'dev' of https://github.com/Project-MONAI/MONAI into fro…
KumoLiu Aug 23, 2023
7ab0a74
remove `BundleManager`
KumoLiu Aug 23, 2023
79bf87d
simplify the specification for
KumoLiu Aug 23, 2023
1a2abea
fix flake8
KumoLiu Aug 23, 2023
a8ae128
fix mypy
KumoLiu Aug 23, 2023
c0c4e01
simplify the specification for
KumoLiu Aug 23, 2023
fcc8b1b
add `create_workflow` and update `load`
KumoLiu Aug 23, 2023
30ebb16
remove `_find_config_file`
KumoLiu Aug 23, 2023
f609f98
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 23, 2023
ee525f8
update docstring
KumoLiu Aug 23, 2023
a9a1163
Merge branch 'from-bundle' of https://github.com/KumoLiu/MONAI into f…
KumoLiu Aug 23, 2023
83cd3d3
minor fix
KumoLiu Aug 24, 2023
559711d
add unittests for load
KumoLiu Aug 24, 2023
7038b90
update the docstring for the `load`
KumoLiu Aug 24, 2023
a5992b9
fix flake8
KumoLiu Aug 24, 2023
0531989
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 24, 2023
a5cfd71
fix mypy
KumoLiu Aug 24, 2023
2229c4d
Merge branch 'from-bundle' of https://github.com/KumoLiu/MONAI into f…
KumoLiu Aug 24, 2023
82a3c23
fix ci
KumoLiu Aug 24, 2023
0ad2364
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 24, 2023
6938718
fix ci
KumoLiu Aug 24, 2023
5da4ea6
Update monai/bundle/__init__.py
KumoLiu Aug 25, 2023
85275d3
fix typo
KumoLiu Aug 25, 2023
63d3a84
update according to comments
KumoLiu Aug 25, 2023
864dadb
fix flake8
KumoLiu Aug 25, 2023
fa3b940
Update monai/bundle/workflows.py
KumoLiu Aug 25, 2023
54c65af
minor fix
KumoLiu Aug 25, 2023
384ff08
Merge branch 'from-bundle' of https://github.com/KumoLiu/MONAI into f…
KumoLiu Aug 25, 2023
c4ac467
fix version ci error
KumoLiu Aug 28, 2023
b267d76
fix flake8
KumoLiu Aug 28, 2023
05e51e5
set "meta_file" as required in `ConfigWorkflow`
KumoLiu Aug 28, 2023
bc3a134
fix flake8
KumoLiu Aug 28, 2023
dcd76dd
minor fix
KumoLiu Aug 28, 2023
3afb12d
Merge branch 'dev' of https://github.com/Project-MONAI/MONAI into fro…
KumoLiu Aug 28, 2023
8e74d04
fix unittests
KumoLiu Aug 28, 2023
fe76258
address comments
KumoLiu Aug 29, 2023
e5c8bfd
address comments
KumoLiu Aug 29, 2023
2789c5f
fix flake8
KumoLiu Aug 29, 2023
561a68e
add deprecated_arg for `net_name`
KumoLiu Aug 30, 2023
9b6bc08
Merge branch 'dev' into from-bundle
KumoLiu Aug 30, 2023
0039860
update `worklfow` to `workflow_type`
KumoLiu Aug 30, 2023
32de5aa
Merge branch 'from-bundle' of https://github.com/KumoLiu/MONAI into f…
KumoLiu Aug 30, 2023
134d1ea
fix flake8
KumoLiu Aug 30, 2023
e93309e
fix ci
KumoLiu Aug 30, 2023
edb8768
address comments
KumoLiu Aug 30, 2023
93a149a
Merge branch 'dev' into from-bundle
wyli Aug 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions monai/bundle/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
from .properties import InferProperties, TrainProperties
from .reference_resolver import ReferenceResolver
from .scripts import (
BundleManager,
ckpt_export,
download,
get_all_bundles_list,
Expand Down
123 changes: 123 additions & 0 deletions monai/bundle/scripts.py
Original file line number Diff line number Diff line change
Expand Up @@ -1539,3 +1539,126 @@
copyfile(str(ckpt_file), str(models_dir / "model.pt"))
elif network is not None:
save_state(network, str(models_dir / "model.pt"))


def _find_bundle_file(root_dir: Path, file_name: str, suffix: Sequence[str] = ("json", "yaml", "yml")) -> str | None:
# find bundle file with possible suffix
for _suffix in suffix:
full_name = f"{file_name}.{_suffix}"
if full_name in os.listdir(root_dir):
wyli marked this conversation as resolved.
Show resolved Hide resolved
return full_name
return None

Check warning on line 1550 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1546-L1550

Added lines #L1546 - L1550 were not covered by tests


class BundleManager:
"""
The `BundleManager` class facilitates the automatic downloading and instantiation of bundles.
KumoLiu marked this conversation as resolved.
Show resolved Hide resolved
It allows users to retrieve bundle properties and meta information.

Typical usage examples:

.. code-block:: python

from monai.bundle import BundleManager

# Create a BundleManager instance for the 'spleen_ct_segmentation' bundle
bundle = BundleManager("spleen_ct_segmentation")

# Get properties defined in `TrainProperties` or `InferProperties`
train_preprocessing = bundle.get("train_preprocessing")
print(train_preprocessing)

# Also support to retrieve meta information from the "metadata.json" file
version = bundle.get(meta="version")
wyli marked this conversation as resolved.
Show resolved Hide resolved
print(version)

Args:
bundle_name: the name of the bundle. Defaults to None.
config_path: file path of the bundle. If a list of file paths is provided,
their contents will be merged. Defaults to None.
bundle_dir: the target directory to store downloaded bundle.
Defaults to the 'bundle' subfolder under `torch.hub.get_dir()`.
target directory to store the downloaded data.
Default is `bundle` subfolder under `torch.hub.get_dir()`.
configs: The name of the config file(s), supporting multiple names.
Defaults to "train".
kwargs: Additional arguments for download or workflow class instantiation.
"""

def __init__(
self,
bundle_name: str | None = None,
config_path: str | Sequence[str] | None = None,
bundle_dir: PathLike | None = None,
configs: str | Sequence[str] = "train",
**kwargs: Any,
) -> None:
if bundle_name is None and config_path is None:
raise ValueError("Must specify bundle_name or config_path.")
configs = ensure_tuple(configs)
if "train" in configs:
workflow = "train"

Check warning on line 1600 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1596-L1600

Added lines #L1596 - L1600 were not covered by tests
else:
workflow = "infer"

Check warning on line 1602 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1602

Added line #L1602 was not covered by tests

if config_path is not None:
_config_path = Path(ensure_tuple(config_path)[0])
if _config_path.is_file():
config_file = config_path
config_root_path = _config_path.parent

Check warning on line 1608 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1604-L1608

Added lines #L1604 - L1608 were not covered by tests
else:
raise FileNotFoundError(f"Cannot find the config file: {config_path}.")

Check warning on line 1610 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1610

Added line #L1610 was not covered by tests
else:
download_args = {

Check warning on line 1612 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1612

Added line #L1612 was not covered by tests
"version": kwargs.pop("version", None),
"args_file": kwargs.pop("args_file", None),
"source": kwargs.pop("source", download_source),
ericspod marked this conversation as resolved.
Show resolved Hide resolved
}
download(bundle_name, bundle_dir=bundle_dir, **download_args)
ericspod marked this conversation as resolved.
Show resolved Hide resolved
bundle_dir = _process_bundle_dir(bundle_dir)
config_root_path = bundle_dir / bundle_name / "configs" # type: ignore
if len(configs) > 0:
config_file = [

Check warning on line 1621 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1617-L1621

Added lines #L1617 - L1621 were not covered by tests
str(config_root_path / _find_bundle_file(config_root_path, _config)) for _config in configs # type: ignore
]
else:
config_file = str(config_root_path / _find_bundle_file(config_root_path, configs[0])) # type: ignore

Check warning on line 1625 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1625

Added line #L1625 was not covered by tests

logging_file = config_root_path / "logging.conf"
self.meta_file = config_root_path / "metadata.json"

Check warning on line 1628 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1627-L1628

Added lines #L1627 - L1628 were not covered by tests
KumoLiu marked this conversation as resolved.
Show resolved Hide resolved

self.workflow = ConfigWorkflow(

Check warning on line 1630 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1630

Added line #L1630 was not covered by tests
config_file=config_file,
meta_file=str(self.meta_file),
logging_file=str(logging_file),
workflow=workflow,
**kwargs,
)
self.workflow.initialize()

Check warning on line 1637 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1637

Added line #L1637 was not covered by tests

def get(self, property: str | None = None, meta: str | None = None) -> Any:
wyli marked this conversation as resolved.
Show resolved Hide resolved
"""
Get information from the bundle.

Args:
property: the target property, defined in `TrainProperties` or `InferProperties`.
meta: meta information retrieved from the "metadata.json" file, such as version, changelog, etc.

"""
if property is not None and meta is not None:
raise ValueError("Incompatible values: both property and meta are specified.")
if property is not None:
if property in self.workflow.properties: # type: ignore
return getattr(self.workflow, property)
raise ValueError(f"Missing property {property} in the bundle.")
if meta is not None:
metadata = ConfigParser.load_config_files(files=self.meta_file)
if meta.lower() in metadata.keys():
return {meta: metadata[meta.lower()]}
raise ValueError(f"Missing meta {meta} informtation in metadata.json.")

Check warning on line 1658 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1648-L1658

Added lines #L1648 - L1658 were not covered by tests

def train(self):
pass

Check warning on line 1661 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1661

Added line #L1661 was not covered by tests

def predict(self):
pass

Check warning on line 1664 in monai/bundle/scripts.py

View check run for this annotation

Codecov / codecov/patch

monai/bundle/scripts.py#L1664

Added line #L1664 was not covered by tests
Loading