3482 Add schema verification for `metadata` #3865

Nic-Ma · 2022-02-28T04:04:27Z

Task step 5 of #3482 .

Description

This PR added the utility to download schema and verify the metadata in a manifest.

Status

Ready

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
Breaking change (fix or new feature that would cause existing functionality to change).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

merge master

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma · 2022-02-28T08:57:09Z

Steps that generated the schema:

Generate a draft schema with online tool:
https://www.liquid-technologies.com/online-json-to-schema-converter
Modify unnecessary or incorrect items in the schema.
Add descriptions to the main items.

Thanks.

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma · 2022-02-28T15:26:08Z

It depends on PR #3832.

Thanks.

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma · 2022-03-01T07:59:23Z

Hi @ericspod @wyli @rijobro ,

Due to the Blossom CI issue, we can't merge any PRs recently, this PR depends on #3832 (marked the dependency logic with FIXME in the code) but it's almost ready for review. Could you please help review this draft PR first?

Thanks in advance.

monai/apps/manifest/utils.py

ericspod

I think it looks good to me, the DS_Store files need to be removed, and some pattern modified in .gitignore so that they don't show up again. We should discuss if adding the jsonschema dependency is a problem but I think it's fine as an optional.

Signed-off-by: Nic Ma <nma@nvidia.com>

…2-add-schema-verify Signed-off-by: Nic Ma <nma@nvidia.com>

Signed-off-by: Nic Ma <nma@nvidia.com>

tests/test_bundle_verify_meta.py

Nic-Ma · 2022-03-11T17:05:30Z

/black

Nic-Ma · 2022-03-11T17:05:37Z

/build

Nic-Ma · 2022-03-12T00:11:10Z

/build

Nic-Ma · 2022-03-12T00:30:07Z

/build

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma · 2022-03-13T02:13:58Z

/black

Nic-Ma · 2022-03-13T02:14:07Z

/build

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma · 2022-03-13T09:44:57Z

/black

Nic-Ma · 2022-03-13T09:45:07Z

/build

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma · 2022-03-14T06:09:09Z

/black

Nic-Ma · 2022-03-14T06:09:16Z

/build

wyli

thanks, looks good, could you move the schema.json to the testing data repository. Cc @ericspod in case there are more comments

monai/bundle/scripts.py

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma · 2022-03-14T13:23:28Z

/black

Nic-Ma · 2022-03-14T13:23:35Z

/build

Nic-Ma · 2022-03-14T13:27:34Z

/build

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma · 2022-03-14T15:25:11Z

/black

Nic-Ma · 2022-03-14T15:25:22Z

/build

Nic-Ma added 5 commits February 1, 2021 19:15

Merge pull request #19 from Project-MONAI/master

42a45e0

merge master

Merge pull request #32 from Project-MONAI/master

cd16a13

merge master

Merge pull request #180 from Project-MONAI/dev

6f87afd

merge master

Merge pull request #214 from Project-MONAI/dev

f398298

merge master

Merge pull request #380 from Project-MONAI/dev

5aa5384

merge master

Nic-Ma mentioned this pull request Feb 28, 2022

Develop MVP of model bundle #3482

Closed

17 tasks

[DLMED] complete temp test schema

09aa7ef

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma added 5 commits February 28, 2022 17:45

[DLMED] add cmd line API

67bd705

Signed-off-by: Nic Ma <nma@nvidia.com>

[DLMED] fix typo

c72088f

Signed-off-by: Nic Ma <nma@nvidia.com>

[DLMED] add more tests

9ed45df

Signed-off-by: Nic Ma <nma@nvidia.com>

[DLMED] add optional import

2cf9d3a

Signed-off-by: Nic Ma <nma@nvidia.com>

[DLMED] test with download link

d3801ee

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma mentioned this pull request Feb 28, 2022

MONAI Archive Specification #3834

Merged

7 tasks

[DLMED] enhance doc-string

b6cd1db

Signed-off-by: Nic Ma <nma@nvidia.com>

[DLMED] fix mypy

c77888a

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma requested review from rijobro, ericspod and wyli March 1, 2022 07:59

ericspod reviewed Mar 1, 2022

View reviewed changes

monai/apps/manifest/utils.py Outdated Show resolved Hide resolved

ericspod reviewed Mar 1, 2022

View reviewed changes

monai/apps/manifest/utils.py Outdated Show resolved Hide resolved

ericspod reviewed Mar 1, 2022

View reviewed changes

monai/apps/manifest/utils.py Outdated Show resolved Hide resolved

ericspod reviewed Mar 1, 2022

View reviewed changes

Nic-Ma mentioned this pull request Mar 2, 2022

3482 add run API for common training, evaluation and inference #3832

Merged

7 tasks

Nic-Ma added 3 commits March 7, 2022 12:27

[DLMED] remove DS_Store

eccc3ab

Signed-off-by: Nic Ma <nma@nvidia.com>

Merge branch 'dev' of https://github.com/Project-MONAI/MONAI into 348…

f37ee56

…2-add-schema-verify Signed-off-by: Nic Ma <nma@nvidia.com>

[DLMED] update according to comments

140b8b6

Signed-off-by: Nic Ma <nma@nvidia.com>

ericspod reviewed Mar 9, 2022

View reviewed changes

tests/test_bundle_verify_meta.py Outdated Show resolved Hide resolved

Nic-Ma requested a review from wyli March 11, 2022 17:05

Merge branch 'dev' into 3482-add-schema-verify

4112f18

Nic-Ma added 2 commits March 12, 2022 22:52

Merge branch 'dev' into 3482-add-schema-verify

4fe728c

[DLMED] update according to comments

9be9daa

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma force-pushed the 3482-add-schema-verify branch from bd35493 to 9be9daa Compare March 13, 2022 02:13

[DLMED] skip windows

caf3421

Signed-off-by: Nic Ma <nma@nvidia.com>

[DLMED] fix typo

0ce58d9

Signed-off-by: Nic Ma <nma@nvidia.com>

wyli approved these changes Mar 14, 2022

View reviewed changes

wyli reviewed Mar 14, 2022

View reviewed changes

monai/bundle/scripts.py Outdated Show resolved Hide resolved

Nic-Ma added 2 commits March 14, 2022 20:10

[DLMED] optimize error log

e7e1d3e

Signed-off-by: Nic Ma <nma@nvidia.com>

[DLMED] update according to comments

7e4c506

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma force-pushed the 3482-add-schema-verify branch from 0b182fa to 7e4c506 Compare March 14, 2022 13:21

[DLMED] update accoding to comments

f80d9a8

Signed-off-by: Nic Ma <nma@nvidia.com>

Nic-Ma enabled auto-merge (squash) March 14, 2022 15:30

Nic-Ma merged commit f15ff93 into Project-MONAI:dev Mar 14, 2022

3482 Add schema verification for metadata #3865

3482 Add schema verification for metadata #3865

Uh oh!

Conversation

Nic-Ma commented Feb 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Status

Types of changes

Uh oh!

Nic-Ma commented Feb 28, 2022

Uh oh!

Nic-Ma commented Feb 28, 2022

Uh oh!

Nic-Ma commented Mar 1, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ericspod left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Nic-Ma commented Mar 11, 2022

Uh oh!

Nic-Ma commented Mar 11, 2022

Uh oh!

Nic-Ma commented Mar 12, 2022

Uh oh!

Nic-Ma commented Mar 12, 2022

Uh oh!

Nic-Ma commented Mar 13, 2022

Uh oh!

Nic-Ma commented Mar 13, 2022

Uh oh!

Nic-Ma commented Mar 13, 2022

Uh oh!

Nic-Ma commented Mar 13, 2022

Uh oh!

Nic-Ma commented Mar 14, 2022

Uh oh!

Nic-Ma commented Mar 14, 2022

Uh oh!

wyli left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Nic-Ma commented Mar 14, 2022

Uh oh!

Nic-Ma commented Mar 14, 2022

Uh oh!

Nic-Ma commented Mar 14, 2022

Uh oh!

Nic-Ma commented Mar 14, 2022

Uh oh!

Nic-Ma commented Mar 14, 2022

Uh oh!

Uh oh!

3482 Add schema verification for `metadata` #3865

3482 Add schema verification for `metadata` #3865

Nic-Ma commented Feb 28, 2022 •

edited

Loading

wyli left a comment •

edited

Loading