Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

886 add MetricsSaver handler to save metrics and details into files #1497

Merged
merged 51 commits into from
Jan 29, 2021
Merged

886 add MetricsSaver handler to save metrics and details into files #1497

merged 51 commits into from
Jan 29, 2021

Conversation

Nic-Ma
Copy link
Contributor

@Nic-Ma Nic-Ma commented Jan 25, 2021 •
edited
Loading

Partly fix ticket Project-MONAI/MetricsReloaded#36

Description

This PR added support to save the metrics final results and details into CSV files.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh --codeformat --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

Nic-Ma and others added 12 commits January 19, 2021 18:25
@Nic-Ma
Merge pull request #10 from Project-MONAI/master
66d59b5 
merge master
@Nic-Ma
Merge pull request #14 from Project-MONAI/master
8e19d91 
merge master
@Nic-Ma
[DLMED] add IterationHandler refer to the EpochHandler in ignite
8f110a0 
Signed-off-by: Nic Ma <nma@nvidia.com>
@monai-bot
[MONAI] python code formatting
560802d 
Signed-off-by: monai-bot <monai.miccai2019@gmail.com>
@Nic-Ma
[DLMED] fix flake8 issue
b2c317e 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] fix the multi-gpu issue
6f2371e 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Merge branch 'master' into 886-add-metrics-report
7aaa14e
@Nic-Ma
[DLMED] fix typo
4b13a4e 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] fix distributed tests
af9fdc6 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] fix flake8 issue
f586b08 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Merge pull request #16 from Project-MONAI/master
8cea632 
merge master
@Nic-Ma
[DLMED] add engine to metrics
8504839 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma Nic-Ma mentioned this pull request Jan 25, 2021
Open
Nic-Ma added 5 commits January 25, 2021 13:20
@Nic-Ma
[DLMED] share metric details in engine
16b1210 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] add metrics report
8036dc0 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] add average value to report
f87c46c 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] add summary report
f247d69 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] add docs
423f724 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 25, 2021

/black

@monai-bot
[MONAI] python code formatting
ae89aa3 
Signed-off-by: monai-bot <monai.miccai2019@gmail.com>
@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 25, 2021

/black

@Nic-Ma Nic-Ma requested a review from wyli January 25, 2021 16:39
@Nic-Ma Nic-Ma force-pushed the 886-add-metrics-report branch from 05ff6d6 to 645daa0 Compare January 25, 2021 16:45
@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 25, 2021

/black

@Nic-Ma Nic-Ma force-pushed the 886-add-metrics-report branch from 645daa0 to 183fb6d Compare January 25, 2021 16:51
@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 25, 2021

/black

@Nic-Ma
[DLMED] fix flake8 issue
35bf113 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma Nic-Ma force-pushed the 886-add-metrics-report branch from 183fb6d to 35bf113 Compare January 25, 2021 17:08
@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 25, 2021

/black

@monai-bot
[MONAI] python code formatting
432177e 
Signed-off-by: monai-bot <monai.miccai2019@gmail.com>
monai-bot and others added 2 commits January 27, 2021 08:08
@monai-bot
[MONAI] python code formatting
d6f1a7d 
Signed-off-by: monai-bot <monai.miccai2019@gmail.com>
@Nic-Ma
[DLMED] remove from min_tests
6d3b08b 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 27, 2021

/black

@Nic-Ma Nic-Ma requested a review from wyli January 27, 2021 12:57
@Nic-Ma
[DLMED] change to standlone APIs to write files
45e5e0a 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 27, 2021

/black

monai-bot and others added 2 commits January 27, 2021 16:45
@monai-bot
[MONAI] python code formatting
1dde629 
Signed-off-by: monai-bot <monai.miccai2019@gmail.com>
@Nic-Ma
[DLMED] add file type check
af77ebb 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 28, 2021

/black

@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 28, 2021

/black

@Nic-Ma
[DLMED] develop standlone API
13954c9 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 29, 2021

/black

@monai-bot
[MONAI] python code formatting
abcb98a 
Signed-off-by: monai-bot <monai.miccai2019@gmail.com>
@Nic-Ma Nic-Ma requested a review from wyli January 29, 2021 02:31
@Nic-Ma
[DLMED] fix flake8 issue
fc274ba 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 29, 2021

/black

@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 29, 2021

/black

@Nic-Ma
[DLMED] fix min test
040470d 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Copy link
Contributor Author

Nic-Ma commented Jan 29, 2021 •
edited by wyli
Loading

/integration-test /black

wyli
wyli approved these changes Jan 29, 2021
View reviewed changes
Copy link
Contributor

@wyli wyli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! could have another iteration to use pandas dataframe or the build-in csv writer

@Nic-Ma Nic-Ma merged commit b3d063c into Project-MONAI:master Jan 29, 2021
@Nic-Ma Nic-Ma deleted the 886-add-metrics-report branch July 2, 2021 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wyli wyli wyli approved these changes

@yiheng-wang-nv yiheng-wang-nv Awaiting requested review from yiheng-wang-nv

@atbenmurray atbenmurray Awaiting requested review from atbenmurray

@ericspod ericspod Awaiting requested review from ericspod

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Nic-Ma @wyli @monai-bot