Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DAOS-6923 test: Offline Reintegration - More tests #4835

Merged
merged 56 commits into from
Mar 30, 2021
Merged

DAOS-6923 test: Offline Reintegration - More tests #4835

merged 56 commits into from
Mar 30, 2021

Conversation

rpadma2
Copy link
Contributor

@rpadma2 rpadma2 commented Mar 3, 2021 •
edited
Loading

Test-tag-hw-medium: pr,hw,medium,ib2 offline_reintegration_daily

Summary:

  • Moved some more common files to osa_utils.py
  • Offline reintegration test changes
    • Added multiple pool infrastructure to test multiple pools fore offline reintegration
    • Test with different object class
    • More ranks excluded and reintegrated
    • Add mechanism to test exclude, reintegration with aggregation turned on.
  • Added daos container check changes.
  • Add mdtest thread methods.
  • Update offline drain to accomodate osa_utils.py changes.
  • Handle multiple containers for OSA testing.
  • Test without checksum enabled.

Signed-off-by: rpadma2 ravindran.padmanabhan@intel.com

@rpadma2
DAOS-6923 test: Offline Reintegration - More tests
112adfa 
Test-tag-hw-medium: pr,hw,medium,ib2 offline_reintegration

Summary:
   - Moved some more common files to osa_utils.py
   - Added the 200 pool test method
   - Test with different object class
   - More ranks excluded and reintegrated

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
daosbuild1
daosbuild1 previously requested changes Mar 3, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4835/1/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules

FYI: Errors found in lines not modified in the patch:

src/tests/ftest/osa/osa_offline_reintegration.py:25:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/osa/osa_offline_reintegration.py:8:
(pylint-unused-import) Unused import threading

src/tests/ftest/util/osa_utils.py:38:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/osa/osa_offline_reintegration.py Outdated Show resolved Hide resolved
src/tests/ftest/osa/osa_offline_reintegration.py Outdated Show resolved Hide resolved
src/tests/ftest/util/osa_utils.py Outdated Show resolved Hide resolved
@daosbuild1
Copy link
Collaborator

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/1/execution/node/72/log

@daosbuild1
Copy link
Collaborator

Test stage Scan CentOS 7 RPMs completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/1/execution/node/1145/log

rpadma2 added 2 commits March 3, 2021 22:58
@rpadma2
Merge branch 'master' into daos_6923
8e7300e
@rpadma2
DAOS-6923 test: Added mdtest feature
9bc2302 
Test-tag-hw-medium: pr,hw,medium,ib2 osa

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
@daosbuild1 daosbuild1 dismissed their stale review March 5, 2021 22:38

Updated patch

daosbuild1
daosbuild1 previously requested changes Mar 5, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4835/2/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules

FYI: Errors found in lines not modified in the patch:

src/tests/ftest/util/osa_utils.py:39:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/osa_utils.py:15:
(pylint-unused-import) Unused IorCommand imported from ior_utils

src/tests/ftest/util/osa_utils.py:16:
(pylint-unused-import) Unused Mpirun imported from job_manager_utils

src/tests/ftest/util/osa_utils.py:17:
(pylint-unused-import) Unused MpioUtils imported from mpio_utils

src/tests/ftest/osa/osa_offline_drain.py:24:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/osa/osa_offline_reintegration.py:25:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/osa_utils.py Outdated Show resolved Hide resolved
src/tests/ftest/util/osa_utils.py Show resolved Hide resolved
src/tests/ftest/util/osa_utils.py Outdated Show resolved Hide resolved
src/tests/ftest/util/osa_utils.py Outdated Show resolved Hide resolved
src/tests/ftest/osa/osa_offline_reintegration.py Outdated Show resolved Hide resolved
src/tests/ftest/osa/osa_offline_reintegration.py Outdated Show resolved Hide resolved
src/tests/ftest/util/mdtest_test_base.py Outdated Show resolved Hide resolved
@daosbuild1
Copy link
Collaborator

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/2/execution/node/72/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on CentOS 7 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/2/execution/node/391/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on Leap 15 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/2/execution/node/396/log

@rpadma2
DAOS-6923 test: Fix checkpatch issues.
7225a5b 
Test-tag-hw-medium: pr,hw,medium,ib2 osa

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
@daosbuild1 daosbuild1 dismissed their stale review March 5, 2021 23:01

Updated patch

daosbuild1
daosbuild1 previously requested changes Mar 5, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4835/3/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules

FYI: Errors found in lines not modified in the patch:

src/tests/ftest/osa/osa_offline_drain.py:24:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/mdtest_test_base.py:23:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/mdtest_test_base.py:33:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/osa_utils.py:39:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/osa_utils.py:15:
(pylint-unused-import) Unused IorCommand imported from ior_utils

src/tests/ftest/util/osa_utils.py:16:
(pylint-unused-import) Unused Mpirun imported from job_manager_utils

src/tests/ftest/util/osa_utils.py:17:
(pylint-unused-import) Unused MpioUtils imported from mpio_utils

src/tests/ftest/osa/osa_offline_reintegration.py Outdated Show resolved Hide resolved
@daosbuild1
Copy link
Collaborator

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/3/execution/node/73/log

@rpadma2
DAOS-6923 test: Fix the typo
77f8c65 
Test-tag-hw-medium: pr,hw,medium,ib2 osa

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
@daosbuild1 daosbuild1 dismissed their stale review March 5, 2021 23:06

Updated patch

daosbuild1
daosbuild1 reviewed Mar 5, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. No errors found by checkpatch.

FYI: Errors found in lines not modified in the patch:

src/tests/ftest/osa/osa_offline_drain.py:24:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/osa/osa_offline_reintegration.py:25:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/mdtest_test_base.py:23:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/mdtest_test_base.py:33:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/osa_utils.py:39:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/osa_utils.py:15:
(pylint-unused-import) Unused IorCommand imported from ior_utils

src/tests/ftest/util/osa_utils.py:16:
(pylint-unused-import) Unused Mpirun imported from job_manager_utils

src/tests/ftest/util/osa_utils.py:17:
(pylint-unused-import) Unused MpioUtils imported from mpio_utils

@rpadma2
DAOS-6923 test: Removed unwanted results parameter
b2cb8c6 
Test-tag-hw-medium: pr,hw,medium,ib2 osa

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
daosbuild1
daosbuild1 previously requested changes Mar 7, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4835/5/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules

FYI: Errors found in lines not modified in the patch:

src/tests/ftest/osa/osa_offline_drain.py:9:
(pylint-syntax-error) Cannot import 'osa_utils' due to syntax error "closing parenthesis ')' does not match opening parenthesis '{' on line 198 (, line 201)"

src/tests/ftest/osa/osa_offline_drain.py:24:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/osa/osa_offline_drain.py:68:
(pylint-attribute-defined-outside-init) Attribute 'pool' defined outside init

src/tests/ftest/osa/osa_offline_drain.py:81:
(pylint-attribute-defined-outside-init) Attribute 'pool' defined outside init

src/tests/ftest/osa/osa_offline_reintegration.py:9:
(pylint-syntax-error) Cannot import 'osa_utils' due to syntax error "closing parenthesis ')' does not match opening parenthesis '{' on line 198 (, line 201)"

src/tests/ftest/osa/osa_offline_reintegration.py:25:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/osa/osa_offline_reintegration.py:67:
(pylint-attribute-defined-outside-init) Attribute 'pool' defined outside init

src/tests/ftest/util/mdtest_test_base.py:23:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/mdtest_test_base.py:33:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/osa/osa_offline_reintegration.py Outdated Show resolved Hide resolved
src/tests/ftest/osa/osa_offline_reintegration.py Outdated Show resolved Hide resolved
src/tests/ftest/util/osa_utils.py Outdated Show resolved Hide resolved
@daosbuild1
Copy link
Collaborator

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/5/execution/node/72/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on CentOS 7 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/5/execution/node/405/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on Leap 15 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/5/execution/node/357/log

@rpadma2
DAOS-6923 test: Address checkpatch issues.
078065e 
Test-tag-hw-medium: pr,hw,medium,ib2 osa

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
@daosbuild1 daosbuild1 dismissed their stale review March 8, 2021 01:52

Updated patch

daosbuild1
daosbuild1 reviewed Mar 8, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. No errors found by checkpatch.

FYI: Errors found in lines not modified in the patch:

src/tests/ftest/osa/osa_offline_drain.py:24:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/osa/osa_offline_reintegration.py:25:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/mdtest_test_base.py:23:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/mdtest_test_base.py:33:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/osa_utils.py:39:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/osa_utils.py:15:
(pylint-unused-import) Unused IorCommand imported from ior_utils

src/tests/ftest/util/osa_utils.py:16:
(pylint-unused-import) Unused Mpirun imported from job_manager_utils

src/tests/ftest/util/osa_utils.py:17:
(pylint-unused-import) Unused MpioUtils imported from mpio_utils

rpadma2 added 2 commits March 9, 2021 00:25
@rpadma2
DAOS-6923 test: Run all the tests once now.
c072ccb 
Test-tag-hw-medium: pr,hw,medium,ib2 osa

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
@rpadma2
Merge branch 'master' into daos_6923
94f82fb 
Test-tag-hw-medium: pr,hw,medium,ib2 osa

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
daosbuild1
daosbuild1 previously requested changes Mar 9, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4835/7/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules

FYI: Errors found in lines not modified in the patch:

src/tests/ftest/osa/osa_offline_drain.py:24:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/mdtest_test_base.py:23:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/mdtest_test_base.py:33:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/osa_utils.py:39:
(pylint-super-with-arguments) Consider using Python 3 style super() without arguments

src/tests/ftest/util/osa_utils.py:15:
(pylint-unused-import) Unused IorCommand imported from ior_utils

src/tests/ftest/util/osa_utils.py:16:
(pylint-unused-import) Unused Mpirun imported from job_manager_utils

src/tests/ftest/util/osa_utils.py:17:
(pylint-unused-import) Unused MpioUtils imported from mpio_utils

src/tests/ftest/osa/osa_offline_reintegration.py Outdated Show resolved Hide resolved
@daosbuild1
Copy link
Collaborator

Test stage Build RPM on Leap 15 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/42/execution/node/378/log

@daosbuild1
Copy link
Collaborator

Test stage NLT completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-4835/42/display/redirect

daosbuild1
daosbuild1 reviewed Mar 28, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. No errors found by checkpatch.

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on CentOS 7 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/43/execution/node/348/log

@daosbuild1
Copy link
Collaborator

Test stage Build DEB on Ubuntu 20.04 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/43/execution/node/395/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on Leap 15 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-4835/43/execution/node/389/log

daosbuild1
daosbuild1 reviewed Mar 28, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. No errors found by checkpatch.

rpadma2 added 2 commits March 28, 2021 19:56
@rpadma2
DAOS-6923 test: Seeing md_test failures.
3787a55 
Test-tag-hw-medium: pr,hw,medium,ib2 osa

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
@rpadma2
DAOS-6923 test: Enable daos cont check for tests
de979b5 
Test-tag-hw-medium: pr,hw,medium,ib2 osa

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
daosbuild1
daosbuild1 reviewed Mar 29, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. No errors found by checkpatch.

@rpadma2
DAOS-6923 test: Fix mdtest_test_base
d17a081 
Test-tag-hw-medium: pr,hw,medium,ib2 osa

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
daosbuild1
daosbuild1 reviewed Mar 29, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. No errors found by checkpatch.

rpadma2 added 2 commits March 29, 2021 10:29
@rpadma2
DAOS-6923 test: Add log messages
3ef55fe 
Test-tag-hw-medium: pr,hw,medium,ib2 offline_reintegration_daily

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
@rpadma2
Merge branch 'master' into daos_6923
e975b4f 
Test-tag-hw-medium: pr,hw,medium,ib2 offline_reintegration_daily
daosbuild1
daosbuild1 reviewed Mar 29, 2021
View reviewed changes
Copy link
Collaborator

@daosbuild1 daosbuild1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. No errors found by checkpatch.

@rpadma2 rpadma2 requested review from wangdi1 and shimizukko March 30, 2021 13:43
@rpadma2 rpadma2 requested review from sylviachanoiyee and a team March 30, 2021 15:36
@rpadma2
Copy link
Contributor Author

rpadma2 commented Mar 30, 2021 •
edited
Loading

This PR is ready to be merged. Once merged, I will work on cherry-picking this change to 1.2 branch,.

saurabhtandan
saurabhtandan approved these changes Mar 30, 2021
View reviewed changes
Copy link
Contributor

@saurabhtandan saurabhtandan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sylviachanoiyee sylviachanoiyee merged commit aa07584 into master Mar 30, 2021
@sylviachanoiyee sylviachanoiyee deleted the daos_6923 branch March 30, 2021 20:50
rpadma2 added a commit that referenced this pull request Mar 30, 2021
@rpadma2
DAOS-6923 test: Offline Reintegration - More tests (#4835)
60cdac6 
Summary:
   - Moved some more common files to osa_utils.py
   - Added the 200 pool test method
   - Test with different object class
   - More ranks excluded and reintegrated

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
rpadma2 added a commit that referenced this pull request Mar 30, 2021
@rpadma2
DAOS-6923 test: Offline Reintegration - More tests (#4835)
a6602e7 
Summary:
   - Moved some more common files to osa_utils.py
   - Added the 200 pool test method
   - Test with different object class
   - More ranks excluded and reintegrated

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
sylviachanoiyee pushed a commit that referenced this pull request Mar 31, 2021
@rpadma2
DAOS-6923 test: Offline Reintegration - More tests (#4835) (#5253)
728ca11 
Summary:
   - Moved some more common files to osa_utils.py
   - Added the 200 pool test method
   - Test with different object class
   - More ranks excluded and reintegrated

Signed-off-by: rpadma2 <ravindran.padmanabhan@intel.com>
@ashleypittman ashleypittman mentioned this pull request Apr 28, 2021
Closed
@ashleypittman ashleypittman mentioned this pull request May 20, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daosbuild1 daosbuild1 daosbuild1 left review comments

@saurabhtandan saurabhtandan saurabhtandan approved these changes

@wangdi1 wangdi1 wangdi1 approved these changes

@shimizukko shimizukko shimizukko approved these changes

@byron-marohn byron-marohn Awaiting requested review from byron-marohn

@sylviachanoiyee sylviachanoiyee Awaiting requested review from sylviachanoiyee

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@rpadma2 @daosbuild1 @jolivier23 @shimizukko @wangdi1 @saurabhtandan @sylviachanoiyee