Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update zos_data_set member description created #791

Merged
merged 15 commits into from
Jun 12, 2023

Conversation

ddimatos
Copy link
Collaborator

@ddimatos ddimatos commented Jun 3, 2023

SUMMARY

Given this change will drive over 700 tests it made sense to collapse all the related zos_data_set into one PR to avoid multiple regression pipelines.

  1. Documentation: When a MEMBER is created, the format type of the member was not described, it should be clearly defined so that a user does not try to insert an executable into the data formatted member. Issue [Documentation] [zos_data_set] Update member doc to explain what type of member is created #631

  2. Bug: A random issue would occur on the pipeline leaving behind VSAM cluster components be it INDEX or DATA, originally this was thought to be a test case issue and a simple test fix but after reviewing I found it much more complex and took a better part of the week to address. To fix this I had to undo most of what was done in PR 325 for bugfix [Bug] [zos_data_set] should not ignore supplied volumes #140. Although this was a good attempt, the issue was that only one of the cluster components were being deleted here with a DELETE USER.PRIVATE.TESTDS.DATA FILE(DD1) VVR while adding the other for INDEX would have been a suitable fix, the issue such a clean up leaves space unclaimed on the volume and not the correct way to back out a data set from z/OS. Thus introduced a flow of tracking state and uncatalog, catalog, delete to achieve this, see image below for most the flow, the flow has since changed with some added exception actions but mostly the flow is correct for reference. Issue [Enabler] [CI/CD] Failing to clean up environtment causes a failure. #635

  3. The catalog commands were failing because of formatting, I might have been happening for a long time because of all this other related issues it might not have been apparent, thus now the commands are formatted correctly and tested. Issue [Bug] zos_data_set data_set.py utils cataloging failures #802

  4. Small .ac command issue where some files are left behind were cleaned up as part of this PR because I was using ./ac --ac-test and it was inhibiting my ability to work to not fix it.

Data set deletion flow.
image

ISSUE TYPE
  • Docs Pull Request

ddimatos added 11 commits June 2, 2023 21:57
Signed-off-by: ddimatos <dimatos@gmail.com>
Signed-off-by: ddimatos <dimatos@gmail.com>
Signed-off-by: ddimatos <dimatos@gmail.com>
Signed-off-by: ddimatos <dimatos@gmail.com>
Signed-off-by: ddimatos <dimatos@gmail.com>
…s user to correctly delete DS not in cat

Signed-off-by: ddimatos <dimatos@gmail.com>
Signed-off-by: ddimatos <dimatos@gmail.com>
Signed-off-by: ddimatos <dimatos@gmail.com>
Signed-off-by: ddimatos <dimatos@gmail.com>
Signed-off-by: ddimatos <dimatos@gmail.com>
Signed-off-by: ddimatos <dimatos@gmail.com>
@ddimatos
Copy link
Collaborator Author

image Jenkins: Job suffix `/job/zosAnsibleAlternate2/41/console` Log: `====================== 760 passed in 33901.51s (9:25:01) =======================` Tests: ``` + ./dependencyfinder.py -p .. -b dev -s functional/modules/test_module_security.py functional/modules/test_zos_apf_func.py -m [Pipeline] echo ../tests/functional/modules/test_zos_lineinfile_func.py ../tests/functional/modules/test_zos_fetch_func.py ../tests/functional/modules/test_zos_mvs_raw_func.py ../tests/functional/modules/test_zos_blockinfile_func.py ../tests/functional/modules/test_zos_find_func.py ../tests/functional/modules/test_zos_copy_func.py ../tests/functional/modules/test_zos_data_set_func.py ../tests/unit/test_zos_mvs_raw_unit.py ../tests/functional/modules/test_zos_backup_restore.py ../tests/functional/modules/test_zos_job_submit_func.py ../tests/functional/modules/test_zos_job_output_func.py ../tests/functional/modules/test_zos_job_query_func.py ../tests/functional/modules/test_zos_volume_init_func.py ../tests/functional/modules/test_zos_mount_func.py ../tests/functional/modules/test_zos_encode_func.py ```

Copy link
Collaborator

@fernandofloresg fernandofloresg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested for changelog and unused import.

Questions about 2 rst files apparently not related to this PR and use of run_command for mvs_cmd

changelogs/fragments/791-zos_data_set-update-vsam.yml Outdated Show resolved Hide resolved
changelogs/fragments/791-zos_data_set-update-vsam.yml Outdated Show resolved Hide resolved
changelogs/fragments/791-zos_data_set-update-vsam.yml Outdated Show resolved Hide resolved
plugins/module_utils/data_set.py Show resolved Hide resolved
plugins/module_utils/data_set.py Outdated Show resolved Hide resolved
docs/source/modules/zos_job_query.rst Outdated Show resolved Hide resolved
docs/source/modules/zos_job_submit.rst Outdated Show resolved Hide resolved
Signed-off-by: ddimatos <dimatos@gmail.com>
Copy link
Collaborator

@fernandofloresg fernandofloresg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is ok for me now, I get that having this in our dev branch will benefit us immediately. mvs_cmd can be standarized later and unused imports actually are shown get using the latest ansible 2.15 sanity scan so we will clean it 🙂 .

ddimatos added 2 commits June 11, 2023 12:44
Signed-off-by: ddimatos <dimatos@gmail.com>
Signed-off-by: ddimatos <dimatos@gmail.com>
@ddimatos
Copy link
Collaborator Author

ddimatos commented Jun 11, 2023

Imports removed from data_sets.py, thought you were referring to the pprint in the test cases, which is referenced by an unused method. Sanity ran clean on 2.15 and 2.16.

@ddimatos ddimatos marked this pull request as ready for review June 11, 2023 19:47
…time 2.16

Signed-off-by: ddimatos <dimatos@gmail.com>
Copy link
Collaborator

@richp405 richp405 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@richp405 richp405 merged commit b2c8b07 into dev Jun 12, 2023
@richp405 richp405 deleted the doc/631/define-member-zos_data_set branch June 12, 2023 13:26
@ddimatos
Copy link
Collaborator Author

Also includes issue #803 which was left out of the description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants