Automate the branch entry to version increment workflow #2602

prudhvigodithi · 2022-09-09T21:58:55Z

Description

Automate the branch entry to version increment workflow

Issues Resolved

Part of: #1375
Solves: #2601

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

* Updated manifests. Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> * Remove 2.2.2 from the check for build as 2.3.0 is about to release Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Add common-utils Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Peter Zhu <zhujiaxi@amazon.com>

prudhvigodithi · 2022-09-09T22:08:36Z

Test fail for mypy strange this works on my local

pipenv run mypy .
Success: no issues found in 323 source files

src/manifests_workflow/input_manifests.py

peterzhuamazon · 2022-09-12T19:07:55Z

@prudhvigodithi can you explain what you are trying to do here?
I see you are trying to load a yaml and check the matrix.
While figuring out what is missing and add new entries, but not very clear without the data. Thanks.

prudhvigodithi · 2022-09-12T19:49:03Z

@prudhvigodithi can you explain what you are trying to do here? I see you are trying to load a yaml and check the matrix. While figuring out what is missing and add new entries, but not very clear without the data. Thanks.

So @peterzhuamazon in order to trigger the version-increment workflow, we need the right branch entries, having these branches, for every branch the a job is created and version increment is performed, now for every release one has to raise a PR to add this branch entry.

With this change (part of this PR) this branch entry with be auto updated as part of the manifest workflow [AUTO] Updated input manifests PR, so next time we dont need to manually raise a PR for 2.x, 3.x etc.

peterzhuamazon · 2022-09-12T20:14:48Z

I see what you mean so if there is any new branches it will update the versions-increment yml workflow so we dont need to manually raise PR.

Thanks.

src/manifests_workflow/input_manifests.py

codecov-commenter · 2022-09-12T21:38:34Z

Codecov Report

Merging #2602 (d3964dc) into main (d502f4a) will decrease coverage by 0.47%.
The diff coverage is 28.12%.

@@             Coverage Diff              @@
##               main    #2602      +/-   ##
============================================
- Coverage     94.41%   93.94%   -0.48%     
  Complexity       28       28              
============================================
  Files           219      219              
  Lines          4462     4494      +32     
  Branches         29       29              
============================================
+ Hits           4213     4222       +9     
- Misses          243      266      +23     
  Partials          6        6

Impacted Files	Coverage Δ
src/manifests_workflow/input_manifests.py	`82.97% <28.12%> (-16.11%)`	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Signed-off-by: prudhvigodithi <pgodithi@amazon.com>

prudhvigodithi · 2022-09-13T16:48:44Z

Thanks @peterzhuamazon, I have also just removed - '2.3' (commit) from the entry as we are already on 2.4, just an overkill to run again for all 2.3 branches

dblock · 2022-09-13T19:32:31Z

I’m late to this PR, but it needs more work even if it seems to work. Please followup / open issues accordingly.

What is the new yaml library business here? It looks like there’s now more than one way to read and write YML, please consolidate into one.
There are no tests that ensure that the correct values were added to the workflow in the various cases of major/minor/etc., just that the add method was called.

dblock · 2022-09-13T19:33:58Z

tests/tests_manifests_workflow/test_input_manifests.py

+    @patch("builtins.open", new_callable=mock_open)
+    @patch("manifests_workflow.input_manifests.InputManifests.add_to_versionincrement_workflow")
+    def test_add_to_versionincrement_workflow(self, *mocks: Any) -> None:
+        input_manifests = InputManifests("test")


This test doesn’t assert anything.

dblock · 2022-09-13T19:34:28Z

src/manifests_workflow/input_manifests.py

+        with open(versionincrement_workflow_file) as f:
+            data = yaml.load(f)
+
+        version_entry = []


This part is not tested.

Let me add this as well, I will raise a new PR

dblock · 2022-09-13T19:35:40Z

src/manifests_workflow/input_manifests.py

@@ -11,6 +11,8 @@
 from abc import abstractmethod
 from typing import Dict, List, Type, Union

+import ruamel.yaml


The code now reads and writes YML files in two different ways. We need to have one.

prudhvigodithi · 2022-09-13T19:44:04Z

Hey @dblock for the new YAML library ruamel.yaml I have have added useful information #2602 (comment), please check. I can create an issue to migrate to ruamel.yaml

prudhvigodithi · 2022-09-13T21:26:06Z

Created a new issue to use ruamel.yaml #2614
@dblock @peterzhuamazon @gaiksaya @bbarani

prudhvigodithi requested a review from a team as a code owner September 9, 2022 21:58

prudhvigodithi force-pushed the versionincrement branch 3 times, most recently from a5e569c to 0212aac Compare September 9, 2022 22:48

peterzhuamazon reviewed Sep 12, 2022

View reviewed changes

src/manifests_workflow/input_manifests.py Show resolved Hide resolved

prudhvigodithi requested a review from peterzhuamazon September 12, 2022 20:13

gaiksaya reviewed Sep 12, 2022

View reviewed changes

src/manifests_workflow/input_manifests.py Outdated Show resolved Hide resolved

prudhvigodithi force-pushed the versionincrement branch from 0212aac to b3a536b Compare September 12, 2022 21:33

gaiksaya reviewed Sep 12, 2022

View reviewed changes

src/manifests_workflow/input_manifests.py Outdated Show resolved Hide resolved

prudhvigodithi force-pushed the versionincrement branch 3 times, most recently from e39b644 to 4139168 Compare September 12, 2022 22:59

prudhvigodithi marked this pull request as draft September 12, 2022 23:03

prudhvigodithi force-pushed the versionincrement branch from 4139168 to d3964dc Compare September 12, 2022 23:29

prudhvigodithi marked this pull request as ready for review September 13, 2022 00:27

prudhvigodithi requested a review from gaiksaya September 13, 2022 02:12

peterzhuamazon approved these changes Sep 13, 2022

View reviewed changes

Upstream merge

76b6280

Signed-off-by: prudhvigodithi <pgodithi@amazon.com>

prudhvigodithi force-pushed the versionincrement branch from d3964dc to 76b6280 Compare September 13, 2022 16:47

prudhvigodithi merged commit b22a570 into opensearch-project:main Sep 13, 2022

prudhvigodithi mentioned this pull request Sep 13, 2022

Automate the branch entry to version increment workflow. #2601

Closed

dblock reviewed Sep 13, 2022

View reviewed changes

prudhvigodithi mentioned this pull request Sep 13, 2022

Migrate the python workflows yaml module to use ruamel.yaml #2614

Open

prudhvigodithi mentioned this pull request Sep 15, 2022

OpenSearch: Automate increment to next development iteration #1375

Closed

29 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automate the branch entry to version increment workflow #2602

Automate the branch entry to version increment workflow #2602

prudhvigodithi commented Sep 9, 2022 •

edited

Loading

prudhvigodithi commented Sep 9, 2022 •

edited

Loading

peterzhuamazon commented Sep 12, 2022

prudhvigodithi commented Sep 12, 2022 •

edited

Loading

peterzhuamazon commented Sep 12, 2022

codecov-commenter commented Sep 12, 2022 •

edited

Loading

prudhvigodithi commented Sep 13, 2022 •

edited

Loading

dblock commented Sep 13, 2022 •

edited

Loading

dblock Sep 13, 2022 •

edited

Loading

dblock Sep 13, 2022

prudhvigodithi Sep 13, 2022

dblock Sep 13, 2022

prudhvigodithi commented Sep 13, 2022

prudhvigodithi commented Sep 13, 2022

Automate the branch entry to version increment workflow #2602

Automate the branch entry to version increment workflow #2602

Conversation

prudhvigodithi commented Sep 9, 2022 • edited Loading

Description

Issues Resolved

prudhvigodithi commented Sep 9, 2022 • edited Loading

peterzhuamazon commented Sep 12, 2022

prudhvigodithi commented Sep 12, 2022 • edited Loading

peterzhuamazon commented Sep 12, 2022

codecov-commenter commented Sep 12, 2022 • edited Loading

Codecov Report

prudhvigodithi commented Sep 13, 2022 • edited Loading

dblock commented Sep 13, 2022 • edited Loading

dblock Sep 13, 2022 • edited Loading

Choose a reason for hiding this comment

dblock Sep 13, 2022

Choose a reason for hiding this comment

prudhvigodithi Sep 13, 2022

Choose a reason for hiding this comment

dblock Sep 13, 2022

Choose a reason for hiding this comment

prudhvigodithi commented Sep 13, 2022

prudhvigodithi commented Sep 13, 2022

prudhvigodithi commented Sep 9, 2022 •

edited

Loading

prudhvigodithi commented Sep 9, 2022 •

edited

Loading

prudhvigodithi commented Sep 12, 2022 •

edited

Loading

codecov-commenter commented Sep 12, 2022 •

edited

Loading

prudhvigodithi commented Sep 13, 2022 •

edited

Loading

dblock commented Sep 13, 2022 •

edited

Loading

dblock Sep 13, 2022 •

edited

Loading