Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(ci): Make sure merge_local_staging handles all subdir #3788

Merged
merged 8 commits into from
Dec 20, 2023
Merged

fix(ci): Make sure merge_local_staging handles all subdir #3788

merged 8 commits into from
Dec 20, 2023

Conversation

Xuanwo
Copy link
Member

@Xuanwo Xuanwo commented Dec 20, 2023

Fix #3787

@Xuanwo
fix(ci): Make sure merge_local_staging handles all subdir
7db8d1b 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo Xuanwo requested a review from PsiACE as a code owner December 20, 2023 03:52
@Xuanwo Xuanwo requested review from tisonkun and suyanhanx December 20, 2023 03:52
@Xuanwo
Add some logs
9a5a80c 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
address comments
3f6fd3d 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Add comments
ba0fe39 
Signed-off-by: Xuanwo <github@xuanwo.io>
@tisonkun
Copy link
Member

In another perspective, the PR description doesn't tell why the preview script doesn't convey all the variants. I guess it's some override rather than append logic. But it's not easy to reverse engineer the patch to understand.

@suyanhanx
Copy link
Member

😦May you test it by dry run?

@tisonkun
Copy link
Member

tisonkun commented Dec 20, 2023 •
edited
Loading

May you test it by dry run?

I ever test the original script by download the final artifacts.

The workflow file already have a ${{ github.event_name != 'pull_request' }} in deploy step but missing a upload artifact for debugging.

But the artifacts for each platform are available so the python script can be tested locally.

@Xuanwo
refactor
4b2b972 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Copy link
Member Author

Xuanwo commented Dec 20, 2023

Tested locally with linux x86_64 & aarch artifacts:

> python ./merge_local_staging.py ./staging /tmp/linux-aarch_64-local-staging/staging /tmp/linux-x86_64-local-staging/staging
Target directory set to staging
Processing /tmp/linux-aarch_64-local-staging/staging
Appending /tmp/linux-aarch_64-local-staging/staging/16a16cf8585f60/.index to staging/16a16cf8585f60/.index
Copying /tmp/linux-aarch_64-local-staging/staging/16a16cf8585f60/org to staging/16a16cf8585f60
Processing /tmp/linux-x86_64-local-staging/staging
Appending /tmp/linux-x86_64-local-staging/staging/16a16cf8585f60/.index to staging/16a16cf8585f60/.index
Copying /tmp/linux-x86_64-local-staging/staging/16a16cf8585f60/org to staging/16a16cf8585f60

image

@Xuanwo
Print tree after merging
fe3beb7 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo
Copy link
Member Author

Xuanwo commented Dec 20, 2023

Also added tree command to check if we merged correctly.

@tisonkun
Copy link
Member

Tested locally with linux x86_64 & aarch artifacts:

Can you download the uploaded artifacts (linux, macos, window) and test the script on those artifacts?

@Xuanwo
Copy link
Member Author

Xuanwo commented Dec 20, 2023 •
edited
Loading

Can you download the uploaded artifacts (linux, macos, window) and test the script on those artifacts?

Yep. I tested it in this way.

@tisonkun
Copy link
Member

@Xuanwo but you test only for linux aarch64 and x8664. Let's try for what exactly the script does to avoid some subtle issue.

@Xuanwo
Copy link
Member Author

Xuanwo commented Dec 20, 2023

Tested on all artifacts downloaded from: https://github.com/apache/incubator-opendal/actions/runs/7046140331

:( python ./merge_local_staging.py ./staging /tmp/linux-aarch_64-local-staging/staging /tmp/linux-x86_64-local-staging/staging /tmp/windows-x86_64-local-staging/staging /tmp/osx-aarch_64-local-staging/staging /tmp/osx-x86_64-local-staging/staging 
Target directory set to staging
Processing /tmp/linux-aarch_64-local-staging/staging
Appending /tmp/linux-aarch_64-local-staging/staging/16a16cf8585f60/.index to staging/16a16cf8585f60/.index
Copying /tmp/linux-aarch_64-local-staging/staging/16a16cf8585f60/org to staging/16a16cf8585f60
Processing /tmp/linux-x86_64-local-staging/staging
Appending /tmp/linux-x86_64-local-staging/staging/16a16cf8585f60/.index to staging/16a16cf8585f60/.index
Copying /tmp/linux-x86_64-local-staging/staging/16a16cf8585f60/org to staging/16a16cf8585f60
Processing /tmp/windows-x86_64-local-staging/staging
Appending /tmp/windows-x86_64-local-staging/staging/16a16cf8585f60/.index to staging/16a16cf8585f60/.index
Copying /tmp/windows-x86_64-local-staging/staging/16a16cf8585f60/org to staging/16a16cf8585f60
Processing /tmp/osx-aarch_64-local-staging/staging
Appending /tmp/osx-aarch_64-local-staging/staging/16a16cf8585f60/.index to staging/16a16cf8585f60/.index
Copying /tmp/osx-aarch_64-local-staging/staging/16a16cf8585f60/org to staging/16a16cf8585f60
Processing /tmp/osx-x86_64-local-staging/staging
Appending /tmp/osx-x86_64-local-staging/staging/16a16cf8585f60/.index to staging/16a16cf8585f60/.index
Copying /tmp/osx-x86_64-local-staging/staging/16a16cf8585f60/org to staging/16a16cf8585f60

index file:

org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.jar=org.apache.opendal:opendal-java:0.43.0:n/a:jar:jar:opendal-java-0.43.0.pom:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip=org.apache.opendal:opendal-java:0.43.0:source-release:zip:zip:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-sources.jar=org.apache.opendal:opendal-java:0.43.0:sources:java-source:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-javadoc.jar=org.apache.opendal:opendal-java:0.43.0:javadoc:javadoc:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-linux-aarch_64.jar=org.apache.opendal:opendal-java:0.43.0:linux-aarch_64:jar:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip.sha512=org.apache.opendal:opendal-java:0.43.0:source-release:zip.sha512:zip.sha512:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.jar.asc=org.apache.opendal:opendal-java:0.43.0:n/a:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.pom.asc=org.apache.opendal:opendal-java:0.43.0:n/a:pom.asc:pom.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip.asc=org.apache.opendal:opendal-java:0.43.0:source-release:zip.asc:zip.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-sources.jar.asc=org.apache.opendal:opendal-java:0.43.0:sources:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-javadoc.jar.asc=org.apache.opendal:opendal-java:0.43.0:javadoc:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-linux-aarch_64.jar.asc=org.apache.opendal:opendal-java:0.43.0:linux-aarch_64:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.jar=org.apache.opendal:opendal-java:0.43.0:n/a:jar:jar:opendal-java-0.43.0.pom:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip=org.apache.opendal:opendal-java:0.43.0:source-release:zip:zip:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-sources.jar=org.apache.opendal:opendal-java:0.43.0:sources:java-source:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-javadoc.jar=org.apache.opendal:opendal-java:0.43.0:javadoc:javadoc:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-linux-x86_64.jar=org.apache.opendal:opendal-java:0.43.0:linux-x86_64:jar:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip.sha512=org.apache.opendal:opendal-java:0.43.0:source-release:zip.sha512:zip.sha512:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.jar.asc=org.apache.opendal:opendal-java:0.43.0:n/a:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.pom.asc=org.apache.opendal:opendal-java:0.43.0:n/a:pom.asc:pom.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip.asc=org.apache.opendal:opendal-java:0.43.0:source-release:zip.asc:zip.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-sources.jar.asc=org.apache.opendal:opendal-java:0.43.0:sources:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-javadoc.jar.asc=org.apache.opendal:opendal-java:0.43.0:javadoc:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-linux-x86_64.jar.asc=org.apache.opendal:opendal-java:0.43.0:linux-x86_64:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.jar=org.apache.opendal:opendal-java:0.43.0:n/a:jar:jar:opendal-java-0.43.0.pom:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip=org.apache.opendal:opendal-java:0.43.0:source-release:zip:zip:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-sources.jar=org.apache.opendal:opendal-java:0.43.0:sources:java-source:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-javadoc.jar=org.apache.opendal:opendal-java:0.43.0:javadoc:javadoc:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-windows-x86_64.jar=org.apache.opendal:opendal-java:0.43.0:windows-x86_64:jar:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip.sha512=org.apache.opendal:opendal-java:0.43.0:source-release:zip.sha512:zip.sha512:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.jar.asc=org.apache.opendal:opendal-java:0.43.0:n/a:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.pom.asc=org.apache.opendal:opendal-java:0.43.0:n/a:pom.asc:pom.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip.asc=org.apache.opendal:opendal-java:0.43.0:source-release:zip.asc:zip.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-sources.jar.asc=org.apache.opendal:opendal-java:0.43.0:sources:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-javadoc.jar.asc=org.apache.opendal:opendal-java:0.43.0:javadoc:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-windows-x86_64.jar.asc=org.apache.opendal:opendal-java:0.43.0:windows-x86_64:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.jar=org.apache.opendal:opendal-java:0.43.0:n/a:jar:jar:opendal-java-0.43.0.pom:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip=org.apache.opendal:opendal-java:0.43.0:source-release:zip:zip:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-sources.jar=org.apache.opendal:opendal-java:0.43.0:sources:java-source:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-javadoc.jar=org.apache.opendal:opendal-java:0.43.0:javadoc:javadoc:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-osx-aarch_64.jar=org.apache.opendal:opendal-java:0.43.0:osx-aarch_64:jar:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip.sha512=org.apache.opendal:opendal-java:0.43.0:source-release:zip.sha512:zip.sha512:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.jar.asc=org.apache.opendal:opendal-java:0.43.0:n/a:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.pom.asc=org.apache.opendal:opendal-java:0.43.0:n/a:pom.asc:pom.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip.asc=org.apache.opendal:opendal-java:0.43.0:source-release:zip.asc:zip.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-sources.jar.asc=org.apache.opendal:opendal-java:0.43.0:sources:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-javadoc.jar.asc=org.apache.opendal:opendal-java:0.43.0:javadoc:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-osx-aarch_64.jar.asc=org.apache.opendal:opendal-java:0.43.0:osx-aarch_64:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.jar=org.apache.opendal:opendal-java:0.43.0:n/a:jar:jar:opendal-java-0.43.0.pom:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip=org.apache.opendal:opendal-java:0.43.0:source-release:zip:zip:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-sources.jar=org.apache.opendal:opendal-java:0.43.0:sources:java-source:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-javadoc.jar=org.apache.opendal:opendal-java:0.43.0:javadoc:javadoc:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-osx-x86_64.jar=org.apache.opendal:opendal-java:0.43.0:osx-x86_64:jar:jar:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip.sha512=org.apache.opendal:opendal-java:0.43.0:source-release:zip.sha512:zip.sha512:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.jar.asc=org.apache.opendal:opendal-java:0.43.0:n/a:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0.pom.asc=org.apache.opendal:opendal-java:0.43.0:n/a:pom.asc:pom.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-source-release.zip.asc=org.apache.opendal:opendal-java:0.43.0:source-release:zip.asc:zip.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-sources.jar.asc=org.apache.opendal:opendal-java:0.43.0:sources:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-javadoc.jar.asc=org.apache.opendal:opendal-java:0.43.0:javadoc:jar.asc:jar.asc:n/a:n/a:n/a:n/a
org/apache/opendal/opendal-java/0.43.0/opendal-java-0.43.0-osx-x86_64.jar.asc=org.apache.opendal:opendal-java:0.43.0:osx-x86_64:jar.asc:jar.asc:n/a:n/a:n/a:n/a

file tree:

./staging
└── 16a16cf8585f60
    ├── .index
    └── org
        └── apache
            └── opendal
                └── opendal-java
                    ├── 0.43.0
                    │   ├── opendal-java-0.43.0.jar
                    │   ├── opendal-java-0.43.0.jar.asc
                    │   ├── opendal-java-0.43.0-javadoc.jar
                    │   ├── opendal-java-0.43.0-javadoc.jar.asc
                    │   ├── opendal-java-0.43.0-linux-aarch_64.jar
                    │   ├── opendal-java-0.43.0-linux-aarch_64.jar.asc
                    │   ├── opendal-java-0.43.0-linux-x86_64.jar
                    │   ├── opendal-java-0.43.0-linux-x86_64.jar.asc
                    │   ├── opendal-java-0.43.0-osx-aarch_64.jar
                    │   ├── opendal-java-0.43.0-osx-aarch_64.jar.asc
                    │   ├── opendal-java-0.43.0-osx-x86_64.jar
                    │   ├── opendal-java-0.43.0-osx-x86_64.jar.asc
                    │   ├── opendal-java-0.43.0.pom
                    │   ├── opendal-java-0.43.0.pom.asc
                    │   ├── opendal-java-0.43.0-source-release.zip
                    │   ├── opendal-java-0.43.0-source-release.zip.asc
                    │   ├── opendal-java-0.43.0-source-release.zip.sha512
                    │   ├── opendal-java-0.43.0-sources.jar
                    │   ├── opendal-java-0.43.0-sources.jar.asc
                    │   ├── opendal-java-0.43.0-windows-x86_64.jar
                    │   └── opendal-java-0.43.0-windows-x86_64.jar.asc
                    └── maven-metadata-nexus.xml

7 directories, 23 files

@Xuanwo
Add more log for merging
a004cf9 
Signed-off-by: Xuanwo <github@xuanwo.io>
tisonkun
tisonkun approved these changes Dec 20, 2023
View reviewed changes
Copy link
Member

@tisonkun tisonkun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

.github/workflows/release_java.yml Outdated Show resolved Hide resolved
@Xuanwo
polish
a19b694 
Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo Xuanwo merged commit e3070e6 into main Dec 20, 2023
21 checks passed
@Xuanwo Xuanwo deleted the fix-java-release branch December 20, 2023 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tisonkun tisonkun tisonkun approved these changes

@PsiACE PsiACE Awaiting requested review from PsiACE

@suyanhanx suyanhanx Awaiting requested review from suyanhanx

Assignees
No one assigned
Labels
releases-note/fix
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

release: Binding Java doesn't have linux and windwos artifacts
3 participants
@Xuanwo @tisonkun @suyanhanx