CI/Build: Add Gradle Build Scans

[snazy]

Gradle Build Scans, free of use, collect a lot of information about a Gradle build, including the actual output of failed test. This becomes quite convenient when inspecting test failures in CI and a lot of other information about Gradle builds.

Example build scan

[jbonofre]

@snazy what about using ASF Develocity ? It's hosted here: https://ge.apache.org/ and "hosted"/approved by the ASF.

I would prefer to use ASF hosted platform here.

[jbonofre]

For context: https://infra.apache.org/gradle.html

[snazy]

Good point, updated the PR.

[snazy]

Well, Apache's instance rejects unauthenticated build scans, which is always the case for PR CI runs (no secrets in those GH workflows). So I think it's fine to let CI runs that don't have the access-key secret publish to Gradle's infra, and those that have it publish to Apache's infra.

[adutra]

Well, Apache's instance rejects unauthenticated build scans, which is always the case for PR CI runs (no secrets in those GH workflows). So I think it's fine to let CI runs that don't have the access-key secret publish to Gradle's infra, and those that have it publish to Apache's infra.

Why can't we configure our PR CI runs to authenticate? The section named "GitHub Actions" in in this page says:

For GitHub Actions builds in the Apache GitHub organization, an access token is stored in the organizational secret GE_ACCESS_TOKEN.

[snazy]

Why can't we configure our PR CI runs to authenticate?

Security thing - to not expose secrets in such CI runs.

[jbonofre]

Two comments:

• we need approval from PPMC and security team to export data in a third party instance
• as other projects (like Apache Beam or Pekko) are using ge.apache.org, I wonder why we would be different 😄

[jbonofre]

See https://cwiki.apache.org/confluence/pages/viewpage.action?spaceKey=INFRA&title=Project+Onboarding+Instructions+for+Develocity

[snazy]

* as other projects (like Apache Beam or Pekko) are using ge.apache.org, I wonder why we would be different 😄

That still requires all GH workflow runs to have access to secrets, which is a big concern IMHO.

[jbonofre]

@snazy For GitHub Actions builds in the Apache GitHub organization, an access token is stored in the organizational secret GE_ACCESS_TOKEN. For security, GitHub Actions do not pass secrets to runners running workflows triggered from forked repositories.
I don't see the issue here as the GE_ACCESS_TOKEN is setup by INFRA.
For local build, users can use their Apache ID.

[snazy]

@jbonofre GH actions run from forks do NOT have any secrets, that's how GH workflows work.

[jbonofre]

@snazy yes, that's expected. That's why I don't the security question :) If an user forks Polaris, if he's a Apache committer, he can use his Apache account, else he doesn't push.

Sorry, I failed to see the point 😄

[snazy]

Yes, you failed the point. See the warning box under the discussion of pull_request_target vs the pull_request trigger. Apache Committer or not does not matter here.

[snazy]

TL;DR both scenarios, pull_request_target for CI against apache/polaris and using branches in the apache/polaris repo with pull_request are both huge security risks - see this post.

[jbonofre]

My point about Apache Committer is for local build (not related to GH Action).

I think I understand your point about PR, but I was talking about "regular" GH Action build.
For PRs, are you proposing to systematically publish build scans ?

[snazy]

For PRs, are you proposing to systematically publish build scans ?

Yes, that's basically the whole point - I want to inspect build scans for PRs, not just test results, but more than that.

[jbonofre]

@snazy so like this one: https://ge.apache.org/s/7hwlge7hwhch4
It's a build on a PR (apache/beam#32400) on Beam (action https://github.com/apache/beam/actions/runs/12012626739).

So it seems to work on Apache Beam.

[snazy]

@jbonofre that uses the insecure pull_request_target, right?

[jbonofre]

@snazy I'm checking this.

[github-actions]

This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[snazy]

Any updates on this @jbonofre ?

[jbonofre]

My apologize, I have been swamped by other stuff. Let me resume work on this. Sorry for the delay 😀

[flyrain]

Hi folk, do you still need this?

[snazy]

Build scans are very useful and much more convenient than downloading some archive, unpacking it, finding the right files, etc.
It's worth tackling this effort.

[snazy]

Closed in favor of #2559