Detect changed workspaces & trigger CI

[tudi2d]

Closes #6

To be reviewed:

At this point it is the goal to get a basic CI up & running. Therefor the only files to be reviewed are the ci.yml workflow & the scripts/ci folder.

All other +150 files in workspaces will be removed once approved. They are only needed to actually see the workflows running & give a minimal example.

Current idea of the flow:

1. Detect workspaces with changes - we expect 100+ plugins in the repository so CI should really only run on changed workspaces
2. Once the workspace is detected we can run CI for Node 18 & 20
3. The same goes for the verification where we check for duplication in the yarn.lock file of the workspace & verify the changeset only mentions non-private packages in the workspace (e.g. no release for the example app)

Questions:

• Are we missing anything basic at the moment?

TODO:

• Verify changesets
• Run tests
• Add api-reports
• Fix template to not throw prettier warnings (Fix template prettier issues backstage#23139)
• Remove example worspaces after review

[tudi2d]

Run into prettier warnings with the default template, which lead to failing CI. That's not great & should be fixed!

[tudi2d]

Here is the PR fixing the prettier issue I ran into earlier: backstage/backstage#23139

[tudi2d]

The current example workspaces will be removed after review & should not be considered in the review. It just helps to showcase the current workflows & their capabilities.

[freben]

Is this one worth having here? Not really for end users to run. Might as well just call this command directly inside the CI script I think.

May also make sense to name these create/create-workspace.js and ci/list-workspaces-with-changes.js or so, helps find the right one in the future

[tudi2d]

yeah - good point - could be removed

[freben]

Suggested change

	const match = name.match("(?<=workspaces/)([^/]+)(?=/)");
	const match = name.match("workspaces/([^/]+)/");

No need for the fancy post-and prefix matching right? :)

Also, i don't know if these are relative or absolute, but it would feel better if we could anchor it with ^ at the start so it doesn't spuriously match on these words later in the path somewhere. Or string.split or whatnot

And speaking of that ... does this work on windows too? Will we have windows based tests later? Does the GitUtils always emit forward slashes or not

[vinzscam]

And speaking of that ... does this work on windows too? Will we have windows based tests later? Does the GitUtils always emit forward slashes or not

it shouldn't matter so much as this script is only meant to be run by the CI on an ubuntu runner

[tudi2d]

Adjusted the anchor.

The path name is comming from GitUtils in @backstage/cli-node, which is basically just running git diff. If I'm not mistaking git always handles paths the UNIX way (e.g. when writing a .gitignore you also have to use forward slashes) - and that should also be the case here with giving back the diff. I could not find a documentation proving my point though (only found example outputs of people running it on windows). My suggestion is to keep this in mind - should be rather easy to fix.

As Vincenzo said we also probably won't need to detect the affected workspaces on windows machines. Though want to clarify that we probably want to run tests & CI for workspaces on windows in the future.

[BethGriggs]

I've taken a look at each of the CI scripts and they LGTM. But, consider this approval more of an endorsement of the overall CI flow so far. It looks like a great starting point, thanks!

I feel it'll probably be easier to iterate and debug edge cases once we have this initial CI flow landed.