feat: refactor sync template workflow to work in a whitelist setting

[zugdev]

Resolves #61

The sync workflow should only propose changes automatically on shared files between repos, I agree those include some of the workflows and ts settings. What I would propose is to enable manual workflow runs with input files too. This way if you want every repo that uses the template to receive a specific new file or to create an overwriting, but not too big PR, you can do it. I believe the AI analysis even though might help, would still need a thorough review and would not reduce any time overhead. I am not sure centralizing the sync workflow is a good idea, if so we would need another way to identify which repos implement the template, to only open PRs in those. If it is really needed we can have a repos list in the workflow, but I think having the sync template in this repo and skipping when it runs here is sufficient to deliver updates to all sync-template workflows across our implementations.

1. Whitelist approach
2. Allow for inputting specific files and running manually
3. Adds sync-template to this repo and skips runs in ts-template

[ubiquity-os-deployer]

4ccb05c

afe768a

496fba1

8e53675

dd0c5f8

457cab9

c030c75

1977209

7df4c59

fc1c8c9

a68c37a

54944f2

baddf8d

c74d52b

da994a2

4c82745

e24cca6

9cbcc57

9aacc8d

0ceefd7

[zugdev]

Here is a QA, though it's failed because I don't have auth setup, you can see through the logs the modified files:

1. can create files with whitelist

2. can create files with additional manual input

3. can delete files

4. can modify files

[zugdev]

My solution enables individual repo configuration, the whitelist for this repo's sync-template.yml doesn't matter. What we should do is evaluate what files are shared between implementation and template and add it to the workflow in that specific implementation's repo. For example: in work.ubq.fi I know that the template's build workflow will work, so I should add it to the whitelist, on the other hand deploy.yml won't work - since we are implementing a KV setup. Another feature I've enabled is the manual synchronization with custom extra inputs, so:

1. If you want to sync the whitelisted files/directories with the template at any time you can manually do that with empty arguments

2. If you want to sync files/directories that aren't in the whitelist you can manually do that with paths separated by commas

Therefore a second step to this PR, is opening PRs in repos that implement ts-template with an initial configuration. I'll get started on that. It seems there a billion repos, what I could do to speed up is to initially setup the very obvious common files such as the .husky directory, and then with use we expand each repo's whitelist.

[ubiquity-os]

@zugdev, this task has been idle for a while. Please provide an update.

[zugdev]

I think this is a good default whitelist. The next steps are:

1. Placing the sync-template workflow in all repos we want it to be.

2. Installing the app and setting secrets in all repos

[0x4007]

Probably should be all .github/* and all root config files at least.

[zugdev]

Probably should be all .github/* and all root config files at least.

I know that the deploy workflow shouldn't be in work.ubq.fi case, perhaps I can add a blacklist so we can add the entire .github dir but remove specific files in each repo with the blacklist.

[ubiquity-os]

@zugdev, this task has been idle for a while. Please provide an update.

[zugdev]

I have the whitelist set to WHITELIST_FILES: ".github/ .husky/ .eslintrc .nvmrc .prettierrc .yarnrc.yml tsconfig.json"

Here is a QA simulating a blacklisted deploy workflow, as we will do in work.ubq.fi:

[1. Action Run]
[2. PR created]

I had to create a PTA to be able to change the .github directory which is protected by workflow write permissions. For some reason the GitHub app does not behave as expected and it's a known problem .

Also if you sync multiple times without merging it simply updates the PR, instead of opening new ones. Notice I've blacklisted sync-template.yml to avoid loss of .env vars, but this will lead to some QA errors since this repo does not yet contain that file.

[ubiquity-os]

@zugdev, this task has been idle for a while. Please provide an update.

[zugdev]

another QA flow:

1. first sync run with no additional arguments creates this PR. the pr body shows the following list of modified files:

This pull request merges changes from the template repository, overwriting or removing the following files:

- .github/ (directory)
- .husky/ (directory)
- .eslintrc
- .nvmrc
- .prettierrc
- .yarnrc.yml
- tsconfig.json

2. second sync run with static/index.html,static/main.ts as additional args, a commit is pushed and the PR is updated

This pull request merges changes from the template repository, overwriting or removing the following files:

- .github/ (directory)
- .husky/ (directory)
- .eslintrc
- .nvmrc
- .prettierrc
- .yarnrc.yml
- tsconfig.json
- static/index.html
- static/main.ts

the idea is an open sync PR can be incremented

[ubiquity-os]

@zugdev, this task has been idle for a while. Please provide an update.

[0x4007]

#61 (comment)

Do you think there is a way to make it a single script instead of a ton in every repo?

[zugdev]

I think so

[ubiquity-os]

@zugdev, this task has been idle for a while. Please provide an update.

[ubiquity-os]

@zugdev, this task has been idle for a while. Please provide an update.

[rndquu]

@zugdev So we need to copy/paste the sync-template.yml to all of our repositories? What if there's a change in the sync-template.yml file itself? How to propagate its update to ~100 of our repositories?

I actually thought there will be a single workflow file in the https://github.com/ubiquity/ts-template repository run on cron and workflow_dispatch with 3 inputs (with default values set for our organization):

• file_paths: to specify which files should be updated
• organizations: to specify which exact organizations should be affected
• repositories: to specify which exact repositories should be affected

[zugdev]

@zugdev So we need to copy/paste the sync-template.yml to all of our repositories? What if there's a change in the sync-template.yml file itself? How to propagate its update to ~100 of our repositories?

I actually thought there will be a single workflow file in the https://github.com/ubiquity/ts-template repository run on cron and workflow_dispatch with 3 inputs (with default values set for our organization):

• file_paths: to specify which files should be updated
• organizations: to specify which exact organizations should be affected
• repositories: to specify which exact repositories should be affected

Im refactoring in favor of this push from ts-template idea. It is better.