Adding a config_automation.yml #435

cansavvy · 2022-02-08T14:42:33Z

Purpose/implementation Section

What changes are being implemented in this Pull Request?

What the user would need to do:

Change github actions in config_automation.yml to yes or no to indicate whether or not they should be on. When changes are made and pushed to config_automation.yml they are automatically incorporated by the manage-gha.yml GHA by it calling and running the scripts/employ-gha-switches.R script.

How scripts/employ-gha-switches.R works

The on triggers for each github action are different for each and they are stored in on-triggers.rds
scripts/employ-gha-switches.R reads in all workflows in the directory .github/workflows. It then reads in the specs from the user in config_automation.yml. It finds the on trigger by where it says TRIGGER-START and TRIGGER-END. It either replaces it with the on trigger (which is from the on-triggers.rds) for that particular gha OR it puts a manual dispatch trigger if its been specified to be turned off. This means all github actions can always be manually triggered by going to Actions, but they will not be run automatically unless specified as such in the config_automation.yml file.

How on triggers are updated -- maintenance purposes

store-triggers.R will update the current state of the GHA triggers. Normally store-triggers.R doesn't need to be run, but if the on trigger for any of the github actions changes, then store-triggers.R needs to be run to store the new version in the on-triggers.rds. BUT, first all GHA need to be turned on in config_automation.yml before store-triggers.R can be run.

Reorganization of Coursera as its own GHA render

***Note that I've also separated out render-coursera out into its own github actions so this is easier to manage! So now there's a separate yml for each bookdown, Leanpub, and Coursera. Bookdown happens first, which then triggers the Coursera render, which then triggers the Leanpub render.

github-actions · 2022-02-08T14:43:49Z

No spelling errors! 🎉
Comment updated at 2022-02-09 with changes from 3502022

github-actions · 2022-02-08T14:53:17Z

Re-rendered previews from the latest commit:

Updated at 2022-02-09 with changes from 64236f0

cansavvy · 2022-02-08T18:29:29Z

@avahoffman Do you want to see what you think about the general concept of this? It's not 100% completed, but it is mostly there.

carriewright11 · 2022-02-08T18:35:47Z

Thanks for adding to the sync file... I guess we don't need to sync get_spell_errors.R right? or git_repo_check.R

cansavvy · 2022-02-08T18:36:51Z

Thanks for adding to the sync file... I guess we don't need to sync get_spell_errors.R right?

I should have put this add in a different PR. Whoops. But actually we don't want to sync get_spell_errors.R so I'll add that.

carriewright11 · 2022-02-08T18:40:33Z

Thanks for adding to the sync file... I guess we don't need to sync get_spell_errors.R right?

I should have put this add in a different PR. Whoops. But actually we should sync get_spell_errors.R so I'll add that.

what about the repo check... I noticed that got deleted from the OTTR Web repo

That's only used in transfer-rendered-files.yml at this point, so you don't need it. - wow I didn't know you could edit other peoples comments :P

carriewright11 · 2022-02-08T18:44:25Z

thanks @cansavvy ! Also this looks like a great start for the plan we discussed!

cansavvy · 2022-02-09T14:56:56Z

Proposed plan of action for @avahoffman review

Do you have an idea for how to make the set up of on-trigger.rds better? It's pretty shoot yourself in the foot right now.
Give it a review overall and if/when it seems reasonabl we can merge it to staging
Test it out on staging branch -- create some pull requests, play around with turning GHA on and off -- how easy/not easy is it to use? Try to break it while testing.

If steps 1 - 3 go well and we address any bugs we find, then lets merge to main and test with a couple PRs, then make a new release and ship it out!

cansavvy · 2022-02-09T15:01:24Z

Additionally, I need to make delete-preview.ymls trigger tied to render-preview.yml

* Add the polishing changes * No manage-gha * No login needed * Make it simpler for commits and pushes

…/main-gha

cansavvy · 2022-02-09T16:16:51Z

scripts/store-triggers.R

@@ -0,0 +1,32 @@
+# Write on-trigger yaml
+# This script stores what the 'on' triggers look like for each github action


This is the bad script that stores whatever is in the current version of the gha which requires you to turn all the github actions ON before running this.

cansavvy · 2022-02-09T16:18:22Z

scripts/employ-gha-switches.R

+ dplyr::filter(basename(gha_file) == gha_files) %>%
+ dplyr::pull("on_or_off")
+
+ if (length(status) == 1) {


It's also bad that the trigger gets overwritten whether or not there is a change specified in config_automation.yml

I think my next idea for this, and for the other problem is what if we store a yaml of all the triggers. (.github/triggers.yml)
So for example:

check-quizzes: status: on on: <What the trigger for on looks like> off: <What the trigger for off looks like> render-bookdown: status: on on: < ... > off: < ... >

This format would be far easier to manage than an RDS because we can 1) Just change the triggers in this file directly using a text editor and 2) We can use it to check the current status with the previous status so we can avoid overwriting everytime.

My hope is that this kind of yaml would allow the store-triggers.R script render unnecessary completely and would make everything a lot more streamlined.

with the latest iteration of this, store-triggers.R is not obsolete and now you just need to specify things in the gha-triggers.yml if you want to change the triggers.

cansavvy · 2022-02-09T16:19:04Z

@avahoffman I commented on the troublesome places with this PR and also neatened it up a bit. Let me know how it goes. I'll brainstorm a bit myself and comment with any ideas.

cansavvy · 2022-02-09T18:52:44Z

Meanwhile, I'm going to look into Python's handling of yamls, since we suspect it handles yamls better than R, but question is does it handle yamls so much better that we can switch strategies to handle yamls as yamls. pyyaml is where I'm going to start.

Edit: Same story with this -- extra spacing lines and comments are removed if we use pyyaml to write yaml files. So from a quick review, it does not appear that switching to python will allow us to get over this particular problem.

avahoffman · 2022-02-09T21:03:42Z

@cansavvy good to know. I'm playing with an alternative approach that might work well right now. Stay tuned!

cansavvy · 2022-02-10T16:19:39Z

Closing this. We have something better going on on #445 , #448, and #449

cansavvy added 17 commits February 7, 2022 20:22

Neatening up and trying to control gha triggers

3a056ae

Add some config mechs

9d154da

One strategy for yamls

964e16d

Fix trigger syntax

bdf4d65

Try to fix syntax part 2

a853cf7

Fix syntax part 3

0d84ced

Add TRIGGER-START

43266db

This is mostly working

996e502

Set up gha

a85d3fd

This seems to work

579c801

Updates to script

a56d5ba

Updates to script 2

6d6ca3b

Nope

975148f

Don't delete last line

35e1d60

script edit

a0d2c02

Added some extra lines

6d36644

Don't add extra spaces

b200741

cansavvy added 2 commits February 8, 2022 09:48

Get rid of packages.bib

e3e658c

No packages.bib

a2f97a6

Add OTTR Website

fa7b618

cansavvy requested a review from avahoffman February 8, 2022 18:26

Update sync

a398c81

Get rid of git_repo_check.R too

cc6c6e4

jhudsl-robot added 2 commits February 9, 2022 08:52

Polishing up comments and etc

60e27ae

No spaces

28fa844

cansavvy added 7 commits February 9, 2022 10:33

Update trigger handling

6c31f07

Fix GHA branch handling

1274e9e

Echo if no changes to commit

5a429cc

Only add github/workflows folder

28c0098

force push

c25a044

Try to fix manage GHA trigger

2fbb6b7

Fix commenting

62afbf7

cansavvy mentioned this pull request Feb 9, 2022

Add the polishing changes from #435 in their own PR #442

Merged

cansavvy added a commit that referenced this pull request Feb 9, 2022

Add the polishing changes from #435 in their own PR (#442)

4a96ab3

* Add the polishing changes * No manage-gha * No login needed * Make it simpler for commits and pushes

cansavvy added 4 commits February 9, 2022 11:10

Merge branch 'staging' into cansavvy/main-gha

70cd861

Fix file changes

97eee77

Merge remote-tracking branch 'origin/cansavvy/main-gha' into cansavvy…

01fab8e

…/main-gha

Merge branch 'staging' into cansavvy/main-gha

96ad996

cansavvy commented Feb 9, 2022

View reviewed changes

Merge branch 'staging' into cansavvy/main-gha

559d556

cansavvy commented Feb 9, 2022

View reviewed changes

cansavvy added 2 commits February 9, 2022 14:30

Add a gha-triggers.yml file

9e2d930

Update workflow name

cba55f8

cansavvy added 2 commits February 9, 2022 16:55

Move yml and fix script

316072d

Don't need store-triggers.R anymore!

0e55b55

cansavvy closed this Feb 10, 2022

cansavvy deleted the cansavvy/main-gha branch February 11, 2022 13:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding a config_automation.yml #435

Adding a config_automation.yml #435

cansavvy commented Feb 8, 2022 •

edited

Loading

github-actions bot commented Feb 8, 2022 •

edited

Loading

github-actions bot commented Feb 8, 2022 •

edited

Loading

cansavvy commented Feb 8, 2022

carriewright11 commented Feb 8, 2022 •

edited

Loading

cansavvy commented Feb 8, 2022 •

edited

Loading

carriewright11 commented Feb 8, 2022 •

edited

Loading

carriewright11 commented Feb 8, 2022

cansavvy commented Feb 9, 2022

cansavvy commented Feb 9, 2022 •

edited

Loading

cansavvy Feb 9, 2022

cansavvy Feb 9, 2022

cansavvy Feb 9, 2022

cansavvy Feb 9, 2022

cansavvy Feb 9, 2022

cansavvy commented Feb 9, 2022

cansavvy commented Feb 9, 2022 •

edited

Loading

avahoffman commented Feb 9, 2022

cansavvy commented Feb 10, 2022

		@@ -0,0 +1,32 @@
		# Write on-trigger yaml
		# This script stores what the 'on' triggers look like for each github action

Adding a config_automation.yml #435

Adding a config_automation.yml #435

Conversation

cansavvy commented Feb 8, 2022 • edited Loading

Purpose/implementation Section

What changes are being implemented in this Pull Request?

What the user would need to do:

How scripts/employ-gha-switches.R works

How on triggers are updated -- maintenance purposes

Reorganization of Coursera as its own GHA render

github-actions bot commented Feb 8, 2022 • edited Loading

github-actions bot commented Feb 8, 2022 • edited Loading

cansavvy commented Feb 8, 2022

carriewright11 commented Feb 8, 2022 • edited Loading

cansavvy commented Feb 8, 2022 • edited Loading

carriewright11 commented Feb 8, 2022 • edited Loading

carriewright11 commented Feb 8, 2022

cansavvy commented Feb 9, 2022

cansavvy commented Feb 9, 2022 • edited Loading

cansavvy Feb 9, 2022

Choose a reason for hiding this comment

cansavvy Feb 9, 2022

Choose a reason for hiding this comment

cansavvy Feb 9, 2022

Choose a reason for hiding this comment

cansavvy Feb 9, 2022

Choose a reason for hiding this comment

cansavvy Feb 9, 2022

Choose a reason for hiding this comment

cansavvy commented Feb 9, 2022

cansavvy commented Feb 9, 2022 • edited Loading

avahoffman commented Feb 9, 2022

cansavvy commented Feb 10, 2022

cansavvy commented Feb 8, 2022 •

edited

Loading

github-actions bot commented Feb 8, 2022 •

edited

Loading

github-actions bot commented Feb 8, 2022 •

edited

Loading

carriewright11 commented Feb 8, 2022 •

edited

Loading

cansavvy commented Feb 8, 2022 •

edited

Loading

carriewright11 commented Feb 8, 2022 •

edited

Loading

cansavvy commented Feb 9, 2022 •

edited

Loading

cansavvy commented Feb 9, 2022 •

edited

Loading