Add github actions to clean and release

[DeafBDoor]

First proposal of the actions used to clean and provide a release branch for the repository.

[vanderpol]

Thanks for this @DeafBDoor, I have a minor suggestion, and a couple questions.

1. I think we should add the 'tools' and '.github' directory to the list of directories to clean up, along with all of the files in the root which should be a trivial change. See attached, as I think we would only want the oval schemas and spec to be included
   

2. Also, is there any way to have this script also generate the ReadTheDocs documentation, or should that be a separate action? Ideally I'd want to create both the .rst files needed for https://oval-community-guidelines.readthedocs.io/en/latest/oval-schema-documentation/index.html, and I'd like to include an HTML version of the docs with the release if at all possible.

3. Just for my own ignorance on how this works, I'm assuming we run this action, and then use github to perform a release, pointing it at the branch that this action just created?

[solind]

@vanderpol I'm not a big Github actions user; @A-Biggs or @maxullman would be better reviewers for this activity.

You may want to look at the ovaldocs target in the old jOVAL SCAP schema project, which builds XML documentation from the XML schema:
https://github.com/joval/jOVAL/blob/master/scap/GNUmakefile

As part of that build process, it also creates .sch and .xsl files for the OVAL schematron rules. I think it would be a good practice to include those in release artifacts as well.

To accomplish that, we included a copy of Saxon in the repository, and ran various transforms on the schema that can be found here:
https://github.com/joval/jOVAL/blob/master/scap/rsrc

[DeafBDoor]

Thanks for this @DeafBDoor, I have a minor suggestion, and a couple questions.

1. I think we should add the 'tools' and '.github' directory to the list of directories to clean up, along with all of the files in the root which should be a trivial change. See attached, as I think we would only want the oval schemas and spec to be included
   

Sure. I think I'm unable to remove the .git directory, because the release mechanism actually uses git tags for that. Note, however, that the checked out repo only has depth 1, thus only brings the latest changes.
But just to confirm: you only want the oval-schemas and oval-specifications directories, aside from generated docs, right?

UPDATE looks like I misunderstood the release mechanism of github. It creates a shallow repo and removes the .git already. So only the .github needs to be removed. I'm on it.

2. Also, is there any way to have this script also generate the ReadTheDocs documentation, or should that be a separate action? Ideally I'd want to create both the .rst files needed for https://oval-community-guidelines.readthedocs.io/en/latest/oval-schema-documentation/index.html, and I'd like to include an HTML version of the docs with the release if at all possible.

Yes, we can add a job before the one I created which executes the tools to generate the docs. I'll add it as an improvement to the PR.

3. Just for my own ignorance on how this works, I'm assuming we run this action, and then use github to perform a release, pointing it at the branch that this action just created?
   Yes. The action generates a tag based on the input sent. I suggest you clone the repo with that change and play with it a bit to see if it's acceptable.

As I mentioned, this is a first version, so I have in mind some improvements, like setting a latest tag, which will depend on your acceptance of the action interface.

[vanderpol]

Merging pull request so we can see how this works.

[vanderpol]

Merging pull request so we can see how this works.

[DeafBDoor]

Hi @vanderpol, I noticed this was closed without a merge, so you probably found a better way to make this work. Should I stop working on the version of the action which handles the documentation?

[vanderpol]

I must have made a mistake, my intention was to merge it, and then nothing happened, so I thought i had to close it. I'll go back and see if I can figure out what I screwed up.

…

On Tue, Oct 15, 2024, 7:11 PM Richard Maciel ***@***.***> wrote: Hi @vanderpol <https://github.com/vanderpol>, I noticed this was closed without a merge, so you probably found a better way to make this work. Should I stop working on the version of the action which handles the documentation? — Reply to this email directly, view it on GitHub <#178 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA264GUUANVRHLFONINHHW3Z3WOLRAVCNFSM6AAAAABP3T5PHCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJVGMZDEMJXGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[vanderpol]

I swear I did this earlier today, but I apparently didn't do something right. Sorry for the confusion, it's merged now (in theory). Let me know if you have any issues with it.

[A-Biggs]

I'm a bit late to the game here, but wanted to comment anyways.

@DeafBDoor if I understand your github action script correctly, if is essentially creating a tag and release branch for which contains a stripped down version of the files applicable to the release.

My thought on this was to make use of the github release functionality (https://github.com/OVAL-Community/OVAL/releases) to create a release, and to trigger the GH action, so instead of:

on:
  workflow_dispatch:
    inputs:
      version_tag:
        description: 'Associated tag to the release version'

it would trigger using something like:

on:
  release:
    types: [ published, edited ]

The use of GH release process forces a tag to be created as part of the release creation, and it will automatically attached the Source Code.zip/tar.gz files. The run command gh release upload <tag> <list of files> --clobber --repo $OWNER/$REPO can then be used to upload the various files that want to be part of the release (which would be all the various .xsd files if we want to mirror the files included in the 5.11.2 release at https://github.com/OVAL-Community/OVAL/releases, but could easily also add .sch and .xsl files if they were built as part of the build process).

We should figure out how we want to deliver a release, as instead of all the various files being attached to a release, it might be better to create a zip file with all the schemas, docs, etc... and just attached the one file, or attach the schema files separately by attach a zip for the generated schema docs (once that is implemented)?

[vanderpol]

@A-Biggs I don't have much to say about how we get there, I'm open to whatever is easier and more consistent to implement, maybe you and @DeafBDoor can work together and come up with a gameplan.

For the releases, I created the 5.12 as a potential template (and starting point for discussions like this!), and I'm open to feedback on what 'should' be in the release, for 5.12 I was thinking that having all of the .xsd file visible, and not just in a zip would make them easily publicly viewable, so content authors could include those static linked files in their content, but I realized that the URL's in the release are an automatic download

https://github.com/OVAL-Community/OVAL/releases/download/5.11.2-import/aix-definitions-schema.xsd
vs
https://raw.githubusercontent.com/OVAL-Community/OVAL/5.11.2-import/oval-schemas/aix-definitions-schema.xsd

I'm not sure the implications of that when it comes to schema validation with XML editing tools? If content authors are going to point xsi:schemaLocation= to the raw.githubusercontent.com URL, then it makes more sense to just provide the zip as the release vs 50+ xsd files.

Thoughts?

[DeafBDoor]

I'm a bit late to the game here, but wanted to comment anyways.

@DeafBDoor if I understand your github action script correctly, if is essentially creating a tag and release branch for which contains a stripped down version of the files applicable to the release.

My thought on this was to make use of the github release functionality (https://github.com/OVAL-Community/OVAL/releases) to create a release, and to trigger the GH action, so instead of:

Hmmmm... I'm not sure I follow this: the release requires a tag, but the tag is associated with a specific branch/commit history. So you need to clean up the files, generate an associated commit and tag it before you can do the release.

The release mechanism provides a way to upload additional assets.

Or am I doing something wrong here?

on:
  workflow_dispatch:
    inputs:
      version_tag:
        description: 'Associated tag to the release version'

it would trigger using something like:

on:
  release:
    types: [ published, edited ]

The use of GH release process forces a tag to be created as part of the release creation, and it will automatically attached the Source Code.zip/tar.gz files. The run command gh release upload <tag> <list of files> --clobber --repo $OWNER/$REPO can then be used to upload the various files that want to be part of the release (which would be all the various .xsd files if we want to mirror the files included in the 5.11.2 release at https://github.com/OVAL-Community/OVAL/releases, but could easily also add .sch and .xsl files if they were built as part of the build process).

Tried that, but it uploaded the release and I didn't find a way to remove the unwanted assets after creating the release. I'll check the docs again and see if I'm doing something wrong.

We should figure out how we want to deliver a release, as instead of all the various files being attached to a release, it might be better to create a zip file with all the schemas, docs, etc... and just attached the one file, or attach the schema files separately by attach a zip for the generated schema docs (once that is implemented)?

Yeah, I definitely need to get more experience with the release process. Need to understand the types of transformations I can do with the files after a release is generated. I'm not sure if the mechanism is that flexible.

[A-Biggs]

Hmmmm... I'm not sure I follow this: the release requires a tag, but the tag is associated with a specific branch/commit history. So you need to clean up the files, generate an associated commit and tag it before you can do the release.
@DeafBDoor If we attach all the artifacts to a release, do we care that the commit that the release references removes all the files that aren't attached as an artifact of that release, since people should be downloading items from the release itself as opposed to the tag?

I created a fork of this repo, and created a GH Action for uploading the schema files to the release assets list, for which the test release created using the GH Action I created match the list of the release @vanderpol created.

If we did want the release auto-attached Source Code files to be the clean version, then I could see that both of these GH actions are used. First clean_for_release workflow would be run, for which that would create a tag, and then that tag could be used in the creation of a new release, for which the GH action I created would then attach whatever assets we want to the release asset list.

As mentioned above, I think there is still some discussion to be had as to what should be all included in a release, and I left a placeholder step for building other artifacts, for which then could also be uploaded in the last step by adding the files to the list of files.

If this makes sense, and think the GH action I created is useful, I can create a PR to merge that from my fork to this repo.

[DeafBDoor]

Yes, it looks good to me.

[A-Biggs]

FYI, I just created PR if people want to review, approve and merge.