Publication source branches/tags and target URLs mapping/aliases

[bact]

This issue is for the discussion/documentation of the source branches/tags and the target URLs mapping for website/RDF publication, including default version/version aliases settings and appropriate version suffixes.

Background

With #1092 resolved, branches in spdx-spec repo got renamed like this:

• "master" -> "main"
• "development/v3.0.1" -> "develop"
• "development/v3.0" -> "support/3.0"

(spdx-3-model is not yet but will soon after 3.0.1 freeze)

Under new branch strategy:

• "main" is for the latest stable version, changes will be merged to this branch periodically (months) from a short-live administrative "release" branch (the "release" branch is created off the "develop")
• "develop" is for under development version, changes will be merged to this branch constantly (days)
• "support/3.0" is for the maintenance of version 3.0.x
• "hotfix" branches can be applied to "main" and "support" branches
• Translation is considered "support"
• See more in README at Branch Structure section

Publication CIs need to be adjusted, see PR #1146 for earlier discussion.

Notes

Note that currently the branch strategy is not entirely true yet:

• "main" should contain 3.0 (3.0.0), but currently is contains pre-3.0 development (the change log show "3.0 (TBD)")
• "support/3.0" contains 3.0, while in fact we can argue that the "support/3.0" should not exist yet at this time (since "develop" still contain 3.0.x development)

This is for historical/transitional reason and will eventually resolved soon after the release of 3.0.1.
It is very likely that at that point "support/3.0" will be post-3.0.1 maintenance, and "develop" will be 3.1 development.

Mapping proposal

I cannot really write it down to a general mapping rule yet, but below are some of the possible scenarios.

Hopefully we can use these for discussion, agreed on publication general mapping rule, and update #1146 with that general rule.

Note that will current mapping proposal, if we merge a "hotfix" to "main", the changes will not be get published unless we make another "release". Current proposal assume that all changes will go to either "develop" or "support" branch first.

To make things easier, we will neglect current facts stated in Notes above.

Scenario 1

• "main" is 3.0 release
• "develop" is 3.0.1 development
• "support/3.0" is not exist yet

Source branch/tag	Target URL
releases/tag/v3.0	https://spdx.github.io/spdx-spec/v3.0/
develop	https://spdx.github.io/spdx-spec/v3.0.1-dev/

Alias	Target version	Notes
(default)	v3.0	If no version given, redirect to the default version.
v3-draft	v3.0
v3.0-RC2	v3.0
v3.0.1	v3.0.1-dev

The "-dev" will be where spec and model developers can check the most recent changes, how HTML and RDF will look like.

Scenario 2

• "main" is 3.0.1 release
• "develop" is 3.1 development
• "support/3.0" is 3.0.x maintenance

Source branch/tag	Target URL	Notes
releases/tag/v3.0	https://spdx.github.io/spdx-spec/v3.0.0/	Republished in a new target URL
releases/tag/3.0.1	https://spdx.github.io/spdx-spec/v3.0.1/	New default
develop	https://spdx.github.io/spdx-spec/v3.1-dev/
support/3.0	https://spdx.github.io/spdx-spec/v3.0.2-dev/

Alias	Target version	Notes
(default)	v3.0.1	Updated
v3-draft	v3.0.1	Updated
v3.0-RC2	v3.0.1	Updated
v3.0	v3.0.1	New
v3.0.0	v3.0.0	New (do we still need this?)
v3.0.2	v3.0.2-dev	New
v3.1	v3.1-dev	New

Scenario 3

Same as Scenario 2 but now we have a release candidate for 3.1.

• "main" is 3.0.1 release
• "develop" is 3.1 development
• "support/3.0" is 3.0.x maintenance

Release candidate will be publish just like the stable release:

• While the version with "-RC" is from "develop" branch and considered under development, at the same time it is considered a "release", in a way that once released it should got fixated (as oppose to constantly changing as in the target URL with "-dev" suffix), so people can make reference to it reliably during the reviewing period.
  • See the existing RC: https://github.com/spdx/spdx-spec/releases/tag/v3.0-RC2
• This means we probably need a short-live "release" branch (similar to stable release) to publish to "-RC" target URL, because our "develop" publishes to "-dev" target URL.

Source branch/tag	Target URL	Notes
releases/tag/3.0.1	https://spdx.github.io/spdx-spec/v3.0.1/	Still default
releases/tag/3.1-RC1	https://spdx.github.io/spdx-spec/v3.1-RC1/
develop	https://spdx.github.io/spdx-spec/v3.1-dev/
support/3.0	https://spdx.github.io/spdx-spec/v3.0.2-dev/

Alias	Target version	Notes
(default)	v3.0.1
v3-draft	v3.0.1
v3.0-RC2	v3.0.1
v3.0	v3.0.1
v3.0.0	v3.0.0
v3.0.2	v3.0.2-dev
v3.1	v3.1-RC1	Updated

Scenario 4

• "main" is 3.1 release
• "develop" is 3.2 development
• "support/3.0" is 3.0.x maintenance
• "support/3.1" is 3.1.x maintenance

Source branch/tag	Target URL	Notes
releases/tag/3.0.1	https://spdx.github.io/spdx-spec/v3.0.1/
releases/tag/3.1	https://spdx.github.io/spdx-spec/v3.1/	New default
develop	https://spdx.github.io/spdx-spec/v3.2-dev/
support/3.0	https://spdx.github.io/spdx-spec/v3.0.2-dev/
support/3.1	https://spdx.github.io/spdx-spec/v3.1.1-dev/

Alias	Target version	Notes
(default)	v3.1	Updated
v3-draft	v3.1	Updated
v3.0-RC2	v3.0.1
v3.0	v3.0.1
v3.0.0	v3.0.0
v3.0.2	v3.0.2-dev
v3.1-RC1	v3.1	New
v3.1.1	v3.1.1-dev	New

Scenario 5

Same as Scenario 4 but now we have a new translation for 3.0.1.

• "main" is 3.1 release
• "develop" is 3.2 development
• "support/3.0" is 3.0.x maintenance
• "support/3.1" is 3.1.x maintenance

Source branch/tag	Target URL	Notes
releases/tag/3.0.1-ja	https://spdx.github.io/spdx-spec/v3.0.1/	Publish a support release (with new translation) to an existing target URL
releases/tag/3.1	https://spdx.github.io/spdx-spec/v3.1/	Still default
develop	https://spdx.github.io/spdx-spec/v3.2-dev/
support/3.0	https://spdx.github.io/spdx-spec/v3.0.2-dev/
support/3.1	https://spdx.github.io/spdx-spec/v3.1.1-dev/

Alias	Target version	Notes
(default)	v3.1
v3-draft	v3.1
v3.0-RC2	v3.0.1
v3.0	v3.0.1
v3.0.0	v3.0.0
v3.0.2	v3.0.2-dev
v3.1-RC1	v3.1
v3.1.1	v3.1.1-dev

(Alias table remains unchanged from Scenario 4)

More questions about CI trigger

• At which point in time/what kind of event should we trigger the publication CI?
  • Under new branch structure, we will have an administrative "release" branch. Should we trigger the publication CI from that "release" branch instead? I'm not entirely understand how "release" branch will be created.
• How are we going to publish the translation addition? Does what proposed in Scenario 5 make sense?

[goneall]

"main" should contain 3.0 (3.0.0), but currently is contains pre-3.0 development (the change log show "3.0 (TBD)")

This looks like an error - Since 3.0.1 is close, let's just fix this once we release 3.0.1 - then the branch history should get fixed when we merge the 3.0.1 changes.

[goneall]

Thanks @bact for thinking through all the scenarios.

I think scenario 1 can be implemented soon - if others agree.

Once we release 3.0.1, we can move to scenario 2.

[goneall]

At which point in time/what kind of event should we trigger the publication CI?

I'm not planning on using release branches for releases - just tags and merging into the other branches. So I think the publication CI would occur when tagged. We can do this manually. We'll need to update our release checklist with whatever we decide.

[goneall]

How are we going to publish the translation addition? Does what proposed in Scenario 5 make sense?

What I'm thinking is the following:

• Translations are created as a pull request against the support/vX branches
• After review, we'll merge into the support branch
• Once we decide we want to release one or more translations, we'll tag the commit in the support branch vX.Y.Z-[cc] - where [cc] is the country code(s)
• After tagging we'll merge into the develop and main branches

Let me know your thoughts on the above process.

If the above process works, we can trigger re-publishing on the merge to the main branch for the currently release version.

For the previously released versions, perhaps when we tag the support branch? Could be a manual process - we just need to document in that case.

[bact]

I think that works.

The translators can review the results live at one of the staging websites for support branches at spdx.github.com/spdx-spec/X.Y.Z-dev, and once everyone are happy, we make a tag, and that tag will be published to spdx.github.com/spdx-spec/X.Y.Z

Btw, we drop "v" from the tag as well. And in the place of "cc" that should be the language code. See examples of valid language codes at https://github.com/spdx/spdx-3-model/blob/main/docs%2Ftranslation.md

[bact]

I think scenario 1 can be implemented soon - if others agree.

Agree. It will give us opportunities as well to try things out before the release of 3.0.1.

[bact]

At which point in time/what kind of event should we trigger the publication CI?

I'm not planning on using release branches for releases - just tags and merging into the other branches. So I think the publication CI would occur when tagged. We can do this manually. We'll need to update our release checklist with whatever we decide.

I'm thinking of using tags as well.
GitHub emits events that we can use for CI to get triggered (it probably already works that way), and yes we always have the manual option as a human-in-the-loop safety.

[zvr]

Looking at the 4 bullets in @goneall last comment, I'm not sure everything is clear -- nor that I agree with everything:

1. I think it presumes that all of a translation (in language XX, say) are in one PR against a support/a.b.c branch. Realistically, we would want to have another branch for "working on translating a.b.c to XX" where multiple commits/PRs will happen. How do we call this branch?
2. "After [finalizing the translation and] review, we'll merge into the support branch"
3. Tagging. Contrary to others, I personally do not view tags as immutable. So I agree with tagging the merged translation as "a.b.c-XX", but I would also change the "a.b.c" tag to also point to this, the latest and most complete version of the a.b.c spec, instead of the English-only version of the spec. We can add an a.b.c-en to the original point, if we want. After a couple of languages are merged, we would have the series of tags (from oldest to newest): a.b.c.-en, a.b.c-de, a.b.c-ja, a.b.c-cn == a.b.c (which includes all translations).
4. "After tagging we'll merge into the develop and main branches." Should we? Until now, the implied workflow is "once something has finalized (tagged), then we start accepting translation work". What is the use of having a frozen "a.b.c in XX" content in a "develop" content that will eventually become "d.e.f" (> a.b.c) ?

[bact]

I don't think we can have a clear a.b.c-lang tag.

For example, if after we already have a first translation from English, let's say that that language is German and we have a.b.c-de.

That a.b.c-de contains two languages: English and German.

Then later, someone starts a translation of Swahili based on existing files (in spdx-spec and spdx-3-model repos). The submitting PR will contain three languages: English, German, and Swahili. Unless we asked them to remove de/ and en/ directories (for spdx-spec) or remove English/German summary/description in model files.

The changeset in PR, in principle, should only contain the addition of a new language - but we are talking about a tag which in GitHub means you can download the entire snapshot of a repo. This is not like a "language pack" that we can selectively pick a particular language (a.b.c-de contains only German, a.b.c-sw contains only Swahili). It is accumulative. One language on top of another. So realistically, it will be "a.b.c-en-de-sw".

[bact]

1. I think it presumes that all of a translation (in language XX, say) are in one PR against a support/a.b.c branch. Realistically, we would want to have another branch for "working on translating a.b.c to XX" where multiple commits/PRs will happen. How do we call this branch?

Can we adapt the "profile" branch approach?

Have a convention that for translation, please prefix it with "lang-" followed by a language code. And work on that branch.

Once ready for review, create a PR against "support/x.y"?

[zvr]

You're thinking it from a wrong point of view: the meaning of a tag a.b.c-XX is that this contains the released content of a.b.c in language XX. It might contain other stuff, as well: the English version will certainly be there, and all the other languages that came before.
Getting a.b.c-XX should get you the XX translation, and getting a.b.c should get you everything.

It makes no sense to use things like a.b.c-XX-YY-ZZ since this is dependent on the set of translations done before XX and why should anyone know this?

[bact]

Thanks. I'm not suggesting that we're going to use that for public facing tag names. I just use that as an instrument to explain what languages are realistically inside at each tag at each state, and to avoid anyone else to think about a "language pack" analogy. I'm glad that that message goes across.

[bact]

3. Tagging. Contrary to others, I personally do not view tags as immutable. So I agree with tagging the merged translation as "a.b.c-XX", but I would also change the "a.b.c" tag to also point to this, the latest and most complete version of the a.b.c spec, instead of the English-only version of the spec. We can add an a.b.c-en to the original point, if we want. After a couple of languages are merged, we would have the series of tags (from oldest to newest): a.b.c.-en, a.b.c-de, a.b.c-ja, a.b.c-cn == a.b.c (which includes all translations).

I think this can work. In a condition that it is well documented, because it is against common conventions (that people think that tag is immutable, but people may forget that a tag can be deleted and a new tag of a different commit can be created by the same name - so practically it is "mutable" if we only consider the tag label).

This can be an easy way to manage and make a reference to. Since we can keep the tag label constant (despite the fact that the underlying commit is actually changing).

From the point of view of CI, this may make it easier too for publication CI to just publish whatever from "x.y.z" without worrying about new languages.

I'm not certain about consequences for other areas (as, for example, the URL of the previous tag label that got replaced will no longer be available? Edit: the tag-label-based URL will still be available with a new content), but for publication purpose this can be a neat way.

[goneall]

Looking at the 4 bullets in @goneall last comment, I'm not sure everything is clear -- nor that I agree with everything:

1. I think it presumes that all of a translation (in language XX, say) are in one PR against a support/a.b.c branch. Realistically, we would want to have another branch for "working on translating a.b.c to XX" where multiple commits/PRs will happen. How do we call this branch?

Yes - I agree this will be done on a separate branch. I personally don't think we need to standardize the naming of the branch - we can leave that up to the translators. They could even work in a separate fork of the repo - as long as it eventually results in a merge back to the appropriate branches.

2. "After [finalizing the translation and] review, we'll merge into the support branch"
3. Tagging. Contrary to others, I personally do not view tags as immutable. So I agree with tagging the merged translation as "a.b.c-XX", but I would also change the "a.b.c" tag to also point to this, the latest and most complete version of the a.b.c spec, instead of the English-only version of the spec. We can add an a.b.c-en to the original point, if we want. After a couple of languages are merged, we would have the series of tags (from oldest to newest): a.b.c.-en, a.b.c-de, a.b.c-ja, a.b.c-cn == a.b.c (which includes all translations).

Although tags are not technically immutable, many people expect that they will not change - so I'd like to avoid changing the commit associated with a tag.

Good point on the naming. We should come up with a better scheme then what was originally proposed.

Perhaps just a number (e.g. a.b.c-l1, a.b.c-l2 ...). Another alternative is to bump the patch version on every language merge.

4. "After tagging we'll merge into the develop and main branches." Should we? Until now, the implied workflow is "once something has finalized (tagged), then we start accepting translation work". What is the use of having a frozen "a.b.c in XX" content in a "develop" content that will eventually become "d.e.f" (> a.b.c) ?

I'm not sure I understand. The develop branch is where the latest development is occurring. Don't we want to include the translations such that the next release will have the translations included? There is a valid question / concern on how we deal with changes in the spec when translated text is modified. This won't happen very often, but when it does need need a process to flag that the text needs re-translation.

[bact]

Another alternative is to bump the patch version on every language merge.

Our patch version is "sensitive" as we decided to have the patch version inside the element IRIs.

So while I like the idea, as it is very straightforward, I'm reluctant to touch the patch version ... unless there's a way we can conveniently (automatically) say that an instance in 3.0.x is the same as in 3.0.y (there's owl:sameAs that @goneall proposed a while ago, but that needs to be implemented).

OR, if we can make the IRIs stationary.
For example , with something that has a similar effect to "sinceVersion" metadata as discussed in spdx/spec-parser#154 (comment) - in that way, anything introduced in 3.0.x will always have 3.0.x IRI, and if there's a new thing in 3.0.y, that new thing will have 3.0.y IRI (and the same model can have 3.0.x, 3.0.y, 3.0.z IRIs mixed -- and all these can be generated, because the version information is no longer hard-coded inside the generator but instead declared in the model files).

If that's the case, I think we are good to increment patch version every time we have a new/updated translation.