Schema development: main or branch, in parallel or at release time?

[handrews]

I'm splitting out the schema-related concerns from #3677 as we decided in a recent TDC meeting that the schema, spec, and gh-pages parts have different priorities (and probably involve different people).

• New schemas are created for each minor or major release
  • Patch releases are limited to changes that would not require a separate schema for each patch release in a minor release line
  • Should new minor/major schemas be created and updated in parallel with spec changes, or deferred and created all at once just prior to the release, as part of the final review effort?
  • If created and updated in parallel, this should be integrated with the development branch process being sorted out in Define and document branch strategy for the spec, both development and publishing #3677, presumably continuing to be published to main in the current schemas/, just as Define and document branch strategy for the spec, both development and publishing #3677 recommends publishing the markdown to the current versions/ location
• Schemas can currently also be changed for bug-fixing or functional improvement at any time, and released independently from the release schedule (see also Define and document our schema publishing process #3715)
  • This includes but is not limited to patch release-related bug fixes
  • This might require a different tagging strategy from whatever we do for spec releases

[handrews]

So... given the proposals for branching in PR #4154 (background in issue #3677), I can see a few options for schemas:

share the X.Y-dev branches, but release independently

This would consolidate the schemas, based on the 3.1.1 files, under src/ (alongside the spec which is src/oas.md) on the dev branch, and we'd do the release-line-specific schema work on the appropriate X.Y-dev branch.

Since schema releases are independent of patch releases, instead of making X.Y.Z-rel branches to rename to the published file location, we'd make branches like 2024-10-22-rel.

PROS:

• Same process and working branches as the spec
• We can merge changes across release lines
• This allows simultaneous spec and schema changes (which is a surprisingly controversial proposal, but I'm putting it under "PRO" based on personal preference)

CONS:

• The release branches are a bit unwieldy, and schemas might be published significantly more often (it's a bit hard to say, and also depends on how define the scope of schemas — see Consider allowing contributions to "linting" schemas #4141 and Improve 3.1+ schema $id UX #4147)

MITIGATION:

• We could probably automate more of the publishing process

Keep the schemas/ layout but do the work on dev

PROS:

• No renaming for release needed
• We can still batch up changes on dev and only merge them to main when we want them published

CONS:

• Schema dev is still divorced from the spec changes that drive it (which I consider a con)
• No merges, we'd have to do git --format-patch | sed | git am -3 hackery still
• Different process to remember compared to spec work

MITIGATION:

• Schemas changes usually aren't so complicated, so manually psuedo-merging isn't quite as horrible

Hybrid 1: Keep /schemas but work on X.Y-dev

In this approach, we don't rename any schema files into the src/ directory.

PROS:

• Spec and schema changes can be in the same PR
• The fundamental branching pattern is the same (just not the file management pattern)

CONS:

• The file management pattern is different
• Still require psuedo-merge messiness

MITIGATIONS:

• Pseudo-merges of schemas aren't as horrible

Hybrid 2: work on X.Y-dev under src/ and release directly from there

This approach more-or-less abandons schemas on dev or main, and just triggers publishing to the spec site directly from the X.Y-dev branch.

PROS:

• Same structure/process as the spec
• No release branches needed
• Changes can merge across release lines
• No weird-looking semi-published schemas on main

CONS:

• Schemas harder to find? Do we leave the old schemas on main?
• No ability to batch up changes if we automatically trigger publishing from X.Y-dev

[lornajane]

We reached a consensus that Hybrid 2 would work well for most people. We will need to take care to provide good development experience to be able to quickly build and try things.

@handrews has the action point to implement this.

[handrews]

@OAI/tsc so I forgot the rename the schemas/v3.1/README.md file to the dev branches. Should I do that, or should we do all of the documentation through the CONTRIBUTING.md file? I'm inclined to start with just that one and only break out per-directory READMEs if there's a clear need. I know it took me a long time to even realize the schema-specific READMEs were in a directory there.

I feel like "there's no README in this directory, guess I'll look at CONTRIBUTING" is a better fallback than "this isn't covered in CONTRIBUTING maybe I should look for a README somewhere else"

[ralfhandl]

If we state in CONTRIBUTING.md that PRs should only change files in the src folder of vX.Y-dev branches, that would be sufficient.

Which reminds me: should the pass and fail folders of the schema tests also be copied into the src/schemas/validation folder, and as part of the release be copied back to the tests/vX.Y folder?

[handrews]

@ralfhandl I'd probably put them beside the directory of schema sources rather than in it? Particularly because the pubilshed tests location tests/vX.Y is beside the published schemas location?

Although IIRC we have not decided whether we want to merge schemas back to main at all yet, as there was some support for the Hybrid-2 option.

[ralfhandl]

I'd probably put them beside the directory of schema sources rather than in it

That would be src/schema-tests/validation/pass, in case we add other schemas next to the validation ones?

This does have the minuscule advantage of being within src over the current location of tests/vX.Y/pass, and the same disadvantage of being a parallel folder tree.