[BWC] How should Backwards Compatiabilty tests be configured as we approach v3.0?

[peternied]

Following up from the conversation in opensearch-project/OpenSearch#3615

As we are building OpenSearch v3.0 we don't really have a backwards compatibility story with version 2.X. While I understand this story is still coming together, what should plugin teams do that use BWC tests as part of their PR checks?

In Security, we have been blocked from migrating to v3.0 because of this issue, but I feel this is slowing our velocity too much on part of the product that isn't addressed until later in the lifecycle. I am disabling these tests to keep us agile, but I feel like this is going to delay dealing with these problems until later. What do others think, how should we handle this?

[dblock]

We have a bcw story with 2.x, only the last 2.x will be directly upgradable to 3.0.

I fixed this for job-scheduler here: opensearch-project/job-scheduler#242 as part of making it work against 3.0 core.

Is there something else needed?

[peternied]

Thanks for getting that change in for job-scheduler. I'll rerun the suite and confirm we are OK to test the 2.4 -> 3.0 workflow, in which case there is no action we need to take. https://github.com/peternied/security/actions/runs/3139197099/jobs/5099360691

Assuming this passes, I'll close this issue

only the last 2.x

This is a soft concept, once the end of the 2.x line is declared having a mechanism to propagate it and lift/ensure everyone to v3.0 would be of value.