[DISCUSSION]: Formally document the backwards compatibility process for FOCUS

[AWS-ZachErdman]

Description

FOCUS has been receiving some backwards incompatible changes in 1.1. We need to:

1. Document our definition of backwards incompatibility
2. Document our process for validating and merging backwards incompatible changes to a future version

Proposed Approach

This documentation should be in the FOCUS repo for future reference, but need not be in the specification document.

GitHub Issue or Reference

No response

Context

No response

Data Submission for Discussion

No response

[jpradocueva]

Action Items from TF-2 call on Oct 3rd:

• [TF2-#578] Zach, @AWS-ZachErdman : Draft initial documentation for review next week.

[jpradocueva]

Action Items from the Mebmers' call on October 3rd:

• [Maintainers-#578] Zach, @AWS-ZachErdman to draft a process for breaking change documentation.

[ijurica]

A few notes on (potential) breaking changes introduced in FOCUS version 1.1 compared to version 1.0...

---

Assuming a breaking change is defined as any modification to an existing FOCUS column, attribute, or Metadata element that could cause previously functioning SQL queries (supporting a specific use case) to fail or behave differently, the following changes introduced in FOCUS version 1.1, compared to version 1.0, are considered breaking changes:

• ConsumedQuantity column update:

  • According to the normative requirements specified in version 1.1, ConsumedQuantity MUST be null if ChargeCategory is 'Usage', CommitmentDiscountStatus is 'Unused', and ChargeClass is not 'Correction'.
  • In contrast, version 1.0 specified that ConsumedQuantity MUST NOT be null if ChargeCategory is 'Usage' and ChargeClass is not 'Correction', regardless of the CommitmentDiscountStatus.
  • Note: An alternative to this breaking change was considered (see PR Have consumedquantity be 0 for unused commitment #598 for details). We discussed the pros and cons and decided to proceed with the current version.

• ConsumedUnit column update:

  • According to the normative requirements specified in version 1.1, ConsumedUnit MUST be null if ChargeCategory is 'Usage', CommitmentDiscountStatus is 'Unused', and ChargeClass is not 'Correction'.
  • In contrast, version 1.0 specified that ConsumedUnit MUST NOT be null if ChargeCategory is 'Usage' and ChargeClass is not 'Correction', regardless of the CommitmentDiscountStatus.
  • Note: An alternative to this breaking change was considered (see PR Have consumedquantity be 0 for unused commitment #598 for details). We discussed the pros and cons and decided to proceed with the current version.

• PricingCategory column update:

  • According to the normative requirements specified in version 1.1, PricingCategory MUST be 'Committed' when the charge is subject to an existing commitment discount and is not related to the purchase of the commitment discount.
  • In contrast, version 1.0 specified that PricingCategory MUST be 'Committed' when the charge is subject to an existing commitment discount, even in cases where the charge pertains to the purchase of the commitment discount.

• CommitmentDiscountStatus / CommitmentDiscountCategory column update:

  • According to the normative requirements specified in version 1.1, CommitmentDiscountStatus MUST be one of the allowed values: 'Used' or 'Unused'. Due to an oversight in version 1.0, the same requirement for the CommitmentDiscountStatus column was incorrectly written as: "The CommitmentDiscountCategory MUST be one of the allowed values: 'Used' or 'Unused'."
  • This oversight might also be classified as an editorial change.

[ijurica]

Some Common Characteristics of Breaking Changes:

• Schema/Structure Changes:

  • Changes in data structure, such as renaming or removing columns or metadata elements, can break existing integrations or queries.
  • Example: Removing or renaming a column in a dataset would require consumers (practitioners, ETL systems, etc.) to adapt their queries or processing logic.

• Modifying Data Types:

  • Changing the data type of a column can cause failures if consumers were designed to handle the previous format or behavior.
  • Example: Changing a column’s data type from String to Decimal.

• Modifying Constraints That Result in a Change in Expected Behavior:

  • Any modification that alters column nullability (e.g., MUST be null vs. MUST NOT be null), the list of available values, or the expected value for a particular use case scenario can potentially cause the same queries to yield different outputs or results.
  • Example: If a query against a dataset returns different results due to changes in column nullability or available value definitions, this constitutes a breaking change.

• Etc.

Additional topics to discuss:

• Should we focus on the practitioner’s (consumer’s) perspective, or should we encompass both consumers and producers?
  • For example, introduction of stricter validation or constraints - such as a more limited value range, or changing from "MAY be null" or "MAY NOT be null" to "MUST", etc.

[shawnalpay]

Assigning this one to @AWS-ZachErdman and @ijurica, per our discussions this past week.

EDIT: per the Oct 10 Member call, adding @ahullah and @rileyjenk as assignees as well.

[jpradocueva]

Action Items during Members' call on Oct 11:

• [Members-#578] Zach, @AWS-ZachErdman , Alex, @ahullah, Riley, @rileyjenk : Create a formal document outlining the backwards compatibility process.
• [Members-#578] Zach, @AWS-ZachErdman , Alex, @ahullah, Riley, @rileyjenk : Develop a checklist of evaluation criteria to be used for assessing proposed changes.
• [Members-#578] Joaquin: Schedule follow-up meetings to review specific scenarios and refine the process.