Merge branch 'working_draft' into FOCUS-551

specification/Makefile

@@ -25,7 +25,13 @@ else
 endif
  ./validate_includes.py columns
  ./validate_includes.py attributes
+ ./validate_includes.py metadata
+ ./validate_includes.py metadata/data_generator
+ ./validate_includes.py metadata/schema
+ ./validate_includes.py metadata/schema/column_definition
  ./validate_includes.py appendix
+ ./validate_includes.py appendix/examples
+ ./validate_includes.py appendix/examples/metadata_examples
  PYTHONPATH=../vendored/ $(MARKDOWNPP) spec.mdpp -o $@
  $(PYMARKDOWNLNT) --config markdownlnt.cfg scan $(SPEC_SOURCE_MDFILES)
  $(PYMARKDOWNLNT) --config markdownlnt.cfg scan $@

specification/appendix/commitment_discounts.md

@@ -14,16 +14,16 @@ While customers are bound to the *term* of a *commitment discounts*, providers o
 * *No Upfront* - The *commitment discounts* is paid on a repeated basis, typically over each [*billing period*](glossary:billing-period) of the *term*.
 * *Partial Upfront* - Some of the *commitment discounts* is paid before the *term* begins, and the rest is paid repeatedly over the *term*.
 
-For example, if a customer buys a 1-year, spend-based *commitment discount* with a $1.00 hourly commit and pays with the partial option, the *commitment discount's* payment consists of a one-time purchase in the beginning of the *term* *and* monthly recurring purchases with the following totals:
+For example, if a customer buys a 1-year, spend-based *commitment discount* with a $1.00 hourly commit and pays with the partial option, the *commitment discount's* payment consists of a one-time purchase in the beginning of the *term* *and* monthly recurring purchases with the following totals:
 
-1. *One-Time* - $4,380 (`24 hours * 365 days * $1.00 * 0.5`)
-2. *Recurring* - $182.50 (`24 hours * 365 days * $1.00 / 12 months`)
+1. *One-Time* - $4,380 (`24 hours * 365 days * $1.00 * 0.5`)
+2. *Recurring* - $182.50 (`24 hours * 365 days * $1.00 / 12 months`)
 
 ## Usage
 
 Commitment discounts follow a "use-it-or-lose-it" model where the [*amortization*](glossary:amortization) of a *commitment discount's* purchase applies evenly to eligible *resources* over each [*charge period*](glossary:charge-period) of the *term*.
 
-For example, if a customer buys a spend-based *commitment discount* with a $1.00 hourly commit in January (31 days), only $1.00 is eligible for consumption for each hourly *charge period*. If a customer has eligible *resources* running during this *charge period*, an amount of up to $1.00 will be allocated to these *resources*. Conversely, if a customer does have eligible *resources* running that fully take advantage of this $1.00 during this *charge period*, then some or all of this amount will go to waste.
+For example, if a customer buys a spend-based *commitment discount* with a $1.00 hourly commit in January (31 days), only $1.00 is eligible for consumption for each hourly *charge period*. If a customer has eligible *resources* running during this *charge period*, an amount of up to $1.00 will be allocated to these *resources*. Conversely, if a customer does have eligible *resources* running that fully take advantage of this $1.00 during this *charge period*, then some or all of this amount will go to waste.
 
 ## Commitment Discounts in FOCUS
 
@@ -35,11 +35,11 @@ All *commitment discount* purchases appear with a positive `BilledCost`, `Pricin
 
 Recurring purchases are allocated across all corresponding *charge periods* of the *term* when `ChargeCategory` is "Purchase", `ChargeFrequency` is "Recurring", and `CommitmentDiscountQuantity` and `CommitmentDiscountUnit` are reflected only for that *charge period*.
 
-Using the same *commitment discount* example as above with a one-year, spend-based *commitment discount* with a $1.00 hourly commit purchased on Jan 1, 2023, various purchase options are available:
+Using the same *commitment discount* example as above with a one-year, spend-based *commitment discount* with a $1.00 hourly commit purchased on Jan 1, 2023, various purchase options are available:
 
 #### Scenario #1: All Upfront
 
-The entire *commitment discount* is billed _once_ during the first *charge period* of the *term* for $8,670 (derived as `24 hours * 365 days * $1.00`).
+The entire *commitment discount* is billed _once_ during the first *charge period* of the *term* for $8,670 (derived as `24 hours * 365 days * $1.00`).
 
 ```json
 [
@@ -63,7 +63,7 @@ The entire *commitment discount* is billed _once_ during the first *charge perio
 
 #### Scenario #2: No Upfront
 
-The *commitment discount* is billed across all 8,760 (`24 hours * 365 days`) *charge periods* of the *term* with $1.00 allocated to each *charge period* over the *term*.
+The *commitment discount* is billed across all 8,760 (`24 hours * 365 days`) *charge periods* of the *term* with $1.00 allocated to each *charge period* over the *term*.
 
 ```json
 [
@@ -89,7 +89,7 @@ The *commitment discount* is billed across all 8,760 (`24 hours * 365 days`) *ch
 
 #### Scenario #3: Partial Upfront
 
-With a 50/50 split, half of the commitment is billed _once_ during the first *charge period* of the *term* for $4,380 (derived as `24 hours * 182.5 days * $1.00`), and the other half is billed across each *charge period* over the term, derived as (`$1.00 * 8,760 hours * 0.5`). Amortized costs incur half of the amount (i.e. $0.50) from the one-time purchase and the other half from the recurring purchase.
+With a 50/50 split, half of the commitment is billed _once_ during the first *charge period* of the *term* for $4,380 (derived as `24 hours * 182.5 days * $1.00`), and the other half is billed across each *charge period* over the term, derived as (`$1.00 * 8,760 hours * 0.5`). Amortized costs incur half of the amount (i.e. $0.50) from the one-time purchase and the other half from the recurring purchase.
 
 ```json
 [
@@ -130,16 +130,16 @@ With a 50/50 split, half of the commitment is billed _once_ during the first *ch
 
 ### Usage *Rows*
 
-*Amortization* of *commitment discounts* occur similarly regardless of how *commitment discount* purchases are made. The same usage-based or spend-based amount is applied evenly across all *charge periods* and potentially allocated to eligible *resources*. Continuing with the same *commitment discount* example, a one-year, spend-based *commitment discount* with a $1.00 hourly commit and 1 *resource* (for simplicity) yields 4 types of scenarios that can occur during a *charge period*:
+*Amortization* of *commitment discounts* occur similarly regardless of how *commitment discount* purchases are made. The same usage-based or spend-based amount is applied evenly across all *charge periods* and potentially allocated to eligible *resources*. Continuing with the same *commitment discount* example, a one-year, spend-based *commitment discount* with a $1.00 hourly commit and 1 *resource* (for simplicity) yields 4 types of scenarios that can occur during a *charge period*:
 
 * Scenario #1: An eligible *resource* fully consumes the allocated amount (100% utilization)
 * Scenario #2: No eligible *resource* consumes the allocated amount (0% utilization)
 * Scenario #3: An eligible *resource* partially consumes the allocated amount (75% utilization)
-* Scenario #4: An eligible *resource* fully consumes the $1.00 hourly commit with an overage (100% utilization + overage)
+* Scenario #4: An eligible *resource* fully consumes the $1.00 hourly commit with an overage (100% utilization + overage)
 
 #### Scenario #1: An eligible *resource* fully consumes the allocated amount (100% utilization)
 
-In this scenario, one eligible *resource* runs for the full hour and consumes $1.00, so one *row* allocated to the *resource* is produced.
+In this scenario, one eligible *resource* runs for the full hour and consumes $1.00, so one *row* allocated to the *resource* is produced.
 
 ```json
 [
@@ -166,7 +166,7 @@ In this scenario, one eligible *resource* runs for the full hour and consumes $1
 
 #### Scenario #2: No eligible *resource* consumes the allocated amount (0% utilization)
 
-In this situation, the full eligible, $1.00 amount remained unutilized and results in 1 unused *row*. In this scenario, it is important to note that while `CommitmentDiscountQuantity` is not because $1 was still drawn down by the *commitment discount* even though, no *resource* was allocated, so `ConsumedQuantity` and `ConsumedUnit` are null.
+In this situation, the full eligible, $1.00 amount remained unutilized and results in 1 unused *row*. In this scenario, it is important to note that while `CommitmentDiscountQuantity` is not because $1 was still drawn down by the *commitment discount* even though, no *resource* was allocated, so `ConsumedQuantity` and `ConsumedUnit` are null.
 
 ```json
 [
@@ -193,7 +193,7 @@ In this situation, the full eligible, $1.00 amount remained unutilized and resul
 
 #### Scenario #3: An eligible *resource* partially consumes the allocated amount (75% utilization)
 
-In this scenario, one eligible *resource* runs for the full hour and consumes $0.75 of the $1.00 allocation. One *row* shows $0.75 to a *resource*, and the other *row* shows that $0.25 was unused.
+In this scenario, one eligible *resource* runs for the full hour and consumes $0.75 of the $1.00 allocation. One *row* shows $0.75 to a *resource*, and the other *row* shows that $0.25 was unused.
 
 ```json
 [
@@ -236,9 +236,9 @@ In this scenario, one eligible *resource* runs for the full hour and consumes $0
 ]
 ```
 
-#### Scenario #4: An eligible *resource* fully consumes the $1.00 hourly commit with an overage (100% utilization + overage)
+#### Scenario #4: An eligible *resource* fully consumes the $1.00 hourly commit with an overage (100% utilization + overage)
 
-In this scenario, one eligible *resource* runs for the full hour and is charged $1.50. One *row* shows that $1.00 was *amortized* from the *commitment discount*, and the other shows that $0.50 was charged as standard, on-demand spend.
+In this scenario, one eligible *resource* runs for the full hour and is charged $1.50. One *row* shows that $1.00 was *amortized* from the *commitment discount*, and the other shows that $0.50 was charged as standard, on-demand spend.
 
 ```json
 [

specification/appendix/examples/examples.mdpp

@@ -2,4 +2,4 @@
 
 *This section is non-normative.*
 
-!INCLUDE "metadata/metadata_examples.mdpp",1
+!INCLUDE "metadata_examples/metadata_examples.mdpp",1

specification/appendix/examples/metadata/adding_new_columns_example.md → specification/appendix/examples/metadata_examples/adding_new_columns_example.md

@@ -2,15 +2,13 @@
 
 ## Scenario
 
-ACME has decided add additional columns to their FOCUS data export. The new columns are x_awesome_column1, x_awesome_column2, and x_awesome_column3. The provider creates a new schema object to represent the new schema, this schema object has a unique SchemaId. The subsequent data exports that use the new schema include the new schema's id as a reference to their corresponding schema object.
+ACME has decided add additional columns to their FOCUS data export. The new columns are x_awesome_column1, x_awesome_column2, and x_awesome_column3. The provider creates a new [Schema](#schema) object to represent the new schema, this schema object has a unique [SchemaId](#schemaid). The subsequent data exports that use the new schema include the new schema's id as a reference to their corresponding schema object.
 
 ## Supplied Metadata
 
-## Location for the new schema object
+Metadata can be provided at a location such as `/FOCUS/metadata/schemas/schema-23456-abcde-23456-abcde-23456.json`.
 
-`/FOCUS/metadata/schemas/schema-23456-abcde-23456-abcde-23456.json`
-
-## Content for the new schema object
+The updated schema related metadata could look like this:
 
 ```json
  {
@@ -75,4 +73,4 @@ ACME has decided add additional columns to their FOCUS data export. The new colu
 }
 ```
 
-For an example of how ACME ensures the schema metadata reference requirement is met see: [Schema Metadata to FOCUS Data Reference](../schema_metadata_reference_example.md)
+For an example of how ACME ensures the schema metadata reference requirement is met see: [Schema Metadata to FOCUS Data Reference](#schemametadatatofocusdatareference)

specification/appendix/examples/metadata/changing_column_metadata_example.md → specification/appendix/examples/metadata_examples/changing_column_metadata_example.md

@@ -1,16 +1,14 @@
-# Changing a Column's Metadata Example
+# Changing Column Metadata
 
 ## Scenario
 
-ACME has decided to change the datatype of column x_awesome_column1 from a string to a number. ACME creates a new schema object with the modification to x_awesome_column2.
+ACME has decided to change the datatype of column x_awesome_column1 from a string to a number. ACME creates a new [Schema](#schema) object with the modification to x_awesome_column2.
 
 ## Supplied Metadata
 
-## Location for the new schema object
+Metadata can be provided at a location such as `/FOCUS/metadata/schemas/schema-67891-abcde-67891-abcde-67891.json`.
 
-`/FOCUS/metadata/schemas/schema-67891-abcde-67891-abcde-67891.json`
-
-## Content for the new schema object
+The updated schema related metadata could look like this:
 
 ```json
  {
@@ -69,4 +67,4 @@ ACME has decided to change the datatype of column x_awesome_column1 from a strin
 }
 ```
 
-For an example of how ACME ensures the schema metadata reference requirement is met see: [Schema Metadata to FOCUS Data Reference](schema_metadata_reference_example.md)
+For an example of how ACME ensures the schema metadata reference requirement is met see: [Schema Metadata to FOCUS Data Reference](#schemametadatatofocusdatareference)

specification/appendix/examples/metadata/correcting_schema_errors_example.md → specification/appendix/examples/metadata_examples/correcting_schema_errors_example.md

@@ -1,16 +1,14 @@
-# Provider has an error in their schema metadata
+# Provider Metadata Error Correction
 
 ## Scenario
 
-ACME has discovered that while their export includes the column x_awesome_column3, the schema metadata does not include this column. In this case, the provider fixes the metadata in existing the schema object and does not need to create a new schema object.  Reference metadata remains the same.
+ACME has discovered that while their export includes the column x_awesome_column3, the [Schema](#schema) metadata does not include this column. In this case, the provider fixes the metadata in the existing schema object and does not need to create a new schema object. Reference metadata remains the same.
 
 ## Supplied Metadata
 
-## Location of the schema object
+Metadata can be provided at a location such as `/FOCUS/metadata/schemas/schema-34567-abcde-34567-abcde-34567.json`.
 
-`/FOCUS/metadata/schemas/schema-34567-abcde-34567-abcde-34567.json`
-
-## Content of the schema object
+The updated schema related metadata could look like this:
 
 ```json
  {
@@ -70,3 +68,4 @@ ACME has discovered that while their export includes the column x_awesome_column
  ]
 }
 ```
+

specification/appendix/examples/metadata_examples/data_generator_example.md

@@ -0,0 +1,18 @@
+# Data Generator Metadata
+
+## Scenario
+
+Acme provides metadata about the data generator as a part of their FOCUS data export. They provide the relevant data via the [Data Generator](#datagenerator) schema object.
+
+## Supplied Metadata
+
+Metadata can be provided at a location such as `/FOCUS/metadata/data_generator.json`.
+
+The updated data generator related metadata could look like this:
+
+```json
+{
+ "DataGenerator": "Acme"
+}
+```
+

specification/appendix/examples/metadata/focus_version_changed_example.md → specification/appendix/examples/metadata_examples/focus_version_changed_example.md

@@ -1,16 +1,14 @@
-# Provider has an error in their schema metadata
+# FOCUS Version Changed
 
 ## Scenario
 
-ACME's previous exports used Focus Version 1.0. They are now going to adopt Focus Version 1.1. It is required that they create a new schema metadata object when using a new FOCUS version regardless of schema changes. In this example, the FOCUS new version adoption doesn't include columns changes. This is to illustrate that FOCUS Version changes are independent of column changes, however, this scenario is unlikely.
+ACME's previous exports used FOCUS version 1.0. They are now going to adopt FOCUS version 1.1. It is required that they create a new schema metadata object which specifies the new FOCUS version via the [FOCUS Version](#focusversion) property - regardless of schema changes. In this example, the new FOCUS version adoption doesn't include columns changes. This is to illustrate that FOCUS version changes are independent of column changes, however, this scenario is unlikely.
 
 ## Supplied Metadata
 
-## Location of the new schema object
+Metadata can be provided at a location such as `/FOCUS/metadata/schemas/schema-45678-abcde-45678-abcde-45678.json`.
 
-`/FOCUS/metadata/schemas/schema-45678-abcde-45678-abcde-45678.json`
-
-## Content of the schema object
+The updated schema related metadata could look like this:
 
 ```json
  {
@@ -69,4 +67,4 @@ ACME's previous exports used Focus Version 1.0. They are now going to adopt Focu
 }
 ```
 
-For an example of how ACME ensures the schema metadata reference requirement is met see: [Schema Metadata to FOCUS Data Reference](schema_metadata_reference_example.md)
+For an example of how ACME ensures the schema metadata reference requirement is met see: [Schema Metadata to FOCUS Data Reference](#schemametadatatofocusdatareference)