Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Breaking Change: Standardize casing for property names and enumerated values in JSON #642

Closed
michaeltlombardi opened this issue Feb 13, 2025 · 7 comments · Fixed by #645 or #648
Closed
Assignees
Labels
Issue-Enhancement The issue is a feature or idea Needs Triage
Milestone

Comments

@michaeltlombardi
Copy link
Collaborator

Summary of the new feature / enhancement

As a user of DSC, resource author, and contributing or integrating developer, I want to rely on consistent casing for JSON that DSC emits and expects, instead of having to double-check properties and values or accidentally run into unexpected validation or (de)serialization errors.

Currently, the general practice is to use camelCase for JSON property names and enumerated values. However, not all of the structs follow this convention, and it's not predictable which values or properties use a different casing.

Because JSON Schemas are case sensitive, this constitutes a breaking change best made sooner than later.

Proposed technical implementation details (optional)

  1. Ensure that the serde attribute for rust structs and enums correctly cases the property names and values.
  2. Update the JSON schemas hosted in the repository to match.
  3. Ensure all future structs and enums that derive the Serialize and/or Deserialize use camel casing.
@SteveL-MSFT
Copy link
Member

@michaeltlombardi I manually reviewed the structs that are (de)serialized. The only cases I saw were in Progress which is already fixed by #644 and tracing supporting line_number which is what the tracing crate uses, but created PR to allow for lineNumber alias. Is there some other instance(s) you're aware of that need to be fixed?

@michaeltlombardi
Copy link
Collaborator Author

I found the following definitions other than progress—I looked primarily for structs and enums that had both serialize/deserialize or the JsonSchema trait:

  • #[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema)]
    pub enum ContextKind {
    Configuration,
    Resource,
    }
  • #[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema)]
    pub enum SecurityContextKind {
    Current,
    Elevated,
    Restricted,
    }
  • #[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema)]
    pub enum Operation {
    Get,
    Set,
    Test,
    Export,
    }
  • #[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema)]
    pub enum ExecutionKind {
    Actual,
    WhatIf,
    }
  • #[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema)]
    pub enum ConfigurationResourceCompletionStatus {
    Success,
    Failure,
    }
  • #[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema)]
    pub enum MessageLevel {
    Error,
    Warning,
    Information,
    }
  • #[derive(Clone, Debug, PartialEq, Deserialize, Serialize, JsonSchema)]
    pub enum Capability {
    /// The resource supports retrieving configuration.
    Get,
    /// The resource supports applying configuration.
    Set,
    /// The resource supports the `_exist` property directly.
    SetHandlesExist,
    /// The resource supports simulating configuration directly.
    WhatIf,
    /// The resource supports validating configuration.
    Test,
    /// The resource supports deleting configuration.
    Delete,
    /// The resource supports exporting configuration.
    Export,
    /// The resource supports resolving imported configuration.
    Resolve,
    }
  • #[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema)]
    pub enum Kind {
    Adapter,
    Group,
    Importer,
    Resource,
    }

I also noticed the progress struct, which only serializes - not sure if we should also have it with a JSON Schema if it's meant to be consumed by other tools, and thus also camelCased:

DSC/dsc_lib/src/util.rs

Lines 42 to 50 in 3963389

#[derive(Default, Debug, Clone, Serialize)]
pub struct Progress {
pub activity: String,
pub percent_complete: u16,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub status: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub seconds_remaining: Option<u64>,
}

@SteveL-MSFT
Copy link
Member

I think PascalCasing makes sense for Enums which are used as property values and not property names. Progress currently is only serialized, when we get to enabling resources to emit progress then deserialization will be important.

michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 18, 2025
Prior to this change, the structs and enums in `dsc_lib` didn't consistently
use camelCase - most property names and enum values use camelCase, but not
all - and this inconsistency isn't predictable for end users, who need to
consult the JSON Schemas to be sure.

This change updates the definitions to rename the fields and values when
serializing and deserializing, which also updates their JSON Schema.

A future change is required to update the canonical schemas in the
repository to match these updates.
@michaeltlombardi
Copy link
Collaborator Author

Pascal casing for enums is currently inconsistent, and some of them can't be changed to PascalCasing without breaking ARM compatibility, like DataType:

#[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema)]
pub enum DataType {
#[serde(rename = "string")]
String,
#[serde(rename = "secureString")]
SecureString,
#[serde(rename = "int")]
Int,
#[serde(rename = "bool")]
Bool,
#[serde(rename = "object")]
Object,
#[serde(rename = "secureObject")]
SecureObject,
#[serde(rename = "array")]
Array,
}

Other enums that currently use camelCasing:

  • #[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema)]
    pub enum InputKind {
    /// The input is accepted as environmental variables.
    #[serde(rename = "env")]
    Env,
    /// The input is accepted as a JSON object via STDIN.
    #[serde(rename = "stdin")]
    Stdin,
    }
  • #[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema)]
    pub enum ReturnKind {
    /// The return JSON is the state of the resource.
    #[serde(rename = "state")]
    State,
    /// The return JSON is the state of the resource and the diff.
    #[serde(rename = "stateAndDiff")]
    StateAndDiff,
    }
  • #[derive(Debug, Clone, PartialEq, Deserialize, Serialize, JsonSchema)]
    pub enum ConfigKind {
    /// The adapter accepts full unprocessed configuration.
    #[serde(rename = "full")]
    Full,
    /// The adapter accepts configuration as a sequence.
    #[serde(rename = "sequence")]
    Sequence,
    }

@SteveL-MSFT
Copy link
Member

ARM compat is a priority, so I guess that means we should make everything camelCased to be consistent. I'll update my PR for this.

@michaeltlombardi
Copy link
Collaborator Author

Relatedly, in my ideal world, which I am not recommending we do in the short-to-mid-term, or without further user feedback:

  1. DSC property names and enum values are case-insensitive
  2. Provide a way for resources to indicate that their properties/values are case-insensitive
  3. DSC is able to inform users about casing issues / canonicalize casing in the dsc config resolve command

Because DSC knows the supplied data and JSON Schema for not only its own data types but also for resource instances, I think we can (relatively cheaply, maintainably) implement a check for case-sensitivity values. The jsonschema crate supports custom keywords we could use for handling this, too.

@SteveL-MSFT
Copy link
Member

I think making property names and enum values case-insensitive makes sense since we don't ever want a situation where we have different properties/enums that differ by casing. It looks like serde has support for this so I'll play with it.

michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 18, 2025
Prior to this change, the structs and enums in `dsc_lib` didn't consistently
use camelCase - most property names and enum values use camelCase, but not
all - and this inconsistency isn't predictable for end users, who need to
consult the JSON Schemas to be sure.

This change updates the definitions to rename the fields and values when
serializing and deserializing, which also updates their JSON Schema.

A future change is required to update the canonical schemas in the
repository to match these updates.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 22, 2025
Prior to this change, the structs and enums in `dsc_lib` didn't consistently
use camelCase - most property names and enum values use camelCase, but not
all - and this inconsistency isn't predictable for end users, who need to
consult the JSON Schemas to be sure.

This change updates the definitions to rename the fields and values when
serializing and deserializing, which also updates their JSON Schema.

A future change is required to update the canonical schemas in the
repository to match these updates.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 22, 2025
This change updates the YAML source definition files for
the JSON Schemas to use the correct casing. It also adds
the missing `context` property for the `Microsoft.DSC`
metadata field.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 22, 2025
This change generates the canonical schemas for in new folders: `v3`,
`v3.0`, and `v3.0.0`, instead of the previous date-based folder
`2024/04`. The changing of the casing is _not_ backwards compatible, and
requires a new location to avoid breaking prior schemas.

This change generates the schemas into three folders to accomodate the
changes requested in PowerShell#138 for moving from date-based to semantic
versioning for the schemas.

Going forward, schema generation should:

- Export to a folder for the current release with the full semantic
  version, like `v3.0.1` or `v3.1.0`.
- Export to a folder for the current `major.minor` schema, like `v3.0` or
  `v3.1`
- Build to a folder for the current major version of the release, like `v3`.

Under this design, you can:

- Get the exact version of the schema that a release shipped with by
  specifying the schema URI as `v<major>.<minor>.<patch>`
- Get the latest schema for your minor version as `v<major>.<minor>`
- Get the latest schema for your major version as `v<major>`

Breaking changes to the schema have the potential to break
integrating software, configuration documents, and resource manifests.
They therefore constitute a breaking change in the application and
require incrementing the major version segment of the semantic version.

This change will be followed by an update to the allowed schema versions
in the code itself, which should only recognize the updated schema at
this time, not previous schemas.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 22, 2025
This commit only touches the `dsc` and `dsc_lib` crates. A future
commit will update the examples, manifests, and tests throughout
the repository for the updated schema URIs. Tests against this
commit specifically will not pass, but the changes to the library
are broken out here for easier review.

This change defines a new module in the library for interacting
with and managing JSON Schemas for DSC. The code in this module
is made public for use in other repository crates, but shouldn't
be used by crates outside of this repository.

It defines new enumerations:

- `SchemaUriPrefix` for handling URIs pointing to either the GitHub repository
  or friendly `aka.ms` locations.
- `SchemaForm` for the different forms the DSC schemas are published to (canonical,
  bundled, and enhanced for VS Code)
- `RecognizedSchemaVersion` for managing the version folders schemas
  are publsihed to for and after GA.

It defines a few helper functions for constructing the various URIs for a
schema. These helper functions may be useful for testing, but should
primarily be used through the new `DscRepoSchema` trait, which requires
a type to define the base name and folder path for its schema and whether
the schema should be published in its bundled form (only applies to top-level
schemas, not subschemas).

The trait provides default implementations for retrieving the schema URIs and
a function to return a generated JSON Schema for the `$schema` keyword, used
by both configuration documents and resource manifests.

As we canonicalize and improve the schemas, this trait should make it simpler
to manage the schemas across new versions and should be the container for
JSON Schema enhancements specific to the DSC schemas.

This change updates the configuration document and resource manifest definitions
to implement the `DscRepoSchema` trait and replaces the per-type enumerations
for schema version URIs by passing the `recognized_schema_uris_subschema` to
the schemars `schema_with` attribute.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 22, 2025
This change updates the schema URIs in the various resource manifests,
configuration documents, and tests to use the new schema URIs:

- `https://aka.ms/dsc/schemas/v3/bundled/config/document.json` for
  configuration documents.
- `https://aka.ms/dsc/schemas/v3/bundled/resource/manifest.json` for
  resource manifests.

This change pins the changes to the major version to reduce how often
these files need to be updated with future releases. It uses the
`aka.ms` prefix instead of the GitHub URI prefix for readability
and because we expect the majority of users to prefer the shorter
URI.

With this change in place, tests broken in the previous commit should
pass again.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 24, 2025
Prior to this change, DSC didn't have a convenient way to verify
whether a configuration or manifest used a recognized schema or
to switch behaviors based on the declared URI.

This change begins the process of plumbing the recognized schemas
into DSC so the engine can correctly validate and handle items
based on the value of the `$schema` keyword. In this first
iteration, the `DscRepoSchema` trait adds a method to validate
whether the item uses a recognized schema. In the future, the
engine should provide further handling than pass/fail.

This change only adds the methods, it doesn't alter how DSC
processes configuration documents or resource manifests.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 25, 2025
Prior to this change, the structs and enums in `dsc_lib` didn't consistently
use camelCase - most property names and enum values use camelCase, but not
all - and this inconsistency isn't predictable for end users, who need to
consult the JSON Schemas to be sure.

This change updates the definitions to rename the fields and values when
serializing and deserializing, which also updates their JSON Schema.

A future change is required to update the canonical schemas in the
repository to match these updates.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 25, 2025
This change updates the YAML source definition files for
the JSON Schemas to use the correct casing. It also adds
the missing `context` property for the `Microsoft.DSC`
metadata field.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 25, 2025
This change generates the canonical schemas for in new folders: `v3`,
`v3.0`, and `v3.0.0`, instead of the previous date-based folder
`2024/04`. The changing of the casing is _not_ backwards compatible, and
requires a new location to avoid breaking prior schemas.

This change generates the schemas into three folders to accomodate the
changes requested in PowerShell#138 for moving from date-based to semantic
versioning for the schemas.

Going forward, schema generation should:

- Export to a folder for the current release with the full semantic
  version, like `v3.0.1` or `v3.1.0`.
- Export to a folder for the current `major.minor` schema, like `v3.0` or
  `v3.1`
- Build to a folder for the current major version of the release, like `v3`.

Under this design, you can:

- Get the exact version of the schema that a release shipped with by
  specifying the schema URI as `v<major>.<minor>.<patch>`
- Get the latest schema for your minor version as `v<major>.<minor>`
- Get the latest schema for your major version as `v<major>`

Breaking changes to the schema have the potential to break
integrating software, configuration documents, and resource manifests.
They therefore constitute a breaking change in the application and
require incrementing the major version segment of the semantic version.

This change will be followed by an update to the allowed schema versions
in the code itself, which should only recognize the updated schema at
this time, not previous schemas.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 25, 2025
This commit only touches the `dsc` and `dsc_lib` crates. A future
commit will update the examples, manifests, and tests throughout
the repository for the updated schema URIs. Tests against this
commit specifically will not pass, but the changes to the library
are broken out here for easier review.

This change defines a new module in the library for interacting
with and managing JSON Schemas for DSC. The code in this module
is made public for use in other repository crates, but shouldn't
be used by crates outside of this repository.

It defines new enumerations:

- `SchemaUriPrefix` for handling URIs pointing to either the GitHub repository
  or friendly `aka.ms` locations.
- `SchemaForm` for the different forms the DSC schemas are published to (canonical,
  bundled, and enhanced for VS Code)
- `RecognizedSchemaVersion` for managing the version folders schemas
  are publsihed to for and after GA.

It defines a few helper functions for constructing the various URIs for a
schema. These helper functions may be useful for testing, but should
primarily be used through the new `DscRepoSchema` trait, which requires
a type to define the base name and folder path for its schema and whether
the schema should be published in its bundled form (only applies to top-level
schemas, not subschemas).

The trait provides default implementations for retrieving the schema URIs and
a function to return a generated JSON Schema for the `$schema` keyword, used
by both configuration documents and resource manifests.

As we canonicalize and improve the schemas, this trait should make it simpler
to manage the schemas across new versions and should be the container for
JSON Schema enhancements specific to the DSC schemas.

This change updates the configuration document and resource manifest definitions
to implement the `DscRepoSchema` trait and replaces the per-type enumerations
for schema version URIs by passing the `recognized_schema_uris_subschema` to
the schemars `schema_with` attribute.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 25, 2025
This change updates the schema URIs in the various resource manifests,
configuration documents, and tests to use the new schema URIs:

- `https://aka.ms/dsc/schemas/v3/bundled/config/document.json` for
  configuration documents.
- `https://aka.ms/dsc/schemas/v3/bundled/resource/manifest.json` for
  resource manifests.

This change pins the changes to the major version to reduce how often
these files need to be updated with future releases. It uses the
`aka.ms` prefix instead of the GitHub URI prefix for readability
and because we expect the majority of users to prefer the shorter
URI.

With this change in place, tests broken in the previous commit should
pass again.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 25, 2025
Prior to this change, DSC didn't have a convenient way to verify
whether a configuration or manifest used a recognized schema or
to switch behaviors based on the declared URI.

This change begins the process of plumbing the recognized schemas
into DSC so the engine can correctly validate and handle items
based on the value of the `$schema` keyword. In this first
iteration, the `DscRepoSchema` trait adds a method to validate
whether the item uses a recognized schema. In the future, the
engine should provide further handling than pass/fail.

This change only adds the methods, it doesn't alter how DSC
processes configuration documents or resource manifests.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 25, 2025
…/config ref

This change drafts a new reference document that explains how the DSC
schemas are published, the URIs DSC recognizes for those schemas, and
updates the documentation for the resource manifest and configuration
document schemas with the updated context.

A future change will address updating the rest of the reference
documentation.
michaeltlombardi added a commit to michaeltlombardi/DSC that referenced this issue Feb 25, 2025
This change updates the reference documentation to align
with the new schema URIs and the changes to the enumeration
and struct casing.
github-merge-queue bot pushed a commit that referenced this issue Feb 25, 2025
…ng-canonicalization

(GH-642) Ensure camelCase for items in `dsc_lib`
SteveL-MSFT pushed a commit to SteveL-MSFT/DSC that referenced this issue Feb 26, 2025
Prior to this change, the structs and enums in `dsc_lib` didn't consistently
use camelCase - most property names and enum values use camelCase, but not
all - and this inconsistency isn't predictable for end users, who need to
consult the JSON Schemas to be sure.

This change updates the definitions to rename the fields and values when
serializing and deserializing, which also updates their JSON Schema.

A future change is required to update the canonical schemas in the
repository to match these updates.
SteveL-MSFT pushed a commit to SteveL-MSFT/DSC that referenced this issue Feb 26, 2025
This change updates the YAML source definition files for
the JSON Schemas to use the correct casing. It also adds
the missing `context` property for the `Microsoft.DSC`
metadata field.
SteveL-MSFT pushed a commit to SteveL-MSFT/DSC that referenced this issue Feb 26, 2025
This change generates the canonical schemas for in new folders: `v3`,
`v3.0`, and `v3.0.0`, instead of the previous date-based folder
`2024/04`. The changing of the casing is _not_ backwards compatible, and
requires a new location to avoid breaking prior schemas.

This change generates the schemas into three folders to accomodate the
changes requested in PowerShell#138 for moving from date-based to semantic
versioning for the schemas.

Going forward, schema generation should:

- Export to a folder for the current release with the full semantic
  version, like `v3.0.1` or `v3.1.0`.
- Export to a folder for the current `major.minor` schema, like `v3.0` or
  `v3.1`
- Build to a folder for the current major version of the release, like `v3`.

Under this design, you can:

- Get the exact version of the schema that a release shipped with by
  specifying the schema URI as `v<major>.<minor>.<patch>`
- Get the latest schema for your minor version as `v<major>.<minor>`
- Get the latest schema for your major version as `v<major>`

Breaking changes to the schema have the potential to break
integrating software, configuration documents, and resource manifests.
They therefore constitute a breaking change in the application and
require incrementing the major version segment of the semantic version.

This change will be followed by an update to the allowed schema versions
in the code itself, which should only recognize the updated schema at
this time, not previous schemas.
SteveL-MSFT pushed a commit to SteveL-MSFT/DSC that referenced this issue Feb 26, 2025
This commit only touches the `dsc` and `dsc_lib` crates. A future
commit will update the examples, manifests, and tests throughout
the repository for the updated schema URIs. Tests against this
commit specifically will not pass, but the changes to the library
are broken out here for easier review.

This change defines a new module in the library for interacting
with and managing JSON Schemas for DSC. The code in this module
is made public for use in other repository crates, but shouldn't
be used by crates outside of this repository.

It defines new enumerations:

- `SchemaUriPrefix` for handling URIs pointing to either the GitHub repository
  or friendly `aka.ms` locations.
- `SchemaForm` for the different forms the DSC schemas are published to (canonical,
  bundled, and enhanced for VS Code)
- `RecognizedSchemaVersion` for managing the version folders schemas
  are publsihed to for and after GA.

It defines a few helper functions for constructing the various URIs for a
schema. These helper functions may be useful for testing, but should
primarily be used through the new `DscRepoSchema` trait, which requires
a type to define the base name and folder path for its schema and whether
the schema should be published in its bundled form (only applies to top-level
schemas, not subschemas).

The trait provides default implementations for retrieving the schema URIs and
a function to return a generated JSON Schema for the `$schema` keyword, used
by both configuration documents and resource manifests.

As we canonicalize and improve the schemas, this trait should make it simpler
to manage the schemas across new versions and should be the container for
JSON Schema enhancements specific to the DSC schemas.

This change updates the configuration document and resource manifest definitions
to implement the `DscRepoSchema` trait and replaces the per-type enumerations
for schema version URIs by passing the `recognized_schema_uris_subschema` to
the schemars `schema_with` attribute.
SteveL-MSFT pushed a commit to SteveL-MSFT/DSC that referenced this issue Feb 26, 2025
This change updates the schema URIs in the various resource manifests,
configuration documents, and tests to use the new schema URIs:

- `https://aka.ms/dsc/schemas/v3/bundled/config/document.json` for
  configuration documents.
- `https://aka.ms/dsc/schemas/v3/bundled/resource/manifest.json` for
  resource manifests.

This change pins the changes to the major version to reduce how often
these files need to be updated with future releases. It uses the
`aka.ms` prefix instead of the GitHub URI prefix for readability
and because we expect the majority of users to prefer the shorter
URI.

With this change in place, tests broken in the previous commit should
pass again.
SteveL-MSFT pushed a commit to SteveL-MSFT/DSC that referenced this issue Feb 26, 2025
Prior to this change, DSC didn't have a convenient way to verify
whether a configuration or manifest used a recognized schema or
to switch behaviors based on the declared URI.

This change begins the process of plumbing the recognized schemas
into DSC so the engine can correctly validate and handle items
based on the value of the `$schema` keyword. In this first
iteration, the `DscRepoSchema` trait adds a method to validate
whether the item uses a recognized schema. In the future, the
engine should provide further handling than pass/fail.

This change only adds the methods, it doesn't alter how DSC
processes configuration documents or resource manifests.
SteveL-MSFT pushed a commit to SteveL-MSFT/DSC that referenced this issue Feb 26, 2025
…/config ref

This change drafts a new reference document that explains how the DSC
schemas are published, the URIs DSC recognizes for those schemas, and
updates the documentation for the resource manifest and configuration
document schemas with the updated context.

A future change will address updating the rest of the reference
documentation.
SteveL-MSFT pushed a commit to SteveL-MSFT/DSC that referenced this issue Feb 26, 2025
This change updates the reference documentation to align
with the new schema URIs and the changes to the enumeration
and struct casing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue-Enhancement The issue is a feature or idea Needs Triage
Projects
None yet
2 participants