Improve IDL 1 and 2 interoperability

VERSION

@@ -1 +1 @@
-1.24.0
+1.25.0

docs/source-2.0/spec/aggregate-types.rst

@@ -168,10 +168,9 @@ Structure member optionality
 
 Whether a structure member is optional is determined by evaluating the
 :ref:`required-trait`, :ref:`default-trait`, :ref:`clientOptional-trait`,
-and :ref:`input-trait`. Authoritative model consumers like servers MAY choose
-to determine optionality using more restrictive rules by ignoring the
-``@input`` and ``@clientOptional`` traits.
-
+:ref:`input-trait`, and :ref:`addedDefault-trait`. Authoritative model
+consumers like servers MAY choose to determine optionality using more
+restrictive rules by ignoring the ``@input`` and ``@clientOptional`` traits.
 
 .. list-table::
     :header-rows: 1
@@ -262,20 +261,28 @@ need to be provided without breaking previously generated code.
 .. rubric:: Migrating ``@required`` to ``@default``
 
 If a ``required`` member no longer needs to be be required, the ``required``
-trait MAY be removed and replaced with the :ref:`default-trait`. The member
-is still considered always present to tools like code generators, but instead
-of requiring the value to be provided by an end-user, a default value is
-automatically provided if missing. For example, the previous ``TimeSpan``
-model can be backward compatibly changed to:
+trait MAY be removed and replaced with the :ref:`default-trait`. Alternatively,
+a ``default`` trait MAY be added to a member marked as ``required`` to
+provide a default value for the member but require that it is serialized.
+Either way, the member is still considered always present to tools like code
+generators, but instead of requiring the value to be provided by an end-user,
+a default value is automatically provided if missing. For example, the previous
+``TimeSpan`` model can be backward compatibly changed to:
 
 .. code-block:: smithy
 
     structure TimeSpan {
-        // @required is replaced with @default
+        // @required is replaced with @default and @addedDefault
+        @addedDefault
         years: Integer = 0
         days: Integer = 0
     }
 
+The :ref:`addeddefault-trait` trait SHOULD be used any time a ``default`` trait is
+added to a previously published member. Some tooling does not treat the
+``required`` trait as non-nullable but does treat the ``default`` trait as
+non-nullable.
+
 .. rubric:: Requiring members to be optional
 
 The :ref:`clientOptional-trait` is used to indicate that a member that is

docs/source-2.0/spec/model.rst

@@ -1288,46 +1288,25 @@ referenced from within any namespace using a relative shape ID.
     @unitType
     structure Unit {}
 
-    @deprecated(
-        message: "Use Boolean instead, and add the @default trait to structure members that targets this shape",
-        since: "2.0"
-    )
+    @default(false)
     boolean PrimitiveBoolean
 
-    @deprecated(
-        message: "Use Byte instead, and add the @default trait to structure members that targets this shape",
-        since: "2.0"
-    )
+    @default(0)
     byte PrimitiveByte
 
-    @deprecated(
-        message: "Use Short instead, and add the @default trait to structure members that targets this shape",
-        since: "2.0"
-    )
+    @default(0)
     short PrimitiveShort
 
-    @deprecated(
-        message: "Use Integer instead, and add the @default trait to structure members that targets this shape",
-        since: "2.0"
-    )
+    @default(0)
     integer PrimitiveInteger
 
-    @deprecated(
-        message: "Use Long instead, and add the @default trait to structure members that targets this shape",
-        since: "2.0"
-    )
+    @default(0)
     long PrimitiveLong
 
-    @deprecated(
-        message: "Use Float instead, and add the @default trait to structure members that targets this shape",
-        since: "2.0"
-    )
+    @default(0)
     float PrimitiveFloat
 
-    @deprecated(
-        message: "Use Double instead, and add the @default trait to structure members that targets this shape",
-        since: "2.0"
-    )
+    @default(0)
     double PrimitiveDouble
 
 

docs/source-2.0/spec/type-refinement-traits.rst

@@ -15,22 +15,20 @@ type of a shape.
 Summary
     Provides a structure member with a default value.
 Trait selector
-    ``structure > member :test(> :is(simpleType, list, map))``
+    ``:is(simpleType, list, map, structure > member :test(> :is(simpleType, list, map)))``
 
-    A member of a structure that targets a simple type, list, or map.
+    A simple type, list, map, or a member of a structure that targets a simple type, list, or map.
 Value type
     Document type.
-Conflicts with
-    * :ref:`required-trait`
 See also
     * :ref:`structure-optionality`
     * :ref:`required-trait`
     * :ref:`clientOptional-trait`
     * :ref:`input-trait`
 
-The ``@default`` trait assigns a default value to a structure member using
-a document type. The following example defines a structure with a "language"
-member that has a default value:
+The ``@default`` trait assigns a default value to a structure member. The
+following example defines a structure with a "language" member that has a
+default value:
 
 .. code-block:: smithy
 
@@ -58,6 +56,40 @@ The above example uses syntactic sugar to apply the ``@default`` trait. The
         language: Language
     }
 
+The ``@default`` trait can be added to root-level simple types, lists, or
+maps. This can serve as a kind of template to enforce default values across
+structure members in a model. Any structure member that targets a shape
+marked with ``@default`` MUST also add a matching ``@default`` trait to the
+member.
+
+.. code-block:: smithy
+
+    @default(0)
+    integer ZeroValueInteger
+
+    structure Message {
+        zeroValueInteger: ZeroValueInteger = 0 // must be repeated and match the target.
+    }
+
+The ``@default`` trait on a structure member can be set to ``null`` to
+explicitly indicate that the member is optional or to override the default
+value requirement of a targeted shape.
+
+.. code-block:: smithy
+
+    @default(0)
+    integer ZeroValueInteger
+
+    structure Message {
+        zeroValueInteger: ZeroValueInteger = null // forces the member to be optional
+    }
+
+.. note::
+
+    * The ``@default`` trait on root-level shapes has no impact when targeted by
+      any other shape than a structure member.
+    * The ``@default`` trait on root-level shapes cannot be set to ``null``.
+
 
 Default value constraints
 -------------------------
@@ -94,6 +126,12 @@ property was omitted.
 Updating default values
 -----------------------
 
+The default value of a root-level shape MUST NOT be changed nor can the default
+trait be added or removed from an existing root-level shape. Changing the
+default value of a root-level shape would cause any member reference to the
+shape to break and could inadvertently impact code generated types for the
+shape.
+
 The default value of a member SHOULD NOT be changed. However, it MAY be
 necessary in extreme cases to change a default value if changing the default
 value addresses a customer-impacting issue or availability issue for a service.
@@ -111,8 +149,45 @@ default values if the member is marked with the :ref:`internal-trait`.
 
 To allow servers to change default values if necessary, clients SHOULD NOT
 serialize default values unless the member is explicitly set to the default
-value. This implies that clients SHOULD implement a kind of "presence tracking"
-of defaulted members.
+value or marked with the :ref:`default-trait`. This implies that clients
+SHOULD implement a kind of "presence tracking" of defaulted members so that
+the member is only serialized if it is explicitly set to the default value.
+
+
+Default and required
+--------------------
+
+A member that is both ``@default`` and ``@required`` SHOULD always be
+serialized, and implementations SHOULD NOT use any form of presence tracking
+to omit a member if the member is not explicitly set to the default value.
+It is a protocol-specific decision as to whether this is enforced in
+serialized messages; some protocols follow this strictly whereas others may
+not.
+
+
+.. smithy-trait:: smithy.api#addedDefault
+.. _addedDefault-trait:
+
+``addedDefault`` trait
+======================
+
+Summary
+    Indicates that the :ref:`default-trait` was added to a structure member
+    after initially publishing the member. This allows tooling to decide
+    whether to ignore the ``@default`` trait if it will break backward
+    compatibility in the tool.
+Trait selector
+    ``structure > member [trait|default]``
+
+    *Member of a structure marked with the default trait*
+Value type
+    Annotation trait.
+See also
+    * :ref:`structure-optionality`
+    * :ref:`default-trait`
+    * :ref:`clientOptional-trait`
+    * :ref:`input-trait`
+    * :ref:`recommended-trait`
 
 
 .. smithy-trait:: smithy.api#required
@@ -130,8 +205,6 @@ Trait selector
     *Member of a structure*
 Value type
     Annotation trait.
-Conflicts with
-    * :ref:`default-trait`
 See also
     * :ref:`structure-optionality`
     * :ref:`default-trait`

smithy-build/src/main/java/software/amazon/smithy/build/plugins/NullabilityReportPlugin.java

@@ -0,0 +1,86 @@
+/*
+ * Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License").
+ * You may not use this file except in compliance with the License.
+ * A copy of the License is located at
+ *
+ *  http://aws.amazon.com/apache2.0
+ *
+ * or in the "license" file accompanying this file. This file is distributed
+ * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+ * express or implied. See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
+package software.amazon.smithy.build.plugins;
+
+import software.amazon.smithy.build.PluginContext;
+import software.amazon.smithy.build.SmithyBuildPlugin;
+import software.amazon.smithy.model.Model;
+import software.amazon.smithy.model.knowledge.NullableIndex;
+import software.amazon.smithy.model.node.Node;
+import software.amazon.smithy.model.node.ObjectNode;
+import software.amazon.smithy.model.shapes.MemberShape;
+import software.amazon.smithy.model.shapes.StructureShape;
+import software.amazon.smithy.model.traits.BoxTrait;
+
+/**
+ * Generates a JSON report that contains a mapping of every structure member to
+ * whether the member is considered nullable in v1 and v2 implementations.
+ */
+public final class NullabilityReportPlugin implements SmithyBuildPlugin {
+
+    static final String NULLABILITY_REPORT_PATH = "nullability-report.json";
+    private static final String NAME = "nullabilityReport";
+
+    @Override
+    public String getName() {
+        return NAME;
+    }
+
+    @Override
+    public boolean requiresValidModel() {
+        return false;
+    }
+
+    @Override
+    public void execute(PluginContext context) {
+        if (context.getOriginalModel().isPresent() && context.getProjection().isPresent()) {
+            context.getFileManifest().writeJson(NULLABILITY_REPORT_PATH, serializeReport(context));
+        }
+    }
+
+    private static Node serializeReport(PluginContext context) {
+        ObjectNode.Builder root = Node.objectNodeBuilder();
+        Model model = context.getModel();
+        NullableIndex index = NullableIndex.of(model);
+
+        for (StructureShape structure : model.getStructureShapes()) {
+            // Only generate for structures in "sources" that have members.
+            if (!structure.getAllMembers().isEmpty() && context.isSourceShape(structure)) {
+                ObjectNode.Builder struct = Node.objectNodeBuilder();
+                for (MemberShape member : structure.getAllMembers().values()) {
+                    ObjectNode.Builder entry = Node.objectNodeBuilder();
+                    entry.withMember("v1", index.isNullable(member));
+                    entry.withMember("v1-via-box", member.getMemberTrait(model, BoxTrait.class).isPresent());
+                    entry.withMember("v2-client",
+                                     index.isMemberNullable(member, NullableIndex.CheckMode.CLIENT));
+                    entry.withMember("v2-client-careful",
+                                     index.isMemberNullable(member, NullableIndex.CheckMode.CLIENT_CAREFUL));
+                    entry.withMember("v2-client-zero-value",
+                                     index.isMemberNullable(member, NullableIndex.CheckMode.CLIENT_ZERO_VALUE_V1));
+                    entry.withMember("v2-client-zero-value-no-input",
+                                     index.isMemberNullable(member,
+                                                            NullableIndex.CheckMode.CLIENT_ZERO_VALUE_V1_NO_INPUT));
+                    entry.withMember("v2-server",
+                                     index.isMemberNullable(member, NullableIndex.CheckMode.SERVER));
+                    struct.withMember(member.getMemberName(), entry.build());
+                }
+                root.withMember(structure.getId().toString(), struct.build());
+            }
+        }
+
+        return root.build();
+    }
+}

smithy-build/src/main/resources/META-INF/services/software.amazon.smithy.build.SmithyBuildPlugin

@@ -1,3 +1,4 @@
 software.amazon.smithy.build.plugins.BuildInfoPlugin
 software.amazon.smithy.build.plugins.ModelPlugin
 software.amazon.smithy.build.plugins.SourcesPlugin
+software.amazon.smithy.build.plugins.NullabilityReportPlugin