Support maps and heterogeneous arrays as attribute values

Resolves open-telemetry#376

Use cases where this is necessary or useful:

1. Specify more than one resource in the telemetry: open-telemetry#579
2. Data coming from external source, e.g. AWS Metadata: open-telemetry#596 (comment)
   or open-telemetry#376 (comment)
3. Capturing HTTP headers: open-telemetry#376 (comment)
4. Structured stack traces: open-telemetry#2841
5. Payloads as attributes: open-telemetry/oteps#219 (comment)

This is a draft PR to see what the change looks like.

If this PR is merged it will be nice to follow it up with:
- A standard way of flattening maps and nested objects when converting from OTLP
  to formats that don't support maps/nested objects.
- Recommendations for semantic conventions to use/not use complex objects.

CHANGELOG.md

@@ -37,6 +37,9 @@ release.
 - Clarify that attribute keys are case-sensitive.
   ([#3784](https://github.com/open-telemetry/opentelemetry-specification/pull/3784))
 
+- Support maps and heterogeneous arrays as attribute values
+  ([#2888](https://github.com/open-telemetry/opentelemetry-specification/pull/2888))
+
 ### Supplementary Guidelines
 
 ## v1.28.0 (2023-12-07)

spec-compliance-matrix.md

@@ -215,6 +215,7 @@ formats is required. Implementing more than one format is optional.
 | A metric Producer accepts an optional metric Filter                                                                                                                    |          |    |      |     |        |      |        |     |      |     |      |       |
 | The metric Reader implementation supports registering metric Filter and passing them  its registered metric Producers                                                  |          |    |      |     |        |      |        |     |      |     |      |       |
 | The metric SDK's metric Producer implementations uses the metric Filter                                                                                                |          |    |      |     |        |      |        |     |      |     |      |       |
+| Non-homogeneous arrays and maps (including nested)                                                                                                                     |          |    |      |     |        |      |        |     |      |     |      |       |
 
 ## Logs
 
@@ -242,6 +243,7 @@ Disclaimer: this list of features is still a work in progress, please refer to t
 | Feature                                                                                                                                     | Optional | Go | Java | JS | Python | Ruby | Erlang | PHP | Rust | C++ | .NET | Swift |
 |---------------------------------------------------------------------------------------------------------------------------------------------|----------|----|------|----|--------|------|--------|-----|------|-----|------|-------|
 | Create from Attributes                                                                                                                      |          | +  | +    | +  | +      | +    | +      | +   | +    | +   | +    | +     |
+| Non-homogeneous arrays and maps (including nested) for attribute values                                                                     |          |    |      |    |        |      |        |     |      |     |      |       |
 | Create empty                                                                                                                                |          | +  | +    | +  | +      | +    | +      | +   | +    | +   | +    | +     |
 | [Merge (v2)](specification/resource/sdk.md#merge)                                                                                           |          | +  | +    |    | +      | +    | +      | +   | +    | +   | +    |       |
 | Retrieve attributes                                                                                                                         |          | +  | +    | +  | +      | +    | +      | +   | +    | +   | +    | +     |

specification/common/README.md

@@ -29,12 +29,21 @@ An `Attribute` is a key-value pair, which MUST have the following properties:
 
 - The attribute key MUST be a non-`null` and non-empty string.
   - Case sensitivity of keys is preserved. Keys that differ in casing are treated as distinct keys.
-- The attribute value is either:
-  - A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer.
-  - An array of primitive type values. The array MUST be homogeneous,
-    i.e., it MUST NOT contain values of different types.
+- The attribute value can be of `any` type, where any is defined as one of the following:
+    - A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer.
+    - A homogeneous array of values of primitive type [before 1.29.0].
+    - An array of `any` values [since 1.29.0].
+    - A key/value map, where key is string and value is `any` value [since 1.29.0].
 
-For protocols that do not natively support non-string values, non-string values SHOULD be represented as JSON-encoded strings.  For example, the expression `int64(100)` will be encoded as `100`, `float64(1.5)` will be encoded as `1.5`, and an empty array of any type will be encoded as `[]`.
+Complex attribute types (such as homogenous arrays, arrays of any, and maps) SHOULD be
+used sparingly, in situations where their use minimizes manipulation of the data’s
+original structure.
+
+When exporting to protocols that do not natively support a particular non-string
+value type the value should be converted to a string JSON-encoding of the value.
+
+For example, the expression `int64(100)` will be encoded as `100`, `float64(1.5)` will
+be encoded as `1.5`, and an empty array of any type will be encoded as `[]`.
 
 Attribute values expressing a numerical value of zero, an empty string, or an
 empty array are considered meaningful and MUST be stored and passed on to

specification/trace/sdk_exporters/jaeger.md

@@ -138,8 +138,8 @@ OpenTelemetry Span `Attribute`(s) MUST be reported as `tags` to Jaeger.
 
 Primitive types MUST be represented by the corresponding types of Jaeger tags.
 
-Array values MUST be serialized to string like a JSON list as mentioned in
-[semantic conventions](../../overview.md#semantic-conventions).
+Array and map values MUST be serialized to a JSON and recorded as a tag of string type
+as mentioned in [attribute value definition](../../common/README.md#attribute).
 
 ### Links
 

specification/trace/sdk_exporters/zipkin.md

@@ -131,8 +131,8 @@ document maps to the strongly-typed fields of Zipkin spans.
 Primitive types MUST be converted to string using en-US culture settings.
 Boolean values MUST use lower case strings `"true"` and `"false"`.
 
-Array values MUST be serialized to string like a JSON list as mentioned in
-[semantic conventions](../../overview.md#semantic-conventions).
+Array and map values MUST be serialized to a JSON and recorded as a tag of string type
+as mentioned in [attribute value definition](../../common/README.md#attribute).
 
 TBD: add examples