Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(api-logs): align AnyValue to spec #4893

Merged
merged 2 commits into from
Aug 9, 2024

Conversation

blumamir
Copy link
Member

@blumamir blumamir commented Aug 1, 2024

Following this PR in contrib I noticed that the AnyValue type in @opentelemetry/api-logs is not accurate, as the spec defined more allowed types. This PR aligns the AnyValue type to match the spec:

Type any
Value of type any can be one of the following:

  • A scalar value: string, boolean, signed 64 bit integer, or double precision floating point (IEEE 754-1985)

  • A byte array,

  • An array (a list) of any values,

  • A map<string, any>,

  • [since 1.31.0] An empty value (e.g. null).

@blumamir blumamir requested a review from a team August 1, 2024 06:58

/**
* AnyValueMap is a map from string to AnyValue (attribute value or a nested map)
*/
export interface AnyValueMap {
[attributeKey: string]: AnyValue | undefined;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewer note: since now AnyValue can be null or undefined, it is not needed to state it here for map specifically

*/
export type AnyValue = AttributeValue | AnyValueMap;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewer note: Since AttributeValue is:

export type AttributeValue =
  | string
  | number
  | boolean
  | Array<null | undefined | string>
  | Array<null | undefined | number>
  | Array<null | undefined | boolean>;

Then removing it and adding the AnyValueScalar and AnyValueArray should already include all the previous allowed type, thus this type only extends the previous one and thus I don't think it's a breaking change

@pichlermarc pichlermarc added bug Something isn't working pkg:api-logs target:next-minor-release This PR is in scope for the next minor release (`main` branch) labels Aug 7, 2024
export type AnyValue = AttributeValue | AnyValueMap;
export type AnyValue =
| AnyValueScalar
| Uint8Array
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several typed array types, do we want to have them all listed here? https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray Maybe having a AnyTypedArray type now

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think any of the other TypedArrays would get borked on OTLP export. The toAnyValue() in otlp-transformer specifically knows about Uint8Array:

if (value instanceof Uint8Array) return { bytesValue: value };

But doesn't know the other TypeArrays.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed what the spec wording that said A byte array, as something that should be supported as any value.

Since byte is 0-255, I think it aligns with Uint8 in contrast to Int8Array for example, which semantically is not byte array I think. as @trentm mentioned, they cannot be serialized into OTLP as bytesValue anyhow as the proto field only accepts bytes bytes_value = 7;.

However, it will still be exported (to my understanding) as arrayValue

  if (Array.isArray(value))
    return { arrayValue: { values: value.map(toAnyValue) } };

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it will be exported as an object with integer-string keys, because:

> a = new Int8Array([1,2,3])
Int8Array(3) [ 1, 2, 3 ]
> Array.isArray(a)
false
> typeof a
'object'

From my playing around on open-telemetry/opentelemetry-js-contrib#2352 (comment) and with a LogRecord with an attribute aInt8Array: new Int8Array([-7,8,9]) the OTLP server sees something like this:

                KeyValue {
                  key: 'aInt8Array',
                  value: AnyValue {
                    kvlistValue: KeyValueList {
                      values: [
                        KeyValue { key: '0', value: AnyValue { intValue: Long { low: -7, high: -1, unsigned: false } } },
                        KeyValue { key: '1', value: AnyValue { intValue: Long { low: 8, high: 0, unsigned: false } } },
                        KeyValue { key: '2', value: AnyValue { intValue: Long { low: 9, high: 0, unsigned: false } } }
                      ]
                    }
                  }
                },

FWIW, this is basically how JSON.stringify also serializes these TypedArrays:

> JSON.stringify(a)
'{"0":1,"1":2,"2":3}'

@@ -14,16 +14,29 @@
* limitations under the License.
*/

import { AttributeValue } from '@opentelemetry/api';
export type AnyValueScalar = string | number | boolean;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should support bigint here as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://opentelemetry.io/docs/specs/otel/logs/data-model/#type-any says:

signed 64 bit integer,

but JS's MAX_SAFE_INTEGER is less than that...

so, probably, yes we should add BigInt?

However BigInt requires Node.js 10.4.0 and api/package.json (the package in which api-logs eventually will land) has:

  "engines": {
    "node": ">=8.0.0"
  },

so, I'm not sure if that is a problem.

There was some discussion around BigInt usage in some exporters-related PR a while back I thought.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, this could be discussed separate from this PR. This change isn't removing BigInt from the type.

@@ -14,16 +14,29 @@
* limitations under the License.
*/

import { AttributeValue } from '@opentelemetry/api';
export type AnyValueScalar = string | number | boolean;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://opentelemetry.io/docs/specs/otel/logs/data-model/#type-any says:

signed 64 bit integer,

but JS's MAX_SAFE_INTEGER is less than that...

so, probably, yes we should add BigInt?

However BigInt requires Node.js 10.4.0 and api/package.json (the package in which api-logs eventually will land) has:

  "engines": {
    "node": ">=8.0.0"
  },

so, I'm not sure if that is a problem.

There was some discussion around BigInt usage in some exporters-related PR a while back I thought.

@@ -14,16 +14,29 @@
* limitations under the License.
*/

import { AttributeValue } from '@opentelemetry/api';
export type AnyValueScalar = string | number | boolean;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, this could be discussed separate from this PR. This change isn't removing BigInt from the type.

export type AnyValue = AttributeValue | AnyValueMap;
export type AnyValue =
| AnyValueScalar
| Uint8Array
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think any of the other TypedArrays would get borked on OTLP export. The toAnyValue() in otlp-transformer specifically knows about Uint8Array:

if (value instanceof Uint8Array) return { bytesValue: value };

But doesn't know the other TypeArrays.

Copy link
Member

@hectorhdzg hectorhdzg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working pkg:api-logs target:next-minor-release This PR is in scope for the next minor release (`main` branch)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants