Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Azure Schema Registry Commitments for JS (July 2021-March 2022) #15959

Closed
7 of 12 tasks
AlexGhiondea opened this issue Jun 24, 2021 · 0 comments
Closed
7 of 12 tasks

Azure Schema Registry Commitments for JS (July 2021-March 2022) #15959

AlexGhiondea opened this issue Jun 24, 2021 · 0 comments
Assignees
@deyaaeldeen
Labels
Client This issue points to a problem in the data-plane of the library. Epic Schema Registry
Milestone
[2022] March

Comments

@AlexGhiondea
Copy link
Contributor

AlexGhiondea commented Jun 24, 2021 •
edited by deyaaeldeen
Loading

Azure Schema Registry is a feature of Event Hubs, which provides a central repository for schema documents for event-driven and messaging-centric applications.

The @azure/schema-registry package provides the SDK for JavaScript & TypeScript developers to write applications that use Azure Schema Registry. Version 1 of this package follows the latest guidelines for writing TypeScript SDKs for Azure services with the goal of being a client library that is developer-friendly, idiomatic to the JavaScript ecosystem, and as consistent across different languages and platforms as possible.

This issue tracks the work we have planned for this package for the duration of July - December 2021

Epic for other languages: .NET, Python, and Java
Open issues: link

New features:

General improvements:

Testing improvements:
@AlexGhiondea AlexGhiondea added Client This issue points to a problem in the data-plane of the library. Schema Registry labels Jun 24, 2021
@AlexGhiondea AlexGhiondea added this to the [2022] January milestone Jun 24, 2021
@AlexGhiondea AlexGhiondea changed the title Azure Schema Registry Commitments for JS Azure Schema Registry Commitments for JS (July-December 2021) Jun 24, 2021
@deyaaeldeen deyaaeldeen self-assigned this Aug 3, 2021
@ramya-rao-a ramya-rao-a changed the title Azure Schema Registry Commitments for JS (July-December 2021) Azure Schema Registry Commitments for JS (July 2021-March 2022) Dec 15, 2021
@deyaaeldeen deyaaeldeen mentioned this issue Jan 14, 2022
Merged
deyaaeldeen added a commit that referenced this issue Jan 26, 2022
@deyaaeldeen
[Schema Registry Avro] New Encoder Design (#19842)
be4578a 
Fixes #20061

## Overview
Revamps the schema registry encoder to work on messages instead of buffers based on the recommendation of the Azure messaging architect.

This changes the APIs as follows:
```ts
  const buffer: NodeJS.Buffer = await serializer.serialize(value, schema);
```
becomes
```ts
  const message: MessageWithMetadata = await encoder.encodeMessageData(value, schema);
```
where `MessageWithMetadata` has a `body` field as well as a `contentType` field. The latter's format is `avro/binary+<Schema ID>`

For derserializing, the change is as follows:
```ts
  const deserializedValue = await serializer.deserialize(buffer);
```
becomes:
```ts
  const decodedObject = await encoder.decodeMessageData(message);
```

## Improvement upon  #15959

This design introduces a new `messageAdapter` option in the encoder constructor to support processing of any message type (e.g. [cloud event](https://github.com/cloudevents/spec/blob/v1.0.1/spec.md)):

```ts
  const encoder = new SchemaRegistryAvroEncoder(schemaRegistryClient, {
    groupName,
    messageAdapter: adapter
  });
```

where `adapter` is a message adapter that follows the following contract:

```ts
interface MessageAdapter<MessageT> {
  produceMessage: (messageWithMetadata: MessageWithMetadata) => MessageT;
  consumeMessage: (message: MessageT) => MessageWithMetadata;
}

 interface MessageWithMetadata {
  body: Uint8Array;
  contentType: string;
}
```

For convenience, the PR adds a couple of convenience adapter factories for Event Hubs's `EventData` and Event Grid's `SendCloudEventInput<Uint8Array>`. For example, the `createCloudEventAdapter` factory can be called to construct an adapter for the latter as follows:

```ts
const adapter = createCloudEventAdapter({
      type: "azure.sdk.eventgrid.samples.cloudevent",
      source: "/azure/sdk/schemaregistry/samples/withEventGrid",
    }),
```

Note that these adapter factories are exported by their respective messaging package without explicitly implementing the contract and the PR adds new encoder tests that check whether the produced adapters follow the contract. This organization could change in the future if we create a new core place for the contract to be imported from.

See the newly added samples for how to send such messages with Event Hubs and Event Grid.

Schema Registry commitment tracking: #15959
Tracking issue: #18608
First iteration design: #18365
@lmazuel lmazuel closed this as completed Mar 2, 2022
@github-actions github-actions bot locked and limited conversation to collaborators Apr 11, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@deyaaeldeen deyaaeldeen

Labels
Client This issue points to a problem in the data-plane of the library. Epic Schema Registry
Projects
None yet
Milestone
[2022] March
Development

No branches or pull requests
4 participants
@lmazuel @deyaaeldeen @AlexGhiondea @ramya-rao-a