TypeScript types

[micheleangioni]

The TypeScript declaration package is missing in https://www.npmjs.com/~types .

More information on this: https://www.typescriptlang.org/docs/handbook/declaration-files/publishing.html

This is very important to ensure a proper diffusion of the cloudevents standards.

[fabiojose]

@micheleangioni Could you make a PR with this? If you can't, please write an example using some piece of code from sdk.

[micheleangioni]

I do not have the complete library definitions. I have the definitions of the Spec01, Spec02 and Cloudevent classes.

These are the definitions I wrote:

declare module 'cloudevents-sdk' {
  export type SpecPayload = {
    [key: string]: any,
  };

  export class Spec01 {
    public payload: SpecPayload & {
      cloudEventsVersion: '0.1',
      eventID: string,
    };

    public check(): void;

    public type(type: string): Spec01;
    public getType(): string|undefined;

    public getSpecversion(): string;

    public eventTypeVersion(eventTypeVersion: string): Spec01;
    public getEventTypeVersion(): string|undefined;

    public source(source: string): Spec01;
    public getSource(): string|undefined;

    public id(id: string): Spec01;
    public getId(): string|undefined;

    public schemaurl(schemaurl: string): Spec01;
    public getSchemaurl(): string|undefined;

    public contenttype(contenttype: string): Spec01;
    public getContenttype(): string|undefined;

    public data(data: any): Spec01;
    public getData(): any;

    public time(time: Date): Spec01;
    public getTime(): string|undefined;

    public addExtension(key: string, value: any): Spec01;
  }

  export class Spec02 {
    public payload: SpecPayload & {
      specversion: '0.2',
      id: string,
    };

    public check(): void;

    public type(type: string): Spec02;
    public getType(): string|undefined;

    public getSpecversion(): string;

    public eventTypeVersion(eventTypeVersion: string): Spec02;
    public getEventTypeVersion(): string|undefined;

    public source(source: string): Spec02;
    // public getSource(): string|undefined; // TODO Missing getter, to be added. See https://github.com/cloudevents/sdk-javascript/pull/8

    public id(id: string): Spec02;
    public getId(): string|undefined;

    public time(time: Date): Spec02;
    public getTime(): string|undefined;

    public schemaurl(schemaurl: string): Spec02;
    public getSchemaurl(): string|undefined;

    public contenttype(contenttype: string): Spec02;
    public getContenttype(): string|undefined;

    public data(data: any): Spec02;
    public getData(): any;

    public addExtension(key: string, value: any): Spec02;
  }

  export type Extensions = {
    [key: string]: any,
  };

  export class JSONFormatter01 {
    public format(payload: SpecPayload): SpecPayload;
    public toString(payload: SpecPayload): string;
  }

  export default class Cloudevent {
    public static specs: {
      0.1: Spec01,
      0.2: Spec02,
    };

    public static formats: {
      'json': JSONFormatter01,
      'json0.1': JSONFormatter01,
    };

    public static bindings: {
      'http-structured': any,
      'http-structured0.1': any,
      'http-structured0.2': any,
      'http-binary0.1': any,
      'http-binary0.2': any,
    };

    public constructor(spec?: Spec01|Spec02, formatter?: JSONFormatter01);

    public format(): SpecPayload;
    public toString(): string;

    public type(type: string): Cloudevent;
    public getType(): string|undefined;

    public getSpecversion(): string;

    public source(type: string): Cloudevent;
    public getSource(): string|undefined;

    public id(id: string|number): Cloudevent;
    public getId(): string|undefined;

    public time(type: Date): Cloudevent;
    public getTime(): string|undefined;

    public schemaurl(schemaurl: string): Cloudevent;
    public getSchemaurl(): string|undefined;

    public contenttype(contenttype: string): Cloudevent;
    public getContenttype(): string|undefined;

    public data(data: any): Cloudevent;
    public getData(): any;

    public addExtension(key: string, value: any): Cloudevent;
    public getExtensions(): Extensions;
  }
}

[fabiojose]

@micheleangioni Now I understood. Do you want to join us and work in this issue?

[micheleangioni]

Yes, sure. I hope to find some time to write the declarations also of the remaining classes, hopefully in a couple of weeks :)

[cwallsfdc]

@micheleangioni, would be awesome!

Novice question, how to use the type definition above w/ cloudevents-sdk? I placed the type definition here lib/@types/cloudevents-sdk/index.d.ts, but it's unclear how to create a Cloudevent with a JSON string.

Thanks.

[cwallsfdc]

@micheleangioni, I had to change the typing to the following to match cloudevents.js's single export of Cloudevent.

https://gist.github.com/cwallsfdc/183c2df8dad88964efa957217b12c496

[fabiojose]

@cwallsfdc Can you help us in how to publish this typing in NPM?

[cwallsfdc]

Honestly, I think a better approach is to convert to Typescript and include the typings in the cloudevents-sdk npm module. This way the lib and typing stay aligned. Migrating to Typescript shouldn't disrupt users.

If in agreement, I can post a PR w/ Typescript changes. We'd then publish a new cloudevents-sdk version w/ typings.

Thoughts?

[fabiojose]

LGTM!

@cwallsfdc, go ahead! Perform your PR!

[grant]

Google Cloud Functions uses it's own TypeScript types because this SDK does not have them.

Google Cloud Functions Framework:
https://github.com/GoogleCloudPlatform/functions-framework-nodejs/blob/v1.0.0/src/invoker.ts#L63

[grant]

Answering questions from email:

When the sdk converted to ts, we can use it in pure js projects?

Yes. When we publish to npm, we publish pure js and we have files for TypeScript types for autocompletion.

I am comfortable with this approach, using definitely typed.

We can publish this npm module instead of using DefinitelyTyped. See https://www.typescriptlang.org/docs/handbook/declaration-files/publishing.html

---

Just let me know when master and dev are the same so I can do the conversion without new changes in the middle. It will take a bit of time.

[fabiojose]

@grant Thanks for the answers.

Because of the changes that are coming from the spec, I think that is more prudence to publish the types using Definitely Typed. This will help the projects written in TypeScript and we save time to model the stuff for the future of the sdk (writing in ts) and to work in the incoming spec changes.

It's fair if we make voting:

• Vote a if you are in favor of to create just the types and publish to DefinitelyTyped
• Vote b if you are in favor of to migrate the entire codebase from JavaScript to TypeScript

What do you thing about that @grant ?

rigth now, master and develop remains the same

[grant]

b is so much better and easier.
This is what Google does with all JavaScript libraries.

[fabiojose]

@micheleangioni
@cwallsfdc

[cwallsfdc]

Thanks for leading this charge @fabiojose.

I agree with @grant. b. TypeScript is natural and well supported.

[fabiojose]

b wins!

Vote b if you are in favor of to migrate the entire codebase from JavaScript to TypeScript

@grant Thanks for your contribution! Let's code?

[fabiojose]

@grant

Will you code for this issue? Just to know.

[grant]

I can't continue since this repo doesn't seem to follow the CNCF standards of having reviews. Need to check on Slack and with others first.

[fabiojose]

Nice!

This is a great opportunity. You can become a new commiter and helps us to review. At this moment I am the only one: maintainer and commiter.

Because of that, I guess, you got that impression of loose standards.

Come, join us! Make this better!

[duglin]

this repo doesn't seem to follow the CNCF standards of having reviews

@grant can you elaborate? As @fabiojose mentioned, we do try to follow "normal" review processes (see the golang repo, it has more people involved) but if there's only one person active in the repo then it's hard to require multiple people for reviews :-)

[grant]

In order for me to contribute, I need master to not change as fast as it is. The commits made to master are not PRs, they are direct to master.

I would like to convert this repo to typescript, so that I can use the SDK with Cloud Functions, but last time I tried, there were commits to master that conflicted with my conversion.

All commits in this repo should be tied to a PR and all PRs are squashed so we can see the history change.

Aside, I don't really understand why we should have exceptions for reviews with certain import repos.

[takethefake]

I would like to join migrating cloudevents to typescript!

[fabiojose]

In order for me to contribute, I need master to not change as fast as it is. The commits made to master are not PRs, they are direct to master.

I would like to convert this repo to typescript, so that I can use the SDK with Cloud Functions, but last time I tried, there were commits to master that conflicted with my conversion.

All commits in this repo should be tied to a PR and all PRs are squashed so we can see the history change.

Aside, I don't really understand why we should have exceptions for reviews with certain import repos.

@grant, in the early days I used gitflow to merge to master, without PRs. Never commit directly.

Now we are using PRs, but I must approve by myself. You can help us in the review of that PRs.

Is that makes senses to you?

[grant]

Now we are using PRs, but I must approve by myself. You can help us in the review of that PRs.
Is that makes senses to you?

Makes sense. I can look into converting this package again. It needs a lot of major changes though.

[fabiojose]

Folks, it's not the best, but now we have types for version 1.0.0

[Gerrit-K]

A remark from a (rather inexperienced) typescript user: the provided types are not really flexible as they are more of an interface than a type definition. I think the whole structure would be much more versatile (i.e. usable in / connectable to other libraries) if you split the event builder pattern from the bare data model and use basic object attributes instead of getters and setters in the type definition.

In my case, I wanted to receive the events from rabbitmq and store them in mongodb but both libraries expect type definitions with bare objects (and for instance iterate over their keys).

[grant]

Nice!

This is a great opportunity. You can become a new commiter and helps us to review. At this moment I am the only one: maintainer and commiter.

Because of that, I guess, you got that impression of loose standards.

Come, join us! Make this better!

Hi @fabiojose,
Do you mind making me a collaborator to this repo? I have an expertise in JavaScript/TypeScript and would really like to make this project better.

At Google, we don't use this CloudEvent SDK (even though we use CloudEvents with Node) because of some of the issues described above. I would like to improve this situation but I need to be able to contribute to this repo and use best practices.

To restart trying with this repo, I've created a simple PR:
#63

After that, I would love to help with doing things like simplifying the multiple versions we have in this repo and conversion to modern TypeScript.

[DavidWells]

Yeah, I guess it just depends on preference.

Personally, I prefer just javascript because there is a wider audience of developers who author code in JS. So in theory, the larger opportunity for OS contributions lies in pure JS. Additionally, I think Typescript has a larger learning curve and can be trickier for folks to pick up.

Both work and I have no dog in this fight 😃. Pros and cons both ways

[duglin]

I finally found time to read this whole thread! oy vey! :-)

To make sure I understand things since (as I've said) I don't know TS at all and don't do anything with JS outside of a browser, early on in the thread it seemed like there were two concerns from @lance w.r.t. the proposed direction:
1 - it would require users of the SDK to rename their files form *.js to *.ts
2 - it would require users of the SDK to add a watch/build step

And, as I understand it, TS is a compiled language in the sense that it needs to be converted from TS to JS before it is executed (interpreted) at runtime.

Sound ok so far?

If so, I have just a couple of questions:

• since TS is a superset of JS, would it make sense (or be possible) to have a JS SDK be leveraged by a TS user in a way that makes sense to them? Or would they expect things to be there (like static typing) that the JS SDK would not have?
• Assuming a JS SDK wouldn't be a good fit for a TS user, would the TS SDK need (or want) to leverage the JS SDK code at all? Meaning, would it ever make sense for the TS SDK to be a layer on top of the JS SDK or is that just an extra layer without a purpose and the TS SDK would be better written to be a native CE processor?
• If a layering approach makes no sense, which implies to me that there's basically no code sharing, would there be any benefit to both code bases being in the same repo? The only thing that comes to mind is a shared conceptual model/design but we (sort of) try to do that today across the various SDK already.

I'm guessing the answer to all 3 bullets is basically "no" which is why we landed on the "new SDK" decision. But I wanted to make sure I'm up to speed.

How far off am I? :-)

[lance]

I finally found time to read this whole thread! oy vey! :-)

Thanks for taking the time - and on a Friday night, even. 😄

it seemed like there were two concerns from @lance w.r.t. the proposed direction:
1 - it would require users of the SDK to rename their files form *.js to *.ts
2 - it would require users of the SDK to add a watch/build step

Not really - those were concerns that were attributed to me by others. Really what I care about is maintainer productivity and the larger pool of potential contributors (and alienating existing contributors who have only recently started to submit code (including myself)).

And, as I understand it, TS is a compiled language in the sense that it needs to be converted from TS to JS before it is executed (interpreted) at runtime.

Correct.

since TS is a superset of JS, would it make sense (or be possible) to have a JS SDK be leveraged by a TS user in a way that makes sense to them?

Yes - see David's comment above. Personally, I think this is the best path forward for a possible migration to TypeScript in the future. This is the compromise I have recently suggested. I honestly do not want this community fractured and I'm trying to be open about this. My preference is pure JS. Clearly, that feeling is not universal. ☯️. I personally would be most comfortable with a graduated approach using the methods suggested by David.

Concerns about doc comments drifting from implementation are valid, but there are ways to deal with this using tools and processes.

[helio-frota]

I have limited TS knowledge and I have no room for TS learning in my free time to be able to be comfortable in contributing.

[grant]

1 - it would require users of the SDK to rename their files form *.js to *.ts
2 - it would require users of the SDK to add a watch/build step
Sound ok so far?

Yes. Those are the two main points, .ts files and running watch/build. Adding tools/processes on top of this is fine. I think another fear of developers is that TS has a steep learning curve (it does not, there is literally no difference if you only use JS features).

TypeScript is required for many projects including those at Google and Microsoft.

The best path forward is a separate implementation with TypeScript. If there is value in merging projects at a later point, that sounds fine.

[erikerikson]

@grant Can you explain the specifics of the Typescript requirement? Are guaranteed to be correct typescript types are not enough?

The js=>ts renaming doesn't seem to be necessary with the proper tsconfig and the addition of watch/build using tsc and enforced by CI has largely been ceded. Do I appear wrong on these points? Insisting on these two points despite direct statement that they are not the concern is a curious behavior.

You seem to be implying that only JavaScript features would be used. I'm curious how you expect that could be affected. Do you expect contributors from across the internet to forgo using TypeScript features despite project conversion and labeling as typescript? Is there a mechanism you have in mind for enforcing that? Are you willing to agree to it's use?

[grant]

@grant Can you explain the specifics of the Typescript requirement? Are guaranteed to be correct typescript types are not enough?

Sure. As a developer:

• I must know the exact properties of a CloudEvent before execution.
• I must never see "undefined" is not a function or equivalent for SDK users.
• I must have autocompletion in my developer tooling/IDE for CloudEvents.
• I must be able to click to definition to see the source code of the SDK.
• I must see linting errors in my code if I use the CloudEvent SDK incorrectly (i.e. invalid type).

There are a few projects that could use an SDK for CloudEvents in Node.

Here is an example project that could use the SDK with TypeScript:

Example at Google

https://www.npmjs.com/package/@google-cloud/functions-framework

It powers serverless functions for Google Cloud. We don't use this module to accept/parse HTTP requests with CloudEvents due to a few issues, one being lack of TS support.

We require autocompletion and static checks to ensure that the module is running correctly. It's vital for ensuring we don't crash our product for thousands of customers on a bad deploy. Many other companies use this superset language.

It's Literally Node

It's literally no different, please browse the code of the TypeScript version:
https://github.com/loopingz/sdk-javascript
This is the literal code change that we're talking about...

It's Incremental

You don't need to learn a new language. We can enforce not using non-JS features.

It's an industry-wide standard that you're probably using right now

If someone was to magically enforce types and IDE integration with types that are not from TypeScript, that's fine. That's really the point of TypeScript though. There is a whole ecosystem and company support around TypeScript.

Example players supporting TS: https://tsconf.io/

I can't find any project that does anything similar.

I assume many people use VS Code for code editing right? That heavily uses TypeScript.

Other Questions

The js=>ts renaming doesn't seem to be necessary with the proper tsconfig and the addition of watch/build using tsc and enforced by CI has largely been ceded. Do I appear wrong on these points? Insisting on these two points despite direct statement that they are not the concern is a curious behavior.

Editors like VS Code will do it's best with just js files or just a tsconfig. You're really using TypeScript under-the-hood. At that point, you're really just avoiding the filename rename for some reason. 🤷

You seem to be implying that only JavaScript features would be used. I'm curious how you expect that could be affected. Do you expect contributors from across the internet to forgo using TypeScript features despite project conversion and labeling as typescript? Is there a mechanism you have in mind for enforcing that? Are you willing to agree to it's use?

I am saying that the main issue we're facing is the fear of change. We don't have to change any of the JavaScript if we don't want to to use TypeScript. There aren't that many contributors to this project for us to know to not accept PRs with extra TS features. We don't even consistently use ES6. I'm willing to agree on not using TypeScript features and we can add a simple lint file to enforce this via tslint.

[johnm]

Finally, caught up with all of this thread and the slack discussion. :-) As I brought up the concern in last week's call and now will say more strongly... I'm against having two separate, "official" "Javascript" sdks. That's just plain bad optics and makes for confusion, bugs, interop, support, etc. issues for users/consumers/providers of CE.

[johnm]

If I understand the camps' positions, @loopingz's code translation and some simple policies such as the tslint enforcement seem like a fair place to start so that we can remain as single JS SDK for CE. As someone who has no personal stake in the pure-JS vs. TS approach, I personally hope that people in the various camps are willing to at least try a joint approach openly before splitting into two separate, competing efforts.

[grant]

As I brought up the concern in last week's call and now will say more strongly... I'm against having two separate, "official" "Javascript" sdks.

Hi @johnm, thanks for your opinion, but there are a couple issues. I literally cannot use this project unless it fully supports TypeScript. I oppose two separate JS-like repos too on the surface. I'm happy to merge separate repos if possible.

What do you expect me to do? This is a deal-blocker.

We currently have separate efforts, literal code duplication already, since there is no TypeScript support.

I'm obviously not as happy to continue a separate effort outside of a this repo or outside of this GitHub org.

[erikerikson]

@grant

I am saying that the main issue we're facing is the fear of change.

Two concerns I have heard are exclusion (or even just reduced welcome) and a potentially reduced contributor pool and thereby speed of improvement and community inclusion.

I understand that typescript is a superset and have indicated personal comfort with it (by admitting use). To be transparent I would welcome a full transition an use of typescript constructs personally but my personal mores are unimportant in this case. Depending on how we introduce typescript (if we do at all) we could create externalities. This includes factors such as a subset of the population seeing unexpected constructs and be put off regardless of what size the learning curve might be.

Thank you for sharing the ability of tslint to enforce sticking to core JavaScript. I'm curious if using that sufficient compromise for those who would like to stick with vanilla JavaScript (@lance or anyone else?). I'm also a bit mystified by the assertion that the technical/concrete requirements would care at all about a file name. If we look at loopingz's repo there are a couple further changes in it but they also seem pretty reasonably limited.

What do you expect me to do? This is a deal-blocker.

It turns out that (as a stand in for an important subset of users) your needs matter too. What we are talking through are the deal-making constraints and options. I apologize that my ignorance is forcing me to ask you to explain options and a bit of how our potential choices interact and what the choices available to us are. I can imagine it is a little frustrating but I can only be where I am, please apply patience as you can.

[johnm]

As I brought up the concern in last week's call and now will say more strongly... I'm against having two separate, "official" "Javascript" sdks.

Hi @johnm, thanks for your opinion, but there are a couple issues. I literally cannot use this project unless it fully supports TypeScript. I oppose two separate JS-like repos too on the surface. I'm happy to merge separate repos if possible.

What do you expect me to do? This is a deal-blocker.

We currently have separate efforts, literal code duplication already, since there is no TypeScript support.

I'm obviously not as happy to continue a separate effort outside of a this repo or outside of this GitHub org.

Hm... Did you respond before reading my second comment? I hope that made clear, as I also commented in the call, was that it seems that there's a combined approach place to come together other. Based on that, are there still deal breakers?

[grant]

Well, thanks for all the replies and trying to figure this out @erikerikson.

Sorry @johnm, I was reacting a bit. This issue has been open for over a year with hundreds of commits to this project directly to master and no progress on issues like this one despite PRs, forks, Slack discussions, calls, etc. The comment of immediately positioning against making progress on this issue via separate repos was surprising and did not suggest a way forward.

[cwallsfdc]

38% of bugs at Airbnb could have been prevented by TypeScript according to postmortem analysis

To type or not to type: quantifying detectable bugs in JavaScript

[johnm]

Sorry @johnm, I was reacting a bit. This issue has been open for over a year with hundreds of commits to this project directly to master and no progress on issues like this one despite PRs, forks, Slack discussions, calls, etc. The comment of immediately positioning against making progress on this issue via separate repos was surprising and did not suggest a way forward.

No worries, I understand the frustration as I've been in similar situations previously.

FWIW, I separated the two different aspects on purpose. There are the problems at technology level but also at the project level (of the impact on the users/etc). I think the context of the project should be keep in mind when dealing with the tradeoffs in making the technology decisions.

[johnm]

38% of bugs at Airbnb could have been prevented by TypeScript according to postmortem analysis

To type or not to type: quantifying detectable bugs in JavaScript

May I come in and talk with you about our strongly typed savior, Rust & Wasm? ;-) :-)

[lance]

• I must know the exact properties of a CloudEvent before execution.
• I must never see "undefined" is not a function or equivalent for SDK users.
• I must have autocompletion in my developer tooling/IDE for CloudEvents.
• I must be able to click to definition to see the source code of the SDK.
• I must see linting errors in my code if I use the CloudEvent SDK incorrectly (i.e. invalid type).

I was under the belief that using tsc and JSDoc would ensure that all of this is possible. Is that not the case?

Editors like VS Code will do it's best with just js files or just a tsconfig. You're really using TypeScript under-the-hood. At that point, you're really just avoiding the filename rename for some reason.

If type checking is enabled with tsc and there is a valid tsconfig, then VSCode should be satisfied, I think. I've recently been experimenting with this on my fork, and have found that it's quite effective.

I am saying that the main issue we're facing is the fear of change.

I might say reluctance to change. There are existing contributors (including myself) who are reluctant to switch fully to TS for a number of reasons. But I don't know that you are representing them in good faith. A reduced contributor pool and alienation of existing contributors are what I care most about. I really don't like the required transpilation stage in order to actually have a JavaScript engine be capable of reading my code, but I can live with that, I suppose. I do like the fact that tslint can enforce pure JS. But I wonder, what's the point of actually converting to TS in that case if we can achieve the user-facing goals (knowing the properties of a CE, autocompletion, linting, etc) with tsc, JSDoc and other tooling?

As @johnm has suggested, I would like to consider a path forward that starts small, using tsc to check the code, jsdoc to enable both IDE autocomplete and help tsc, and type declaration generation. Do this with an eye towards eventually moving to TS when this whole thing has simmered down a bit. I understand that there are potential benefits to TS. But there are reasons to take it slowly. I would like to think that this is a fair compromise.

To be quite honest, I would be surprised if Google immediately started using this project if we did nothing but switch to TypeScript by changing file names and adding a watch command. Those first compromising steps seem necessary in any case.

Rather than a switch to TypeScript, my most pressing concerns are:

• Reduce the surface area of the API to be much simpler, e.g. const { CloudEvent, HTTPEmitter, HTTPReceiver } = require('cloudevents-sdk');
• Add JSDoc to the entire public facing API (those classes above)
• Design the API so that it is more consistent with the spec
• Release 2.0.0 soon.

Honestly, I think these add much more value than renaming a bunch of files. I'd like to focus on these before making any major repository-wide changes if possible.

[grant]

tsc is just a tool to transpile TypeScript to JavaScript.

One of the big point of TypeScript is to not have to duplicate your code inside of docs. Right now we don't have many comments in the code. The suggestion here is to add JSDoc and annotate types everywhere. That's going to create a lot of boilerplate.

There are 2 options:

1. Add JSDoc everywhere. Keep them up-to-date with refactors.
2. Rename files from .js to .ts and automatically understand types (even without annotations).

The above comment chooses option 1. I'm suggesting 2.

Again, switching to TypeScript is not a big change that will prevent you from having to write all that JSDoc.

---

To be quite honest, I would be surprised if Google immediately started using this project if we did nothing but switch to TypeScript by changing file names and adding a watch command. Those first compromising steps seem necessary in any case.

I literally work for Google, for Events, and have said multiple times I'd use this module if it was useful and supported TypeScript. See #9 (comment)

[lance]

Fixed in: #155