Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(open-payments): adding open-payments package #669

Merged
merged 18 commits into from
Oct 24, 2022
Merged

Conversation

mkurapov
Copy link
Contributor

@mkurapov mkurapov commented Oct 12, 2022

Changes proposed in this pull request

This is a first PR for roadmap item: interledger/roadmap#20. More discussion can be found there.

This PR deals with creating a new package, open-payments. It uses the openapi-typescript package to generate Typescript types from the Open API resource server spec, and then exports the types (to be used in the /backend package). It also exports a client to be used when making rafiki to rafiki calls via the Open Payments API.

The usage of the client is as follows:
Screen Shot 2022-10-12 at 3 27 16 PM

Additional changes are to follow, want to keep the PR sizes manageable:

  • Open API validation (from open-api package)
  • Usage of client in the /backend package, replacing how the incoming-payments and connections are grabbed in the existing OpenPaymentsClientService

Context

The related issues are #663, and #662.

Checklist

  • Related issues linked using fixes #number
  • Tests added/updated
  • Documentation added
  • Make sure that all checks pass

@@ -0,0 +1,64 @@
import { components } from './types'
import axios, { AxiosInstance } from 'axios'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

considered universal/isomorphic fetch ?
node 18 LTS is due very soon
https://dev.to/cloudx/nodejs-18-fetch-api-test-runner-module-and-more-2ckg

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, looks to be out Oct 25th: https://github.com/nodejs/release#release-schedule. I have axios for now as I was mirroring the existing OpenPaymentsClientService https://github.com/interledger/rafiki/blob/main/packages/backend/src/open_payments/client/service.ts#L153-L180. I know we use axios in a few places so maybe have a team discussion whether we want to upgrade node and replace axios with fetch altogether once node 18 is out

},
"devDependencies": {
"@types/node": "^18.7.12",
"openapi-typescript": "^4.5.0",
Copy link
Contributor Author

@mkurapov mkurapov Oct 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrested quite a bit with this, but I wasn't able to use the newest version of this library. Using the programmatic way of generating types via openapiTS anything above v5 does not work and result in ERR_REQUIRE_ESM errors. Versions 5 and above do not support backwards compatibility with CommonJS, especially given the fact that having type: module in package.json breaks using this package as a workspace in the /backend project.

one two three are the related issues in the openapi-typescript repo.

@@ -0,0 +1,5 @@
export default {
OPEN_PAYMENTS_OPEN_API_URL:
Copy link
Contributor Author

@mkurapov mkurapov Oct 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

want to make sure the same URL is used for type generation and (soon to be added) open API response validation

@mkurapov mkurapov changed the title feat(open-payments-client): initial POC of open-payments-client feat(open-payments): initial POC of open-payments-client Oct 14, 2022
@mkurapov mkurapov changed the title feat(open-payments): initial POC of open-payments-client feat(open-payments): initial POC of open-payments package Oct 14, 2022
@github-actions github-actions bot added the type: tests Testing related label Oct 14, 2022
@mkurapov mkurapov changed the title feat(open-payments): initial POC of open-payments package feat(open-payments): adding open-payments package Oct 14, 2022

await get(axiosInstance, {
url: `${baseUrl}/incoming-payment`,
accessToken: 'accessToken'

Check failure

Code scanning / CodeQL

Hard-coded credentials

The hard-coded value "accessToken" is used as [authorization header](1).
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(this is a test file)

`${baseUrl}/incoming-payment`,
{
headers: {
Authorization: 'GNAP accessToken',

Check failure

Code scanning / CodeQL

Hard-coded credentials

The hard-coded value "GNAP accessToken" is used as [authorization header](1).
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(this is a test file)

Comment on lines +54 to +67
export const createClient = (
args?: CreateOpenPaymentClientArgs
): OpenPaymentsClient => {
const axios = createAxiosInstance(args)

return {
incomingPayment: {
get: (args: GetArgs) => get<IncomingPayment>(axios, args)
},
ilpStreamConnection: {
get: (args: GetArgs) => get<ILPStreamConnection>(axios, args)
}
}
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wanted to keep very simple for the first PR, will be adding open-api validation next.

import { components } from './generated/types'

export type IncomingPayment = components['schemas']['incoming-payment']
export type ILPStreamConnection = components['schemas']['ilp-stream-connection']
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can definitely be changed to something less wordy than ILPStreamConnection, but I was just following the naming in the spec

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@github-actions github-actions bot added the type: ci Changes to the CI label Oct 14, 2022
@mkurapov
Copy link
Contributor Author

Will add documentation once there is a proper usage of the client in the /backend package.

@mkurapov mkurapov marked this pull request as ready for review October 14, 2022 14:00
packages/open-payments/src/config.ts Outdated Show resolved Hide resolved
@@ -0,0 +1,702 @@
/**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At one point we had (tried to have) a check that the generated files matched the result of generate (here generate:types) script.
Do you think that's worth doing here?
https://github.com/interledger/rafiki/pull/453/files

Copy link
Contributor Author

@mkurapov mkurapov Oct 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wilsonianb what we can do is instead of that check, is actually just run the script on a pre-commit hook to generate the types and stage the updated file for committing. Instead of doing it on every commit, we can just do that only if the src/types.ts file has been changed, for example. Thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good to me.
We did recently remove pre-commit hooks, but hopefully it won't be an issue to support this

import { components } from './generated/types'

export type IncomingPayment = components['schemas']['incoming-payment']
export type ILPStreamConnection = components['schemas']['ilp-stream-connection']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wilsonianb
wilsonianb previously approved these changes Oct 14, 2022
@mkurapov
Copy link
Contributor Author

mkurapov commented Oct 21, 2022

@wilsonianb the openpayments spec split has caused the type generation to break, it will require a change to that repo to support generating these types. Basically, when referencing the shared schema, the types are generated as such:

Screen Shot 2022-10-21 at 3 11 19 PM

Basically, nothing gets generated for the shared schema:
Screen Shot 2022-10-21 at 3 11 28 PM

This is because the shared schemas do not list components under components.schemas types, but directly use fields at the root level.

eg the shared schema is declared as $ref: ../../shared/schemas.yaml#/access, instead of ../../shared/schemas.yaml#/components/schemas/access. Having those declarations under components.schemas will make type generation work properly.

@mkurapov mkurapov merged commit fd658a0 into main Oct 24, 2022
@mkurapov mkurapov deleted the 663/mk/op-client-1 branch October 24, 2022 13:40
omertoast pushed a commit that referenced this pull request Oct 28, 2022
* feat(open-payments-client): initial POC of open-payments-client

* feat(open-payments-client): cleanup

* feat(open-payments): remove open-payments-client folder

* feat(open-payments): make open-payments folder and use script for type generation

* feat(open-payments): fix rootDir

* feat(open-payments): downgrading generation lib, updating how script is ran

* feat(open-payments): update default config usage

* feat(open-payments): update pnpm-lock.yaml

* feat(open-payments): update pnpm-lock.yaml

* feat(open-payments): adding tests

* feat(open-payments): update pnpm-lock.yaml

* feat(open-payments): simplify test

* feat(open-payments): updating workflows

* feat(open-payments): pin the open api spec to the most recent commit

* feat(open-payments): use updated RS spec

* feat(open-payments): building open-api package during workflow

* Revert "feat(open-payments): building open-api package during workflow"

This reverts commit e11ec65.
@wilsonianb wilsonianb mentioned this pull request Jan 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: ci Changes to the CI type: tests Testing related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants