Skip to content

feat: new kms api example module branch #6119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 13 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,4 @@ modules/**/dist/
modules/**/pack-scoped/
coverage
/.direnv/
*.db
45 changes: 45 additions & 0 deletions modules/express-kms-api-example/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# KMS API example (REST API)

Based on TDD specification [On-Prem Wallets(https://docs.google.com/document/d/1ku2agwirV3tHCJF350VF_uaVx73D6vu7yUBaDp-cxL0/edit?tab=t.0#heading=h.165ukudv7ejt)]

Made with ExpressJS, Typescript and sqlite3.

# Installation steps / setup

1 - Clone the BitGoJS repo and navigate in the terminal until you reach this folder (express-kms-api-example)
2 - Set the current node version with node version manager: $nvm use
3 - Install all the packages: $npm install
4 - Create a .env file in the root of the project (at the side of package.json)
5 - Edit the .env file and set this couple of variables:

USER_PROVIDER_CLASS="aws"
BACKUP_PROVIDER_CLASS="aws"

you could use different class names between user provider and backup provider or the same, it depends on your particular setup.
Important notes: in the /providers folder, you may be able to add your providers (aws, azure, mock, custom, etc), for adding a new custom provider just follow the structure of the AWS one, starting from the root folder:

providers/
|
|-aws/
|
|--aws-kms.ts

the "aws" part on "aws-kms" filename is the same as the USER_PROVIDER_CLASS or BACKUP_PROVIDER_CLASS

In the case of aws, in aws-kms.ts you may find the AwsKmsProvider class declared inside the file.
For your custom provider, suppose that the provider is called "custom", you may need this folder structure:

providers/
|
|-custom/
|
|--custom-kms.ts

Inside custom-kms.ts ==> class CustomKmsProvider (implements the common interface)

6 - Implement your custom providers if necessary or use the aws/azure implementation included
7 - Run the project(test): $npm run dev

Extras:

- If you want to use the mock server, instead of "aws" replace with "mock" on step 5
39 changes: 39 additions & 0 deletions modules/express-kms-api-example/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{
"name": "examplekmsapi",
"version": "1.0.0",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"dev": "npx tsx --watch src/app.ts",
"dev-test": "npx tsx --watch src/index-test.ts",
"build": "tsc -p .",
"run": "node dist/src/app.js",
"build-and-run": "tsc -p .; node dist/src/app.js"
},
"keywords": [],
"author": "",
"license": "ISC",
"description": "",
"dependencies": {
"@aws-sdk/client-kms": "^3.810.0",
"@aws-sdk/credential-providers": "^3.817.0",
"@azure/identity": "^4.10.0",
"@azure/keyvault-keys": "^4.9.0",
"body-parser": "^2.2.0",
"express": "^5.1.0",
"sqlite3": "^5.1.7",
"swagger-jsdoc": "^6.2.8",
"swagger-ui-express": "^5.0.1",
"zod": "^3.24.4"
},
"devDependencies": {
"@tsconfig/node22": "^22.0.1",
"@types/express": "^5.0.1",
"@types/express-serve-static-core": "^5.0.6",
"@types/node": "^22.15.17",
"@types/swagger-jsdoc": "^6.0.4",
"@types/swagger-ui-express": "^4.1.8",
"tsx": "^4.19.4",
"typescript": "^5.8.3"
}
}
13 changes: 13 additions & 0 deletions modules/express-kms-api-example/scripts/create-tables.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
-- TODO: delete this comments! questions for Pranav/Mohammad
-- Some fields with VARCHAR(X) not sure how many characters
-- do we need for a coin.

-- I took the fields from the TDD doc on page 3
-- Not sure also about the prv and pub key length, should we limit them?
CREATE TABLE PRIVATE_KEYS(
id TEXT PRIMARY KEY,
coin VARCHAR(30) NOT NULL,
source VARCHAR(15) CHECK(source IN ('user', 'backup')) NOT NULL,
type VARCHAR(15) CHECK(type IN ('independent', 'tss')) NOT NULL,
prv TEXT NOT NULL,
pub TEXT NOT NULL);
102 changes: 102 additions & 0 deletions modules/express-kms-api-example/src/api/handlers/GET.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
import { NextFunction, Request, Response } from 'express';
import db from '../../db';

type GetParamsType = {
pub: string;
};

/**
* @openapi
* /key/{pub}:
* get:
* summary: Retrieve a private key stored
* tags:
* - key management service
* parameters:
* - in: path
* name: pub
* required: true
* schema:
* type: string
* description: Public key related to the priv key to retrieve
* - in: query
* name: source
* required: true
* schema:
* type: string
* enum:
* - user
* - backup
* description: The kind of key to retrieve
* responses:
* 200:
* description: Private key retrieved
* content:
* application/json:
* schema:
* type: object
* required:
* - prv
* - pub
* - coin
* - source
* - type
* properties:
* prv:
* type: string
* pub:
* type: string
* source:
* type: string
* enum:
* - user
* - backup
* type:
* type: string
* enum:
* - user
* - backup
* example:
* prv: "MIICXAIBAAKBgH3D4WKfdvhhj9TSGrI0FxAmdfiyfOphuM/kmLMIMKdahZLE5b8YoPL5oIE5NT+157iyQptb7q7qY9nA1jw86Br79FIsi6hLOuAne+1u4jVyJi4PLFAK5gM0c9klGjiunJ+OSH7fX+HQDwykZm20bdEa2fRU4dqT/sRm4Ta1iwAfAgMBAAEC"
* pub: "MIGeMA0GCSqGSIb3DQEBAQUAA4GMADCBiAKBgH3D4WKfdvhhj9TSGrI0FxAmdfiyfOphuM/kmLMIMKdahZLE5b8YoPL5oIE5NT+157iyQptb7q7qY9nA1jw86Br79FIsi6hLOuAne+1u4jVyJi4PLFAU4dqT/sRm4Ta1iwAfAgMBAAE="
* source: "user"
* type: "independent"
* 404:
* description: Private key not found
* 500:
* description: Internal server error
*/
export async function GET(req: Request<GetParamsType>, res: Response, next: NextFunction): Promise<void> {
//TODO: fix type, it says that the prop doesn't exists
// but in fact it's a incorect type declaration
const userKeyProvider = req.body.userKeyProvider;

const { pub } = req.params;

// fetch from DB
const source = req.query.source;
const data = await db.fetchOne('SELECT encryptedPrv, kmsKey, type FROM PRIVATE_KEY WHERE pub = ? AND source = ?', [
pub,
source,
]);

if (!data) {
res.status(404);
res.send({ message: `Not Found` });
return;
}

const { encryptedPrv, kmsKey, type } = data;

const kmsRes = await userKeyProvider.getKey(kmsKey, encryptedPrv, {});
if ('code' in kmsRes) {
res.status(500);
res.send({ message: 'Internal server error' });
return;
}
const { prv } = kmsRes;

res.status(200);
res.json({ prv, pub, source, type });
next();
}
148 changes: 148 additions & 0 deletions modules/express-kms-api-example/src/api/handlers/POST.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
import { NextFunction, Request, Response } from 'express';
import db from '../../db';
import { KmsErrorRes, PostKeyKmsRes } from '../../providers/kms-interface/kmsInterface';
import { ZodPostKeySchema } from '../schemas/postKeySchema';

/**
* @openapi
* /key:
* post:
* summary: Store a new private key
* tags:
* - key management service
* requestBody:
* required: true
* content:
* application/json:
* schema:
* type: object
* required:
* - prv
* - pub
* - coin
* - source
* - type
* properties:
* prv:
* type: string
* pub:
* type: string
* coin:
* type: string
* source:
* type: string
* enum:
* - user
* - backup
* type:
* type: string
* enum:
* - independent
* - mpc
* responses:
* 200:
* description: Successfully stored key
* content:
* application/json:
* schema:
* type: object
* required:
* - prv
* - pub
* - coin
* - source
* - type
* properties:
* keyId:
* type: string
* coin:
* type: string
* source:
* type: string
* enum:
* - user
* - backup
* type:
* type: string
* enum:
* - independent
* - mpc
* pub:
* type: string
* example:
* keyId: "MIICXAIBAAKBgH3D4WKfdvhhj9TSGrI0FxAmdfiyfOphuM/kmLMIMKdahZLE5b8YoPL5oIE5NT+157iyQptb7q7qY9nA1jw86Br79FIsi6hLOuAne+1u4jVyJi4PLFAK5gM0c9klGjiunJ+OSH7fX+HQDwykZm20bdEa2fRU4dqT/sRm4Ta1iwAfAgMBAAEC"
* coin: "sol"
* source: "user"
* type: "tss"
* pub: "MIGeMA0GCSqGSIb3DQEBAQUAA4GMADCBiAKBgH3D4WKfdvhhj9TSGrI0FxAmdfiyfOphuM/kmLMIMKdahZLE5b8YoPL5oIE5NT+157iyQptb7q7qY9nA1jw86Br79FIsi6hLOuAne+1u4jVyJi4PLFAU4dqT/sRm4Ta1iwAfAgMBAAE="
* 400:
* description: Invalid data
* 409:
* description: Duplicate key
* 500:
* description: Internal server error
*/
export async function POST(req: Request, res: Response, next: NextFunction): Promise<void> {

// parse request
try {
console.log('POST /key', req.body);

Check warning

Code scanning / CodeQL

Log injection Medium

Log entry depends on a
user-provided value
.

Copilot Autofix

AI 2 days ago

To fix the issue, we need to sanitize the user-provided input (req.body) before logging it. Specifically:

  1. Remove any newline (\n) or carriage return (\r) characters from the input to prevent log injection.
  2. Clearly mark the user input in the log entry to distinguish it from other log data.

This can be achieved by using JSON.stringify to serialize the req.body object and then replacing newline and carriage return characters with an empty string. This ensures that the logged data is safe and does not introduce unintended log entries.


Suggested changeset 1
modules/express-kms-api-example/src/api/handlers/POST.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/modules/express-kms-api-example/src/api/handlers/POST.ts b/modules/express-kms-api-example/src/api/handlers/POST.ts
--- a/modules/express-kms-api-example/src/api/handlers/POST.ts
+++ b/modules/express-kms-api-example/src/api/handlers/POST.ts
@@ -88,3 +88,4 @@
   try {
-    console.log('POST /key', req.body);
+    const sanitizedBody = JSON.stringify(req.body).replace(/[\n\r]/g, '');
+    console.log('POST /key', sanitizedBody);
     ZodPostKeySchema.parse(req.body);
EOF
@@ -88,3 +88,4 @@
try {
console.log('POST /key', req.body);
const sanitizedBody = JSON.stringify(req.body).replace(/[\n\r]/g, '');
console.log('POST /key', sanitizedBody);
ZodPostKeySchema.parse(req.body);
Copilot is powered by AI and may make mistakes. Always verify output.
ZodPostKeySchema.parse(req.body);
} catch (e) {
res.status(400);
res.send({ message: 'Invalid data provided from client' });
return;
}

const { prv, pub, coin, source, type, userKeyProvider, backupKeyProvider } = req.body;

Check notice

Code scanning / CodeQL

Unused variable, import, function or class Note

Unused variable backupKeyProvider.

Copilot Autofix

AI 2 days ago

To fix the issue, we should remove the unused variable backupKeyProvider from the destructuring assignment on line 97. This will eliminate the unnecessary variable and improve code clarity and maintainability. No other changes are required, as the removal of this variable does not affect the functionality of the code.


Suggested changeset 1
modules/express-kms-api-example/src/api/handlers/POST.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/modules/express-kms-api-example/src/api/handlers/POST.ts b/modules/express-kms-api-example/src/api/handlers/POST.ts
--- a/modules/express-kms-api-example/src/api/handlers/POST.ts
+++ b/modules/express-kms-api-example/src/api/handlers/POST.ts
@@ -96,3 +96,3 @@
 
-  const { prv, pub, coin, source, type, userKeyProvider, backupKeyProvider } = req.body;
+  const { prv, pub, coin, source, type, userKeyProvider } = req.body;
 
EOF
@@ -96,3 +96,3 @@

const { prv, pub, coin, source, type, userKeyProvider, backupKeyProvider } = req.body;
const { prv, pub, coin, source, type, userKeyProvider } = req.body;

Copilot is powered by AI and may make mistakes. Always verify output.

// check for duplicates
const keyObject = await db.fetchAll('SELECT * from PRIVATE_KEYS WHERE pub = ? AND source = ?', [pub, source]);
if (keyObject.length !== 0) {
res.status(409);
res.send({ message: `Error: Duplicated Key for source: ${source} and pub: ${pub}` });
return;
}

// db script to fetch kms key from the database, if any exist
let kmsKey = await db.fetchOne('SELECT kmsKey from PRIVATE_KEYS WHERE provider = ? LIMIT 1', ['mock']);
if (!kmsKey) {
const kmsRes = await userKeyProvider.createKmsKey({});
if ('code' in kmsRes) {
res.status(kmsRes.code);
res.send({ message: 'Internal server error. Failed to create top-level kms key in KMS' });
return;
}
kmsKey = kmsRes.kmsKey;
}

// send to kms
const kmsRes: PostKeyKmsRes | KmsErrorRes = await userKeyProvider.postKey(kmsKey, prv, {});
if ('code' in kmsRes) {
res.status(kmsRes.code);
res.send({ message: 'Internal server error. Failed to encrypt private key in KMS' });
return;
}

// insert into database
// TODO: better catching
await db
.run('INSERT INTO PRIVATE_KEYS values (?, ?, ?, ?, ?, ?, ?)', [
pub,
source,
kmsRes.encryptedPrv,
userKeyProvider.providerName,
kmsRes.topLevelKeyId,
coin,
type,
])
.catch((err) => {
res.status(500);
res.send({ message: 'Internal server error' });
return; // TODO: test this
});

res.status(200);
res.json({ coin, source, type, pub });
next();
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { z } from 'zod';
import { KeySource, KeyType, MultiSigCoins } from './types';

export const ZodPostKeySchema = z.object({
prv: z.string(), // TODO: min/max length?
pub: z.string(), // TODO: min/max length?
coin: z.enum(MultiSigCoins),
source: z.enum(KeySource),
type: z.enum(KeyType),
userKeyProvider: z.any(), // TODO: move this away from schema
backupKeyProvider: z.any(),
});
8 changes: 8 additions & 0 deletions modules/express-kms-api-example/src/api/schemas/types.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
// TODO: add the full list of supported coins
export const MultiSigCoins = ['btc', 'heth'] as const;
export const KeySource = ['user', 'backup'] as const;
export const KeyType = ['independent', 'tss'] as const;

export type MultiSigCoinsType = (typeof MultiSigCoins)[number];
export type KeySourceType = (typeof KeySource)[number];
export type KeyTypeType = (typeof KeyType)[number];
Loading
Loading