Pid 2645 implement aws secret storage provider #848

martinsaporiti · 2024-11-19T13:04:34Z

No description provided.

…ub.com:0xPolygonID/issuer-node into PID-2645-implement-aws-secret-storage-provider

gitguardian · 2024-11-19T13:04:40Z

⚠️ GitGuardian has uncovered 5 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
14295020	Triggered	Generic High Entropy Secret	`129144c`	internal/kms/aws_secret_storage_provider_test.go	View secret
8664826	Triggered	Generic Password	`004c9a9`	k8s/helm/values.yaml	View secret
14295020	Triggered	Generic High Entropy Secret	`129144c`	internal/kms/aws_secret_storage_provider_test.go	View secret
14295020	Triggered	Generic High Entropy Secret	`129144c`	internal/kms/aws_secret_storage_provider_test.go	View secret
14295020	Triggered	Generic High Entropy Secret	`bc715b3`	internal/kms/aws_secret_storage_provider_test.go	View secret

🛠 Guidelines to remediate hardcoded secrets

Understand the implications of revoking this secret by investigating where it is used in your code.
Replace and store your secrets safely. Learn here the best practices.
Revoke and rotate these secrets.
If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

following these best practices for managing and storing secrets including API keys and other credentials
install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

…vider

OBrezhniev · 2024-11-20T22:59:03Z

cmd/kms_priv_key_importer/main.go

-	if issuerKMSEthPluginVar != config.LocalStorage && issuerKMSEthPluginVar != config.Vault && issuerKMSEthPluginVar != config.AWS {
-		log.Error(ctx, "issuer kms eth provider is not set or is not localstorage or vault or aws", "plugin: ", issuerKMSEthPluginVar)
+	if issuerKMSETHProviderToUse != config.LocalStorage && issuerKMSETHProviderToUse != config.Vault && issuerKMSETHProviderToUse != config.AWS {
+		log.Error(ctx, "issuer kms eth provider is not set or is not localstorage or vault or aws", "plugin: ", issuerKMSETHProviderToUse)


This error message is messy. Cleaner one would be issuer kms eth provider is invalid, supported values are: localstorage, vault, aws

OBrezhniev · 2024-11-20T23:07:34Z

internal/kms/aws_secret_storage_provider.go

+)
+
+const (
+	localstackSecretManagerEndpoint = "http://localhost:4566"


It should not be a constant, should be configurable through env vars or config param.

It's default value (coming from the env var / config) should not be localhost, but container name (e.g. http://localstack:4566)

Name should not include "localstack" because user may switch to something else, better use "local" as in the name of key storage config.

ok

ok

I prefer URL for the name of the variable. So AWS config will be:

ISSUER_KMS_AWS_ACCESS_KEY=<aws_access_key> ISSUER_KMS_AWS_SECRET_KEY=<aws_secret_key> ISSUER_KMS_AWS_REGION=<aws_region> # if this value is `local` then `ISSUER_KMS_AWS_URL` is used ISSUER_KMS_AWS_URL=<aws_url> # aws secret store URL

Note: ISSUER_KMS_AWS_REGION=local is for running tests.

OBrezhniev · 2024-11-20T23:09:29Z

infrastructure/local/docker-compose-infra.yml

+  localstack:
+    image: localstack/localstack:latest
+    ports:
+      - "4566:4566"


Don't do port forwarding. Please remove the same for other containers (in a separate PR if too many changes are needed).

We need port forwarding because localstack is used for running tests and those tests are outside docker

OBrezhniev · 2024-11-21T00:30:01Z

internal/kms/file_storage_provider.go

+}
+
+// NewLocalStorageFileProvider - creates new local storage file manager
+func NewLocalStorageFileProvider(file string) StorageManager {


Better rename local storage to file storage everywhere (including in config params and file name, and deprecate local as a storage option, but still support it), so that here we would have NewFileStorageManager (because it returns StorageManager, not StoragePrivider).

@demonsh What do you think about introducing new kms config option file and deprecating local? I think it should be easier to understand what it is and how it works.

OBrezhniev · 2024-11-21T00:30:41Z

internal/kms/file_storage_provider.go

+	return &localStorageFileProvider{file}
+}
+
+func (ls *localStorageFileProvider) SaveKeyMaterial(ctx context.Context, keyMaterial map[string]string, id string) error {


*fileStorageManager

OBrezhniev · 2024-11-21T00:52:30Z

internal/kms/local_bjj_key_provider.go

-	PrivateKey string `json:"private_key"`
+type localBJJKeyProvider struct {
+	keyType          KeyType
+	reIdenKeyPathHex *regexp.Regexp // RE of key path bounded to identity


bound, not bounded

OBrezhniev · 2024-11-21T00:53:26Z

internal/kms/local_bjj_key_provider.go

+type localBJJKeyProvider struct {
+	keyType          KeyType
+	reIdenKeyPathHex *regexp.Regexp // RE of key path bounded to identity
+	reAnonKeyPathHex *regexp.Regexp // RE of key path not bounded to identity


OBrezhniev · 2024-11-21T00:56:21Z

internal/kms/local_bjj_key_provider.go

-	KeyType    string `json:"key_type"`
-	KeyPath    string `json:"key_path"`
-	PrivateKey string `json:"private_key"`
+type localBJJKeyProvider struct {


fileBJJKeyProvider

local means that the issuer node handles the creation of the keys unlike Vault or AWS KMS.
localBJJKeyProvider uses a storageManager to store the keys, which can be a text file or an AWS secret manager.

OBrezhniev · 2024-11-21T00:56:40Z

internal/kms/local_eth_key_provider.go

-	keyType                 KeyType
-	reIdenKeyPathHex        *regexp.Regexp // RE of key path bounded to identity
-	localStorageFileManager LocalStorageFileManager
+type localEthKeyProvider struct {


fileEthKeyProvider

local means that the issuer node handles the creation of the keys unlike Vault or AWS KMS.
localETHKeyProvider uses a storageManager to store the keys, which can be a text file or an AWS secret manager.

OBrezhniev · 2024-11-21T01:00:11Z

internal/kms/utils_test.go

+)
+
+func Test_getKeyID(t *testing.T) {
+	identity, err := w3c.ParseDID("did:polygonid:polygon:amoy:2qQ68JkRcf3ySMcjPtGKJhGgu7jh4zzhrz6UxpXwPw")


Please use did:iden3

martinsaporiti added 10 commits October 18, 2024 07:01

chore: add secret manager

fbdcbd8

chore: improve code and add tests

129144c

chore: merged with develop

004c9a9

chore: add localstack

b177c27

fix: dependencies

4dda2fc

chore: add support for multiples keys

bc715b3

chore: change secret name

b1e1281

fix: revert to Iden3commRevocationStatusV1.0

af64c32

chore: remove constant

6d4f3c2

Merge branch 'PID-2645-implement-aws-secret-storage-provider' of gith…

556c132

…ub.com:0xPolygonID/issuer-node into PID-2645-implement-aws-secret-storage-provider

martinsaporiti requested a review from a team as a code owner November 19, 2024 13:04

martinsaporiti self-assigned this Nov 19, 2024

Merge branch 'develop' into PID-2645-implement-aws-secret-storage-pro…

c2b6c3b

…vider

OBrezhniev requested changes Nov 21, 2024

View reviewed changes

chore: improvements after code review

6636892

martinsaporiti requested a review from OBrezhniev November 21, 2024 15:11

martinsaporiti added 5 commits November 22, 2024 14:45

chore: add aws kms option

edb3ca5

chore: add tests and improve code

2219567

chore: add more tests

0652a1f

chore: remove comments

8723ea9

fix: log

ebac8ae

OBrezhniev approved these changes Nov 27, 2024

View reviewed changes

demonsh self-requested a review November 27, 2024 18:13

demonsh approved these changes Nov 27, 2024

View reviewed changes

martinsaporiti merged commit cc52be0 into develop Dec 2, 2024
3 checks passed

martinsaporiti deleted the PID-2645-implement-aws-secret-storage-provider branch December 2, 2024 09:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pid 2645 implement aws secret storage provider #848

Pid 2645 implement aws secret storage provider #848

martinsaporiti commented Nov 19, 2024

gitguardian bot commented Nov 19, 2024 •

edited

Loading

OBrezhniev Nov 20, 2024

OBrezhniev Nov 20, 2024

martinsaporiti Nov 21, 2024

OBrezhniev Nov 20, 2024

martinsaporiti Nov 21, 2024

OBrezhniev Nov 21, 2024

OBrezhniev Nov 21, 2024

OBrezhniev Nov 21, 2024

OBrezhniev Nov 21, 2024

OBrezhniev Nov 21, 2024

OBrezhniev Nov 21, 2024

martinsaporiti Nov 26, 2024

OBrezhniev Nov 21, 2024

martinsaporiti Nov 26, 2024

OBrezhniev Nov 21, 2024

Pid 2645 implement aws secret storage provider #848

Pid 2645 implement aws secret storage provider #848

Conversation

martinsaporiti commented Nov 19, 2024

gitguardian bot commented Nov 19, 2024 • edited Loading

⚠️ GitGuardian has uncovered 5 secrets following the scan of your pull request.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gitguardian bot commented Nov 19, 2024 •

edited

Loading