feat(security): Implementation to set up Consul ACL #3215

jim-wang-intel · 2021-03-02T23:33:01Z

This PR is part of Phase 1 to secure Consul. Main feature added is to enable Consul's ACL and bootstrap ACL. The following are detailed changes:

Add Consul's ACL configuration file with "allow" policy
Disable Consul's DNS port
Add ACL related configuration toml on security-bootstrapper
Add setupRegistryACL subcommand to security-bootstrapper
Add checking and retry logic for "non-empty" Consul leader
Add implementation for setting up Consul's ACL, including bootstrap ACL and configure Consul secrets access for Vault

Closes: #3156

Signed-off-by: Jim Wang yutsung.jim.wang@intel.com

PR Checklist

Please check if your PR fulfills the following requirements:

[x ] Tests for the changes have been added (for bug fixes / features)
Docs have been added / updated (for bug fixes / features)

If your build fails due to your commit message not passing the build checks, please review the guidelines here: https://github.com/edgexfoundry/edgex-go/blob/master/.github/Contributing.md.

What is the current behavior?

No ACL of Consul as of today.

Issue Number: #3156

What is the new behavior?

Start up Consul with ACL enabled with "allow" policy and also bootstrap its ACL for the first time use. Once bootstrapping is successfully, we will use Vault mgmt token to configure consul secret access.

Does this PR introduce a breaking change?

Yes
[x ] No

New Imports

Yes
[x ] No

Specific Instructions

Are there any specific instructions or things that should be known prior to reviewing?
This implements the majority of consul bootstrap process in golang code and can be running in docker-compose. The implementation for snap would be in a separate PR.

Other information

To verify this locally, one needs to git clone this PR and then add environment override ENABLE_REGISTRY_ACL=true into both secretstore-setup and consul service of docker-compose file:

   environment:
      ENABLE_REGISTRY_ACL: "true"

Also make sure to make the volume of secrets to have written permission in the Consul container:

volumes:
      - edgex-init:/edgex-init:ro,z
      - /tmp/edgex/secrets/edgex-consul:/tmp/edgex/secrets/edgex-consul:z

as the generated Consul will be written into that secrets volume.

internal/security/bootstrapper/command/setupacl/aclbootstrap.go

internal/security/bootstrapper/command/setupacl/command.go

internal/security/bootstrapper/interfaces/command.go

internal/security/bootstrapper/command/setupacl/command.go

jim-wang-intel · 2021-03-04T00:05:47Z

what cleans up the file when the is a restart of all the services?

no need to clean up even when restarting all services since Consul's ACL is in a good bootstrapped ACL state so no need to re-run the setup registry ACL again. Unless you want to clean out all the volumes.

lenny-goodell · 2021-03-04T00:09:00Z

Unless you want to clean out all the volumes.

Right, so I see an issue here when volumes are cleared, but sentinel file is left behind. Is there a Consul API that will tell us that it is setup already or not?

jim-wang-intel · 2021-03-04T00:11:45Z

Unless you want to clean out all the volumes.

Right, so I see an issue here when volumes are cleared, but sentinel file is left behind. Is there a Consul API that will tell us that it is setup already or not?

the sentinel file is on the volume. I am NOT aware of any Consul API can tell us the ACL bootstrap status.

lenny-goodell · 2021-03-04T00:34:21Z

the sentinel file is on the volume

ohhh! Didn't realize that. ;-) Then I am good!

lenny-goodell

LGTM, I'll concede on the 5 sec wait... ;-)

jim-wang-intel · 2021-03-04T01:13:55Z

LGTM, I'll concede on the 5 sec wait... ;-)

Thanks, it rarely happens and I've put the issue link notes. Since last time I saw, I've never seen it happened again on my box afterwards. So I can not even re-produce, and my guess is probably last time i've had some left-over state for some reason in consul as Consul is stateful machine.

jim-wang-intel · 2021-03-04T15:46:32Z

recheck

cmd/security-bootstrapper/entrypoint-scripts/consul_wait_install.sh

cmd/security-bootstrapper/res/configuration.toml

internal/security/bootstrapper/command/setupacl/aclbootstrap.go

internal/security/bootstrapper/command/setupacl/command.go

jim-wang-intel · 2021-03-04T16:59:29Z

Hi @ernestojeda , I am starting seeing the snap pulling core18 from internet error in the CI pipeline and it happens 2-in-a-row now, could you please look into that?

ernestojeda · 2021-03-04T17:37:04Z

Not much I can do other than re-run the job. I will open a ticket with the LF to see if they can diagnose any network issue.

jim-wang-intel · 2021-03-04T17:38:31Z

Not much I can do other than re-run the job. I will open a ticket with the LF to see if they can diagnose any network issue.

Ok, thanks for the information.

Mainly feature added is to enable Consul's ACL and bootstrap ACL. The following are detailed changes: - Add Consul's ACL configuration file with "allow" policy - Disable Consul's DNS port - Add ACL related configuration toml on security-bootstrapper - Add setupRegistryACL subcommand to security-bootstrapper - Add checking and retry logic for "non-empty" Consul leader - Add implementation for setting up Consul's ACL, including bootstrap ACL and configure Consul secrets access for Vault Closes: edgexfoundry#3156 Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>

Added the acl done sentinel file once Consul's registry had been setup successfully once This is to prevent the error if it is re-run second time or later. Like 2nd time compose-up or re-run the consul service in the snap. Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>

Address Lenny's PR comments and feedback Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>

Remove unused waitGroup Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>

- add Sentinel filepath configuration in toml Address Bryon's PR feedback/comments Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>

sonarqubecloud · 2021-03-04T21:58:56Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.6% Duplication

lenny-goodell

LGTM

jim-wang-intel added enhancement New feature or request security-services 3-high priority denoting release-blocking issues ireland labels Mar 2, 2021

jim-wang-intel added this to the Ireland milestone Mar 2, 2021

jim-wang-intel requested review from beaufrusetta, hutchic, tonyespy, cloudxxx8, bnevis-i and lenny-goodell March 2, 2021 23:33

jim-wang-intel self-assigned this Mar 2, 2021

lenny-goodell requested changes Mar 3, 2021

View reviewed changes

internal/security/bootstrapper/command/setupacl/command.go Outdated Show resolved Hide resolved

jim-wang-intel force-pushed the bootstrap-consul-acl-config-access branch from 0b77750 to 2c33bc0 Compare March 3, 2021 23:15

jim-wang-intel requested a review from lenny-goodell March 4, 2021 00:14

lenny-goodell previously approved these changes Mar 4, 2021

View reviewed changes

jim-wang-intel dismissed lenny-goodell’s stale review via 88047cb March 4, 2021 14:58

bnevis-i reviewed Mar 4, 2021

View reviewed changes

jim-wang-intel added 2 commits March 4, 2021 14:58

jim-wang-intel added 3 commits March 4, 2021 14:58

fix: Address PR feedback and comments

b8970f0

Address Lenny's PR comments and feedback Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>

fix: code clean up

7c9b397

Remove unused waitGroup Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>

feat(security): Add sentinel file config and also address PR feedback

191f7d1

- add Sentinel filepath configuration in toml Address Bryon's PR feedback/comments Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>

jim-wang-intel force-pushed the bootstrap-consul-acl-config-access branch from 88047cb to 191f7d1 Compare March 4, 2021 21:58

jim-wang-intel requested review from bnevis-i and lenny-goodell March 4, 2021 21:58

bnevis-i approved these changes Mar 4, 2021

View reviewed changes

lenny-goodell approved these changes Mar 4, 2021

View reviewed changes

jim-wang-intel merged commit 8a56253 into edgexfoundry:master Mar 4, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(security): Implementation to set up Consul ACL #3215

feat(security): Implementation to set up Consul ACL #3215

jim-wang-intel commented Mar 2, 2021

jim-wang-intel commented Mar 4, 2021

lenny-goodell commented Mar 4, 2021

jim-wang-intel commented Mar 4, 2021 •

edited

Loading

lenny-goodell commented Mar 4, 2021

lenny-goodell left a comment

jim-wang-intel commented Mar 4, 2021

jim-wang-intel commented Mar 4, 2021

jim-wang-intel commented Mar 4, 2021

ernestojeda commented Mar 4, 2021

jim-wang-intel commented Mar 4, 2021

sonarqubecloud bot commented Mar 4, 2021

lenny-goodell left a comment

feat(security): Implementation to set up Consul ACL #3215

feat(security): Implementation to set up Consul ACL #3215

Conversation

jim-wang-intel commented Mar 2, 2021

PR Checklist

What is the current behavior?

Issue Number: #3156

What is the new behavior?

Does this PR introduce a breaking change?

New Imports

Specific Instructions

Other information

jim-wang-intel commented Mar 4, 2021

lenny-goodell commented Mar 4, 2021

jim-wang-intel commented Mar 4, 2021 • edited Loading

lenny-goodell commented Mar 4, 2021

lenny-goodell left a comment

Choose a reason for hiding this comment

jim-wang-intel commented Mar 4, 2021

jim-wang-intel commented Mar 4, 2021

jim-wang-intel commented Mar 4, 2021

ernestojeda commented Mar 4, 2021

jim-wang-intel commented Mar 4, 2021

sonarqubecloud bot commented Mar 4, 2021

lenny-goodell left a comment

Choose a reason for hiding this comment

jim-wang-intel commented Mar 4, 2021 •

edited

Loading