[full-ci] Introduce TLS Settings for go-micro based grpc services and clients

[rhafer]

TLS for the services can be configure by setting the OCIS_GRPC_TLS_ENABLED" "OCIS_GRPC_TLS_CERTIFICATE" and "OCIS_GRPC_TLS_KEY" enviroment variables.

TLS for the clients can configured by setting the "OCIS_GRPC_CLIENT_TLS_MODE" and "OCIS_GRPC_CLIENT_TLS_CACERT" variables.

By default TLS is disabled.

[mmattel]

I see a problem there, not with the implementation or the env description texts.

The problem is, that none of the envs starting with OCIS_ will be part of the documentation because they are not assigned to a particular services (global scope) and currently not automatically catched by the process.

This impacts that we have important envs to configure but nobody knows...

Referncing #3917 (Harmonize env variable naming for those which are not bound to a particular service) @dragonchaser fyi

[rhafer]

The problem is, that none of the envs starting with OCIS_ will be part of the documentation because they are not assigned to a particular services (global scope) and currently not automatically catched by the process.

This is not what is happening here. Actually the env vars are assigned to specific services. Even if the struct are defined on a "global" level (mainly to avoid to much duplication and copy and pasting the same thing all over the code base, also because currently there is no need to override these settings on a per service base). Each service that needs those settings instantiates that struct in it's own config struct. And the documentation will get properly generated. Exactly the same thing was done in: #4798 (e.g. for the REVA_GATEWAY_TLS_MODE and REVA_GATEWAY_TLS_CACERT settings) and the documentation is properly generated for those. See https://owncloud.dev/services/users/configuration/

[butonic]

I'm not sure about the verious places where you configure the GRPC default client:

err = ogrpc.Configure(ogrpc.GetClientOptions(cfg.GRPCClientTLS)...)

That only works because GRPCClientTLS is a *shared.GRPCClientTLS that is filled by EnsureDefaults(), only has a single OCIS_GRPC_CLIENT_TLS_* set of env vars and no service specific config.

While it looks fragile to me and I think we may need to add service specific certificates in the future, this is a great step in the right direction.

[butonic]

I set

					// grpc server config
					"OCIS_GRPC_TLS_ENABLED": "on",
					"OCIS_GRPC_TLS_CERTIFICATE": "/home/vscode/.ocis/idm/ldap.crt",
					"OCIS_GRPC_TLS_KEY":  "/home/vscode/.ocis/idm/ldap.key",

					// grpc client config
					"OCIS_GRPC_CLIENT_TLS_MODE": "insecure",

now getting:

2022-11-02T12:28:03Z ERR error starting the grpc server error="unable to register services: rgrpc: grpc service publicshareprovider could not be started,: rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: first record does not look like a TLS handshake\"" service=sharing

🤔

[butonic]

my bad it has to be "OCIS_GRPC_TLS_ENABLED": "true", and it works nicely!

[butonic]

looks like the PR needs a rebase on / merge of master as some of the expected test lines look shifted.

[rhafer]

That only works because GRPCClientTLS is a *shared.GRPCClientTLS that is filled by EnsureDefaults(), only has a single OCIS_GRPC_CLIENT_TLS_* set of env vars and no service specific config.

Yes. And I did that on purpose. Mainly to avoid spreading even more code duplication across all the services' pkg/config.

While it looks fragile to me and I think we may need to add service specific certificates in the future, this is a great step in the right direction.

Two questions:

1. Why do you think it's fragile?
2. In which case to you think we need per service certificates? I am having a hard time coming up with a setup where it would really be needed to have per service env vars for configuring the certificates. When starting separate ocis instance per service you can always set "OCIS_GRPC_TLS_CERTIFICATE to a different value per ocis instance if really needed. If you run a single ocis instance, why would you need separate certificates? And if you really want that, you can even configure it in the config file (each service's configuration section can have it's own tls configuration).

[rhafer]

I set

					// grpc server config
					"OCIS_GRPC_TLS_ENABLED": "on",
					"`": "/home/vscode/.ocis/idm/ldap.crt",
					"OCIS_GRPC_TLS_KEY":  "/home/vscode/.ocis/idm/ldap.key",

					// grpc client config
					"OCIS_GRPC_CLIENT_TLS_MODE": "insecure",

BTW, you can even leave OCIS_GRPC_TLS_CERTIFICATE and OCIS_GRPC_TLS_KEY empty. It will just generate (insecure) certificates in-memory (at every start of ocis).

[mmattel]

you can even leave OCIS_GRPC_TLS_CERTIFICATE and OCIS_GRPC_TLS_KEY empty. It will just generate (insecure) certificates in-memory (at every start of ocis).

A sentence could be added to the description reflecting this behaviour.

[rhafer]

you can even leave OCIS_GRPC_TLS_CERTIFICATE and OCIS_GRPC_TLS_KEY empty. It will just generate (insecure) certificates in-memory (at every start of ocis).

A sentence could be added to the description reflecting this behaviour.

It's there: https://github.com/owncloud/ocis/pull/4901/files#diff-bc4f34bf14cfb981d81d2f12ed8f70c7c34023813282c8a82d4bc54c3938de4bR44 😄

[mmattel]

Added some commitable suggestions, from a docs pov, looks fine to me though I would be happy to have an answer to #4901 (comment) for a general improvment of that env descriptions - which can be fixed post merging this PR but should be considered.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
16 Code Smells

0.0% Coverage
44.6% Duplication

[butonic]

• Why do you think it's fragile?

Reusing a shared config requires initializing it with EnsureDefaults ... I'm just beginning to hold a grudge against all the boilerplate, which I consider fragile when trying to create a new service. At the moment I think it is still nearld impossible to add a service and get the config and env var parsing right ... not really a critique af this PR, I admit.

• In which case to you think we need per service certificates? I am having a hard time coming up with a setup where it would really be needed to have per service env vars for configuring the certificates. When starting separate ocis instance per service you can always set "OCIS_GRPC_TLS_CERTIFICATE to a different value per ocis instance if really needed. If you run a single ocis instance, why would you need separate certificates? And if you really want that, you can even configure it in the config file (each service's configuration section can have it's own tls configuration).

Yeah, I had the same thought but did not really follow up on it. Thank you for driving the nail into the coffin by confirming it.