Need certificate for signing binaries: Windows

[codeboten]

A discussion in the security SIG about open-telemetry/opentelemetry-collector-releases#207 led us to realizing that if we want to sign binaries for Windows, we would need an OpenTelemetry certificate. The discussion then led us to realize that this should likely be owned by the TC.

Related spec issue: open-telemetry/opentelemetry-specification#2560

[pellared]

The code signing is (usually) done using SignTool.exe. AFAIK the certificate used for code signing should be available in Personal Information Exchange (PFX) format.

AFAIK http://timestamp.digicert.com/ is a commonly used timestamp server. More: https://knowledge.digicert.com/generalinformation/INFO4231.html

[tigrannajaryan]

This requires a few things:

1. Buying a code signing certificate. This normally requires proof of identity, i.e. you need to prove to the issuing organization that you are who you claim you are. For businesses they typically want articles of incorporations, etc. Not sure what the requirements are for an org that is part of CNCF. I would imagine this to be something that the GC should do.
2. Once the certificate is issued it needs to be securely and privately stored somewhere. I don't know where that is. The access to the certificate should be guarded, any unauthorized access compromises the certificate and requires revocation and obtaining a new one. I have only done this for proprietary software in the past, no idea how do this for an OSS project. We may need to borrow a best practice from some other OSS project. Like only TC will have access to the certificate.
3. Any Windows executable that needs to be signed needs access to the certificate (and yes, signtool.exe is used for signing process). I don't think this can run as a regular github action on a public Collector repo. That would make trivial to expose the certificate. So this may require running the signing process somewhere else (a private github repo? outside github at all?)

I suggest that we come up with answers for 2 and 3 before we start working on 1.

[codeboten]

I don't think this can run as a regular github action on a public Collector repo. That would make trivial to expose the certificate

Can you say more about this? My understanding is that using a github org-level secret would work here, but it's possible i'm missing some context.

[tigrannajaryan]

Can you say more about this? My understanding is that using a github org-level secret would work here, but it's possible i'm missing some context.

I may be wrong, but I think it is trivial to update the github action to print the secret, which will be then publicly visible in the github output.

[codeboten]

I may be wrong, but I think it is trivial to update the github action to print the secret, which will be then publicly visible in the github output.

I hope this isn't possible. Would this not mean that today we run the risk to any and all secrets being available to leak and use maliciously? Can you point to some proof of concept?

If exposing GH secrets through actions is trivial, should we avoid using them altogether and remove their use today?

[tigrannajaryan]

I may be wrong, but I think it is trivial to update the github action to print the secret, which will be then publicly visible in the github output.

I hope this isn't possible. Would this not mean that today we run the risk to any and all secrets being available to leak and use maliciously? Can you point to some proof of concept?

If exposing GH secrets through actions is trivial, should we avoid using them altogether and remove their use today?

I haven't actually tried, but here is an example: https://stackoverflow.com/a/74731496

[codeboten]

@tigrannajaryan this does seem trivial. According to the github docs only users w/ write permissions would have read access to the secrets. https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#using-secrets

Consider requiring review for access to secrets

You can use required reviewers to protect environment secrets. A workflow job cannot access environment secrets until approval is granted by a reviewer. For more information about storing secrets in environments or requiring reviews for environments, see "Using secrets in GitHub Actions" and "Using environments for deployment."

I wonder would requiring reviews for accessing secrets be enough to secure those secrets?

To follow up to my previous comments, we should definitely reconsider the use of secrets altogether if they're at risk of being leaked.

[tigrannajaryan]

To follow up to my previous comments, we should definitely reconsider the use of secrets altogether if they're at risk of being leaked.

If I am understanding Github behavior correctly the only way for credentials to leak is for an approver (someone with Write access) is to modify the Github action and print the credential to console or ssh to the machine. So, a random PR from a random person can't result in a leak. (Someone please confirm my understanding is correct).

I think this is a good enough protection for most of our secrets we use but I am open to rethinking this if we want stronger protection.

The code signing certificate seems a bit more critical to me since if it leaks the process of fixing the problem is more complicated: need to revoke the existing cert - not sure how exactly, then need to obtain a new cert, then need to use that new cert. For other secrets it is a simpler process - you just update a password (or whatever is leaked). This was the primary reason I am more reluctant about using the same approach for code signing.

Again, I may be missing something, I haven't looked in-depth into this issue, so feel free to correct me.

[trask]

a random PR from a random person can't result in a leak. (Someone please confirm my understanding is correct).

this is correct, here is the important part from https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#accessing-secrets

Workflows triggered using the pull_request event have read-only permissions and have no access to secrets.

[codeboten]

I think this is a good enough protection for most of our secrets we use but I am open to rethinking this if we want stronger protection.

@tigrannajaryan it would be good to get a consensus on what's secure enough. Credentials used to publish packages would may have a similar if not larger blast radius if they were leaked. So far we've worked with the assumption that github secrets were good enough, but again, i would like to make sure that's the case moving forward.

I agree that the remediation of signing certs is pretty terrible in general, so I get the reluctance.

[tigrannajaryan]

I think this is a good topic for SIG security to figure out.

[pellared]

I think this is a good topic for SIG security to figure out.

The issue was created because of the discussion we had during the SIG security meeting. I think that this is something that could be driven by .NET SIGs.

@reyang are you able to help as a TC member, OTel .NET approver and Microsoft employee?

[reyang]

@reyang are you able to help as a TC member, OTel .NET approver and Microsoft employee?

I want to, and I'm interested in this topic as well. The problem is that I have recurring conflicts so we need to find a time that we can collaborate.

[RassK]

Bump, any updates? Progress?

[austinlparker]

Something to consider as an option: https://azure.microsoft.com/en-us/products/trusted-signing?msockid=2d0894ca7a9d65c4016c80bc7bba64d2