Security releases and coordinated release

[AdamMajer]

The current security practice for nodejs project is to notify everyone of upcoming security release approximately a week in advance and then do a release centered around these security issues only. The security issues could be Node specific or in the currently bundled 3rd party dependencies.

Security notifications are limited to members of https://groups.google.com/g/nodejs-sec

There are two-fold issues here when it comes to Linux distros (or BSD),

1. most distributions are subscribed to issues via oss-security mailing list - https://oss-security.openwall.org/wiki/mailing-lists/oss-security . CC'ing this mailing list would raise awareness of security issues.
2. coordinated release dates (CRD) for embargoed bugs via https://oss-security.openwall.org/wiki/mailing-lists/distros

Regarding the second point, and I will use OpenSSL as an example, normally 1 week prior to a public release of a security fix, a member of openssl project would mail the distros mailing list a description of the bug along with pending patches. This allowed distributions to analyze these issues and backport fixes while keeping the issue non-public. Then when the issue is made public (generally by same person that sent the embargoed report in the first place), distributions are ready with the fixes, or at least they've had some time to analyze the impact. There is a header-only archive where you can see this followed few days later by public announcement,

https://www.openwall.com/lists/linux-distros/2020/12/01/8
https://www.openssl.org/news/secadv/20191206.txt

The question that remains is Why? The main answer is that the lifecycle of Node releases in some distributions is longer than the LTS lifecycle in the nodejs project. At SUSE we still have nodejs8 maintained for some customers. Furthermore, the release process needs to account for additional testing by distributions which also requires time. A few days advance warning would prevent this race between the issue being public and being fixable by all the integrators. This was especially important in more visible linux kernel or openssl issues in the recent past.

Finally, this is not about the 3rd party bugs, like bugs in V8 or OpenSSL or ICU (since those issues are public before being fixed in Node), but reserved to the core node code.

So, my questions here are,

1. would NodeJS project consider using CRD via the already curated security-centered web-of-trust that is the distros mailing list?
2. could oss-security ML be cc'ed when there is a security release from node?

[Trott]

@nodejs/security @nodejs/releasers

[MylesBorins]

I think this should likely be transferred to https://github.com/nodejs/release for discussion by the release wg

[richardlau]

I think this should likely be transferred to https://github.com/nodejs/release for discussion by the release wg

I dunno, it feels like this is more security policy rather than a release policy (that's not to say we can't also discuss this in the Release WG).

[targos]

I vaguely remember an existing related discussion. It was more generally about giving early access to security patches to some trusted entities.

[Trott]

@nodejs/distros

[Trott]

I vaguely remember an existing related discussion. It was more generally about giving early access to security patches to some trusted entities.

Same and I'm frustrated I can't find it now.

[mhdawson]

• With my TSC "hat" on:

  • There has been discussion of special notice, CRDs etc. in the past several times over the years. From what I've seen
    the project has always been open to these, but what has been missing are people who are willing to champion them
    and help do extra work/co-ordination etc.
  • There are some harder questions in terms of how you define the line of who can/cannot get special access that will
    take effort to work through and then there is likely going to be additional ongoing work to support additional
    process.

• Putting on my Red Hat "hat".

  • Our RH collaborators work closely with the our RHEL team that builds Node.js and we've explored if
    championing this on our own would make sense. The answer has been that it would be more of a "nice-to-have" and
    have no current plans to do so by ourselves.

• Looking at the specific requests:

  • would NodeJS project consider using CRD via the already curated security-centered web-of-trust that
    is the distros mailing list?
  • could oss-security ML be cc'ed when there is a security release from node?

The second one is a small extension on what we do, and is reasonable to be PR'd into https://github.com/nodejs/node/blob/master/doc/guides/security-release-process.md (@danbev please sanity check me on this one as somebody else who has stewarded security releases).

The first one I think would require people from the distro's most interested getting more involved in the Node.js project
and then helping to champion and support the execution of the required co-ordination, planning etc. It's been proposed/discussed in the past but without somebody willing to champion and take on the work the discussions have stalled out.

[danbev]

The second one is a small extension on what we do, and is reasonable to be PR'd into https://github.com/nodejs/node/blob/master/doc/guides/security-release-process.md (@danbev please sanity check me on this one as somebody else who has stewarded security releases).

Yes, this sounds like something we could do 👍

[AdamMajer]

The first one I think would require people from the distro's most interested getting more involved in the Node.js project
and then helping to champion and support the execution of the required co-ordination, planning etc. It's been proposed/discussed in the past but without somebody willing to champion and take on the work the discussions have stalled out.

Indeed, this would be an unattainable task for a single project hence the mention of the distros mailing list. It's already self-organized.

https://github.com/nodejs/node/blob/master/doc/guides/security-release-process.md

In the process that already exists, during the per-annoucement the patches already exist and the CVEs are assigned. The planned release date is already, well, planned and within the maximum embargoed timeframe. My request here is to forward the patches for medium and higher priority CVEs to the distros ML at the time of the pre-annoucement. Further co-ordination is not really required on the part of the project.

Regarding additional work, I understand that every addition to the security release process is like adding straw camel's back. If desired, I could help in coordinating this with distros in the future, though I'm not involved in the day to day activity of either nodejs nor am I a member of distros (we do have people at SUSE that are members).

In many ways this is a wishlist issue. All security issues in Node have been very well handled and are not difficult to backport thanks to their unit tests that accompany them.

[mcollina]

The technical reason why we cannot do this at this time is that the patches are not final when we announce the security release. We cannot run CI for the security patches until we lock down CI for the security release. Adding the requirement to announce the patches one week before the release would lock our CI for 2 weeks instead of one.

Ideally this should not be needed and we would have the means to run CI for the security fixes in a secure way. However this is not possible right now and I do not think we have the bandwidth or resources to do it.

[mhdawson]

Since @AdamMajer has said using CRD is a "wishlist" issue, my take would be that we add the extra CC to our process and leave it at that until there is a strong need/champions to do more. @danbev do you want to PR the change to the security process or should I do that?

[danbev]

@danbev do you want to PR the change to the security process or should I do that?

I created nodejs/node#39191 for this yesterday 👍

[AdamMajer]

The technical reason why we cannot do this at this time is that the patches are not final when we announce the security release. We cannot run CI for the security patches until we lock down CI for the security release.

Thank you for the explanation. Since there are still technical issues here, the issue with the CRD should probably be punted into the future for now.

[mhdawson]

@danbev thanks!

@AdamMajer thanks for the follow up. Since @danbev has opened the PR to add the CC is there anything else we should discuss before closing out this issue?

[AdamMajer]

@mhdawson I believe everything has been discussed now.