Test for conformance to NIST SP 800-52

[dcooper16]

The reason that I started contributing to testssl.sh is that I was tasked with developing a test suite that tests a web server's configuration to the National Institute of Standards and Technology's (NIST) Special Publication 800-52 Revision 1, Guidelines for the Selection, Configuration, and Use of Transport Layer Security (TLS) Implementations. I'm using testssl.sh since it already does most of what is needed, and the code that is there will make the implementation of the rest much easier.

If necessary, the version of the code that tests for conformance to SP 800-52 will be maintained as a separate fork, but if there's interest in incorporating the additional tests into the baseline code that would be my preference. My thought would be to add a "--SP800-52" flag and then only test for things that are specific to that document if the flag is set.

Would this be of interest, or should I not bother to submit PRs for code that would test conformance to SP 800-52, but that wouldn't be of interest to anyone who isn't trying to test conformance to SP 800-52?

Thanks,

David

[drwetter]

Hi David,

Am 30. März 2016 17:10:49 MESZ, schrieb David Cooper notifications@github.com:

The reason that I started contributing to testssl.sh is that I was
tasked with developing a test suite that tests a web server's
configuration to the National Institute of Standards and
Technology's (NIST)
Special Publication 800-52 Revision 1, Guidelines for the Selection,
Configuration, and Use of Transport Layer Security (TLS)
Implementations. I'm
using testssl.sh since it already does most of what is needed, and the
code that is there will make the implementation of the rest much
easier.

Good to hear!

If necessary, the version of the code that tests for conformance to SP
800-52 will be maintained as a separate fork, but if there's interest
in incorporating the additional tests into the baseline code that would
be my preference. My thought would be to add a "--SP800-52" flag and
then only test for things that are specific to that document if the
flag is set.

I like both ideas, a) having one common code b) a NIST check.

Besides it's kind of an honor for the project.

Would this be of interest, or should I not bother to submit PRs for
code that would test conformance to SP 800-52, but that wouldn't be of
interest to anyone who isn't trying to test conformance to SP 800-52?

Can't speak for others but to me it sounds pretty useful! It probably boils down to the question "how". There's a techical how and an organizing how.

Atm I have no bigger picture how much additional code that would be. I have gotten much trust in your work in the past but if there are large chunks coming maybe it's a good idea that we discuss a concept / design first.

What would be your time line? Just asking because some Linux distributions like Fedora, Debian and Ubuntu and others picked up testssl.sh. Unfortunately the 2.6 release as I marked it as stable. Thus I would love to release 2.8 stable soon. For this the known bugs need to be addressed. (I bet they are also in 2.6 but at the time of release I didn't know).

Thx, Dirk

Set from my mobile. Excuse my brevity&typos

[dcooper16]

At this point I'm just getting started on the work, and I started with the things seemed easiest to implement. My idea is to submit PRs over time as I create the code in order to make code's testing of SP 800-52 conformance incrementally more complete over time.

I definitely would not suggest delaying a 2.8 release for this. If I submit a PR that for a change that wouldn't be appropriate for the 2.8 release, you could just delay incorporating it until after 2.8 has been forked from the master branch.

So far, I have only made real progress on two SP 800-52 conformance issues. The first is version negotiation. SP 800-52 Rev. 1 says:

Some server implementations are known to implement version negotiation incorrectly. For example, there are TLS 1.0 servers that terminate the connection when the client offers a version newer than TLS 1.0. Servers that incorrectly implement TLS version negotiation shall not be used.

I don't know what server implementations this refers to, but I added some code to run_protocols() that calls tls_sockets "05" "$TLS12_CIPHER" and then verifies that the connection was successful and that $DETECTED_TLS_VERSION is equal to the latest version of TLS supported by the server. The code only runs if the server supports a SSLv3 or newer and both $using_sockets and $EXPERIMENTAL are set to true.

The second item I've been working on is testing the set of cipher suites supported by the server against the set of cipher suites that SP 800-52 Rev.1 says servers SHALL, SHOULD, or SHALL NOT support. This list differs from what is tested by run_std_cipherlists(), so it would only be tested if the "--SP800-52" flag were set.

At the moment I wrote this code to use tls_sockets if both $using_sockets and $EXPERIMENTAL are set to true and OpenSSL otherwise. I did this since using tls_sockets allows for more thorough testing, especially in the "SHALL NOT" case, since tls_sockets allows me to test for every "bad" cipher suite and not just the "bad" ones that are still supported by the version of OpenSSL being used. The only problem is that including both the tls_sockets and OpenSSL versions makes the code longer, and thus harder to read.

[drwetter]

Am 03/30/2016 um 10:53 PM schrieb David Cooper:

At this point I'm just getting started on the work, and I started with the things seemed
easiest to implement. My idea is to submit PRs over time as I create the code in order to make
code's testing of SP 800-52 conformance incrementally more complete over time.

I definitely would not suggest delaying a 2.8 release for this. If I submit a PR that for a
change that wouldn't be appropriate for the 2.8 release, you could just delay incorporating it
until after 2.8 has been forked from the master branch.

Ok.

So far, I have only made real progress on two SP 800-52 conformance issues. The first is
version negotiation. SP 800-52 Rev. 1 says:

Some server implementations are known to implement version negotiation incorrectly. For
example, there are TLS 1.0 servers that terminate the connection when the client offers a
version newer than TLS 1.0. Servers that incorrectly implement TLS version negotiation
*shall not* be used.

I don't know what server implementations this refers to, but I added some code to
run_protocols() that calls tls_sockets "05" "$TLS12_CIPHER" and then verifies that the
connection was successful and that $DETECTED_TLS_VERSION is equal to the latest version of TLS
supported by the server. The code only runs if the server supports a SSLv3 or newer and both
$using_sockets and $EXPERIMENTAL are set to true.

look into that later.

That could be the thing why the widespread introduction of TLS 1.2 was delayed. IIRC the server
side TLS stack
was surprised that all of a sudden a client advertised a different TLS version on the record
layer as on the handshake layer. Some servers then just tore down the connection and the
client gave up.

I remember have drafted already a TLS tolerance check where that was part of it (as well as
checking
for TLS 1.3) but atm cannot find it.

Cheers, Dirk

[drwetter]

Am 03/30/2016 um 10:53 PM schrieb David Cooper:

At this point I'm just getting started on the work, and I started with the things seemed
easiest to implement. My idea is to submit PRs over time as I create the code in order to make
code's testing of SP 800-52 conformance incrementally more complete over time.

I definitely would not suggest delaying a 2.8 release for this. If I submit a PR that for a
change that wouldn't be appropriate for the 2.8 release, you could just delay incorporating it
until after 2.8 has been forked from the master branch.

So far, I have only made real progress on two SP 800-52 conformance issues. The first is
version negotiation. SP 800-52 Rev. 1 says:

Some server implementations are known to implement version negotiation incorrectly. For
example, there are TLS 1.0 servers that terminate the connection when the client offers a
version newer than TLS 1.0. Servers that incorrectly implement TLS version negotiation
*shall not* be used.

I don't know what server implementations this refers to, but I added some code to
run_protocols() that calls tls_sockets "05" "$TLS12_CIPHER" and then verifies that the
connection was successful and that $DETECTED_TLS_VERSION is equal to the latest version of TLS
supported by the server. The code only runs if the server supports a SSLv3 or newer and both
$using_sockets and $EXPERIMENTAL are set to true.

remark #2: IIS 7+ servers seem not very tolerant with my socket-based handshake for TLS
1.2 (only). Probably it expects something in the TLS extensions which I haven't done
good enough yet, see #122, #90.

You may want to keep that in mind.

Cheers, Dirk

[dcooper16]

One of the servers that I've tested against is an IIS server that incorrectly reports that TLS 1.2 is not supported if I set EXPERIMENTAL to true. However, I noticed comments in socksend_tls_clienthello() that said:

#FIXME: for TLS 1.2 and IIS servers we need extension_signature_algorithms!!

#FIXME: we (probably) need extension_signature_algorithms here. TLS 1.2 fails on IIS otherwise

So, I changed socksend_tls_clienthello() so that it includes the signature algorithms extension (a trivial change since almost all of the work had already been done), and that seemed to make everything work.

Is there reason to believe that adding the signature algorithms extension to the client hello that socksend_tls_clienthello() currently sends isn't enough?

David

[drwetter]

Am 03/31/2016 um 03:58 PM schrieb David Cooper:

One of the servers that I've tested against is an IIS server that incorrectly reports that TLS
1.2 is not supported if I set EXPERIMENTAL to true. However, I noticed comments in
socksend_tls_clienthello() that said:

#FIXME: for TLS 1.2 and IIS servers we need extension_signature_algorithms!!

#FIXME: we (probably) need extension_signature_algorithms here. TLS 1.2 fails on IIS otherwise

So, I changed socksend_tls_clienthello() so that it includes the signature algorithms extension
(a trivial change since almost all of the work had already been done), and that seemed to make
everything work.

oh, ok...

Is there reason to believe that adding the signature algorithms extension to the client hello
that socksend_tls_clienthello() currently sends isn't enough?

Can't recall this exactly as it must be > 1 year ago. I guess it was more the EC related,
i.e. when the client advertises EC ciphers but doesn't send a EC point format or ECs.

If you have an improvement: a PR would be much appreciated!

In any case: the heartbleed or ccs client hello is more complete as far as the
TLS extensions are concerned. But less flexible

Cheers, Dirk

Set from my mobile. Excuse my brevity&typos

[dcooper16]

I've been doing a lot of work over the past week on the code I've been developing. I've modified socksend_tls_clienthello() to include several additional extensions, including signature algorithms and, when ECC cipher suites are present, the EC point formats and supported elliptic curves. This seems to work, but I should test against more servers before submitting a PR.

I've also done a near complete re-write of parse_tls_serverhello(). I'm trying to make it possible to use tls_sockets() wherever "$OPENSSL s_client" is currently used. So, I revised the code so that it extracts the list of extensions in the server hello, the server's certificate and intermediate certificates, any OCSP response, and the size of any ephemeral (DH or ECDH) key used.

As a proof-of-concept, I wrote a version of run_allciphers that uses tls_sockets, and I think I could do the same with run_server_defaults, but I haven't tried yet.

If you're interested in seeing what I've done so far, I've posted it to https://github.com/dcooper16/testssl.sh/blob/extended_tls_sockets/testssl.sh.

Thanks,

Dave

[drwetter]

Hi David,

Am 04/08/2016 um 10:56 PM schrieb David Cooper:

I've been doing a lot of work over the past week on the code I've been developing. I've modified socksend_tls_clienthello() to include several additional extensions, including signature algorithms and, when ECC cipher suites are present, the EC point formats and supported elliptic curves.

cool!

This seems to work, but I should test against more servers before submitting a PR.

sounds good! I do testing by using the file option on a number of different targets and diff'ing the output (use -q best) but pretests always help/are appreciated.

I've also done a near complete re-write of parse_tls_serverhello(). I'm trying to make it possible to > use tls_sockets() wherever "$OPENSSL s_client" is currently used. So, I revised the code so that > it extracts the list of extensions in the server hello, the server's certificate and intermediate
certificates, any OCSP response, and the size of any ephemeral (DH or ECDH) key used.

wow.

As a proof-of-concept, I wrote a version of run_allciphers that uses tls_sockets, and I think I could do the same with run_server_defaults, but I haven't tried yet.

That's definitely more than you indicated, great!

I think a step by step approach would be great:

• Have a solid TLS 1.2 handshake
• put the socket-based TLS 1.2 protocol check into production
• [anything else]
• [more]

If you're interested in seeing what I've done so far, I've posted it to https://github.com/dcooper16/testssl.sh/blob/extended_tls_sockets/testssl.sh.

glanced at it: Tip my hat! :-)

The question to you now is: How long can it wait to be integrated into mainline?
I don't want to disgruntle you.

As indicated I would love to squash known bugs (https://github.com/drwetter/testssl.sh/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+label%3Abug+-label%3Awontfix)
first, and do a release.

problem is: atm I lack time. Realistically I can have the bugs squashed in ~4 weeks (help appreciated).

Best, Dirk

[dcooper16]

At this point I'm not in any rush to see my code integrated into the mainline. I'll continue to work on the code, and I'll post things to https://github.com/dcooper16/testssl.sh as I go along (so that others can see them if they are interested), but I'll try to avoid submitting any PRs for anything that would likely be considered inappropriate for the 2.8 release until after that release is done.

Hopefully the PR I submitted for socksend_tls_clienthello() will help towards the issues that you listed above:

• Have a solid TLS 1.2 handshake
• put the socket-based TLS 1.2 protocol check into production

David

[drwetter]

Am 04/18/2016 um 06:19 PM schrieb David Cooper:

At this point I'm not in any rush to see my code integrated into the mainline. I'll continue to work on the code, and I'll post things to https://github.com/dcooper16/testssl.sh as I go along (so that others can see them if they are interested), but I'll try to avoid submitting any PRs for anything that would likely be considered inappropriate for the 2.8 release until after that release is done.

I would not choose the term inappropriate here because your work would certainly appear not as good as deserved. ;-)

But as suggested: Lets get 2.8 off the table and then I am glad for anything further extension of functionality.

Set from my mobile. Excuse my brevity&typos

[dcooper16]

FYI, a draft version of NIST SP 800-52 Revision 2, Guidelines for the Selection, Configuration, and Use of Transport Layer Security (TLS) Implementations, is out for public comment until February 1, 2018: https://csrc.nist.gov/publications/detail/sp/800-52/rev-2/draft.

As described previously in this issue, I've been maintaining a branch of testssl.sh that includes some checks for conformance to SP 800-52, but I decided to base the checks on the upcoming Revision 2 rather than Revision 1. It took longer than expected for a draft of Revision 2 to be ready due to a decision to include information related to TLSv1.3. But, now that a draft of Revision 2 has been posted, I hope to make the branch that I've been maintaining available soon.

In the meantime, if anyone has time to look at the draft of NIST SP 800-52 Revision 2 and has any comments, they would be appreciated.

[drwetter]

Tip of the hat 👍 to the author!

So, late February / March 2018 can the release be expected?

Good step forward is the appreciation of TLS 1.3 and soon demise of RSA PKCS #1 v1.5 , TLS 1.1, and TLS 1.0.

[dcooper16]

So, late February / March 2018 can the release be expected?

Unfortunately, things never happen that quickly. I was involved in writing one document where the public comment period ended March 1, 2016, and the final version of the document hasn't been released yet. 😞

Hopefully this one will be released much faster, but it will be at least a few months after the comment period ends before the final version is released.

[dcooper16]

I posted a version of the 3.1dev branch at https://github.com/dcooper16/testssl.sh/tree/nist that tailors some of the checks in testssl.sh to check a server's configuration against the checks in NIST SP 800-52 (if the --nist option is used or the NIST environment variable is set to true). I'm sure there are some places in which it could be made better, but I think it is fairly useful for those who need to follow NIST guidelines.

One issue that I have not yet tried to tackle is the grading system. I don't think reporting the grade that would have been given by SSL Labs would be appropriate, as it could provide a high grade to a server that only supports cipher suites that are not approved for use by NIST. However, I haven't really thought about how to modify to grading. So, for the moment the branch just sets do_rating to false if NIST is true.

Any comments or suggestions that people have on this branch would be appreciated.

Thanks,

David

[drwetter]

Hi David,

cool! The difference in the output of your nist branch is not really apparent (except the missing gradrng), one need to look at the code.

If you ask me I find a NIST compliance check better than SSLlabs.Since Ivan left Qualys it looks to me like there wasn't so much progress. On their rating probably it was only little (see date @ 31 January 2020). Unfortunately there isn't even TLS 1.3 mentioned. In testssl.sh it is there because Magnus was so kind to implement it. At a certain point we should however improve the rating (and e.g. rename it to ~ on the basis of) or as indicated switch to a template where every user can have a choice. Or at a certain point switch completely to NIST.

That is probably not easy at the moment because we would need to separate test from screen and file output. The first step for that would be defining variables / arrays which hold the results, like e.g. censys does that (see example https://censys.io/ipv4/81.169.166.184/table). Once the variables or a batch their of have a value, we can apply any grading. So that would be more of a two step process that we do now.

[Dejavu610]

I posted a version of the 3.1dev branch at https://github.com/dcooper16/testssl.sh/tree/nist that tailors some of the checks in testssl.sh to check a server's configuration against the checks in NIST SP 800-52 (if the --nist option is used or the NIST environment variable is set to true). I'm sure there are some places in which it could be made better, but I think it is fairly useful for those who need to follow NIST guidelines.

One issue that I have not yet tried to tackle is the grading system. I don't think reporting the grade that would have been given by SSL Labs would be appropriate, as it could provide a high grade to a server that only supports cipher suites that are not approved for use by NIST. However, I haven't really thought about how to modify to grading. So, for the moment the branch just sets do_rating to false if NIST is true.

Any comments or suggestions that people have on this branch would be appreciated.

Thanks,

David

Hi David，
Thanks for your work!
Does the version of the 3.1dev branch at https://github.com/dcooper16/testssl.sh/tree/nist can check a server's configuration against the checks in NIST SP 800-52 Revison 2?
Is there any other tools can check a server's configuration in NIST SP 800-52 Revison 2?

[dcooper16]

Hi @KongLynn,

If you use the branch at https://github.com/dcooper16/testssl.sh/tree/nist and specify the --nist option at the command line, then it would check for conformance to NIST SP 800-52 Revision 2. It does not check for conformance to all of the requirements in NIST SP 800-52 Revision 2, but it does, for example, rate ciphers and key lengths according to NIST guidance, which is not always the same as the rating that testssl.sh would give by default.

I am not aware of any other tools that can perform these checks.

[Dejavu610]

Hi @KongLynn,

If you use the branch at https://github.com/dcooper16/testssl.sh/tree/nist and specify the --nist option at the command line, then it would check for conformance to NIST SP 800-52 Revision 2. It does not check for conformance to all of the requirements in NIST SP 800-52 Revision 2, but it does, for example, rate ciphers and key lengths according to NIST guidance, which is not always the same as the rating that testssl.sh would give by default.

I am not aware of any other tools that can perform these checks.

Hi @dcooper16 ，
Thank you for your reply. I use the branch to test my server, and it reports
Unapproved 256-bit ciphers offered (NOT ok)
Why 256-bit ciphers is not secure? For example，TLS_CHACHA20_POLY1305_SHA256 is recommended in https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-4.

[dcooper16]

Hi @KongLynn,

I am not suggesting that TLS_CHACHA20_POLY1305_SHA256 is insecure. As you note, it is one of the cipher suites recommended by the IETF. It is also rated as a good cipher suite by testssl.sh (without the --nist option). However, neither ChaCha nor Poly1305 has ever been NIST approved. So, U.S. federal agencies can not use this cipher suite to protect sensitive information.

The only symmetric encryption algorithms that are NIST approved at the moment are AES and 3DES. 3DES is no longer allowed (for U.S. federal agencies) for use with TLS, and it will soon be disallowed for all purposes, leaving AES as the only approved option.

So, when specifying the --nist option to test against NIST guidelines, the use of any cipher suite that does use AES for symmetric encryption will be marked as an issue.

[Dejavu610]

Hi @KongLynn,

I am not suggesting that TLS_CHACHA20_POLY1305_SHA256 is insecure. As you note, it is one of the cipher suites recommended by the IETF. It is also rated as a good cipher suite by testssl.sh (without the --nist option). However, neither ChaCha nor Poly1305 has ever been NIST approved. So, U.S. federal agencies can not use this cipher suite to protect sensitive information.

The only symmetric encryption algorithms that are NIST approved at the moment are AES and 3DES. 3DES is no longer allowed (for U.S. federal agencies) for use with TLS, and it will soon be disallowed for all purposes, leaving AES as the only approved option.

So, when specifying the --nist option to test against NIST guidelines, the use of any cipher suite that does use AES for symmetric encryption will be marked as an issue.

Hi @KongLynn,

I am not suggesting that TLS_CHACHA20_POLY1305_SHA256 is insecure. As you note, it is one of the cipher suites recommended by the IETF. It is also rated as a good cipher suite by testssl.sh (without the --nist option). However, neither ChaCha nor Poly1305 has ever been NIST approved. So, U.S. federal agencies can not use this cipher suite to protect sensitive information.

The only symmetric encryption algorithms that are NIST approved at the moment are AES and 3DES. 3DES is no longer allowed (for U.S. federal agencies) for use with TLS, and it will soon be disallowed for all purposes, leaving AES as the only approved option.

So, when specifying the --nist option to test against NIST guidelines, the use of any cipher suite that does use AES for symmetric encryption will be marked as an issue.

Hi， @dcooper16 ,
Got it. Thank you very much.