OpenSSL engine support is broken #2195

peternied · 2022-10-26T02:13:56Z

The following settings were used to enable the use of OpenSSL in JDK11 runtime. After investigating this is not working at all and OpenSSL has been broken for an indeterminate amount of time.

plugins.security.ssl.http.enable_openssl_if_available: true
plugins.security.ssl.transport.enable_openssl_if_available: true
opensearch.unsafe.use_netty_default_allocator: true

After quickly probing the forums I see years old posts with no resolution:

While there was some interest our modern JDK platforms support many more cryptographic protocols and make this unneeded for the majority use case. Note! This could be a solid enhancement to the platform for faster execution and better footprint.

Fixing OpenSSL support has the following steps at least...

Fix OpenSSL support so you can use the OpenSSL Engine
[BUG] OpenSSLTest is not using the OpenSSL Provider #2208
Update documentation with steps on how to get the feature functional

The text was updated successfully, but these errors were encountered:

opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com>

stephen-crawford · 2022-10-28T14:26:41Z

I think it makes sense to deprecate a feature if the feature is only used on a deprecated platform. That being said, the question of who is using OpenSSL may be hard fro this specific GitHub community to answer.

I would suspect that those using OpenSSL in other ways would be more likely to include it in any OpenSearch implementations they operated. Apparently up to 4000+ companies have reported using it (https://stackshare.io/openssl) but just because a bunch of people do something does not necessarily make it the right decision for everyone in every case.

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. #2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. #2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. #2195 Signed-off-by: Peter Nied <petern@amazon.com>

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. #2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. #2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. #2195 Signed-off-by: Peter Nied <petern@amazon.com> (cherry picked from commit a57fd0a)

cwperks · 2022-10-31T16:48:00Z

As far as I can tell, these tests (and corresponding feature) are not working properly as it is. See the output of OpenSSLTest.testHttpsAndNodeSSLKeyPass below:

---------------- Starting JUnit-test: OpenSSLTest testHttpsAndNodeSSLKeyPass ----------------
tcpClusterManagerPorts: [7130]/tcpAllPorts: [7130, 8115, 9070]/httpPorts: [9302, 9470, 9615] for (6024-11023) fork 1
[2022-10-24T13:42:35,917][WARN ][org.opensearch.node.Node] version [3.0.0-SNAPSHOT] is a pre-release version of OpenSearch and is not suitable for production
[2022-10-24T13:42:35,918][WARN ][org.opensearch.security.ssl.util.SSLCertificateHelper] Certificate chain for alias node-0 contains a root certificate
[2022-10-24T13:42:35,968][WARN ][org.opensearch.security.ssl.util.SSLCertificateHelper] Certificate chain for alias node-0 contains a root certificate
[2022-10-24T13:42:35,996][WARN ][org.opensearch.security.OpenSearchSecurityPlugin] OpenSearch Security plugin run in ssl only mode. No authentication or authorization is performed
[2022-10-24T13:42:36,020][WARN ][org.opensearch.gateway.DanglingIndicesState] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
{"client.type":"node","cluster.initial_cluster_manager_nodes":["127.0.0.1:7130"],"cluster.name":"utest_n287_fnull_t1691768839757","cluster.routing.allocation.disk.threshold_enabled":"false","discovery.initial_state_timeout":"8s","discovery.seed_hosts":["127.0.0.1:7130"],"http.compression":"false","http.cors.enabled":"true","http.port":"9302","http.type":"org.opensearch.security.ssl.http.netty.SecuritySSLNettyHttpServerTransport","http.type.default":"netty4","node.attr.shard_indexing_pressure_enabled":"true","node.max_local_storage_nodes":"3","node.name":"node_utest_n287_fnull_t1691768839757_num3","node.roles":["cluster_manager"],"path.data":["/home/runner/work/security/security/build/testrun/test/target/data/utest_n287_fnull_t1691768839757/data"],"path.home":"/home/runner/work/security/security/build/testrun/test/target","path.logs":"/home/runner/work/security/security/build/testrun/test/target/data/utest_n287_fnull_t1691768839757/logs","plugins.security.ssl.http.clientauth_mode":"REQUIRE","plugins.security.ssl.http.enable_openssl_if_available":"true","plugins.security.ssl.http.enabled":"true","plugins.security.ssl.http.keystore_alias":"node-0","plugins.security.ssl.http.keystore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/node-0-keystore.jks","plugins.security.ssl.http.keystore_keypassword":"changeit","plugins.security.ssl.http.truststore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/truststore.jks","plugins.security.ssl.transport.enable_openssl_if_available":"true","plugins.security.ssl.transport.enabled":"true","plugins.security.ssl.transport.enforce_hostname_verification":"false","plugins.security.ssl.transport.keystore_alias":"node-0","plugins.security.ssl.transport.keystore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/node-0-keystore.jks","plugins.security.ssl.transport.keystore_keypassword":"changeit","plugins.security.ssl.transport.resolve_hostname":"false","plugins.security.ssl.transport.truststore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/truststore.jks","plugins.security.ssl_only":"true","transport.tcp.port":"7130","transport.type":"org.opensearch.security.ssl.http.netty.SecuritySSLNettyTransport","transport.type.default":"netty4"}
[2022-10-24T13:42:36,029][WARN ][org.opensearch.node.Node] version [3.0.0-SNAPSHOT] is a pre-release version of OpenSearch and is not suitable for production
[2022-10-24T13:42:36,030][WARN ][org.opensearch.security.ssl.util.SSLCertificateHelper] Certificate chain for alias node-0 contains a root certificate
[2022-10-24T13:42:36,094][WARN ][org.opensearch.security.ssl.util.SSLCertificateHelper] Certificate chain for alias node-0 contains a root certificate
[2022-10-24T13:42:36,121][WARN ][org.opensearch.security.OpenSearchSecurityPlugin] OpenSearch Security plugin run in ssl only mode. No authentication or authorization is performed
[2022-10-24T13:42:36,145][WARN ][org.opensearch.gateway.DanglingIndicesState] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
{"client.type":"node","cluster.initial_cluster_manager_nodes":["127.0.0.1:7130"],"cluster.name":"utest_n287_fnull_t1691768839757","cluster.routing.allocation.disk.threshold_enabled":"false","discovery.initial_state_timeout":"8s","discovery.seed_hosts":["127.0.0.1:7130"],"http.compression":"false","http.cors.enabled":"true","http.port":"9470","http.type":"org.opensearch.security.ssl.http.netty.SecuritySSLNettyHttpServerTransport","http.type.default":"netty4","node.attr.shard_indexing_pressure_enabled":"true","node.max_local_storage_nodes":"3","node.name":"node_utest_n287_fnull_t1691768839757_num2","node.roles":["data"],"path.data":["/home/runner/work/security/security/build/testrun/test/target/data/utest_n287_fnull_t1691768839757/data"],"path.home":"/home/runner/work/security/security/build/testrun/test/target","path.logs":"/home/runner/work/security/security/build/testrun/test/target/data/utest_n287_fnull_t1691768839757/logs","plugins.security.ssl.http.clientauth_mode":"REQUIRE","plugins.security.ssl.http.enable_openssl_if_available":"true","plugins.security.ssl.http.enabled":"true","plugins.security.ssl.http.keystore_alias":"node-0","plugins.security.ssl.http.keystore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/node-0-keystore.jks","plugins.security.ssl.http.keystore_keypassword":"changeit","plugins.security.ssl.http.truststore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/truststore.jks","plugins.security.ssl.transport.enable_openssl_if_available":"true","plugins.security.ssl.transport.enabled":"true","plugins.security.ssl.transport.enforce_hostname_verification":"false","plugins.security.ssl.transport.keystore_alias":"node-0","plugins.security.ssl.transport.keystore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/node-0-keystore.jks","plugins.security.ssl.transport.keystore_keypassword":"changeit","plugins.security.ssl.transport.resolve_hostname":"false","plugins.security.ssl.transport.truststore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/truststore.jks","plugins.security.ssl_only":"true","transport.tcp.port":"8115","transport.type":"org.opensearch.security.ssl.http.netty.SecuritySSLNettyTransport","transport.type.default":"netty4"}
[2022-10-24T13:42:36,154][WARN ][org.opensearch.node.Node] version [3.0.0-SNAPSHOT] is a pre-release version of OpenSearch and is not suitable for production
[2022-10-24T13:42:36,155][WARN ][org.opensearch.security.ssl.util.SSLCertificateHelper] Certificate chain for alias node-0 contains a root certificate
[2022-10-24T13:42:36,221][WARN ][org.opensearch.security.ssl.util.SSLCertificateHelper] Certificate chain for alias node-0 contains a root certificate
[2022-10-24T13:42:36,247][WARN ][org.opensearch.security.OpenSearchSecurityPlugin] OpenSearch Security plugin run in ssl only mode. No authentication or authorization is performed
[2022-10-24T13:42:36,265][WARN ][org.opensearch.gateway.DanglingIndicesState] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
{"client.type":"node","cluster.initial_cluster_manager_nodes":["127.0.0.1:7130"],"cluster.name":"utest_n287_fnull_t1691768839757","cluster.routing.allocation.disk.threshold_enabled":"false","discovery.initial_state_timeout":"8s","discovery.seed_hosts":["127.0.0.1:7130"],"http.compression":"false","http.cors.enabled":"true","http.port":"9615","http.type":"org.opensearch.security.ssl.http.netty.SecuritySSLNettyHttpServerTransport","http.type.default":"netty4","node.attr.shard_indexing_pressure_enabled":"true","node.max_local_storage_nodes":"3","node.name":"node_utest_n287_fnull_t1691768839757_num1","node.roles":["data"],"path.data":["/home/runner/work/security/security/build/testrun/test/target/data/utest_n287_fnull_t1691768839757/data"],"path.home":"/home/runner/work/security/security/build/testrun/test/target","path.logs":"/home/runner/work/security/security/build/testrun/test/target/data/utest_n287_fnull_t1691768839757/logs","plugins.security.ssl.http.clientauth_mode":"REQUIRE","plugins.security.ssl.http.enable_openssl_if_available":"true","plugins.security.ssl.http.enabled":"true","plugins.security.ssl.http.keystore_alias":"node-0","plugins.security.ssl.http.keystore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/node-0-keystore.jks","plugins.security.ssl.http.keystore_keypassword":"changeit","plugins.security.ssl.http.truststore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/truststore.jks","plugins.security.ssl.transport.enable_openssl_if_available":"true","plugins.security.ssl.transport.enabled":"true","plugins.security.ssl.transport.enforce_hostname_verification":"false","plugins.security.ssl.transport.keystore_alias":"node-0","plugins.security.ssl.transport.keystore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/node-0-keystore.jks","plugins.security.ssl.transport.keystore_keypassword":"changeit","plugins.security.ssl.transport.resolve_hostname":"false","plugins.security.ssl.transport.truststore_filepath":"/home/runner/work/security/security/build/resources/test/ssl/truststore.jks","plugins.security.ssl_only":"true","transport.tcp.port":"9070","transport.type":"org.opensearch.security.ssl.http.netty.SecuritySSLNettyTransport","transport.type.default":"netty4"}
{
  "principal" : "CN=node-0.example.com,OU=SSL,O=Test,L=Test,C=DE",
  "peer_certificates" : "3",
  "ssl_protocol" : "TLSv1.2",
  "ssl_cipher" : "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
  "ssl_openssl_available" : false,
  "ssl_openssl_version" : -1,
  "ssl_openssl_version_string" : null,
  "ssl_openssl_non_available_cause" : "java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSLContext",
  "ssl_openssl_supports_key_manager_factory" : false,
  "ssl_openssl_supports_hostname_validation" : false,
  "ssl_provider_http" : "JDK",
  "ssl_provider_transport_server" : "JDK",
  "ssl_provider_transport_client" : "JDK"
}

This test isn't even using OpenSSL as the ssl provider and I'm not sure when that stopped being the case. These 2 PRs may be related: #422 and #1649

I believe the test is working because this block will return the JDK provider instead of the OpenSSL provider so that cluster is still able to setup SSL: https://github.com/opensearch-project/security/blob/main/src/main/java/org/opensearch/security/ssl/DefaultSecurityKeyStore.java#L161-L169

There is no specific assertion in the test to ensure it was brought up with the OpenSSL provider, so as far as I can tell this feature was regressed on at some point and it was not noticed.

cwperks · 2022-10-31T16:56:25Z

Note: According to this issue on Cassandra (https://issues.apache.org/jira/browse/CASSANDRA-10735) OpenSSL can provide performance improvements on the JDK SSL provider. It may be worth it to look into this further and see what it takes to support OpenSSL.

peternied · 2022-10-31T17:25:17Z

I believe the test is working

Sounds like the OpenSSL tests do not verify its functionality 🤯 I am going to rewrite this issue as "OpenSSL does not work" Help wanted, and we can triage this afternoon.

cwperks · 2022-10-31T17:33:35Z

@peternied I just created an issue ;)

#2208

peternied · 2022-10-31T17:35:30Z

Note; this is not a platform support issue, as it does not work on any platforms!

cwperks · 2022-10-31T19:13:32Z

[Triage] @krisfreedain Could you please look at see if any users have been asking about this in the forums or on Github.

cwperks · 2022-10-31T19:36:00Z

[Triage] @peternied Can you file an issue on the documentation website with affected versions?

peternied · 2022-10-31T22:07:07Z

Can you file an issue on the documentation website with affected versions?

Last change in this space was during the 1.3 build on Mar 17, 2022. After consulting previous CI logs from https://github.com/opensearch-project/security/actions/runs/3200296006 which was built on 1.3.7, it only shows ssl_provider_transport_server defined as 'JDK'. Its been broken since at least then, if not longer.

@krisdreedain Could you please look at see if any users have been asking about this in the forums or on Github.

I found two issues in the forum seems like this feature has been hard to use for a while, if not downright broken for a considerable amount of time. How would you feel about crafting a message to the effect of "We discovered that the OpenSSL feature has been broken for what seems like the past year if not longer, for those that are interested in adding support checkout this github issue, ... {link to this issue}. This could be a good opportunity to contribute if there is value for you / your use cases"?

peternied · 2022-10-31T22:07:42Z

@nibix Do you have context on the impact of this feature being broken?

nibix · 2022-11-01T08:52:57Z

@nibix Do you have context on the impact of this feature being broken?

Maybe I can give some history:

OpenSSL support originally built because the TLS support that came with the JRE was lacking in regards of performance and modern cipher suites. Java 11, however, brought a fundamentally improved TLS support which fixed the performance and cipher suite issues. Thus OpenSSL support lost its significant advantage while keeping significant disadvantages: Hard to configure, thus being a noticeable support burden. So, OpenSSL support was disabled at some point in time - which was actually before ODFE was created.

Later, in 2020, AWS re-enabled OpenSSL support - however only for Java versions 11 and older:

#422

The PR does not really give a rationale on doing this - thus I don't know about the motivation here.

So, generally only users on old Java versions should be impacted. As OpenSearch is quite young, I do not really expect many users to actually rely on old Java versions. Thus, I personally do not expect a significant impact.

reta · 2022-11-02T21:24:38Z

@cwperks @peternied some findings with regards to OpenSSL support by netty (found out while working on #2208):

netty-tcnative brings OpenSSL support to limited set of plafforms (https://netty.io/wiki/forked-tomcat-native.html) and is a required dependecy
netty-tcnative needs OpenSSL 1.0.x and does not work with OpenSSL 1.1.x (Cannot use Openssl 1.1.1f with netty-tcnative, we get UnsatisfiedLinkError: .. libssl.so.1.0.0 netty/netty-tcnative#551)

And tons of other issues;

I believe I could make it run with OpenSSL 1.0.x but do we really want to take this route?

peternied · 2022-11-02T21:29:08Z

@reta Thanks for checking in. I'm of the opinion that we should not work any more on this issue and update the documentation site. I say we don't invest more time into OpenSSL until we find someone(s) that have good reason not to use the JDK's SSL Engine and OpenSSL is a good fit.

I'm going to re-add the untriaged tag so we have another discussion about this issue on Monday with the maintainers and give everyone time to weigh in before we make a more conclusive decision.

cwperks · 2022-11-07T20:23:53Z

[Triage] @peternied after discussion in Triage with @reta we are planning to remove the OpenSSL-related code as this feature has been broken for a while.

…nsearch-project#2205) Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com> (cherry picked from commit a57fd0a)

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com> Signed-off-by: Stephen Crawford <steecraw@amazon.com>

…nsearch-project#2205) Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com> (cherry picked from commit a57fd0a) Signed-off-by: Stephen Crawford <steecraw@amazon.com>

peternied · 2022-11-28T15:54:06Z

While this issue was discovered during 2.4.0 it isn't slated for a fix in that release.

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com>

reta · 2023-01-24T15:51:18Z

@peternied I think we could close this one with a note that BoringSSL seems to work but Netty's OpenSSL support is only available for OpenSSL 1.x versions.

…nsearch-project#2205) Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com> (cherry picked from commit a57fd0a)

peternied added enhancement New feature or request untriaged Require the attention of the repository maintainers and may need to be prioritized breaking This issue is or proposes a breaking change labels Oct 26, 2022

peternied added a commit to peternied/security that referenced this issue Oct 26, 2022

Disable OpenSSL on Windows and Mac

d9aa6cf

opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com>

peternied mentioned this issue Oct 26, 2022

Add CI for Windows and MacOS platforms #2190

Merged

3 tasks

peternied mentioned this issue Oct 28, 2022

[Backport 2.x] Add CI for Windows and MacOS platforms (#2190) #2205

Merged

3 tasks

peternied added the windows label Oct 28, 2022

peternied mentioned this issue Oct 28, 2022

[Meta] OpenSearch / OpenSearch Dashboards - Windows distribution (X64) opensearch-project/opensearch-build#33

Closed

19 tasks

peternied added the v2.4.0 'Issues and PRs related to version v2.4.0' label Oct 28, 2022

peternied changed the title ~~Should Windows / Mac platforms support OpenSSL?~~ Should Windows platforms support OpenSSL? Oct 28, 2022

peternied mentioned this issue Oct 31, 2022

Release Version 2.4.0 #2160

Closed

23 tasks

peternied removed the breaking This issue is or proposes a breaking change label Oct 31, 2022

peternied added help wanted Community contributions are especially encouraged for these issues. and removed windows labels Oct 31, 2022

peternied changed the title ~~Should Windows platforms support OpenSSL?~~ OpenSSL engine support is broken Oct 31, 2022

cwperks removed the untriaged Require the attention of the repository maintainers and may need to be prioritized label Oct 31, 2022

peternied added the triaged Issues labeled as 'Triaged' have been reviewed and are deemed actionable. label Oct 31, 2022

peternied added the untriaged Require the attention of the repository maintainers and may need to be prioritized label Nov 2, 2022

cwperks removed the untriaged Require the attention of the repository maintainers and may need to be prioritized label Nov 7, 2022

peternied removed the v2.4.0 'Issues and PRs related to version v2.4.0' label Nov 28, 2022

cwperks mentioned this issue Dec 5, 2022

Remove OpenSSL provider as an option. JDK SSL Provider will be used by default. #2298

Closed

3 tasks

peternied closed this as completed Jan 24, 2023

cwperks mentioned this issue Aug 23, 2024

Add Nils Bandener (Github: nibix) as a maintainer #4672

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OpenSSL engine support is broken #2195

OpenSSL engine support is broken #2195

peternied commented Oct 26, 2022 •

edited

Loading

stephen-crawford commented Oct 28, 2022

cwperks commented Oct 31, 2022 •

edited

Loading

cwperks commented Oct 31, 2022

peternied commented Oct 31, 2022

cwperks commented Oct 31, 2022 •

edited

Loading

peternied commented Oct 31, 2022

cwperks commented Oct 31, 2022

cwperks commented Oct 31, 2022

peternied commented Oct 31, 2022

peternied commented Oct 31, 2022

nibix commented Nov 1, 2022 •

edited

Loading

reta commented Nov 2, 2022 •

edited

Loading

peternied commented Nov 2, 2022

cwperks commented Nov 7, 2022

peternied commented Nov 28, 2022

reta commented Jan 24, 2023

OpenSSL engine support is broken #2195

OpenSSL engine support is broken #2195

Comments

peternied commented Oct 26, 2022 • edited Loading

stephen-crawford commented Oct 28, 2022

cwperks commented Oct 31, 2022 • edited Loading

cwperks commented Oct 31, 2022

peternied commented Oct 31, 2022

cwperks commented Oct 31, 2022 • edited Loading

peternied commented Oct 31, 2022

cwperks commented Oct 31, 2022

cwperks commented Oct 31, 2022

peternied commented Oct 31, 2022

peternied commented Oct 31, 2022

nibix commented Nov 1, 2022 • edited Loading

reta commented Nov 2, 2022 • edited Loading

peternied commented Nov 2, 2022

cwperks commented Nov 7, 2022

peternied commented Nov 28, 2022

reta commented Jan 24, 2023

peternied commented Oct 26, 2022 •

edited

Loading

cwperks commented Oct 31, 2022 •

edited

Loading

cwperks commented Oct 31, 2022 •

edited

Loading

nibix commented Nov 1, 2022 •

edited

Loading

reta commented Nov 2, 2022 •

edited

Loading