Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elasticsearch disable CA retrieval when ssl is disabled #2475

Conversation

Anaethelion
Copy link
Contributor

What does this PR do?

This makes optional the retrieval of the Elasticsearch generated CA for version 8 when CA generation has been explicitly disabled.

Why is it important?

While Elastic supports security on by default and users should use the CA and authentication, the container shouldn't fail if one of the TLS config option has been explicitly disabled.

How to test this PR

Tests come with the PR.

Follow-ups

Fixing this would allow to finish the adaptation of the go-elasticsearch client integration tests to testcontainers!

@Anaethelion Anaethelion requested a review from a team as a code owner April 9, 2024 15:52
Copy link

netlify bot commented Apr 9, 2024

Deploy Preview for testcontainers-go ready!

Name Link
🔨 Latest commit 186156d
🔍 Latest deploy log https://app.netlify.com/sites/testcontainers-go/deploys/66156443ceabe70008ae7c0b
😎 Deploy Preview https://deploy-preview-2475--testcontainers-go.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@Anaethelion Anaethelion changed the title Elasticsearch disable ca without ssl Elasticsearch disable CA retrieval when ssl is disabled Apr 9, 2024
Copy link
Member

@mdelapenya mdelapenya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!!

@mdelapenya
Copy link
Member

Once the CI passes, I'll merge this one. Thank you!

@mdelapenya mdelapenya self-assigned this Apr 9, 2024
@mdelapenya mdelapenya added the enhancement New feature or request label Apr 9, 2024
@mdelapenya mdelapenya merged commit a3ff7aa into testcontainers:main Apr 9, 2024
102 checks passed
@Anaethelion Anaethelion deleted the elasticsearch_disable_ca_without_ssl branch April 10, 2024 10:27
mdelapenya added a commit to coffeegoddd/testcontainers-go that referenced this pull request Apr 12, 2024
* main: (115 commits)
  chore: create TLS certs in a consistent manner (testcontainers#2478)
  chore(deps): bump idna from 3.6 to 3.7 (testcontainers#2480)
  Elasticsearch disable CA retrieval when ssl is disabled (testcontainers#2475)
  fix: handle dockerignore exclusions properly (testcontainers#2476)
  chore: prepare for next minor development cycle (0.31.0)
  chore: use new version (v0.30.0) in modules and examples
  Fix url creation to handle query params when using HTTP wait strategy (testcontainers#2466)
  fix: data race on container run (testcontainers#2345)
  fix: logging deadlock (testcontainers#2346)
  feat(k6):Add remote test scripts (testcontainers#2350)
  feat: optimizes file copies to and from containers (testcontainers#2450)
  fix(exec): updates the `Multiplexed` opt to combine stdout and stderr (testcontainers#2452)
  Upgrade neo4j module to use features from v0.29.1 of testcontainers-go (testcontainers#2463)
  bug:Fix AMQPS url (testcontainers#2462)
  chore: more compose updates in comments
  chore: use "docker compose" (v2) instead of "docker-compose" (v1) (testcontainers#2464)
  chore(deps): bump github/codeql-action from 2.22.12 to 3.24.9 (testcontainers#2459)
  refactor: Add Weaviate modules tests (testcontainers#2447)
  feat(exitcode): Add exit code sugar method (testcontainers#2342)
  feat: add module to support InfluxDB v1.x (testcontainers#1703)
  ...
mdelapenya pushed a commit to mdelapenya/testcontainers-go that referenced this pull request Apr 23, 2024
…rs#2475)

* skip search for CACert if ssl has been turned off

* add tests with and without ssl enabled

* add all config keys that disable CA gen, restrict check to version 8

* rename test to match content
mdelapenya added a commit to mdelapenya/testcontainers-go that referenced this pull request Apr 23, 2024
* main:
  fix: don't retry on permanent APIClient errors (testcontainers#2506)
  feat: support overriding the default recreate options for compose (testcontainers#2511)
  feat: support passing io.Reader for compose files when creating a compose instance (testcontainers#2509)
  chore: add funding button for testcontainers (testcontainers#2510)
  feat: support Ryuk for the compose module (testcontainers#2485)
  chore(deps): bump golang.org/x/net in modules (minio, gcloud, weaviate, compose, qdrant, couchbase, k3s, milvus, mockserver, pulsar, kafka) (testcontainers#2505)
  fix: fallback to URL-path when parsing auth config URL without scheme (testcontainers#2488)
  fix(postgres): Fix the non-default dbname error (testcontainers#2489)
  feat: Bump default postgres version (testcontainers#2481)
  support Dolt (testcontainers#2177)
  chore: create TLS certs in a consistent manner (testcontainers#2478)
  chore(deps): bump idna from 3.6 to 3.7 (testcontainers#2480)
  Elasticsearch disable CA retrieval when ssl is disabled (testcontainers#2475)
  fix: handle dockerignore exclusions properly (testcontainers#2476)
mdelapenya added a commit that referenced this pull request Apr 24, 2024
* chore: start a foundational package for interacting with Docker networks

* feat: add an SSH tunnel forwarding a host port to a container

* fix: rename struct

* chore: pass the original context to the exposeHostPorts function

* chore: start tunnel using context

* chore: push goroutines to the method where they are used

* fix: proper eval of first network

* fix: handle dockerignore exclusions properly (#2476)

* chore: only include the dockerignore if it contains ignore files

* fix: the inclusions must be relative to the context

* docs: document the dockerignore feature

* chore: only include the dockerignore file if it exists

* Elasticsearch disable CA retrieval when ssl is disabled (#2475)

* skip search for CACert if ssl has been turned off

* add tests with and without ssl enabled

* add all config keys that disable CA gen, restrict check to version 8

* rename test to match content

* chore(deps): bump idna from 3.6 to 3.7 (#2480)

Bumps [idna](https://github.com/kjd/idna) from 3.6 to 3.7.
- [Release notes](https://github.com/kjd/idna/releases)
- [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst)
- [Commits](kjd/idna@v3.6...v3.7)

---
updated-dependencies:
- dependency-name: idna
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: create TLS certs in a consistent manner (#2478)

* fix: remove suspicious filepath.Join

* chore: fix lint

* fix: handle error

* chore: reverse assertion for lint

* feat: support generating TLS certificates on the fly

* chore: apply to cockroachdb

* chore: support saving the cert and priv key files to disk

* chore: apply to rabbitmq

* chore: simplify

* chore: use in redpanda module

* chore: lint

* chore: set validFrom internally

* fix: properly use the new API in redpanda

* docs: document the TLS helpers

* chore: simplify WithParent to accept the struct directly

* chore: use tlscert package instead

* fix: use non-deprecated API

* docs: update

* docs: fix examples

* chore: use released version of tlscert

* fix: add common name for the node cert

* support Dolt (#2177)

* /modules/dolt: wip, kinda working

* /modules/dolt: get tests passing

* /{.github,.vscode,docs,mkdocs,modules,sonar-project}: use modulegen tool

* /modules/dolt/{dolt.go,examples_test.go}: run linter

* /modules/dolt/{dolt.go,examples_test.go}: add methods for cloning

* /{docs, modules}: add with creds file

* /{docs,modules}: pr feedback, cleanup

* /modules/dolt/examples_test.go: remove panics, lint

* chore: run mod tidy

* chore: include MustConnectionString method

* chore: do not use named returns

* chore: perform initialisation before the container has started

---------

Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

* feat: Bump default postgres version (#2481)

* Bump default postgres version

* Bump to use latest pg

* Bump version from non-ancient version

---------

Co-authored-by: bstrausser <bstrausser@locusrobotics.com>

* fix(postgres): Fix the non-default dbname error (#2489)

* Fix the non-default dbname error

The linked issue described in great detail an issue where we assumed everyone would use the default database user, whose home DB defaults to the postgres database. When that was not the case, the snapshots would fail silently as the user would not connect to the right database to take the commands.

This PR fixes the issue by adding the dbname by default in the command, and adds a test to validate this works as intended. In addition, it also adds some logic to handle any error that does not cause the exec command to fail, such as database access failures.

Run the added test to test this works as intended.

Closes #2474

* Document the postgres dbname issue in the docs

* fix: fallback to URL-path when parsing auth config URL without scheme (#2488)

* chore(deps): bump golang.org/x/net in modules (minio, gcloud, weaviate, compose, qdrant, couchbase, k3s, milvus, mockserver, pulsar, kafka) (#2505)

* chore(deps): bump golang.org/x/net in /modules/kafka

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0.
- [Commits](golang/net@v0.17.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump golang.org/x/net in /modules/pulsar

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0.
- [Commits](golang/net@v0.17.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump golang.org/x/net in /modules/mockserver

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0.
- [Commits](golang/net@v0.17.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump golang.org/x/net in /modules/milvus

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0.
- [Commits](golang/net@v0.17.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump golang.org/x/net from 0.19.0 to 0.23.0 in /modules/k3s

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.19.0 to 0.23.0.
- [Commits](golang/net@v0.19.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump golang.org/x/net in /modules/couchbase

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0.
- [Commits](golang/net@v0.20.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump golang.org/x/net in /modules/qdrant

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0.
- [Commits](golang/net@v0.20.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump golang.org/x/net in /modules/compose

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0.
- [Commits](golang/net@v0.20.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump golang.org/x/net in /modules/weaviate

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0.
- [Commits](golang/net@v0.20.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump golang.org/x/net in /modules/gcloud

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.21.0 to 0.23.0.
- [Commits](golang/net@v0.21.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump golang.org/x/net in /modules/minio

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.21.0 to 0.23.0.
- [Commits](golang/net@v0.21.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: support Ryuk for the compose module (#2485)

* feat: add testcontainers labels to compose containers

* feat: support reaper for compose

* chore: increase ryuk reconnection timeout on CI

* chore: cache containers on UP

* chore: more tuning for compose

* chore: more consistent assertion

* chore: the compose stack asks for the reaper, but each container then connects to it

* chore: use different error groups

the first time wait is called, the context is cancelled

* chore: the lookup method include cache checks

* chore: update tests to make them deterministic

* chore: rename local compose testss

* chore: support returning the dynamic port in the helper function

* chore: try with default reconnection timeout

* feat: support removing networks from compose

* chore: support naming test services with local and api

It will allow the tests to be more deterministic, as there could be service containers started from the local test suite with the same name as in the API test suite.

* Revert "chore: try with default reconnection timeout"

This reverts commit 336760c.

* fix: typo

* chore: add funding button for testcontainers (#2510)

* feat: support passing io.Reader for compose files when creating a compose instance (#2509)

* feat: support passing io.Reader when creating a compose instance

* docs: change title

* feat: support overriding the default recreate options for compose (#2511)

* feat: support overriding the default recreate options for compose

* chore: validate recreation values

* fix: don't retry on permanent APIClient errors (#2506)

* fix: don't retry on permanent APIClient errors

* fix: add more tests for un-retryable scenarios

* chore: run mod tidy

* chore: implement the port-forwarding correctly

* chore: use new sshd image

* chore: simplify channel creation to avoid allocations

* fix: do not leak goroutines

Detected with go.uber.org/goleak

* chore: expose host internal constant

* fix: update variables

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Laurent Saint-Félix <laurent.saintfelix@elastic.co>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dustin Brown <dustin@dolthub.com>
Co-authored-by: Barrett Strausser <bearrito@users.noreply.github.com>
Co-authored-by: bstrausser <bstrausser@locusrobotics.com>
Co-authored-by: Guillaume St-Pierre <guillaume.stpierre@docker.com>
Co-authored-by: Patrick Jahn <33724206+p-jahn@users.noreply.github.com>
mdelapenya added a commit to stevenh/testcontainers-go that referenced this pull request Apr 24, 2024
* main: (22 commits)
  feat: forward host ports to a container using an SSH tunnel (testcontainers#2471)
  Update follow_logs.md with adding missing package (testcontainers#2513)
  fix: don't retry on permanent APIClient errors (testcontainers#2506)
  feat: support overriding the default recreate options for compose (testcontainers#2511)
  feat: support passing io.Reader for compose files when creating a compose instance (testcontainers#2509)
  chore: add funding button for testcontainers (testcontainers#2510)
  feat: support Ryuk for the compose module (testcontainers#2485)
  chore(deps): bump golang.org/x/net in modules (minio, gcloud, weaviate, compose, qdrant, couchbase, k3s, milvus, mockserver, pulsar, kafka) (testcontainers#2505)
  fix: fallback to URL-path when parsing auth config URL without scheme (testcontainers#2488)
  fix(postgres): Fix the non-default dbname error (testcontainers#2489)
  feat: Bump default postgres version (testcontainers#2481)
  support Dolt (testcontainers#2177)
  chore: create TLS certs in a consistent manner (testcontainers#2478)
  chore(deps): bump idna from 3.6 to 3.7 (testcontainers#2480)
  Elasticsearch disable CA retrieval when ssl is disabled (testcontainers#2475)
  fix: handle dockerignore exclusions properly (testcontainers#2476)
  chore: prepare for next minor development cycle (0.31.0)
  chore: use new version (v0.30.0) in modules and examples
  Fix url creation to handle query params when using HTTP wait strategy (testcontainers#2466)
  fix: data race on container run (testcontainers#2345)
  ...
mdelapenya added a commit to mdelapenya/testcontainers-go that referenced this pull request Apr 26, 2024
* main: (34 commits)
  break: return error from Customize request option (testcontainers#2267)
  fix: wrong copy paste (testcontainers#2515)
  docs: add documentation for Exec method (testcontainers#2451)
  docs: document the SSHd tunnel (testcontainers#2514)
  fix: enhance host configuration port binding (testcontainers#2512)
  feat: forward host ports to a container using an SSH tunnel (testcontainers#2471)
  Update follow_logs.md with adding missing package (testcontainers#2513)
  fix: don't retry on permanent APIClient errors (testcontainers#2506)
  feat: support overriding the default recreate options for compose (testcontainers#2511)
  feat: support passing io.Reader for compose files when creating a compose instance (testcontainers#2509)
  chore: add funding button for testcontainers (testcontainers#2510)
  feat: support Ryuk for the compose module (testcontainers#2485)
  chore(deps): bump golang.org/x/net in modules (minio, gcloud, weaviate, compose, qdrant, couchbase, k3s, milvus, mockserver, pulsar, kafka) (testcontainers#2505)
  fix: fallback to URL-path when parsing auth config URL without scheme (testcontainers#2488)
  fix(postgres): Fix the non-default dbname error (testcontainers#2489)
  feat: Bump default postgres version (testcontainers#2481)
  support Dolt (testcontainers#2177)
  chore: create TLS certs in a consistent manner (testcontainers#2478)
  chore(deps): bump idna from 3.6 to 3.7 (testcontainers#2480)
  Elasticsearch disable CA retrieval when ssl is disabled (testcontainers#2475)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants