Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable ssl between logstash and agent #1669

Merged
merged 7 commits into from
Feb 14, 2024
Merged

Conversation

bhapas
Copy link
Contributor

@bhapas bhapas commented Feb 9, 2024

Currently ssl is disabled between logstash and agent when running on local stack. This PR enables ssl communication between logstash and agent.

Closes #1658

@bhapas
Copy link
Contributor Author

bhapas commented Feb 13, 2024

There are two approaches to add the client certificates when fleet output is added through config kibana.yml

@bhapas bhapas force-pushed the enable_ssl branch 2 times, most recently from 6ed276d to 3cd61cd Compare February 13, 2024 13:31
@bhapas
Copy link
Contributor Author

bhapas commented Feb 13, 2024

  • Importing the certificates generated locally into the kibana config using facters. But this is also blocked as Fleet is removing new line in the ssl certificates leading to error in elastic-agent

A workaround to fix the indentation in the yaml config - https://github.com/elastic/elastic-package/pull/1669/files#diff-9fd733fc9d8f10a5c2dd55a062523d3dcb0655bc1db097648b3adabbd3460173R206

@bhapas
Copy link
Contributor Author

bhapas commented Feb 13, 2024

/test

# logstash expects the key in pkcs8 format. Hence converting the key.pem to pkcs8 format using openssl.
# Also logstash-filter-elastic_integration plugin is installed by default to run ingest pipelines in logstash.
# elastic-package#1637 made improvements to enable logstash stats through port 9600.
command: bash -c 'openssl pkcs8 -inform PEM -in /usr/share/logstash/config/certs/key.pem -topk8 -nocrypt -outform PEM -out /tmp/logstash.pkcs8.key && chmod +x /tmp/logstash.pkcs8.key && if [[ ! $(bin/logstash-plugin list) == *"logstash-filter-elastic_integration"* ]]; then echo "Missing plugin logstash-filter-elastic_integration, installing now" && bin/logstash-plugin install logstash-filter-elastic_integration; fi && bin/logstash -f /usr/share/logstash/pipeline/logstash.conf'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shall be extracted to a script in #1668

internal/stack/certs.go Show resolved Hide resolved
@@ -171,6 +184,45 @@ func applyResources(profile *profile.Profile, stackVersion string) error {
return nil
}

func addClientCertsToResources(resourceManager *resource.Manager, certResources []resource.Resource) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use an approach like this one for serverless? So we provide the certificates when creating the output and we don't need to update it later?

Copy link
Contributor Author

@bhapas bhapas Feb 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But for serverless the certificates are created at a later stage than adding the agent policy. And to add agent policy the output needs to be created

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could move applyServerlessResources to an early stage if needed. It is only now in startLocalServices because so far it was only needed for local services. But we could run this earlier if needed to create the project.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will try this in another PR

internal/stack/resources.go Outdated Show resolved Hide resolved
internal/stack/resources.go Outdated Show resolved Hide resolved
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

@bhapas bhapas merged commit ff60d25 into elastic:main Feb 14, 2024
3 checks passed
@bhapas bhapas deleted the enable_ssl branch February 14, 2024 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[logstash] Enable ssl for communication between agent and logstash in local stack
3 participants