Elastic 8.x charts?

[agonzalezm]

When are elastic 8.x charts being released?. I am trying to deploy 8.0.1 cluster but failing with current charts. Why they are not released same time than production release of elastic?

[jmlrt]

Hi @agonzalezm,
Elastic stack 8.x products are introducing some important changes like the security by default for Elasticsearch and we are working toward making these changes compatible with the Helm charts. The 8.x release for charts will be published when it will be ready.

[agonzalezm]

meantime is there any beta or something that can be tested and works at least for elasticsearch and kibana? not all product.

[jmlrt]

The charts from main branch is working for Elasticsearch 8.x but Kibana is still broken for now.

[agonzalezm]

can you update here when kibana is also working? thanks!

[jasonyic]

should the latest chart be 7.17.1, instead of 8.1.0?

[agonzalezm]

elastic 8.1 is already released, any update when elastic and kibana 8.x can be deployed with these charts?

[jmlrt]

Sorry, we don't have any update for now.

[agonzalezm]

Hello! any update of charts for elasticsearch and kibana 8.x? still kibana not working?

[fajulleiro]

Yes please, we really need this new version. Thanks

[rotemsh15]

What is the latest stable version of the charts?

[agonzalezm]

are these charts going to be updated on time or should we move to use ECK operator?

[jmlrt]

What is the latest stable version of the charts?

https://github.com/elastic/helm-charts/releases/tag/v7.17.1 (should also work with 7.17.2 imageTag but not with any 8.x version.

are these charts going to be updated on time or should we move to use ECK operator?

We are working toward fixing Kibana chart for 8.x, however we don't have any release date.

Meanwhile, ECK is still the recommended way to deploy Elastic products on Kubernetes as mentioned in the helm-charts repo README. We highly recommend taking a look at it and using it if that's an option for you as it is a lot more mature than the Helm charts now. Indeed it benefits from a real programming language for the operator instead of a simple templating language for Helm charts, in addition to a development team fully dedicated to maintaining it.

[NeckBeardPrince]

I have a feeling that the Helm charts are no longer going to be maintained. The main branch hasn't been updated in 28 days, and the last time a PR was closed/merged was 20 days ago. Elastic is also pushing heavy on the operator, which is fine. Just wish some kind of official communication would be given.

[rgarcia89]

What is the latest stable version of the charts?

https://github.com/elastic/helm-charts/releases/tag/v7.17.1 (should also work with 7.17.2 imageTag but not with any 8.x version.

are these charts going to be updated on time or should we move to use ECK operator?

We are working toward fixing Kibana chart for 8.x, however we don't have any release date.

Meanwhile, ECK is still the recommended way to deploy Elastic products on Kubernetes as mentioned in the helm-charts repo README. We highly recommend taking a look at it and using it if that's an option for you as it is a lot more mature than the Helm charts now. Indeed it benefits from a real programming language for the operator instead of a simple templating language for Helm charts, in addition to a development team fully dedicated to maintaining it.

I would be totally fine with using ECK if it would not require a different license, which is btw more expensive...

[stasctopi]

I would be totally fine with using ECK if it would not require a different license, which is btw more expensive...

Very true, I don't understand why the recommended and apparently more maintained way of deployment isn't available for all license tiers.

[tranvansang]

Can anyone tell me where to confirm the license requirement of this Elastic Cloud on Kubernetes (ECK)?
As I check at its github homepage, it doesn't mention anything about license requirements.

https://github.com/elastic/cloud-on-k8s

I would like to know the requirement and decide to give it up before spending time on the setup.

[agonzalezm]

I think ECK is also available for free tier (basic license).

When you install the default distribution of ECK, you receive a Basic license. Any Elastic stack application you manage through ECK will also be Basic licensed

[tranvansang]

I think ECK is also available for free tier (basic license).

When you install the default distribution of ECK, you receive a Basic license. Any Elastic stack application you manage through ECK will also be Basic licensed

And with this repo helm chart, everything is free. Am I correct?

[ztdietmaier]

Can anyone tell me where to confirm the license requirement of this Elastic Cloud on Kubernetes (ECK)?

https://www.elastic.co/de/subscriptions -- search for "kubernetes". ECK is not available for gold and platinum subscriptions.

[stasctopi]

Can anyone tell me where to confirm the license requirement of this Elastic Cloud on Kubernetes (ECK)? As I check at its github homepage, it doesn't mention anything about license requirements.

https://github.com/elastic/cloud-on-k8s

I would like to know the requirement and decide to give it up before spending time on the setup.

Here https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-licensing.html

Its available for free, basic and enterprise users

We are running platinum so no soup for us 🤷🏻‍♂️

[rgarcia89]

@stasctopi same here 👎

[ppawel]

Oof, this does not look good. We are looking at alternatives, e.g. https://github.com/grafana/loki.

[tranvansang]

Oof, this does not look good. We are looking at alternatives, e.g. https://github.com/grafana/loki.

Thank you very much for the suggestion. I didn't know about this project before.

I am building a minimal vendor-agnostic infra with k8s. The final part is the logging (I was also considering cloudfront but not satisfy with the vendor-lock). Grafana loki seems best fit.

<3

[sagar-raythatha]

waiting for the release for Elasticsearch 8.x.x , Do we have any dates, when will it be released?

[c4road]

Please fix the README.md of elasticsearch chart and do not refer to do this helm install elasticsearch ./helm-charts/elasticsearch --set imageTag=8.1.0 there, link this issue or make a disclaimer with the proper way of installing latest version (I think it is ECK). Since it suggest that 8.1 support is available. It will save countless hours of debugging.

[jmlrt]

Please fix the README.md of elasticsearch chart and do not refer to do this helm install elasticsearch ./helm-charts/elasticsearch --set imageTag=8.1.0 there, link this issue or make a disclaimer with the proper way of installing latest version (I think it is ECK). Since it suggest that 8.1 support is available. It will save countless hours of debugging.

@c4road sorry for your bad experience here. The Elasticsearch Readme on the main branch has already a disclaimer mentioning that it should be used for development versions only:

The README from the last released version (https://github.com/elastic/helm-charts/blob/v7.17.3/elasticsearch/README.md) is mentioning 7.17.3 version as expected.

On the Artifact Hub page (https://artifacthub.io/packages/helm/elastic/elasticsearch) the instructions are also correct:

That being said, I agree that it can lead to confusion. With our current workflow where main branch is for the development of the next version and only tagged commit are used for the released versions, we didn't find a better way to document it.

[BenB196]

Just posting this here in case anyone has the same question I did. Since Logstash isn't supported by ECK, you can still you the 7.17.3 to deploy the 8.x Logstash versions, you'll just need to change the tag to use the version you want. I test 8.2.2 Logstash against the 7.17.3 and from what I can tell, everything functioned normally.

[christianhuening]

i find it a bit disconcerting that 1) ECK appears not to support all things and 2) months after the 8.x release there still are no published helm charts and no update on when we might get them. i do not have a good feeling...

[stasctopi]

Another month and still no chart, can we get any update?

[darthf1]

Or a migration guide how to migrate to the operator? 😊

[NeckBeardPrince]

Is it really that difficult to give your users an update? If you were in our shoes, wouldn't you want some kind of update?

[djarami726]

I've literally spent days trying to get an 8.x working and it is all over the place. I'm using the examples in the helm-charts repo and read the docuentation and I still can't come up with a working combination.

using this as a starting config and get a copy of it locally.
https://github.com/elastic/helm-charts/blob/main/elasticsearch/examples/minikube/values.yaml
helm install elasticsearch elastic/elasticsearch -f ./values.yaml --set imageTag=8.1.0

I then get an exception:
java.lang.IllegalArgumentException: unknown setting [node.ml] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:563)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:509)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:479)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:449)
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:151)
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:56)
at org.elasticsearch.node.Node.(Node.java:447)
at org.elasticsearch.node.Node.(Node.java:284)

I go through the docs with regards to node.roles with 8.x and I put this in the values.yaml:
node.roles: [ master ] and get this exception
java.lang.IllegalArgumentException: unknown setting [node.master] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:563)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:509)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:479)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:449)
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:151)
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:56)
at org.elasticsearch.node.Node.(Node.java:447)
at org.elasticsearch.node.Node.(Node.java:284)

Also tried the 7.x style and get the next exception
roles:
master: "true"
java.lang.IllegalArgumentException: unknown setting [node.ml] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:563)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:509)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:479)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:449)
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:151)
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:56)
at org.elasticsearch.node.Node.(Node.java:447)

Then I tried this and got the same exception
roles:
master: "true"
ml: "false"
java.lang.IllegalArgumentException: unknown setting [node.ml] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:563)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:509)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:479)
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:449)
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:151)
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:56)
at org.elasticsearch.node.Node.(Node.java:447)
at org.elasticsearch.node.Node.(Node.java:284)

Is there a working scenarion for helm charts with 8.x - please point me to a working example - not the documentation because that doesn't work. Documentation without examples is not very helpful.

[ppawel]

We plan to switch to OpenSearch and never look back. Elastic just wants to milk money from you for subscriptions since they went public while the product quality goes down the drain. Good luck with that.

[nkammah]

Sorry for the delay! We've started breaking up the tasks in the background — issues for elastic/beats#31930 and elastic/kibana#134096, which needs to switch to Service Accounts and requires some bigger changes. As always, contributions are very welcome!

This will be the last release for 8.x. Elastic will focus on the Kubernetes Operator going forward.

If there is interest from the community to keep the Helm charts alive, we would be more than happy to let someone take ownership. The Elasticsearch Puppet module is a great example for this.

Please, let us know if you would be interested and we're happy to work with you on that transition.

[darthf1]

Sorry for the delay! We've started breaking up the tasks in the background — issues for elastic/beats#31930 and elastic/kibana#134096, which needs to switch to Service Accounts and requires some bigger changes. As always, contributions are very welcome!

This will be the last release for 8.x. Elastic will focus on the Kubernetes Operator going forward.

If there is interest from the community to keep the Helm charts alive, we would be more than happy to let someone take ownership. The Elasticsearch Puppet module is a great example for this.

Please, let us know if you would be interested and we're happy to work with you on that transition.

Hi @nkammah, since this helm chart will be deprecated; can you provide a step by step guide how to migrate from this helm chart to the k8s operator?

[ztdietmaier]

This will be the last release for 8.x. Elastic will focus on the Kubernetes Operator going forward.

@nkammah will the operator also be usable with platinum licenses?

[djarami726]

Sorry for the delay! We've started breaking up the tasks in the background — issues for elastic/beats#31930 and elastic/kibana#134096, which needs to switch to Service Accounts and requires some bigger changes. As always, contributions are very welcome!
This will be the last release for 8.x. Elastic will focus on the Kubernetes Operator going forward.
If there is interest from the community to keep the Helm charts alive, we would be more than happy to let someone take ownership. The Elasticsearch Puppet module is a great example for this.
Please, let us know if you would be interested and we're happy to work with you on that transition.

Hi @nkammah, since this helm chart will be deprecated; can you provide a step by step guide how to migrate from this helm chart to the k8s operator?

Yes, please. I've gone through all the docs available on this on the elastic documentation and it's not enough to get going with. Maybe some suggested links/pointers on how to get started would be helpful.

[NeckBeardPrince]

This will be the last release for 8.x. Elastic will focus on the Kubernetes Operator going forward.

@nkammah will the operator also be usable with platinum licenses?

@nkammah @ztdietmaier This is a critical question and really needs an answer. I feel like you can't deprecate a deployment method for another if the another doesn't have parity to the one being deprecated. This effects paying customers and users.

[ebuildy]

hello @djarami726, we have just switch to last version 8.3.3, using elastic Helm charts, and after disabled security.

It works straighaway like 7.X versions. (elasticsearch + kibana).

The bug you have should not be here with latest Helm charts version: https://github.com/elastic/helm-charts/blob/main/elasticsearch/templates/statefulset.yaml#L317 there is no more node.ml env var.

So please use latest version, even with elasticsearch 8.3.3, this will work.

[tuxerrante]

Hi,
while waiting for the 8 release it would be nice to have critical CVEs fixed.
Currently Grype detects NVD - CVE-2021-29281 (nist.gov) due to an old "archiver" npm package

They have already referenced ^5.3.1 in the main branch kibana/package.json at main · elastic/kibana (github.com)
While in 7.17.6 is still 5.2.0 kibana/package.json at v7.17.6 · elastic/kibana (github.com)

Would that be possible thanks?

[xeraa]

@tuxerrante

1. Security issues should be reported as described on https://www.elastic.co/community/security
2. This would be a discussion for Kibana and is unrelated to the Helm Charts. The upgrade is in main, but wasn't backported to 7.x
3. I have a hunch about the quality of your vulnerability scanner...

[NeckBeardPrince]

This will be the last release for 8.x. Elastic will focus on the Kubernetes Operator going forward.

@nkammah will the operator also be usable with platinum licenses?

@nkammah @ztdietmaier This is a critical question and really needs an answer. I feel like you can't deprecate a deployment method for another if the another doesn't have parity to the one being deprecated. This effects paying customers and users.

It's so confusing why nobody has responded to this.

[tuxerrante]

3. I have a hunch about the quality of your vulnerability scanner...

@xeraa It's the official successor of Anchore.. not exactly "mine" ;)

[gnauhnoj]

Just posting this here in case anyone has the same question I did. Since Logstash isn't supported by ECK, you can still you the 7.17.3 to deploy the 8.x Logstash versions, you'll just need to change the tag to use the version you want. I test 8.2.2 Logstash against the 7.17.3 and from what I can tell, everything functioned normally.

@BenB196 sorry for the very delayed ping. I tested this out and wasn't able to get things to work because of the deprecated node.<role> properties. Any tips for getting around that? The roles key in the top level template didn't seem to work in 7.17.3

[jmlrt]

👋🏻 FYI, 8.5.1 just got released. We also published an official communication in #1731 about the future of this repository.