Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ci): us sh instead of bash; increase tag by patch #190

Merged
merged 1 commit into from
Oct 26, 2022

Conversation

szalai1
Copy link
Contributor

@szalai1 szalai1 commented Oct 26, 2022

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable)

@szalai1 szalai1 requested a review from pedro93 October 26, 2022 12:29
@szalai1 szalai1 merged commit 5a68a62 into master Oct 26, 2022
@szalai1 szalai1 deleted the fix-tag-to-patch branch October 26, 2022 12:31
sb-sebkim added a commit to sendbird/datahub-helm that referenced this pull request Nov 21, 2022
* upgrading to 8.40 (acryldata#139)

* Option to add podAnnotations to jobs. (acryldata#137)

* feat(helm): add pod annotations to jobs

* feat(helm): add pod annotation to cron jobs.

* docs(helm): Add podAnnotation documentation and default values.

* feat(helm) chart version bump.

* update to v0.8.41 (acryldata#143)

* fix(mysql): update mysql helm chart version (acryldata#146)

* fix(metrics): fix Prometheus env variable in the MCE consumer deployment (acryldata#148)

* feat(datahub-frontend): Option to configure OIDC auth easily (acryldata#142)

* fix(kafka): update kafka chart version (acryldata#147)

* feat(datahub-frontend): Allow lifecycle hooks (acryldata#141)

* <feat>(datahub-frontend): Allow lifecycle hooks

This PR adds a lifecycle node to the datahub-frontend deployment template.
It can be used to execute lifecycle hooks, e.g. postStart actions, like
replacing the default user.props file.

* Bump chart version to 0.2.86

Co-authored-by: Pedro Silva <pedro.cls93@gmail.com>

* Update to v0.8.43 (acryldata#151)

* feat(upgrade): allow setting batch args (acryldata#155)

* feat(datahub-upgrade): Restore Indices Resources (acryldata#152)

* <feat>(datahub-upgrade): Restore Indices Resources

This PR adds the ability for end-users to specify resource requests and limits
for the datahub upgrade job 'restore indices'. This is needed for situations
where the restore indices job will OOM due to a large backfill of data. I've
seen this happening with our deployment of datahub and am unable to tune
the resources for the restore indices job because the current requests
and limits are hard-coded.

* fix(gms): remove unused JMX sidecar (acryldata#156)

* fix(gms): remove unused JMX sidecar
in favour of the already included JVM agent

* Update Chart.yaml

* Update Chart.yaml

* Bumping to v0.8.44 (acryldata#160)

* feat(ci): close stale issues/prs (acryldata#164)

* feat: add extra labels to ingress configurations (acryldata#163)

* feat(prerequisites): add postgresql to prerequistes Helm chart as an optional dependency (acryldata#121)

* feat(charts): add an option to configure revisionHistoryLimit to subcharts (acryldata#159)

* feat: datahub-frontend add oidc authentication client secret reference (acryldata#165)

* fix(monitoring) Fixes monitoring default values + gms chart version (acryldata#161)

* chore: bump datahub-actions version (acryldata#168)

* Feat(actions): Support mounting a customer defined k8s secrets as files needed in ingestion recipes (acryldata#169)

* Mount a customer defined k8s secret for ssl secrets needed in ingestion recipes

* Bump chart versions

* Address comment

* Update readme

* Address Pedro's comment

* feat(monitoring): enable monitoring on frontend (acryldata#171)

* Add kafka-exporter to expose prometheus metrics for kafka cluster (acryldata#170)

* Add kafka-exporter to expose prometheus metrics for kafka cluster

* Kafka-exporter: Updated values.yaml to support tls/sasl authentication while connecting kafka cluster

* Kafka-exporter: Updated values.yaml to remove default values and correct intendation

* Add kafka-exporter to expose prometheus metrics for kafka cluster

* Kafka-exporter: Updated values.yaml to support tls/sasl authentication while connecting kafka cluster

* Kafka-exporter: Updated values.yaml to remove default values and correct intendation

* Updated chart version

* Bumping actions (acryldata#173)

* feat(elasticsearch-setup): Add support for insecure curl connections (acryldata#166)

* fix versions, clean jmx exporter (acryldata#175)

* bumping helm charts for 0.8.45 (acryldata#178)

* bumping to 0.8.45

* Fixing

* feat(monitoring): Add optional ServiceMonitors to all JVM based services (acryldata#181)

* feat(frontend): add service monitoring

* feat(gms): add service monitoring

* gms: servicemonitor

* feat(mae): enable service monitoring

* update READMEs

* fix service ports

* bump version

* Add missing default value

* Update charts/datahub/Chart.yaml

* delete jmx config and rename port to http

* feat(secrets): Allow users to specify their own secret values at first boot (acryldata#180)

* fix(auth-secret): Make template use correct retrieval method

`datahub-auth-secret` yaml was incorrectly using the `index` template function which is for list variables. For dictionaries as is the case with the `$secret.data` object, the correct method is: https://helm.sh/docs/chart_template_guide/function_list/#get

Tested locally, that this works against a running cluster.

* Updates encryption secret to use get helm function

* feat(secrets): Allow users to specify their own secret values at first boot

* fix(datahub-frontend): Remove unused env var (acryldata#176)

* fix(datahub-frontend): Remove unused env var

* Bump chart version

* Change default versions 0.9.0 (acryldata#184)

* feat(release): automatically bumbp versions for all subcharts (acryldata#185)

* feat(release): automatically bumbp versions for all subcharts

* Remove kafka export as subchart
Kafka exporter is useful for monitoring datahub, but it's not part of it. It should be deploy sepratly, as we do in production

* Add script to bump versions together

* Add tagging to the workflow

* Don't run release on CI commits

* fix(CI): remove duplicate id (acryldata#189)

* fix(ci): us sh instead of bash; increase tag by patch (acryldata#190)

* DataHub Release v0.9.1  (acryldata#191)

* fix(v0.9.1): Fix v0.9.1 release versions (acryldata#192)

* Release v0.9.1 Helm

* Bump chart versions

* Fixing final 210

* Bumping to release Helm charts for v0.9.2

* stg actions image upgrade

* service annotations scp applied

* filters deprecated in favor of orFilters

* filters deprecated in favor of orFilters version applied in yaml

* release(v0.9.2) Release Helm Charts for DataHub v0.9.2 (acryldata#194)

* Bumping to release Helm charts for v0.9.2

* bump version by running:
 ./.github/scripts/bump-chart-versions.sh 0.2.112

Co-authored-by: szalai1 <szalaipeti.vagyok@gmail.com>

* fix(gha): revert auto bump release changes (acryldata#197)

Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Lukáš Novotný <42157644+novotl@users.noreply.github.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
Co-authored-by: Justin Marozas <justinas.marozas@gmail.com>
Co-authored-by: Tomáš Kubín <tomas.kubin@blindspot.ai>
Co-authored-by: Toby Irmer <toby@irmer.com>
Co-authored-by: Pedro Silva <pedro.cls93@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Hunter Elbourn <hunterelbourn@gmail.com>
Co-authored-by: Peter Szalai <szalaipeti.vagyok@gmail.com>
Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: Upendra Rao Vedullapalli <upendrao@gmail.com>
Co-authored-by: Felix Lüdin <13187726+Masterchen09@users.noreply.github.com>
Co-authored-by: Bumsoo Kim <bskim45@gmail.com>
Co-authored-by: Tony Ouyang <43738225+TonyOuyangGit@users.noreply.github.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Jinlin Yang <86577891+jinlintt@users.noreply.github.com>
Co-authored-by: Navin Sharma <103643430+NavinSharma13@users.noreply.github.com>
Co-authored-by: Bogdan Antoniu <bogdan.antoniu@gmail.com>
sb-sebkim added a commit to sendbird/datahub-helm that referenced this pull request Jan 31, 2023
* upgrading to 8.40 (acryldata#139)

* Option to add podAnnotations to jobs. (acryldata#137)

* feat(helm): add pod annotations to jobs

* feat(helm): add pod annotation to cron jobs.

* docs(helm): Add podAnnotation documentation and default values.

* feat(helm) chart version bump.

* update to v0.8.41 (acryldata#143)

* fix(mysql): update mysql helm chart version (acryldata#146)

* fix(metrics): fix Prometheus env variable in the MCE consumer deployment (acryldata#148)

* feat(datahub-frontend): Option to configure OIDC auth easily (acryldata#142)

* fix(kafka): update kafka chart version (acryldata#147)

* feat(datahub-frontend): Allow lifecycle hooks (acryldata#141)

* <feat>(datahub-frontend): Allow lifecycle hooks

This PR adds a lifecycle node to the datahub-frontend deployment template.
It can be used to execute lifecycle hooks, e.g. postStart actions, like
replacing the default user.props file.

* Bump chart version to 0.2.86

Co-authored-by: Pedro Silva <pedro.cls93@gmail.com>

* Update to v0.8.43 (acryldata#151)

* feat(upgrade): allow setting batch args (acryldata#155)

* feat(datahub-upgrade): Restore Indices Resources (acryldata#152)

* <feat>(datahub-upgrade): Restore Indices Resources

This PR adds the ability for end-users to specify resource requests and limits
for the datahub upgrade job 'restore indices'. This is needed for situations
where the restore indices job will OOM due to a large backfill of data. I've
seen this happening with our deployment of datahub and am unable to tune
the resources for the restore indices job because the current requests
and limits are hard-coded.

* fix(gms): remove unused JMX sidecar (acryldata#156)

* fix(gms): remove unused JMX sidecar
in favour of the already included JVM agent

* Update Chart.yaml

* Update Chart.yaml

* Bumping to v0.8.44 (acryldata#160)

* feat(ci): close stale issues/prs (acryldata#164)

* feat: add extra labels to ingress configurations (acryldata#163)

* feat(prerequisites): add postgresql to prerequistes Helm chart as an optional dependency (acryldata#121)

* feat(charts): add an option to configure revisionHistoryLimit to subcharts (acryldata#159)

* feat: datahub-frontend add oidc authentication client secret reference (acryldata#165)

* fix(monitoring) Fixes monitoring default values + gms chart version (acryldata#161)

* chore: bump datahub-actions version (acryldata#168)

* Feat(actions): Support mounting a customer defined k8s secrets as files needed in ingestion recipes (acryldata#169)

* Mount a customer defined k8s secret for ssl secrets needed in ingestion recipes

* Bump chart versions

* Address comment

* Update readme

* Address Pedro's comment

* feat(monitoring): enable monitoring on frontend (acryldata#171)

* Add kafka-exporter to expose prometheus metrics for kafka cluster (acryldata#170)

* Add kafka-exporter to expose prometheus metrics for kafka cluster

* Kafka-exporter: Updated values.yaml to support tls/sasl authentication while connecting kafka cluster

* Kafka-exporter: Updated values.yaml to remove default values and correct intendation

* Add kafka-exporter to expose prometheus metrics for kafka cluster

* Kafka-exporter: Updated values.yaml to support tls/sasl authentication while connecting kafka cluster

* Kafka-exporter: Updated values.yaml to remove default values and correct intendation

* Updated chart version

* Bumping actions (acryldata#173)

* feat(elasticsearch-setup): Add support for insecure curl connections (acryldata#166)

* fix versions, clean jmx exporter (acryldata#175)

* bumping helm charts for 0.8.45 (acryldata#178)

* bumping to 0.8.45

* Fixing

* feat(monitoring): Add optional ServiceMonitors to all JVM based services (acryldata#181)

* feat(frontend): add service monitoring

* feat(gms): add service monitoring

* gms: servicemonitor

* feat(mae): enable service monitoring

* update READMEs

* fix service ports

* bump version

* Add missing default value

* Update charts/datahub/Chart.yaml

* delete jmx config and rename port to http

* feat(secrets): Allow users to specify their own secret values at first boot (acryldata#180)

* fix(auth-secret): Make template use correct retrieval method

`datahub-auth-secret` yaml was incorrectly using the `index` template function which is for list variables. For dictionaries as is the case with the `$secret.data` object, the correct method is: https://helm.sh/docs/chart_template_guide/function_list/#get

Tested locally, that this works against a running cluster.

* Updates encryption secret to use get helm function

* feat(secrets): Allow users to specify their own secret values at first boot

* fix(datahub-frontend): Remove unused env var (acryldata#176)

* fix(datahub-frontend): Remove unused env var

* Bump chart version

* Change default versions 0.9.0 (acryldata#184)

* feat(release): automatically bumbp versions for all subcharts (acryldata#185)

* feat(release): automatically bumbp versions for all subcharts

* Remove kafka export as subchart
Kafka exporter is useful for monitoring datahub, but it's not part of it. It should be deploy sepratly, as we do in production

* Add script to bump versions together

* Add tagging to the workflow

* Don't run release on CI commits

* fix(CI): remove duplicate id (acryldata#189)

* fix(ci): us sh instead of bash; increase tag by patch (acryldata#190)

* DataHub Release v0.9.1  (acryldata#191)

* fix(v0.9.1): Fix v0.9.1 release versions (acryldata#192)

* Release v0.9.1 Helm

* Bump chart versions

* Fixing final 210

* release(v0.9.2) Release Helm Charts for DataHub v0.9.2 (acryldata#194)

* Bumping to release Helm charts for v0.9.2

* bump version by running:
 ./.github/scripts/bump-chart-versions.sh 0.2.112

Co-authored-by: szalai1 <szalaipeti.vagyok@gmail.com>

* fix(gha): revert auto bump release changes (acryldata#197)

* fix(datahub-upgrade): Add system auth credentials to datahub upgrade jobs (acryldata#199)

* fix(datahub-upgrade): Add system auth credentials to datahub upgrade jobs

* bump chart.yaml version

* feat(sql): extend sql username/password configuration (acryldata#198)

* Ebean credential configuration flexbility

Allows the EBEAN_DATASOURCE_USERNAME and EBEAN_DATASOURCE_PASSWORD to be specified either in global or in the gms config.
Also, allows a string value to be specified as the password to enable vault secret webhooks.

* Extended postgres creds config

* Extended mysql setup job creds config

* bump chart version

* bump chart version

* show example in values.yaml

* Added explanation to README

* fix typo

* tidy values.yaml examples

* consistent quote

Use the same quoting solution for sql password and usernames across the file

* remove space error

* Bump gms sub-chart

* Update Chart.yaml

* add nil condition to gms deploy

* ad nil conditional to mysql job

* add nil conditional to postgres job

* fix(frontend): increase recommended defaults (acryldata#202)

* docs: add secret key information (acryldata#195)

Added information about the key name in the secret, or it will fail like this:
Error: couldn't find key neo4j-password in Secret default/neo4j-secrets

* fix(doc): correct doc for serviceMonitor.create (acryldata#204)

* Bumping charts to v0.9.3 (acryldata#207)

* Update Chart.yaml (acryldata#208)

* doc Add global.elasticsearch.useSSL to values.yaml (acryldata#209)

* fix: only add DATAHUB_SYSTEM_CLIENT envs when metadata service auth enabled (acryldata#205)

Co-authored-by: Pedro Silva <pedro@acryl.io>

* fix(frontend): Correct lifecycle templating for frontend (acryldata#201)

* Fix lifecycle templating for frontend
Fixes `throws ValidationError(Deployment.spec.template.spec.containers[0].lifecycle): invalid type for io.k8s.api.core.v1.Lifecycle: got "string", expected "map"`

* bump up Chart.yaml version

* set default value for lifecycle

* Bump versions

Co-authored-by: Pedro Silva <pedro@acryl.io>

* fix(ingestion-cron): Make hostAliases work in datahub-ingestion-cron (acryldata#206)

* fix: make hostAliases work in datahub-ingestion-cron
* Bump ingestion cron subchart version

Fixes acryldata#200

Co-authored-by: Pedro Silva <pedro@acryl.io>

* bugfix: datahub-upgrade use datasource password value if set (acryldata#210)

* feat(global-version): add option to globally define versions (acryldata#212)

* Update versions & bitnami index (acryldata#219)

* Update prerequisite versions & bitnami repo

* fix: move default tag to global.datahub.version (acryldata#214)

image.tag in datahub-gms and datahub-fronted shadowed the
global.datahub.version from the top datahub chart, so the resulting
image was head.

* fix: move default tag to global in subcharts (acryldata#220)

* fix(jobs): Bump kafka setup resources + make all jobs resources configurable (acryldata#224)

* Remove release for master

* fix(jobs): Bump kafka setup resources + make all jobs resources configurable

* Add documentation

* Bump chart.yaml

* Lock upgrade job to v0.9.4

Co-authored-by: Dexter Lee <dexter@acryl.io>

* fix: add extra space before comment to make yamllint pass (acryldata#226)

* fix(release): Revert workflow change & change upgrade job to 0.9.3 (acryldata#227)

* feat(mce-consumer): removing dependency on gms for mce consumer (acryldata#216)

* feat(mce-consumer): removing dependency on gms for mce consumer envs

* feat(elasticsearch): adding elasticsearch index job (acryldata#228)

* Feat/elasticsearch optimization ext (acryldata#229)

* feat(elasticsearch): allow document count mismatch override

* Bumping the versions as part of v0.9.5 release (acryldata#230)

* Fix 232 (acryldata#233)

* fix(cron-ingestion): referecing $ values

* bump versions

* feat(elasticsearch):  build-indices job (acryldata#231)

* feat(elasticsearch): adding elasticsearch index job
* feat(elasticsearch): allow document count mismatch override
* Adjusting hook order build-indices job

* config(gms): enable elasticsearch reindex by default (acryldata#235)

* fix(jobs): Make job pull policy consistent with subchart (acryldata#237)

* fix(jobs): Make job pull policy consistent with subchart

* Explicitly set schema registry type

* Update charts/datahub/values.yaml

Co-authored-by: Peter Szalai  <szalaipeti.vagyok@gmail.com>

Co-authored-by: Peter Szalai  <szalaipeti.vagyok@gmail.com>

* fix(mce-consumer): Correct env vars & configure resources for all components (acryldata#238)

* fix(mce-consumer): Correct env vars & configure resources for all components

* Update quickstart values to use global datahub version

* Update elasticsearch memory configs

* fix(version) Update remaining values for 0.9.6 upgrade  (acryldata#239)

* v0.9.6.1 release (acryldata#241)

* Release bump fix (acryldata#243)

* fix(release version): fix missing global release bump to 0.9.6.1

* Update values.yaml (acryldata#244)

* Update values.yaml

* Update Chart.yaml

* fix(restore-indices): restore indices should use global datastore values (acryldata#247)

* expose oidc scope (acryldata#249)

* Update Chart.yaml (acryldata#251)

* fix(kafka-setup): Make kafka setup use explicit topic names (acryldata#245)

* fix(kafka-setup): Make kafka setup use explicit topic names

* Set kafka env vars for all deployments

* Update gms & actions pod env vars

* Add documentation for new properties

* Version bump (acryldata#250)

* bump versions

* GRAPH_SERVICE_DIFF_MODE_ENABLED false, it's important if we use neo4j

* graph_service_impl to elasticsearch

---------

Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Lukáš Novotný <42157644+novotl@users.noreply.github.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
Co-authored-by: Justin Marozas <justinas.marozas@gmail.com>
Co-authored-by: Tomáš Kubín <tomas.kubin@blindspot.ai>
Co-authored-by: Toby Irmer <toby@irmer.com>
Co-authored-by: Pedro Silva <pedro.cls93@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Hunter Elbourn <hunterelbourn@gmail.com>
Co-authored-by: Peter Szalai <szalaipeti.vagyok@gmail.com>
Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: Upendra Rao Vedullapalli <upendrao@gmail.com>
Co-authored-by: Felix Lüdin <13187726+Masterchen09@users.noreply.github.com>
Co-authored-by: Bumsoo Kim <bskim45@gmail.com>
Co-authored-by: Tony Ouyang <43738225+TonyOuyangGit@users.noreply.github.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Jinlin Yang <86577891+jinlintt@users.noreply.github.com>
Co-authored-by: Navin Sharma <103643430+NavinSharma13@users.noreply.github.com>
Co-authored-by: Bogdan Antoniu <bogdan.antoniu@gmail.com>
Co-authored-by: Callum Richard Edwards <34392823+CallumREdwards@users.noreply.github.com>
Co-authored-by: Álvaro González <alvaro.gonzalez@auctane.com>
Co-authored-by: David Haglund <548769+daha@users.noreply.github.com>
Co-authored-by: Andrea Scarpino <andrea@scarpino.dev>
Co-authored-by: Nurlan Farajov <nur.rico@gmail.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Adam Price <adam.price@afreshtechnologies.com>
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
sb-sebkim added a commit to sendbird/datahub-helm that referenced this pull request Jul 6, 2023
* upgrading to 8.40 (acryldata#139)

* Option to add podAnnotations to jobs. (acryldata#137)

* feat(helm): add pod annotations to jobs

* feat(helm): add pod annotation to cron jobs.

* docs(helm): Add podAnnotation documentation and default values.

* feat(helm) chart version bump.

* update to v0.8.41 (acryldata#143)

* fix(mysql): update mysql helm chart version (acryldata#146)

* fix(metrics): fix Prometheus env variable in the MCE consumer deployment (acryldata#148)

* feat(datahub-frontend): Option to configure OIDC auth easily (acryldata#142)

* fix(kafka): update kafka chart version (acryldata#147)

* feat(datahub-frontend): Allow lifecycle hooks (acryldata#141)

* <feat>(datahub-frontend): Allow lifecycle hooks

This PR adds a lifecycle node to the datahub-frontend deployment template.
It can be used to execute lifecycle hooks, e.g. postStart actions, like
replacing the default user.props file.

* Bump chart version to 0.2.86

Co-authored-by: Pedro Silva <pedro.cls93@gmail.com>

* Update to v0.8.43 (acryldata#151)

* feat(upgrade): allow setting batch args (acryldata#155)

* feat(datahub-upgrade): Restore Indices Resources (acryldata#152)

* <feat>(datahub-upgrade): Restore Indices Resources

This PR adds the ability for end-users to specify resource requests and limits
for the datahub upgrade job 'restore indices'. This is needed for situations
where the restore indices job will OOM due to a large backfill of data. I've
seen this happening with our deployment of datahub and am unable to tune
the resources for the restore indices job because the current requests
and limits are hard-coded.

* fix(gms): remove unused JMX sidecar (acryldata#156)

* fix(gms): remove unused JMX sidecar
in favour of the already included JVM agent

* Update Chart.yaml

* Update Chart.yaml

* Bumping to v0.8.44 (acryldata#160)

* feat(ci): close stale issues/prs (acryldata#164)

* feat: add extra labels to ingress configurations (acryldata#163)

* feat(prerequisites): add postgresql to prerequistes Helm chart as an optional dependency (acryldata#121)

* feat(charts): add an option to configure revisionHistoryLimit to subcharts (acryldata#159)

* feat: datahub-frontend add oidc authentication client secret reference (acryldata#165)

* fix(monitoring) Fixes monitoring default values + gms chart version (acryldata#161)

* chore: bump datahub-actions version (acryldata#168)

* Feat(actions): Support mounting a customer defined k8s secrets as files needed in ingestion recipes (acryldata#169)

* Mount a customer defined k8s secret for ssl secrets needed in ingestion recipes

* Bump chart versions

* Address comment

* Update readme

* Address Pedro's comment

* feat(monitoring): enable monitoring on frontend (acryldata#171)

* Add kafka-exporter to expose prometheus metrics for kafka cluster (acryldata#170)

* Add kafka-exporter to expose prometheus metrics for kafka cluster

* Kafka-exporter: Updated values.yaml to support tls/sasl authentication while connecting kafka cluster

* Kafka-exporter: Updated values.yaml to remove default values and correct intendation

* Add kafka-exporter to expose prometheus metrics for kafka cluster

* Kafka-exporter: Updated values.yaml to support tls/sasl authentication while connecting kafka cluster

* Kafka-exporter: Updated values.yaml to remove default values and correct intendation

* Updated chart version

* Bumping actions (acryldata#173)

* feat(elasticsearch-setup): Add support for insecure curl connections (acryldata#166)

* fix versions, clean jmx exporter (acryldata#175)

* bumping helm charts for 0.8.45 (acryldata#178)

* bumping to 0.8.45

* Fixing

* feat(monitoring): Add optional ServiceMonitors to all JVM based services (acryldata#181)

* feat(frontend): add service monitoring

* feat(gms): add service monitoring

* gms: servicemonitor

* feat(mae): enable service monitoring

* update READMEs

* fix service ports

* bump version

* Add missing default value

* Update charts/datahub/Chart.yaml

* delete jmx config and rename port to http

* feat(secrets): Allow users to specify their own secret values at first boot (acryldata#180)

* fix(auth-secret): Make template use correct retrieval method

`datahub-auth-secret` yaml was incorrectly using the `index` template function which is for list variables. For dictionaries as is the case with the `$secret.data` object, the correct method is: https://helm.sh/docs/chart_template_guide/function_list/#get

Tested locally, that this works against a running cluster.

* Updates encryption secret to use get helm function

* feat(secrets): Allow users to specify their own secret values at first boot

* fix(datahub-frontend): Remove unused env var (acryldata#176)

* fix(datahub-frontend): Remove unused env var

* Bump chart version

* Change default versions 0.9.0 (acryldata#184)

* feat(release): automatically bumbp versions for all subcharts (acryldata#185)

* feat(release): automatically bumbp versions for all subcharts

* Remove kafka export as subchart
Kafka exporter is useful for monitoring datahub, but it's not part of it. It should be deploy sepratly, as we do in production

* Add script to bump versions together

* Add tagging to the workflow

* Don't run release on CI commits

* fix(CI): remove duplicate id (acryldata#189)

* fix(ci): us sh instead of bash; increase tag by patch (acryldata#190)

* DataHub Release v0.9.1  (acryldata#191)

* fix(v0.9.1): Fix v0.9.1 release versions (acryldata#192)

* Release v0.9.1 Helm

* Bump chart versions

* Fixing final 210

* release(v0.9.2) Release Helm Charts for DataHub v0.9.2 (acryldata#194)

* Bumping to release Helm charts for v0.9.2

* bump version by running:
 ./.github/scripts/bump-chart-versions.sh 0.2.112

Co-authored-by: szalai1 <szalaipeti.vagyok@gmail.com>

* fix(gha): revert auto bump release changes (acryldata#197)

* fix(datahub-upgrade): Add system auth credentials to datahub upgrade jobs (acryldata#199)

* fix(datahub-upgrade): Add system auth credentials to datahub upgrade jobs

* bump chart.yaml version

* feat(sql): extend sql username/password configuration (acryldata#198)

* Ebean credential configuration flexbility

Allows the EBEAN_DATASOURCE_USERNAME and EBEAN_DATASOURCE_PASSWORD to be specified either in global or in the gms config.
Also, allows a string value to be specified as the password to enable vault secret webhooks.

* Extended postgres creds config

* Extended mysql setup job creds config

* bump chart version

* bump chart version

* show example in values.yaml

* Added explanation to README

* fix typo

* tidy values.yaml examples

* consistent quote

Use the same quoting solution for sql password and usernames across the file

* remove space error

* Bump gms sub-chart

* Update Chart.yaml

* add nil condition to gms deploy

* ad nil conditional to mysql job

* add nil conditional to postgres job

* fix(frontend): increase recommended defaults (acryldata#202)

* docs: add secret key information (acryldata#195)

Added information about the key name in the secret, or it will fail like this:
Error: couldn't find key neo4j-password in Secret default/neo4j-secrets

* fix(doc): correct doc for serviceMonitor.create (acryldata#204)

* Bumping charts to v0.9.3 (acryldata#207)

* Update Chart.yaml (acryldata#208)

* doc Add global.elasticsearch.useSSL to values.yaml (acryldata#209)

* fix: only add DATAHUB_SYSTEM_CLIENT envs when metadata service auth enabled (acryldata#205)

Co-authored-by: Pedro Silva <pedro@acryl.io>

* fix(frontend): Correct lifecycle templating for frontend (acryldata#201)

* Fix lifecycle templating for frontend
Fixes `throws ValidationError(Deployment.spec.template.spec.containers[0].lifecycle): invalid type for io.k8s.api.core.v1.Lifecycle: got "string", expected "map"`

* bump up Chart.yaml version

* set default value for lifecycle

* Bump versions

Co-authored-by: Pedro Silva <pedro@acryl.io>

* fix(ingestion-cron): Make hostAliases work in datahub-ingestion-cron (acryldata#206)

* fix: make hostAliases work in datahub-ingestion-cron
* Bump ingestion cron subchart version

Fixes acryldata#200

Co-authored-by: Pedro Silva <pedro@acryl.io>

* bugfix: datahub-upgrade use datasource password value if set (acryldata#210)

* feat(global-version): add option to globally define versions (acryldata#212)

* Update versions & bitnami index (acryldata#219)

* Update prerequisite versions & bitnami repo

* fix: move default tag to global.datahub.version (acryldata#214)

image.tag in datahub-gms and datahub-fronted shadowed the
global.datahub.version from the top datahub chart, so the resulting
image was head.

* fix: move default tag to global in subcharts (acryldata#220)

* fix(jobs): Bump kafka setup resources + make all jobs resources configurable (acryldata#224)

* Remove release for master

* fix(jobs): Bump kafka setup resources + make all jobs resources configurable

* Add documentation

* Bump chart.yaml

* Lock upgrade job to v0.9.4

Co-authored-by: Dexter Lee <dexter@acryl.io>

* fix: add extra space before comment to make yamllint pass (acryldata#226)

* fix(release): Revert workflow change & change upgrade job to 0.9.3 (acryldata#227)

* feat(mce-consumer): removing dependency on gms for mce consumer (acryldata#216)

* feat(mce-consumer): removing dependency on gms for mce consumer envs

* feat(elasticsearch): adding elasticsearch index job (acryldata#228)

* Feat/elasticsearch optimization ext (acryldata#229)

* feat(elasticsearch): allow document count mismatch override

* Bumping the versions as part of v0.9.5 release (acryldata#230)

* Fix 232 (acryldata#233)

* fix(cron-ingestion): referecing $ values

* bump versions

* feat(elasticsearch):  build-indices job (acryldata#231)

* feat(elasticsearch): adding elasticsearch index job
* feat(elasticsearch): allow document count mismatch override
* Adjusting hook order build-indices job

* config(gms): enable elasticsearch reindex by default (acryldata#235)

* fix(jobs): Make job pull policy consistent with subchart (acryldata#237)

* fix(jobs): Make job pull policy consistent with subchart

* Explicitly set schema registry type

* Update charts/datahub/values.yaml

Co-authored-by: Peter Szalai  <szalaipeti.vagyok@gmail.com>

Co-authored-by: Peter Szalai  <szalaipeti.vagyok@gmail.com>

* fix(mce-consumer): Correct env vars & configure resources for all components (acryldata#238)

* fix(mce-consumer): Correct env vars & configure resources for all components

* Update quickstart values to use global datahub version

* Update elasticsearch memory configs

* fix(version) Update remaining values for 0.9.6 upgrade  (acryldata#239)

* v0.9.6.1 release (acryldata#241)

* Release bump fix (acryldata#243)

* fix(release version): fix missing global release bump to 0.9.6.1

* Update values.yaml (acryldata#244)

* Update values.yaml

* Update Chart.yaml

* fix(restore-indices): restore indices should use global datastore values (acryldata#247)

* expose oidc scope (acryldata#249)

* Update Chart.yaml (acryldata#251)

* fix(kafka-setup): Make kafka setup use explicit topic names (acryldata#245)

* fix(kafka-setup): Make kafka setup use explicit topic names

* Set kafka env vars for all deployments

* Update gms & actions pod env vars

* Add documentation for new properties

* Version bump (acryldata#250)

* bump versions

* Search improvements updates (acryldata#254)

* Set system-update job to enabled by global flag

* Datahub upgrade topic (acryldata#255)

* v0.9.6.1 release

* fix(release version): fix missing global release bump to 0.9.6.1

* bump versions

* Update for search improvements and generic system update job

* enable default graphl db to elasticsearch

* lint

* Set system-update job to enabled by global flag

* Adding support for non-default datahub upgrade history topic

* Datahub upgrade topic (acryldata#256)

* v0.9.6.1 release

* fix(release version): fix missing global release bump to 0.9.6.1

* bump versions

* Update for search improvements and generic system update job

* enable default graphl db to elasticsearch

* lint

* Set system-update job to enabled by global flag

* Adding support for non-default datahub upgrade history topic

* feat(gms): Add hazelcast distributed caching when replica count > 1.0 (acryldata#240)

Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* feat(neo4j-password): be able to deploy without neo4j secret (acryldata#248)

* be able to deploy without neo4j secret

---------

Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* feat(gas): remove release name from service's name (acryldata#252)

* service name without release name

---------

Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* Bump versions v0.10.0 (acryldata#257)

* Release v0.10.0

* feat: allow extra labels on deployments (acryldata#258)

* feat: allow extra labels on deployments

* feat: add examples for extraLabels

* feat(elasticsearch): allow plain text password (acryldata#264)

* feat(elasticsearch): allow plain text password

* chart version bumps

* fix strange line move

* fix(mce): remove duplicated serviceAccount section (acryldata#260)

* fix(mce): remove duplicated serviceAccount section

Fixes: acryldata#259

* Update Chart.yaml

* Update Chart.yaml

* Add input validation for managed ingestion (acryldata#268)

* Fix(managed ingestion): Add checks for MCE processor replica count with managed ingestion

* Bump chart.yaml version

* Add flag to disable helm chart validations

* fix: remove duplicated EBEAN env vars on datahub-upgrade jobs (acryldata#269)

Fixes: [acryldata#261](acryldata#261)

Co-authored-by: Justas Cernas <JustasCe@users.noreply.github.com>

* Update README.md (acryldata#271)

* fix: add missing service account in acryl-datahub-actions (acryldata#272)

* Emit mcl config (acryldata#274)

* v0.9.6.1 release

* fix(release version): fix missing global release bump to 0.9.6.1

* bump versions

* config(mcl): add mcl emission flag

* lint

* Emit mcl config (acryldata#275)

* fix(mcl): fix mcl emit for default true

* feat(flags): adding various configuration options (acryldata#277)

* Enabling new environment configuration

* Diff mode flag (acryldata#278)

* Fix options for true defaults for lineage/search cache

* fix: remove duplicated EBEAN env vars on datahub-upgrade jobs (acryldata#269) (acryldata#270)

Signed-off-by: Tomáš Novák <tomas.novak@bcas.cz>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* Release Charts for DataHub v0.10.1 (acryldata#288)

* fix(secrets): Make secret creation optional

Makes datahub-auth-secrets creation dependant on whether metadata service authentication has been enabled or not.

* Bump chart version & default datahub version

* fix: secretkey for systemClientSecret in values.yaml file (acryldata#280)

* fix: systemClientSecret key in values.yaml

Updated systemClientSecret with proper secretKey from the datahub-auth-secrets file

* fix: Updated Chart version

---------

Co-authored-by: Upendra Vedullapalli <upendra.rao.vedullapalli@entur.org>

* feat(PFP-102): Make Elasticsearch use 1 master node by default (acryldata#283)

Our dependencies chart is setup to use 3 ES nodes in 3 AZ-equivalents, but
is broadly intended for use as a test harness which often runs in single-node events.

So let's fix this and have it be single-node by default

* feat: change k8s secret mount default mode to accept value override (acryldata#289)

* update deployment yaml file

* bump chart version

* add missing chart version update

* address comments to add descriptions in README and values.yaml

* address comment to remove default value of defaultMode

* Update helm charts for datahub 0.10.2 release (acryldata#298)

Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>

* feat: enable sidecar configuration (acryldata#276)

* feat: enable sidecar configuration

* feat: values for PSQL db name, elastic prefix (acryldata#313)

* Values for PSQL database name, elastic prefix
---------

Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* feat(search,schema-registry): updates for v0.10.3 release (acryldata#311)

* Update kafka chart to 22.1.3 for kafka 3.4.0 (acryldata#316)

* feat: add parameters to cleanupJob resources settings (acryldata#317)

* feat: Add ability to specify extraPodLabels per deployment (acryldata#310)

* feat: Add ability to specify extraPodLabels per deployment

* Update Chart.yaml

---------

Co-authored-by: jorrick <jorrick.sleijster@adyen.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* fix: use common labels for (Cron)Jobs (acryldata#303)

* fix(datahub): use common labels for (Cron)Jobs

* chore: update version

---------

Co-authored-by: Matthijs van der Loos <matthijs.vanderloos@utopiamusic.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* fix: add missing global values to subchart values (acryldata#302)

* fix(datahub): add missing global values to subchart values

* chore: update versions

---------

Co-authored-by: Matthijs van der Loos <matthijs.vanderloos@utopiamusic.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* feat: allow pulling ebean username from secrets alongside password (acryldata#291)

* chore(secrets): use configurable refs instead of fixed names (acryldata#323)

* chore(secrets): use configurable refs instead of fixed names

* Update Chart.yaml

---------

Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* Update Default version to v0.10.4 (acryldata#330)

* fix: Fixed indentation in  datahub-cleanup-job-template.yml (acryldata#328)

* feat(healthcheck): use new healthcheck endpoint for GMS (acryldata#331)

Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>

* chore(version): version bump & indent (acryldata#324)

* feat: Default User Credentials (acryldata#321)

Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

* feat(cloud-sql-proxy): add support for running gcloud sql proxy as prerequisite (acryldata#332)

* mce-consumer subcharts typo fix

* datahubUpgradeBuildIndices enabled false

* kibana 7.17.3 upgrade

* finalize

* elastic replicas 3

* lint fix

* docker build multi platform supported

* upload script fix

---------

Signed-off-by: Tomáš Novák <tomas.novak@bcas.cz>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Lukáš Novotný <42157644+novotl@users.noreply.github.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
Co-authored-by: Justin Marozas <justinas.marozas@gmail.com>
Co-authored-by: Tomáš Kubín <tomas.kubin@blindspot.ai>
Co-authored-by: Toby Irmer <toby@irmer.com>
Co-authored-by: Pedro Silva <pedro.cls93@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Hunter Elbourn <hunterelbourn@gmail.com>
Co-authored-by: Peter Szalai <szalaipeti.vagyok@gmail.com>
Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: Upendra Rao Vedullapalli <upendrao@gmail.com>
Co-authored-by: Felix Lüdin <13187726+Masterchen09@users.noreply.github.com>
Co-authored-by: Bumsoo Kim <bskim45@gmail.com>
Co-authored-by: Tony Ouyang <43738225+TonyOuyangGit@users.noreply.github.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Jinlin Yang <86577891+jinlintt@users.noreply.github.com>
Co-authored-by: Navin Sharma <103643430+NavinSharma13@users.noreply.github.com>
Co-authored-by: Bogdan Antoniu <bogdan.antoniu@gmail.com>
Co-authored-by: Callum Richard Edwards <34392823+CallumREdwards@users.noreply.github.com>
Co-authored-by: Álvaro González <alvaro.gonzalez@auctane.com>
Co-authored-by: David Haglund <548769+daha@users.noreply.github.com>
Co-authored-by: Andrea Scarpino <andrea@scarpino.dev>
Co-authored-by: Nurlan Farajov <nur.rico@gmail.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Adam Price <adam.price@afreshtechnologies.com>
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
Co-authored-by: Mike <90835468+justmike1@users.noreply.github.com>
Co-authored-by: Matthijs van der Loos <33864538+matthijsvanderloos@users.noreply.github.com>
Co-authored-by: Matt Matravers <mattmatravers@hotmail.com>
Co-authored-by: Maiara Reinaldo <72740386+maiarareinaldo@users.noreply.github.com>
Co-authored-by: Justas Cernas <JustasCe@users.noreply.github.com>
Co-authored-by: Tomáš Novák <tom.nov96@gmail.com>
Co-authored-by: Upendra Vedullapalli <upendra.rao.vedullapalli@entur.org>
Co-authored-by: Kevin Meyer <meyerkev@users.noreply.github.com>
Co-authored-by: Indy Prentice <iprentic@users.noreply.github.com>
Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
Co-authored-by: KonstantinVishnivetskii <120646195+KonstantinVishnivetskii@users.noreply.github.com>
Co-authored-by: Jorrick Sleijster <jorricks3@gmail.com>
Co-authored-by: jorrick <jorrick.sleijster@adyen.com>
Co-authored-by: Matthijs van der Loos <matthijs.vanderloos@utopiamusic.com>
Co-authored-by: Max Pospischil <maxpospischil@gmail.com>
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
Co-authored-by: TusharM <tushar.madhukar@gmail.com>
Co-authored-by: seokyun.ha <127274415+Seokyun-Ha@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants