From 15b00fb3a55d9a20e534bcf40a218391968e3689 Mon Sep 17 00:00:00 2001 From: maskarb Date: Thu, 23 Mar 2023 16:01:10 -0400 Subject: [PATCH 1/6] update docs --- docs/csv-description.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/csv-description.md b/docs/csv-description.md index 0f9ad9548..568068e9f 100644 --- a/docs/csv-description.md +++ b/docs/csv-description.md @@ -12,6 +12,8 @@ The Koku Metrics Operator (`koku-metrics-operator`) collects the metrics require * Packaging the CSV report files into tarballs. #### Additional Capabilities: +* Resource Optimization metrics collection (the default is to not collect these metrics). +* The operator can be configured to gather all previous data within the configured retention period or a maximum of 90 days. The default data collection period is the 14 previous days. This setting is only applicable to newly created KokuMetricsConfigs. * The operator can be configured to automatically upload the packaged reports to cost management through Red Hat Insights Ingress service. * The operator can create a source in console.redhat.com. A source is required for cost management to process the uploaded packages. * PersistentVolumeClaim (PVC) configuration: The KokuMetricsConfig CR can accept a PVC definition and the operator will create and mount the PVC. If one is not provided, a default PVC will be created. @@ -48,6 +50,27 @@ To use the default specification, the follow assumptions must be met: If these assumptions are not met, the operator will not deploy correctly. In these cases, storage must be manually configured. After configuring storage, a valid PVC template should be supplied in the `volume_claim_template` spec of the KokuMetricsConfig CR. +## Configrable parameters: +* ### authentication: + * `type: choice (basic, token)` -> The authentication method for connecting to `console.redhat.com`. The default and preferred method is `token`. `basic` is used when the openshift-config pull-secret does not contain a token for `cloud.redhat.com`. + * `secret_name` -> The Secret used by the operator when the authentication type is `basic`. +* ### packaging: + * `max_reports_to_store` -> The number of reports to store when configured in air-gapped mode. The default is 30, with a minimum of 1 and no maximum. When the operator is not configured in air-gapped mode, this parameter has no effect. Reports are removed as soon as they are uploaded. + * `max_size` -> The maximum size for packaged files in Megabytes prior to compression. The default is 100, with a minimum of 1 and maximum of 100. +* ### prometheus_config: + * `collect_previous_data` -> Toggle for collecting all available data in Prometheus **upon KokuMetricsConfig creation** (This parameter will start to appear in KokuMetricsConfigs that were created prior to v2.0.0 but will not have any effect unless the KokuMetricsConfig is deleted and recreated). The default is `true`. The operator will first look for a `retention` period in the `cluster-monitoring-config` ConfigMap in the `openshift-monitoring` namespace and gather data over this time period up to a maximum of 90 days. If this configuration is not set, the default is 14 days. (New in v2.0.0) + * `disable_metrics_collection_cost_management` -> Toggle for disabling the collection of metrics for Cost Management. The default is false. (New in v2.0.0) + * `disable_metrics_collection_resource_optimization` -> Toggle for disabling the collection of metrics for Resource Optimization. The default is true. (New in v2.0.0) + * `context_timeout` -> The time in seconds before Prometheus queries timeout due to exceeding context timeout. The default is 120, with a minimum of 10 and maximum of 180. +* ### source: + * `name`: -> The name of the Source the operator will create in `console.redhat.com`. The default is `INSERT_SOURCE_NAME` which is a placeholder. + * `create_source` -> Toggle for whether or not the operator will create the Source in `console.redhat.com`. The default is False. This parameter should be switched to True when a Source does not already exist in `console.redhat.com` for this cluster. + * `check_cycle` -> The time in minutes to wait between checking if a Source exists for this cluster. The default is 1440 minutes (24 hrs). +* ### upload: + * `upload_cycle` -> The time in minutes between payload uploads. The default is 360 (6 hours). + * `upload_toggle` -> Toggle to turn upload on or off -> true means upload, false means do not upload (false == air-gapped mode). The default is `true`. + * `upload_wait` -> The amount of time (in seconds) to pause before uploading a payload. The default is a random number between 0 and 35. This is used to decrease service load, but may be set to `0` if desired. + ## Configure the koku-metrics-operator **Note** There are separate instructions for configuring the `koku-metrics-operator` to run in a restricted network. ##### Configure authentication From d818f0da8dfd7d56b5106e70b89d773138a6d277 Mon Sep 17 00:00:00 2001 From: maskarb Date: Thu, 23 Mar 2023 16:02:23 -0400 Subject: [PATCH 2/6] add a new in v2 section to be filled out --- docs/csv-description.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/csv-description.md b/docs/csv-description.md index 568068e9f..ea8205169 100644 --- a/docs/csv-description.md +++ b/docs/csv-description.md @@ -19,6 +19,9 @@ The Koku Metrics Operator (`koku-metrics-operator`) collects the metrics require * PersistentVolumeClaim (PVC) configuration: The KokuMetricsConfig CR can accept a PVC definition and the operator will create and mount the PVC. If one is not provided, a default PVC will be created. * Restricted network installation: this operator can function on a restricted network. In this mode, the operator stores the packaged reports for manual retrieval. +## New in v2.0.0: +* + ## Limitations and Pre-Requisites #### Limitations (Potential for metrics data loss) * A source **must** exist in console.redhat.com for an uploaded payload to be processed by cost management. The operator sends the payload to the Red Hat Insights Ingress service which usually returns successfully, but the operator does not currently confirm with cost management that the payload was processed. After Ingress accepts the uploaded payload, the payload is removed from the operator and is gone forever. If the data within the payload is not processed, a gap will be introduced in the usage metrics. From c7a112732db6690d5616658dbd4bf2eb19334489 Mon Sep 17 00:00:00 2001 From: maskarb Date: Tue, 28 Mar 2023 22:42:21 -0400 Subject: [PATCH 3/6] more docs --- docs/csv-description.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/csv-description.md b/docs/csv-description.md index ea8205169..16cdba26b 100644 --- a/docs/csv-description.md +++ b/docs/csv-description.md @@ -20,7 +20,8 @@ The Koku Metrics Operator (`koku-metrics-operator`) collects the metrics require * Restricted network installation: this operator can function on a restricted network. In this mode, the operator stores the packaged reports for manual retrieval. ## New in v2.0.0: -* +* Adds metrics and report generation for resource optimization. This feature will collect additional usage metrics and create a new report in the payload. These metrics are disabled by default, but can be enabled by setting `disable_metrics_collection_resource_optimization` to `false`. +* Collect all available Prometheus data upon CR creation. This feature only applies to newly created KokuMetricsConfigs. The operator will check the monitoring stack configuration in the `openshift-monitoring` namespace. The operator will use the `retention` period set in the `cluster-monitoring-config` ConfigMap if defined, up to a maximum of 90 days. Otherwise it will fall back to collecting 14 days of data, if available. This data collection may be disabled by setting `collect_previous_data` to `false`. Turning this feature off results in the operator collecting metrics from the time the KokuMetricsConfig is created, forward. ## Limitations and Pre-Requisites #### Limitations (Potential for metrics data loss) @@ -53,7 +54,7 @@ To use the default specification, the follow assumptions must be met: If these assumptions are not met, the operator will not deploy correctly. In these cases, storage must be manually configured. After configuring storage, a valid PVC template should be supplied in the `volume_claim_template` spec of the KokuMetricsConfig CR. -## Configrable parameters: +## Configurable parameters: * ### authentication: * `type: choice (basic, token)` -> The authentication method for connecting to `console.redhat.com`. The default and preferred method is `token`. `basic` is used when the openshift-config pull-secret does not contain a token for `cloud.redhat.com`. * `secret_name` -> The Secret used by the operator when the authentication type is `basic`. From 8a03f6fb678fc0b08e54c7af5b1b0b36c7d009b2 Mon Sep 17 00:00:00 2001 From: maskarb Date: Tue, 28 Mar 2023 22:48:41 -0400 Subject: [PATCH 4/6] more docs --- docs/csv-description.md | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/docs/csv-description.md b/docs/csv-description.md index 16cdba26b..097f7e751 100644 --- a/docs/csv-description.md +++ b/docs/csv-description.md @@ -55,24 +55,24 @@ To use the default specification, the follow assumptions must be met: If these assumptions are not met, the operator will not deploy correctly. In these cases, storage must be manually configured. After configuring storage, a valid PVC template should be supplied in the `volume_claim_template` spec of the KokuMetricsConfig CR. ## Configurable parameters: -* ### authentication: - * `type: choice (basic, token)` -> The authentication method for connecting to `console.redhat.com`. The default and preferred method is `token`. `basic` is used when the openshift-config pull-secret does not contain a token for `cloud.redhat.com`. - * `secret_name` -> The Secret used by the operator when the authentication type is `basic`. -* ### packaging: - * `max_reports_to_store` -> The number of reports to store when configured in air-gapped mode. The default is 30, with a minimum of 1 and no maximum. When the operator is not configured in air-gapped mode, this parameter has no effect. Reports are removed as soon as they are uploaded. - * `max_size` -> The maximum size for packaged files in Megabytes prior to compression. The default is 100, with a minimum of 1 and maximum of 100. -* ### prometheus_config: - * `collect_previous_data` -> Toggle for collecting all available data in Prometheus **upon KokuMetricsConfig creation** (This parameter will start to appear in KokuMetricsConfigs that were created prior to v2.0.0 but will not have any effect unless the KokuMetricsConfig is deleted and recreated). The default is `true`. The operator will first look for a `retention` period in the `cluster-monitoring-config` ConfigMap in the `openshift-monitoring` namespace and gather data over this time period up to a maximum of 90 days. If this configuration is not set, the default is 14 days. (New in v2.0.0) - * `disable_metrics_collection_cost_management` -> Toggle for disabling the collection of metrics for Cost Management. The default is false. (New in v2.0.0) - * `disable_metrics_collection_resource_optimization` -> Toggle for disabling the collection of metrics for Resource Optimization. The default is true. (New in v2.0.0) - * `context_timeout` -> The time in seconds before Prometheus queries timeout due to exceeding context timeout. The default is 120, with a minimum of 10 and maximum of 180. -* ### source: - * `name`: -> The name of the Source the operator will create in `console.redhat.com`. The default is `INSERT_SOURCE_NAME` which is a placeholder. - * `create_source` -> Toggle for whether or not the operator will create the Source in `console.redhat.com`. The default is False. This parameter should be switched to True when a Source does not already exist in `console.redhat.com` for this cluster. - * `check_cycle` -> The time in minutes to wait between checking if a Source exists for this cluster. The default is 1440 minutes (24 hrs). -* ### upload: - * `upload_cycle` -> The time in minutes between payload uploads. The default is 360 (6 hours). - * `upload_toggle` -> Toggle to turn upload on or off -> true means upload, false means do not upload (false == air-gapped mode). The default is `true`. +* #### `authentication`: + * `type: token` -> The authentication method for connecting to `console.redhat.com`. The default and preferred method is `token`. `basic` is used when the openshift-config pull-secret does not contain a token for `cloud.redhat.com`. + * `secret_name` -> The Secret used by the operator when the authentication type is `basic`. This parameter is required **only if** the authentication type is `basic`. +* #### `packaging`: + * `max_reports_to_store: 30` -> The number of reports to store when configured in air-gapped mode. The default is 30, with a minimum of 1 and no maximum. When the operator is not configured in air-gapped mode, this parameter has no effect. Reports are removed as soon as they are uploaded. + * `max_size: 100` -> The maximum size for packaged files in Megabytes prior to compression. The default is 100, with a minimum of 1 and maximum of 100. +* #### `prometheus_config`: + * `collect_previous_data: true` -> Toggle for collecting all available data in Prometheus **upon KokuMetricsConfig creation** (This parameter will start to appear in KokuMetricsConfigs that were created prior to v2.0.0 but will not have any effect unless the KokuMetricsConfig is deleted and recreated). The default is `true`. The operator will first look for a `retention` period in the `cluster-monitoring-config` ConfigMap in the `openshift-monitoring` namespace and gather data over this time period up to a maximum of 90 days. If this configuration is not set, the default is 14 days. (New in v2.0.0) + * `disable_metrics_collection_cost_management: false` -> Toggle for disabling the collection of metrics for Cost Management. The default is false. (New in v2.0.0) + * `disable_metrics_collection_resource_optimization: true` -> Toggle for disabling the collection of metrics for Resource Optimization. The default is true. (New in v2.0.0) + * `context_timeout: 120` -> The time in seconds before Prometheus queries timeout due to exceeding context timeout. The default is 120, with a minimum of 10 and maximum of 180. +* #### `source`: + * `name: INSERT_SOURCE_NAME` -> The name of the Source the operator will create in `console.redhat.com`. The default is `INSERT_SOURCE_NAME` which is a placeholder. + * `create_source: false` -> Toggle for whether or not the operator will create the Source in `console.redhat.com`. The default is False. This parameter should be switched to True when a Source does not already exist in `console.redhat.com` for this cluster. + * `check_cycle: 1440` -> The time in minutes to wait between checking if a Source exists for this cluster. The default is 1440 minutes (24 hrs). +* #### `upload`: + * `upload_cycle: 360` -> The time in minutes between payload uploads. The default is 360 (6 hours). + * `upload_toggle: true` -> Toggle to turn upload on or off -> true means upload, false means do not upload (false == air-gapped mode). The default is `true`. * `upload_wait` -> The amount of time (in seconds) to pause before uploading a payload. The default is a random number between 0 and 35. This is used to decrease service load, but may be set to `0` if desired. ## Configure the koku-metrics-operator @@ -163,7 +163,7 @@ Configure the koku-metrics-operator by creating a `KokuMetricsConfig`. ``` upload: - upload_cycle: 360, + upload_cycle: 360 upload_toggle: false ``` @@ -232,6 +232,6 @@ Create source: ## Upload the reports to cost managment Uploading reports to cost managment is done through curl: - $ curl -vvvv -F "file=@FILE_NAME.tar.gz;type=application/vnd.redhat.hccm.tar+tgz" https://cloud.redhat.com/api/ingress/v1/upload -u USERNAME:PASS + $ curl -vvvv -F "file=@FILE_NAME.tar.gz;type=application/vnd.redhat.hccm.tar+tgz" https://console.redhat.com/api/ingress/v1/upload -u USERNAME:PASS where `USERNAME` and `PASS` correspond to the user credentials for [console.redhat.com](https://console.redhat.com), and `FILE_NAME` is the name of the report to upload. From f3d480013938b0c7dcc0da8ae8596d3ec0b82c06 Mon Sep 17 00:00:00 2001 From: maskarb Date: Tue, 28 Mar 2023 23:13:01 -0400 Subject: [PATCH 5/6] add volume_claim_template option --- docs/csv-description.md | 39 ++++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/docs/csv-description.md b/docs/csv-description.md index 097f7e751..8a2288a37 100644 --- a/docs/csv-description.md +++ b/docs/csv-description.md @@ -55,25 +55,26 @@ To use the default specification, the follow assumptions must be met: If these assumptions are not met, the operator will not deploy correctly. In these cases, storage must be manually configured. After configuring storage, a valid PVC template should be supplied in the `volume_claim_template` spec of the KokuMetricsConfig CR. ## Configurable parameters: -* #### `authentication`: - * `type: token` -> The authentication method for connecting to `console.redhat.com`. The default and preferred method is `token`. `basic` is used when the openshift-config pull-secret does not contain a token for `cloud.redhat.com`. - * `secret_name` -> The Secret used by the operator when the authentication type is `basic`. This parameter is required **only if** the authentication type is `basic`. -* #### `packaging`: - * `max_reports_to_store: 30` -> The number of reports to store when configured in air-gapped mode. The default is 30, with a minimum of 1 and no maximum. When the operator is not configured in air-gapped mode, this parameter has no effect. Reports are removed as soon as they are uploaded. - * `max_size: 100` -> The maximum size for packaged files in Megabytes prior to compression. The default is 100, with a minimum of 1 and maximum of 100. -* #### `prometheus_config`: - * `collect_previous_data: true` -> Toggle for collecting all available data in Prometheus **upon KokuMetricsConfig creation** (This parameter will start to appear in KokuMetricsConfigs that were created prior to v2.0.0 but will not have any effect unless the KokuMetricsConfig is deleted and recreated). The default is `true`. The operator will first look for a `retention` period in the `cluster-monitoring-config` ConfigMap in the `openshift-monitoring` namespace and gather data over this time period up to a maximum of 90 days. If this configuration is not set, the default is 14 days. (New in v2.0.0) - * `disable_metrics_collection_cost_management: false` -> Toggle for disabling the collection of metrics for Cost Management. The default is false. (New in v2.0.0) - * `disable_metrics_collection_resource_optimization: true` -> Toggle for disabling the collection of metrics for Resource Optimization. The default is true. (New in v2.0.0) - * `context_timeout: 120` -> The time in seconds before Prometheus queries timeout due to exceeding context timeout. The default is 120, with a minimum of 10 and maximum of 180. -* #### `source`: - * `name: INSERT_SOURCE_NAME` -> The name of the Source the operator will create in `console.redhat.com`. The default is `INSERT_SOURCE_NAME` which is a placeholder. - * `create_source: false` -> Toggle for whether or not the operator will create the Source in `console.redhat.com`. The default is False. This parameter should be switched to True when a Source does not already exist in `console.redhat.com` for this cluster. - * `check_cycle: 1440` -> The time in minutes to wait between checking if a Source exists for this cluster. The default is 1440 minutes (24 hrs). -* #### `upload`: - * `upload_cycle: 360` -> The time in minutes between payload uploads. The default is 360 (6 hours). - * `upload_toggle: true` -> Toggle to turn upload on or off -> true means upload, false means do not upload (false == air-gapped mode). The default is `true`. - * `upload_wait` -> The amount of time (in seconds) to pause before uploading a payload. The default is a random number between 0 and 35. This is used to decrease service load, but may be set to `0` if desired. + * `authentication`: + * `type: token` -> The authentication method for connecting to `console.redhat.com`. The default and preferred method is `token`. `basic` is used when the openshift-config pull-secret does not contain a token for `cloud.redhat.com`. + * `secret_name` -> The Secret used by the operator when the authentication type is `basic`. This parameter is required **only if** the authentication type is `basic`. + * `packaging`: + * `max_reports_to_store: 30` -> The number of reports to store when configured in air-gapped mode. The default is 30, with a minimum of 1 and no maximum. When the operator is not configured in air-gapped mode, this parameter has no effect. Reports are removed as soon as they are uploaded. + * `max_size: 100` -> The maximum size for packaged files in Megabytes prior to compression. The default is 100, with a minimum of 1 and maximum of 100. + * `prometheus_config`: + * `collect_previous_data: true` -> Toggle for collecting all available data in Prometheus **upon KokuMetricsConfig creation** (This parameter will start to appear in KokuMetricsConfigs that were created prior to v2.0.0 but will not have any effect unless the KokuMetricsConfig is deleted and recreated). The default is `true`. The operator will first look for a `retention` period in the `cluster-monitoring-config` ConfigMap in the `openshift-monitoring` namespace and gather data over this time period up to a maximum of 90 days. If this configuration is not set, the default is 14 days. (New in v2.0.0) + * `disable_metrics_collection_cost_management: false` -> Toggle for disabling the collection of metrics for Cost Management. The default is false. (New in v2.0.0) + * `disable_metrics_collection_resource_optimization: true` -> Toggle for disabling the collection of metrics for Resource Optimization. The default is true. (New in v2.0.0) + * `context_timeout: 120` -> The time in seconds before Prometheus queries timeout due to exceeding context timeout. The default is 120, with a minimum of 10 and maximum of 180. + * `source`: + * `name: INSERT_SOURCE_NAME` -> The name of the Source the operator will create in `console.redhat.com`. The default is `INSERT_SOURCE_NAME` which is a placeholder. + * `create_source: false` -> Toggle for whether or not the operator will create the Source in `console.redhat.com`. The default is False. This parameter should be switched to True when a Source does not already exist in `console.redhat.com` for this cluster. + * `check_cycle: 1440` -> The time in minutes to wait between checking if a Source exists for this cluster. The default is 1440 minutes (24 hrs). + * `upload`: + * `upload_cycle: 360` -> The time in minutes between payload uploads. The default is 360 (6 hours). + * `upload_toggle: true` -> Toggle to turn upload on or off -> true means upload, false means do not upload (false == air-gapped mode). The default is `true`. + * `upload_wait` -> The amount of time (in seconds) to pause before uploading a payload. The default is a random number between 0 and 35. This is used to decrease service load, but may be set to `0` if desired. + * `volume_claim_template`: see the "Storage configuration prerequisite" section above. ## Configure the koku-metrics-operator **Note** There are separate instructions for configuring the `koku-metrics-operator` to run in a restricted network. From 6fe58616c2068c987be1db1d3792b7350cbe8f65 Mon Sep 17 00:00:00 2001 From: maskarb Date: Tue, 28 Mar 2023 23:13:44 -0400 Subject: [PATCH 6/6] add volume_claim_template option --- docs/csv-description.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/csv-description.md b/docs/csv-description.md index 8a2288a37..48ecf3bd9 100644 --- a/docs/csv-description.md +++ b/docs/csv-description.md @@ -74,7 +74,7 @@ If these assumptions are not met, the operator will not deploy correctly. In the * `upload_cycle: 360` -> The time in minutes between payload uploads. The default is 360 (6 hours). * `upload_toggle: true` -> Toggle to turn upload on or off -> true means upload, false means do not upload (false == air-gapped mode). The default is `true`. * `upload_wait` -> The amount of time (in seconds) to pause before uploading a payload. The default is a random number between 0 and 35. This is used to decrease service load, but may be set to `0` if desired. - * `volume_claim_template`: see the "Storage configuration prerequisite" section above. + * `volume_claim_template` -> see the "Storage configuration prerequisite" section above. ## Configure the koku-metrics-operator **Note** There are separate instructions for configuring the `koku-metrics-operator` to run in a restricted network.