[patch] Fixes and updates for IBMCloud COS support

ibm/mas_devops/roles/cos/README.md

@@ -1,23 +1,18 @@
 cos
-===
-
+===============================================================================
 This role provides support for:
-- Provisioning and Configuring Cloud Object Storage in MAS. It currently supports two providers:
-
-  - In-cluster Ceph Object Storage leveraging OpenShift Container Storage
-  - IBM Cloud Object Storage
 
+- Provisioning and Configuring Cloud Object Storage in MAS. It currently supports two providers:
+    - In-cluster Ceph Object Storage leveraging OpenShift Container Storage
+    - IBM Cloud Object Storage
 - Deprovision Cloud Object Store. It currently supports one provider:
     - IBM Cloud Object Storage
 
 Currently this role only supports generating a system-scoped ObjectStorageCfg resource, but the generated file can be modified if you wish to use other scopes.
 
-Role Variables
---------------
-
-# IBM Cloud Object Storage (ibm)
-# ---------------------------------------------------------------------------------------------------------------------
 
+Role Variables - General
+-------------------------------------------------------------------------------
 ### cos_type
 Required.  Which COS provider to use; can be set to either `ibm` for IBM Cloud Object Storage or `ocs` for OpenShift Container Storage
 
@@ -30,6 +25,29 @@ Required.  Which action you want to run for the COS instance. You can either `pr
 - Environment Variable: `COS_ACTION`
 - Default Value: `provision`
 
+### cos_use_hmac
+Supported values are true and false, this is used when ibm cloud-cos to be setup with hmac encrypted credentials.
+
+- Environment Variable: COS_USE_HMAC
+- Default: false
+
+### cluster ingres tls secret name
+Specify the name of the cluster's ingres tls secret which contains the default router certificate.
+
+- Optional
+- Environment Variable: `OCP_INGRESS_TLS_SECRET_NAME`
+- Default Value: router-certs-default
+
+### custom_labels
+List of comma separated key=value pairs for setting custom labels on instance specific resources.
+
+- Optional
+- Environment Variable: `CUSTOM_LABELS`
+- Default Value: None
+
+
+Role Variables - IBM COS
+-------------------------------------------------------------------------------
 ### cos_instance_name
 Provide an optional name for the Object Storage instance.  This is only used when cos_type is set to `ibm` for IBM Cloud Object Storage.
 
@@ -54,20 +72,35 @@ Required (For Provisioning). The COS region location url endpoint. Needed to gen
 ### cos_resource_key_iam_role
 Provide an optional role when cos service credential is getting created during COS provisioning.
   - Environment Variable: `COS_RESOURCE_KEY_IAM_ROLE`
-  - Default Value: `Manager` 
+  - Default Value: `Manager`
+
+### cos_apikey
+Required if cos_type is set to `ibm`.  Provide your less priviledged IBM Cloud API Key for COS only
+
+- Environment Variable: `COS_APIKEY`
+- Default Value: `ibmcloud_apikey`
 
 ### ibmcloud_apikey
-Required if cos_type is set to `ibm`.  Provide your IBM Cloud API Key.
+Required if cos_type is set to `ibm`.  Provide your IBM Cloud API Key that will be used as the default API Key across multiple roles in this collection.
 
 - Environment Variable: `IBMCLOUD_APIKEY`
 - Default Value: None
 
-### ibmcloud_resourcegroup
+### cos_resourcegroup
 Only used when cos_type is set to `ibm`.  Provide the name of the resource group which will own the COS instance.
 
+- Environment Variable: `COS_RESOURCEGROUP`
+- Default Value: `ibmcloud_resourcegroup`
+
+### ibmcloud_resourcegroup
+Only used when cos_type is set to `ibm`.  Provide the name of the default resource group used across multiple roles in this collection.
+
 - Environment Variable: `IBMCLOUD_RESOURCEGROUP`
 - Default Value: `Default`
 
+
+Role Variables - MAS Configuration
+-------------------------------------------------------------------------------
 ### mas_instance_id
 The instance ID of Maximo Application Suite that the ObjectStorageCfg configuration will target.  If this or `mas_config_dir` are not set then the role will not generate a ObjectStorageCfg template.
 
@@ -80,37 +113,19 @@ Local directory to save the generated ObjectStorageCfg resource definition.  Thi
 - Environment Variable: `MAS_CONFIG_DIR`
 - Default Value: None
 
-### cos_use_hmac
-Supported values are true and false, this is used when ibm cloud-cos to be setup with hmac encrypted credentials.
-
-- Environment Variable: COS_USE_HMAC
-- Default: false
-
-### cluster ingres tls secret name
-Specify the name of the cluster's ingres tls secret which contains the default router certificate.
-
-- Optional
-- Environment Variable: `OCP_INGRESS_TLS_SECRET_NAME`
-- Default Value: router-certs-default
-
-### custom_labels
-List of comma separated key=value pairs for setting custom labels on instance specific resources.
-
-- Optional
-- Environment Variable: `CUSTOM_LABELS`
-- Default Value: None
-
 ### include_cluster_ingress_cert_chain
 Optional. When set to `True`, includes the complete certificates chain in the generated MAS configuration, when a trusted certificate authority is found in your cluster's ingress.
 
 - Optional
 - Environment Variable: `INCLUDE_CLUSTER_INGRESS_CERT_CHAIN`
 - Default: `False`
 
+
 Example Playbook
-----------------
+-------------------------------------------------------------------------------
 
 Create the Ceph Object store on the existing OCS cluster and prepare the objectstorageCfg yaml to mas_config_dir.
+
 ```yaml
 - hosts: localhost
   any_errors_fatal: true
@@ -121,7 +136,9 @@ Create the Ceph Object store on the existing OCS cluster and prepare the objects
   roles:
     - ibm.mas_devops.cos
 ```
+
 Create the IBM Cloud Object storage Instance and prepare the objectstorageCfg yaml to mas_config_dir.
+
 ```yaml
 - hosts: localhost
   any_errors_fatal: true
@@ -134,6 +151,5 @@ Create the IBM Cloud Object storage Instance and prepare the objectstorageCfg ya
     - ibm.mas_devops.cos
 ```
 License
--------
-
+-------------------------------------------------------------------------------
 EPL-2.0

ibm/mas_devops/roles/cos/defaults/main.yml

@@ -19,6 +19,7 @@ cos_include_isrg_root_cert: "{{ lookup('env', 'COS_INCLUDE_ISRG_ROOT_CERT') | de
 ibmcloud_apikey: "{{ lookup('env', 'IBMCLOUD_APIKEY') }}"
 ibmcloud_resourcegroup: "{{ lookup('env', 'IBMCLOUD_RESOURCEGROUP') | default('Default', true) }}"
 cos_resourcegroup: "{{ lookup('env', 'COS_RESOURCEGROUP') | default(ibmcloud_resourcegroup, true) }}"
+cos_apikey: "{{ lookup('env', 'COS_APIKEY') | default(ibmcloud_apikey, true) }}"
 cos_instance_name: "{{ lookup('env', 'COS_INSTANCE_NAME') }}"
 cos_location_info: "{{ lookup('env', 'COS_LOCATION') | default('global', True) }}"
 cos_plan_type: "{{ lookup('env', 'COS_PLAN') | default('standard', True) }}"

ibm/mas_devops/roles/cos/tasks/main.yml

@@ -9,7 +9,7 @@
     that: cos_action is defined and cos_action != ""
     fail_msg: "cos_action property is required"
 
-# 1. Configure COS using IBM Cloud Object Storage (ibmcos) or OpenShift Container Storage (ocs)
+# 1. Configure COS using IBM Cloud Object Storage (COS) or OpenShift Container Storage (ocs)
 # ---------------------------------------------------------------------------------------------------------------------
 - name: "Run the task for the appropriate provider"
   include_tasks: tasks/providers/{{ cos_type }}/{{ cos_action }}.yml

ibm/mas_devops/roles/cos/tasks/providers/ibm/deprovision.yml

@@ -1,8 +1,8 @@
 ---
-- name: "Fail if ibmcloud_apikey is not provided"
+- name: "Fail if cos_apikey is not provided"
   assert:
-    that: ibmcloud_apikey is defined and ibmcloud_apikey != ""
-    fail_msg: "ibmcloud_apikey property is required"
+    that: cos_apikey is defined and cos_apikey != ""
+    fail_msg: "cos_apikey property is required"
 
 # 1. Determine the instance name
 # ---------------------------------------------------------------------------------------------------------------------
@@ -39,7 +39,7 @@
 - name: Fetch IBM Cloud Resource Group Id
   ibm.cloudcollection.ibm_resource_group_info:
     name: "{{ cos_resourcegroup }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey }}"
+    cos_apikey: "{{ cos_apikey }}"
   register: rg_info
 
 - name: Fetch IBM COS Information
@@ -48,7 +48,7 @@
     resource_group_id: "{{ rg_info.resource.id }}"
     service: "{{ cos_service }}"
     location: "{{ cos_location_info }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey  }}"
+    cos_apikey: "{{ cos_apikey  }}"
   register: cos_info
   failed_when: cos_info.rc != 0 and "No resource instance found" not in cos_info.stderr
 
@@ -77,7 +77,7 @@
         service: "cloud-object-storage"
         plan: "{{ cos_plan_type }}"
         location: "{{ cos_location_info }}"
-        ibmcloud_api_key: "{{ ibmcloud_apikey  }}"
+        cos_apikey: "{{ cos_apikey  }}"
         state: absent
       register: cos_deprovision_output
 

ibm/mas_devops/roles/cos/tasks/providers/ibm/provision.yml

@@ -1,8 +1,8 @@
 ---
-- name: "Fail if ibmcloud_apikey is not provided"
+- name: "Fail if cos_apikey is not provided"
   assert:
-    that: ibmcloud_apikey is defined and ibmcloud_apikey != ""
-    fail_msg: "ibmcloud_apikey property is required"
+    that: cos_apikey is defined and cos_apikey != ""
+    fail_msg: "cos_apikey property is required"
 
 # 1. Determine the instance name
 # ---------------------------------------------------------------------------------------------------------------------
@@ -37,7 +37,7 @@
 - name: "ibm : Retrieve resource group guid"
   ibm.cloudcollection.ibm_resource_group_info:
     name: "{{ cos_resourcegroup }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey }}"
+    ibmcloud_api_key: "{{ cos_apikey }}"
   register: rg_info
 
 - name: "ibm : Debug group GUID"
@@ -61,10 +61,10 @@
     service: "{{ cos_service }}"
     plan: "{{ cos_plan_type }}"
     location: "{{ cos_location_info }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey  }}"
+    ibmcloud_api_key: "{{ cos_apikey  }}"
   register: cos_output
 
-- name: "Fail if IBMCOS resource ID not found"
+- name: "Fail if COS resource ID not found"
   assert:
     that:
       - not cos_output.failed
@@ -76,14 +76,14 @@
     resource_group_id: "{{ resourceGID }}"
     service: "{{ cos_service }}"
     location: "{{ cos_location_info }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey  }}"
+    ibmcloud_api_key: "{{ cos_apikey  }}"
   register: cos_info
 
-- name: "ibm : Debug IBMCOS resource ID"
+- name: "ibm : Debug COS resource ID"
   debug:
     msg: "COS Instance resource ID ... {{ cos_info.resource.id }}"
 
-- name: "ibmcos: Set resource instance id variable"
+- name: "cos: Set resource instance id variable"
   when: cos_info.resource.id is defined
   set_fact:
     cos_resource_id: "{{ cos_info.resource.id }}"
@@ -96,15 +96,15 @@
     name: "{{ mas_instance_id }}-key"
     role: "{{ cos_resource_key_iam_role }}"
     resource_instance_id: "{{ cos_resource_id }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey }}"
+    ibmcloud_api_key: "{{ cos_apikey }}"
     parameters:
       HMAC: "{{ cos_use_hmac }}"
 
 - name: "ibm : Retrieve cos service credential for this MAS instance"
   ibm.cloudcollection.ibm_resource_key_info:
     name: "{{ mas_instance_id }}-key"
     resource_instance_id: "{{ cos_resource_id }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey }}"
+    ibmcloud_api_key: "{{ cos_apikey }}"
   register: cos_key_info
 
 - name: "ibm : Debug COS Key Resource"

ibm/mas_devops/roles/cos_bucket/defaults/main.yml

@@ -12,6 +12,8 @@ cos_bucket_action: "{{ lookup('env', 'COS_BUCKET_ACTION') | default('create', tr
 # ---------------------------------------------------------------------------------------------------------------------
 ibmcloud_apikey: "{{ lookup('env', 'IBMCLOUD_APIKEY') }}"
 ibmcloud_resourcegroup: "{{ lookup('env', 'IBMCLOUD_RESOURCEGROUP') | default('Default', true) }}"
+cos_resourcegroup: "{{ lookup('env', 'COS_RESOURCEGROUP') | default(ibmcloud_resourcegroup, true) }}"
+cos_apikey: "{{ lookup('env', 'COS_APIKEY') | default(ibmcloud_apikey, true) }}"
 cos_service: cloud-object-storage
 cos_instance_name: "{{ lookup('env', 'COS_INSTANCE_NAME') }}"
 cos_location_info: global

ibm/mas_devops/roles/cos_bucket/tasks/providers/ibm/create.yml

@@ -1,9 +1,9 @@
 ---
-# ibmcloud_apikey is required in order to be able to lookup cos resources
-- name: "Fail if ibmcloud_apikey is not provided"
+# cos_apikey is required in order to be able to lookup cos resources
+- name: "Fail if cos_apikey is not provided"
   assert:
-    that: ibmcloud_apikey is defined and ibmcloud_apikey != ""
-    fail_msg: "ibmcloud_apikey property is required"
+    that: cos_apikey is defined and cos_apikey != ""
+    fail_msg: "cos_apikey property is required"
 
 # cos_instance_name is required in order to be able to create cos resources i.e credentials and buckets
 - name: "Fail if cos_instance_name is not provided"
@@ -53,7 +53,7 @@
 - name: "ibm : Retrieve resource group guid"
   ibm.cloudcollection.ibm_resource_group_info:
     name: "{{ ibmcloud_resourcegroup }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey }}"
+    ibmcloud_api_key: "{{ cos_apikey }}"
   register: rg_info
 
 - name: "ibm : Debug group GUID"
@@ -75,7 +75,7 @@
     resource_group_id: "{{ resourceGID }}"
     service: "{{ cos_service }}"
     location: "{{ cos_location_info }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey  }}"
+    ibmcloud_api_key: "{{ cos_apikey  }}"
   register: cos_info
 
 - name: "Debug IBM COS resource ID"

ibm/mas_devops/roles/cos_bucket/tasks/providers/ibm/create_cross_region_location_bucket.yml

@@ -2,7 +2,7 @@
 - name: "Lookup IBM COS bucket named '{{ cos_bucket_name }}' in COS instance '{{ cos_instance_name }}'"
   ibm.cloudcollection.ibm_cos_bucket_info:
     resource_instance_id: "{{ cos_resource_id }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey  }}"
+    ibmcloud_api_key: "{{ cos_apikey  }}"
     bucket_name: "{{ cos_bucket_name }}"
     bucket_type: "cross_region_location"
     bucket_region: "{{ cos_bucket_region_location }}"
@@ -23,7 +23,7 @@
   - name: "Create cos bucket if it does not exist"
     ibm.cloudcollection.ibm_cos_bucket:
       resource_instance_id: "{{ cos_resource_id }}"
-      ibmcloud_api_key: "{{ ibmcloud_apikey  }}"
+      ibmcloud_api_key: "{{ cos_apikey  }}"
       bucket_name: "{{ cos_bucket_name }}"
       storage_class: "{{ cos_bucket_storage_class }}"
       cross_region_location: "{{ cos_bucket_region_location }}"

ibm/mas_devops/roles/cos_bucket/tasks/providers/ibm/create_region_location_bucket.yml

@@ -2,7 +2,7 @@
 - name: "Lookup IBM COS bucket named '{{ cos_bucket_name }}' in COS instance '{{ cos_instance_name }}'"
   ibm.cloudcollection.ibm_cos_bucket_info:
     resource_instance_id: "{{ cos_resource_id }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey  }}"
+    ibmcloud_api_key: "{{ cos_apikey  }}"
     bucket_name: "{{ cos_bucket_name }}"
     bucket_type: "region_location"
     bucket_region: "{{ cos_bucket_region_location }}"
@@ -23,7 +23,7 @@
   - name: "Create cos bucket if it does not exist"
     ibm.cloudcollection.ibm_cos_bucket:
       resource_instance_id: "{{ cos_resource_id }}"
-      ibmcloud_api_key: "{{ ibmcloud_apikey  }}"
+      ibmcloud_api_key: "{{ cos_apikey  }}"
       bucket_name: "{{ cos_bucket_name }}"
       storage_class: "{{ cos_bucket_storage_class }}"
       region_location: "{{ cos_bucket_region_location }}"

ibm/mas_devops/roles/cos_bucket/tasks/providers/ibm/create_service_credentials.yml

@@ -6,14 +6,14 @@
     name: "{{ cos_service_credentials_name }}"
     role: "{{ cos_resource_key_iam_role }}"
     resource_instance_id: "{{ cos_resource_id }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey  }}"
+    ibmcloud_api_key: "{{ cos_apikey  }}"
     parameters: {'HMAC': true }
 
 - name: "Retrieve IBM COS service credentials {{ cos_service_credentials_name }}"
   ibm.cloudcollection.ibm_resource_key_info:
     name: "{{ cos_service_credentials_name }}"
     resource_instance_id: "{{ cos_resource_id }}"
-    ibmcloud_api_key: "{{ ibmcloud_apikey  }}"
+    ibmcloud_api_key: "{{ cos_apikey  }}"
   register: cos_key_info
 
 - name: "IBM COS: Set access key id & secret access key variables"