Skip to content

Commit

Permalink
Support online/offline installation of PowerMax Observability (#112)
Browse files Browse the repository at this point in the history
  • Loading branch information
@P-Cao
P-Cao authored Feb 21, 2023
1 parent 711d0ce commit a191fc5
Show file tree
Hide file tree
Showing 2 changed files with 153 additions and 14 deletions.
48 changes: 35 additions & 13 deletions installer/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ The installation script is located at https://github.com/dell/karavi-observabili
- Copy the vxflexos-config Secret from the CSI PowerFlex namespace into the Karavi Observability namespace (if not already copied)
- Copy the powerstore-config Secret from the CSI PowerStore namespace into the Karavi Observability namespace (if not already copied)
- Copy the isilon-creds Secret from the CSI PowerScale namespace into the Karavi Observability namespace (if not already copied)
- Copy the powermax-reverseproxy-config Configmap and corresponding Secret from the CSI PowerMax namespace into the Karavi Observability namespace (if not already copied)
- Install the CertManager CRDs (if not already installed)
- Install the Karavi Observability helm chart
- Wait for the Karavi Observability pods to become ready
Expand Down Expand Up @@ -69,6 +70,7 @@ Options:
--csi-powerflex-namespace[=]<csi powerflex namespace> Namespace where CSI PowerFlex is installed, default is 'vxflexos'
--csi-powerstore-namespace[=]<csi powerstore namespace> Namespace where CSI PowerStore is installed, default is 'csi-powerstore'
--csi-powerscale-namespace[=]<csi powerscale namespace> Namespace where CSI PowerScale is installed, default is 'isilon'
--csi-powermax-namespace[=]<csi powermax namespace> Namespace where CSI PowerMax is installed, default is 'powermax'
--set-file Set values from files used during helm installation (can be specified multiple times)
--skip-verify Skip verification of the environment
--values[=]<values.yaml> Values file, which defines configuration values
Expand Down Expand Up @@ -206,10 +208,11 @@ or
*
* Downloading and saving Docker images
dellemc/csm-topology:v1.4.0
dellemc/csm-metrics-powerflex:v1.4.0
dellemc/csm-metrics-powerstore:v1.4.0
dellemc/csm-metrics-powerscale:v1.1.0
dellemc/csm-topology:v1.5.0
dellemc/csm-metrics-powerflex:v1.5.0
dellemc/csm-metrics-powerstore:v1.5.0
dellemc/csm-metrics-powerscale:v1.2.0
dellemc/csm-metrics-powermax:v1.0.0
otel/opentelemetry-collector:0.42.0
nginxinc/nginx-unprivileged:1.20
Expand Down Expand Up @@ -239,10 +242,11 @@ or
*
* Loading, tagging, and pushing Docker images to registry <my-registry>:5000/
dellemc/csm-topology:v1.4.0 -> <my-registry>:5000/csm-topology:v1.4.0
dellemc/csm-metrics-powerflex:v1.4.0 -> <my-registry>:5000/csm-metrics-powerflex:v1.4.0
dellemc/csm-metrics-powerstore:v1.4.0 -> <my-registry>:5000/csm-metrics-powerstore:v1.4.0
dellemc/csm-metrics-powerscale:v1.1.0 -> <my-registry>:5000/csm-metrics-powerscale:v1.1.0
dellemc/csm-topology:v1.5.0 -> <my-registry>:5000/csm-topology:v1.5.0
dellemc/csm-metrics-powerflex:v1.5.0 -> <my-registry>:5000/csm-metrics-powerflex:v1.5.0
dellemc/csm-metrics-powerstore:v1.5.0 -> <my-registry>:5000/csm-metrics-powerstore:v1.5.0
dellemc/csm-metrics-powerscale:v1.2.0 -> <my-registry>:5000/csm-metrics-powerscale:v1.2.0
dellemc/csm-metrics-powermax:v1.0.0 -> <my-registry>:5000/csm-metrics-powermax:v1.0.0
otel/opentelemetry-collector:0.42.0 -> <my-registry>:5000/opentelemetry-collector:0.42.0
nginxinc/nginx-unprivileged:1.20 -> <my-registry>:5000/nginx-unprivileged:1.20
```
Expand Down Expand Up @@ -280,10 +284,20 @@ Example command to copy the Secret from the isilon namespace to the karavi names
[user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# kubectl get secret isilon-creds -n isilon -o yaml | sed 's/namespace: isilon/namespace: karavi/' | kubectl create -f -
```

6. (Optional) The following steps can be performed to enable Karavi Observability for PowerFlex/PowerScale to use an existing instance of Karavi Authorization for accessing the REST API for the given storage systems.
6. The powermax-reverseproxy-config Configmap and corresponding Secreta from the namespace where CSI Driver for Dell PowerMax is installed must be copied to the namespace where Karavi Observability is to be installed.
```
[user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# kubectl get configmap powermax-reverseproxy-config -n powermax -o yaml | sed 's/namespace: powermax/namespace: karavi/' | kubectl create -f -
[user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# for secret in\
$(kubectl get configmap powermax-reverseproxy-config -n powermax -o jsonpath="{.data.config\.yaml}" | grep arrayCredentialSecret | awk 'BEGIN{FS=":"}{print $2}' | uniq); do\
kubectl get secret $secret -n powermax -o yaml | sed "s/namespace: powermax/namespace: karavi/" \
| kubectl create -f -;done
```

# TODO
7. (Optional) The following steps can be performed to enable Karavi Observability for PowerFlex/PowerScale/PowerMax to use an existing instance of Karavi Authorization for accessing the REST API for the given storage systems.
You need to provide your own configurations. A sample values.yaml file is located [here](https://github.com/dell/helm-charts/blob/main/charts/karavi-observability/values.yaml).

In your own configuration values.yaml, you need to enable PowerFlex/PowerScale Authorization, and provide the location of the sidecar-proxy Docker image and URL of the Karavi Authorization proxyHost address.
In your own configuration values.yaml, you need to enable PowerFlex/PowerScale/PowerMax Authorization, and provide the location of the sidecar-proxy Docker image and URL of the Karavi Authorization proxyHost address.

PowerFlex:
Copy the vxflexos-config-params Configmap, and Copy karavi-authorization-config, proxy-server-root-certificate and proxy-authz-tokens Secrets into the Karavi Observability namespace:
Expand All @@ -300,8 +314,16 @@ Copy the isilon-config-params Configmap, and Copy karavi-authorization-config, p
[user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# kubectl get secret karavi-authorization-config proxy-server-root-certificate proxy-authz-tokens -n isilon -o yaml | sed 's/namespace: isilon/namespace: karavi/' | sed 's/name: karavi-authorization-config/name: isilon-karavi-authorization-config/' | sed 's/name: proxy-server-root-certificate/name: isilon-proxy-server-root-certificate/' | sed 's/name: proxy-authz-tokens/name: isilon-proxy-authz-tokens/' | kubectl create -f -
```
PowerMax:
Copy the powermax-config-params Configmap, and Copy karavi-authorization-config, proxy-server-root-certificate and proxy-authz-tokens Secrets into the Karavi Observability namespace:

```
[user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# kubectl get configmap powermax-config-params -n powermax -o yaml | sed 's/namespace: powermax/namespace: karavi/' | kubectl create -f -
[user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# kubectl get secret karavi-authorization-config proxy-server-root-certificate proxy-authz-tokens -n powermax -o yaml | sed 's/namespace: powermax/namespace: karavi/' | sed 's/name: karavi-authorization-config/name: powermax-karavi-authorization-config/' | sed 's/name: proxy-server-root-certificate/name: powermax-proxy-server-root-certificate/' | sed 's/name: proxy-authz-tokens/name: powermax-proxy-authz-tokens/' | kubectl create -f -
```

7. Now that the required images have been made available and the Helm chart's configuration updated with references to the internal registry location, installation can proceed by following the instructions that are documented within the Helm chart's repository.
8. Now that the required images have been made available and the Helm chart's configuration updated with references to the internal registry location, installation can proceed by following the instructions that are documented within the Helm chart's repository.

**Note**: Optionally, you could provide your own configurations. A sample values.yaml file is located [here](https://github.com/dell/helm-charts/blob/main/charts/karavi-observability/values.yaml).

Expand Down Expand Up @@ -351,13 +373,13 @@ Follow [Offline Karavi Observability Helm Chart Installer](#unpack-the-offline-b
[user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# kubectl apply --validate=false -f cert-manager.crds.yaml
```

3. (Optional) The following steps can be performed to enable Karavi Observability for PowerFlex/PowerScale to use an existing instance of Karavi Authorization for accessing the REST API for the given storage systems.
3. (Optional) The following steps can be performed to enable Karavi Observability for PowerFlex/PowerScale/PowerMax to use an existing instance of Karavi Authorization for accessing the REST API for the given storage systems.

**Note**: Assuming that If the Karavi Observability's Authorization has been enabled in the phase of [Offline Karavi Observability Helm Chart Installer](#offline-karavi-observability-helm-chart-installer), the Authorization Secrets/Configmap have been copied to the Karavi Observability namespace.

You need to provide your own configurations. A sample values.yaml file is located [here](https://github.com/dell/helm-charts/blob/main/charts/karavi-observability/values.yaml).

In your own configuration values.yaml, you need to enable PowerFlex/PowerScale Authorization, and provide the location of the sidecar-proxy Docker image and URL of the Karavi Authorization proxyHost address.
In your own configuration values.yaml, you need to enable PowerFlex/PowerScale/PowerMax Authorization, and provide the location of the sidecar-proxy Docker image and URL of the Karavi Authorization proxyHost address.

4. Now that the required images have been made available and the Helm chart's configuration updated with references to the internal registry location, installation can proceed by following the instructions that are documented within the Helm chart's repository.

Expand Down
119 changes: 118 additions & 1 deletion installer/karavi-observability-install.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,18 @@ NC='\033[0m' # No Color
DEFAULT_CSI_POWERFLEX_NAMESPACE="vxflexos"
DEFAULT_CSI_POWERSTORE_NAMESPACE="csi-powerstore"
DEFAULT_CSI_POWERSCALE_NAMESPACE="isilon"
DEFAULT_CSI_POWERMAX_NAMESPACE="powermax"
CSI_POWERFLEX_NAMESPACE=""
CSI_POWERSTORE_NAMESPACE=""
CSI_POWERSCALE_NAMESPACE=""
CSI_POWERMAX_NAMESPACE=""
NAMESPACE=""
VALUES=""

DISABLE_POWERFLEX_COMPONENTS=false
DISABLE_POWERSTORE_COMPONENTS=false
DISABLE_POWERSCALE_COMPONENTS=false
DISABLE_POWERMAX_COMPONENTS=false

VERBOSE=0

Expand All @@ -38,8 +41,10 @@ RELEASE="karavi-observability"
FAIL_IF_AUTHORIZATION_NOT_AVAILABLE=0
ENABLE_AUTHORIZATION_DURING_INSTALL=0
KARAVICTL_INSTALLED=0
# TODO
KARAVI_POWERFLEX_AUTHORIZATION_ENTITIES_EXIST=0
KARAVI_POWERSCALE_AUTHORIZATION_ENTITIES_EXIST=0
KARAVI_POWERMAX_AUTHORIZATION_ENTITIES_EXIST=0

HELM_SET_FILES=()

Expand Down Expand Up @@ -89,6 +94,44 @@ function create_namespace() {
fi
}

# is_csi_powermax_installed returns 0 if CSI Driver for PowerMax is installed
function is_csi_powermax_installed() {
NUM=$(run_command kubectl get configmap -n ${CSI_POWERMAX_NAMESPACE} 2> /dev/null | grep -e '^powermax-reverseproxy-config\s' | wc -l)
if [ "${NUM}" != "0" ]; then
return 0
else
return 1
fi
}

# copy the powermax-reverseproxy-config ConfigMap and corresponding Secret from the CSI PowerScale namespace into the namespace for Karavi Observability
function copy_powermax_config_secret() {
NUM=$(run_command kubectl get configmap -n ${NAMESPACE} 2> /dev/null | grep -e '^powermax-reverseproxy-config\s' | wc -l)
if [ "${NUM}" == "0" ]; then
log step "Copying ConfigMap from ${CSI_POWERMAX_NAMESPACE} to ${NAMESPACE}" "small"
run_command "kubectl get configmap powermax-reverseproxy-config -n ${CSI_POWERMAX_NAMESPACE} -o yaml | sed 's/namespace: ${CSI_POWERMAX_NAMESPACE}/namespace: ${NAMESPACE}/' | kubectl create -f - > ${DEBUGLOG} 2>&1"

if [ $? -eq 1 ]; then
log step_failure
log error "Unable to copy ConfigMap from namespace ${CSI_POWERMAX_NAMESPACE} to ${NAMESPACE}."
else
log step_success
fi

log step "Copying Secret from ${CSI_POWERMAX_NAMESPACE} to ${NAMESPACE}" "small"
for secret in $(kubectl get configmap powermax-reverseproxy-config -n ${CSI_POWERMAX_NAMESPACE} -o jsonpath="{.data.config\.yaml}" | grep arrayCredentialSecret | awk 'BEGIN{FS=":"}{print $2}' | uniq)
do
run_command "kubectl get secret $secret -n ${CSI_POWERMAX_NAMESPACE} -o yaml | sed 's/namespace: ${CSI_POWERMAX_NAMESPACE}/namespace: ${NAMESPACE}/' | kubectl create -f - > ${DEBUGLOG} 2>&1"
if [ $? -eq 1 ]; then
log step_failure
log error "Unable to copy Secret from namespace ${CSI_POWERMAX_NAMESPACE} to ${NAMESPACE}."
else
log step_success
fi
done
fi
}

# is_csi_powerscale_installed returns 0 if CSI Driver for PowerScale is installed
function is_csi_powerscale_installed() {
NUM=$(run_command kubectl get secret -n ${CSI_POWERSCALE_NAMESPACE} 2> /dev/null | grep -e '^isilon-creds\s' | wc -l)
Expand Down Expand Up @@ -222,6 +265,35 @@ function copy_powerscale_authorization_entities() {
fi
}

# copy the proxy-authz-tokens, karavi-authorization-config, and proxy-server-root-certificate Secrets and powermax-config-params ConfigMap from the CSI PowerMax namespace into the namespace for Karavi Observability for Karavi Authorization
function copy_powermax_authorization_entities() {
NUM=$(run_command kubectl get configmap --namespace "${NAMESPACE}" | grep -e '^powermax-config-params\s' | wc -l)
if [ "${NUM}" == "0" ]; then
log arrow
log smart_step "Copying ConfigMap from ${CSI_POWERMAX_NAMESPACE} to ${NAMESPACE}" "small"
run_command "kubectl get configmap powermax-config-params -n ${CSI_POWERMAX_NAMESPACE} -o yaml | sed 's/namespace: ${CSI_POWERMAX_NAMESPACE}/namespace: ${NAMESPACE}/' | kubectl create -f - > ${DEBUGLOG} 2>&1"
if [ $? -eq 1 ]; then
log step_failure
log error "Unable to copy ConfigMap from namespace ${CSI_POWERMAX_NAMESPACE} to ${NAMESPACE}."
else
log step_success
fi
fi

NUM2=$(run_command kubectl get secret --namespace "${NAMESPACE}" | grep -e '^powermax-proxy-authz-tokens\s' -e '^powermax-karavi-authorization-config\s' -e '^powermax-proxy-server-root-certificate\s' | wc -l)
if [ "${NUM2}" != "3" ]; then
log arrow
log smart_step "Copying Karavi Authorization Secrets from ${CSI_POWERMAX_NAMESPACE} to ${NAMESPACE}" "small"
run_command "kubectl get secret proxy-authz-tokens karavi-authorization-config proxy-server-root-certificate -n ${CSI_POWERMAX_NAMESPACE} -o yaml | sed 's/namespace: ${CSI_POWERMAX_NAMESPACE}/namespace: ${NAMESPACE}/' | sed 's/name: karavi-authorization-config/name: powermax-karavi-authorization-config/' | sed 's/name: proxy-server-root-certificate/name: powermax-proxy-server-root-certificate/' | sed 's/name: proxy-authz-tokens/name: powermax-proxy-authz-tokens/' | kubectl create -f - > ${DEBUGLOG} 2>&1"
if [ $? -eq 1 ]; then
log step_failure
log error "Unable to copy Karavi Authorization Secrets from namespace ${CSI_POWERMAX_NAMESPACE} to ${NAMESPACE}."
else
log step_success
fi
fi
}

# enable the authorization sidecar-proxy for observability
function enable_auth_for_observability() {
run_command "kubectl get secrets,deployments -n ${NAMESPACE} -o yaml | kubectl apply -f - > /dev/null 2>&1"
Expand Down Expand Up @@ -294,6 +366,9 @@ function install_karavi_observability() {
if [ "$DISABLE_POWERSCALE_COMPONENTS" == "true" ]; then
OPT_VALUES_ARG+="--set karaviMetricsPowerscale.enabled=false "
fi
if [ "$DISABLE_POWERMAX_COMPONENTS" == "true" ]; then
OPT_VALUES_ARG+="--set karaviMetricsPowermax.enabled=false "
fi

log step "Installing Karavi Observability helm chart"
run_command "helm install \
Expand Down Expand Up @@ -367,7 +442,7 @@ function verify_karavi_observability() {
log info "Skipping verification of the environment"
return
fi
verify_k8s_versions "1.22" "1.25"
verify_k8s_versions "1.22" "1.26"
verify_openshift_versions "4.8" "4.11"
verify_helm_3
}
Expand Down Expand Up @@ -485,6 +560,10 @@ function validate_params() {
if [ -z "${CSI_POWERSCALE_NAMESPACE}" ]; then
CSI_POWERSCALE_NAMESPACE="${DEFAULT_CSI_POWERSCALE_NAMESPACE}"
fi
# if not overriding csi powermax namespace, use the default
if [ -z "${CSI_POWERMAX_NAMESPACE}" ]; then
CSI_POWERMAX_NAMESPACE="${DEFAULT_CSI_POWERMAX_NAMESPACE}"
fi
}

# determines the version of OpenShift
Expand Down Expand Up @@ -544,6 +623,7 @@ function usage() {
decho " --csi-powerflex-namespace[=]<csi powerflex namespace> Namespace where CSI PowerFlex is installed, default is 'vxflexos'"
decho " --csi-powerstore-namespace[=]<csi powerstore namespace> Namespace where CSI PowerStore is installed, default is 'csi-powerstore'"
decho " --csi-powerscale-namespace[=]<csi powerscale namespace> Namespace where CSI PowerScale is installed, default is 'isilon'"
decho " --csi-powermax-namespace[=]<csi powermax namespace> Namespace where CSI PoPowerMax is installed, default is 'powermax'"
decho " --set-file Set values from files used during helm installation (can be specified multiple times)"
decho " --skip-verify Skip verification of the environment"
decho " --values[=]<values.yaml> Values file, which defines configuration values"
Expand Down Expand Up @@ -602,6 +682,13 @@ while getopts ":h-:" optchar; do
csi-powerscale-namespace=*)
CSI_POWERSCALE_NAMESPACE=${OPTARG#*=}
;;
csi-powermax-namespace)
CSI_POWERMAX_NAMESPACE="${!OPTIND}"
OPTIND=$((OPTIND + 1))
;;
csi-powermax-namespace=*)
CSI_POWERMAX_NAMESPACE=${OPTARG#*=}
;;
set-file)
HELM_SET_FILES+=(${!OPTIND})
OPTIND=$((OPTIND + 1))
Expand Down Expand Up @@ -679,6 +766,19 @@ function powerscale_authorization_entities_exist() {
fi
}

function powermax_authorization_entities_exist() {
NUM=$(run_command kubectl get secret --namespace "${CSI_POWERMAX_NAMESPACE}" 2> /dev/null | grep -e '^proxy-authz-tokens\s' -e '^karavi-authorization-config\s' -e '^proxy-server-root-certificate\s' | wc -l)
NUM2=$(run_command kubectl get configmap --namespace "${CSI_POWERMAX_NAMESPACE}" 2> /dev/null | grep -e '^isilon-config-params\s' | wc -l)
if [[ "${NUM}" == "3" && "${NUM2}" == "1" ]]; then
KARAVI_POWERMAX_AUTHORIZATION_ENTITIES_EXIST=1
else
KARAVI_POWERMAX_AUTHORIZATION_ENTITIES_EXIST=0
if [[ "${FAIL_IF_AUTHORIZATION_NOT_AVAILABLE}" == "1" ]]; then
log error "Unable to use Karavi Authorization for PowerMax because the entities do not exist in namespace ${CSI_POWERMAX_NAMESPACE}"
fi
fi
}

function verify_authorization_environment() {
karavictl_exists
vxflexos_authorization_entities_exist
Expand Down Expand Up @@ -742,6 +842,17 @@ case $MODE in
log step_success
DISABLE_POWERSCALE_COMPONENTS=true
fi

is_csi_powermax_installed
if [[ $? == "0" ]]; then
log step "CSI Driver for PowerMax is installed"
log step_success
copy_powermax_config_secret
else
log step "CSI Driver for PowerMax is not installed" "small"
log step_success
DISABLE_POWERMAX_COMPONENTS=true
fi

install_certmanager_crds

Expand All @@ -754,6 +865,9 @@ case $MODE in
if [[ "${KARAVI_POWERSCALE_AUTHORIZATION_ENTITIES_EXIST}" == "1" ]]; then
copy_powerscale_authorization_entities
fi
if [[ "${KARAVI_POWERMAX_AUTHORIZATION_ENTITIES_EXIST}" == "1" ]]; then
copy_powermax_authorization_entities
fi
enable_auth_for_observability
fi

Expand Down Expand Up @@ -783,6 +897,9 @@ case $MODE in
if [[ "${KARAVI_POWERSCALE_AUTHORIZATION_ENTITIES_EXIST}" == "1" ]]; then
copy_powerscale_authorization_entities
fi
if [[ "${KARAVI_POWERMAX_AUTHORIZATION_ENTITIES_EXIST}" == "1" ]]; then
copy_powermax_authorization_entities
fi
enable_auth_for_observability
wait_on_pods
fi
Expand Down

0 comments on commit a191fc5

Please sign in to comment.