[Docs] - Organize upgrade steps (#13738)

openmetadata-docs/content/partials/v1.2/deployment/upgrade/post-upgrade-steps.md

@@ -0,0 +1,28 @@
+# Post-Upgrade Steps
+
+### Reindex
+
+{% partial file="/v1.2/deployment/reindex.md" /%}
+
+Since this is required after the upgrade, we want to reindex `All` the entities.
+
+### (Optional) Update your OpenMetadata Ingestion Client
+
+If you are running the ingestion workflows **externally** or using a custom Airflow installation, you need to make sure that the Python Client you use is aligned
+with the OpenMetadata server version.
+
+For example, if you are upgrading the server to the version `x.y.z`, you will need to update your client with
+
+```bash
+pip install openmetadata-ingestion[<plugin>]==x.y.z
+```
+
+The `plugin` parameter is a list of the sources that we want to ingest. An example would look like this `openmetadata-ingestion[mysql,snowflake,s3]==1.2.0`.
+You will find specific instructions for each connector [here](/connectors).
+
+Moreover, if working with your own Airflow deployment - not the `openmetadata-ingestion` image - you will need to upgrade
+as well the `openmetadata-managed-apis` version:
+
+```bash
+pip install openmetadata-managed-apis==x.y.z
+```

openmetadata-docs/content/partials/v1.2/deployment/upgrade/upgrade-prerequisites.md

@@ -1,8 +1,8 @@
-## Prerequisites
+# Prerequisites
 
 Everytime that you plan on upgrading OpenMetadata to a newer version, make sure to go over all these steps:
 
-### 1. Backup your Metadata
+### Backup your Metadata
 
 Before upgrading your OpenMetadata version we strongly recommend backing up the metadata.
 
@@ -64,46 +64,7 @@ You can refer to the following guide to get more details about the backup and re
   {% /inlineCallout %}
 {% /inlineCalloutContainer %}
 
-### 2. Review the Deprecation Notice and Breaking Changes
-
-Releases might introduce deprecations and breaking changes that you should be aware of and understand before moving forward.
-
-Below in this page you will find the details for the latest release, and you can find older release notes [here](/deployment/upgrade/versions).
-
-The goal is to answer questions like:
-- *Do I need to update my configurations?*
-- *If I am running connectors externally, did their service connection change?*
-
-Carefully reviewing this will prevent easy errors.
-
-### (Optional) 3. Update your OpenMetadata Ingestion Client
-
-If you are running the ingestion workflows **externally** or using a custom Airflow installation, you need to make sure that the Python Client you use is aligned
-with the OpenMetadata server version.
-
-For example, if you are upgrading the server to the version `x.y.z`, you will need to update your client with
-
-```bash
-pip install openmetadata-ingestion[<plugin>]==x.y.z
-```
-
-The `plugin` parameter is a list of the sources that we want to ingest. An example would look like this `openmetadata-ingestion[mysql,snowflake,s3]==1.2.0`.
-You will find specific instructions for each connector [here](/connectors).
-
-Moreover, if working with your own Airflow deployment - not the `openmetadata-ingestion` image - you will need to upgrade
-as well the `openmetadata-managed-apis` version:
-
-```bash
-pip install openmetadata-managed-apis==x.y.z
-```
-
-## Deprecation Notice
-
-- OpenMetadata only supports Python version 3.8 to 3.10. We will add support for 3.11 in the release 1.3.
-
-## Breaking Changes for 1.2 Stable Release
-
-### BEFORE the migration - Update `sort_buffer_size` (MySQL) or `work_mem` (Postgres)
+### Update `sort_buffer_size` (MySQL) or `work_mem` (Postgres)
 
 Before running the migrations, it is important to update these parameters to ensure there are no runtime errors.
 A safe value would be setting them to 20MB.
@@ -141,27 +102,29 @@ during the migration after bumping this value, you can increase them further.
 
 After the migration is finished, you can revert this changes.
 
+# Deprecation Notice
+
+- OpenMetadata only supports Python version 3.8 to 3.10. We will add support for 3.11 in the release 1.3.
+- OpenMetadata version 0.13.x is deprecated.
+
+# Breaking Changes
+
 ### Version Upgrades
 
-- OpenMetadata now supports **Elasticsearch** up to version **8.10.2** and **Opensearch** up to version **2.7**
 - The OpenMetadata Server is now based on **JDK 17**
-
-#### Elasticsearch & Opensearch troubleshooting
+- OpenMetadata now supports **Elasticsearch** up to version **8.10.2** and **Opensearch** up to version **2.7**
 
 There is no direct migration to bump the indexes to the new supported versions. You might see errors like:
 
 ```
-"java.lang.IllegalStateException: cannot upgrade a node from version [7.16.3] directly to version [8.5.1], upgrade to version [7.17.0] 
+"java.lang.IllegalStateException: cannot upgrade a node from version [7.16.3] directly to version [8.5.1]
 ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/elasticsearch.log
 ERROR: Elasticsearch exited unexpectedly
 ```
 
 In order to move forward, you can remove volumes / delete the indexes directly from your search instances. Note that
-OpenMetadata stores everything in the database, so indexes can be recreated directly from the UI.
-
-{% partial file="/v1.2/deployment/reindex.md" /%}
-
-Since this is required after the upgrade, we want to reindex `All` the entities.
+OpenMetadata stores everything in the database, so indexes can be recreated directly from the UI. We will
+show you how in the [Post-Upgrade Steps](/deployment/upgrade#reindex).
 
 ### Query Entity
 

openmetadata-docs/content/v1.2.x/deployment/upgrade/bare-metal.md

@@ -88,6 +88,4 @@ binaries. You may restart the server by running the following command.
 ./bin/openmetadata.sh start
 ```
 
-## Step 7: Re-index all your metadata
-
-{% partial file="/v1.2/deployment/reindex.md" /%}
+{% partial file="/v1.2/deployment/upgrade/post-upgrade-steps.md" /%}

openmetadata-docs/content/v1.2.x/deployment/upgrade/docker.md

@@ -20,7 +20,7 @@ Below we have highlighted the steps needed to upgrade to the latest version with
 
 # Upgrade Process
 
-## Step 1: Replace the docker compose file
+## Replace the docker compose file
 
 - Stop the running compose deployment with below command 
 ```
@@ -40,54 +40,10 @@ Please make sure to go through [breaking changes and release highlights](/deploy
 docker compose -f docker-compose.yml up -d
 ```
 
-## Step 2: Re-index all your metadata
+{% partial file="/v1.2/deployment/upgrade/post-upgrade-steps.md" /%}
 
-{% partial file="/v1.2/deployment/reindex.md" /%}
 
----
-
-## Guide for Upgrading ingestion patch versions
-
-During the release lifespan we may publish new patch versions of `openmetadata-ingestion`. If you deployed
-the ingestion container and require one of the fixes or improvements from a new patch release, there's usually no need
-to re-deploy the full ingestion container.
-
-{% note %}
-
-Note that this process will only work if we are moving from PATCH versions. For example: `0.13.1.1` -> `0.13.1.2`.
-
-This method won't work when upgrading from `0.13.1.X` -> `0.13.2.X`, as that will also require to upgrade the
-server version.
-
-{% /note %}
-
-The steps to follow are:
-
-- Connect to the ingestion container. If using our docker compose files or `metadata docker` CLI, this translates to
-   ```
-     docker exec -it openmetadata_ingestion bash
-     ```
-- Validate your `metadata` version via ```metadata --version```. You will get back something like:
-   ```
-   metadata 0.13.1.5 from /home/airflow/.local/lib/python3.9 (python 3.9)
-   ```
-- Upgrade the `openmetadata-ingestion` package via ```pip install "openmetadata-ingestion==0.13.1.X"```,for example,
-   ```
-   pip install "openmetadata-ingestion==0.13.1.7" 
-   ```
-   You can find the list of all released versions of
-   the `openmetadata-ingestion` package [here](https://pypi.org/project/openmetadata-ingestion/#history).
-- Exit the container by typing `exit`.
-- Restart the ingestion container with `docker restart openmetadata_ingestion`. This will need a few minutes to
-   to stop the container and start it again. Now, Airflow will start with the upgraded `metadata` version.
-- Connect to the ingestion container and validate the `metadata` version:
-    ```
-    docker exec -it openmetadata_ingestion bash
-    ```
-   - ```metadata version```: where we expect to get the same version that was previously installed.
-
-
-## Troubleshooting
+# Troubleshooting
 
 #### Permission Denied when running  ```metadata openmetadata-imports-migration```
 If you have a `Permission Denied` error thrown when running ```metadata openmetadata-imports-migration --change-config-file-path``` you might need to change the permission on the `/opt/airflow/dags` folder. SSH into the ingestion container and check the permission on the folder running the below commands

openmetadata-docs/content/v1.2.x/deployment/upgrade/index.md

@@ -5,34 +5,38 @@ slug: /deployment/upgrade
 
 # Upgrade OpenMetadata
 
+In this guide, you will find all the necessary information to safely upgrade your OpenMetadata instance to 1.2.x.
+
 {% partial file="/v1.2/deployment/upgrade/upgrade-prerequisites.md" /%}
 
-## Upgrade your installation
+# Upgrade your installation
 
-Once your metadata is safe, follow the required upgrade instructions:
+Once your metadata is safe, follow the required upgrade instructions based on your environment:
 
 {% inlineCalloutContainer %}
   {% inlineCallout
     color="violet-70"
     icon="fit_screen"
     bold="Upgrade a Kubernetes Deployment"
-    href="/deployment/upgrade/kubernetes" %}
+    href="/deployment/upgrade/kubernetes#upgrade-process" %}
       Upgrade your Kubernetes installation
   {% /inlineCallout %}
 
   {% inlineCallout
     color="violet-70"
     icon="celebration"
     bold="Upgrade a Docker Deployment"
-    href="/deployment/upgrade/docker" %}
+    href="/deployment/upgrade/docker#upgrade-process" %}
       Upgrade your Docker installation
   {% /inlineCallout %}
 
   {% inlineCallout
     color="violet-70"
     icon="storage"
     bold="Upgrade a Bare Metal Deployment"
-    href="/deployment/upgrade/bare-metal" %}
+    href="/deployment/upgrade/bare-metal#upgrade-process" %}
       Upgrade your Bare Metal installation
   {% /inlineCallout %}
 {% /inlineCalloutContainer %}
+
+{% partial file="/v1.2/deployment/upgrade/post-upgrade-steps.md" /%}

openmetadata-docs/content/v1.2.x/deployment/upgrade/kubernetes.md

@@ -93,11 +93,9 @@ You can learn more about how the migration process works [here](/deployment/upgr
 
 {% /note %}
 
-## Step 5: Re-index all your metadata
+{% partial file="/v1.2/deployment/upgrade/post-upgrade-steps.md" /%}
 
-{% partial file="/v1.2/deployment/reindex.md" /%}
-
-## Troubleshooting
+# Troubleshooting
 
 ### Helm Upgrade fails with additional property airflow not allowed
 
@@ -168,27 +166,6 @@ openmetadata:
 
 Run the [helm lint](https://helm.sh/docs/helm/helm_lint/) command on your custom values after making the changes to validate with the JSON Schema.
 
-### With 0.13.0 Release
-
-If your helm dependencies upgrade fails with the below command result -
-
-```
-Error: UPGRADE FAILED: cannot patch "mysql" with kind StatefulSet: StatefulSet.apps "mysql" is invalid: spec: 
-Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' 
-and 'minReadySeconds' are forbidden
-```
-
-This is probably because with `0.13.0`, we have **default size of mysql persistence set to 50Gi**.
-
-Kubernetes does not allow changes to Persistent volume with helm upgrades.
-
-In order to work around this issue, you can either default the persistence size to 8Gi or run the below command which will patch Persistent Volumes and Persistent Volume Claims for mysql helm and then run the above `helm upgrade` command.
-
-```
-kubectl patch pvc data-mysql-0 -p '{"spec":{"resources":{"requests":{"storage":"50Gi"}}}}'
-kubectl patch pv <mysql-pv> -p '{"spec":{"storage":"50Gi"}}'
-```
-
 ### MySQL Pod fails on Upgrade
 
 {% note %}