Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

procedures: Add PostgreSQL migration guides #2163

Closed
wants to merge 20 commits into from
Closed
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
2e542c1
Database migration guides
tolusha Nov 15, 2021
734b733
Fixup
tolusha Nov 16, 2021
439974a
fixups
tolusha Nov 16, 2021
364ad3c
Merge branch 'master' into backuprestoredb
tolusha Nov 17, 2021
639e6f8
Merge branch 'master' into backuprestoredb
tolusha Nov 23, 2021
22a45e1
Merge branch 'master' into backuprestoredb
themr0c Dec 9, 2021
0e5c536
Merge branch 'master' into backuprestoredb
themr0c Dec 9, 2021
30d48f9
Merge branch 'master' into backuprestoredb
themr0c Dec 10, 2021
70d5033
Merge branch 'master' into backuprestoredb
themr0c Dec 16, 2021
dd76b1c
Merge branch 'master' into backuprestoredb
themr0c Dec 22, 2021
d2be4ed
Merge branch 'master' into backuprestoredb
themr0c Jan 21, 2022
5b31623
Update modules/administration-guide/partials/snip_finding-che-user-na…
themr0c Jan 25, 2022
34508cf
Update modules/administration-guide/partials/proc_cleaning-up-postgre…
max-cx Oct 26, 2022
6ad0aee
Update modules/administration-guide/partials/proc_cleaning-up-postgre…
max-cx Oct 26, 2022
b4ee1ad
Update modules/administration-guide/partials/proc_cleaning-up-postgre…
max-cx Oct 26, 2022
38bc83a
Update modules/administration-guide/partials/proc_cleaning-up-postgre…
max-cx Oct 26, 2022
4fa32d5
Update modules/administration-guide/partials/proc_cleaning-up-postgre…
max-cx Oct 26, 2022
ced5002
Update modules/administration-guide/partials/proc_cleaning-up-postgre…
max-cx Oct 26, 2022
8e0fced
Update modules/administration-guide/partials/proc_backuping-postgresq…
max-cx Oct 26, 2022
da9c72a
Update modules/administration-guide/partials/proc_backuping-postgresq…
max-cx Oct 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions modules/administration-guide/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,10 @@
*** xref:define-backup-server-for-operator.adoc[]
** xref:persistent-volumes-backups.adoc[]
** xref:external-database-setup.adoc[]
** xref:migration-to-external-database-setup.adoc[]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
** xref:migration-to-external-database-setup.adoc[]
** xref:migration-to-an-external-database-setup.adoc[]


* xref:migration-from-postgresql-9-to-postgresql-13.adoc[]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* xref:migration-from-postgresql-9-to-postgresql-13.adoc[]
* xref:upgrading-postgresql.adoc[]

* xref:migration-from-helm-charts-to-operator.adoc[]

* xref:readiness-init-containers.adoc[]
** xref:enabling-and-disabling-readiness-init-containers-for-the-operator-installer.adoc[]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[id="migration-from-helm-charts-to-operator"]
:navtitle: Migration from Helm Charts to Operator
:keywords: administration-guide, migration, helm-charts, operator, database
:page-aliases: .:migration-from-helm-charts-to-operator

include::partial$assembly_migration-from-helm-charts-to-operator.adoc[]
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[id="migration-from-postgresql-9-to-postgresql-13"]
:navtitle: Migration from PostgreSQL 9 to PostgreSQL 13
Comment on lines 1 to 2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[id="migration-from-postgresql-9-to-postgresql-13"]
:navtitle: Migration from PostgreSQL 9 to PostgreSQL 13
[id="upgrading-postgresql"]
:navtitle: Upgrading PostgreSQL

:keywords: administration-guide, migration
:page-aliases: .:migration, .:database
:keywords: administration-guide, migration, database
:page-aliases: .:migration-database

include::partial$proc_migration_from_postgresql_9_to_postgresql_13.adoc[]
include::partial$assembly_migration-from-postgresql-9-to-postgresql-13.adoc[]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
include::partial$assembly_migration-from-postgresql-9-to-postgresql-13.adoc[]
include::partial$assembly_upgrading-postgresql.adoc[]

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[id="migration-to-external-database-setup"]
:navtitle: Migration to external database setup
Comment on lines +1 to +2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[id="migration-to-external-database-setup"]
:navtitle: Migration to external database setup
[id="migration-to-an-external-database-setup"]
:navtitle: Migration to an external database setup

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll probably need to change the file name too.

:keywords: administration-guide, external-database, migration
:page-aliases: .:migration-to-external-database-setup

include::partial$assembly_migration-to-external-database-setup.adoc[]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might have suggested renaming that file too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
include::partial$assembly_migration-to-external-database-setup.adoc[]
include::partial$assembly_migration-to-an-external-database-setup.adoc[]

Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@


:parent-context-of-migration-from-helm-charts-to-operator: {context}

[id="migration-from-helm-charts-to-operator_{context}"]
= Migration from Helm Charts to Operator

:context: migration-from-helm-charts-to-operator

Follow the procedures below to migrate from Helm Charts to Operator successfully without any data loss.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Follow the procedures below to migrate from Helm Charts to Operator successfully without any data loss.
You can migrate from Helm Charts to Operator without losing data.


.Prerequisites

* The `{orch-cli}` tool is available.
* The `{prod-cli}` management tool is available. See xref:overview:using-the-chectl-management-tool.adoc[].
* An instance of {prod-short} running in {orch-name}.

include::partial$proc_backuping-postgresql.adoc[leveloffset=+1]

include::partial$proc_reinstalling-che.adoc[leveloffset=+1]

include::partial$proc_cleaning-up-postgresql.adoc[leveloffset=+1]

include::partial$proc_restoring-internal-postgresql.adoc[leveloffset=+1]

include::partial$snip_verification-che-working.adoc[]

:context: {parent-context-of-migration-from-helm-charts-to-operator}

Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@


:parent-context-of-migration-from-postgresql-9-to-postgresql-13: {context}

[id="migration-from-postgresql-9-to-postgresql-13_{context}"]
= Migration from PostgreSQL 9 to PostgreSQL 13

:context: migration-from-postgresql-9-to-postgresql-13
Comment on lines +3 to +8
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:parent-context-of-migration-from-postgresql-9-to-postgresql-13: {context}
[id="migration-from-postgresql-9-to-postgresql-13_{context}"]
= Migration from PostgreSQL 9 to PostgreSQL 13
:context: migration-from-postgresql-9-to-postgresql-13
:parent-context-of-upgrading-postgresql: {context}
[id="upgrading-postgresql_{context}"]
= Upgrading PostgreSQL
:context: upgrading-postgresql


By the 11th of November, 2021, the PostgreSQL version 9.6 came out of support, and {prod-short} team recommends that all users undergo migrating to version 13.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a release note to me. If we need to tell this information to customers, I propose that we create a release note and link to this procedure from the release note.
Let me know what you think about this possibility.


Follow the procedures below to migrate to a newer version of PostgreSQL successfully without any data loss.
Copy link
Contributor

@max-cx max-cx Oct 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Follow the procedures below to migrate to a newer version of PostgreSQL successfully without any data loss.
You can upgrade to a newer version of PostgreSQL without losing data.


.Prerequisites

* The `{orch-cli}` tool is available.
* An instance of {prod-short} running in {orch-name}.

include::partial$proc_backuping-postgresql.adoc[leveloffset=+1]

include::partial$proc_updating-postgresql-version.adoc[leveloffset=+1]

include::partial$proc_provisioning-postgresql.adoc[leveloffset=+1]

include::partial$proc_restoring-internal-postgresql.adoc[leveloffset=+1]

include::partial$snip_verification-che-working.adoc[]

:context: {parent-context-of-migration-from-postgresql-9-to-postgresql-13}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:context: {parent-context-of-migration-from-postgresql-9-to-postgresql-13}
:context: {parent-context-of-upgrading-postgresql}


Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@


:parent-context-of-migration-to-external-database-setup: {context}

[id="migration-to-external-database-setup_{context}"]
= Migration to external database setup
Comment on lines +5 to +6
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[id="migration-to-external-database-setup_{context}"]
= Migration to external database setup
[id="migration-to-an-external-database-setup_{context}"]
= Migration to an external database setup

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll probably need to change the file name too.


:context: migration-to-external-database-setup

Follow the procedures below to migrate from internal to external database setup successfully without any data loss.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Follow the procedures below to migrate from internal to external database setup successfully without any data loss.
You can migrate from the internal database to an external database without losing data.


.Prerequisites

* The `{orch-cli}` tool is available.
* An instance of {prod-short} running in {orch-name}.

include::partial$proc_backuping-postgresql.adoc[leveloffset=+1]

:reuse-database-user: true
include::partial$proc_configuring-external-PostgreSQL.adoc[leveloffset=+1]

include::partial$proc_switching-che-to-work-with-external-postgresql.adoc[leveloffset=+1]

include::partial$proc_restoring-external-postgresql.adoc[leveloffset=+1]

include::partial$snip_verification-che-working.adoc[]

:context: {parent-context-of-migration-to-external-database-setup}

Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ The recommended configuration of the example database is:
|Automated backups | enabled (30 days)
|===

:reuse-database-user: false
include::partial$proc_configuring-external-PostgreSQL.adoc[leveloffset=+1]

include::partial$proc_configuring-che-to-work-with-external-PostgreSQL.adoc[leveloffset=+1]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
[id="backuping-postgresql{context}"]
= Backuping PostgreSQL
Comment on lines +1 to +2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
[id="backuping-postgresql{context}"]
= Backuping PostgreSQL
[id="backing-up-postgresql{context}"]
= Backing up PostgreSQL

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll probably need to change the file name too.


.Procedure

. Save and push changes back to the Git repositories for all running workspaces of the {prod-short} instance.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
. Save and push changes back to the Git repositories for all running workspaces of the {prod-short} instance.
. Save and push changes to the Git repositories for all running workspaces of the {prod-short} instance.

+
. Stop all workspaces in the {prod-short} instance.
+
include::partial$snip_scaling-down-che.adoc[]

include::partial$snip_scaling-down-keycloak.adoc[]

include::partial$snip_finding-postgresql-pod.adoc[]

include::partial$snip_finding-che-database-name.adoc[]

. Backup {prod-short} database:
+
[subs="+quotes,+attributes"]
----
{orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "pg_dump $CHE_POSTGRES_DB > /tmp/che.sql"
{orch-cli} cp {prod-namespace}/$POSTGRES_POD:/tmp/che.sql che.sql
Comment on lines +22 to +23
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "pg_dump $CHE_POSTGRES_DB > /tmp/che.sql"
{orch-cli} cp {prod-namespace}/$POSTGRES_POD:/tmp/che.sql che.sql
$ {orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "pg_dump $CHE_POSTGRES_DB > /tmp/che.sql"
$ {orch-cli} cp {prod-namespace}/$POSTGRES_POD:/tmp/che.sql che.sql

----

. Backup {identity-provider} database:
+
[subs="+quotes,+attributes"]
----
{orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "pg_dump keycloak > /tmp/keycloak.sql"
{orch-cli} cp {prod-namespace}/$POSTGRES_POD:/tmp/keycloak.sql keycloak.sql
Comment on lines +30 to +31
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "pg_dump keycloak > /tmp/keycloak.sql"
{orch-cli} cp {prod-namespace}/$POSTGRES_POD:/tmp/keycloak.sql keycloak.sql
$ {orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "pg_dump keycloak > /tmp/keycloak.sql"
$ {orch-cli} cp {prod-namespace}/$POSTGRES_POD:/tmp/keycloak.sql keycloak.sql

----
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
[id="cleaning-up-postgresql{context}"]
= Cleaning up PostgreSQL

.Procedure

include::partial$snip_scaling-down-che.adoc[]

include::partial$snip_scaling-down-keycloak.adoc[]

include::partial$snip_finding-postgresql-pod.adoc[]

include::partial$snip_finding-che-database-name.adoc[]

. Drop {identity-provider} database:
+
[subs="+quotes,+attributes"]
----
{orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "dropdb keycloak"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "dropdb keycloak"
$ {orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "dropdb keycloak"

----

. Create {identity-provider} database:
+
[subs="+quotes,+attributes"]
----
{orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "psql postgres -tAc \"CREATE DATABASE keycloak\""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "psql postgres -tAc \"CREATE DATABASE keycloak\""
$ {orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "psql postgres -tAc \"CREATE DATABASE keycloak\""

----

. Drop {prod-short} database:
+
[subs="+quotes,+attributes"]
----
{orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "dropdb $CHE_POSTGRES_DB"
----

. Create {prod-short} database:
+
[subs="+quotes,+attributes"]
----
{orch-cli} exec -it $POSTGRES_POD -n {prod-namespace} -- bash -c "psql postgres -tAc \"CREATE DATABASE $CHE_POSTGRES_DB\""
----

Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
[id="configuring-external-PostgreSQL_{context}"]
= Configuring external PostgreSQL

By configuring the external PostgreSQL, you can make the workspace metadata and the user information persistent.
By configuring the external PostgreSQL, you can make the workspace metadata and the user information persistent.

.Procedure

Expand All @@ -12,15 +12,24 @@ By configuring the external PostgreSQL, you can make the workspace metadata and
--
* `<database-user>` is the {prod-short} server database user name
* `<database-password>` is the {prod-short} server database password
* `<database>` is the {prod-short} server database name
* `<database>` is the {prod-short} server database name
--
ifeval::["{reuse-database-user}" == "true"]
WARNING: Database user name should be reused from the existed {prod} deployment. Run the following command on machine with access to the {kubernetes} or OpenShift cluster and use obtained value as placeholder above:
+
[subs="+quotes,+attributes"]
----
include::partial$snip_finding-che-user-name.adoc[]
echo $CHE_USER_NAME
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
echo $CHE_USER_NAME
$ echo $CHE_USER_NAME

----
endif::[]

. Use the following SQL script to create a user and a database for the {prod-short} server to make workspace metadata persistent:
+
[subs="+quotes,+attributes"]
----
CREATE USER <database-user> WITH PASSWORD '<database-password>'
CREATE DATABASE <database>
CREATE USER <database-user> WITH PASSWORD '<database-password>'
CREATE DATABASE <database>
GRANT ALL PRIVILEGES ON DATABASE <database> TO <database-user>
ALTER USER <database-user> WITH SUPERUSER
Comment on lines +31 to 34
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
CREATE USER <database-user> WITH PASSWORD '<database-password>'
CREATE DATABASE <database>
GRANT ALL PRIVILEGES ON DATABASE <database> TO <database-user>
ALTER USER <database-user> WITH SUPERUSER
$ CREATE USER <database-user> WITH PASSWORD '<database-password>'
$ CREATE DATABASE <database>
$ GRANT ALL PRIVILEGES ON DATABASE <database> TO <database-user>
$ ALTER USER <database-user> WITH SUPERUSER

----
Expand All @@ -35,7 +44,7 @@ ALTER USER <database-user> WITH SUPERUSER
+
[subs="+quotes,+attributes"]
----
CREATE USER keycloak WITH PASSWORD '<identity-database-password>'
CREATE USER keycloak WITH PASSWORD '<identity-database-password>'
CREATE DATABASE keycloak
GRANT ALL PRIVILEGES ON DATABASE keycloak TO keycloak
Comment on lines +47 to 49
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
CREATE USER keycloak WITH PASSWORD '<identity-database-password>'
CREATE DATABASE keycloak
GRANT ALL PRIVILEGES ON DATABASE keycloak TO keycloak
$ CREATE USER keycloak WITH PASSWORD '<identity-database-password>'
$ CREATE DATABASE keycloak
$ GRANT ALL PRIVILEGES ON DATABASE keycloak TO keycloak

----
----

This file was deleted.

Loading