Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Block when cP accts have DBs on distro PgSQL #327

Merged
merged 1 commit into from
Jan 19, 2024
Merged

Block when cP accts have DBs on distro PgSQL #327

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions docs-website-src/content/blockers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,7 @@ At any given time, the upgrade process may use at or more than 5 GB. If you have

The following software is known to lead to a corrupt install if this script is used. We block elevation when it is detected:

* **cPanel CCS Calendar Server** - Requires Postgresql < 10.0
* **Postgresql** - ELevate upgrades you to Postgresql 10.x which makes it impossible to downgrade to a 9.x Postgresql.
* **cPanel CCS Calendar Server** - Requires Postgresql older than 10.0

## Things you need to upgrade first.

Expand Down Expand Up @@ -64,6 +63,21 @@ You can discover many of these issues by downloading `elevate-cpanel` and runnin

The following is a list of other known issues that could prevent your server's successful elevation.

## PostgreSQL

If you are using the PostgreSQL software provided by your distro (which includes PostgreSQL as installed by cPanel), ELevate will upgrade the software packages. However, your PostgreSQL service is unlikely to start properly. The reason for this is that ELevate will **not** attempt to update the data directory being used by your PostgreSQL instance to store settings and databases; and PostgreSQL will detect this condition and refuse to start, to protect your data from corruption, until you have performed this update.

To ensure that you are aware of this requirement, if it detects that one or more cPanel accounts have associated PostgreSQL databases, ELevate will block you from beginning the upgrade process until you have created a file at `/var/cpanel/acknowledge_postgresql_for_elevate`.

### Updating the PostgreSQL data directory

Once ELevate has completed, you should then perform the update to the PostgreSQL data directory. Although we defer to the information [in the PostgreSQL documentation itself](https://www.postgresql.org/docs/10/pgupgrade.html), and although [Red Hat has provided steps in their documentation](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/deploying_different_types_of_servers/using-databases#migrating-to-a-rhel-8-version-of-postgresql_using-postgresql) which should be mostly applicable to all distros derived from RHEL 8, we found that the following steps worked in our testing to update the PostgreSQL data directory. (Please note that these steps assume that your server's data directory is located at `/var/lib/pgsql/data`; your server may be different. You should also consider making a backup copy of your data directory before starting, because **cPanel cannot guarantee the correctness of these steps for any arbitrary PostgreSQL installation**.)

1. Install the `postgresql-upgrade` package: `dnf install postgresql-upgrade`
2. Within your PostgreSQL config file at `/var/lib/pgsql/data/postgresql.conf`, if there exists an active option `unix_socket_directories`, change that phrase to read `unix_socket_directory`. This is necessary to work around a difference between the CentOS 7 PostgreSQL 9.2 and the PostgreSQL 9.2 helpers packaged by your new operating system's `postgresql-upgrade` package.
3. Invoke the `postgresql-setup` tool: `/usr/bin/postgresql-setup --upgrade`.
4. In the root user's WHM, navigate to the "Configure PostgreSQL" area and click on "Install Config". This should restore the additions cPanel makes to the PostgreSQL access controls in order to allow phpPgAdmin to function.

## Using OVH proactive intervention monitoring

If you are using a dedicated server hosted at OVH, you should **disable the `proactive monitoring` before starting** the elevation process.
Expand Down
74 changes: 71 additions & 3 deletions elevate-cpanel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -467,8 +467,10 @@ BEGIN { # Suppress load of all of these at earliest point.

use cPstrict;

use Cpanel::OS ();
use Cpanel::Version::Tiny ();
use Cpanel::OS ();
use Cpanel::Version::Tiny ();
use Cpanel::JSON ();
use Cpanel::SafeRun::Simple ();

# use Elevate::Blockers::Base();
our @ISA;
Expand All @@ -477,9 +479,13 @@ BEGIN { # Suppress load of all of these at earliest point.
# use Log::Log4perl qw(:easy);
INIT { Log::Log4perl->import(qw{:easy}); }

use constant POSTGRESQL_ACK_TOUCH_FILE => q[/var/cpanel/acknowledge_postgresql_for_elevate];

sub check ($self) {
my $ok = 1;
$self->_warning_if_postgresql_installed;
my $ok = $self->_blocker_old_mysql;
$ok = 0 unless $self->_blocker_acknowledge_postgresql_datadir;
$ok = 0 unless $self->_blocker_old_mysql;
$ok = 0 unless $self->_blocker_mysql_upgrade_in_progress;
$self->_warning_mysql_not_enabled();
return $ok;
Expand All @@ -498,6 +504,68 @@ BEGIN { # Suppress load of all of these at earliest point.
return 2;
}

sub _blocker_acknowledge_postgresql_datadir ($self) {

return 0 unless Cpanel::Pkgr::is_installed('postgresql-server');

my $touch_file = POSTGRESQL_ACK_TOUCH_FILE;
return 0 if -e $touch_file;

my @users_with_dbs = $self->_has_mapped_postgresql_dbs();
return 0 unless scalar @users_with_dbs;

my $message = <<~"EOS";
One or more users on your system have associated PostgreSQL databases.
ELevate may upgrade the software packages associated with PostgreSQL
automatically, but if it does, it will *NOT* automatically update the
PostgreSQL data directory to work with the new version. Without an update
to the data directory, the upgraded PostgreSQL software will not start, in
order to ensure that your data does not become corrupted.

For more information about PostgreSQL upgrades, please consider the
following resources:

https://cpanel.github.io/elevate/blockers/#postgresql
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/deploying_different_types_of_servers/using-databases#migrating-to-a-rhel-8-version-of-postgresql_using-postgresql
https://www.postgresql.org/docs/10/pgupgrade.html

When you are ready to acknowledge that you have prepared to update the
PostgreSQL data directory, or that this warning does not apply to you,
please touch the following file to continue with the ELevate process:

> touch $touch_file

The following user(s) have PostgreSQL databases associated with their cPanel accounts:
EOS

$message .= join "\n", sort(@users_with_dbs);

return $self->has_blocker($message);
}

sub _has_mapped_postgresql_dbs ($self) {

local $ENV{HTTP_COOKIE} = 'session_locale=en';

my $users_json = Cpanel::SafeRun::Simple::saferunnoerror(qw(/usr/local/cpanel/bin/whmapi1 --output=json list_users));
my $users_parsed = Cpanel::JSON::Load($users_json);
die "WHM API1 list_users call returned result 0: $users_parsed->{metadata}->{reason}" unless $users_parsed->{metadata}->{result};
my @users = grep { $_ ne "root" } $users_parsed->{data}->{users}->@*;

my @users_with_dbs;
foreach my $user (@users) {
my $dbs_json = Cpanel::SafeRun::Simple::saferunnoerror( "/usr/local/cpanel/bin/uapi", "--user=$user", qw(--output=json Postgresql list_databases) );
my $dbs_parsed = Cpanel::JSON::Load($dbs_json);
if ( !$dbs_parsed->{result}->{status} ) {
return if ( $dbs_parsed->{result}->{errors}->[0] // '' ) eq 'This server does not support this functionality.'; # sanity check
die( "UAPI Postgresql::list_databases call returned status 0 for user $user:\n" . join( "\n", $dbs_parsed->{result}->{errors}->@* ) );
}
push @users_with_dbs, $user if scalar $dbs_parsed->{result}->{data}->@*;
}

return @users_with_dbs;
}

sub _blocker_old_mysql ( $self, $mysql_version = undef ) {

$mysql_version //= $self->cpconf->{'mysql-version'} // '';
Expand Down
77 changes: 74 additions & 3 deletions lib/Elevate/Blockers/Databases.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,22 @@ Blockers for datbase: MySQL, PostgreSQL...

use cPstrict;

use Cpanel::OS ();
use Cpanel::Version::Tiny ();
use Cpanel::OS ();
use Cpanel::Version::Tiny ();
use Cpanel::JSON ();
use Cpanel::SafeRun::Simple ();

use parent qw{Elevate::Blockers::Base};

use Log::Log4perl qw(:easy);

use constant POSTGRESQL_ACK_TOUCH_FILE => q[/var/cpanel/acknowledge_postgresql_for_elevate];

sub check ($self) {
my $ok = 1;
$self->_warning_if_postgresql_installed;
my $ok = $self->_blocker_old_mysql;
$ok = 0 unless $self->_blocker_acknowledge_postgresql_datadir;
$ok = 0 unless $self->_blocker_old_mysql;
$ok = 0 unless $self->_blocker_mysql_upgrade_in_progress;
$self->_warning_mysql_not_enabled();
return $ok;
Expand All @@ -40,6 +46,71 @@ sub _warning_if_postgresql_installed ($self) {
return 2;
}

sub _blocker_acknowledge_postgresql_datadir ($self) {

return 0 unless Cpanel::Pkgr::is_installed('postgresql-server');

my $touch_file = POSTGRESQL_ACK_TOUCH_FILE;
return 0 if -e $touch_file;

my @users_with_dbs = $self->_has_mapped_postgresql_dbs();
return 0 unless scalar @users_with_dbs;

my $message = <<~"EOS";
One or more users on your system have associated PostgreSQL databases.
ELevate may upgrade the software packages associated with PostgreSQL
automatically, but if it does, it will *NOT* automatically update the
PostgreSQL data directory to work with the new version. Without an update
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we run the script to upgrade them? Why can't we use the psql channel to switch them to 8.3 instead of 10?

Copy link
Contributor Author

@sloanebernstein sloanebernstein Dec 20, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can't be 8.3 because downgrades are not possible.

We can upgrade them. It could be a candidate for a leapp actor, even, though that assumes that AlmaLinux wants something like that in their fork. (I'm assuming that there's a reason Red Hat never did something similar for their original version of leapp, in favor of producing documentation instead.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I meant 9.6 See these old instructions:

-    You have postgresql-server version 9.2 installed.
-    This is upgraded irreversably to version 10.0 when you switch to AlmaLinux 8
-    We recommend data backup and removal of all postgresql packages before upgrade to AlmaLinux 8.
-    To re-install postgresql 9 on AlmaLinux 8, you can run: `dnf -y module enable postgresql:9.6; dnf -y install postgresql-server`

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory we could do this for them...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but we would still need to perform the data directory update, because prior to PostgreSQL 10, the first two elements of the version number are considered the "major" version: 9.2.x to 9.6.x is a major upgrade.

to the data directory, the upgraded PostgreSQL software will not start, in
order to ensure that your data does not become corrupted.

For more information about PostgreSQL upgrades, please consider the
following resources:

https://cpanel.github.io/elevate/blockers/#postgresql
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/deploying_different_types_of_servers/using-databases#migrating-to-a-rhel-8-version-of-postgresql_using-postgresql
https://www.postgresql.org/docs/10/pgupgrade.html

When you are ready to acknowledge that you have prepared to update the
PostgreSQL data directory, or that this warning does not apply to you,
please touch the following file to continue with the ELevate process:

> touch $touch_file

The following user(s) have PostgreSQL databases associated with their cPanel accounts:
EOS

$message .= join "\n", sort(@users_with_dbs);

return $self->has_blocker($message);
}

sub _has_mapped_postgresql_dbs ($self) {

# Normalize locale:
local $ENV{HTTP_COOKIE} = 'session_locale=en';

# Get a list of cPanel users:
my $users_json = Cpanel::SafeRun::Simple::saferunnoerror(qw(/usr/local/cpanel/bin/whmapi1 --output=json list_users));
my $users_parsed = Cpanel::JSON::Load($users_json);
die "WHM API1 list_users call returned result 0: $users_parsed->{metadata}->{reason}" unless $users_parsed->{metadata}->{result};
my @users = grep { $_ ne "root" } $users_parsed->{data}->{users}->@*;

# Compile a list of users with PgSQL DBs:
my @users_with_dbs;
foreach my $user (@users) {
my $dbs_json = Cpanel::SafeRun::Simple::saferunnoerror( "/usr/local/cpanel/bin/uapi", "--user=$user", qw(--output=json Postgresql list_databases) );
sloanebernstein marked this conversation as resolved.
Show resolved Hide resolved
my $dbs_parsed = Cpanel::JSON::Load($dbs_json);
if ( !$dbs_parsed->{result}->{status} ) {
return if ( $dbs_parsed->{result}->{errors}->[0] // '' ) eq 'This server does not support this functionality.'; # sanity check
die( "UAPI Postgresql::list_databases call returned status 0 for user $user:\n" . join( "\n", $dbs_parsed->{result}->{errors}->@* ) );
}
push @users_with_dbs, $user if scalar $dbs_parsed->{result}->{data}->@*;
}

return @users_with_dbs;
}

sub _blocker_old_mysql ( $self, $mysql_version = undef ) {

$mysql_version //= $self->cpconf->{'mysql-version'} // '';
Expand Down
85 changes: 85 additions & 0 deletions t/blocker-Databases.t
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,4 +157,89 @@ my $mock_elevate = Test::MockFile->file('/var/cpanel/elevate');

}

{
note "PostgreSQL 9.2->10 acknowledgement";

my $pkgr_mock = Test::MockModule->new('Cpanel::Pkgr');
my %installed = ( 'postgresql-server' => 9.2 );
$pkgr_mock->redefine( 'is_installed' => sub ($rpm) { return defined $installed{$rpm} ? 1 : 0 } );
$pkgr_mock->redefine( 'get_package_version' => sub ($rpm) { return $installed{$rpm} } );

my $mock_touchfile = Test::MockFile->file('/var/cpanel/acknowledge_postgresql_for_elevate');

my @mock_users = qw(cpuser1 cpuser2);
$db_mock->redefine( _has_mapped_postgresql_dbs => sub { return @mock_users } );

my $expected = {
id => q[Elevate::Blockers::Databases::_blocker_acknowledge_postgresql_datadir],
msg => <<~'EOS'
One or more users on your system have associated PostgreSQL databases.
ELevate may upgrade the software packages associated with PostgreSQL
automatically, but if it does, it will *NOT* automatically update the
PostgreSQL data directory to work with the new version. Without an update
to the data directory, the upgraded PostgreSQL software will not start, in
order to ensure that your data does not become corrupted.

For more information about PostgreSQL upgrades, please consider the
following resources:

https://cpanel.github.io/elevate/blockers/#postgresql
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/deploying_different_types_of_servers/using-databases#migrating-to-a-rhel-8-version-of-postgresql_using-postgresql
https://www.postgresql.org/docs/10/pgupgrade.html

When you are ready to acknowledge that you have prepared to update the
PostgreSQL data directory, or that this warning does not apply to you,
please touch the following file to continue with the ELevate process:

> touch /var/cpanel/acknowledge_postgresql_for_elevate

The following user(s) have PostgreSQL databases associated with their cPanel accounts:
cpuser1
cpuser2
EOS
};
chomp $expected->{msg};
is(
$db->_blocker_acknowledge_postgresql_datadir(),
$expected,
"PostgreSQL with cPanel users having databases and without ACK touch file is a blocker"
);

%installed = ();
is( $db->_blocker_acknowledge_postgresql_datadir(), 0, "No blocker if no postgresql-server package" );
%installed = ( 'postgresql-server' => 9.2 );

$mock_touchfile->touch();
is( $db->_blocker_acknowledge_postgresql_datadir(), 0, "No blocker if touch file present" );
$mock_touchfile->unlink();

@mock_users = ();
is( $db->_blocker_acknowledge_postgresql_datadir(), 0, "No blocker if no users have PgSQL DBs" );
@mock_users = qw(cpuser1 cpuser2);
}

{
note "check for PostgreSQL databases";

my $mock_saferun = Test::MockModule->new('Cpanel::SafeRun::Simple');

$mock_saferun->redefine(
saferunnoerror => sub {
return $_[0] eq '/usr/local/cpanel/bin/whmapi1'
? '{"metadata":{"command":"list_users","version":1,"reason":"OK","result":1},"data":{"users":["root","cpuser2","cpuser1"]}}'
: '{"apiversion":3,"module":"Postgresql","func":"list_databases","result":{"warnings":null,"data":[{"disk_usage":9001,"database":"dontcare","users":["dontcare"]}],"errors":null,"metadata":{"transformed":1},"status":1,"messages":null}}';
}
);

is(
[ $db->_has_mapped_postgresql_dbs() ],
bag {
item 'cpuser1';
item 'cpuser2';
end();
},
"_has_mapped_postgresql_dbs returns expected list of users"
);
}

done_testing();