NC | Online Upgrade | Replace failure if system.json contains hosts that do not exist in --expected_hosts with a warning

[romayalon]

Explain the changes

1. Replaced the error coming from the condition for failing config dir upgrade if the --expected_hosts list does not match the hosts registered in system.json with a warning in the logs. This change is needed because NooBaa is not cluster aware and we can not take system.json as the source of truth of the current hosts status in the cluster, therefore we will rely on CES to send us the expected hosts list that should contain only hosts the upgraded their noobaa RPM.
2. Note - we still verify that the hosts in the expected hosts list have upgraded noobaa version and that we have information in system.json about the NooBaa version of the host.
3. Changed tests that tested this condition to expect a successful upgrade.

Issues: Fixed #xxx / Gap #xxx

Testing Instructions:

Automatic tests -
sudo jest --testRegex=jest_tests/test_cli_upgrade

Manual tests -

1. Check that upgrade won't fail if system.json contains a host that doesn't exist in expected_hosts (Artificially)-
   a. sudo mkdir /etc/noobaa.conf.d/
   b. checkout to 5.16
   c. start noobaa - sudo node src/cmd/nsfs.js --debug 5, stop noobaa CTRL+C
   d. checkout to 5.18
   e. start noobaa - sudo node src/cmd/nsfs.js --debug 5, stop noobaa CTRL+C
   f. edit system.json and add another hostname object with key that doesn't exist, for example -
   vi cat /etc/noobaa.conf.d/system.json

{"existing_hostname":{"current_version":"5.18.0","upgrade_history":{"successful_upgrades":[{"timestamp":1734620619190,"from_version":"5.16.5","to_version":"5.18.0"}]}}
+ "non_existing_hostname":{"current_version":"5.18.0","upgrade_history":{"successful_upgrades":[{"timestamp":1734620619190,"from_version":"5.16.5","to_version":"5.18.0"}]}}
}

g. run config dir upgrade start with expected_hosts having non only the existing_host -
noobaa-cli upgrade start --expected_version 5.18.0 --expected_hosts existing_host_name
and expect a successful upgrade

{
  "response": {
    "code": "UpgradeSuccessful",
    ....

Inside the logs you can see a warning about it but no failure -

[WARN] core.upgrade.nc_upgrade_manager:: _verify_config_dir_upgrade - system.json contains one or more hosts info that are not specified in expected_hosts hosts_data...

2. Check that upgrade still fails if expected_hosts contains a host that doesn't exist in system.json -
   a. sudo mkdir /etc/noobaa.conf.d/
   b. checkout to 5.16
   c. start noobaa - sudo node src/cmd/nsfs.js --debug 5, stop noobaa CTRL+C
   d. checkout to 5.18
   e. start noobaa - sudo node src/cmd/nsfs.js --debug 5, stop noobaa CTRL+C
   f. run config dir upgrade start with expected_hosts having non existing host -
   noobaa-cli upgrade start --expected_version 5.18.0 --expected_hosts existing_host_name,non_existing_hostname
   and expect a failure -

{
  "error": {
    "code": "UpgradeFailed",
    "message": "Upgrade request failed",
    "cause": "Error: config dir upgrade can not be started - expected_hosts contains one or more hosts that are not specified in system.json hosts_data=...."
  }
}

• Doc added/updated
• Tests added

[shirady]

LGTM

[shirady]

LGTM