[ftr] pass password for UI login + improve login/logout steps with pr…

…oper wait logic (elastic#166936) ## Summary Hopefully closes elastic#167104 closes elastic#167130 closes elastic#167100 closes elastic#167013 closes elastic#166964 Fixing a few issues with login/logout: 1. Failed to login in "before" hook <img width="1336" alt="Screenshot 2023-09-25 at 12 37 45" src="https://github.com/elastic/kibana/assets/10977896/e3b2830e-7b0d-4467-9b90-261b385bf71e"> My theory is that we are loading `/login` route too soon while log out was not completed yet. When we navigate to `https://localhost:5620/logout` there are multiple url re-directions with final page being Cloud login form. This PR makes sure we wait for this form to be displayed + 2500 ms extra to avoid "immediate" /login navigation 2. Failed login on MKI: Updating login via UI for serverless to pass password valid for deployment: currently FTR uses `changeme` for both Kibana CI & MKI. 3. ES activate user profile call returning 500 We saw some login failures that are preceded with the following logs: ``` [00:03:27] │ debg Find.clickByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000 [00:03:27] │ debg Find.findByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000 [00:03:27] │ debg Find.waitForDeletedByCssSelector('.kibanaWelcomeLogo') with timeout=10000 [00:03:27] │ proc [kibana] [2023-09-19T07:08:26.126+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic) [00:03:27] │ info [o.e.x.s.s.SecurityIndexManager] [ftr] security index does not exist, creating [.security-profile-8] with alias [.security-profile] [00:03:27] │ proc [kibana] [2023-09-19T07:08:26.140+00:00][ERROR][plugins.security.user-profile] Failed to activate user profile: {"error":{"root_cause":[{"type":"validation_exception","reason":"Validation Failed: 1: this action would add [1] shards, but this cluster currently has [27]/[27] maximum normal shards open;"}],"type":"validation_exception","reason":"Validation Failed: 1: this action would add [1] shards, but this cluster currently has [27]/[27] maximum normal shards open;"},"status":400}. [00:03:27] │ proc [kibana] [2023-09-19T07:08:26.140+00:00][ERROR][http] 500 Server Error [00:03:27] │ warn browser[SEVERE] http://localhost:5620/internal/security/login - Failed to load resource: the server responded with a status of 500 (Internal Server Error) ``` User activation happens during `POST internal/security/login` call to Kibana server. ~~The only improvement that we can do from FTR perspective is to call this end-point via API to makes sure user is activated and only after proceed with UI login.~~ While working on issue #4 and talking to @jeramysoucy I believe retrying login via UI will work here as well. We are checking if we are still on login page (similar to incorrect password login), waiting 2500 ms and pressing login button again. 4. Failed to login with Kibana reporting UNEXPECTED_SESSION_ERROR and been re-directed to Cloud login page ``` proc [kibana] [2023-09-25T11:35:12.794+00:00][INFO ][plugins.security.authentication] Authentication attempt failed: UNEXPECTED_SESSION_ERROR ``` Temporary solution is to retry login from scratch (navigation to Kibana login page & re-login ) Flaky-test-runner for functional obtl tests 50x https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3215 This PR is not fixing random 401 response when user navigates to some apps with direct url --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
semd · Sep 28, 2023 · 2eddd54 · 2eddd54
1 parent 0d97cc7
commit 2eddd54
Show file tree

Hide file tree

Showing 2 changed files with 85 additions and 4 deletions.
diff --git a/x-pack/test/functional/page_objects/security_page.ts b/x-pack/test/functional/page_objects/security_page.ts
@@ -168,7 +168,7 @@ export class SecurityPageObject extends FtrService {
     );
   }
 
-  private async isLoginFormVisible() {
+  public async isLoginFormVisible() {
     return await this.testSubjects.exists('loginForm');
   }
 
@@ -323,7 +323,14 @@ export class SecurityPageObject extends FtrService {
         if (alert?.accept) {
           await alert.accept();
         }
-        return !(await this.browser.getCurrentUrl()).includes('/logout');
+
+        if (this.config.get('serverless')) {
+          // Logout might trigger multiple redirects, but in the end we expect the Cloud login page
+          this.log.debug('Wait 5 sec for Cloud login page to be displayed');
+          return await this.find.existsByDisplayedByCssSelector('.login-form-password', 5000);
+        } else {
+          return !(await this.browser.getCurrentUrl()).includes('/logout');
+        }
       });
     }
   }

diff --git a/x-pack/test_serverless/functional/page_objects/svl_common_page.ts b/x-pack/test_serverless/functional/page_objects/svl_common_page.ts
@@ -10,16 +10,90 @@ import { FtrProviderContext } from '../ftr_provider_context';
 export function SvlCommonPageProvider({ getService, getPageObjects }: FtrProviderContext) {
   const testSubjects = getService('testSubjects');
   const config = getService('config');
-  const pageObjects = getPageObjects(['security']);
+  const pageObjects = getPageObjects(['security', 'common']);
+  const retry = getService('retry');
+  const deployment = getService('deployment');
+  const log = getService('log');
+  const browser = getService('browser');
+
+  const delay = (ms: number) =>
+    new Promise((resolve) => {
+      setTimeout(resolve, ms);
+    });
 
   return {
+    async navigateToLoginForm() {
+      const url = deployment.getHostPort() + '/login';
+      await browser.get(url);
+      // ensure welcome screen won't be shown. This is relevant for environments which don't allow
+      // to use the yml setting, e.g. cloud
+      await browser.setLocalStorageItem('home:welcome:show', 'false');
+
+      log.debug('Waiting for Login Form to appear.');
+      await retry.waitForWithTimeout('login form', 10_000, async () => {
+        return await pageObjects.security.isLoginFormVisible();
+      });
+    },
+
     async login() {
       await pageObjects.security.forceLogout({ waitForLoginPage: false });
-      return await pageObjects.security.login(config.get('servers.kibana.username'));
+
+      // adding sleep to settle down logout
+      await pageObjects.common.sleep(2500);
+
+      await retry.waitForWithTimeout(
+        'Waiting for successful authentication',
+        90_000,
+        async () => {
+          if (!(await testSubjects.exists('loginUsername', { timeout: 1000 }))) {
+            await this.navigateToLoginForm();
+
+            await testSubjects.setValue('loginUsername', config.get('servers.kibana.username'));
+            await testSubjects.setValue('loginPassword', config.get('servers.kibana.password'));
+            await testSubjects.click('loginSubmit');
+          }
+
+          if (await testSubjects.exists('userMenuButton', { timeout: 10_000 })) {
+            log.debug('userMenuButton is found, logged in passed');
+            return true;
+          } else {
+            throw new Error(`Failed to login to Kibana via UI`);
+          }
+        },
+        async () => {
+          // Sometimes authentication fails and user is redirected to Cloud login page
+          // [plugins.security.authentication] Authentication attempt failed: UNEXPECTED_SESSION_ERROR
+          const currentUrl = await browser.getCurrentUrl();
+          if (currentUrl.startsWith('https://cloud.elastic.co')) {
+            log.debug(
+              'Probably authentication attempt failed, we are at Cloud login page. Retrying from scratch'
+            );
+          } else {
+            const authError = await testSubjects.exists('promptPage', { timeout: 2500 });
+            if (authError) {
+              log.debug('Probably SAML callback page, doing logout again');
+              await pageObjects.security.forceLogout({ waitForLoginPage: false });
+            } else {
+              const isOnLoginPage = await testSubjects.exists('loginUsername', { timeout: 1000 });
+              if (isOnLoginPage) {
+                log.debug(
+                  'Probably ES user profile activation failed, waiting 2 seconds and pressing Login button again'
+                );
+                await delay(2000);
+                await testSubjects.click('loginSubmit');
+              } else {
+                log.debug('New behaviour, trying to navigate and login again');
+              }
+            }
+          }
+        }
+      );
+      log.debug('Logged in successfully');
     },
 
     async forceLogout() {
       await pageObjects.security.forceLogout({ waitForLoginPage: false });
+      log.debug('Logged out successfully');
     },
 
     async assertProjectHeaderExists() {