Multi tenant support (#2523)

* Split sap system database aggregates (#2445) * Add new database aggregate code * Remove database logic from sap system aggregate * Redirect router commands to database aggregate * Adapt deregistration process manager * Adapt enrichment protocol * Adapt sap system discovery policy * Adapt remaining code alias and tests * Adapt related e2e tests * Update the aggregates docstring * Leftover dosctring and pattern matching entries * Move database events (#2452) * Create new databases context file * Move the database events to its folder * Sap system deregistration/restore based on the database (#2456) * Add relationship between database and sap systems readmodel * Add enriched database id to RegisterApplicationInstance command * Add database id to SapSystemRegistered event * Sap system projector projects database association * Changed sap system view to reflect readmodels changes * Update test factory * Add explicit Restore/Deregistration to SapSystem aggregate * Add event handlers for deregistration/restore of sap systems * mix credo * Refactor database instances relationship in sap systems rm * Route new sap system commands in commanded * Add db host and tenant in restore sap system command * Fix database restore event handler test * Addressing feedbacks, fix testing * Database rollup (#2469) * Create required rollup command and events * Update the database aggregate to use the rollup logic * Add rollup events to database lifespan * Handle the rollup events * Use database rollup in deregistration process manager * Move database projections (#2454) * Update database read models and projections * Adapt all the aliases and new fields * Adapt tests to use the new database projections * Fix dialyzer warning * Fix conflicted aliases and tests * Move database controller views (#2472) * Move the database controller * Move the database view * Remove fields that cause errors in view rendering * Adapt frontend to use the new database_id field (#2473) * Adapt frontend to use the new database_id field * Fix host details view story to show the sap instance type * Update frontend to correlate SAP systems and databases properly (#2480) * Update backend and fix tests * Fix frontend links, test and update factory * Update open api schema and fix test * Refactor test and update database_factory * Add database to keep reference * Apply database health to sap system (#2474) * Create and modify new commands and events to add database health * Implement the domain logic * Add new event listener to listen to db health change events * Update the other event handler and middleware functions * Add e2e test case * Handle old database events (#2485) * Add migration to remove old sap system entries * Upcast old events to use the proper database id * Handle old events that do not apply now * Handle legacy rolled up event * Add common upcaster function * Test the events upcasting logic * Use a macro to upcast all the related events * Test legacy events are being ignored * Handle database legacy events in the sap system aggregate * Use a enrichment protocol instead of aggregate code * Make the rollback possible in the migration * Fix Incorrect RDBMS Value Display on SAP Systems Overview (#2492) * The RDBMS value in sap systems overview shows the database sid * Remove unused attached Rdbms data from DatabasesOverview * Add database_sid on sap system registration event * Refactor view and tests * Add test for sap system view render * Address comments about test and view * Refactor sap_system_view test * Fix get systems by cluster selector (#2499) * Fix getSystemsByClusterHosts selector function * Add check in e2e test * Multi tenant discovery (#2500) * wip * Database aggregate support for multiple tenants * Sap system policy support for multi tenant discovery * removed tenant information from DatabaseInstanceRegistered event * Database tenants on DatabaseReadModel * Mix format * wip * Enrich application instance middleware with updated tenant query * health summary service extract tenant from sap system * HealthOverview returns database_sid as deprecated tenant field the new field database_sid is added for clarity and the frontend is changed to reflect the new api field * Fix database view and controller * mix credo * fix mispell * Npm run format * Addressing review feedbacks * Add upcasting regression e2e test to the CI (#2504) * Add upcasting regression e2e test to the CI * Use gh-action for restore and switch to plain sql * Pin ubuntu container to 22.04 * Remove fabriziosestito and add janvhs in the maintainers * Revert wrongly changed path (#2509) * Revert wrongly changed path * Remove newline * Remove sid duplication from view (#2516) * Remove sid duplication from view * Filter by id instead of sid * Fix state mockup and move import * Apply cosmetics * Add photofinish scenario for multi-tenant case (#2521) * Add photofinish scenario for multi-tenant case * Update correct agent_id in filenames * Add photofinish scenario usage in CI * Adjust agent version --------- Co-authored-by: Carmine Di Monaco <carmine.dimonaco@suse.com> Co-authored-by: Eugen Maksymenko <emaksymenko@suse.com> Co-authored-by: Rubén Torrero Marijnissen <rtorreromarijnissen@suse.com>
trento-project · Apr 16, 2024 · d35fe34 · d35fe34
1 parent d7922f5
commit d35fe34
Show file tree

Hide file tree

Showing 166 changed files with 12,873 additions and 4,628 deletions.
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -4,6 +4,7 @@ on:
   push:
     branches: [main]
   pull_request:
+      types: [opened, synchronize, labeled]
   release:
     types: [published]
   workflow_dispatch:
@@ -15,6 +16,8 @@ env:
   OTP_VERSION: 26
   MIX_ENV: test
   NODE_VERSION: "16"
+  MANTAINERS: '["cdimonaco", "dottorblaster", "janvhs", "rtorrero", "nelsonkopliku", "arbulu89","jagabomb","emaksy","jamie-suse"]'
+  RG_TEST_LABEL: regression
 
 jobs:
   elixir-deps:
@@ -436,6 +439,116 @@ jobs:
           name: e2e-screenshots
           path: test/e2e/cypress/screenshots/
 
+  check-regression-label:
+    name: Check if the environment creation criteria are met, store in the job output
+    runs-on: ubuntu-22.04
+    outputs:
+      run_regression_test: ${{ steps.check.outputs.run_regression_test }}
+    steps:
+      - id: check
+        run: echo "run_regression_test=${{ contains(fromJson(env.MANTAINERS), github.event.sender.login) && contains(github.event.pull_request.labels.*.name, env.RG_TEST_LABEL) }}" >> "$GITHUB_OUTPUT"
+
+  regression-test-e2e:
+    name: Regression tests
+    needs: [elixir-deps, npm-deps, npm-e2e-deps, api-bc-check]
+    runs-on: ubuntu-22.04
+    env:
+      MIX_ENV: dev
+    steps:
+      - name: Cancel Previous Runs
+        uses: styfle/cancel-workflow-action@0.12.1
+        with:
+          access_token: ${{ github.token }}
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - name: Setup
+        uses: erlef/setup-beam@v1
+        with:
+          elixir-version: ${{ env.ELIXIR_VERSION }}
+          otp-version: ${{ env.OTP_VERSION }}
+        env:
+          ImageOS: ubuntu20
+      - name: Setup Node
+        uses: actions/setup-node@v4
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+      - name: Retrieve Cached Dependencies
+        uses: actions/cache@v4
+        id: mix-cache
+        with:
+          path: |
+            deps
+            _build/dev
+            priv/plts
+          key: ${{ runner.os }}-${{ env.OTP_VERSION }}-${{ env.ELIXIR_VERSION }}-${{ hashFiles('mix.lock') }}
+      - name: Retrieve NPM Cached Dependencies
+        uses: actions/cache@v4
+        id: npm-cache
+        with:
+          path: |
+            assets/node_modules
+          key: ${{ runner.os }}-${{ env.NODE_VERSION }}-${{ hashFiles('assets/package-lock.json') }}
+      - name: Retrieve E2E NPM Cached Dependencies
+        uses: actions/cache@v4
+        id: npm-e2e-cache
+        with:
+          path: |
+            test/e2e/node_modules
+          key: ${{ runner.os }}-${{ env.NODE_VERSION }}-${{ hashFiles('test/e2e/package-lock.json') }}
+      - name: "Docker compose dependencies"
+        uses: isbang/compose-action@v1.5.1
+        with:
+          compose-file: "./docker-compose.yaml"
+          down-flags: "--volumes"
+      - name: Postgres trento_dev restore
+        uses: tj-actions/pg-restore@v6
+        with:
+          database_url: "postgresql://postgres:postgres@localhost:5433/postgres"
+          backup_file: "test/fixtures/regression/sap_system_split/trento_dev.sql"
+          postgresql_version: "15"
+      - name: Postgres trento_eventstore_dev restore
+        uses: tj-actions/pg-restore@v6
+        with:
+          database_url: "postgresql://postgres:postgres@localhost:5433/postgres"
+          backup_file: "test/fixtures/regression/sap_system_split/trento_eventstore_dev.sql"
+          postgresql_version: "15"
+      - name: Run DB migrations
+        run: mix ecto.migrate
+      - name: Run trento detached
+        run: mix phx.server &
+      - name: Install photofinish
+        uses: jaxxstorm/action-install-gh-release@v1.11.0
+        with:
+          repo: trento-project/photofinish
+          tag: v1.3.0
+          cache: enable
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      - name: Give executable permissions to photofinish
+        run: chmod +x $(whereis photofinish | cut -d" " -f2)
+      - name: Cypress run
+        uses: cypress-io/github-action@v6
+        env:
+          cypress_video: false
+          cypress_db_host: postgres
+          cypress_db_port: 5432
+          cypress_photofinish_binary: $(whereis photofinish | cut -d" " -f2)
+        with:
+          working-directory: test/e2e
+          spec: |
+            cypress/e2e/databases_overview.cy.js
+            cypress/e2e/sap_systems_overview.cy.js
+          wait-on-timeout: 30
+          config: baseUrl=http://localhost:4000
+      - name: Upload cypress test screenshots
+        uses: actions/upload-artifact@v4
+        if: failure()
+        with:
+          name: regression-e2e-screenshots
+          path: test/e2e/cypress/screenshots/
+
   main-branch-deps:
     name: Rebuild main branch dependencies
     runs-on: ubuntu-20.04
@@ -697,6 +810,7 @@ jobs:
           photofinish run healthy-27-node-SAP-cluster -u "http://$TRENTO_DEMO_IP/api/collect" "$TRENTO_API_KEY"
           photofinish run hana-scale-out -u "http://$TRENTO_DEMO_IP/api/collect" "$TRENTO_API_KEY"
           photofinish run hana-diskless-sbd -u "http://$TRENTO_DEMO_IP/api/collect" "$TRENTO_API_KEY"
+          photofinish run multi-tenant -u "http://$TRENTO_DEMO_IP/api/collect" "$TRENTO_API_KEY"
 
   obs-commit-image:
     name: Commit to OBS to generate container image

diff --git a/.github/workflows/pr_env.yaml b/.github/workflows/pr_env.yaml
@@ -159,4 +159,5 @@ jobs:
         run: |
           photofinish run healthy-27-node-SAP-cluster -u "https://$TRENTO_PR_ENV_URL/api/v1/collect"
           photofinish run hana-scale-out -u "https://$TRENTO_PR_ENV_URL/api/v1/collect"
-          photofinish run hana-diskless-sbd -u "https://$TRENTO_PR_ENV_URL/api/v1/collect"
+          photofinish run hana-diskless-sbd -u "https://$TRENTO_PR_ENV_URL/api/v1/collect"
+          photofinish run multi-tenant -u "https://$TRENTO_PR_ENV_URL/api/v1/collect"
diff --git a/.photofinish.toml b/.photofinish.toml
@@ -22,6 +22,10 @@ directories = ["./test/fixtures/scenarios/aws-landscape"]
 
 directories = ["./test/fixtures/scenarios/gcp-landscape"]
 
+[multi-tenant]
+
+directories = ["./test/fixtures/scenarios/multi-tenant"]
+
 [sap-system-detail-GRAY]
 
 files = [

diff --git a/assets/js/lib/test-utils/data/databases.js b/assets/js/lib/test-utils/data/databases.js
@@ -9,7 +9,7 @@ export default [
         https_port: 51014,
         instance_hostname: 'vmhdbdev02',
         instance_number: '10',
-        sap_system_id: 'f534a4ad-cef7-5234-b196-e67082ffb50c',
+        database_id: 'f534a4ad-cef7-5234-b196-e67082ffb50c',
         sid: 'HDD',
         start_priority: '0.3',
         system_replication: 'Secondary',
@@ -24,7 +24,7 @@ export default [
         https_port: 51014,
         instance_hostname: 'vmhdbdev01',
         instance_number: '10',
-        sap_system_id: 'f534a4ad-cef7-5234-b196-e67082ffb50c',
+        database_id: 'f534a4ad-cef7-5234-b196-e67082ffb50c',
         sid: 'HDD',
         start_priority: '0.3',
         system_replication: 'Primary',
@@ -47,7 +47,7 @@ export default [
         https_port: 51014,
         instance_hostname: 'vmhdbprd01',
         instance_number: '10',
-        sap_system_id: '6c9208eb-a5bb-57ef-be5c-6422dedab602',
+        database_id: '6c9208eb-a5bb-57ef-be5c-6422dedab602',
         sid: 'HDP',
         start_priority: '0.3',
         system_replication: 'Primary',
@@ -62,7 +62,7 @@ export default [
         https_port: 51014,
         instance_hostname: 'vmhdbprd02',
         instance_number: '10',
-        sap_system_id: '6c9208eb-a5bb-57ef-be5c-6422dedab602',
+        database_id: '6c9208eb-a5bb-57ef-be5c-6422dedab602',
         sid: 'HDP',
         start_priority: '0.3',
         system_replication: 'Secondary',
@@ -85,7 +85,7 @@ export default [
         https_port: 51014,
         instance_hostname: 'vmhdbqas01',
         instance_number: '10',
-        sap_system_id: 'cd52e571-c897-5bba-b0f9-e155ceca1fff',
+        database_id: 'cd52e571-c897-5bba-b0f9-e155ceca1fff',
         sid: 'HDQ',
         start_priority: '0.3',
         system_replication: 'Primary',
@@ -100,7 +100,7 @@ export default [
         https_port: 51014,
         instance_hostname: 'vmhdbqas02',
         instance_number: '10',
-        sap_system_id: 'cd52e571-c897-5bba-b0f9-e155ceca1fff',
+        database_id: 'cd52e571-c897-5bba-b0f9-e155ceca1fff',
         sid: 'HDQ',
         start_priority: '0.3',
         system_replication: 'Secondary',

diff --git a/assets/js/lib/test-utils/factories/databases.js b/assets/js/lib/test-utils/factories/databases.js
@@ -8,7 +8,7 @@ const features = () =>
   faker.helpers.arrayElements(['HDB', 'HDB_WORKER', 'HDB_STANDBY']);
 
 export const databaseInstanceFactory = Factory.define(() => ({
-  sap_system_id: faker.string.uuid(),
+  database_id: faker.string.uuid(),
   health: healthEnum(),
   host_id: faker.string.uuid(),
   http_port: faker.internet.port(),
@@ -32,7 +32,7 @@ export const databaseFactory = Factory.define(({ params }) => {
     sid,
     health: healthEnum(),
     database_instances: databaseInstanceFactory.buildList(2, {
-      sap_system_id: id,
+      database_id: id,
       sid,
     }),
   };

diff --git a/assets/js/lib/test-utils/factories/sapSystems.js b/assets/js/lib/test-utils/factories/sapSystems.js
@@ -35,14 +35,16 @@ export const sapSystemApplicationInstanceFactory = Factory.define(() => ({
 export const sapSystemFactory = Factory.define(({ params }) => {
   const id = params.id || faker.string.uuid();
   const sid = params.sid || faker.string.alphanumeric(3, { casing: 'upper' });
-
+  const databaseId = params.database_id || faker.string.uuid();
+  const databaseSid =
+    params.database_sid || faker.string.alphanumeric(3, { casing: 'upper' });
   return {
     application_instances: sapSystemApplicationInstanceFactory.buildList(2, {
       sap_system_id: id,
       sid,
     }),
     database_instances: databaseInstanceFactory.buildList(2, {
-      sap_system_id: id,
+      database_id: databaseId,
       sid: faker.string.alphanumeric(3, { casing: 'upper' }),
     }),
     db_host: faker.internet.ip(),
@@ -52,6 +54,8 @@ export const sapSystemFactory = Factory.define(({ params }) => {
     id,
     sid,
     tags: [],
+    database_sid: databaseSid,
     tenant: faker.string.alphanumeric(3, { casing: 'upper' }),
+    database_id: databaseId,
   };
 });
diff --git a/assets/js/pages/ClusterDetails/ClustersList.jsx b/assets/js/pages/ClusterDetails/ClustersList.jsx
@@ -7,6 +7,7 @@ import { getClusterTypeLabel } from '@lib/model/clusters';
 
 import { addTagToCluster, removeTagFromCluster } from '@state/clusters';
 import { getAllSAPInstances } from '@state/selectors/sapSystem';
+import { getInstanceID } from '@state/instances';
 
 import PageHeader from '@common/PageHeader';
 import SapSystemLink from '@common/SapSystemLink';
@@ -79,7 +80,7 @@ function ClustersList() {
               <SapSystemLink
                 key={`${id}-${singleSid}`}
                 systemType={sapSystemData?.type}
-                sapSystemId={sapSystemData?.sap_system_id}
+                sapSystemId={getInstanceID(sapSystemData)}
               >
                 {singleSid}
               </SapSystemLink>,

diff --git a/assets/js/pages/DatabasesOverview/DatabasesOverview.jsx b/assets/js/pages/DatabasesOverview/DatabasesOverview.jsx
@@ -128,11 +128,10 @@ function DatabasesOverview({
     id: database.id,
     health: database.health,
     sid: database.sid,
-    attachedRdbms: database.tenant,
     tenant: database.tenant,
     dbAddress: database.db_host,
     databaseInstances: filter(databaseInstances, {
-      sap_system_id: database.id,
+      database_id: database.id,
     }),
     tags: (database.tags && database.tags.map((tag) => tag.value)) || [],
   }));

diff --git a/assets/js/pages/DatabasesOverview/DatabasesOverview.stories.jsx b/assets/js/pages/DatabasesOverview/DatabasesOverview.stories.jsx
@@ -31,11 +31,11 @@ const databaseWithSR = databaseFactory.build();
 
 const systemReplicationInstances = [
   databaseInstanceFactory.build({
-    sap_system_id: databaseWithSR.id,
+    database_id: databaseWithSR.id,
     system_replication: 'Primary',
   }),
   databaseInstanceFactory.build({
-    sap_system_id: databaseWithSR.id,
+    database_id: databaseWithSR.id,
     system_replication: 'Secondary',
     system_replication_status: 'ACTIVE',
   }),
@@ -44,10 +44,10 @@ const systemReplicationInstances = [
 const databaseWithAbsentInstances = databaseFactory.build();
 const absentInstance = [
   databaseInstanceFactory.build({
-    sap_system_id: databaseWithAbsentInstances.id,
+    database_id: databaseWithAbsentInstances.id,
   }),
   databaseInstanceFactory.build({
-    sap_system_id: databaseWithAbsentInstances.id,
+    database_id: databaseWithAbsentInstances.id,
     absent_at: faker.date.past().toISOString(),
   }),
 ];

diff --git a/assets/js/pages/HealthSummary/HomeHealthSummary.jsx b/assets/js/pages/HealthSummary/HomeHealthSummary.jsx
@@ -86,7 +86,7 @@ const healthSummaryTableConfig = {
       key: 'hostsHealth',
       className: 'text-center w-1/6',
       render: (content, item) => {
-        const linkToHosts = `/hosts?sid=${item.sid}&sid=${item.tenant}`;
+        const linkToHosts = `/hosts?sid=${item.sid}&sid=${item.databaseSid}`;
         return (
           <Link to={linkToHosts}>
             <HealthIcon health={content} centered />
@@ -155,7 +155,7 @@ function HomeHealthSummary({ sapSystemsHealth, loading }) {
     id: summaryDataEntry.id,
     sapsystemHealth: summaryDataEntry.sapsystem_health,
     sid: summaryDataEntry.sid,
-    tenant: summaryDataEntry.tenant,
+    databaseSid: summaryDataEntry.database_sid,
   }));
 
   return loading ? (

diff --git a/assets/js/pages/HealthSummary/HomeHealthSummary.test.jsx b/assets/js/pages/HealthSummary/HomeHealthSummary.test.jsx
@@ -16,7 +16,7 @@ const homeHealthSummaryData = [
     hosts_health: 'critical',
     sapsystem_health: 'passing',
     sid: 'NWD',
-    tenant: 'HDD',
+    database_sid: 'HDD',
   }),
   healthSummaryFactory.build({
     application_cluster_health: 'passing',

diff --git a/assets/js/pages/HostDetailsPage/HostDetails.stories.jsx b/assets/js/pages/HostDetailsPage/HostDetails.stories.jsx
@@ -1,18 +1,27 @@
 import React from 'react';
 import { MemoryRouter } from 'react-router-dom';
 import { faker } from '@faker-js/faker';
+import { APPLICATION_TYPE, DATABASE_TYPE } from '@lib/model/sapSystems';
 
 import {
   clusterFactory,
   hostFactory,
+  databaseInstanceFactory,
   sapSystemApplicationInstanceFactory,
   catalogFactory,
 } from '@lib/test-utils/factories';
 import HostDetails from './HostDetails';
 
 const host = hostFactory.build({ provider: 'azure', agent_version: '2.0.0' });
 const cluster = clusterFactory.build({ id: host.cluster_id });
-const sapInstances = sapSystemApplicationInstanceFactory.buildList(2);
+const sapInstances = sapSystemApplicationInstanceFactory
+  .buildList(1)
+  .map((instance) => ({ ...instance, type: APPLICATION_TYPE }))
+  .concat(
+    databaseInstanceFactory
+      .buildList(1)
+      .map((instance) => ({ ...instance, type: DATABASE_TYPE }))
+  );
 
 function ContainerWrapper({ children }) {
   return (

diff --git a/assets/js/pages/HostDetailsPage/tableConfigs.jsx b/assets/js/pages/HostDetailsPage/tableConfigs.jsx
@@ -1,6 +1,8 @@
 import React from 'react';
 import { Features } from '@pages/SapSystemDetails';
 
+import { getInstanceID } from '@state/instances';
+
 export const subscriptionsTableConfiguration = {
   usePadding: false,
   columns: [
@@ -43,7 +45,11 @@ export const subscriptionsTableConfiguration = {
 export const sapInstancesTableConfiguration = {
   usePadding: false,
   columns: [
-    { title: 'ID', key: 'sap_system_id' },
+    {
+      title: 'ID',
+      key: '',
+      render: (_content, item) => getInstanceID(item),
+    },
     { title: 'SID', key: 'sid' },
     { title: 'Type', key: 'type' },
     {