Testing BA-MBSE Persistence Refactor Merge #259

HuiJun · 2024-02-07T08:16:34Z

deconflicted branch of #235

…ers. Introduces a unified persistence layer interface with a federated implementation. Added msosa, view, and oauth modules

…th persistence implementation

…preventing them from getting transaction proxies created

into develop # Conflicts: # crud/src/main/java/org/openmbee/mms/crud/services/DefaultBranchService.java

rdb/src/main/java/org/openmbee/mms/rdb/config/DatabaseDefinitionService.java

            }
-            statement.executeUpdate(connection.nativeSQL("DROP DATABASE " + databaseProjectString(project)));
+            statement.executeUpdate(connection.nativeSQL("DROP DATABASE " + databaseProjectString(projectId)));


dlamoris

have not went over permissions related changes yet
things to check:

some performance optimizations in federated classes where it's getting entire commit json instead of just commit info from db
updating deleted elements, deleted key in json
series of inits across sub/superclasses during element get/post, check logic if element post has deletes as well as updates

dlamoris · 2024-02-07T22:45:03Z

core/src/main/java/org/openmbee/mms/core/dao/NodePersistence.java

+
+    NodeChangeInfo commitChanges(NodeChangeInfo nodeChangeInfo);
+
+    NodeGetInfo findById(String elementId, String refId, String commitId, String id);


first argument is projectId, last argument is elementId

fixed in #260

dlamoris · 2024-02-07T23:54:29Z

cameo/src/main/java/org/openmbee/mms/cameo/services/CameoNodeService.java

@@ -80,38 +82,37 @@ public ElementsResponse read(String projectId, String refId, ElementsRequest req
    }

    @Override
-    public void extraProcessPostedElement(ElementJson element, Node node, NodeChangeInfo info) {
-        node.setNodeType(cameoHelper.getNodeType(element).getValue());


this was replaced by https://github.com/Open-MBEE/exec-mms/blob/feature/persistence_refactor/federatedpersistence/src/main/java/org/openmbee/mms/federatedpersistence/domain/FederatedNodeChangeDomain.java#L140

dlamoris · 2024-02-07T23:59:49Z

cameo/src/main/java/org/openmbee/mms/cameo/services/CameoNodeService.java

-        String commitId = params.getOrDefault("commitId", null);
-        ContextHolder.setContext(projectId, refId);
-        NodeGetInfo info = nodeGetHelper.processGetJson(req.getElements(), commitId, this);
+        String commitId = params.getOrDefault(CameoConstants.COMMITID, null);


commitId is a generic request param, not cameo specific one

fixed in #260

dlamoris · 2024-02-08T00:18:13Z

view/src/main/java/org/openmbee/mms/view/controllers/VeController.java

+        ViewService viewService = genericServiceFactory.getServiceForSchema( ViewService.class , getProjectType(projectId));
+        ElementsResponse res = viewService.createOrUpdate(projectId, refId, req, params, auth.getName());
+        // ToDo : the following is just a workaround till the indexing part is synchronized W
+        try {


fixed in #260 think this was because the added commit check done during read

dlamoris · 2024-02-08T00:29:11Z

rdb/src/main/java/org/openmbee/mms/rdb/repositories/node/NodeDAOImpl.java

+            if (keyHolder.getKey() != null) {
+                node.setId(keyHolder.getKey().longValue());
+            } else {
+                throw new NotFoundException("Key value was null");


transaction?

dlamoris · 2024-02-12T22:53:46Z

...ce/src/main/java/org/openmbee/mms/federatedpersistence/domain/FederatedNodeChangeDomain.java

+            throw new InternalErrorException("Unexpected NodeChangeInfo type in FederatedNodeChangeDomain");
+        }
+        Node n = ((FederatedNodeChangeInfo) info).getExistingNodeMap().get(element.getId());
+        if(n != null) {


fixed in #260

dlamoris · 2024-02-13T01:34:21Z

cameo/src/main/java/org/openmbee/mms/cameo/services/CameoViewService.java

        }
        return res;
 }

    @Override
-    public void extraProcessPostedElement(ElementJson element, Node node, NodeChangeInfo info) {
+    public void extraProcessPostedElement(NodeChangeInfo info, ElementJson element) {


there shouldn't be any _childViews posted starting with ve 5, can indicate incompatibility with ve < 5 going forward

dlamoris · 2024-02-13T01:43:27Z

ldap/src/main/java/org/openmbee/mms/ldap/LdapSecurityConfig.java

+                userPersistence.save(user);
+                //Add groups to user
+
+                Set<GroupJson> addGroups = new HashSet<>();


actually don't need to be syncing local db with ldap groups, this can be taken out , just the granted authority (and token) need to have the groups

will fix in a separate issue/pr

dlamoris · 2024-02-13T04:05:43Z

cameo/src/main/java/org/openmbee/mms/cameo/services/CameoViewService.java

+        List<ElementJson> groups = getNodePersistence().findAllByNodeType(projectId, refId, commitId,
+            CameoNodeType.GROUP.getValue());
+
+        ElementsResponse res = this.read(projectId, refId, buildRequestFromJsons(groups), params);


groups already is the right list of json

fixed in #260

dlamoris · 2024-02-13T04:21:52Z

cameo/src/main/java/org/openmbee/mms/cameo/services/CameoViewService.java

@@ -75,42 +82,47 @@ public void addChildViews(ElementsResponse res, Map<String, String> params) {
    }

    public ElementsResponse getGroups(String projectId, String refId, Map<String, String> params) {


add override

HuiJun · 2024-02-13T19:19:54Z

crud/src/main/java/org/openmbee/mms/crud/services/DefaultCommitService.java

        }
-        res.getCommits().addAll(commitIndex.elementHistory(elementId, commitIds));
+        res.getCommits().addAll(commitPersistence.elementHistory(projectId, elementId, commitIds));


…persistence_refactor

sonarqubecloud · 2024-02-13T21:51:54Z

Quality Gate failed

Failed conditions
1 Security Hotspot
0.0% Coverage on New Code (required ≥ 80%)
7.8% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

…vice doesn't need to pass in commitIds (this is so it's not getting json just to get ids as getting entire commit json is expensive for federated persistence)

…ndling ve structural changes and has been taken out of ve (5+)

…istence_refactor_fixes merging from develop, deconflict branch from commit

…istence_refactor_fixes

dlamoris · 2024-03-18T21:28:58Z

core/src/main/java/org/openmbee/mms/core/services/DefaultPermissionService.java

-                    Branch b = new Branch(getProject(projectId), branchId, false);
-                    branchRepo.save(b);
-                    return b;
+                    RefJson b = new RefJson();


before was only creating global branch, check what side effects using persistence.save have
theoretically it should never create here

fixed in #260

dlamoris · 2024-03-19T02:19:49Z

...src/main/java/org/openmbee/mms/permissions/delegation/DefaultPermissionsDelegateFactory.java

@@ -1,36 +0,0 @@
-package org.openmbee.mms.permissions.delegation;


moved into federated module

dlamoris · 2024-03-26T00:29:08Z

...in/java/org/openmbee/mms/federatedpersistence/permissions/DefaultOrgPermissionsDelegate.java

@@ -71,6 +74,17 @@ public boolean hasPermission(String user, Set<String> groups, String privilege)
        return false;
    }

+    @Override
+    public boolean hasGroupPermissions(String group, String privilege) {


this is actually not used anywhere, just here to fulfill an interface

dlamoris · 2024-03-26T00:32:00Z

...va/org/openmbee/mms/federatedpersistence/permissions/AbstractDefaultPermissionsDelegate.java

@@ -60,7 +60,7 @@ protected Pair<Group, Role> getGroupAndRole(PermissionUpdateRequest.Permission p
        Optional<Group> group = getGroupRepo().findByName(p.getName());
        Optional<Role> role = getRoleRepo().findByName(p.getRole());
        if (!role.isPresent()) {
-            return Pair.of(group.orElse(null), null);
+            return null;


line 67 should keep the returned instance

fixed in #260

fix argument in DefaultPermissionService.setProjectInherit

dlamoris · 2024-03-30T23:56:05Z

...openmbee/mms/federatedpersistence/permissions/FederatedPermissionUpdatesResponseBuilder.java

+    private FederatedPermissionsUpdateResponseBuilder groupsBuilder = new FederatedPermissionsUpdateResponseBuilder();
+
+    @Override
+    public FederatedPermissionUpdatesResponseBuilder setInherit(Boolean inherit) {


setInherit, setPublic, insert functions are never called on this class, only on parent class, can be removed

fixed in #260

dlamoris · 2024-03-30T23:57:33Z

...openmbee/mms/federatedpersistence/permissions/FederatedPermissionsUpdateResponseBuilder.java

@@ -0,0 +1,99 @@
+package org.openmbee.mms.federatedpersistence.permissions;


these were functions from PermissionsUpdateResponseBuilder that uses the DB classes

dlamoris · 2024-03-30T23:58:21Z

core/src/main/java/org/openmbee/mms/core/builders/PermissionUpdateResponseBuilder.java

@@ -50,93 +44,6 @@ public PermissionUpdateResponseBuilder insert(PermissionUpdateResponse updateUse
        return this;
    }

-    public void insertPermissionUpdates_OrgUserPerm(PermissionUpdateResponse.Action action, Collection<OrgUserPerm> perms) {


these are moved to federated module

dlamoris · 2024-03-31T00:00:54Z

...ava/org/openmbee/mms/federatedpersistence/permissions/DefaultProjectPermissionsDelegate.java

@@ -88,6 +85,17 @@ public boolean hasPermission(String user, Set<String> groups, String privilege)
        return false;
    }

+    @Override
+    public boolean hasGroupPermissions(String group, String privilege) {


never called, just implements interface

dlamoris · 2024-03-31T00:01:39Z

...ava/org/openmbee/mms/federatedpersistence/permissions/DefaultProjectPermissionsDelegate.java

@@ -120,15 +128,28 @@ public boolean setInherit(boolean isInherit) {
        return false;
    }

+    @Override
+    public PermissionResponse getInherit() {


never called

… info instead of reinit data

… from passed in nodeChangeInfo

Fix/persistence refactor fixes

sonarqubecloud · 2024-04-09T17:42:19Z

Quality Gate failed

Failed conditions
1 Security Hotspot
50.6% Coverage on New Code (required ≥ 80%)
7.2% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

BA-MBSE and others added 19 commits March 17, 2023 15:11

Decouples Federated Persistence layer from controller and service lay…

cd851b0

…ers. Introduces a unified persistence layer interface with a federated implementation. Added msosa, view, and oauth modules

Remove redundant commit save

9e17c7b

CodeQL updates

240359a

More CodeQL changes

d1cd63d

Added timeout to database delete statement

1cc6a06

Removing overarching transactional annotations

9ddd679

Removed transactional annotations from DefaultPermissionService

bc41d28

Removed @transactional annotation from modules that are not tasked wi…

ff3d4fd

…th persistence implementation

Fixes for failing crud tests

b6a0ede

Increased request delay for crud tests

9808317

Working on fixing permissions test failures

a9595f6

increased delay in permissions test

23848a0

Fixing problem with bean creation for permissions delegates that was …

b07f73b

…preventing them from getting transaction proxies created

Fixed artifacts behavior

307a8b9

extending artifacts test delay to 1000 ms

8eb0abb

Fixing elastic test failures

a95b385

Fixing groups test failures

066a212

Merge branch 'develop' into ba_persistence_refactor

87d24ad

Merge branch 'ba_persistence_refactor' of https://github.com/BA-MBSE/mms

2c0daeb

into develop # Conflicts: # crud/src/main/java/org/openmbee/mms/crud/services/DefaultBranchService.java

github-advanced-security bot found potential problems Feb 7, 2024

View reviewed changes

dlamoris requested changes Feb 13, 2024

View reviewed changes

HuiJun commented Feb 13, 2024

View reviewed changes

Merge commit '4e4e8586b744bc371db774f2b7115647003d2965' into feature/…

f5ef049

…persistence_refactor

Doris Lam added 3 commits February 13, 2024 13:53

fix NodePersistence.findById interface argument names

fe50baf

fix FederatedNodeChangeDomain superclass constructor call

ceb011f

change CommitPersistence.elementHistory interface so DefaultCommitSer…

d042a36

…vice doesn't need to pass in commitIds (this is so it's not getting json just to get ids as getting entire commit json is expensive for federated persistence)

dlamoris mentioned this pull request Feb 13, 2024

Fix/persistence refactor fixes #260

Merged

Doris Lam added 2 commits February 13, 2024 14:44

remove redundant groups read from CameoViewService

1190f31

remove posted _childViews handling from CameoViewService, this was ha…

8fab816

…ndling ve structural changes and has been taken out of ve (5+)

Doris Lam added 2 commits February 24, 2024 13:14

update project permissions after changing orgs if needed

c6aecc5

continue with element delete even if json not found

1af38d9

dlamoris mentioned this pull request Feb 28, 2024

Reconcile OpenMBEE and Boeing #258

Closed

Doris Lam added 3 commits March 10, 2024 16:07

Merge commit 'f7ac4a8af45c3f3347eec2a2b4d0e9a5fe12f144' into fix/pers…

814b718

…istence_refactor_fixes merging from develop, deconflict branch from commit

add more tests for branching behavior

e8fc607

Merge commit '83ec1f6db8a540e1c8bade294c49fa62a62cec25' into fix/pers…

b58f138

…istence_refactor_fixes

dlamoris reviewed Mar 18, 2024

View reviewed changes

dlamoris reviewed Mar 19, 2024

View reviewed changes

Doris Lam added 2 commits March 18, 2024 20:06

use returned instance from spring data repository save

13318b9

fix branch not found behavior when init branch perms

e51fa08

dlamoris reviewed Mar 26, 2024

View reviewed changes

remove unused things in FederatedPermissionUpdatesResponseBuilder

a55b5ca

fix argument in DefaultPermissionService.setProjectInherit

dlamoris reviewed Mar 30, 2024

View reviewed changes

dlamoris reviewed Mar 31, 2024

View reviewed changes

dlamoris mentioned this pull request Mar 31, 2024

Persistence refactor #235

Closed

Doris Lam and others added 8 commits March 30, 2024 20:04

remove unneeded element delete processing

f178665

users list is available to authenticated users

7162f86

tweak commit update in case twc module is used

844e9eb

commit fail cleanup

7858dfb

fix commit delete

26300d3

in FederatedNodeChangeDomain.primeNodeChangeInfo, add to any existing…

404ef00

… info instead of reinit data

only further process transacted elements but keep other existing info…

27f056a

… from passed in nodeChangeInfo

Merge pull request #260 from Open-MBEE/fix/persistence_refactor_fixes

fce5e38

Fix/persistence refactor fixes

dlamoris merged commit 7de6dd0 into develop Apr 9, 2024
3 of 5 checks passed

dlamoris deleted the feature/persistence_refactor branch August 27, 2024 16:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Testing BA-MBSE Persistence Refactor Merge #259

Testing BA-MBSE Persistence Refactor Merge #259

HuiJun commented Feb 7, 2024 •

edited by dlamoris

Loading

dlamoris left a comment

dlamoris Feb 7, 2024

dlamoris Feb 13, 2024

dlamoris Feb 7, 2024

dlamoris Feb 13, 2024

dlamoris Feb 7, 2024

dlamoris Feb 21, 2024

dlamoris Feb 8, 2024

dlamoris Feb 24, 2024

dlamoris Feb 8, 2024

dlamoris Feb 12, 2024

dlamoris Feb 14, 2024

dlamoris Feb 13, 2024

dlamoris Feb 13, 2024

dlamoris Mar 21, 2024

dlamoris Feb 13, 2024

dlamoris Feb 22, 2024

dlamoris Feb 13, 2024

HuiJun Feb 13, 2024

sonarqubecloud bot commented Feb 13, 2024

dlamoris Mar 18, 2024 •

edited

Loading

dlamoris Mar 25, 2024

dlamoris Mar 19, 2024

dlamoris Mar 26, 2024

dlamoris Mar 26, 2024

dlamoris Mar 26, 2024 •

edited

Loading

dlamoris Mar 30, 2024

dlamoris Mar 30, 2024

dlamoris Mar 30, 2024

dlamoris Mar 30, 2024

dlamoris Mar 31, 2024

dlamoris Mar 31, 2024

sonarqubecloud bot commented Apr 9, 2024


		NodeChangeInfo commitChanges(NodeChangeInfo nodeChangeInfo);

		NodeGetInfo findById(String elementId, String refId, String commitId, String id);

		@@ -75,42 +82,47 @@ public void addChildViews(ElementsResponse res, Map<String, String> params) {
		}

		public ElementsResponse getGroups(String projectId, String refId, Map<String, String> params) {

		@@ -1,36 +0,0 @@
		package org.openmbee.mms.permissions.delegation;

		@@ -0,0 +1,99 @@
		package org.openmbee.mms.federatedpersistence.permissions;

Testing BA-MBSE Persistence Refactor Merge #259

Testing BA-MBSE Persistence Refactor Merge #259

Conversation

HuiJun commented Feb 7, 2024 • edited by dlamoris Loading

dlamoris left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Feb 13, 2024

Quality Gate failed

dlamoris Mar 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dlamoris Mar 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Apr 9, 2024

Quality Gate failed

HuiJun commented Feb 7, 2024 •

edited by dlamoris

Loading

dlamoris Mar 18, 2024 •

edited

Loading

dlamoris Mar 26, 2024 •

edited

Loading