Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LPD-42482 Rework data set serialization I (URL and actions) #4669

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

dsanz
Copy link
Collaborator

@dsanz dsanz commented Jan 17, 2025

Reference: https://liferay.atlassian.net/browse/LPD-42482
Parent task: https://liferay.atlassian.net/browse/LPD-42481

Once both java interfaces necessary to describe a data set (https://liferay.atlassian.net/browse/LPD-44141) and FDS API URL builder (https://liferay.atlassian.net/browse/LPD-44140) are in place, we can homogeneize seralization.

This is the first of 2 PRs where we will add some components in charge of serializing different parts of the Data Set. Main features of this proposal are:

  • Common serialization API, that can be spetialized for specific serialization needs depending on the type of thing being serialized
  • Parallelism between custom and system data set serialization mechanisms, via annotated OSGi components.
  • This separation makes serialization logic simpler, focused on a single thing and much more testable. To illustrate, currently, fragment handles serialization of the entire custom data set. Part of this PR borrows code from the fragment
  • Also, splitting serialization logic in this way allows to reuse it in the future when custom serialization needs to serialize a system action for example.
  • Each commit handles serialization for a specific piece in the DS ecosystem, along with tests
  • The Serializer name has been inspired on the existing FDSFilterSerializer and FDSViewSerializer classes, which will be integrated into this new architecture in the second PR

This first PR handles serialization of URLs and actions (item, creation menu and bulk)

Additional notes:

  • I've added support for bulk actions, even if DSM does not support this yet. This means that serialization of bulk actions for custom data sets is just a no-op. I added it for completion but we could remove it as well
  • As LPD-44140 is not yet in upstream master, I've added 8 commits from Brian's origin before the real stuff. They'll dissappear soon, safe to ignore.
  • These changes leave existing data set fragment untouched. Final wiring will take place later on with a FF. Only when we remove that FF, we can get rid of old fragment code in favor of the new serializers

@dsanz dsanz requested review from markocikos and removed request for liferay-frontend January 17, 2025 09:34
@liferay-continuous-integration
Copy link
Collaborator

CI is automatically triggering the following test suites:

  •     ci:test:sf

@dsanz
Copy link
Collaborator Author

dsanz commented Jan 17, 2025

ci:test:relevant

@liferay-continuous-integration
Copy link
Collaborator

❌ ci:test:sf - 0 out of 1 jobs passed in 34 minutes

Click here for more details.

Base Branch:

Branch Name: master
Branch GIT ID: d1f1ba2114b232828a89aa605c18ef368ad23905

Sender Branch:

Branch Name: LPD-37531_LPD-42482
Branch GIT ID: 24755a9a90bec3967ade58af95971b05ca042e1e

0 out of 1jobs PASSED
For more details click here.
format-source-files:
     [java] Loading file:/opt/dev/projects/github/liferay-portal/portal-impl/classes/system.properties
     [java] java.lang.Exception: Found 3 formatting issues:
     [java] 1: Incorrect commit message in SHA 9dd12a4d06994066177a38200ede11d50962589d: Each breaking change should have one, and only one "# breaking", "## What", "## Why" and ## (Optional). Use "----" to split each breaking change.: ./modules/apps/frontend-data-set/frontend-data-set-api/bnd.bnd (SourceCheck:BNDBreakingChangeCommitMessageCheck)
     [java] 2: Incorrect commit message in SHA 9dd12a4d06994066177a38200ede11d50962589d: The commit message contains "# breaking" should end with "\n\n----": ./modules/apps/frontend-data-set/frontend-data-set-api/bnd.bnd (SourceCheck:BNDBreakingChangeCommitMessageCheck)
     [java] 3: ./modules/apps/frontend-data-set/frontend-data-set-api/src/main/java/com/liferay/frontend/data/set/action/FDSItemActionList.java expected:<...pyrightText: (c) 202[4] Liferay, Inc. https...> but was:<...pyrightText: (c) 202[5] Liferay, Inc. https...>
     [java] 
     [java]   at com.liferay.source.formatter.SourceFormatter.format(SourceFormatter.java:472)
     [java]   at com.liferay.source.formatter.SourceFormatter.main(SourceFormatter.java:301)
[stopwatch] [run.batch.test.action: 31:44.952 sec]
     [echo] The following error occurred while executing this line:
     [echo] /opt/dev/projects/github/liferay-portal/portal-impl/build.xml:449: The following error occurred while executing this line:
     [echo] /opt/dev/projects/github/liferay-portal/portal-impl/build.xml:651: Java returned: 1
   [delete] Deleting: /opt/dev/projects/github/liferay-portal/null1438725624.properties

merge-test-results:
[mkdir] Created dir: /opt/dev/projects/github/liferay-portal/test-results
[beanshell] Truncating errors in /opt/dev/projects/github/liferay-portal/portal-impl/test-results/TEST-JenkinsLogAssertTest.xml
[junitreport] Processing /opt/dev/projects/github/liferay-portal/test-results/TESTS-TestSuites.xml to /tmp/null1949916906
[junitreport] Loading stylesheet ile:/opt/java/ant/lib/ant-junit.jar!/org/apache/tools/ant/taskdefs/optional/junit/xsl/junit-frames.xsl
[junitreport] Transform time: 284ms
[junitreport] Deleting: /tmp/null1949916906
[echo] A report with all the test results can be found at test-results/html/index.html.
[mkdir] Created dir: /opt/dev/projects/github/liferay-jenkins-ee/test-results

@liferay-continuous-integration
Copy link
Collaborator

Jenkins Report:jenkins-report.html
Jenkins Suite:sf
Testray Routine:EE Pull Request
Testray Build ID:111032910

@liferay-continuous-integration
Copy link
Collaborator

✔️ ci:test:stable - 24 out of 24 jobs passed

❌ ci:test:relevant - 44 out of 52 jobs passed in 2 hours 7 minutes

Click here for more details.

Base Branch:

Branch Name: master
Branch GIT ID: d1f1ba2114b232828a89aa605c18ef368ad23905

Upstream Comparison:

Branch GIT ID: 381187e44482e95de0a8f15a6844495628ee9081
Jenkins Build URL: EE Development Acceptance (master) - 1122 - 2025-01-16[12:37:12]

ci:test:stable - 24 out of 24 jobs PASSED
24 Successful Jobs:
    ci:test:relevant - 44 out of 52 jobs PASSED

    8 Failed Jobs:

    44 Successful Jobs:
      For more details click here.

      Failures unique to this pull:


      Failures in common with acceptance upstream results at 381187e:
      1. ...
      Test bundle downloads:

      @liferay-continuous-integration
      Copy link
      Collaborator

      @dsanz dsanz force-pushed the LPD-37531_LPD-42482 branch from 24755a9 to 200111d Compare January 17, 2025 12:12
      @dsanz
      Copy link
      Collaborator Author

      dsanz commented Jan 17, 2025

      ci:test:sf

      @dsanz
      Copy link
      Collaborator Author

      dsanz commented Jan 17, 2025

      ci:test:sf

      @dsanz
      Copy link
      Collaborator Author

      dsanz commented Jan 17, 2025

      ci:test:relevant

      dsanz added 6 commits January 20, 2025 09:19
      … data sets, with unit tests
      
      # breaking
      
      ## What modules/apps/frontend-data-set/frontend-data-set-api/src/main/java/com/liferay/frontend/data/set/action/FDSCreationMenu.java
      
      deleted method
      	* public CreationMenu getCreationMenu(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse) throws PortalException;
      
      added method
      	* public CreationMenu getCreationMenu(HttpServletRequest httpServletRequest)
      
      ## Why
      
      Response is not necessary to compute the creation menu. Also, we want implementations to handle their exceptions and return an empty menu if something goes wrong, so that the serialization does not stop. This is a recently added interface, not yet implemented. Customers are not affected at the time of this writing
      
      ----
      …ts, with unit tests. Custom data sets don't yet support definition of bulk actions so we provide a placeholder implementation to facilitate reference handling in the renderer
      @dsanz dsanz force-pushed the LPD-37531_LPD-42482 branch from 812aea6 to 67de3a9 Compare January 20, 2025 08:23
      @dsanz
      Copy link
      Collaborator Author

      dsanz commented Jan 20, 2025

      ci:test:sf

      Copy link
      Collaborator

      @markocikos markocikos left a comment

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      @dsanz please see comments inline

      HttpServletRequest httpServletRequest,
      HttpServletResponse httpServletResponse)
      throws PortalException;
      public CreationMenu getCreationMenu(HttpServletRequest httpServletRequest);
      Copy link
      Collaborator

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      This will conflict with my actions import task. It's no problem, just a bit of extra work for one of us.

      Copy link
      Collaborator Author

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      I'm homogeneizing this because I had to artificially add the httpServletResponse parameter in the PoC, then we never removed it. Request shall be enough to calculate this

      HttpServletRequest httpServletRequest,
      HttpServletResponse httpServletResponse)
      throws PortalException;
      HttpServletRequest httpServletRequest);
      Copy link
      Collaborator

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      Also conflict

      /**
      * @author Daniel Sanz
      */
      public class FDSTypes {
      Copy link
      Collaborator

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      I'm not sure if this will be acceptable. Recently, because of brianchandotcom#157046 (comment) I needed to change FDSTimeZoneBehaviors into FDSTimeZoneBehaviorConstants. This might need to be FDSTypeConstants or FDSConstants.

      Copy link
      Collaborator Author

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      sure, this naming is improvable

      * @author Daniel Sanz
      */
      @Component(
      property = "dataset.type=" + FDSTypes.CUSTOM,
      Copy link
      Collaborator

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      Name of the property does not seem right. Maybe fds.entry.type? To be consistent with constants, FDSTypes should probably be FDSEntryTypeConstants.

      Copy link
      Collaborator Author

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      I feel a bit more comfortable with dataset here because it is detached from the storage mechanism. fdsEntry sounds to me as biased towards java-based stuff.

      Nevertheless, I understand consistency might be more important here, so I'm changing it.

      Comment on lines +33 to +68
      @Override
      public CreationMenu serialize(
      String fdsName, HttpServletRequest httpServletRequest) {

      CreationMenu creationMenu = new CreationMenu();

      for (ObjectEntry objectEntry :
      _customFDSSerializerHelper.getCreationActionObjectEntries(
      fdsName, httpServletRequest)) {

      creationMenu.addPrimaryDropdownItem(
      dropdownItem -> {
      Map<String, Object> properties =
      objectEntry.getProperties();

      dropdownItem.putData(
      "disableHeader",
      (boolean)Validator.isNull(properties.get("title")));

      dropdownItem.putData(
      "permissionKey", properties.get("permissionKey"));
      dropdownItem.putData("size", properties.get("modalSize"));
      dropdownItem.putData("title", properties.get("title"));

      dropdownItem.setHref(properties.get("url"));
      dropdownItem.setIcon(
      String.valueOf(properties.get("icon")));
      dropdownItem.setLabel(
      String.valueOf(properties.get("label")));
      dropdownItem.setTarget(
      String.valueOf(properties.get("target")));
      });
      }

      return creationMenu;
      }
      Copy link
      Collaborator

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      To me, this is not really a serialization. Serialization is transformation of complex data structure into a string, or a similar simple format, some linear series of bytes. Here, we are transforming one type of object (ObjectEntry) into another (CreationMenu). But, something like FDSCreationMenuTransformer also doesn't sound right to me. I would expect this type of functionality to be a method rather than a class, or a part of constructor. Something like

      CreationMenu creationMenu = new CreationMenu(creationActionObjectEntries);
      

      or

      CreationMenu creationMenu = SomeHelperUtil.getCreationMenu(creationActionObjectEntries);
      

      Copy link
      Collaborator Author

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      This is an important thing about naming.

      Original, pre-existing serializers are FDSViewSerializer and FDSFilterSerializer which basically return a JSONArray from the list of java objects representing views and filters.

      Originally, I planned to do a similar thing for the list of actions (creation menu and friends).

      Nevertheless, CreationMenu happens to extend HashMap, one of the valid things we put on the props map that gets serialized for react render.

      In light of the above, it makes no sense to convert creation menu related data into something different than CreationMenu class. This is why the dedicated serializer class extends the generic one parameterizing T with CreationMenu directly:

      public interface FDSCreationMenuSerializer extends FDSSerializer<CreationMenu> {}
      

      From this reasoning, implementations of FDSCreationMenuSerializer for custom and system data sets do directly provide an instance of CreationMenu

      All the above sounds reasonable to me, except naming. Because as you said, this is not exactly serialization but more like "conversion". Even, for the case of SystemFDSCreationMenuSerializer, implementation just fetches the object from the registry so there is even no conversion at all.

      I'm open to other names, as long as we consider all the variants we have.

      }

      private FDSAPIURLBuilder _addNestedFields(
      FDSAPIURLBuilder fdsapiurlBuilder,
      Copy link
      Collaborator

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      This should be fdsAPIURLBuilder

      Copy link
      Collaborator Author

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      let me recheck the SF

      Comment on lines +82 to +83
      System.arraycopy(
      fieldNameList, 0, fieldsName, 0, fieldNameList.length - 1);
      Copy link
      Collaborator

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      Without going too deep into what's going on here, there is probably a nicer way to achieve it, a util or a tweak when making an array instance.

      Copy link
      Collaborator Author

      Choose a reason for hiding this comment

      The reason will be displayed to describe this comment to others. Learn more.

      I took the existing code in our fragment, so I guess we can consider this later on

      @liferay-continuous-integration
      Copy link
      Collaborator

      ❌ ci:test:sf - 0 out of 1 jobs passed in 31 minutes

      Click here for more details.

      Base Branch:

      Branch Name: master
      Branch GIT ID: 4035a2f3f4c922f3108d0bf0a3939c66802ed5ba

      Sender Branch:

      Branch Name: LPD-37531_LPD-42482
      Branch GIT ID: 67de3a917be39cb152d9a1808d2a342a8a70d924

      0 out of 1jobs PASSED
      For more details click here.
      format-source-files:
           [java] Loading file:/opt/dev/projects/github/liferay-portal/portal-impl/classes/system.properties
           [java] java.lang.Exception: Found 7 formatting issues:
           [java] 1: ./modules/third-party/jenkins-core/build.gradle expected:<...", version: "3.0.1"
           [java] [
           [java] ]  compileOnly group: ...> but was:<...", version: "3.0.1"
           [java] []  compileOnly group: ...>
           [java] 2: ./modules/third-party/org-apache-axis/build.gradle expected:<...", version: "1.6.3"
           [java] [
           [java] ]  compileOnly group: ...> but was:<...", version: "1.6.3"
           [java] []  compileOnly group: ...>
           [java] 3: ./modules/third-party/org-outerj-daisy-diff/build.gradle expected:<...ff", version: "1.2"
           [java] [
           [java] ]  compileOnly group: ...> but was:<...ff", version: "1.2"
           [java] []  compileOnly group: ...>
           [java] 4: ./modules/third-party/org-hibernate-core-3.6.10/build.gradle expected:<...sion: "3.2.0.Final"
           [java] [
           [java] ]  compileOnly group: ...> but was:<...sion: "3.2.0.Final"
           [java] []  compileOnly group: ...>
           [java] 5: ./modules/third-party/org-opensearch-java-client/build.gradle expected:<... version: "2.25.53"
           [java] [
           [java] ]  compileOnly group: ...> but was:<... version: "2.25.53"
           [java] []  compileOnly group: ...>
           [java] 6: ./modules/third-party/org-drools-core/build.gradle expected:<...sion: "5.4.0.Final"
           [java] [
           [java] ]  compileOnly group: ...> but was:<...sion: "5.4.0.Final"
           [java] []  compileOnly group: ...>
           [java] 7: ./modules/third-party/org-opensearch-rest-client/build.gradle expected:<..., version: "4.4.15"
           [java] [
           [java] ]  compileOnly group: ...> but was:<..., version: "4.4.15"
           [java] []  compileOnly group: ...>
           [java] 
           [java]   at com.liferay.source.formatter.SourceFormatter.format(SourceFormatter.java:472)
           [java]   at com.liferay.source.formatter.SourceFormatter.main(SourceFormatter.java:301)
      [stopwatch] [run.batch.test.action: 28:20.136 sec]
           [echo] The following error occurred while executing this line:
           [echo] /opt/dev/projects/github/liferay-portal/portal-impl/build.xml:449: The following error occurred while executing this line:
           [echo] /opt/dev/projects/github/liferay-portal/portal-impl/build.xml:651: Java returned: 1
         [delete] Deleting: /opt/dev/projects/github/liferay-portal/null1991035310.properties
      

      merge-test-results:

      @liferay-continuous-integration
      Copy link
      Collaborator

      Jenkins Report:jenkins-report.html
      Jenkins Suite:sf
      Testray Routine:EE Pull Request
      Testray Build ID:113094676

      @liferay-continuous-integration
      Copy link
      Collaborator

      ✔️ ci:test:stable - 24 out of 24 jobs passed

      ✔️ ci:test:relevant - 31 out of 31 jobs passed in 1 hour 18 minutes

      Click here for more details.

      Base Branch:

      Branch Name: master
      Branch GIT ID: 09cc2b52634eb72cdebf54b7aac9afe4ba545311

      Upstream Comparison:

      Branch GIT ID: 09cc2b52634eb72cdebf54b7aac9afe4ba545311
      Jenkins Build URL: EE Development Acceptance (master) - 1128 - 2025-01-19[20:37:59]

      ci:test:stable - 24 out of 24 jobs PASSED
      24 Successful Jobs:
      ci:test:relevant - 31 out of 31 jobs PASSED
      31 Successful Jobs:
      For more details click here.
      Test bundle downloads:

      @liferay-continuous-integration
      Copy link
      Collaborator

      @dsanz
      Copy link
      Collaborator Author

      dsanz commented Jan 20, 2025

      last sf runs returns unrelated failures, probably due to some change in the CI.

      @dsanz
      Copy link
      Collaborator Author

      dsanz commented Jan 20, 2025

      ci:test:sf

      @liferay-continuous-integration
      Copy link
      Collaborator

      ❌ ci:test:sf - 0 out of 1 jobs passed in 37 minutes

      Click here for more details.

      Base Branch:

      Branch Name: master
      Branch GIT ID: 4035a2f3f4c922f3108d0bf0a3939c66802ed5ba

      Sender Branch:

      Branch Name: LPD-37531_LPD-42482
      Branch GIT ID: 67de3a917be39cb152d9a1808d2a342a8a70d924

      0 out of 1jobs PASSED
      For more details click here.
      format-source-files:
           [java] Loading file:/opt/dev/projects/github/liferay-portal/portal-impl/classes/system.properties
           [java] java.lang.Exception: Found 7 formatting issues:
           [java] 1: ./modules/third-party/org-apache-axis/build.gradle expected:<...", version: "1.6.3"
           [java] [
           [java] ]  compileOnly group: ...> but was:<...", version: "1.6.3"
           [java] []  compileOnly group: ...>
           [java] 2: ./modules/third-party/org-hibernate-core-3.6.10/build.gradle expected:<...sion: "3.2.0.Final"
           [java] [
           [java] ]  compileOnly group: ...> but was:<...sion: "3.2.0.Final"
           [java] []  compileOnly group: ...>
           [java] 3: ./modules/third-party/org-drools-core/build.gradle expected:<...sion: "5.4.0.Final"
           [java] [
           [java] ]  compileOnly group: ...> but was:<...sion: "5.4.0.Final"
           [java] []  compileOnly group: ...>
           [java] 4: ./modules/third-party/org-opensearch-java-client/build.gradle expected:<... version: "2.25.53"
           [java] [
           [java] ]  compileOnly group: ...> but was:<... version: "2.25.53"
           [java] []  compileOnly group: ...>
           [java] 5: ./modules/third-party/org-opensearch-rest-client/build.gradle expected:<..., version: "4.4.15"
           [java] [
           [java] ]  compileOnly group: ...> but was:<..., version: "4.4.15"
           [java] []  compileOnly group: ...>
           [java] 6: ./modules/third-party/jenkins-core/build.gradle expected:<...", version: "3.0.1"
           [java] [
           [java] ]  compileOnly group: ...> but was:<...", version: "3.0.1"
           [java] []  compileOnly group: ...>
           [java] 7: ./modules/third-party/org-outerj-daisy-diff/build.gradle expected:<...ff", version: "1.2"
           [java] [
           [java] ]  compileOnly group: ...> but was:<...ff", version: "1.2"
           [java] []  compileOnly group: ...>
           [java] 
           [java]   at com.liferay.source.formatter.SourceFormatter.format(SourceFormatter.java:472)
           [java]   at com.liferay.source.formatter.SourceFormatter.main(SourceFormatter.java:301)
      [stopwatch] [run.batch.test.action: 34:25.524 sec]
           [echo] The following error occurred while executing this line:
           [echo] /opt/dev/projects/github/liferay-portal/portal-impl/build.xml:449: The following error occurred while executing this line:
           [echo] /opt/dev/projects/github/liferay-portal/portal-impl/build.xml:651: Java returned: 1
         [delete] Deleting: /opt/dev/projects/github/liferay-portal/null420668434.properties
      

      merge-test-results:

      @liferay-continuous-integration
      Copy link
      Collaborator

      Jenkins Report:jenkins-report.html
      Jenkins Suite:sf
      Testray Routine:EE Pull Request
      Testray Build ID:113157722

      Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
      Projects
      None yet
      Development

      Successfully merging this pull request may close these issues.

      3 participants