COV-78 Feat/multi tab explorer

[mfshao]

This PR fulfills https://occ-data.atlassian.net/browse/COV-78

Allow Guppy Data Explorer to render multiple tabs as configured in the configuration file, each tab will be serving from one ES index

Deployed at https://mingfei.planx-pla.net/explorer

Added new config explorerConfig to repleace existing dataExplorerConfig and fileExplorerConfig

explorerConfig is an array which contains multiple configs (one for each explorer tab), e.g.:

"explorerConfig":[
    {                                       ---> 1st tab
      "tabTitle": "Data",         ---> optional tab title, if omitted, will default to use the value of guppyConfig.dataType of this tab
      "charts": {
        "project_id": {
          "chartType": "count",
          "title": "Projects"
        },
        ...
      },
      "filters": {
        "tabs": [
          {
            "title": "Case",
            "fields": [
              "project_id",
              ...
            ]
          }
        ]
      },
      "table": {
        "enabled": true,
        "fields": [
          "project_id",
          ...
        ]
      },
      "guppyConfig": {
        "dataType": "case",
        "nodeCountTitle": "Cases",
        "fieldMapping": [
          {"field": "_aliquots_count", "name": "Aliquots Count"},
          ....
        ],
        "manifestMapping": {
          "resourceIndexType": "file",
          "resourceIdField": "object_id",
          "referenceIdFieldInResourceIndex": "case_id",
          "referenceIdFieldInDataIndex": "case_id"
        },
        "accessibleFieldCheckList": ["project_id"],
        "accessibleValidationField": "project_id"
      },
      "buttons": [
         ....
      ]
    },
    {                                                  ---> 2nd tab
      "charts": {
        "data_type": {
          "chartType": "stackedBar",
          "title": "File Type"
        },
        ...
      },
      "filters": {
        ...
      },
      "table": {
        "enabled": true,
        "fields": [
          "project_id",
          ...
        ]
      },
      "guppyConfig": {
        "dataType": "file",
        "fieldMapping": [
          { "field": "object_id", "name": "GUID" }
        ],
        "nodeCountTitle": "Files",
        "manifestMapping": {
          "resourceIndexType": "case",
          "resourceIdField": "case_id",
          "referenceIdFieldInResourceIndex": "object_id",
          "referenceIdFieldInDataIndex": "object_id"
        },
        "accessibleFieldCheckList": ["project_id"],
        "accessibleValidationField": "project_id",
        "downloadAccessor": "object_id"
      },
      "buttons": [
        {
          "enabled": true,
          "type": "file-manifest",
          "title": "Download Manifest",
          "leftIcon": "datafile",
          "rightIcon": "download",
          "fileName": "file-manifest.json"
        }
      ],
      "dropdowns": {}
    },
    {                                     ---> 3rd tab
      "charts": {
           ...
      },
      "filters": {
        "tabs": [
          {
            "title": "Subject",
            "fields": [
              "project_id",
              "subject_id"
            ]
          }
        ]
      },
      "table": {
        "enabled": true,
        "fields": [
          "project_id",
          "subject_id"
        ]
      },
      "guppyConfig": {
        "dataType": "subject",
        "fieldMapping": [
        ],
        "nodeCountTitle": "Subjects",
        "manifestMapping": {
        },
        "accessibleFieldCheckList": ["project_id"],
        "accessibleValidationField": "project_id"
      },
      "buttons": [
      ],
      "dropdowns": {}
    }
  ]

New Features

• Multi-tab data explorer

Deployment Changes

• New explorerConfig which can take configs for multiple tabs, backward compatiable with current dataExplorerConfig and fileExplorerConfig

Improvements

• Updated format for data explorer config
• Decoupled file explorer from data explorer

[mfshao]

need to merge the login redirect fix first

[mfshao]

marked it as ready again after merging a bug fix, thanks @ZakirG 👍

also added a doc as suggested by @paulineribeyre 😺

[paulineribeyre]

this doc should probably be moved to this repo. it doesn't need to be private and it contains essential information for external people trying to configure windmill
what do you think about moving it

[mfshao]

i could just copy the part that relates to portal configs over? idk if the rests (etl and manifest) should belong to here as well

[paulineribeyre]

this worries me a bit. The first tab won't always be the one where we want the heatmap, and we could want different heatmaps in multiple tabs. could we move the dataAvailabilityToolConfig to the explorerConfig items config in a backwards compatible way? 😬the heatmap will just be another kind of chart

[mfshao]

right! the dataAvailabilityToolConfig seems need some tweak. I'll fix it

[paulineribeyre]

you can also create a ticket if it's a lot of work?

[mfshao]

I did a few tries, seems for now the easiest option is to put the dataAvailabilityToolConfig as a whole inside a single tab's config

It would be cool if we can just put mainField, aggFields etc in the guppyConfig for one tab, and get rid of the dataAvailabilityToolConfig. But right now it won't work because I think if you set aggFields for guppyConfig the guppy wrapper will preforms sub-aggregations, and that will breaks other UI components 🤷‍♂️

I can create a ticket for that future improvement

[paulineribeyre]

right, the DAT is using a separate guppywrapper so i think it's normal that it has a separate guppyconfig? but a ticket is fine :)