subscription interface

[oliver-sanders]

Context: The result of a few conversations about isolating views from GraphQL and filtering which hadn't been written to an issue.

The current subscription interface:

• The UI has multiple "views" each of which requires a different slice of the same data.
• The data requirements of a view might change as a user interacts with it.
• We currently merge the subscriptions to avoid duplicate request / storage / handling of data.
• We have one subscription for all workflows (dashboard, gscan).
• And one subscription per workflow.

Possible simplification:

• Rather than having views provide a GraphQL query in full have them provide a list of the fields they require for each object (workflow, task, etc).
• Have the workflow service construct / merge this into a subscription.
• Later on we can add extra fields for filtering.

For example a simple tree view might provide the following subscription:

subscription = {
  'workflow': [
    'id',
    'status'
  ],
  'task': [
    'id',
    'status'
  ],
  'job': [
    'id',
    'status'
  ],
  'task-filter': [
    'failed',
    'submit-failed'
  ]
}

Which the workflow service could construct into:

workflow {
  id
  status
  task(states: ["failed", "submit-failed"]) {
    id
    status
  }
  job {
    id
    status
  }
}

If a second subscription came along with a blank task-filter we could handle that intelligently in the UI code:

-  task(states: ["failed", "submit-failed"]) {
+  task {

[oliver-sanders]

[Update 2023] We've had a while to play around with queries now. The data requirements of views are generally quite simple, but the subscriptions are quite lengthy as a result of boilerplate.

The boiler plate results from the requirements of the store itself which imposes a set of rules:

• The id must be requested for every element requested.
• You must request pruned deltas for every type you request.
• You must request the fields you want using fragments which must follow a standard naming pattern (e.g. WorkflowData, TaskProxyData, etc) to avoid requesting duplicate data.
• If you want the family-tree you must request the following fields on FamilyProxies:
  • __typename
  • ancestors { name }
  • childTasks { id }
• You must request workflow { reloaded } if you request any of these fields to ensure the data store is correctly flushed on reload:
  • Jobs
  • Tasks
  • Families
  • TaskProxies
  • FamilyProxies
  • Nodes
  • Edges

Which is quite a lot of rules which makes view writing harder than it has to be and easier to mess up.

Here's the query for the SimpleTree which serves as developer documentation, most of which is boilerplate:

cylc-ui/src/views/SimpleTree.vue

Lines 118 to 175 in 2d81ad2

	const QUERY = gql`
	subscription SimpleTreeSubscription ($workflowId: ID) {
	deltas(workflows: [$workflowId]) {
	...Deltas
	}
	}
	fragment Deltas on Deltas {
	added {
	...AddedDelta
	}
	updated (stripNull: true) {
	...UpdatedDelta
	}
	pruned {
	...PrunedDelta
	}
	}
	fragment AddedDelta on Added {
	taskProxies {
	...TaskProxyData
	}
	jobs {
	...JobData
	}
	}
	fragment UpdatedDelta on Updated {
	taskProxies {
	...TaskProxyData
	}
	jobs {
	...JobData
	}
	}
	# We must list all of the types we request data for here to enable automatic
	# housekeeping.
	fragment PrunedDelta on Pruned {
	workflow
	taskProxies
	jobs
	}
	# We must always request the "id" for ALL types.
	# The only field this view requires beyond that is the status.
	fragment TaskProxyData on TaskProxy {
	id
	state
	}
	# Same for jobs.
	fragment JobData on Job {
	id
	state
	}
	`

As a result of the complexity of these rules as well as other factors incl maintainability and duplicate-fragment warnings, I think that templating requested fields into the query rather than attempting to merge the pre-formed queries is looking like the way to go (i.e. what this issue is about).

Note, filtering at the query level (as referenced in the OP) is a more advanced feature which cylc-ui is yet to develop.