The run jobs page is slow to respond

[iaindillingham]

The run jobs page is slow to respond. This may be where a study has many actions. For example, long-covid-symptoms/long-covid-symptoms has 200 actions.

Thank you for reporting this, @kevwing.

[ghickman]

todo:

• get stats from honeycomb about these pages across various workspaces
• check if this is related to the extra checks added for codelist rot
• check honeycomb for the requests made in the background (honeycomb has telemetry for the requests library)

[madwort]

https://ui.honeycomb.io/bennett-institute-for-applied-data-science/environments/production/datasets/job-server/result/nwdBZ8xhqQt

[ghickman]

Locally it takes ~2s for the run jobs page of long-covid-symptoms (the linked workspace) to load with 223 actions. Nearly all of this time is spent in dispatch. The 3 slowest parts of the code, in the order they execute, are:

1. 0.422s: get project.yaml from GitHub
2. 0.273s: load (and validate) project.yaml into a Pipeline object
3. 0.929s: check codelist status

On a page with a much smaller number of actions, such as test-age-distribution which has 3 actions, it takes ~1s to load locally, with those calls from above taking:

1. 0.312s: get project.yaml from GitHub
2. 0.012s: load (and validate) project.yaml into a Pipeline object
3. 0.581s: check codelist status

Not at all a scientific approach (I saw a 300ms variation in opencodelist response time while getting these numbers!), but the smaller number of actions clearly helps with the load_pipeline call.

I think we have 3 options to tackle this:

1. switch the view to async and wrap up the GitHub and OpenCodelist API calls behind async functions
2. use a pool of threads/processes/etc to run the API calls
3. nothing, and accept that this view needs to be rebuilt as a wizard, likely with a react, or similar, front end

[tomodwyer]

No current plans to optimise. The job submission flow will eventually be redesigned to fix some of these issues.