Completion cron should fetch data for all learners in one request

[garemoko]

Currently the completion cron queries the LRS once for each learner/module instance combination.

A better approach would be to make one request and store the outcome as a cache in Moodle's database so that Moodle does not re-query the LRS if it has recently fetched statements.

'Recently' needs some consideration. If it's too long a time period we risk completions being missed because the learner completed too recently. If it's too short, we risk the cron task not completing in time.

Some options:

• It may be that we can use the cache only when running the cron by storing a flag to indicate that the cron task is currently running and therefore the cache should be used.
• Alternatively the cache duration could be a configuration setting so that performance conscious sites can set a higher values and sites that care about completion being displayed to the learner in 'real time' can set something lower. If this config setting is added, it should also feed into the regularity that completion is checked by the launch page because there's no point checking completion more regularly than the cache is updated.

[mediabounds]

I've recently set up the cron job on our Moodle server and am noticing that it takes quite a while to check for completion (70 xAPI launch link activities with ~70 students). Cron job takes over 30 minutes.

One option could be to set the since query parameter and download all statements matching completed or voided since the last time it checked then iterate through the statements to update the student records.

[garemoko]

@mediabounds Yes, that sounds like a good improvement, although you would have to be careful to make sure it didn't mark people previously flagged as complete as un-complete. Also, I imagine that in most cases it would have no impact on performance if the number of qualifying statements per student is less than the limit of the LRS (sometimes set to 100), which seems likely in most cases.

Please can you create a separate issue for this?

[garemoko]

@mediabounds most of the reason for the slow time is that for 70 students and 70 modules, 490 requests will be sent to the LRS. Reducing that to 70 requests (one per module as described in this ticket) would probably be around 30 or more times faster.