Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(TaskProcessingApi): Add endpoint for getting the next task #45391

Merged
merged 2 commits into from
Jul 1, 2024

Conversation

provokateurin
Copy link
Member

Summary

The ExApps need a way to get the next task.
@marcelklehr I think there is a bug as the tasks are not marked as running when the getNextScheduledTask method of the service is called (or maybe it was also intentional and the endpoint should do it manually).

Checklist

@provokateurin provokateurin added the 2. developing Work in progress label May 17, 2024
@provokateurin provokateurin added this to the Nextcloud 30 milestone May 17, 2024
@marcelklehr
Copy link
Member

Ah, I missed this, yes.

@marcelklehr I think there is a bug as the tasks are not marked as running when the getNextScheduledTask method of the service is called (or maybe it was also intentional and the endpoint should do it manually).

Yeah, let's do this manually directly in the endpoint

@marcelklehr marcelklehr added the pending documentation This pull request needs an associated documentation update label May 18, 2024
@provokateurin provokateurin force-pushed the feat/taskprocessingapi/next-task-endpoint branch from 5800675 to 6d86511 Compare June 4, 2024 10:58
@provokateurin provokateurin added 3. to review Waiting for reviews and removed 2. developing Work in progress labels Jun 4, 2024
@provokateurin provokateurin marked this pull request as ready for review June 4, 2024 11:15
@bigcat88
Copy link
Member

bigcat88 commented Jun 5, 2024

We need endpoint for getting nextTask by specifying multiple task_types.
One provider can process multiple task types and we do not want to spam NC with one nextTask request for each task_type.

Or just to change string $taskTypeId to array of strings.

Copy link
Member

@julien-nc julien-nc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I agree with @marcelklehr that it makes more sense to change the task status to "running" in the controller method.
    The Manager::getNextScheduledTask method might be used in some context where we don't want to change to task status.

  • Also, how about having a setTaskRunning parameter (with true as default value) in TaskProcessingApiController::getNextScheduledTask ? Just in case one wants to get the next scheduled task without actually processing it.

  • About requesting multiple task types in getNextScheduledTask, much needed indeed. @marcelklehr Would you go with another endpoint or change the current one?

@provokateurin
Copy link
Member Author

I don't see a reason why you would ever need to know the next task without also running it at the same time, but I can change it if you really want 🤷‍♀️

@marcelklehr
Copy link
Member

I don't see a reason why you would ever need to know the next task without also running it at the same time

The synchronous background job will re-schedule itself based on the fact whether there is another job waiting to be executed.

@marcelklehr
Copy link
Member

Would you go with another endpoint or change the current one?

We can add this to the current endpoint.

@provokateurin provokateurin requested a review from julien-nc June 7, 2024 06:42
@provokateurin provokateurin force-pushed the feat/taskprocessingapi/next-task-endpoint branch from 711a3fe to 7e6e94a Compare June 7, 2024 06:55
@provokateurin provokateurin changed the base branch from master to feat/securitymiddleware/bypass-app_api June 7, 2024 06:55
@provokateurin provokateurin force-pushed the feat/taskprocessingapi/next-task-endpoint branch from 7e6e94a to 8063ff5 Compare June 7, 2024 09:35
@provokateurin provokateurin changed the base branch from feat/securitymiddleware/bypass-app_api to master June 7, 2024 09:35
@bigcat88
Copy link
Member

ExApp needs to call these APIs to process tasks:

getNextScheduledTask
getFileContents
setProgress
setResult

more precisely, there should be two versions of getFileContents - one for the UI with userId and other one for ExApp with [ExAppRequired] without checking for userId

cause only having those 4 endpoint ExApp can have no "system" flag to process tasks from what I see

Copy link
Member

@nickvergessen nickvergessen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The security middleware looks good now

@bigcat88 bigcat88 requested a review from marcelklehr June 28, 2024 14:31
@bigcat88
Copy link
Member

bigcat88 commented Jun 28, 2024

I didn't change the logic inside getNextScheduledTask - but it doesn't seem right to me for these reasons:

  1. $task->setStatus(Task::STATUS_RUNNING); won't write anything to the database, right?
  2. Reading and writing to the database must be atomic within “getNextScheduledTask” (so that two requests for processing a task in parallel do not receive the same task)
  3. I didn’t understand the logic at all why $providerIds is there as input and what it should do, that’s why I didn’t change the entry to atomic inside “getNextScheduledTask”

Signed-off-by: provokateurin <kate@provokateurin.de>
@provokateurin provokateurin force-pushed the feat/taskprocessingapi/next-task-endpoint branch from b1cc4fb to ed62b4c Compare July 1, 2024 12:43
@nickvergessen
Copy link
Member

btw Psalm failing is related:

Error: lib/private/TaskProcessing/Db/TaskMapper.php:25:7: InvalidTemplateParam: Extended template param T expects type OCP\AppFramework\Db\Entity, type OCP\TaskProcessing\Task given (see https://psalm.dev/183)
Error: lib/private/TaskProcessing/Db/TaskMapper.php:42:15: UndefinedPropertyFetch: Static property OCP\TaskProcessing\Task::$columns is not defined (see https://psalm.dev/039)
Error: lib/private/TaskProcessing/Db/TaskMapper.php:57:15: UndefinedPropertyFetch: Static property OCP\TaskProcessing\Task::$columns is not defined (see https://psalm.dev/039)
Error: lib/private/TaskProcessing/Db/TaskMapper.php:89:15: UndefinedPropertyFetch: Static property OCP\TaskProcessing\Task::$columns is not defined (see https://psalm.dev/039)
Error: lib/private/TaskProcessing/Db/TaskMapper.php:109:15: UndefinedPropertyFetch: Static property OCP\TaskProcessing\Task::$columns is not defined (see https://psalm.dev/039)
Error: lib/private/TaskProcessing/Db/TaskMapper.php:130:15: UndefinedPropertyFetch: Static property OCP\TaskProcessing\Task::$columns is not defined (see https://psalm.dev/039)
Error: lib/private/TaskProcessing/Db/TaskMapper.php:152:32: ImplementedParamTypeMismatch: Argument 1 of OC\TaskProcessing\Db\TaskMapper::update has wrong type 'OCP\AppFramework\Db\Entity', expecting 'OCP\TaskProcessing\Task' as defined by OCP\AppFramework\Db\QBMapper::update (see https://psalm.dev/199)
Error: lib/private/TaskProcessing/Manager.php:584:29: InvalidArgument: Argument 1 of OC\TaskProcessing\Db\TaskMapper::insert expects OCP\TaskProcessing\Task, but OC\TaskProcessing\Db\Task provided (see https://psalm.dev/004)
Error: lib/private/TaskProcessing/Manager.php:595:29: InvalidArgument: Argument 1 of OC\TaskProcessing\Db\TaskMapper::delete expects OCP\TaskProcessing\Task, but OC\TaskProcessing\Db\Task provided (see https://psalm.dev/004)
Error: lib/private/TaskProcessing/Manager.php:601:24: UndefinedMethod: Method OCP\TaskProcessing\Task::toPublicTask does not exist (see https://psalm.dev/022)
Error: lib/private/TaskProcessing/Manager.php:619:30: InvalidArgument: Argument 1 of OC\TaskProcessing\Db\TaskMapper::update expects OCP\TaskProcessing\Task, but OC\TaskProcessing\Db\Task provided (see https://psalm.dev/004)
Error: lib/private/TaskProcessing/Manager.php:635:30: InvalidArgument: Argument 1 of OC\TaskProcessing\Db\TaskMapper::update expects OCP\TaskProcessing\Task, but OC\TaskProcessing\Db\Task provided (see https://psalm.dev/004)
Error: lib/private/TaskProcessing/Manager.php:684:30: InvalidArgument: Argument 1 of OC\TaskProcessing\Db\TaskMapper::update expects OCP\TaskProcessing\Task, but OC\TaskProcessing\Db\Task provided (see https://psalm.dev/004)
Error: lib/private/TaskProcessing/Manager.php:699:24: UndefinedMethod: Method OCP\TaskProcessing\Task::toPublicTask does not exist (see https://psalm.dev/022)
Error: lib/private/TaskProcessing/Manager.php:771:24: UndefinedMethod: Method OCP\TaskProcessing\Task::toPublicTask does not exist (see https://psalm.dev/022)
Error: lib/private/TaskProcessing/Manager.php:784:60: UndefinedMethod: Method OCP\TaskProcessing\Task::toPublicTask does not exist (see https://psalm.dev/022)
Error: lib/private/TaskProcessing/Manager.php:795:60: UndefinedMethod: Method OCP\TaskProcessing\Task::toPublicTask does not exist (see https://psalm.dev/022)
Error: lib/public/AppFramework/Db/QBMapper.php:153:18: MismatchingDocblockParamType: Parameter $entity has wrong type 'OCP\TaskProcessing\Task', should be 'OCP\AppFramework\Db\Entity'

@provokateurin
Copy link
Member Author

Yeah I saw it, will fix all of it

Signed-off-by: provokateurin <kate@provokateurin.de>
@provokateurin provokateurin force-pushed the feat/taskprocessingapi/next-task-endpoint branch from ed62b4c to f5ff813 Compare July 1, 2024 15:11
@provokateurin provokateurin enabled auto-merge July 1, 2024 17:09
@provokateurin provokateurin merged commit 547dab4 into master Jul 1, 2024
165 checks passed
@provokateurin provokateurin deleted the feat/taskprocessingapi/next-task-endpoint branch July 1, 2024 17:10
@provokateurin provokateurin removed the pending documentation This pull request needs an associated documentation update label Jul 18, 2024
@blizzz blizzz mentioned this pull request Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3. to review Waiting for reviews
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants