Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose cronWorkflowWorkers as config item. #11454

Closed
sarabala1979 opened this issue Jul 26, 2023 · 4 comments
Closed

Expose cronWorkflowWorkers as config item. #11454

sarabala1979 opened this issue Jul 26, 2023 · 4 comments
Labels
area/controller Controller issues, panics area/cron-workflows type/feature Feature request

Comments

@sarabala1979
Copy link
Member

Summary

Currently `cronWorkflowWorkers' was hardcoded. In some scenarios, if the user is using cron workflow heavily, they need more workers to process the cronworkflow.

What change needs making?

Use Cases

When would you use this?


Message from the maintainers:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

@sarabala1979 sarabala1979 added the type/feature Feature request label Jul 26, 2023
@tooptoop4
Copy link
Contributor

does it mean if I have 9 different cronworkflows set to run hourly that one will be missed?

@agilgur5
Copy link

agilgur5 commented Jul 29, 2023

Added in #11457 (albeit as an argument, not a config option)

@agilgur5
Copy link

agilgur5 commented Jul 29, 2023

does it mean if I have 9 different cronworkflows set to run hourly that one will be missed?

I believe it's more that there are 8 (lightweight) threads running by default. So if you have 9 cron schedules trigger simultaneously, they all get added to the queue and up to 8 can be processed in parallel (also depends on number of cores and other things running on the CPU etc).
A CronWorkflow primarily creates Workflows, so I'd imagine all 8 could be processed in less than a second, and then the 9th gets processed by the first thread that opens up.

@sarabala1979 feel free to correct me as you obviously know this code better than me!

@agilgur5
Copy link

(^that is also given a single controller instance without any sharding)

@agilgur5 agilgur5 closed this as completed Aug 8, 2023
@agilgur5 agilgur5 added area/controller Controller issues, panics area/cron-workflows labels Aug 8, 2023
@argoproj argoproj locked as resolved and limited conversation to collaborators Apr 30, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/controller Controller issues, panics area/cron-workflows type/feature Feature request
Projects
None yet
Development

No branches or pull requests

3 participants