Quartz scheduler does not support ConcurrentExecution.SKIP correctly

[davidcurrie]

Describe the bug

When using the Quartz scheduler with multiple nodes, ConcurrentExecution.SKIP as implemented under #10203 only skips executions that happen to be scheduled to a node that is still running a previous execution. Given that driver to switch from the simple scheduler to Quartz is often to get scheduling across multiple nodes, this behavior is not as expected. This problem is discussed in #13048 although that issue is looking for something different.

Expected behavior

Concurrent execution should be skipped if a job is still executing anywhere in the cluster. See https://stackoverflow.com/a/5801514 for an example of how to implement this with Quartz.

Actual behavior

Concurrent execution is only skipped if a job is still executing in the same process that a subsequent execution is scheduled to.

[quarkus-bot]

/cc @machi1990, @mkouba

[mkouba]

Scheduled#concurrentExecution() is not intended to work accross the cluster.

@machi1990 Do you happen to know if it would be possible to map the Scheduled#concurrentExecution() to a Quartz-specific feature?

[davidcurrie]

@mkouba - it seems odd to me that, by switching from the simple scheduler to Quartz, I go from running a job in every node to running that job in only one node, yet the semantics of the concurrent execution option don't mirror that. I'm not sure what the usage pattern is that would mean that you wouldn't want it to run concurrently in the same node but it would be fine to run it concurrently in a different one. (It certainly caught us by surprise.)

I'm no Quartz expert but I don't think there is a direct equivalent (@DisallowConcurrentExecution queues jobs rather than skipping them) but, as per the SO post I linked to, if you have access to the JobExecutionContext it is easy enough to manually check whether the same trigger is still being executed elsewhere and skip.

[mkouba]

I'm not a Quartz expert either but AFAIK the @DisallowConcurrentExecution annotation can only be used to prevent multiple job executions on a single node and the org.quartz.Scheduler.getCurrentlyExecutingJobs() method's javadoc (mentioned in the SO post) reads: "This method is not cluster aware. That is, it will only return Jobs currently executing in this Scheduler instance, not across the entire cluster".

[davidcurrie]

Apologies for the bum steer. The information is patently in the Quartz DB but it is not obviously exposed in any API. Seems I will have to look elsewhere for what seems like such an obvious capability.

[machi1990]

Scheduled#concurrentExecution() is not intended to work accross the cluster.

@machi1990 Do you happen to know if it would be possible to map the Scheduled#concurrentExecution() to a Quartz-specific feature?

@mkouba I do not think / know if there is an equivalent.