Skip to content

Latest commit

 

History

History
241 lines (193 loc) · 21.8 KB

faq.adoc

File metadata and controls

241 lines (193 loc) · 21.8 KB
Raw

Frequently Answered Questions about Quartz

Table of Contents

Frequently Answered Questions about Quartz

General Questions:

  1. What is Quartz?

    Quartz is a job scheduling library that can be integrated with, or used along side virtually any other software system. The term "job scheduler" seems to conjure different ideas for different people. As you read the tutorial, you should be able to get a firm idea of what we mean when we use this term, but in short, a job scheduler is a service component that is responsible for executing (or notifying) other software components when a pre-determined (scheduled) time arrives.

See Introduction for more.

What Isn’t Quartz?

Quartz is not a job queue - though it is often used as one, and can do a reasonable job of such at a small scale.

Quartz is not a grid computation engine - though simple, small scale usage as such can be achieved with effort.

Quartz is not a job execution service for use by business personnel - it is a code library that can be easily embeded into applications that need the capability to schedule application-related tasks to occur in the future or on a recurring basis.

What is Quartz - From a Software Component View?

Quartz is distributed as a small java library (.jar file) that contains all of the core Quartz functionality. The main interface (API) to this functionality is the Scheduler interface. It provides simple operations such as scheduling/unscheduling jobs, starting/stopping/pausing the scheduler.

If you wish to schedule your own software components for execution they must implement the simple Job interface, which contains the method execute(). If you wish to have components notified when a scheduled fire-time arrives, then the components should implement either the TriggerListener or JobListener interface.

The main Quartz 'process' can be started and ran within your own application, as a stand-alone application (with an RMI interface), or within a J2EE app. server to be used as a resource by your J2EE components.

Why not just use java.util.Timer?
Since JDK 1.3, Java has "built-in" timer capabilities, through the java.util.Timer and java.util.TimerTask classes - why would someone use Quartz rather than these standard features?

There are many reasons! Here are a few:

  • Timers have no persistence mechanism.

  • Timers have inflexible scheduling (only able to set start-time & repeat interval, nothing based on dates, time of day, etc.)

  • Timers don’t utilize a thread-pool (one thread per timer)

  • Timers have no real management schemes - you’d have to write your own mechanism for being able to remember, organize and retrieve your tasks by name, etc.

    …​of course to some simple applications these features may not be important, in which case it may then be the right decision not to use Quartz.

What are the available alternatives to Quartz?

There are no known competing open source projects (there are a few other open source schedulers, but they are basically just Cron replacements written in Java).

Commercially, you will want to look at the well-regarded Flux scheduler.

How do I build the Quartz source?

Although Quartz ships "pre-built" many people like to make their own alterations and/or build the latest 'non-released' version of Quartz from source. To do this, follow the instructions in the build.adoc documentation.

How do I disable the update check?

Quartz contains an "update check" feature that connects to a server to check if there is a new version of Quartz available for download. This check runs asynchronously and does not affect startup/initialization time of Quartz, and it fails gracefully if the connection cannot be made. If the check runs, and an update is found, it will be reported as available in Quartz’s logs.

You can disable the update check with the Quartz config property "org.quartz.scheduler.skipUpdateCheck: true" or the system property "org.terracotta.quartz.skipUpdateCheck=true" (which you can set in your system environment or as a -D on the java command line). It is recommended that you disable the update check for production deployments.

Miscellaneous Questions

  1. How many jobs is Quartz capable of running?

    This is a tough question to answer…​ the answer is basically "it depends".

    I know you hate that answer, so here’s some information about what it depends "on".

    First off, the JobStore that you use plays a significant factor. The RAM-based JobStore is MUCH (1000x) faster than the JDBC-based JobStore. The speed of JDBC-JobStore depends almost entirely on the speed of the connection to your database, which data base system that you use, and what hardware the database is running on. Quartz actually does very little processing itself, nearly all of the time is spent in the database. Of course RAMJobStore has a more finite limit on how many Jobs and Triggers can be stored, as you’re sure to have less RAM than hard-drive space for a database. You may also look at the FAQ "How do I improve the performance of JDBC-JobStore?"

    So, the limiting factor of the number of Triggers and Jobs Quartz can "store" and monitor is really the amount of storage space available to the JobStore (either the amount of RAM or the amount of disk space).

    Now, aside from "how many can I store?" is the question of "how many jobs can Quartz be running at the same moment in time?"

    One thing that CAN slow down quartz itself is using a lot of listeners (TriggerListeners, JobListeners, and SchedulerListeners). The time spent in each listener obviously adds into the time spent "processing" a job’s execution, outside of actual execution of the job. This doesn’t mean that you should be terrified of using listeners, it just means that you should use them judiciously - don’t create a bunch of "global" listeners if you can really make more specialized ones. Also don’t do "expensive" things in the listeners, unless you really need to. Also be mindful that many plug-ins (such as the "history" plugin) are actually listeners.

    The actual number of jobs that can be running at any moment in time is limited by the size of the thread pool. If there are five threads in the pool, no more than five jobs can run at a time. Be careful of making a lot of threads though, as the JVM, Operating System, and CPU all have a hard time juggling lots of threads, and performance degrades as the system spends time managing threads. In most cases performance starts to tank as you get into the several hundreds of threads (or fewer if the code being executed on the threads is intensive). Be mindful that if you’re running within an application server, it probably has created at least a few dozen threads of its own!

    Aside from those factors, it really comes down to what your jobs DO. If your jobs take a long time to complete their work, and/or their work is very CPU-intensive, then you’re obviously not going to be able to run very many jobs at once, nor very many in a given span of time.

    Finally, if you just can’t get enough horse-power out of one Quartz instance, you can always load-balance many Quartz instances (on separate machines). Each will run the jobs out of the shared database on a first-come first-serve basis, as quickly as the triggers need fired.

    The clustering feature works best for scaling out long-running and/or cpu-intensive jobs (distributing the work-load over multiple nodes). If you need to scale out to support thousands of short-running (e.g 1 second) jobs, consider partitioning the set of jobs by using multiple distinct schedulers. Using one scheduler forces the use of a cluster-wide lock, a pattern that degrades performance as you add more clients.

    So here you are this far into the answer of "how many", and I still haven’t given you an actual number. And I really hate to, because of all of the variables mentioned above. So let me just say, there are installments of Quartz out there that are managing hundreds-of-thousands of Jobs and Triggers, and that are, at any given moment in time executing several dozens of jobs – without even utilizing Quartz’s load-balancing capabilities. With this in mind, most people should feel confident that they can get the performance out of Quartz that they need.

  2. I’m having issues with using Quartz via RMI

    RMI can be a bit problematic, especially if you don’t have an understanding of how class loading via RMI works. I highly recommend reading all of the JavaDOC available about RMI, and strongly suggest you read the following references, dug up by a kind Quartz user (Mike Curwen)

    An excellent description of RMI and codebase: http://www.kedwards.com/jini/codebase.html. One of the important points is to realize that "codebase" is used by the client!

    The important 'take away' of the Java API is:

    RMI's class loader will not download any classes from remote locations if no security manager has been set.

Questions About Jobs

  1. How can I control the instantiation of Jobs?

    See org.quartz.spi.JobFactory and the org.quartz.Scheduler.setJobFactory(..) method.

  2. How do I keep a Job from being removed after it completes?

    Set the property JobDetail.setDurability(true) - which instructs Quartz not to delete the Job when it becomes an "orphan" (when the Job not longer has a Trigger referencing it).

  3. How do I keep a Job from firing concurrently?

    Annotate your job with @DisallowConcurrentExecution and/or @PersistJobDataAfterExecution.

  4. How do I stop a Job that is currently executing?

    See the org.quartz.InterruptableJob interface, and the Scheduler.interrupt(String, String) method. Questions About Triggers

  5. How do I chain Job execution? Or, how do I create a workflow?

    There currently is no "direct" or "free" way to chain triggers with Quartz. However there are several ways you can accomplish it without much effort. Below is an outline of a couple approaches:

    One way is to use a listener (i.e. a TriggerListener, JobListener or SchedulerListener) that can notice the completion of a job/trigger and then immediately schedule a new trigger to fire. This approach can get a bit involved, since you’ll have to inform the listener which job follows which - and you may need to worry about persistence of this information. See the listener org.quartz.listeners.JobChainingJobListener which ships with Quartz - as it already has some of this functionality.

    Another way is to build a Job that contains within its JobDataMap the name of the next job to fire, and as the job completes (the last step in its execute() method) have the job schedule the next job. Several people are doing this and have had good luck. Most have made a base (abstract) class that is a Job that knows how to get the job name and group out of the JobDataMap using pre-defined keys (constants) and contains code to schedule the identified job. This abstract Job’s implementation of execute() delegates to an abstract template method such as "doWork()" (where the extending Job class’s real work goes) and then it contains the code for scheduling the follow-up job. Then they simply make extensions of this class that included the work the job should do. The usage of 'durable' jobs, or the overloaded addJob(JobDetail, boolean, boolean) method (added in Quartz 2.2) helps the application define all the jobs at once with their proper data, without yet creating triggers to fire them (other than one trigger to fire the first job in the chain).

    In the future, Quartz will provide a much cleaner way to do this, but until then, you’ll have to use one of the above approaches, or think of yet another that works better for you.

  6. Why isn’t my trigger firing?

    The most common reason for this is not having called Scheduler.start(), which tells the scheduler to start firing triggers.

    The second most common reason is that the trigger or trigger group has been paused. Daylight Saving Time and Triggers

    CronTrigger and SimpleTrigger each handle daylight savings time in their own way - each in the way that is intuitive to the trigger type.

    First, as a review of what daylight savings time is, please read this resource: https://secure.wikimedia.org/wikipedia/en/wiki/Daylight_saving_time_around_the_world. Some readers may be unaware that the rules are different for different nations/contents. For example, the 2005 daylight savings time started in the United States on April 3, but in Egypt on April 29. It is also important to know that not only the dates are different for different locals, but the time of the shift is different as well. Many places shift at 2:00 am, but others shift time at 1:00 am, others at 3:00 am, and still others right at midnight.

    SimpleTrigger allows you to schedule jobs to fire every N milliseconds. As such, it has to do nothing in particular with respect to daylight savings time in order to "stay on schedule" - it simply keeps firing every N milliseconds. Regardless your SimpleTrigger is firing every 10 seconds, or every 15 minutes, or every hour or every 24 hours it will continue to do so. However the implication of this which confuses some users is that if your SimpleTrigger is firing say every 12 hours, before daylight savings switches it may be firing at what appears to be 3:00 am and 3:00 pm, but after daylight savings 4:00 am and 4:00 pm. This is not a bug - the trigger has kept firing exacly every N milliseconds, it just that the "name" of that time that humans impose on that moment has changed.

    CronTrigger allows you to schedule jobs to fire at certain moments with respect to a "Gregorian calendar". Hence, if you create a trigger to fire every day at 10:00 am, before and after daylight savings time switches it will continue to do so. However, depending on whether it was the Spring or Autumn daylight savings event, for that particular Sunday, the actual time interval between the firing of the trigger on Sundary morning at 10:00 am since its firing on Saturday morning at 10:00 am will not be 24 hours, but will instead be 23 or 25 hours respectively.

    There is one additional point users must understand about CronTrigger with respect to daylight savings. This is that you should take careful thought about creating schedules that fire between midnight and 3:00 am (the critical window of time depends on your trigger’s locale, as explained above). The reason is that depending on your trigger’s schedule, and the particular daylight event, the trigger may be skipped or may appear to not fire for an hour or two. As examples, say you are in the United States, where daylight savings events occur at 2:00 am. If you have a CronTrrigger that fires every day at 2:15 am, then on the day of the beginning of daylight savings time the trigger will be skipped, since, 2:15 am never occurs that day. If you have a CronTrigger that fires every 15 minutes of every hour of every day, then on the day daylight savings time ends you will have an hour of time for which no triggerings occur, because when 2:00 am arrives, it will become 1:00 am again, however all of the firings during the one o’clock hour have already occurred, and the trigger’s next fire time was set to 2:00 am - hence for the next hour no triggerings will occur.

    In summary, all of this makes perfect sense, and should be easy to remember if you keep these two rules in mind:

    SimpleTrigger ALWAYS fires exactly every N seconds, with no relation to the time of day.
CronTrigger ALWAYS fires at a given time of day and then computes its next time to fire. If that time does not occur on a given day, the trigger will be skipped. If the time occurs twice in a given day, it only fires once, because after firing on that time the first time, it computes the next time of day to fire on.

    Other trigger types that are based on sliding along a calendar (rather than exact amounts of time), such as CalenderIntervalTrigger, will be similarly affected - but rather than missing a firing, or firing twice, may end up having it’s fire time shifted by an hour.

Questions About JDBCJobStore

  1. How do I improve the performance of JDBC-JobStore?

    There are a few known ways to speed up JDBC-JobStore, only one of which is very practical.

    First, the obvious, but not-so-practical:

    • Buy a better (faster) network between the machine that runs Quartz, and the machine that runs your RDBMS.

    • Buy a better (more powerful) machine to run your database on.

    • Buy a better RDBMS.

      Now for something simple, but effective: Build indexes on the Quartz tables.

      Most database systems will automatically put indexes on the primary-key fields, many will also automatically do it for the foreign-key field. Make sure yours does this, or make the indexes on all key fields of every table manually.

      Next, manually add some additional indexes: most important to index are the TRIGGER table’s "next_fire_time" and "state" fields. Last (but not as important), add indexes to every column on the FIRED_TRIGGERS table.

      create index idx_qrtz_t_next_fire_time on qrtz_triggers(NEXT_FIRE_TIME);
create index idx_qrtz_t_state on qrtz_triggers(TRIGGER_STATE);
create index idx_qrtz_t_nf_st on qrtz_triggers(TRIGGER_STATE,NEXT_FIRE_TIME);
create index idx_qrtz_ft_trig_name on qrtz_fired_triggers(TRIGGER_NAME);
create index idx_qrtz_ft_trig_group on qrtz_fired_triggers(TRIGGER_GROUP);
create index idx_qrtz_ft_trig_name on qrtz_fired_triggers(TRIGGER_NAME);
create index idx_qrtz_ft_trig_n_g on \
    qrtz_fired_triggers(TRIGGER_NAME,TRIGGER_GROUP);
create index idx_qrtz_ft_trig_inst_name on qrtz_fired_triggers(INSTANCE_NAME);
create index idx_qrtz_ft_job_name on qrtz_fired_triggers(JOB_NAME);
create index idx_qrtz_ft_job_group on qrtz_fired_triggers(JOB_GROUP);
create index idx_qrtz_t_next_fire_time_misfire on \
    qrtz_triggers(MISFIRE_INSTR,NEXT_FIRE_TIME);
create index idx_qrtz_t_nf_st_misfire on \
    qrtz_triggers(MISFIRE_INSTR,NEXT_FIRE_TIME,TRIGGER_STATE);
create index idx_qrtz_t_nf_st_misfire_grp on \
    qrtz_triggers(MISFIRE_INSTR,NEXT_FIRE_TIME,TRIGGER_GROUP,TRIGGER_STATE);

      The clustering feature works best for scaling out long-running and/or cpu-intensive jobs (distributing the work-load over multiple nodes). If you need to scale out to support thousands of short-running (e.g 1 second) jobs, consider partitioning the set of jobs by using multiple distinct schedulers (and hence multiple sets of tables (with distinct prefixes)). Using one scheduler forces the use of a cluster-wide lock, a pattern that degrades performance as you add more clients.

  2. My DB Connections don’t recover properly if the database server is restarted.

    If you’re having Quartz create the connection data source (by specifying the connection parameters in the quartz properties file) make sure you have a connection validation query specified, such as:

    org.quartz.dataSource.myDS.validationQuery=select 0 from dual

    This particular query is extremely efficient for Oracle. For other databases, you’ll need to think of an efficient query that always works as long as the connection is good.

    If you’re datasource is managed by your application server, make sure the datasource is configured in such a way that it can detect failed connections.

Questions About Transactions

  1. I’m using JobStoreCMT and I’m seeing deadlocks, what can I do?

    JobStoreCMT is in heavy use, under heavy load by many people. It is believed to be free of bugs that can cause deadlock. However, every now and then we get complaints about deadlocks. In all cases thus far, the problem has turned out to be "user error", thus the list below is some things for you to check if you are experiencing deadlocks.

    Some databases falsely detect deadlocks when a tx takes a long time.

    • Make sure you have put indexes on your tables (see improving performance of JDBCJobStore).

    • Make sure you have at least number-of-threads-in-thread-pool + 2 connections in your datasources.

    • Make sure you have both a managed and non-managed datasource configured for Quartz to use.

    • Make sure that all work you do with the Scheduler interface is done from within a transaction. Accomplish this by using the Scheduler within a SessionBean that has its tx settings "Required" and "Container". Or within a MessageDrivenBean with similar settings.

    • Finally, start a UserTransaction yourself, and commit the work when done.

      If your Jobs' execute() methods use the Scheduler, make sure a transaction is in progress by using a UserTransaction or by setting the Quartz config property "org.quartz.scheduler.wrapJobExecutionInUserTransaction=true".

  2. I’m using Oracle RAC and I’m seeing deadlocks, what can I do?

    Oracle RAC has many limitations relating to transactions and locking. Quartz is known to work fine with Oracle RAC if you’re careful about the setup. The QTZ-149 issue contains some discussion and links that may help you if you’re having issues. Questions about Clustering, (Scaling and High-Availability) Features

  3. What clustering capabilities exist with Quartz?

    Quartz ships with well-proven clustering capabilities that offer scaling and high availability features. You can read about these features in the Quartz Configuration Reference.

    Additional clustering features that do not rely upon a backing database are available (free of cost) from Terracotta.

Questions About Spring

  1. I’m using Quartz via Spring’s scheduler wrappers, and I need help…​

    Check with the Spring community…​

  2. I’m seeing triggers stuck in the ACQUIRED state, or other weird data problems.

    Spring defaults the Quartz property "org.quartz.jobStore.dontSetAutoCommitFalse" to "true" - which means Quartz will not turn off auto-commit mode on the database connections that it uses. This is the opposite of Quartz’s own default for this setting. If your connection is defaulting to have auto-commit on, then you’ll run into all sorts of strange problems relating to data inconsistencies — the most common symptom being triggers that are "stuck" in the "ACQUIRED" state. Fix this by explicitly setting the property to "false".