Home

Description

As I found other Rails background project to unstable, complex or too memory hogging, I decided to make this.
WorkerQueue can handle large amounts of data, perform tasks on a separate machine and can execute tasks
either in parallel or in sequence. Got Work?

Installation

git clone git://github.com/barttenbrinke/worker_queue.git vendor/plugins/worker_queue

Switch to the 1_0_stable branch if you don’t like living on the edge.

Usage

Whenever you want to background a task, just do the following:

work = WorkerQueue::WorkerQueueItem.new work.class_name = ‘Someclass’ work.method_name = ‘some_method’ work.argument_hash = {:monkey => :tail} work.binary_data = ‘1234567890’ work.save!

This will be picked up by the worker when the following command is called:

WorkerQueue.work

Which in turn will call:

Someclass.some_method({:monkey => :tail, :binary_data => ‘1234567890’)

It is advised too set up a cronjob that runs the supplied rake task:
rake worker_queue:work

Note that WorkerQueue will run only one task of a single class at the same time.
So if you want to queue a execute the items one after the other, make sure all items are set
to the same class.
This is also very useful for things like uploaded files that need to be handled in the order in which
they were uploaded.

If you just want to execute items in the background, supply a unique classname for each item and
WorkerQueue will try its best to complete them as fast as possible.

After the WorkerQueue has executed a task successfully, it will clear the binary (if any) and move on
to the next item. WorkerQueue has no problem with multiple instances of it being run a the same time.

Use the following migration to get the show on the road:

class CreateWorkerQueueItems < ActiveRecord::Migration def self.up create_table :worker_queue_items, :options => ‘ENGINE=InnoDB DEFAULT CHARSET=UTF8’ do |t| t.column :task_name, :string t.column :task_group, :string t.column :class_name, :string t.column :method_name, :string t.column :argument_hash, :text t.column :data, :text t.column :start, :datetime t.column :status, :integer, :default => 0, :null => false t.column :error_message, :string t.column :filename, :string t.column :lock_version, :integer, :default => 0, :null => false t.timestamps end change_column :worker_queue_items, :data, :text, :limit => 256.megabytes + 1 add_index :worker_queue_items, :status end def self.down drop_table :worker_queue_items end end

Notes

• The specs only run in a rails project with the WorkerQueue, rspec and rspec_on_rails plugins installed.
  This needs fixing.

• If you want to store big files in the binary_data field, make sure you MYSQL config accepts
  inserts that are big enough.

• The tasks that get executed expect a Class.method(args_hash, binary_blob) that returns true
  or false.