[Backgroundrb-devel] BackgrounDRb version 1.0RC1 available now

hemant gethemant at gmail.com
Wed Nov 14 00:01:43 EST 2007

Hi Folks,

BackgrounDRb is a Ruby job server and scheduler. Its main intent is to be
used with Ruby on Rails applications for offloading long-running tasks.
Since a Rails application blocks while serving a request it is best to
move long-running tasks off into a background process that is divorced
from http request/response cycle.

This new release of BackgrounDRb is also modular and can be used without
Rails. So any Ruby program or framework can use it.

Copyright (c) 2006 Ezra Zygmuntowicz,skaar[at]waste[dot]org,
Copyright (c) 2007 Hemant Kumar (mail[at]gnufied[dot]org)

== Usage

=== Installation
Getting the code:
  svn co http://svn.devjavu.com/backgroundrb/branches/version10/

Installation with svn externals:
  svn propedit svn:externals vendor/plugins
  [add the following line:]
  backgroundrb http://svn.devjavu.com/backgroundrb/branches/version10/
  [exit editor]

  svn ci -m 'updating svn:external svn property for backgroundrb' vendor/plugins
  svn up vendor/plugins
  rake backgroundrb:setup

Installation with piston:
  piston import
http://svn.devjavu.com/backgroundrb/branches/version10/ backgroundrb

=== Configuration
  Use rake task for initial configuration:

* Cron style scheduling and config

  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006
  | :schedules:
  |   :foo_worker:
  |     :worker_method: foobar
  |     :trigger_args: */5 * * * * * *

* Normal Unix scheduler
  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006
  | :schedules:
  |   :foo_worker:
  |     :start: <%= Time.now + 5.seconds %>
  |     :end: <%= Time.now + 10.minutes %>
  |     :repeat_interval: <%= 1.minute %>

* Plain config
  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006

=== Scheduling
  There are three schemes for periodic execution and scheduling.
  - Cron Scheduling
    You can use configuration file for cron scheduling of workers.
Method specified in configuration
    file would be called periodically. You should take care of the
fact that, time gap between periodic
    invocation of a method should be more than the time thats actually
required to execute the method.
    If a method takes longer time than the time window specified, your
method invocations would lag
  - Normal Scheduler
    You can use second form of scheduling as shown in config file.
  - add_periodic_timer method
    A third and very basic form of scheduling that you can use is,
"add_periodic_timer" method. You can call
    method from anywhere in your worker.

           def create
             add_periodic_timer(5) { say_hello }

    Above snippet would register the proc for periodic execution at
every 5 seconds.

=== Code
  Install the plugin, and run setup task. Create a worker, using
worker generator.

          ./script/generatr worker bar

  You will have a bar_worker.rb in your RAILS_ROOT/lib/workers/(
called WORKER_ROOT henceforth ).
  Generated code will look like this:

  class BarWorker < BackgrounDRb::MetaWorker
    set_worker_name :bar_worker
    def create
      # this method is called, when worker is loaded for the first time
      puts "starting a bar worker"

    def process_request(p_data)
      user_input = p_data[:data]
      result = self.send(user_input[:method],user_input[:data])

  'create' method gets called, when worker is loaded and created. Each
worker runs in its
  own process and you can use 'create' for initializing worker specific stuff.
  All the requests, sent from rails, to bdrb would be received in
process_request method.
  Received data, contains client address as well, which will be used
to send the response
  back to client. You just need to extract the data from received data
and call appropriate
  method. Once, you are done with executing your method, you can use
"send_response" to send
  the result back to client ( rails in our case ).

  It should be noted that, you must pass original data as an parameter
with send_response
  method because original data contains client address.

  Following code snippet, would ask bdrb to execute method
'add_values' in 'foo_worker' with
  arguments '10+10' and return the result.

       MiddleMan.send_request(:worker => :foo_worker, :method =>
:add_values,:data => "10+10")

  You can also use register_status as described in following snippet
to register status of
  your worker with master, which can be directly queried from rails.


  From rails, you can query status object using following code:

       MiddleMan.ask_status(:worker => :foo_worker)

  Above code would return status object of 'foo_worker'. When you call
  from a worker, it replaces older state of the worker with master.
Since, master process
  stores status of the worker, all the status queries are served by
master itself. It can be
  used to store result hashes and stuff. Unlike previous versions of
bdrb, there shouldn't be
  any data corruption.

=== Legacy and deprecated stuff

   Although, You need to wrap your head a bit for understanding
"evented" model of network programming,
   but it gets easier once you get hang of it. Much of the older stuff
is deprecated. Here is a brief list:

   - ACL : gone, trust to thy firewalls.
   - Runtime scheduling through MiddleMan proxy object.
     I don't know if many people used this feature, and is probably
easy to implement back.
   - Threads: find .|grep 'Thread' , gone
   - Passing of arguments from configuration file.
     Again, I am not sure many people used it either, if you are
passing arguments to periodic
     methods from configuration files, you may as well hardcode that
argument in worker itself.

   Some people, asked, if we can do "progress bar" or "file_upload"
with new code base. The answer is yes, but
   again, changes would be required. "progress bar" example would look
like this:

     class ProgressWorker < BackgrounDRb::MetaWorker
       set_worker_name :progress_worker
       def create
         @counter = 0
         add_peridic_timer(2) { increment_counter }
       def increment_counter
         @counter += 1

   And using MiddleMan proxy, you can keep queering status of your progress bar:

       MiddleMan.ask_status(:worker => :progress_worker)

   I would welcome, anyone who contributes more examples back. You can
even use callbacks
   or Deferable pattern for invoking callbacks.

=== Exciting new stuff
  * Rock solid stable ( or will be , after few bug reports )
  * Master Process is using a hash for storing status of each worker.
I am thinking of adding something like this:

          storage: db
          database: worker_status
    Above code would fullfill thy wishes of persitent worker status
storage. Any more ideas, welcome.
  * Each worker comes with Event loop of its own and can potentially
do lots of fancy stuff. Two noteworthy methods are:


    If you are familiar with EventMachine or Twisted style of network
programming, above methods allow you to
    start tcp servers inside your workers or lets you connect to
external tcp servers. For Each accepted client or
    connected socket a instance of Handler class would be created and
integrated with main event loop.
    This can be used for worker to worker communication between
backgroundrb servers running on two machines.

    You are encouraged to look into framework directory, and see the
code that implements all this stuff.Guts of
    new bdrb is based on this library, which would be released soon as

== Online Resources
- http://svn.devjavu.com/backgroundrb/branches/version10/
- http://backgroundrb.devjavu.com (trac)
- http://backgroundrb.rubyforge.org (rdoc)

Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.


More information about the Backgroundrb-devel mailing list