[Backgroundrb-devel] BackgrounDRb version 1.0RC1 available now

Wood, Peter p.wood at lancaster.ac.uk
Wed Nov 14 08:20:00 EST 2007

Heya all,

A coworker and I have checked out the new version of BackgrounDRb in
order to build one of our latest projects, so far we've just tried to
replicate some of the functionality we had when testing the old trunk

I may be missing something obvious, but from the code it doesn't appear
that you can set off multiple copies of the same worker like you could
in the old trunk version. We intend to use BackgrounDRb to set off snmp
scans of various devices on our network, some of which take some time to
complete. It is likely that this will happen more than once at any
particular moment in time.

At the moment, using the sample code we've moved the reply to the
MiddleMan to before the work, therefore actually letting the calling
application carry on (which is what we need). However a second call to
the same worker obviously hangs until the first call has completed.

I think what I'm essentially saying is that we want multiple instances
of the same worker which can be identified by a key (as it was in the
trunk). Is this possible with the new version of BackgrounDRb, or is it
on the map?


Peter A. Wood                     
Network Security Specialist       
Technical Services Group          
Lancaster University

PGP Fingerprint:
C3B5 376B 16C5 F8D1 E3AE  617F 5718 C338 1D06 0689

-----Original Message-----
From: backgroundrb-devel-bounces at rubyforge.org
[mailto:backgroundrb-devel-bounces at rubyforge.org] On Behalf Of hemant
Sent: 14 November 2007 05:02
To: backgroundrb-devel at rubyforge.org
Subject: [Backgroundrb-devel] BackgrounDRb version 1.0RC1 available now

Hi Folks,

BackgrounDRb is a Ruby job server and scheduler. Its main intent is to
be used with Ruby on Rails applications for offloading long-running
Since a Rails application blocks while serving a request it is best to
move long-running tasks off into a background process that is divorced
from http request/response cycle.

This new release of BackgrounDRb is also modular and can be used without
Rails. So any Ruby program or framework can use it.

Copyright (c) 2006 Ezra Zygmuntowicz,skaar[at]waste[dot]org,
Copyright (c) 2007 Hemant Kumar (mail[at]gnufied[dot]org)

== Usage

=== Installation
Getting the code:
  svn co http://svn.devjavu.com/backgroundrb/branches/version10/

Installation with svn externals:
  svn propedit svn:externals vendor/plugins
  [add the following line:]
  backgroundrb http://svn.devjavu.com/backgroundrb/branches/version10/
  [exit editor]

  svn ci -m 'updating svn:external svn property for backgroundrb'
  svn up vendor/plugins
  rake backgroundrb:setup

Installation with piston:
  piston import
http://svn.devjavu.com/backgroundrb/branches/version10/ backgroundrb

=== Configuration
  Use rake task for initial configuration:

* Cron style scheduling and config

  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006
  | :schedules:
  |   :foo_worker:
  |     :worker_method: foobar
  |     :trigger_args: */5 * * * * * *

* Normal Unix scheduler
  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006
  | :schedules:
  |   :foo_worker:
  |     :start: <%= Time.now + 5.seconds %>
  |     :end: <%= Time.now + 10.minutes %>
  |     :repeat_interval: <%= 1.minute %>

* Plain config
  | :backgroundrb:
  |   :ip: localhost
  |   :port: 11006

=== Scheduling
  There are three schemes for periodic execution and scheduling.
  - Cron Scheduling
    You can use configuration file for cron scheduling of workers.
Method specified in configuration
    file would be called periodically. You should take care of the fact
that, time gap between periodic
    invocation of a method should be more than the time thats actually
required to execute the method.
    If a method takes longer time than the time window specified, your
method invocations would lag
  - Normal Scheduler
    You can use second form of scheduling as shown in config file.
  - add_periodic_timer method
    A third and very basic form of scheduling that you can use is,
"add_periodic_timer" method. You can call
    method from anywhere in your worker.

           def create
             add_periodic_timer(5) { say_hello }

    Above snippet would register the proc for periodic execution at
every 5 seconds.

=== Code
  Install the plugin, and run setup task. Create a worker, using worker

          ./script/generatr worker bar

  You will have a bar_worker.rb in your RAILS_ROOT/lib/workers/( called
WORKER_ROOT henceforth ).
  Generated code will look like this:

  class BarWorker < BackgrounDRb::MetaWorker
    set_worker_name :bar_worker
    def create
      # this method is called, when worker is loaded for the first time
      puts "starting a bar worker"

    def process_request(p_data)
      user_input = p_data[:data]
      result = self.send(user_input[:method],user_input[:data])

  'create' method gets called, when worker is loaded and created. Each
worker runs in its
  own process and you can use 'create' for initializing worker specific
  All the requests, sent from rails, to bdrb would be received in
process_request method.
  Received data, contains client address as well, which will be used to
send the response
  back to client. You just need to extract the data from received data
and call appropriate
  method. Once, you are done with executing your method, you can use
"send_response" to send
  the result back to client ( rails in our case ).

  It should be noted that, you must pass original data as an parameter
with send_response
  method because original data contains client address.

  Following code snippet, would ask bdrb to execute method 'add_values'
in 'foo_worker' with
  arguments '10+10' and return the result.

       MiddleMan.send_request(:worker => :foo_worker, :method =>
:add_values,:data => "10+10")

  You can also use register_status as described in following snippet to
register status of
  your worker with master, which can be directly queried from rails.


  From rails, you can query status object using following code:

       MiddleMan.ask_status(:worker => :foo_worker)

  Above code would return status object of 'foo_worker'. When you call
  from a worker, it replaces older state of the worker with master.
Since, master process
  stores status of the worker, all the status queries are served by
master itself. It can be
  used to store result hashes and stuff. Unlike previous versions of
bdrb, there shouldn't be
  any data corruption.

=== Legacy and deprecated stuff

   Although, You need to wrap your head a bit for understanding
"evented" model of network programming,
   but it gets easier once you get hang of it. Much of the older stuff
is deprecated. Here is a brief list:

   - ACL : gone, trust to thy firewalls.
   - Runtime scheduling through MiddleMan proxy object.
     I don't know if many people used this feature, and is probably easy
to implement back.
   - Threads: find .|grep 'Thread' , gone
   - Passing of arguments from configuration file.
     Again, I am not sure many people used it either, if you are passing
arguments to periodic
     methods from configuration files, you may as well hardcode that
argument in worker itself.

   Some people, asked, if we can do "progress bar" or "file_upload"
with new code base. The answer is yes, but
   again, changes would be required. "progress bar" example would look
like this:

     class ProgressWorker < BackgrounDRb::MetaWorker
       set_worker_name :progress_worker
       def create
         @counter = 0
         add_peridic_timer(2) { increment_counter }
       def increment_counter
         @counter += 1

   And using MiddleMan proxy, you can keep queering status of your
progress bar:

       MiddleMan.ask_status(:worker => :progress_worker)

   I would welcome, anyone who contributes more examples back. You can
even use callbacks
   or Deferable pattern for invoking callbacks.

=== Exciting new stuff
  * Rock solid stable ( or will be , after few bug reports )
  * Master Process is using a hash for storing status of each worker.
I am thinking of adding something like this:

          storage: db
          database: worker_status
    Above code would fullfill thy wishes of persitent worker status
storage. Any more ideas, welcome.
  * Each worker comes with Event loop of its own and can potentially do
lots of fancy stuff. Two noteworthy methods are:


    If you are familiar with EventMachine or Twisted style of network
programming, above methods allow you to
    start tcp servers inside your workers or lets you connect to
external tcp servers. For Each accepted client or
    connected socket a instance of Handler class would be created and
integrated with main event loop.
    This can be used for worker to worker communication between
backgroundrb servers running on two machines.

    You are encouraged to look into framework directory, and see the
code that implements all this stuff.Guts of
    new bdrb is based on this library, which would be released soon as

== Online Resources
- http://svn.devjavu.com/backgroundrb/branches/version10/
- http://backgroundrb.devjavu.com (trac)
- http://backgroundrb.rubyforge.org (rdoc)

Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

Backgroundrb-devel mailing list
Backgroundrb-devel at rubyforge.org

More information about the Backgroundrb-devel mailing list