[Backgroundrb-devel] [ANN] BackgrounDRb 1.0 pre-release available now
emarceta at gmail.com
Tue Dec 11 04:53:13 EST 2007
On Dec 11, 2007 12:47 AM, hemant <gethemant at gmail.com> wrote:
> Hi Folks,
> We are glad to announce shiny new release of BackgrounDRb, which will
> soon become 1.0.
> A quick summary of changes:
> - BackgrounDRb is no londer DRb, its based on event driven network
> programming library packet ( http://www.packet.googlecode.com ) .
> - Since we moved to packet, many nasty thread issues, result hash
> corruption issues are totally gone. Lots of work has went in
> making scheduler rock solid stable.
Does this means that slave/daemons are not the dependency anymore?
> - Each worker, still runs in its own process, but each worker has a
> event loop of its own and all the events are triggered by the internal
> reactor loop. In a nutshell, you are not encouraged to use threads
> in your workers now.
By 'not encouraged' do you mean that 1.0 is not supporting multiple
threads in the worker or just as a general guidance?
Could you please comment, how would you approach the following
scenario with 1.0. Currently, we have a worker that creates threads
that process financial payment transactions. An http request sends
several 10s or 100s payment transaction records. They are handled by
the single worker instance. Within the worker there is a pool of
threads created that is calculated based on the number of
transactions. For example for 200 transactions there will be 20
threads where each thread handles 10 requests in a squence. Each
transaction takes about 3-5 seconds, so our throughput is
significantly improved by internal worker parallelization with a
thread pool. The worker periodically updates custom backgroundjob
databse record, so that following ajax request from the client can
read the status of the worker process. The job is identified with the
worker key that is stored in the session.
> All the workers are already concurrent, but you
> are encouraged to use co-operative multitasking, rather than
> pre-emptive. A simple example is,
> For implement something like progress bar in old version of bdrb, you would:
> - start your processing a thread (so as your worker can receive
> further request from rails ) and have a instance
> variable ( protected by mutex ) which is updated on progress and
> can be send to rails.
> With new backgroundrb, progrss bar would be:
> - process your damn request and just use register_status() to
> register status of your worker. Just because
> you are doing some processing won't mean that your worker will
> block. It can still receive requests from rails.
How this works with fastcgi or multiple mongrel based engines where it
is not guaranteed to hit the same process with the next request? We
are using custom database tables and code for sharing the status
information now but I was wandering whether the plumbing includes
something to address this.
More information about the Backgroundrb-devel