[Backgroundrb-devel] Please comment on upcoming changes of backgroundrb

Stevie Clifton stevie at slowbicycle.com
Fri Jun 20 11:22:29 EDT 2008


Hey Hemant,

This is great.  Thanks for putting the effort to getting the git
version tagged so people feel comfortable using it in production!  I'm
assuming when you say that you'll tag the git release you'll also push
out new bdrd and packet gems, correct?

I had a quick question about fetch_parallely/run_concurrently.  Is it
basically doing the same thing as defer(), but just with a thread-safe
callback?  If so, it might be more intuitive to make the name of the
method reflect that, such as "defer_concurrently" so as not to make
people think that they do completely different things.

Also, if the above is true (run_concurrently is a thread-safe version
of defer), I think it would be less cumbersome if you just extended
defer() to allow the same functionality by passing the name of a
callback method in an options hash instead of creating an entirely new
method.  This way people could just use the block passed to defer() as
the request_proc, and another method at the worker level for the
response_proc instead of creating a bunch of procs to pass to the
method.  For example, instead of:

run_concurrently(args, request_proc, response_proc)

You could do:

def example_worker(args)
  defer(args, :callback => :my_callback_method) |args|
    # same as the body of request_proc
  end
end

def my_callback_method(result)
  # same as body of response_proc (call register_status or whatever)
end

I looked through the code in git for fetch_parallely, and I'm not sure
if this would give you the same flexibility, but it would be a more
intuitive solution to a user IMHO.

stevie

On Wed, Jun 18, 2008 at 4:44 PM, hemant <gethemant at gmail.com> wrote:
> Folks,
>
> I am getting ready for a new release of BackgrounDRb. It will be
> mostly tagging of git release which is being used in production. Not
> to mention, Packet and BackgrounDRb has seen a _lot_ of improvement
> and fixes since last release.
>
> Also, there are few changes that I want to introduce, please comment on them:
>
> 1. As posted earlier - A way of running threads inside backgroundrb
> and yet able to use result saving and stuff like that easily.
> Currently method is named fetch_parallely which i am planning to
> rename to run_concurrently. #defer stays as it is.
>
> http://gnufied.org/2008/06/12/unthreaded-threads-of-hobbiton/
>
> 2. Ability to cluster and connect to multiple backgroundrb servers. It
> involves some additions to backgroundrb.yml like:
>
> # following section is totally optional, and only useful if you are
> trying to cluster of backgroundrb server
> # if you do not specify this section backgroundrb will assume that,
> from rails you are connecting to the
> # backgroundrb server which has been specified in previous section
> :client:
>  :drb_servers: "10.0.0.1:11006,10.0.0.2:11007"
>
> So, when you say:
>
> MiddleMan.worker(:hello_worker).fooo(@user)
>
> BackgrounDRb will delegate the task to your backgroundrb servers in
> round robin manner. By using:
>
> MiddleMan.worker(:hello_worker)
>
> you can one specific instance of worker which is tied to one
> particular server. Also, #new_worker will be working a round robin
> manner, but you must call #delete on returned object.
>
> 3. With clustering, there comes the question of storing worker
> results. So far, backgroundrb result storage has been a bit of
> problem. In new version, I am planning to rename ask_status to
> ask_result. Also, register_status will be deprecated and for storing
> results, you will have to use:
>
> cache[some_key]  = result
>
> where cache is local cache in your worker.  note that, here you are
> defining a key for your result. But in memory process storage of
> results won't work if backgroundrb servers are clustered and hence,
> you will have to use memcache based storage if you are going to
> cluster your workers. mechanism will be the same, you will specify the
> key which will be combined with worker_name and worker_key.
>
> Also, job_key wherever used will be replaced with worker_key since I
> find that name confusing.
>
> Thats all folks. Please try git version. Stress test it and let me
> know about any problems.
>
> http://gnufied.org/2008/05/21/bleeding-edge-version-of-backgroundrb-for-better-memory-usage/
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>


More information about the Backgroundrb-devel mailing list