[Backgroundrb-devel] extending bdrb / running multiple servers

hemant gethemant at gmail.com
Tue Mar 25 18:27:42 EDT 2008


On Tue, Mar 25, 2008 at 10:47 PM, Stephen Weiss <sweiss at stylesight.com> wrote:
> Hi,
>
>  Thanks for the help getting the new backgroundrb working, things are
>  working fabulously now.
>
>  I have two questions:
>
>  1)  I have a few basic methods I need available on all my workers in
>  order to communicate properly with my application... it seems to make
>  the most sense right now to subclass BackgrounDRb::MetaWorker, or make
>  a mixin module, because I'm actually extending some of the base
>  methods as well (like the exit method, which I have extended to update
>  status information in a central queue monitor).  Is there a proper
>  place to put in custom code that extends BackgrounDrb, so that it is
>  automatically loaded by the server before the workers are instantiated
>  (sort of like an environment.rb)?

For the time being, you can put that file in lib folder of your rails
application and require that file in environment.rb.

>
>  2)  I have one application server that runs Rails and does all the
>  user interface, but several (10+) servers running backgroundrb to do
>  all the heavy lifting (we do a lot of hi-res image processing,
>  requires many CPU's).  I have a central monitor that monitors what
>  servers are available to run which tasks, and load balances the tasks
>  across all the available backgroundrb servers, instantiating a
>  MiddleMan for the correct server and then sending the job request to
>  that server.  Needless to say, this process is going a lot easier with
>  the new version, except for one catch.  With the old backgroundrb, I
>  was able to connect to an arbitrary server like this:
>
>  thisMiddleMan = DRbObject.new(nil, "druby://#{server_name}:22223");
>
>  Now this doesn't work.  So, I took a look at
>  BackgrounDRb::WorkerProxy, and it seems like it can only connect to
>  the server listed in the config file.  Is there supposed to be a way
>  to manage multiple backgroundrb servers?  I've read the manual, it's
>  not in there.  My solution so far is to subclass WorkerProxy, with a
>  new init method that takes the host and port as parameters:
>
>  class BackgrounDRb::CustomProxy < BackgrounDRb::WorkerProxy
>    def self.init(ip, port)
>      @@config = BackgrounDRb::Config.read_config("#{BACKGROUNDRB_ROOT}/
>  config/backgroundrb.yml")
>      @@server_ip = ip
>      @@server_port = port
>      new
>    end
>  end
>
>  But this doesn't seem very elegant to me, I worry about future
>  software updates...  Is there a clean method for doing this or is this
>  just not supported (yet)?
>

Since, each connection to BackgrounDRb server is nothing but a plain
tcp socket connection. Now, before composing a reply to your message,
I actually committed a little code change, which allows you do do:

Loading development environment (Rails 2.0.2)
server ip: 0.0.0.0 11006
>> foo = BackgrounDRb::WorkerProxy.custom_connection("localhost",11006)
server ip: localhost 11006
=> #<BackgrounDRb::WorkerProxy:0xb7477e14 @mutex=#<Mutex:0xb7477dec>,
@connection_status=true, @connection=#<TCPSocket:0xb7477ce8>>
>> foo
=> #<BackgrounDRb::WorkerProxy:0xb7477e14 @mutex=#<Mutex:0xb7477dec>,
@connection_status=true, @connection=#<TCPSocket:0xb7477ce8>>
>> foo.all_worker_info
=> [{:job_key=>"", :status=>:running, :worker=>:log_worker}]
>> foo.query_all_worker_info
NoMethodError: undefined method `query_all_worker_info' for
#<BackgrounDRb::WorkerProxy:0xb7477e14>
        from (irb):4
>> foo.query_all_workers
=> {:log_worker=>nil}

So, update the code from git and use it.
I hope this helps.


More information about the Backgroundrb-devel mailing list