[Backgroundrb-devel] BackgrounDRb version 1.0RC1 available now

hemant gethemant at gmail.com
Wed Nov 14 08:55:51 EST 2007


On Nov 14, 2007 6:50 PM, Wood, Peter <p.wood at lancaster.ac.uk> wrote:
> Heya all,
>
> A coworker and I have checked out the new version of BackgrounDRb in
> order to build one of our latest projects, so far we've just tried to
> replicate some of the functionality we had when testing the old trunk
> version.
>
> I may be missing something obvious, but from the code it doesn't appear
> that you can set off multiple copies of the same worker like you could
> in the old trunk version. We intend to use BackgrounDRb to set off snmp
> scans of various devices on our network, some of which take some time to
> complete. It is likely that this will happen more than once at any
> particular moment in time.
>
> At the moment, using the sample code we've moved the reply to the
> MiddleMan to before the work, therefore actually letting the calling
> application carry on (which is what we need). However a second call to
> the same worker obviously hangs until the first call has completed.
>
> I think what I'm essentially saying is that we want multiple instances
> of the same worker which can be identified by a key (as it was in the
> trunk). Is this possible with the new version of BackgrounDRb, or is it
> on the map?
>

With new version you can't have two instances of a worker active at
the same time. Although, I do think of having that facility soon, but
for the time being you will have to put up with writing copies of
workers. BTW, how many copies do you need and how much time, does
it take to complete scan of devices on network?



>
> -----Original Message-----
> From: backgroundrb-devel-bounces at rubyforge.org
> [mailto:backgroundrb-devel-bounces at rubyforge.org] On Behalf Of hemant
> Sent: 14 November 2007 05:02
> To: backgroundrb-devel at rubyforge.org
> Subject: [Backgroundrb-devel] BackgrounDRb version 1.0RC1 available now
>
> Hi Folks,
>
>
> BackgrounDRb is a Ruby job server and scheduler. Its main intent is to
> be used with Ruby on Rails applications for offloading long-running
> tasks.
> Since a Rails application blocks while serving a request it is best to
> move long-running tasks off into a background process that is divorced
> from http request/response cycle.
>
> This new release of BackgrounDRb is also modular and can be used without
> Rails. So any Ruby program or framework can use it.
>
> Copyright (c) 2006 Ezra Zygmuntowicz,skaar[at]waste[dot]org,
> Copyright (c) 2007 Hemant Kumar (mail[at]gnufied[dot]org)
>
> == Usage
>
> === Installation
> Getting the code:
>   svn co http://svn.devjavu.com/backgroundrb/branches/version10/
>
> Installation with svn externals:
>   svn propedit svn:externals vendor/plugins
>   [add the following line:]
>   backgroundrb http://svn.devjavu.com/backgroundrb/branches/version10/
>   [exit editor]
>
>   svn ci -m 'updating svn:external svn property for backgroundrb'
> vendor/plugins
>   svn up vendor/plugins
>   rake backgroundrb:setup
>
> Installation with piston:
>   piston import
> http://svn.devjavu.com/backgroundrb/branches/version10/ backgroundrb
>
> === Configuration
>   Use rake task for initial configuration:
>
> * Cron style scheduling and config
>
>   | :backgroundrb:
>   |   :ip: localhost
>   |   :port: 11006
>   |
>   | :schedules:
>   |   :foo_worker:
>   |     :worker_method: foobar
>   |     :trigger_args: */5 * * * * * *
>
> * Normal Unix scheduler
>   | :backgroundrb:
>   |   :ip: localhost
>   |   :port: 11006
>   | :schedules:
>   |   :foo_worker:
>   |     :start: <%= Time.now + 5.seconds %>
>   |     :end: <%= Time.now + 10.minutes %>
>   |     :repeat_interval: <%= 1.minute %>
>
> * Plain config
>   | :backgroundrb:
>   |   :ip: localhost
>   |   :port: 11006
>
> === Scheduling
>   There are three schemes for periodic execution and scheduling.
>   - Cron Scheduling
>     You can use configuration file for cron scheduling of workers.
> Method specified in configuration
>     file would be called periodically. You should take care of the fact
> that, time gap between periodic
>     invocation of a method should be more than the time thats actually
> required to execute the method.
>     If a method takes longer time than the time window specified, your
> method invocations would lag
>     perpetually.
>   - Normal Scheduler
>     You can use second form of scheduling as shown in config file.
>   - add_periodic_timer method
>     A third and very basic form of scheduling that you can use is,
> "add_periodic_timer" method. You can call
>     method from anywhere in your worker.
>
>            def create
>              add_periodic_timer(5) { say_hello }
>            end
>
>     Above snippet would register the proc for periodic execution at
> every 5 seconds.
>
> === Code
>   Install the plugin, and run setup task. Create a worker, using worker
> generator.
>
>           ./script/generatr worker bar
>
>   You will have a bar_worker.rb in your RAILS_ROOT/lib/workers/( called
> WORKER_ROOT henceforth ).
>   Generated code will look like this:
>
>   class BarWorker < BackgrounDRb::MetaWorker
>     set_worker_name :bar_worker
>     def create
>       # this method is called, when worker is loaded for the first time
>       puts "starting a bar worker"
>     end
>
>     def process_request(p_data)
>       user_input = p_data[:data]
>       result = self.send(user_input[:method],user_input[:data])
>       send_response(p_data,result)
>     end
>   end
>
>   'create' method gets called, when worker is loaded and created. Each
> worker runs in its
>   own process and you can use 'create' for initializing worker specific
> stuff.
>   All the requests, sent from rails, to bdrb would be received in
> process_request method.
>   Received data, contains client address as well, which will be used to
> send the response
>   back to client. You just need to extract the data from received data
> and call appropriate
>   method. Once, you are done with executing your method, you can use
> "send_response" to send
>   the result back to client ( rails in our case ).
>
>   It should be noted that, you must pass original data as an parameter
> with send_response
>   method because original data contains client address.
>
>   Following code snippet, would ask bdrb to execute method 'add_values'
> in 'foo_worker' with
>   arguments '10+10' and return the result.
>
>        MiddleMan.send_request(:worker => :foo_worker, :method =>
> :add_values,:data => "10+10")
>
>   You can also use register_status as described in following snippet to
> register status of
>   your worker with master, which can be directly queried from rails.
>
>        register_status(some_status_data)
>
>   From rails, you can query status object using following code:
>
>        MiddleMan.ask_status(:worker => :foo_worker)
>
>   Above code would return status object of 'foo_worker'. When you call
> register_status
>   from a worker, it replaces older state of the worker with master.
> Since, master process
>   stores status of the worker, all the status queries are served by
> master itself. It can be
>   used to store result hashes and stuff. Unlike previous versions of
> bdrb, there shouldn't be
>   any data corruption.
>
> === Legacy and deprecated stuff
>
>    Although, You need to wrap your head a bit for understanding
> "evented" model of network programming,
>    but it gets easier once you get hang of it. Much of the older stuff
> is deprecated. Here is a brief list:
>
>    - ACL : gone, trust to thy firewalls.
>    - Runtime scheduling through MiddleMan proxy object.
>      I don't know if many people used this feature, and is probably easy
> to implement back.
>    - Threads: find .|grep 'Thread' , gone
>    - Passing of arguments from configuration file.
>      Again, I am not sure many people used it either, if you are passing
> arguments to periodic
>      methods from configuration files, you may as well hardcode that
> argument in worker itself.
>
>    Some people, asked, if we can do "progress bar" or "file_upload"
> with new code base. The answer is yes, but
>    again, changes would be required. "progress bar" example would look
> like this:
>
>      class ProgressWorker < BackgrounDRb::MetaWorker
>        set_worker_name :progress_worker
>        def create
>          @counter = 0
>          add_peridic_timer(2) { increment_counter }
>        end
>        def increment_counter
>          @counter += 1
>          register_status(@counter)
>        end
>      end
>
>    And using MiddleMan proxy, you can keep queering status of your
> progress bar:
>
>        MiddleMan.ask_status(:worker => :progress_worker)
>
>    I would welcome, anyone who contributes more examples back. You can
> even use callbacks
>    or Deferable pattern for invoking callbacks.
>
> === Exciting new stuff
>   * Rock solid stable ( or will be , after few bug reports )
>   * Master Process is using a hash for storing status of each worker.
> I am thinking of adding something like this:
>
>         #backgroundrb.yml
>         status_storage:
>           storage: db
>           database: worker_status
>     Above code would fullfill thy wishes of persitent worker status
> storage. Any more ideas, welcome.
>   * Each worker comes with Event loop of its own and can potentially do
> lots of fancy stuff. Two noteworthy methods are:
>
>          connect(ip,port,Handler)
>          start_worker(ip,port,Handler)
>
>     If you are familiar with EventMachine or Twisted style of network
> programming, above methods allow you to
>     start tcp servers inside your workers or lets you connect to
> external tcp servers. For Each accepted client or
>     connected socket a instance of Handler class would be created and
> integrated with main event loop.
>     This can be used for worker to worker communication between
> backgroundrb servers running on two machines.
>
>     You are encouraged to look into framework directory, and see the
> code that implements all this stuff.Guts of
>     new bdrb is based on this library, which would be released soon as
> separately.
>
> == Online Resources
> - http://svn.devjavu.com/backgroundrb/branches/version10/
> - http://backgroundrb.devjavu.com (trac)
> - http://backgroundrb.rubyforge.org (rdoc)
>
>
>
>
> --
> Let them talk of their oriental summer climes of everlasting
> conservatories; give me the privilege of making my own summer with my
> own coals.
>
> http://gnufied.org
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org


More information about the Backgroundrb-devel mailing list