[Backgroundrb-devel] thread_pooling sleeping

hemant gethemant at gmail.com
Wed Dec 19 03:24:03 EST 2007

On Dec 19, 2007 6:58 AM, Jason LaPier <jason.lapier at gmail.com> wrote:
> I'm trying to run a single worker that could perform a periodic task
> for a given user.
> >From a controller, I imagine something like:
> def start_job
>     MiddleMan.ask_work(:worker => :foo_worker, :worker_method => :perform_task,
>       :data => { :user_id = current_user.id })
> end
> def check_job
>     @status = MiddleMan.ask_status(:worker => :foo_worker)[current_user.id]
> end
> My worker is something like:
> class FooWorker < BackgrounDRb::MetaWorker
>   set_worker_name :foo_worker
>   def create(args=nil)
>     @mutex = Mutex.new
>     @mutex.synchronize do
>       @statuses = {}
>       register_status(@statuses)
>     end
>   end
>   def do_task(some_user_id)
>     thread_pool.defer(some_user_id) do |user_id|
>       user = User.find user_id
>       save_status user_id, :progress, "Starting Task"
>       user.do_some_database_stuff
>       save_status user_id, :progress, "Task Stage 2"
>       user.do_some_other_database_stuff
>       save_status user_id, :progress, "Task Stage 3"
>       save_status user_id, :completed, true
>     end
>   end
>   def save_status(user_id, key, data,)
>     @mutex.synchronize do
>       if @statuses[user_id].nil?
>         @statuses[user_id] = {}
>         @statuses[user_id][:completed] = false
>       end
>       @statuses[user_id][key] = data
>       register_status(@statuses)
>       logger.info "statuses synced for #{user_id}, #{key}, #{data}"
>     end
>   end
> end
> Problem is, when I use thread_pool, it gets as far as the first part
> of the task and then just dies, and the backgroundrb script just
> outputs "going to sleep for a while" over and over until I kill it. I
> tried taking out the mutex and the register_status bits and the same
> problem happens. When I use "Thread.new" in place of the thread_pool
> line, everything works. However, I get some "Some read error" messages
> during the "do_task" work. Everything happens like it should - I even
> ran two sessions concurrently and initiated the task with two
> different users at the same time. The work gets done; although a bit
> slowly and with those strange errors. Oddly enough, when I run the
> jobs from the rails irb console, they go without any error - even if I
> start a couple jobs for multiple users and spam "MiddleMan.ask_status"
> while the jobs are still running. (of course, when I start the server
> using the 'Thread.new' method, I get a "no marshal_dump is defined for
> class Thread" exception in the backgroundrb_server log. I'm guessing
> this has something to do with why I'm supposed to be using thread_pool
> instead of Thread).
> When I get those read errors, the backgroundrb_debug.log shows "Client
> disconected" (sic). I don't know if this is normal for rails talking
> to bgrb or if it's an error. I don't see it on every line, but it
> seems to coincide with the "some read error" messages in the script.
> Am I doing something wrong? Anyone have a working example using
> thread_pool? Also, I've never used Mutex before, so if my usage is
> off, please let me know.
> Thanks very much,


I am having a look at the problem. Do not worry about "some read
error" messages, they just indicate that connection that rails was
having with backgroundrb server is closed. This happens quite often
because whenever you do:


A new connection to backgroundrb server is initiated and older one is
closed and at that time you get that message. I put those "puts" for
debugging purpose.

Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.


More information about the Backgroundrb-devel mailing list