[Backgroundrb-devel] thread_pooling sleeping

[Jason LaPier]

On Dec 19, 2007 1:18 AM, hemant <gethemant at gmail.com> wrote:
>
> Hi Jason,
>
> Looks like there is an exception being thrown thats not handled in
> your code. Here is my sample worker code, similar to yours. I also got
> similar behaviour as you described, but then i set
> "Thread.abort_on_exception = true" and also ran backgroundrb in
> foreground mode(see README file for details).
>
> And then it turned out, there was an exception being thrown in my code
> and hence it was not going beyond first stage. Also, you don't need to
> protect register_status() with mutex, but rather you should protect
> status variable with a mutex as demonstrated below.
>
>
> class ErrorWorker < BackgrounDRb::MetaWorker
>   set_worker_name :error_worker
>   attr_accessor :worker_status
>   def create(args = nil)
>     @worker_status = { }
>     @status_mutex = Mutex.new
>     register_status(@worker_status)
>     Thread.abort_on_exception = true
>   end
>
>   def do_task(user_id)
>     thread_pool.defer(user_id) do |user_id|
>       save_status(user_id,:progress,"Started")
>       t_user = World.find_by_id(user_id)
>       t_user.change_message
>       logger.info "message changed"
>       save_status(user_id,:progress,"message_changed")
>       t_user.change_name
>       logger.info "name changed"
>       save_status(user_id,:progress,"name changed")
>       t_user.save
>     end
>   end
>
>   def save_status(user_id,key,data)
>     @status_mutex.synchronize do
>       @worker_status[user_id] = { :key => key, :data => data}
>     end
>     register_status(@worker_status)
>     logger.info "status synced for user #{user_id} #{key} #{data}"
>   end
> end
>
> To stress test I did this:
>
> require File.join(File.dirname(__FILE__) + "/../config/environment")
> p MiddleMan.ask_status(:worker => :error_worker)
>
> (7..200).to_a.each do |world_id|
>   MiddleMan.ask_work(:worker => :error_worker, :worker_method =>
> :do_task, :data => world_id)
>   p MiddleMan.ask_status(:worker => :error_worker)
> end
>
> and it worked flawlessly.
>
> See if this helps.
>

Ok, awesome - thanks very much for your help. With the
abort_on_exception flag and the foreground logging, I was able to
track down the problem. If you try something like:

def do_task(some_user_id)
  logger.info "do_task: some_user_id: #{some_user_id.class}"
  logger.info "starting task - defer to the thread pool"

  thread_pool.defer(some_user_id) do |user_id|
    logger.info "user_id (in pool): #{user_id.class}"
    # and the rest of it
  end
end

You'll see in the log:
do_task: some_user_id: Fixnum
starting task - defer to the thread pool
user_id (in pool): Array

So some_user_id is being converted to [user_id] for the block. Calling
AR's find method with an array gives you back an array (which was the
source of my original exception - trying to act on an Array as if it
were a User object). When you use User.find_by_id in your example, AR
is taking just the first element in the array and doing a find on
that.

I suspect that's a bug in thread_pool, or at least if it's not, it's a
little confusing syntactically. Now that I know how it works, I've got
my code working, so if it's not a bug, no problem :)

- Jason L.

-- 
My Rails and Linux Blog: http://offtheline.net