[Backgroundrb-devel] thread_pooling sleeping

Jason LaPier jason.lapier at gmail.com
Tue Dec 18 20:28:18 EST 2007


I'm trying to run a single worker that could perform a periodic task
for a given user.

>From a controller, I imagine something like:
def start_job
    MiddleMan.ask_work(:worker => :foo_worker, :worker_method => :perform_task,
      :data => { :user_id = current_user.id })
end

def check_job
    @status = MiddleMan.ask_status(:worker => :foo_worker)[current_user.id]
end

My worker is something like:
class FooWorker < BackgrounDRb::MetaWorker
  set_worker_name :foo_worker
  def create(args=nil)
    @mutex = Mutex.new
    @mutex.synchronize do
      @statuses = {}
      register_status(@statuses)
    end
  end

  def do_task(some_user_id)
    thread_pool.defer(some_user_id) do |user_id|
      user = User.find user_id
      save_status user_id, :progress, "Starting Task"
      user.do_some_database_stuff
      save_status user_id, :progress, "Task Stage 2"
      user.do_some_other_database_stuff
      save_status user_id, :progress, "Task Stage 3"
      save_status user_id, :completed, true
    end
  end

  def save_status(user_id, key, data,)
    @mutex.synchronize do
      if @statuses[user_id].nil?
        @statuses[user_id] = {}
        @statuses[user_id][:completed] = false
      end
      @statuses[user_id][key] = data
      register_status(@statuses)
      logger.info "statuses synced for #{user_id}, #{key}, #{data}"
    end
  end

end


Problem is, when I use thread_pool, it gets as far as the first part
of the task and then just dies, and the backgroundrb script just
outputs "going to sleep for a while" over and over until I kill it. I
tried taking out the mutex and the register_status bits and the same
problem happens. When I use "Thread.new" in place of the thread_pool
line, everything works. However, I get some "Some read error" messages
during the "do_task" work. Everything happens like it should - I even
ran two sessions concurrently and initiated the task with two
different users at the same time. The work gets done; although a bit
slowly and with those strange errors. Oddly enough, when I run the
jobs from the rails irb console, they go without any error - even if I
start a couple jobs for multiple users and spam "MiddleMan.ask_status"
while the jobs are still running. (of course, when I start the server
using the 'Thread.new' method, I get a "no marshal_dump is defined for
class Thread" exception in the backgroundrb_server log. I'm guessing
this has something to do with why I'm supposed to be using thread_pool
instead of Thread).

When I get those read errors, the backgroundrb_debug.log shows "Client
disconected" (sic). I don't know if this is normal for rails talking
to bgrb or if it's an error. I don't see it on every line, but it
seems to coincide with the "some read error" messages in the script.

Am I doing something wrong? Anyone have a working example using
thread_pool? Also, I've never used Mutex before, so if my usage is
off, please let me know.

Thanks very much,

- Jason L.


-- 
My Rails and Linux Blog: http://offtheline.net


More information about the Backgroundrb-devel mailing list