[Backgroundrb-devel] worker process hanging around, spiking memory

Clint Troxel clint at ctro.net
Wed Feb 27 19:24:52 EST 2008


Greg,

that is helpful.  thanks!

calling exit() from the workers does indeed kill the process.

Two more questions:

1) before the process is killed by exit() I repeatedly see the memory
of the bdrb process spike from 18 to 45 M -- is there an explanation
for this?  Also, the process seems to take a long time to finish --
minutes just to send 4 emails.  Could some sort of configuration be
causing behavior like this?

2) More importantly, what is the recommended version to run?  And, how
do I check the version in my vendor dir? (I can't remember what
version you were at when I imported into my rails proj)

Thanks again,
-clint

On 2/27/08, Greg Campbell <gtcampbell at gmail.com> wrote:
> Hi Clint,
>
>
> On Wed, Feb 27, 2008 at 3:26 PM, Clint Troxel <clint at ctro.net> wrote:
> > Hi. Posted a general question earlier today.  I have a specific
> > question this time -- hoping someone can help out.  I think I'm
> > probably doing something incorrectly.  Here goes:
> >
> > I have a Worker method that sends a lot of emails.  The method looks like
> this:
> >
> >
> > def send_participant_reminders(args=nil)
> >    args[:emails].each do |email|
> >
> >      unless email.notified?
> >        Emailer.deliver_survey_reminder(email)
> >      end
> >
> >    end
> >  end
> >
> >
> > I call that worker from a controller like so:
> >
> >    MiddleMan.new_worker(:worker => :participant_worker)
> >    MiddleMan.ask_work(:worker => :participant_worker,
> >      :worker_method => :send_participant_reminders,
> >      :data => {:emails => @emails})
> >
> > ----------------------------
> >
> > My emails are being sent, but I'm seeing a new process created *each
> > time I invoke the code in the controller*.  That process starts out at
> > about 17m of memory then, once all work appears to have been
> > completed, jumps to about 50m.  The process stays around until
> > "script/backgroundrb stop".
> >
> > Maybe I've been reading the examples incorrectly?
> >
> > Any help would be appreciated.  Seeing this same problem (I think)
> > suck up 4G of ram on a production server.  yuck.
> >
> > Thanks again,
> > clint
>
> Calling new_worker does indeed create a new backgroundrb process (which may
> or may not be necessary in your case, see below).  If you do in fact need to
> create a new process for each task, you'll need to call either exit (from
> within the worker) or MiddleMan.delete_worker (from the controller) once the
> task is done. If you need to keep track of these worker tasks while they're
> running (to call delete_worker from the controller, or to use ask_status),
> you should also use a job_key when you create it.
>
> You may not actually need the new process, however.  If the likely use case
> is that there will only be one of these tasks running at a time, then you
> can just omit the new_worker call entirely and direct all calls to a single
> running instance of the worker process.  You may also want to investigate
> using thread_pool.defer inside the worker to run the individual tasks in
> threads (rather than spawning new worker processes for each one) if it seems
> likely that there will be multiple requests running simultaneously.
>
> Thanks,
> Greg
> p.s. If I've misrepresented anything, I'd like to invite Hemant or anyone
> else more knowledgeable than I am to correct me, as I've only started
> working on backgroundrb in the last few weeks.
>


More information about the Backgroundrb-devel mailing list