[Backgroundrb-devel] Rather ominous resource consumption figures
ezmobius at gmail.com
Fri Jun 30 10:48:11 EDT 2006
On Jun 30, 2006, at 7:05 AM, Douglass Turner wrote:
> GC is your friend. I use RMagick to prep my image before processing
> by my C++ engine. It was not releasing mems between successive
> calls to the worker. I now call GC.start after I nolonger need the
> image object created by RMagick.
> However, t still does appear that I have incurred a memory hit
> from the initial call to my worker that is never released even when
> the worker dies.
> On 6/30/06, Douglass Turner <douglass.turner at gmail.com> wrote:
> Yesterday I posted about how to email from a worker. I decided to
> be a bit piggy and just inhale all of rails. Sorted. I am noticing
> some rather ominous behavior though.
> The setup: My worker takes an uploaded image as a param and passes
> it (via system call) to a C++ image processing engine. Take a look
> at these memory consumption figures for successive runs uploading
> the identical image each time:
> All figures in megabytes
> 343 C++ image processing idle
> 365 max memory consumption during 3rd C++ image processing run
> 318 C++ image processing idle
> 340 max memory consumption during 2nd C++ image processing run
> 290 C++ image processing idle
> 310 max memory consumption during 1st C++ image processing run
> 256 fire up rails + lighttpd
> 225 fire up backgroundrb
> 207 base (Linux box idle)
> As Sly Stone used to say ".. I want to take you hiya ..". Any
> thoughts on how to tame this beast?
> Doug Turner
> skype: dduuggllaa
In the current implementation you are responsible for calling
MiddleMan.gc! with a timestamp. This will delete all workers older
then the timestamp you pass to gc!. So this could acount for a
perceived memory leak. So you either need to set up a cron script to
expire things older then a certain age. Or you need to use
MiddleMan.delete_worker(key). The reason for this is that the workers
are held in the @jobs hash. So even after the worker is done working,
it is not allowed to be GC'ed by ruby since the jobs hash still holds
a ref to it. The next release will add a time to live param and a
timer built in for expiring jobs and caches on a timed basis. For now
you need to delete_worker when your done or use a cron script like this:
Here is a ruby script you could run from cron that will delete all
cached items older then 30 minutes.
MiddleMan = DRbObject.new(nil, "druby://localhost:22222")
MiddleMan.gc!(Time.now - 60*30)
Let me know if this helps you or if you still have problems. Then we
can figure out what the issue is and correct it for you.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Backgroundrb-devel