[Backgroundrb-devel] Rather ominous resource consumption figures

Ezra Zygmuntowicz ezmobius at gmail.com
Fri Jun 30 10:48:11 EDT 2006


Hey

On Jun 30, 2006, at 7:05 AM, Douglass Turner wrote:

> Ah,
>
> GC is your friend. I use RMagick to prep my image before processing  
> by my C++ engine. It was not releasing mems between successive  
> calls to the worker. I now call GC.start after I nolonger need the  
> image object created by RMagick.
>
> However, t still does appear that  I have incurred a memory hit  
> from the initial call to my worker that is never released even when  
> the worker dies.
>
> -Doug
>
>
> On 6/30/06, Douglass Turner <douglass.turner at gmail.com> wrote:
> Hello,
>
> Yesterday I posted about how to email from a worker. I decided to  
> be a bit piggy and just inhale all of rails. Sorted. I am noticing  
> some rather ominous behavior though.
>
> The setup: My worker takes an uploaded image as a param and passes  
> it (via system call) to a C++ image processing engine. Take a look  
> at these memory consumption figures for successive runs uploading  
> the identical image each time:
>
> All figures in megabytes
>
> 343 C++ image processing idle
> 365 max memory consumption during 3rd C++ image processing run
>
> 318 C++ image processing idle
> 340 max memory consumption during 2nd C++ image processing run
>
> 290 C++ image processing idle
> 310 max memory consumption during 1st C++ image processing run
>
> 256 fire up rails + lighttpd
> 225 fire up backgroundrb
>
> 207 base (Linux box idle)
>
>
> As Sly Stone used to say ".. I want to take you hiya ..". Any  
> thoughts on how to tame this beast?
>
> Cheers,
> Doug Turner
> skype: dduuggllaa
>

	In the current implementation you are responsible for calling  
MiddleMan.gc! with a timestamp. This will delete all workers older  
then the timestamp you pass to gc!. So this could acount for a  
perceived memory leak. So you either need to set up a cron script to  
expire things older then a certain age. Or you need to use  
MiddleMan.delete_worker(key). The reason for this is that the workers  
are held in the @jobs hash. So even after the worker is done working,  
it is not allowed to be GC'ed by ruby since the jobs hash still holds  
a ref to it. The next release will add a time to live param and a  
timer built in for expiring jobs and caches on a timed basis. For now  
you need to delete_worker when your done or use a cron script like this:

Here is a ruby script you could run from cron that will delete all  
workers and
cached items older then 30 minutes.

#!/usr/bin/env ruby
require "drb"
DRb.start_service
MiddleMan = DRbObject.new(nil, "druby://localhost:22222")
MiddleMan.gc!(Time.now - 60*30)

	Let me know if this helps you or if you still have problems. Then we  
can figure out what the issue is and correct it for you.

Cheers-
-Ezra


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060630/9e744c77/attachment.html 


More information about the Backgroundrb-devel mailing list