 |
Forums |
Admin  Start New Thread
| Message: 1995 |
|
BY: Tim Hunter (rmagick)
DATE: 2004-09-18 15:25
SUBJECT: Help! My script runs out of memory! Normally we don't have to worry about memory management in Ruby scripts because Ruby's automatic garbage collection class ("GC") handles it for us. Scripts that use built-in Ruby objects like arrays and hashes should rarely, if ever, run out of memory. However, it is possible for a script that uses RMagick image objects to run out of memory. Here's why, and what to do when that happens.
Usually the problem occurs in a script that is processing a lot of large images in a loop. How many is "a lot" depends on the size of the images, how much memory is available, and how ImageMagick/GraphicsMagick is configured. When configured using the recommended QuantumDepth of 8, ImageMagick/GraphicsMagick uses a little over 4 bytes per pixel to load the image in memory. So, for example, a 4 megapixel image from your digital camera will use over 16Mb of memory! All of this memory is allocated by ImageMagick/GraphicsMagick. (If you configure ImageMagick/GraphicsMagick to use QuantumDepth 16, the images need 2x as much memory!)
To construct a Ruby object from this in-memory image, RMagick attaches the 16Mb image to a tiny Ruby object stub. This stub is allocated by Ruby and the memory it uses is the only memory that Ruby manages.
When RMagick constructs an image object, it stores a pointer to a "free hook" in the object. The free hook points to an RMagick internal function that GC calls when the image is no longer referenced and can be deallocated. When GC calls the free hook, RMagick calls ImageMagick/GraphicsMagick to release the memory used by the image. Until Ruby calls the free hook, RMagick can't return the memory because your script may still be using the image.
Keep in mind that GC doesn't "kick in" until Ruby has allocated a certain (fairly large) amount of memory. Suppose you read 10 4-megapixel images. Your script is using over 160MB, but Ruby only knows about the 10 little stubs. GC doesn't run because Ruby hasn't allocated very much memory yet. Your script can run out of memory before GC ever runs.
Notice that Ruby, RMagick, and ImageMagick/GraphicsMagick are all doing the "right" things: ImageMagick/GraphicMagick can't release the memory until RMagick tells it to. RMagick can't tell it to until GC calls the free hook. GC doesn't run because Ruby hasn't allocated very much memory.
The fix is to add one or more calls to `GC.start' in your script at the point(s) where you know that old images are no longer referenced. The `GC.start' method forces GC to run. GC will call the RMagick free hook for the images that are no longer in use, and RMagick will tell ImageMagick/GraphicsMagick to release the memory.
For example, suppose you're reading images from a directory in a loop. Every time you read an image, you assign the reference to the new image in the same variable, replacing the reference to the old image. Add the call to GC.start after calling `read'. This will force GC to run. Since there the old image is unreferenced, RMagick will cause the memory it is using to be released.
require 'RMagick'
include Magick
pics = Dir["*.jpg"]
pics.each do |pic|
img = Image.read(pic).first
GC.start
# process new image
end
| |
Thread View
Post a followup to this message
|
|
Help! My script runs out of memory!