[rjb-users] Memory Growth

[Chris O'Meara]

Hi Everyone,

I spent some time creating a much smaller test case to demonstrate the
problem we're experiencing.

I have a test program, gc_experiment.rb, which creates a bunch of Java
strings, measures the effect of that work on the process rsize and
creates a bunch of Ruby strings and does the same.  The program also
dumps out an object count after each run.

I'll include the test results and the program below.  The results
point to a dramatic growth in rsize after the run with Java strings
but the contents of ObjectSpace look relatively similar between the
two runs.

Just to be thorough, I connected my profiler to the running test
program and checked the heap usage.  Heap size followed the expected
sawtooth pattern when graphed over time and the final usage was < 1mb.

Can anyone help me find my missing memory?

Thanks,
Chris

Test run results:

RSIZE before: 14364; after: 183136
 14189  String
   665  Array
   450  Class
   197  Regexp
   120  Proc
    88  Gem::Version
    82  Gem::Version::Requirement
    68  Module
    50  Range
    44  Gem::Specification
    44  Time
    37  Gem::Dependency
    36  Hash
    18  Float
     6  Object
     5  Gem::Security::Policy
     4  IO
     4  Bignum
     2  Process::Status
     2  Date::Infinity
     2  #<Class:0x513a38>
     2  YAML::Syck::Resolver
     1  Mutex
     1  Binding
     1  Rjb::Java_lang_String
     1  SystemStackError
     1  MatchData
     1  Gem::SourceIndex
     1  Rjb::Java_lang_String$CaseInsensitiveComparator
     1  NoMemoryError
     1  DL::Symbol
     1  Rjb::Java_lang_Class
     1  Thread
     1  fatal
     1  ThreadGroup
     1  Gem::GemPathSearcher
     1  File
     1  LoadError
RSIZE before: 183140; after: 183140
 14213  String
   703  Array
   450  Class
   197  Regexp
   121  Proc
    88  Gem::Version
    82  Gem::Version::Requirement
    68  Module
    50  Range
    44  Gem::Specification
    44  Time
    37  Gem::Dependency
    37  Hash
    18  Float
     6  Object
     5  Gem::Security::Policy
     4  Bignum
     4  IO
     2  YAML::Syck::Resolver
     2  Date::Infinity
     2  Process::Status
     1  Mutex
     1  SystemStackError
     1  Binding
     1  Rjb::Java_lang_String
     1  MatchData
     1  Gem::SourceIndex
     1  NoMemoryError
     1  Rjb::Java_lang_String$CaseInsensitiveComparator
     1  DL::Symbol
     1  fatal
     1  Rjb::Java_lang_Class
     1  Thread
     1  ThreadGroup
     1  File
     1  Gem::GemPathSearcher
     1  #<Class:0x513a38>
     1  LoadError

Test program (gc_experiment.rb):

require 'rubygems'
require 'rjb'

Rjb::load '', [ '-Djava.awt.headless=true', '-Xms8m', '-Xmx16m' ]
# Rjb::load '', [  '-agentlib:yjpagent', '-Djava.awt.headless=true',
'-Xms8m', '-Xmx16m' ]
JString = Rjb::import('java.lang.String')

def process_rsize
 GC.start
 `ps -orss= -p #{Process.pid}`.gsub(/[^0-9]/, '')
end

def report_rsize
 rsize_before = process_rsize
 10000000.times { yield }
 rsize_after = process_rsize
 puts "RSIZE before: #{rsize_before}; after: #{rsize_after}"
end

def report_object_counts
 counts = Hash.new{|h,k| h[k] = 0}
 ObjectSpace.each_object do |o|
   counts[o.class] += 1
 end
 counts.sort_by{|k,c| c}.reverse.each{|k,c| puts '%7d  %s' % [c, k]}
end

report_rsize do
 JString.new_with_sig 'Ljava.lang.String;', 'This is a Java string'
end
report_object_counts

report_rsize do
 String.new 'This is a Ruby string'
end
report_object_counts

# puts 'Hanging around if you want to connect profiler.  Press any key
to contiue.'
# gets