Bugs: Browse | Submit New | Admin

[#18392] Serious GetText 1.90.x memory leaks with Rails 2.0

Date:
2008-02-26 05:40
Priority:
3
Submitted By:
Simone Carletti (weppos)
Assigned To:
Nobody (None)
Category:
None
State:
Closed
Summary:
Serious GetText 1.90.x memory leaks with Rails 2.0

Detailed description
I upgraded from GetText 1.10 to 1.90 and I noticed some serious memory leaks.
Each Mongrel instance started to grow up from ~30Mb to ~600Mb (and counting).

With GetText 1.90 Mongrel instance increases 0.80 Mb each new connected client.

Have a look at the following tests:


config.cache_classes = false
GetText? Version	From 	To 	Logs
GetText? 1.9.0 	29.55	139.76	(1)
GetText? 1.10.0 	29.55	148.98	(2)
GetText? 1.90.0 	29.57	199.93	(3)

1

Transactions:                   1250 hits
Availability:                 100.00 %
Elapsed time:                 214.79 secs
Data transferred:               6.58 MB
Response time:                  7.87 secs
Transaction rate:               5.82 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   45.79
Successful transactions:        1250
Failed transactions:               0
Longest transaction:           11.19
Shortest transaction:           0.14

2

Transactions:                   1250 hits
Availability:                 100.00 %
Elapsed time:                 218.57 secs
Data transferred:               6.58 MB
Response time:                  8.06 secs
Transaction rate:               5.72 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   46.07
Successful transactions:        1250
Failed transactions:               0
Longest transaction:           21.09
Shortest transaction:           0.14

3

Transactions:                   1250 hits
Availability:                 100.00 %
Elapsed time:                 273.25 secs
Data transferred:               6.58 MB
Response time:                 10.10 secs
Transaction rate:               4.57 trans/sec
Throughput:                     0.02 MB/sec
Concurrency:                   46.22
Successful transactions:        1250
Failed transactions:               0
Longest transaction:           25.20
Shortest transaction:           0.14

config.cache_classes = true ΒΆ
GetText? Version	From 	To 	Logs
GetText? 1.9.0 	29.52	29.59	(4)
GetText? 1.10.0 	29.55	29.61	(5)
GetText? 1.90.0 	29.54	90.14	(6)

4

Transactions:                   1250 hits
Availability:                 100.00 %
Elapsed time:                  67.05 secs
Data transferred:               6.58 MB
Response time:                  2.02 secs
Transaction rate:              18.64 trans/sec
Throughput:                     0.10 MB/sec
Concurrency:                   37.67
Successful transactions:        1250
Failed transactions:               0
Longest transaction:            3.30
Shortest transaction:           0.03

5

Transactions:                   1250 hits
Availability:                 100.00 %
Elapsed time:                  68.71 secs
Data transferred:               6.58 MB
Response time:                  2.32 secs
Transaction rate:              18.19 trans/sec
Throughput:                     0.10 MB/sec
Concurrency:                   42.13
Successful transactions:        1250
Failed transactions:               0
Longest transaction:            2.83
Shortest transaction:           0.04

6

Transactions:                   1250 hits
Availability:                 100.00 %
Elapsed time:                  80.55 secs
Data transferred:               6.58 MB
Response time:                  2.62 secs
Transaction rate:              15.52 trans/sec
Throughput:                     0.08 MB/sec
Concurrency:                   40.71
Successful transactions:        1250
Failed transactions:               0
Longest transaction:            4.77
Shortest transaction:           0.07
 

Add A Comment: Notepad

Please login


Followup

Message
Date: 2008-07-24 13:26
Sender: Masao Mutoh

After some investigation, I fixed it again.

I hope the latest version fixed your problem.
Date: 2008-06-18 13:07
Sender: ilan berci

@Masao, 

Whoops, sorry it took me so long to respond but I got sidetracked
at work and then forgot about it.. :(  By the way, thank you
kindly for investing so much of your time to maintain this beauty,
it's very much appreciated.

As for your question, I believe that "ret" in this
case will never return an empty string as it is always checked
vs a model or class.

Secondly, all you really want to key on is the class, but you
can't use the class id as it will be reloaded upon every request
(within development environment anyways) so to avert this issue,
you just id on the class's name.. hence: ret.name.hash.

I hope this will suffice and it certainly works for me.. I don't
have any test code as I just needed to get it done asap.

Thanks once again

ilan
Date: 2008-06-13 08:04
Sender: Masao Mutoh

ilan,

I couldn't understand to use ret.name.hash instead of ret.id.
Sometimes ret.name may returns ""(empty strings) and
it's not good usage as an ID.

I hope to fix this problem carefully.
Could you post your sample/test script?
# Simple but reproduce your problem is recommanded.
Date: 2008-05-26 12:12
Sender: ilan berci

Big thanks to Steffen!

As an aside you can keep the caching benefit and avoid the leak
by changing bound_target()'s cache keying algorithm to use the
class's name's hash:  id = ret.name.hash instead of ret.id

Lastly, the leak is still present in 1.91

Thanks for posting the cause.... very much appreciated!

ilan

module GetText
  def bound_target(klass = self) # :nodoc:
    ret = (klass.kind_of? Module) ? klass : klass.class
    id = ret.name.hash
    if cached?
     tgt = @@__cache_bound_target[id]
     return tgt if tgt
    end

    if ret.name =~ /^\#<|^$/ or ret == GetText
       #GC for dead object_ids.
       ret = Object
       if @@__cache_bound_target.size >
CACHE_BOUND_TARGET_MAX_SIZE
          @@__cache_bound_target.clear
       end
    end
    @@__cache_bound_target[id] = ret
    ret
  end
end
Date: 2008-04-12 15:26
Sender: Masao Mutoh

Finally, I found 3 memory leaks and fixed them.

Try the CVS version of gettext if you need.

Thanks for reporting.
Date: 2008-04-08 05:07
Sender: Edek ZKrainykredek

It's the same for me. I'll try the patch submitted and let you
know if it solves my issue.
Date: 2008-03-17 13:24
Sender: Steffen Rusitschka

I've had the same problem and dugg deep into the gettext
implementation. There is a huge memory leak in lib/gettext.rb:

The method "bound_target" uses a static hash to speed
up object -> class lookups. Having object as key isn't the
wisest descicion: With every request, a new ActionController,
filled with all instance variables gets added to this hash.

"patching" the method with my implementation below
fixed the problem for me. It would be nice to have this patch
included in the next version ;-)

  def bound_target(klass = self) # :nodoc:
    ret = nil
    if klass.kind_of? Class or klass.kind_of? Module
      ret = klass
    else
      ret = klass.class
    end
    ret = GetText if ret.name =~ /^\#<|^$/
    ret
  end

it's basically the old (1.90.0) implementation with all caching
kicked out...
Date: 2008-02-27 04:20
Sender: Simone Carletti

I run further investigations and I found a small memory leak
also in previous version that could probably be one of the
causes of this issue.

As normal, when selected locale doens't have .po or .mo
files, gettext returns default untranslated language strings.
This is true for any software using gettext and for gettext
itself as well.

When you set in your software a locale name and gettext gem
doesn't have any .po or .mo file for this locale (in its own
installation), memory consuming immediately grow up.

Let me post a better example.
Rollback to gettext 1.10.0 and create an empty rails
installation and suppose it will support 3 locales: uk, it
and es.

Now, if I run the test with it or es it works great
(remember, 1.10 doesn't have the same memory leaks as 1.90).
If I switch locale to 'uk' or whatever language gettext
library doesn't provide .mo/.po files in its proper
/ruby/gems/1.8/gems/gettext-1.10.0/data/locale folder, then
mongrel starts to require more and more memory than usual
and never release it!

It seems like the library has some small bug handling memory
allocation when a default string is returned due to an
unavailable translated one.

Attached Files:

Name Description Download
No Files Currently Attached

Changes:

Field Old Value Date By
close_date2009-03-22 05:532009-03-22 05:53mutoh
status_idOpen2009-03-22 05:53mutoh
resolution_idAccepted2009-03-22 05:52mutoh
status_idClosed2008-06-13 08:04mutoh
resolution_idNone2008-04-12 15:26mutoh
close_date2008-04-12 15:262008-04-12 15:26mutoh
close_date2008-04-12 15:262008-04-12 15:26mutoh
status_idOpen2008-04-12 15:26mutoh