[Mongrel] Recommentation: Sessions and PStore
wyhaines at gmail.com
Sun Sep 3 19:10:09 EDT 2006
On 9/3/06, Zed Shaw <zedshaw at zedshaw.com> wrote:
> But, explain this:
> First one leaks, second one doesn't (with graphs even). What's worse is
> the inverse is true on win32. These scripts have no Mongrel code, no
> Rails code, they're just short Ruby scripts.
> Now, it's not about users, it's about the two bug reductions that
> simulate how Mongrel uses processes demonstrating a leak in both 1.8.5
> and 1.8.4. And it's insulting that you compare my careful examination
> of the problem and numerous people's countless investigation into this
> as nothing more than tribal Sun worship. If anything the Ruby morons
> (not you) running around ignoring Ruby's problems are the religious
> fanatics who need to get a grip and fix this crap before guys like me
> get fed up and run happily to a platform that doesn't suck.
No. People who thought that the sun revolved around the earth made a
perfectly logical assumption, given the evidence that they observed.
They were just missing some information that would have led them to a
more accurate conclusion.
When I read your original post about Mutex and Sync, and looked at the
graphs, my first thought was, "Ugh. Maybe that's where my memory leak
comes from!" I slept on it and when I woke in the morning, I dug into
your test scripts, the code for Mutex and Sync, and the GC subsystem.
I hacked the GC to give me detailed information about allocations,
sweeps, free hash spots, etc... I modified your scripts to give me
detailed object counts. I ran numerous variations of your tests.
I couldn't find a leak. In fact, your test script itself (with Mutex)
ate memory until it reached a stable point, then dropped a bit, then
climbed back up to that stable point. I could make it behave worse by
making the thread do something that actually creates a lot of garbage
objects for the GC to deal with, but it still did not seem to be a
It did exhibit some curious behavior that I am not satisfied that I
understand, though. When in a work loop that actually creates a lot
of garbage objects, many, many more of them are accumulated when
running under Mutex than when running under Sync. They seem to
accumulate in greater numbers than they are cleared by the GC.
However, doing nothing more than slowing the Mutex operation down to
around the same speed as a Sync fixed that problem.
Sync uses more up front resources, and is slower than Mutex. Both are
implemented in pure ruby, and if Mutex leaks, there is no hope,
because it is simpler than almost anything anybody writes in Ruby.
Have you tried monitoring objectspace counts in a Mongrel process that
appears to be leaking? Have you tried running it under valgrind?
Have you tried putting diagnostic hooks into a specially compiled Ruby
so that you can see when GC is called and when space is added to the
heap while running a leaking Mongrel?
I don't have any doubt at all that something funny is happening, but I
have yet to see anything, including your test scripts and graphs, that
actually indict Mutex.
BTW, in my own case, I did end up finding a memory leak in Ruby. It
was a stupid error in Hash causing some memory to never be freed. I
would not be all surprised if there are other similar errors hiding in
More information about the Mongrel-users