Garbage collection outside of request cycle?
luke at lukemelia.com
Thu May 13 16:29:04 EDT 2010
> On May 6, 2010, at 4:57 PM, Eric Wong wrote:
>> ==> big_app_gc.rb <==
>> # This shouldn't hurt overall performance as long as the server cluster
>> # is at <=50% CPU capacity, and improves the performance of most memory
>> # intensive requests. This serves to improve _client-visible_
>> # performance (possibly at the cost of overall performance).
I thought the list might be interested in how this worked for us. I applied the patch to execute GC between each request. I'm using NewRelic to measure the app. Prior to the patch our, we spent about 25% of our aggregate time serving a request in GC and our application was running at around 20-30% CPU load. Our running app shows up as using ~330MB of memory.
Applying the patch cut the time spent in GC time to nearly zero and as predicted CPU spiked. Client-perceived responsiveness increased as well. Unfortunately, during our busiest time of the day, CPU load got so high that nginx locked up, so we rolled back the patch.
I made a simple change to execute GC once every 5 requests and applied it again. Aggregate time spent in GC reduced to out 10% of total request time. This resulted in a bout a 25% overall improvement in client response time. Big win! CPU maxes out at about 80% with this configuration..
One other thing I did was force GC to execute before_fork, on the theory that with COW, we would want to fork in the tidiest state possible. I have not measured this on it's own to evaluate it's impact.
Thanks again for the help and code on this, Eric.
Considering how useful this is, perhaps unicorn should have an after_request hook, to avoid the need to monkey-patch?
More information about the mongrel-unicorn