Page request roundtrip time increases substantially after a bit of use

[Eric Wong]

chris mckenzie <kristopolous at yahoo.com> wrote:
> Hi Eric,
> 
> I'll prepare a more formal response in a bit, but here is my test run:
> rainbows -c config.rb rackup.ru

Thanks, I'm trying to reproduce it now.  Are you able to reproduce it
more quickly by throwing some ab or httperf runs in the mix to make more
requests?  Reducing worker_processes usually help reproduce issues more
quickly, too, especially if it's a resource leak.

> The memory footprint remained flat.
> The CPU usage did not spike noticeably
> netstat -an did reveal some CLOSE_WAIT values on the ports but nothing that 
> hadn't previously been pointed out.

Also, do you have any iptables/firewall/QoS settings on that machine
that could be interfering?

I haven't noticed anything in CLOSE_WAIT since I started testing it a
few minutes ago.  Maybe it takes longer, but CLOSE_WAIT has always been
a rarity to see in my years of working with TCP client/servers...

> The (CSV) output of the test can be seen here:
> http://qaa.ath.cx/single-request.csv.gz

Just covering all my bases here, you don't have a super slow
disk/filesystem that bogs down your entire system once your logs grow to
a certain size, right?

> For those of you without any visualization software, I made a rudimentary graph 
> from the data here:
> 
> http://qaa.ath.cx/single-request.png
> 
> You can clearly see how the delay increases and then doesn't ever go back down 
> to previous levels.

Very strange.

I'm testing on my 32-bit machine with ruby 1.8.7 (2010-12-23 patchlevel
330) [i686-linux] straight off of ruby-lang.org .  I usually use
Rainbows! with 1.9.2-p136 and will try that if I can reproduce the bug
under 1.8.7, too.

> I'll research answers to your previous questions now.  Thanks for looking into 
> this!

Alright, thanks!

-- 
Eric Wong