Sure<br><br>Mongrel <a href="http://0.3.13.4">0.3.13.4</a><br>Mongrel Cluster 0.2.0<br>Ruby 1.8.4<br>Rails 1.1.6<br>Apache 2.2.2<br>RHEL 4<br><br>The symptom is that we are getting frequent application 500 errors. Monitoring the mongrel cluster shows some of the servers in Err status at any given moment. We run 40 mongrel instances in the cluster and only a few of them are in Err status at a time. Does a mongrel instance in Err status return an application 500 error? I have not found documentation on the cluster monitor.
<br><br>The Apache config setup is standard, taken from Coda Hale's blog.<br><br>The Ruby on Rails code is straight forward. Very few gems are used, the memory footprint is only 35MB for each mongrel instance. I have not been able to find any error messages in the production log. What types of things would be major no-no's? I don't use sessions. The web application mainly uses AJAX to coordinate messaging between a server and the client.
<br><br>I have not been able to put the servers into debug mode for the past couple of days since they are running in production at our colo, which is the only place we've seen these application errors occur. Restarting them causes chats to get dropped as well searches our users are performing.
<br><br>The linux kernel's TCP/IP settings have been tweaked to the following:<br><pre>net.ipv4.tcp_fin_timeout = 30<br>net.ipv4.tcp_tw_recycle = 1<br>net.ipv4.tcp_keepalive_time = 1800<br>net.ipv4.tcp_window_scaling = 0<br>
net.ipv4.tcp_dsack = 0<br>net.ipv4.tcp_sack = 0<br>net.ipv4.tcp_timestamps = 0<br>net.ipv4.icmp_echo_ignore_broadcasts = 0<br>net.ipv4.inet_peer_threshold = 16536<br>net.ipv4.ipfrag_high_thresh = 5000000<br>net.ipv4.ipfrag_low_thresh
= 3000000<br>net.ipv4.conf.all.rp_filter = 0<br>net.ipv4.conf.lo.rp_filter = 0<br>net.ipv4.conf.eth0.rp_filter = 0<br>net.ipv4.conf.default.rp_filter = 0<br>net.core.netdev_max_backlog = 2500<br>net.core.optmem_max = 102400
<br>net.core.rmem_default = 262141<br>net.core.rmem_max = 262141<br>net.core.wmem_default = 262141<br>net.core.wmem_max = 262141<br>net.ipv4.route.gc_interval = 5<br>net.ipv4.route.gc_elasticity = 3<br>net.ipv4.route.gc_min_interval
= 1<br>net.ipv4.route.gc_timeout = 30<br>net.ipv4.route.max_size = 65536<br>net.ipv4.route.gc_thresh = 256<br>fs.file-max = 32768<br>net.ipv4.ip_local_port_range = 1024 65535</pre>Although we were noticing the application 500 errors before these tweaks and removing these tweaks did not seem to make a difference.
<br><br>Switching servers seems to resolve the problem for a day or so.<br><br><br><div><span class="gmail_quote">On 10/30/06, <b class="gmail_sendername">Zed A. Shaw</b> <<a href="mailto:zedshaw@zedshaw.com">zedshaw@zedshaw.com
</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">On Mon, 30 Oct 2006 10:02:29 -0500<br>"Jared Brown" <<a href="mailto:mongrelmail@gmail.com">
mongrelmail@gmail.com</a>> wrote:<br><br>> Configuration:<br>><br>> (2) Dual Core Opterons<br>> 8GB RAM<br>> Apache used to balance 40 mongrel instances<br>><br>> We receive Application 500 Errors. Nothing suspect appears in the log, so we
<br>> are at a lost as to what to do next.<br>><br>> Any advice would be welcome and/or an explanation of what types of things<br>> cause Application 500 Errors in mongrel.<br><br>Jared, you write software right? What would you do if someone came running into your office babbling about some kind of bug, but couldn't tell you important information you needed to know to fix the bug?
<br><br>As a programmer I expect other programmers to treat me as they want to be treated. If you ask a question on the list also report: the versions of your software, how you're using it, if you're doing anything odd, operating systems used, etc.
<br><br>C'mon, you know how to do this right.<br><br>--<br>Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu<br><a href="http://www.zedshaw.com/">http://www.zedshaw.com/</a><br><a href="http://safari.oreilly.com/0321483502">
http://safari.oreilly.com/0321483502</a> -- The Mongrel Book<br><a href="http://mongrel.rubyforge.org/">http://mongrel.rubyforge.org/</a><br><a href="http://www.lingr.com/room/3yXhqKbfPy8">http://www.lingr.com/room/3yXhqKbfPy8
</a> -- Come get help.<br>_______________________________________________<br>Mongrel-users mailing list<br><a href="mailto:Mongrel-users@rubyforge.org">Mongrel-users@rubyforge.org</a><br><a href="http://rubyforge.org/mailman/listinfo/mongrel-users">
http://rubyforge.org/mailman/listinfo/mongrel-users</a><br></blockquote></div><br>