From lists at ruby-forum.com Tue Apr 1 09:55:53 2008 From: lists at ruby-forum.com (Michael Kinney) Date: Tue, 1 Apr 2008 15:55:53 +0200 Subject: [Mongrel] Mongrels stop responding In-Reply-To: References: Message-ID: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> Evan Weaver wrote: > Gdb the stuck mongrel and force it to raise a backtrace. I had to wait for one of them to hang again. This time, no CLOSE_WAIT. Also note, I've never used gdb before. I googled up how to attach to a PID and hit the "backtrace" command. If there's anything else I need to do, the GDB session is still available to me for a limited time (until enough people complain about the app being down). GNU gdb 6.4.90-debian Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i486-linux-gnu". Attaching to process 31635 Reading symbols from /usr/bin/ruby1.8...(no debugging symbols found)...done. Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1". Reading symbols from /usr/lib/libruby1.8.so.1.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libruby1.8.so.1.8 Reading symbols from /lib/tls/i686/cmov/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] [New Thread -1211282784 (LWP 31635)] [New Thread -1262781520 (LWP 31645)] Loaded symbols for /lib/tls/i686/cmov/libpthread.so.0 Reading symbols from /lib/tls/i686/cmov/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/tls/i686/cmov/libdl.so.2 Reading symbols from /lib/tls/i686/cmov/libcrypt.so.1... (no debugging symbols found)...done. Loaded symbols for /lib/tls/i686/cmov/libcrypt.so.1 Reading symbols from /lib/tls/i686/cmov/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/tls/i686/cmov/libm.so.6 Reading symbols from /lib/tls/i686/cmov/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/tls/i686/cmov/libc.so.6 Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/lib/ruby/1.8/i486-linux/etc.so... (no debugging symbols found)...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/etc.so Reading symbols from /usr/lib/ruby/1.8/i486-linux/stringio.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/stringio.so Reading symbols from /usr/lib/ruby/1.8/i486-linux/syck.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/syck.so Reading symbols from /usr/lib/ruby/1.8/i486-linux/socket.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/socket.so Reading symbols from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/lib/http11.so...done. Loaded symbols for /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/http11.so Reading symbols from /usr/lib/ruby/gems/1.8/gems/fastthread-1.0.1/lib/fastthread.so...done. Loaded symbols for /usr/lib/ruby/gems/1.8/gems/fastthread-1.0.1/lib/fastthread.so Reading symbols from /usr/lib/ruby/1.8/i486-linux/zlib.so...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/zlib.so Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /usr/lib/ruby/1.8/i486-linux/strscan.so...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/strscan.so Reading symbols from /usr/lib/ruby/1.8/i486-linux/bigdecimal.so...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/bigdecimal.so Reading symbols from /usr/lib/ruby/1.8/i486-linux/nkf.so...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/nkf.so Reading symbols from /usr/local/lib/site_ruby/1.8/i486-linux/oci8lib.so...done. Loaded symbols for /usr/local/lib/site_ruby/1.8/i486-linux/oci8lib.so Reading symbols from /usr/lib/oracle/11.1.0.1/client/lib/libclntsh.so.11.1...done. Loaded symbols for /usr/lib/oracle/11.1.0.1/client/lib/libclntsh.so.11.1 Reading symbols from /usr/lib/oracle/11.1.0.1/client/lib/libnnz11.so...done. Loaded symbols for /usr/lib/oracle/11.1.0.1/client/lib/libnnz11.so Reading symbols from /lib/tls/i686/cmov/libnsl.so.1...done. Loaded symbols for /lib/tls/i686/cmov/libnsl.so.1 Reading symbols from /usr/lib/libaio.so.1...done. Loaded symbols for /usr/lib/libaio.so.1 Reading symbols from /usr/lib/oracle/11.1.0.1/client/lib/libociicus.so...done. Loaded symbols for /usr/lib/oracle/11.1.0.1/client/lib/libociicus.so Reading symbols from /lib/tls/i686/cmov/libnss_compat.so.2...done. Loaded symbols for /lib/tls/i686/cmov/libnss_compat.so.2 Reading symbols from /lib/tls/i686/cmov/libnss_nis.so.2...done. Loaded symbols for /lib/tls/i686/cmov/libnss_nis.so.2 Reading symbols from /lib/tls/i686/cmov/libnss_files.so.2...done. Loaded symbols for /lib/tls/i686/cmov/libnss_files.so.2 Reading symbols from /usr/lib/ruby/1.8/i486-linux/fcntl.so...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/fcntl.so Reading symbols from /usr/lib/ruby/1.8/i486-linux/digest/md5.so...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/digest/md5.so Reading symbols from /usr/lib/ruby/1.8/i486-linux/digest.so...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/digest.so Reading symbols from /usr/lib/ruby/1.8/i486-linux/racc/cparse.so...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/racc/cparse.so Reading symbols from /usr/lib/ruby/1.8/i486-linux/iconv.so...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/iconv.so Reading symbols from /usr/lib/ruby/1.8/i486-linux/openssl.so...done. Loaded symbols for /usr/lib/ruby/1.8/i486-linux/openssl.so Reading symbols from /usr/lib/i686/cmov/libssl.so.0.9.8...done. Loaded symbols for /usr/lib/i686/cmov/libssl.so.0.9.8 Reading symbols from /usr/lib/i686/cmov/libcrypto.so.0.9.8...done. Loaded symbols for /usr/lib/i686/cmov/libcrypto.so.0.9.8 Reading symbols from /lib/tls/i686/cmov/libnss_dns.so.2...done. Loaded symbols for /lib/tls/i686/cmov/libnss_dns.so.2 Reading symbols from /lib/tls/i686/cmov/libresolv.so.2...done. Loaded symbols for /lib/tls/i686/cmov/libresolv.so.2 Failed to read a valid object file image from memory. 0xb7f45410 in ?? () (gdb) backtrace #0 0xb7f45410 in ?? () #1 0xbf9f0fa8 in ?? () #2 0xbf9f0e90 in ?? () #3 0xbf9f0f10 in ?? () #4 0xb7d99131 in select () from /lib/tls/i686/cmov/libc.so.6 #5 0xb7e99f91 in rb_thread_schedule () from /usr/lib/libruby1.8.so.1.8 #6 0xb7e9ac80 in rb_thread_wait_for () from /usr/lib/libruby1.8.so.1.8 #7 0xb7ee750b in Init_process () from /usr/lib/libruby1.8.so.1.8 #8 0xb7e94653 in rb_provide () from /usr/lib/libruby1.8.so.1.8 #9 0xb7e9bdc2 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #10 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #11 0xb7ea4997 in rb_apply () from /usr/lib/libruby1.8.so.1.8 #12 0xb7ea75be in rb_apply () from /usr/lib/libruby1.8.so.1.8 #13 0xb7ea88bb in rb_apply () from /usr/lib/libruby1.8.so.1.8 #14 0xb7e9463e in rb_provide () from /usr/lib/libruby1.8.so.1.8 #15 0xb7e9bdc2 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #16 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #17 0xb7ea4997 in rb_apply () from /usr/lib/libruby1.8.so.1.8 #18 0xb7ea5d0c in rb_apply () from /usr/lib/libruby1.8.so.1.8 #19 0xb7ea75be in rb_apply () from /usr/lib/libruby1.8.so.1.8 #20 0xb7e9faf4 in ruby_stop () from /usr/lib/libruby1.8.so.1.8 #21 0xb7e94247 in rb_provide () from /usr/lib/libruby1.8.so.1.8 #22 0xb7e9bdc2 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #23 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #24 0xb7e9ce47 in rb_obj_call_init () from /usr/lib/libruby1.8.so.1.8 #25 0xb7e9cea2 in rb_obj_call_init () from /usr/lib/libruby1.8.so.1.8 #26 0xb7e94653 in rb_provide () from /usr/lib/libruby1.8.so.1.8 #27 0xb7e9bdc2 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #28 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #29 0xb7ea486c in rb_apply () from /usr/lib/libruby1.8.so.1.8 #30 0xb7ea5d0c in rb_apply () from /usr/lib/libruby1.8.so.1.8 #31 0xb7ea2e39 in rb_apply () from /usr/lib/libruby1.8.so.1.8 #32 0xb7e9c6e1 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #33 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #34 0xb7ea486c in rb_apply () from /usr/lib/libruby1.8.so.1.8 #35 0xb7e9c6e1 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #36 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #37 0xb7ea486c in rb_apply () from /usr/lib/libruby1.8.so.1.8 #38 0xb7e9c6e1 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #39 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #40 0xb7ea486c in rb_apply () from /usr/lib/libruby1.8.so.1.8 #41 0xb7ea4d31 in rb_apply () from /usr/lib/libruby1.8.so.1.8 #42 0xb7ea9baa in rb_load () from /usr/lib/libruby1.8.so.1.8 #43 0xb7eaa487 in rb_f_require () from /usr/lib/libruby1.8.so.1.8 #44 0xb7e94653 in rb_provide () from /usr/lib/libruby1.8.so.1.8 #45 0xb7e9bdc2 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #46 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 #47 0xb7ea4997 in rb_apply () from /usr/lib/libruby1.8.so.1.8 ---Type to continue, or q to quit--- #48 0xb7eaa618 in rb_load_protect () from /usr/lib/libruby1.8.so.1.8 #49 0xb7eaa662 in ruby_exec () from /usr/lib/libruby1.8.so.1.8 #50 0xb7eaa69f in ruby_run () from /usr/lib/libruby1.8.so.1.8 #51 0x08048612 in main () (gdb) -- Posted via http://www.ruby-forum.com/. From evan at cloudbur.st Tue Apr 1 10:10:03 2008 From: evan at cloudbur.st (Evan Weaver) Date: Tue, 1 Apr 2008 10:10:03 -0400 Subject: [Mongrel] Mongrels stop responding In-Reply-To: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> Message-ID: Oh, I meant a Ruby backtrace: http://eigenclass.org/hiki.rb?ruby+live+process+introspection Evan On Tue, Apr 1, 2008 at 9:55 AM, Michael Kinney wrote: > Evan Weaver wrote: > > Gdb the stuck mongrel and force it to raise a backtrace. > > I had to wait for one of them to hang again. This time, no CLOSE_WAIT. > Also note, I've never used gdb before. I googled up how to attach to a > PID and hit the "backtrace" command. If there's anything else I need to > do, the GDB session is still available to me for a limited time (until > enough people complain about the app being down). > > GNU gdb 6.4.90-debian > Copyright (C) 2006 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for > details. > This GDB was configured as "i486-linux-gnu". > Attaching to process 31635 > Reading symbols from /usr/bin/ruby1.8...(no debugging symbols > found)...done. > Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1". > Reading symbols from /usr/lib/libruby1.8.so.1.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libruby1.8.so.1.8 > Reading symbols from /lib/tls/i686/cmov/libpthread.so.0...(no debugging > symbols found)...done. > [Thread debugging using libthread_db enabled] > [New Thread -1211282784 (LWP 31635)] > [New Thread -1262781520 (LWP 31645)] > Loaded symbols for /lib/tls/i686/cmov/libpthread.so.0 > Reading symbols from /lib/tls/i686/cmov/libdl.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib/tls/i686/cmov/libdl.so.2 > Reading symbols from /lib/tls/i686/cmov/libcrypt.so.1... > (no debugging symbols found)...done. > Loaded symbols for /lib/tls/i686/cmov/libcrypt.so.1 > Reading symbols from /lib/tls/i686/cmov/libm.so.6...(no debugging > symbols found)...done. > Loaded symbols for /lib/tls/i686/cmov/libm.so.6 > Reading symbols from /lib/tls/i686/cmov/libc.so.6...(no debugging > symbols found)...done. > Loaded symbols for /lib/tls/i686/cmov/libc.so.6 > Reading symbols from /lib/ld-linux.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/ld-linux.so.2 > Reading symbols from /usr/lib/ruby/1.8/i486-linux/etc.so... > (no debugging symbols found)...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/etc.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/stringio.so...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/stringio.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/syck.so...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/syck.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/socket.so...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/socket.so > Reading symbols from > /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/lib/http11.so...done. > Loaded symbols for > /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/http11.so > Reading symbols from > /usr/lib/ruby/gems/1.8/gems/fastthread-1.0.1/lib/fastthread.so...done. > Loaded symbols for > /usr/lib/ruby/gems/1.8/gems/fastthread-1.0.1/lib/fastthread.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/zlib.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/zlib.so > Reading symbols from /usr/lib/libz.so.1...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /usr/lib/ruby/1.8/i486-linux/strscan.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/strscan.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/bigdecimal.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/bigdecimal.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/nkf.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/nkf.so > Reading symbols from > /usr/local/lib/site_ruby/1.8/i486-linux/oci8lib.so...done. > Loaded symbols for /usr/local/lib/site_ruby/1.8/i486-linux/oci8lib.so > Reading symbols from > /usr/lib/oracle/11.1.0.1/client/lib/libclntsh.so.11.1...done. > Loaded symbols for /usr/lib/oracle/11.1.0.1/client/lib/libclntsh.so.11.1 > Reading symbols from > /usr/lib/oracle/11.1.0.1/client/lib/libnnz11.so...done. > Loaded symbols for /usr/lib/oracle/11.1.0.1/client/lib/libnnz11.so > Reading symbols from /lib/tls/i686/cmov/libnsl.so.1...done. > Loaded symbols for /lib/tls/i686/cmov/libnsl.so.1 > Reading symbols from /usr/lib/libaio.so.1...done. > Loaded symbols for /usr/lib/libaio.so.1 > Reading symbols from > /usr/lib/oracle/11.1.0.1/client/lib/libociicus.so...done. > Loaded symbols for /usr/lib/oracle/11.1.0.1/client/lib/libociicus.so > Reading symbols from /lib/tls/i686/cmov/libnss_compat.so.2...done. > Loaded symbols for /lib/tls/i686/cmov/libnss_compat.so.2 > Reading symbols from /lib/tls/i686/cmov/libnss_nis.so.2...done. > Loaded symbols for /lib/tls/i686/cmov/libnss_nis.so.2 > Reading symbols from /lib/tls/i686/cmov/libnss_files.so.2...done. > Loaded symbols for /lib/tls/i686/cmov/libnss_files.so.2 > Reading symbols from /usr/lib/ruby/1.8/i486-linux/fcntl.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/fcntl.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/digest/md5.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/digest/md5.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/digest.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/digest.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/racc/cparse.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/racc/cparse.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/iconv.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/iconv.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/openssl.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/openssl.so > Reading symbols from /usr/lib/i686/cmov/libssl.so.0.9.8...done. > Loaded symbols for /usr/lib/i686/cmov/libssl.so.0.9.8 > Reading symbols from /usr/lib/i686/cmov/libcrypto.so.0.9.8...done. > Loaded symbols for /usr/lib/i686/cmov/libcrypto.so.0.9.8 > Reading symbols from /lib/tls/i686/cmov/libnss_dns.so.2...done. > Loaded symbols for /lib/tls/i686/cmov/libnss_dns.so.2 > Reading symbols from /lib/tls/i686/cmov/libresolv.so.2...done. > Loaded symbols for /lib/tls/i686/cmov/libresolv.so.2 > Failed to read a valid object file image from memory. > 0xb7f45410 in ?? () > (gdb) backtrace > #0 0xb7f45410 in ?? () > #1 0xbf9f0fa8 in ?? () > #2 0xbf9f0e90 in ?? () > #3 0xbf9f0f10 in ?? () > #4 0xb7d99131 in select () from /lib/tls/i686/cmov/libc.so.6 > #5 0xb7e99f91 in rb_thread_schedule () from /usr/lib/libruby1.8.so.1.8 > #6 0xb7e9ac80 in rb_thread_wait_for () from /usr/lib/libruby1.8.so.1.8 > #7 0xb7ee750b in Init_process () from /usr/lib/libruby1.8.so.1.8 > #8 0xb7e94653 in rb_provide () from /usr/lib/libruby1.8.so.1.8 > #9 0xb7e9bdc2 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #10 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #11 0xb7ea4997 in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #12 0xb7ea75be in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #13 0xb7ea88bb in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #14 0xb7e9463e in rb_provide () from /usr/lib/libruby1.8.so.1.8 > #15 0xb7e9bdc2 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #16 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #17 0xb7ea4997 in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #18 0xb7ea5d0c in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #19 0xb7ea75be in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #20 0xb7e9faf4 in ruby_stop () from /usr/lib/libruby1.8.so.1.8 > #21 0xb7e94247 in rb_provide () from /usr/lib/libruby1.8.so.1.8 > #22 0xb7e9bdc2 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #23 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #24 0xb7e9ce47 in rb_obj_call_init () from /usr/lib/libruby1.8.so.1.8 > #25 0xb7e9cea2 in rb_obj_call_init () from /usr/lib/libruby1.8.so.1.8 > #26 0xb7e94653 in rb_provide () from /usr/lib/libruby1.8.so.1.8 > #27 0xb7e9bdc2 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #28 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #29 0xb7ea486c in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #30 0xb7ea5d0c in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #31 0xb7ea2e39 in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #32 0xb7e9c6e1 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #33 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #34 0xb7ea486c in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #35 0xb7e9c6e1 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #36 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #37 0xb7ea486c in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #38 0xb7e9c6e1 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #39 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #40 0xb7ea486c in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #41 0xb7ea4d31 in rb_apply () from /usr/lib/libruby1.8.so.1.8 > #42 0xb7ea9baa in rb_load () from /usr/lib/libruby1.8.so.1.8 > #43 0xb7eaa487 in rb_f_require () from /usr/lib/libruby1.8.so.1.8 > #44 0xb7e94653 in rb_provide () from /usr/lib/libruby1.8.so.1.8 > #45 0xb7e9bdc2 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #46 0xb7e9cb38 in rb_iter_break () from /usr/lib/libruby1.8.so.1.8 > #47 0xb7ea4997 in rb_apply () from /usr/lib/libruby1.8.so.1.8 > ---Type to continue, or q to quit--- > #48 0xb7eaa618 in rb_load_protect () from /usr/lib/libruby1.8.so.1.8 > #49 0xb7eaa662 in ruby_exec () from /usr/lib/libruby1.8.so.1.8 > #50 0xb7eaa69f in ruby_run () from /usr/lib/libruby1.8.so.1.8 > #51 0x08048612 in main () > (gdb) > -- > > > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -- Evan Weaver Cloudburst, LLC From lists at ruby-forum.com Tue Apr 1 10:48:35 2008 From: lists at ruby-forum.com (Michael Kinney) Date: Tue, 1 Apr 2008 16:48:35 +0200 Subject: [Mongrel] Mongrels stop responding In-Reply-To: References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> Message-ID: <828e2e088d45810e49a9d170ec6e38d0@ruby-forum.com> Evan Weaver wrote: > Oh, I meant a Ruby backtrace: I believe I've blown this opportunity. I got my gdb environment set up as described in the article, attached to the process, and did "rb_backtrace". All I got out of it was "[Switching to Thread -1211282784 (LWP 31635)] $1 = 32". Then I noticed the article says something about redirecting stdout. I issued that command and then did the backtrace again, but the only thing I got that time was "$3 = 32". In an effort to make sure the ruby debugging commands were working properly, I tried one of the "easier" ones: (gdb) rb_object_counts Program received signal SIGSEGV, Segmentation fault. 0xb7eff6fe in st_lookup () from /usr/lib/libruby1.8.so.1.8 When I exited gdb, the hung mongrel had died. So it seems that any meaningful output was lost because I failed to redirect stdout first...? I checked the log files (where I'd expect stdout to go if it weren't redirected), and there is no additional information there either. Please correct me if I'm wrong, but the next chance I get, it seems the proper sequence of commands goes: gdb ruby PID_of_bad_mongrel (gdb) session-ruby (gdb) redirect_stdout (gdb) rb_backtrace Thanks. -- Posted via http://www.ruby-forum.com/. From lists at ruby-forum.com Tue Apr 1 11:04:59 2008 From: lists at ruby-forum.com (Michael Kinney) Date: Tue, 1 Apr 2008 17:04:59 +0200 Subject: [Mongrel] Mongrels stop responding In-Reply-To: <828e2e088d45810e49a9d170ec6e38d0@ruby-forum.com> References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> <828e2e088d45810e49a9d170ec6e38d0@ruby-forum.com> Message-ID: A second app on the same box has gone belly-up. This app does use mysql, but has all of the appropriate fixes put in place as with the first app. This time I was able to get the backtrace out of the /tmp file: ["/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `loop'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `initialize'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `new'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:128:in `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/command.rb:212:in `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:281"] This app is the beta test copy of our "intranet portal" application. It never has problems on the production server. For reference, the other application is referred to internally as "paperless-mpcs". -- Posted via http://www.ruby-forum.com/. From evan at cloudbur.st Tue Apr 1 11:31:00 2008 From: evan at cloudbur.st (Evan Weaver) Date: Tue, 1 Apr 2008 11:31:00 -0400 Subject: [Mongrel] Mongrels stop responding In-Reply-To: References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> <828e2e088d45810e49a9d170ec6e38d0@ruby-forum.com> Message-ID: Stuck in the sleeper thread... that's odd: 285: $mongrel_sleeper_thread = Thread.new { loop { sleep 1 } } Kirk, any ideas? Evan On Tue, Apr 1, 2008 at 11:04 AM, Michael Kinney wrote: > A second app on the same box has gone belly-up. This app does use mysql, > but has all of the appropriate fixes put in place as with the first app. > > This time I was able to get the backtrace out of the /tmp file: > > ["/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in > `run'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in > `loop'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in > `run'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in > `initialize'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in > `new'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in > `run'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:128:in > `run'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/command.rb:212:in > `run'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:281"] > > This app is the beta test copy of our "intranet portal" application. It > never has problems on the production server. > > For reference, the other application is referred to internally as > "paperless-mpcs". > > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -- Evan Weaver Cloudburst, LLC From cnk at caltech.edu Tue Apr 1 13:37:44 2008 From: cnk at caltech.edu (Cynthia Kiser) Date: Tue, 1 Apr 2008 10:37:44 -0700 Subject: [Mongrel] Mongrels stop responding In-Reply-To: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> Message-ID: <20080401173744.GF16856@sue.caltech.edu> You said this app was not using MySQL. Is it using Oracle? if so, ok. Otherwise, why did your trace wander through the Oracle client libraries? (I spent a lot of time one day trying to trace some Oracle library oddities. The very un-Linux like solution of rebotting the VM fixed things but I am now quite leary of unexplained Oracle detours. There is also a very odd interaction with LDAP and the Oracle LDAP libraries - http://lists.rubyonrails.org/pipermail/rails/2006-April/032583.html.) Quoting Michael Kinney : > Reading symbols from /usr/lib/libz.so.1...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /usr/lib/ruby/1.8/i486-linux/strscan.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/strscan.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/bigdecimal.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/bigdecimal.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/nkf.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/nkf.so > Reading symbols from > /usr/local/lib/site_ruby/1.8/i486-linux/oci8lib.so...done. > Loaded symbols for /usr/local/lib/site_ruby/1.8/i486-linux/oci8lib.so > Reading symbols from > /usr/lib/oracle/11.1.0.1/client/lib/libclntsh.so.11.1...done. > Loaded symbols for /usr/lib/oracle/11.1.0.1/client/lib/libclntsh.so.11.1 > Reading symbols from > /usr/lib/oracle/11.1.0.1/client/lib/libnnz11.so...done. > Loaded symbols for /usr/lib/oracle/11.1.0.1/client/lib/libnnz11.so > Reading symbols from /lib/tls/i686/cmov/libnsl.so.1...done. > Loaded symbols for /lib/tls/i686/cmov/libnsl.so.1 > Reading symbols from /usr/lib/libaio.so.1...done. > Loaded symbols for /usr/lib/libaio.so.1 > Reading symbols from > /usr/lib/oracle/11.1.0.1/client/lib/libociicus.so...done. > Loaded symbols for /usr/lib/oracle/11.1.0.1/client/lib/libociicus.so > Reading symbols from /lib/tls/i686/cmov/libnss_compat.so.2...done. > Loaded symbols for /lib/tls/i686/cmov/libnss_compat.so.2 > Reading symbols from /lib/tls/i686/cmov/libnss_nis.so.2...done. > Loaded symbols for /lib/tls/i686/cmov/libnss_nis.so.2 > Reading symbols from /lib/tls/i686/cmov/libnss_files.so.2...done. > Loaded symbols for /lib/tls/i686/cmov/libnss_files.so.2 > Reading symbols from /usr/lib/ruby/1.8/i486-linux/fcntl.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/fcntl.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/digest/md5.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/digest/md5.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/digest.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/digest.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/racc/cparse.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/racc/cparse.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/iconv.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/iconv.so > Reading symbols from /usr/lib/ruby/1.8/i486-linux/openssl.so...done. > Loaded symbols for /usr/lib/ruby/1.8/i486-linux/openssl.so > Reading symbols from /usr/lib/i686/cmov/libssl.so.0.9.8...done. > Loaded symbols for /usr/lib/i686/cmov/libssl.so.0.9.8 > Reading symbols from /usr/lib/i686/cmov/libcrypto.so.0.9.8...done. > Loaded symbols for /usr/lib/i686/cmov/libcrypto.so.0.9.8 > Reading symbols from /lib/tls/i686/cmov/libnss_dns.so.2...done. > Loaded symbols for /lib/tls/i686/cmov/libnss_dns.so.2 > Reading symbols from /lib/tls/i686/cmov/libresolv.so.2...done. > Loaded symbols for /lib/tls/i686/cmov/libresolv.so.2 > Failed to read a valid object file image from memory. > 0xb7f45410 in ?? () From lists at ruby-forum.com Tue Apr 1 15:12:53 2008 From: lists at ruby-forum.com (Michael Kinney) Date: Tue, 1 Apr 2008 21:12:53 +0200 Subject: [Mongrel] Mongrels stop responding In-Reply-To: <20080401173744.GF16856@sue.caltech.edu> References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> <20080401173744.GF16856@sue.caltech.edu> Message-ID: <50a6dab47ab6beee4590002fd8f49e00@ruby-forum.com> Cynthia Kiser wrote: > You said this app was not using MySQL. Is it using Oracle? if so, > ok. Otherwise, why did your trace wander through the Oracle client > libraries? The paperless-mpcs app does use Oracle. This app is where the gdb backtrace came from. I am waiting for this app to fail again so that I can get a ruby backtrace. The intranet portal app uses MySQL. This is the app that generated the ruby backtrace that shows mongrel "stuck in the sleeper thread". > (I spent a lot of time one day trying to trace some Oracle library > oddities. The very un-Linux like solution of rebotting the VM fixed > things but I am now quite leary of unexplained Oracle detours. Are you suggesting I reboot my server? I can make that happen. I would hate to call that a "fix" though. As you say, very un-Linux-like. > There > is also a very odd interaction with LDAP and the Oracle LDAP libraries > - http://lists.rubyonrails.org/pipermail/rails/2006-April/032583.html.) I don't use Ruby/LDAP on this server yet, but I will in the future. Thanks for pointing that out, you've almost certainly saved me some serious headaches! -- Posted via http://www.ruby-forum.com/. From lists at ruby-forum.com Tue Apr 1 17:45:53 2008 From: lists at ruby-forum.com (Michael Kinney) Date: Tue, 1 Apr 2008 23:45:53 +0200 Subject: [Mongrel] Mongrels stop responding In-Reply-To: References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> Message-ID: Just as I was preparing to leave for the day, another Mongrel stopped working. This time, the paperless-mpcs app. Now, I may still not be doing this gdb thing right, because rb_backtrace didn't generate any output. But then I tried 'eval "caller"' and this is what I got: ["/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `loop'", "/usr/lib/ruby/gems/1. 8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `initialize'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1. 1.4/bin/../lib/mongrel/configurator.rb:285:in `new'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:12 8:in `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/command.rb:212:in `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:281", "/usr/bin/mongrel_rails:16:in `load'", "/u sr/bin/mongrel_rails:16"] Seems to be the same thing that's hanging up the portal app? Thanks in advance... -- Posted via http://www.ruby-forum.com/. From ezmobius at gmail.com Tue Apr 1 17:54:06 2008 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Tue, 1 Apr 2008 14:54:06 -0700 Subject: [Mongrel] Mongrels stop responding In-Reply-To: References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> Message-ID: <16AEA934-8A79-4DCB-BFA1-76127713A7A3@gmail.com> On Apr 1, 2008, at 2:45 PM, Michael Kinney wrote: > Just as I was preparing to leave for the day, another Mongrel stopped > working. This time, the paperless-mpcs app. Now, I may still not be > doing this gdb thing right, because rb_backtrace didn't generate any > output. But then I tried 'eval "caller"' and this is what I got: > > ["/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/ > configurator.rb:285:in > `run'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/ > configurator.rb:285:in > `loop'", "/usr/lib/ruby/gems/1. > 8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `run'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/ > configurator.rb:285:in > `initialize'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1. > 1.4/bin/../lib/mongrel/configurator.rb:285:in `new'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/ > configurator.rb:285:in > `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails: > 12 > 8:in `run'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/ > command.rb:212:in > `run'", > "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:281", > "/usr/bin/mongrel_rails:16:in `load'", "/u > sr/bin/mongrel_rails:16"] > > Seems to be the same thing that's hanging up the portal app? > > Thanks in advance...\ Michael- What version and patch level of ruby are you running? If you are using 1.8.6 you need *at least* patch level 110, any patch levels before that have threading issues and should not be used. Cheers- - Ezra Zygmuntowicz -- Founder & Software Architect -- ezra at engineyard.com -- EngineYard.com From rochkind at jhu.edu Tue Apr 1 17:56:36 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Tue, 01 Apr 2008 17:56:36 -0400 Subject: [Mongrel] Mongrels stop responding In-Reply-To: <16AEA934-8A79-4DCB-BFA1-76127713A7A3@gmail.com> References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> <16AEA934-8A79-4DCB-BFA1-76127713A7A3@gmail.com> Message-ID: <47F2AF94.40908@jhu.edu> I hadn't heard about threading issues. What if I'm still on 1.8.5 (I'm not the one who asked the question, but I do use mongrels and threading)--should I be concerned? Jonathan Ezra Zygmuntowicz wrote: > On Apr 1, 2008, at 2:45 PM, Michael Kinney wrote: > >> Just as I was preparing to leave for the day, another Mongrel stopped >> working. This time, the paperless-mpcs app. Now, I may still not be >> doing this gdb thing right, because rb_backtrace didn't generate any >> output. But then I tried 'eval "caller"' and this is what I got: >> >> ["/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/ >> configurator.rb:285:in >> `run'", >> "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/ >> configurator.rb:285:in >> `loop'", "/usr/lib/ruby/gems/1. >> 8/gems/mongrel-1.1.4/bin/../lib/mongrel/configurator.rb:285:in `run'", >> "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/ >> configurator.rb:285:in >> `initialize'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1. >> 1.4/bin/../lib/mongrel/configurator.rb:285:in `new'", >> "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/ >> configurator.rb:285:in >> `run'", "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails: >> 12 >> 8:in `run'", >> "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/ >> command.rb:212:in >> `run'", >> "/usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:281", >> "/usr/bin/mongrel_rails:16:in `load'", "/u >> sr/bin/mongrel_rails:16"] >> >> Seems to be the same thing that's hanging up the portal app? >> >> Thanks in advance...\ >> > > > Michael- > > What version and patch level of ruby are you running? If you are > using 1.8.6 you need *at least* patch level 110, any patch levels > before that have threading issues and should not be used. > > Cheers- > > - Ezra Zygmuntowicz > -- Founder & Software Architect > -- ezra at engineyard.com > -- EngineYard.com > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From wyhaines at gmail.com Tue Apr 1 18:06:32 2008 From: wyhaines at gmail.com (Kirk Haines) Date: Tue, 1 Apr 2008 16:06:32 -0600 Subject: [Mongrel] Mongrels stop responding In-Reply-To: <47F2AF94.40908@jhu.edu> References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> <16AEA934-8A79-4DCB-BFA1-76127713A7A3@gmail.com> <47F2AF94.40908@jhu.edu> Message-ID: On Tue, Apr 1, 2008 at 3:56 PM, Jonathan Rochkind wrote: > I hadn't heard about threading issues. What if I'm still on 1.8.5 (I'm > not the one who asked the question, but I do use mongrels and > threading)--should I be concerned? There are two issues. When 1.8.5 was out, it was discovered that Array#shift had a stupid bug in it that effectively leaked memory. Mutex uses an Array to keep a queue of threads that are waiting for a mutex, and it uses Array#push and Array#shift to put things into the queue and take them out of it.. There was a crude workaround, which was to monkey patch Mutex to use Array#unshift and Array#pop, instead, which didn't leak. However, an intrepid Rubyist took this opportunity to reimplement the Ruby threading primitives as a C extension instead of pure Ruby. Thus, the fastthread library was born. Fastthread was accepted into Ruby 1.8.6 to replace the pure ruby primitives, but the first several releases/patch levels had bugs. Mongrel will use fastthread, so Ruby 1.8.5 w/ fastthread is a perfectly fine platform. If you are on Ruby 1.8.6, though, you need to be on p110 or higher, as that patch level has both properly working threading primitives, and has Array#shift fixed so it doesn't leak. Kirk Haines From cnk at caltech.edu Tue Apr 1 18:32:51 2008 From: cnk at caltech.edu (Cynthia Kiser) Date: Tue, 1 Apr 2008 15:32:51 -0700 Subject: [Mongrel] Mongrels stop responding In-Reply-To: <50a6dab47ab6beee4590002fd8f49e00@ruby-forum.com> References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> <20080401173744.GF16856@sue.caltech.edu> <50a6dab47ab6beee4590002fd8f49e00@ruby-forum.com> Message-ID: <20080401223251.GD16856@sue.caltech.edu> Quoting Michael Kinney : > > (I spent a lot of time one day trying to trace some Oracle library > > oddities. The very un-Linux like solution of rebotting the VM fixed > > things but I am now quite leary of unexplained Oracle detours. > > Are you suggesting I reboot my server? I can make that happen. I would > hate to call that a "fix" though. As you say, very un-Linux-like. No I am not suggesting a server reboot if you can restart your mongrel otherwise. I just mentioned it because we didn't even think about rebooting the machine just because we couldn't restart our mongrels. It was a couple of hours before we tried that irrational - but immediately successful - strategy. -- Cynthia Kiser From lists at ruby-forum.com Tue Apr 1 22:28:28 2008 From: lists at ruby-forum.com (Michael Kinney) Date: Wed, 2 Apr 2008 04:28:28 +0200 Subject: [Mongrel] Mongrels stop responding In-Reply-To: <16AEA934-8A79-4DCB-BFA1-76127713A7A3@gmail.com> References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> <16AEA934-8A79-4DCB-BFA1-76127713A7A3@gmail.com> Message-ID: <6dc4e2b55d14d2b43af8061127f7f75a@ruby-forum.com> Ezra Zygmuntowicz wrote: > What version and patch level of ruby are you running? If you are > using 1.8.6 you need *at least* patch level 110, any patch levels > before that have threading issues and should not be used. ruby --version reports 1.8.5 (2006-08-25) That said, this is a Debian 4.0 system so they may have applied backported security patches. I don't know how to get it to report the patch level. I do also have fastthread 1.0.1 and cgi_multipart_eof_fix 2.5.0 installed. -- Posted via http://www.ruby-forum.com/. From mike.engelhart at gmail.com Wed Apr 2 11:14:59 2008 From: mike.engelhart at gmail.com (Michael Engelhart) Date: Wed, 2 Apr 2008 11:14:59 -0400 Subject: [Mongrel] X_FORWARDED_PROTO issues Message-ID: <42642c790804020814i213cbc26gae9cd54b719772d0@mail.gmail.com> Hi all - I'm having some behavior that i can't track down and I'm not sure if this is an Apache problem, a mongrel problem or neither. Anyway - in my production setting I have two Apache 2.2 proxy servers pointing to two servers running mongrel instances. In my SSL.conf file I have this: RequestHeader set X_FORWARDED_PROTO 'https' It seems that if I reboot apache or reboot the mongrel instances without rebooting both in sequence that this proto value gets corrupted and request.ssl? in rails stops working properly. It's as if request.ssl? is coming in incorrectly. Just wondering if anyone has seen this behavior before. THanks Mike From lists at ruby-forum.com Wed Apr 2 18:07:33 2008 From: lists at ruby-forum.com (Michael Kinney) Date: Thu, 3 Apr 2008 00:07:33 +0200 Subject: [Mongrel] Mongrels stop responding In-Reply-To: <6dc4e2b55d14d2b43af8061127f7f75a@ruby-forum.com> References: <22292c3eafee3a5a40676025371bf269@ruby-forum.com> <16AEA934-8A79-4DCB-BFA1-76127713A7A3@gmail.com> <6dc4e2b55d14d2b43af8061127f7f75a@ruby-forum.com> Message-ID: <79e80b429f19821883edf9e4979a8366@ruby-forum.com> I looked up the Debian package info for the ruby1.8 deb and it looks like they've applied a series of numbered patches up to 164. -- Posted via http://www.ruby-forum.com/. From lists at ruby-forum.com Sun Apr 6 04:06:24 2008 From: lists at ruby-forum.com (Luca Reghellin) Date: Sun, 6 Apr 2008 10:06:24 +0200 Subject: [Mongrel] no more mongrel on leopard! Message-ID: <1dfb1fa6680365dfe5086c01161af5ba@ruby-forum.com> Hi! I've got a problem: just changed my mac, now I run leopard. Just updated original gems to rails 2.0.2 and cleaned up each of them. But, when I run script/server, there's no more mongrel. It starts up webrick! I want mongrel, since it's better for debug. How can I substitute it? Below is a sudo gem list: actionmailer (2.0.2) actionpack (2.0.2) actionwebservice (1.2.6) activerecord (2.0.2) activeresource (2.0.2) activesupport (2.0.2) dnssd (0.6.0) fcgi (0.8.7) gem_plugin (0.2.3) hpricot (0.6) libxml-ruby (0.3.8.4) mongrel (1.1.4) needle (1.3.0) net-sftp (1.1.1) net-ssh (1.1.2) rails (2.0.2) rake (0.8.1) RedCloth (3.0.4) ruby-openid (2.0.4) ruby-yadis (0.3.4) rubygems-update (1.1.0) rubynode (0.1.3) sources (0.0.1) sqlite3-ruby (1.2.1) termios (0.9.4) -- Posted via http://www.ruby-forum.com/. From diwadm at gmail.com Mon Apr 7 02:09:42 2008 From: diwadm at gmail.com (Diwa del Mundo) Date: Mon, 7 Apr 2008 14:09:42 +0800 Subject: [Mongrel] no more mongrel on leopard! In-Reply-To: <1dfb1fa6680365dfe5086c01161af5ba@ruby-forum.com> References: <1dfb1fa6680365dfe5086c01161af5ba@ruby-forum.com> Message-ID: <75b2d2c90804062309g5c52ad8dl283cadec811bafc@mail.gmail.com> Why not run "mongrel_rails start"? On Sun, Apr 6, 2008 at 4:06 PM, Luca Reghellin wrote: > > Hi! I've got a problem: just changed my mac, now I run leopard. Just > updated original gems to rails 2.0.2 and cleaned up each of them. But, > when I run script/server, there's no more mongrel. It starts up webrick! > I want mongrel, since it's better for debug. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080407/a61cb3b3/attachment.html From lists at ruby-forum.com Mon Apr 7 04:14:42 2008 From: lists at ruby-forum.com (Luca Reghellin) Date: Mon, 7 Apr 2008 10:14:42 +0200 Subject: [Mongrel] no more mongrel on leopard! In-Reply-To: <75b2d2c90804062309g5c52ad8dl283cadec811bafc@mail.gmail.com> References: <1dfb1fa6680365dfe5086c01161af5ba@ruby-forum.com> <75b2d2c90804062309g5c52ad8dl283cadec811bafc@mail.gmail.com> Message-ID: Ok, I've done it, but it tell me this: `report_activate_error': Could not find RubyGem daemons (>= 1.0.3) (Gem::LoadError) from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems.rb:311:in `activate' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems.rb:337:in `activate' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems.rb:336:in `each' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems.rb:336:in `activate' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems.rb:65:in `active_gem_with_options' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems.rb:50:in `gem' from /usr/bin/mongrel_rails:18 :( Diwa del Mundo wrote: > Why not run "mongrel_rails start"? -- Posted via http://www.ruby-forum.com/. From diwadm at gmail.com Mon Apr 7 04:24:10 2008 From: diwadm at gmail.com (Diwa del Mundo) Date: Mon, 7 Apr 2008 16:24:10 +0800 Subject: [Mongrel] no more mongrel on leopard! In-Reply-To: References: <1dfb1fa6680365dfe5086c01161af5ba@ruby-forum.com> <75b2d2c90804062309g5c52ad8dl283cadec811bafc@mail.gmail.com> Message-ID: <75b2d2c90804070124l54a8135gc4ae7422a5e55e5b@mail.gmail.com> Hey, Try installing the daemons gem: sudo gem install daemons On Mon, Apr 7, 2008 at 4:14 PM, Luca Reghellin wrote: > > Ok, I've done it, but it tell me this: > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080407/040d1508/attachment.html From lists at ruby-forum.com Mon Apr 7 04:39:11 2008 From: lists at ruby-forum.com (Luca Reghellin) Date: Mon, 7 Apr 2008 10:39:11 +0200 Subject: [Mongrel] no more mongrel on leopard! In-Reply-To: <75b2d2c90804070124l54a8135gc4ae7422a5e55e5b@mail.gmail.com> References: <1dfb1fa6680365dfe5086c01161af5ba@ruby-forum.com> <75b2d2c90804062309g5c52ad8dl283cadec811bafc@mail.gmail.com> <75b2d2c90804070124l54a8135gc4ae7422a5e55e5b@mail.gmail.com> Message-ID: Ok, thank you! I needed to install fastthread and cgi_multipart_eof_fix as well. Now I don't even need anymore to do 'mongrel_rails start': mongrel start soon on script/server! :) Thank you a lot! -- Posted via http://www.ruby-forum.com/. From lists at ruby-forum.com Mon Apr 7 09:58:23 2008 From: lists at ruby-forum.com (Michael Kinney) Date: Mon, 7 Apr 2008 15:58:23 +0200 Subject: [Mongrel] Mongrels stop responding In-Reply-To: References: Message-ID: I think I may have it: The "portal" application has been well behaved since I installed the mysql gem v2.7. It had been using the AR internal adapter. The "paperless" application needed an earlier version of the composite_primary_keys gem. Since downgrading, the app has not hung up. Thanks. -- Posted via http://www.ruby-forum.com/. From public at misuse.org Tue Apr 8 02:14:47 2008 From: public at misuse.org (Steve Midgley) Date: Mon, 07 Apr 2008 23:14:47 -0700 Subject: [Mongrel] Mongrel performance study Message-ID: <20080408061941.190341858632@rubyforge.org> Hi mongrelians, I found myself with an unloaded full production server recently which presented an opportunity to run some performance tests against Mongrel and nginx. I thought this list might have some interest in the results of my tinkering! I actually ran two sets of studies, and Wayne Seguin kindly provided a lot of guidance and support in getting them running. Thanks Wayne! Also, EngineYard provided the hardware for the test. The first test looked at Mongrels running with nginx fair proxy vs. without it: http://www.misuse.org/science/2008/04/07/thin-ruby-on-rails-nginx-fair-proxy-performance-testing/ The second test looked at Mongrels vs Thin as the application handler for Rails: http://www.misuse.org/science/2008/04/07/thin-vs-mongrel-a-ruby-on-rails-performance-shootout/ Overall the results were not so surprising but at least there's a little more hard data for people to look at while making decisions about this stuff. Summary 1: Fair proxy can boost performance significantly on sites that have a mix of 95% fast (<1s) and 5% slow-ish (4s) pages. Summary 2: Unix sockets are probably 4% faster than IP sockets (and therefore Thin is about 4% faster than Mongrel). One thing I did notice was that under heavy load nginx fair proxy caused some weird "clustering" of requests - the graphs (unpublished) show it pretty clearly while the statistics (published above) do not. So while fair proxy looks pretty good overall, if you are heavily loaded, I might recommend from the study that you do not use fair proxy, at least the version I tested (current stable release). Your comments and feedback are welcome. I hope the studies' information and quality are usable to you. It's a very small contribution back to this great community and the people who have contributed so much more. Steve From zedshaw at zedshaw.com Tue Apr 8 03:20:38 2008 From: zedshaw at zedshaw.com (Zed A. Shaw) Date: Tue, 8 Apr 2008 03:20:38 -0400 Subject: [Mongrel] Mongrel performance study In-Reply-To: <20080408061941.190341858632@rubyforge.org> References: <20080408061941.190341858632@rubyforge.org> Message-ID: <20080408032038.06f1e5c4.zedshaw@zedshaw.com> On Mon, 07 Apr 2008 23:14:47 -0700 Steve Midgley wrote: > Hi mongrelians, > > http://www.misuse.org/science/2008/04/07/thin-ruby-on-rails-nginx-fair-proxy-performance-testing/ Yay! You mentioned standard deviations. Bad! You didn't give any. That'd help people figure out if the 4% is even worth it. > The second test looked at Mongrels vs Thin as the application handler > for Rails: > > http://www.misuse.org/science/2008/04/07/thin-vs-mongrel-a-ruby-on-rails-performance-shootout/ "Thin is the newest incarnation of Zed Shaw?s (and now community managed) masterwork Mongrel." That statement isn't correct. Please don't make it since I don't think the author of Thin or the authors of Mongrel would say that. Oh, well they'd say my stuff is a masterwork, just not that Thin is an incarnation of Mongrel. :-) > Overall the results were not so surprising but at least there's a > little more hard data for people to look at while making decisions > about this stuff. > > Summary 1: Fair proxy can boost performance significantly on sites that > have a mix of 95% fast (<1s) and 5% slow-ish (4s) pages. > > Summary 2: Unix sockets are probably 4% faster than IP sockets (and > therefore Thin is about 4% faster than Mongrel). I don't think it's unix sockets vs. ip sockets. These days those aren't really that much faster than a localhost connection thanks to advancements in performance for internal IPC. Still, 4% increase for all that work done in Thin is kind of lame. These kinds of results keep bringing me back to the real problem: Ruby. It's not what IO loop you use, or threading model, or socket type, but more just that Ruby is slow and it's processing sucks. Another comment I'd make is, why not also release the methodology you gave. I'm actually working on a presentation for RuPy so if you'd like to hack on a repeatable study with an automated report let me know. Could be a good test for what I'm writing for the conference. -- Zed A. Shaw - Hate: http://savingtheinternetwithhate.com/ - Good: http://www.zedshaw.com/ - Evil: http://yearofevil.com/ From jftucker at gmail.com Tue Apr 8 08:18:43 2008 From: jftucker at gmail.com (James Tucker) Date: Tue, 8 Apr 2008 13:18:43 +0100 Subject: [Mongrel] Mongrel performance study In-Reply-To: <20080408032038.06f1e5c4.zedshaw@zedshaw.com> References: <20080408061941.190341858632@rubyforge.org> <20080408032038.06f1e5c4.zedshaw@zedshaw.com> Message-ID: <01071344-410C-480D-9C08-A9C4294BF515@gmail.com> On 8 Apr 2008, at 08:20, Zed A. Shaw wrote: > > I don't think it's unix sockets vs. ip sockets. These days those > aren't really that much faster than a localhost connection thanks to > advancements in performance for internal IPC. > > Still, 4% increase for all that work done in Thin is kind of lame. > These kinds of results keep bringing me back to the real problem: > Ruby. It's not what IO loop you use, or threading model, or socket > type, but more just that Ruby is slow and it's processing sucks. I believe this is all correct, however there are other advantages to unix sockets for certain setups. They're easier to secure (this is not always relevant, but it can be). In some scenarios they can be easier to manage too (you can glob them, it's harder to 'glob' using netstat). I know that was a discussion about performance, this is just my $0.02 :-) From ry at tinyclouds.org Tue Apr 8 08:56:38 2008 From: ry at tinyclouds.org (ry dahl) Date: Tue, 8 Apr 2008 14:56:38 +0200 Subject: [Mongrel] Mongrel performance study In-Reply-To: <20080408032038.06f1e5c4.zedshaw@zedshaw.com> References: <20080408061941.190341858632@rubyforge.org> <20080408032038.06f1e5c4.zedshaw@zedshaw.com> Message-ID: <3ae7f4480804080556r1eedc5fag53c2358e615bed94@mail.gmail.com> > I'm actually working on a presentation for RuPy so if you'd like > to hack on a repeatable study with an automated report let me know. ebb includes four different benchmarks that i often use to measure against thin, mongrel, fastcgi, and evented mongrel. http://github.com/ry/ebb/tree/master/benchmark but i will present these at RuPy too! ry From public at misuse.org Tue Apr 8 17:36:16 2008 From: public at misuse.org (Steve Midgley) Date: Tue, 08 Apr 2008 14:36:16 -0700 Subject: [Mongrel] Mongrel performance study Message-ID: <20080408213621.7AFDD18585D2@rubyforge.org> >On Mon, 07 Apr 2008 23:14:47 -0700 > > > > > http://www.misuse.org/science/2008/04/07/thin-ruby-on-rails-nginx-fair-proxy-performance-testing/ > >Yay! You mentioned standard deviations. Bad! You didn't give any. >That'd help people figure out if the 4% is even worth it. I did! I did! The raw data (including std dev's) were linked off the main post. That link is: http://www.misuse.org/downloads/thin_mongrel_test_results.html - I thought that level of info on the main post would be too distracting/confusing for many readers.. I tried to be meticulous in collecting these data, and Zed, for every std dev cut and paste out of JMeter I thought off you. :) >[snip] >"Thin is the newest incarnation of Zed Shaw's (and now community >managed) masterwork Mongrel." > >That statement isn't correct. Please don't make it since I don't >think >the author of Thin or the authors of Mongrel would say that. Oh, well >they'd say my stuff is a masterwork, just not that Thin is an >incarnation of Mongrel. :-) Ok - that's fixed. I said that b/c I thought Thin is using your/mongrel's ragel http states. But you know (way) more than me on this. So I changed the language. >[snip] >I don't think it's unix sockets vs. ip sockets. These days those >aren't really that much faster than a localhost connection thanks to >advancements in performance for internal IPC. The Thin author Marc said the same thing. I'll adjust that in the article soon. >[snip] >Another comment I'd make is, why not also release the methodology you >gave. I'm actually working on a presentation for RuPy so if you'd >like >to hack on a repeatable study with an automated report let me know. >Could be a good test for what I'm writing for the conference. Thanks for the encouragement. I just published my methodology, which was the same for both tests, at the end of the second report: http://www.misuse.org/science/2008/04/07/thin-vs-mongrel-a-ruby-on-rails-performance-shootout/ Feedback and criticism are welcome on that methodology. I have done a fair bit of performance testing in the past, but not in such a general sense as comparing two different application servers, so I may have done something wrong. I'd also be open to performing more tests using this methodology and codebase, but I don't have access to an unloaded high performance webserver anymore. I could probably talk Ezra or Wayne into loaning me one though. :) Drop me a line here or personally if that sounds useful. Steve From lists at ruby-forum.com Tue Apr 15 13:33:16 2008 From: lists at ruby-forum.com (Greg Lazarev) Date: Tue, 15 Apr 2008 19:33:16 +0200 Subject: [Mongrel] General Mongrel Question Message-ID: <17ccd3de77903c3d172bd51ba877368e@ruby-forum.com> Could someone please clarify for me what is mongrel cluster vs. proxy vs. all that other stuff you use with mongrel? I'm basically confused why I can't simply run one mongrel server and be happy. Why do I need a cluster? Why can't I just run: mongrel_rails start -n 3 wouldn't that mean I have 3 mongrel servers running and they can handle more load? Or do I still need something in front of these 3 servers to distribute requests to them. Why do people use Apache infront of Mongrel? So what about Pound? Basically I'm very confused with all these options out there to run in production environment using Mongrel. Please help me and explain to me why all this is needed. Thank you in advance. -- Posted via http://www.ruby-forum.com/. From wyhaines at gmail.com Tue Apr 15 14:05:25 2008 From: wyhaines at gmail.com (Kirk Haines) Date: Tue, 15 Apr 2008 12:05:25 -0600 Subject: [Mongrel] General Mongrel Question In-Reply-To: <17ccd3de77903c3d172bd51ba877368e@ruby-forum.com> References: <17ccd3de77903c3d172bd51ba877368e@ruby-forum.com> Message-ID: On Tue, Apr 15, 2008 at 11:33 AM, Greg Lazarev wrote: > Why do I need a cluster? Why can't I just run: > > mongrel_rails start -n 3 That starts 3 mongrel instances, each listening on a different port. > wouldn't that mean I have 3 mongrel servers running and they can handle > more load? Or do I still need something in front of these 3 servers to > distribute requests to them. Why do people use Apache infront of > Mongrel? So what about Pound? People put something in front of the mongrels because they need a way to balance the load, which is all coming into one ip/port, across multiple processes, each of which are listening on a separate port. This mechanism can be a web server with proxying capabilities, like Apache or nginx, a proxy with web serving capabilities, like Swiftiply, or a straight proxy like Pound or HAProxy. Kirk Haines From lists at ruby-forum.com Tue Apr 15 15:14:27 2008 From: lists at ruby-forum.com (Greg Lazarev) Date: Tue, 15 Apr 2008 21:14:27 +0200 Subject: [Mongrel] General Mongrel Question In-Reply-To: References: <17ccd3de77903c3d172bd51ba877368e@ruby-forum.com> Message-ID: Then what is the point behind mongrel_cluster? -- Posted via http://www.ruby-forum.com/. From wyhaines at gmail.com Tue Apr 15 15:45:36 2008 From: wyhaines at gmail.com (Kirk Haines) Date: Tue, 15 Apr 2008 13:45:36 -0600 Subject: [Mongrel] General Mongrel Question In-Reply-To: References: <17ccd3de77903c3d172bd51ba877368e@ruby-forum.com> Message-ID: On Tue, Apr 15, 2008 at 1:14 PM, Greg Lazarev wrote: > Then what is the point behind mongrel_cluster? It is an attempt to make it simpler to manage the mongrel processes. Kirk Haines From koji.lin at gmail.com Thu Apr 17 05:20:41 2008 From: koji.lin at gmail.com (koji Lin) Date: Thu, 17 Apr 2008 17:20:41 +0800 Subject: [Mongrel] Error log, when using apache2.2.x modproxy with mongrel Message-ID: <747e82370804170220p5f27e67aja72fc90a8c1362c5@mail.gmail.com> Hi all im new to use mongrel, my mongrel version is 1.1.4 and apache is 2.2.8 when i using apapche+mod_proxy+mongrel, there are some error in my log file and make apache 502 prxoy error. ex: proxy: error reading status line from remote server localhost proxy: Error reading from remote server returned by /test/show But it works well if i access mongrel directly. and the mongrel.log will have error like below: then i try to put the REQUEST DATA into test_http11.rb, looks like "Connection: Keep-Alive\r\nGET /test/show HTTP/1.1\r\n " makes it confuse. REQUEST DATA: "GET /test/show HTTP/1.1\r\nHost: www.testkoji.com\r\nUser-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: zh-tw,en-us;q=0.7,en;q=0.3\r\nAccept-Encoding: gzip,deflate\r\nAccept-Charset: Big5,utf-8;q=0.7,*;q=0.7\r\nX-Forwarded-For: 127.0.0.1\r\nX-Forwarded-Host: www.testkoji.com\r\nX-Forwarded-Server: www.testkoji.com\r\nConnection: Keep-Alive\r\nGET /test/show HTTP/1.1\r\nHost: www.testkoji.com\r\nUser-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: zh-tw,en-us;q=0.7,en;q=0.3\r\nAccept-Encoding: gzip,deflate\r\nAccept-Charset: Big5,utf-8;q=0.7,*;q=0.7\r\nX-Forwarded-For: 127.0.0.1\r\nX-Forwarded-Host: www.testkoji.com\r\nX-Forwarded-Server: www.testkoji.com\r\nConnection: Keep-Alive\r\n\r\n" --- PARAMS: {"HTTP_X_FORWARDED_HOST"=>"www.testkoji.com", "HTTP_ACCEPT_ENCODING"=>"gzip,deflate", "HTTP_USER_AGENT"=>"Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5", "HTTP_ACCEPT_LANGUAGE"=>"zh-tw,en-us;q=0.7,en;q=0.3", "HTTP_HOST"=>"www.testkoji.com", "REQUEST_PATH"=>"/test/show", "HTTP_ACCEPT_CHARSET"=>"Big5,utf-8;q=0.7,*;q=0.7", "HTTP_VERSION"=>"HTTP/1.1", "HTTP_X_FORWARDED_SERVER"=>"www.testkoji.com", "REQUEST_URI"=>"/test/show", "HTTP_X_FORWARDED_FOR"=>"127.0.0.1", "HTTP_ACCEPT"=>"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "HTTP_CONNECTION"=>"Keep-Alive", "REQUEST_METHOD"=>"GET"} --- -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080417/7232a034/attachment.html From _ at whats-your.name Thu Apr 17 10:31:09 2008 From: _ at whats-your.name (cdr) Date: Thu, 17 Apr 2008 10:31:09 -0400 Subject: [Mongrel] fastthread only needed on ruby <=1.8 Message-ID: <20080417143109.GA3200@m> hey.. havent learned gemspec stuff yet, maybe someone who already knows can make the fastthread dep optional? its just kind of annoying, since 'gem install mongrel' always breaks on 1.9, even tho mongrel works fine, plus 'gem install mongrel' doesnt even leave a local gem around to manually hack/install plus there doesnt appear to be a 'gem fetch mongrel' plus mongrel.org is some domain squatter peddling links to dog supplies so the bottom line is the user has to google for mongrel, dl a tarball, install manually all cuz of some dep that isnt really a dep cheers From luislavena at gmail.com Thu Apr 17 11:18:49 2008 From: luislavena at gmail.com (Luis Lavena) Date: Thu, 17 Apr 2008 12:18:49 -0300 Subject: [Mongrel] fastthread only needed on ruby <=1.8 In-Reply-To: <20080417143109.GA3200@m> References: <20080417143109.GA3200@m> Message-ID: <71166b3b0804170818t76fefa4fpc3380227ea6d737e@mail.gmail.com> On Thu, Apr 17, 2008 at 11:31 AM, cdr <_ at whats-your.name> wrote: > hey.. havent learned gemspec stuff yet, maybe someone who already knows can make the fastthread dep optional? > fastthread is required for ruby version <= 1.8.5, which is most stable one distributed across servers "in the wild" 1.8.6 with patchlevel higher than 36 ships with it, but is not widely installed. Latest code in stable branch make it optional. > its just kind of annoying, since 'gem install mongrel' always breaks on 1.9, even tho mongrel works fine, > Err, correction: if it compiles doesn't meant it works properly. You need to run the tests to see what it working and what isn't. > plus 'gem install mongrel' doesnt even leave a local gem around to manually hack/install Can you rephrase? Maybe what you need is grab the sources from repository (svn) and hack yourself the need patches. > plus there doesnt appear to be a 'gem fetch mongrel' gem fetch is a RubyGems functionality, not mongrels. > plus mongrel.org is some domain squatter peddling links to dog supplies funny, you need to take a lok at this book then: http://rubyurl.com/EQlo > so the bottom line is the user has to google for mongrel, dl a tarball, install manually > Use SVN, is not powerful as Git (and all the git lovers) but works. Patches are welcome > all cuz of some dep that isnt really a dep Is just me, my bad english or you don't spell english properly? -- Luis Lavena Multimedia systems - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams From _ at whats-your.name Thu Apr 17 11:21:42 2008 From: _ at whats-your.name (cdr) Date: Thu, 17 Apr 2008 11:21:42 -0400 Subject: [Mongrel] [patch] RSTRING fixes for 1.9 Message-ID: <20080417152141.GB3200@m> doing what ebb did. change RSTRING(str)->ptr and RSTRING(str)->len to RSTRING_PTR(str) and RSTRING_LEN(str) and define a backwards-compatible macro.. i dont think i had to fix this before. maybe thin or something already had installed a working http11.so maybe the mongrel parser should be its own package, since so much stuff uses it w/o the rest of mongrel? side note. mongrel.rubyforge trac doesnt use the same accts as toplevel rubyforge? (why im mailing this) best, C -------------- next part -------------- --- mongrel-1.1.4/ext/http11/http11.c 2008-02-23 14:18:08.000000000 -0600 +++ mongrel/ext/http11/http11.c 2008-04-17 10:10:07.000000000 -0500 @@ -9,6 +9,11 @@ #include "http11_parser.h" #include +#ifndef RSTRING_PTR +# define RSTRING_PTR(s) (RSTRING(s)->ptr) +# define RSTRING_LEN(s) (RSTRING(s)->len) +#endif + static VALUE mMongrel; static VALUE cHttpParser; static VALUE eHttpParserError; @@ -74,7 +79,7 @@ f = rb_str_dup(global_http_prefix); f = rb_str_buf_cat(f, field, flen); - for(ch = RSTRING(f)->ptr, end = ch + RSTRING(f)->len; ch < end; ch++) { + for(ch = RSTRING_PTR(f), end = ch + RSTRING_LEN(f); ch < end; ch++) { if(*ch == '-') { *ch = '_'; } else { @@ -169,12 +174,12 @@ rb_hash_aset(req, global_gateway_interface, global_gateway_interface_value); if((temp = rb_hash_aref(req, global_http_host)) != Qnil) { /* ruby better close strings off with a '\0' dammit */ - colon = strchr(RSTRING(temp)->ptr, ':'); + colon = strchr(RSTRING_PTR(temp), ':'); if(colon != NULL) { - rb_hash_aset(req, global_server_name, rb_str_substr(temp, 0, colon - RSTRING(temp)->ptr)); + rb_hash_aset(req, global_server_name, rb_str_substr(temp, 0, colon - RSTRING_PTR(temp))); rb_hash_aset(req, global_server_port, - rb_str_substr(temp, colon - RSTRING(temp)->ptr+1, - RSTRING(temp)->len)); + rb_str_substr(temp, colon - RSTRING_PTR(temp)+1, + RSTRING_LEN(temp))); } else { rb_hash_aset(req, global_server_name, temp); rb_hash_aset(req, global_server_port, global_port_80); @@ -295,8 +300,8 @@ DATA_GET(self, http_parser, http); from = FIX2INT(start); - dptr = RSTRING(data)->ptr; - dlen = RSTRING(data)->len; + dptr = RSTRING_PTR(data); + dlen = RSTRING_LEN(data); if(from >= dlen) { rb_raise(eHttpParserError, "Requested start is after data buffer end."); From luislavena at gmail.com Thu Apr 17 11:31:29 2008 From: luislavena at gmail.com (Luis Lavena) Date: Thu, 17 Apr 2008 12:31:29 -0300 Subject: [Mongrel] [patch] RSTRING fixes for 1.9 In-Reply-To: <20080417152141.GB3200@m> References: <20080417152141.GB3200@m> Message-ID: <71166b3b0804170831w3eebcecap37f3ea79c227c583@mail.gmail.com> On Thu, Apr 17, 2008 at 12:21 PM, cdr <_ at whats-your.name> wrote: > doing what ebb did. change RSTRING(str)->ptr and RSTRING(str)->len to RSTRING_PTR(str) and RSTRING_LEN(str) and define a backwards-compatible macro.. > > i dont think i had to fix this before. maybe thin or something already had installed a working http11.so > This is already fixed in Mongrel http://mongrel.rubyforge.org/changeset/895 > maybe the mongrel parser should be its own package, since so much stuff uses it w/o the rest of mongrel? Mongrel HTTP parser is core of mongrel, it shouldn't be splitted. > side note. mongrel.rubyforge trac doesnt use the same accts as toplevel rubyforge? (why im mailing this) No, RubyForge do not offer Trac functionality, after all RubyForge is GForge / SourceForge software (PHP) and not Python. Mongrel Trac is not hosted at rubyforge, even so, trac and GForge software cannot share authorization information (well, not without tweaking). -- Luis Lavena Multimedia systems - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams From mental at rydia.net Thu Apr 17 16:22:03 2008 From: mental at rydia.net (MenTaLguY) Date: Thu, 17 Apr 2008 13:22:03 -0700 Subject: [Mongrel] fastthread only needed on ruby In-Reply-To: <71166b3b0804170818t76fefa4fpc3380227ea6d737e@mail.gmail.com> References: <71166b3b0804170818t76fefa4fpc3380227ea6d737e@mail.gmail.com> Message-ID: <36d0d69a22a5af23cd4cf5e2b5480bae@localhost> On Thu, 17 Apr 2008 12:18:49 -0300, "Luis Lavena" wrote: > fastthread is required for ruby version <= 1.8.5, which is most stable > one distributed across servers "in the wild" 1.8.6 with patchlevel > higher than 36 ships with it, but is not widely installed. There are problems with 1.8.6's version of fastthread until p111 or so. -mental From normalperson at yhbt.net Thu Apr 17 19:34:01 2008 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 17 Apr 2008 16:34:01 -0700 Subject: [Mongrel] qrp followup, deployment results Message-ID: <20080417233401.GB12363@untitled> I posted about qrp (http://qrp.rubyforge.org/) many weeks ago, but only deployed it to a live site a few weeks ago (after a few bug fixes leading up to qrp v0.4.0). qrp was deployed late on 2008-03-27 to roughly half our servers, and then fully deployed on 2008-03-28. So far, the results have been fairly decent (see below). The only change I needed to make to Mongrel was the following patch to disable the excessive logging since I disabled concurrency in Mongrel: --- a/mongrel.rb 2008-03-03 16:42:04.000000000 -0800 +++ b/mongrel.rb 2008-04-17 15:30:57.313952784 -0700 @@ -210,7 +210,7 @@ # after the reap is done. It only runs if there are workers to reap. def reap_dead_workers(reason='unknown') if @workers.list.length > 0 - STDERR.puts "#{Time.now}: Reaping #{@workers.list.length} threads for slow workers because of '#{reason}'" + #STDERR.puts "#{Time.now}: Reaping #{@workers.list.length} threads for slow workers because of '#{reason}'" error_msg = "Mongrel timed out this thread: #{reason}" mark = Time.now @workers.list.each do |worker| @@ -278,7 +278,7 @@ worker_list = @workers.list if worker_list.length >= @num_processors - STDERR.puts "Server overloaded with #{worker_list.length} processors (#@num_processors max). Dropping connection." + #STDERR.puts "Server overloaded with #{worker_list.length} processors (#@num_processors max). Dropping connection." client.close rescue nil reap_dead_workers("max processors") else As far as I know, we're the only Rails site running qrp in our configuration, but it should be safe now that we're doing it ;) Results: While I don't have hard numbers for average response time and standard deviation, they have not changed much since qrp was deployed. However, our metric for requests taking over 10 seconds has improved greatly since qrp was deployed[1]. The Date is actually shifted by one day (so the report that I received on 2008-03-02 was actually for the previous days traffic). While a few hundredths of one percent doesn't sound like a lot, that's still a reasonable amount of unhappy users that get bogged down. Date | % of requests taking >10s, (0-100) 2008-03-01 | 0.1192 | ***************** 2008-03-02 | 0.1537 | *********************** 2008-03-03 | 0.0634 | ********* 2008-03-04 | 0.1094 | **************** 2008-03-05 | 0.1241 | ****************** 2008-03-06 | 0.1075 | **************** 2008-03-07 | 0.1086 | **************** 2008-03-08 | 0.1664 | ************************ 2008-03-09 | 0.1647 | ************************ 2008-03-10 | 0.0705 | ********** 2008-03-11 | 0.1190 | ***************** 2008-03-12 | 0.1754 | ************************** 2008-03-13 | 0.1202 | ****************** 2008-03-14 | 0.1351 | ******************** 2008-03-15 | 0.1463 | ********************* 2008-03-16 | 0.1468 | ********************** 2008-03-17 | 0.1425 | ********************* 2008-03-18 | 0.1271 | ******************* 2008-03-19 | 0.1260 | ****************** 2008-03-20 | 0.1209 | ****************** 2008-03-21 | 0.1438 | ********************* 2008-03-23 | 0.1139 | ***************** 2008-03-24 | 0.0916 | ************* 2008-03-25 | 0.1469 | ********************** 2008-03-26 | 0.1316 | ******************* 2008-03-26 | 0.1323 | ******************* 2008-03-27 | 0.1397 | ******************** 2008-03-28 | 0.0927 | ************* 2008-03-29 | 0.0425 | ****** 2008-03-30 | 0.0440 | ****** 2008-03-31 | 0.0461 | ****** 2008-04-01 | 0.0357 | ***** 2008-04-02 | 0.0319 | **** 2008-04-03 | 0.0325 | **** 2008-04-04 | 0.0314 | **** 2008-04-05 | 0.0664 | ********* 2008-04-05 | 0.0652 | ********* 2008-04-06 | 0.0823 | ************ 2008-04-07 | 0.0605 | ********* 2008-04-08 | 0.0553 | ******** 2008-04-09 | 0.0537 | ******** 2008-04-10 | 0.1166 | ***************** 2008-04-11 | 0.0512 | ******* 2008-04-12 | 0.0546 | ******** 2008-04-13 | 0.0619 | ********* 2008-04-14 | 0.0519 | ******* 2008-04-15 | 0.0421 | ****** 2008-04-16 | 0.0441 | ****** 2008-04-17 | 0.0409 | ****** We had some internal problems on 2008-04-10 so things went to hell that day. Once again, qrp is needed for a Rails site I work on because: a) we unfortunately use a web service run by folks who suck at the Internet. Unfortunately the tech folks like myself have little control of this. b) One of our internal backend services have some pathologically bad corner cases we occasionally hit. Eliminating them isn't possible due to strange business requirements (and some of the troublesome backend code is proprietary and we can't improve it). [1] yes, I realize that saying that the number of >10s responses have dropped is like saying we've won the Special Olympics :) -- Eric Wong From lists at ruby-forum.com Mon Apr 21 02:39:43 2008 From: lists at ruby-forum.com (Sandeep Gudibanda) Date: Mon, 21 Apr 2008 08:39:43 +0200 Subject: [Mongrel] Shared memory across mongrels Message-ID: Hi, I have written an application which on initiliazation calculates a graph based on data in DB. And then any updates on the graph(global variable) , i am doing both on graph and db. Everything works fine as long as i work on single mongrel. When i use 2 mongrels, second mongrel cannot see graph in the memory. It gets a null variable. How do i do this so that graph built in the memory is accessible to all mongrels? Regards, Sandeep G -- Posted via http://www.ruby-forum.com/. From rogerpack2005 at gmail.com Mon Apr 21 09:43:03 2008 From: rogerpack2005 at gmail.com (Roger Pack) Date: Mon, 21 Apr 2008 07:43:03 -0600 Subject: [Mongrel] Shared memory across mongrels In-Reply-To: References: Message-ID: <966599840804210643n2d1d2d66s441277085129bd8d@mail.gmail.com> > How do i do this so that graph built in the memory is accessible to all > mongrels? Save data to a DB, or to a session variable ? remember it's like having two separate processes--they don't share memory at all by default. -R From jgeiger at gmail.com Mon Apr 21 09:56:47 2008 From: jgeiger at gmail.com (Joey Geiger) Date: Mon, 21 Apr 2008 08:56:47 -0500 Subject: [Mongrel] Shared memory across mongrels In-Reply-To: <966599840804210643n2d1d2d66s441277085129bd8d@mail.gmail.com> References: <966599840804210643n2d1d2d66s441277085129bd8d@mail.gmail.com> Message-ID: <466af3440804210656g5cdfbc41sae822f5bf07c9162@mail.gmail.com> Any this really isn't a "mongrel" issue, but more an issue with the framework you're using. On Mon, Apr 21, 2008 at 8:43 AM, Roger Pack wrote: > > How do i do this so that graph built in the memory is accessible to all > > mongrels? > Save data to a DB, or to a session variable ? > remember it's like having two separate processes--they don't share > memory at all by default. > -R > > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > From piyush.pr at gmail.com Mon Apr 21 10:54:04 2008 From: piyush.pr at gmail.com (Piyush Ranjan) Date: Mon, 21 Apr 2008 20:24:04 +0530 Subject: [Mongrel] Shared memory across mongrels In-Reply-To: <466af3440804210656g5cdfbc41sae822f5bf07c9162@mail.gmail.com> References: <966599840804210643n2d1d2d66s441277085129bd8d@mail.gmail.com> <466af3440804210656g5cdfbc41sae822f5bf07c9162@mail.gmail.com> Message-ID: <325148f70804210754n61c54785q8550f653a4570f36@mail.gmail.com> Sandeep you may use a different ruby process to contain this graph and your mongrel based rails processes may talk to this drb instance and access the graph. 2 or more mongrel process do not share any data as they are completely different processes. Piyush On Mon, Apr 21, 2008 at 7:26 PM, Joey Geiger wrote: > Any this really isn't a "mongrel" issue, but more an issue with the > framework you're using. > > On Mon, Apr 21, 2008 at 8:43 AM, Roger Pack > wrote: > > > How do i do this so that graph built in the memory is accessible to > all > > > mongrels? > > Save data to a DB, or to a session variable ? > > remember it's like having two separate processes--they don't share > > memory at all by default. > > -R > > > > > > _______________________________________________ > > Mongrel-users mailing list > > Mongrel-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/mongrel-users > > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080421/7b51c02b/attachment.html From lists at ruby-forum.com Wed Apr 23 01:54:43 2008 From: lists at ruby-forum.com (Supriya Agarwal) Date: Wed, 23 Apr 2008 07:54:43 +0200 Subject: [Mongrel] Offloading Background Tasks In-Reply-To: <1205047001.16817.3.camel@shire> References: <2d8ffda38541bfdc3425ca8511bfcd00@ruby-forum.com> <20080302025200.5B44F1858660@rubyforge.org> <42642c790803040636x5830a20x5f20ce43aa058784@mail.gmail.com> <1205047001.16817.3.camel@shire> Message-ID: <2574818fe57c5319cd3afaf120904ebe@ruby-forum.com> Hemant Kumar wrote: > On Sun, 2008-03-09 at 05:58 +0100, Nathan Esquenazi wrote: >> Thanks for your responses. Just wanted to reply back that I installed >> backgroundrb (the new version) and I definitely appreciate how useful it >> is and how it is fairly easy to use. >> >> Thanks for all your hard work Hermant. > > No problems! > > If you are having any problems feel free to shoot your questions on > BackgrounDRb mailing list. Hi! Im using this backgroundrb plugin! The steps i followed for using this plugin are as follows: Kindly correct me where i went wrong! 1)rails_apps/app_name/ruby script/plugin install http://svn.devjavu.com/backgroundrb/trunk 2) rake backgroundrb:setup 3) ruby script/generate worker example 4)opened example_worker.rb and just wrote : 1. class ExampleWorker < BackgrounDRb::MetaWorker 2. set_worker_name :example_worker 3. def create(args = nil) 4. # this method is called, when worker is loaded for the first time 5. logger.info "here" 6. end 7. end after that i tried to run #ruby script/backgroundrb start and it gave me load error :bdrb_config.rb not found! I tried copying it to some directory and then ultimately it popped up another error : fork function is umimplemented on this machine! I have also installed the required gems :packets and chronic.. Can someone help me where i went wrong! or give me a step by step procedure to start backgroundrb running. Thanks in advance! Sup -- Posted via http://www.ruby-forum.com/. From cnk at caltech.edu Thu Apr 24 00:48:38 2008 From: cnk at caltech.edu (Cynthia Kiser) Date: Wed, 23 Apr 2008 21:48:38 -0700 Subject: [Mongrel] Offloading Background Tasks In-Reply-To: <2574818fe57c5319cd3afaf120904ebe@ruby-forum.com> References: <2d8ffda38541bfdc3425ca8511bfcd00@ruby-forum.com> <20080302025200.5B44F1858660@rubyforge.org> <42642c790803040636x5830a20x5f20ce43aa058784@mail.gmail.com> <1205047001.16817.3.camel@shire> <2574818fe57c5319cd3afaf120904ebe@ruby-forum.com> Message-ID: <20080424044838.GB24123@inky.caltech.edu> Quoting Supriya Agarwal : > after that i tried to run #ruby script/backgroundrb start > and it gave me load error :bdrb_config.rb not found! I tried copying it > to some directory and then ultimately it popped up another error : > fork function is umimplemented on this machine! > I have also installed the required gems :packets and chronic.. This isn't really the right list for this but... "fork function is umimplemented on this machine!" Are you running on Windows? I think the error above indicates that your OS fundamentally can't do what backgroundRb tries to do. See Wikipedia for more info on forking http://en.wikipedia.org/wiki/Fork_%28operating_system%29 -- Cynthia Kiser From mattias at oncotype.dk Thu Apr 24 05:11:19 2008 From: mattias at oncotype.dk (Mattias Bodlund) Date: Thu, 24 Apr 2008 11:11:19 +0200 Subject: [Mongrel] Date not changing... Message-ID: I run Apache/2.2.8 (FreeBSD), mongrel_cluster (1.0.5) and mongrel (1.1.3) The app I have issues with is a Rails 1.2.6 app. In one controller I have a before filter that sets the month, year variables to the current month and year if it's not passed in the request. The problem that I now have, after changing to the above configuration from a lighttpd/fast-cgi), is that the date doesn't get updated. Lets say I restart mongrel today all is fine but if I let it run untill tomorrow I still get the date of yesterday. As I also mentioned - this is a new behavior I see after I migrated to mongrel. Any ideas? Cheers Mattias From sam at three60.com Thu Apr 24 09:04:29 2008 From: sam at three60.com (Sam) Date: Thu, 24 Apr 2008 09:04:29 -0400 Subject: [Mongrel] Date not changing... In-Reply-To: References: Message-ID: <48AD8613-699D-493E-909E-9A439470E552@three60.com> Hi, I had a similar problem, however it wasn't a mongrel issue but stupidity on my part. My date variables were being cached at startup, I had been setting a constant to date values. On Apr 24, 2008, at 5:11 AM, Mattias Bodlund wrote: > I run Apache/2.2.8 (FreeBSD), mongrel_cluster (1.0.5) and mongrel > (1.1.3) The app I have issues with is a Rails 1.2.6 app. > > In one controller I have a before filter that sets the month, year > variables to the current month and year if it's not passed in the > request. The problem that I now have, after changing to the above > configuration from a lighttpd/fast-cgi), is that the date doesn't get > updated. > Lets say I restart mongrel today all is fine but if I let it run > untill tomorrow I still get the date of yesterday. > > As I also mentioned - this is a new behavior I see after I migrated to > mongrel. > > Any ideas? > > Cheers > Mattias > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080424/51d6e239/attachment.html From zedshaw at zedshaw.com Fri Apr 25 11:44:04 2008 From: zedshaw at zedshaw.com (Zed A. Shaw) Date: Fri, 25 Apr 2008 11:44:04 -0400 Subject: [Mongrel] Go ahead and use Git Message-ID: <20080425114404.2dae4314.zedshaw@zedshaw.com> Hey, I heard a rumor that there's some FUD going around I won't let people on the Mongrel project use Git. Subversion blows, so go ahead and use Git if it makes people happy. I actually never said you can't use Git. My objection previously was that Evan wanted to move the Subversion repository onto his own computers and off of RubyForge. This would have meant that Evan was in total control of the source, and while I trust him, it would have made people not trust the project. Mongrel has to be housed in a neutral location so that people can get at it no matter what disputes the team might have, and outside of any corporate or special interests. RubyForge is the only place that I (and others) trust, so keep there there. Outside of that, do as you want. That's all. Now that RubyForge supports Git you can go to town on it and rock on. Oh, one thing to consider though is that Windows developers are screwed if you use Git. You would probably have to find a way to mirror in svn or get it working on windows. Consult with Luis about this since he knows more. -- Zed A. Shaw - Hate: http://savingtheinternetwithhate.com/ - Good: http://www.zedshaw.com/ - Evil: http://yearofevil.com/ From luislavena at gmail.com Fri Apr 25 12:27:51 2008 From: luislavena at gmail.com (Luis Lavena) Date: Fri, 25 Apr 2008 13:27:51 -0300 Subject: [Mongrel] Go ahead and use Git In-Reply-To: <20080425114404.2dae4314.zedshaw@zedshaw.com> References: <20080425114404.2dae4314.zedshaw@zedshaw.com> Message-ID: <71166b3b0804250927l47e74ef1x63506045921549ce@mail.gmail.com> On Fri, Apr 25, 2008 at 12:44 PM, Zed A. Shaw wrote: > Hey, > > I heard a rumor that there's some FUD going around I won't let people > on the Mongrel project use Git. Subversion blows, so go ahead and use > Git if it makes people happy. Blame me, I used your comment as reference dealing with the repository code for One-Click Ruby Installer project. We have discussed that on our devel list here [1] and [2]. But I never said that you don't let us, just said that you don't want the svn repo moved into a closed source hosting facility/location. > I actually never said you can't use Git. My objection previously was > that Evan wanted to move the Subversion repository onto his own > computers and off of RubyForge. This would have meant that Evan was in > total control of the source, and while I trust him, it would have made > people not trust the project. Mongrel has to be housed in a neutral > location so that people can get at it no matter what disputes the team > might have, and outside of any corporate or special interests. > > RubyForge is the only place that I (and others) trust, so keep there > there. Outside of that, do as you want. > I keep a bzr clone of mongrel repository here, and even the latest changes I made into stable are pushed thanks to bzr-svn integration. > That's all. Now that RubyForge supports Git you can go to town on it > and rock on. > > Oh, one thing to consider though is that Windows developers are screwed > if you use Git. You would probably have to find a way to mirror in svn > or get it working on windows. Consult with Luis about this since he > knows more. > I've been contributing back to DataMapper and a bit to rubinius using Git, so far, the msysGit team did a great work providing something that "just works", have a few glitches, like svn did, but on average, is good. [1] http://rubyforge.org/pipermail/rubyinstaller-devel/2008-April/000302.html [2] http://rubyforge.org/pipermail/rubyinstaller-devel/2008-April/000306.html -- Luis Lavena Multimedia systems - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams From lists at ruby-forum.com Mon Apr 28 16:14:51 2008 From: lists at ruby-forum.com (Roger Pack) Date: Mon, 28 Apr 2008 22:14:51 +0200 Subject: [Mongrel] leaking weirdness Message-ID: <1b85c2fbe14ffbc79ab5a1509255ddc3@ruby-forum.com> Previously posted on ruby talk. Response...silence :) ...... Perhaps someone out there can help give me a clue about the following situation: My mongrel processes seemed to be leaking. As an example of this, if I added this code to the bottom of environment.rb Thread.new {loop do; print 'w'; end} it didn't affect memory. At first. However after the web server got its first hit its memory consumption would sky-rocket. [i.e. it would monotonically grow by about 50MB/s] Then I ran the same scenario using webrick, instead of mongrel [1.1.3]. No leak. Then I'm thinking 'hmm maybe it's mongrel and the mongrel guys have fixed this.' Update my gem to 1.1.4 from 1.1.3. Using mongrel this time. No leak! Yea! Now this is the weird part. Attempted to recreate the bug. Uninstalled mongrel 1.1.4, reinstalled 1.1.3 uninstalled daemons [its dependency] 1.0.10, reinstalled 1.0.9 [what I had before]. Ran it with Mongrel again [which is where it used to leak]. No leak. I am so confused! Perhaps rubygems was updated and so it "compiled it right this time"? Note that there don't appear to be significant code changes in mongrel from 1.1.3 to 1.1.4, though there could be, I'm not sure. It's probably not even a mongrel problem. Maybe if somebody else who 'suspects' a memory leak could try my code and see if it leaks theirs, that would be well appreciated. The good news is that "maybe" updating rubygems and recompiling mongrel helps with memory leaks in rails apps. Thoughts? -R OS X PPC patchlevel 111 -- Posted via http://www.ruby-forum.com/. From evan at cloudbur.st Mon Apr 28 16:23:22 2008 From: evan at cloudbur.st (Evan Weaver) Date: Mon, 28 Apr 2008 16:23:22 -0400 Subject: [Mongrel] leaking weirdness In-Reply-To: <1b85c2fbe14ffbc79ab5a1509255ddc3@ruby-forum.com> References: <1b85c2fbe14ffbc79ab5a1509255ddc3@ruby-forum.com> Message-ID: Maybe the upgrade installed fastthread? Most likely though it's related to the Rails app itself, not Mongrel, and isn't immediately reproducible. Cache freshness could have a large effect on which codepaths get traversed. What did you add that Thread loop for? I don't understand. Evan On Mon, Apr 28, 2008 at 4:14 PM, Roger Pack wrote: > Previously posted on ruby talk. Response...silence :) > > ...... > Perhaps someone out there can help give me a clue about the following > situation: > > My mongrel processes seemed to be leaking. > As an example of this, if I added this code to the bottom of > environment.rb > Thread.new {loop do; print 'w'; end} > it didn't affect memory. At first. However after the web server got > its > first hit its memory consumption would sky-rocket. [i.e. it would > monotonically grow by about 50MB/s] > > Then I ran the same scenario using webrick, instead of mongrel [1.1.3]. > No leak. > > Then I'm thinking 'hmm maybe it's mongrel and the mongrel guys have > fixed this.' > > Update my gem to 1.1.4 from 1.1.3. > Using mongrel this time. > No leak! Yea! > > Now this is the weird part. > Attempted to recreate the bug. > Uninstalled mongrel 1.1.4, reinstalled 1.1.3 > uninstalled daemons [its dependency] 1.0.10, reinstalled 1.0.9 [what I > had before]. > > Ran it with Mongrel again [which is where it used to leak]. > No leak. > I am so confused! > Perhaps rubygems was updated and so it "compiled it right this time"? > Note that there don't appear to be significant code changes in mongrel > from 1.1.3 to 1.1.4, though there could be, I'm not sure. It's probably > not even a mongrel problem. > > Maybe if somebody else who 'suspects' a memory leak could try my code > and see if it leaks theirs, that would be well appreciated. > > The good news is that "maybe" updating rubygems and recompiling mongrel > helps with memory leaks in rails apps. > > Thoughts? > -R > OS X PPC patchlevel 111 > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -- Evan Weaver From luislavena at gmail.com Mon Apr 28 16:24:50 2008 From: luislavena at gmail.com (Luis Lavena) Date: Mon, 28 Apr 2008 17:24:50 -0300 Subject: [Mongrel] leaking weirdness In-Reply-To: <1b85c2fbe14ffbc79ab5a1509255ddc3@ruby-forum.com> References: <1b85c2fbe14ffbc79ab5a1509255ddc3@ruby-forum.com> Message-ID: <71166b3b0804281324n71b4ff3dt8c59a3b468f8ef78@mail.gmail.com> On Mon, Apr 28, 2008 at 5:14 PM, Roger Pack wrote: > Previously posted on ruby talk. Response...silence :) > Didn't saw that post, sorry. > ...... > Perhaps someone out there can help give me a clue about the following > situation: > > My mongrel processes seemed to be leaking. > As an example of this, if I added this code to the bottom of > environment.rb > Thread.new {loop do; print 'w'; end} > it didn't affect memory. At first. However after the web server got > its > first hit its memory consumption would sky-rocket. [i.e. it would > monotonically grow by about 50MB/s] > > Then I ran the same scenario using webrick, instead of mongrel [1.1.3]. > No leak. > > Then I'm thinking 'hmm maybe it's mongrel and the mongrel guys have > fixed this.' > > Update my gem to 1.1.4 from 1.1.3. > Using mongrel this time. > No leak! Yea! > > Now this is the weird part. > Attempted to recreate the bug. > Uninstalled mongrel 1.1.4, reinstalled 1.1.3 > uninstalled daemons [its dependency] 1.0.10, reinstalled 1.0.9 [what I > had before]. > > Ran it with Mongrel again [which is where it used to leak]. > No leak. > I am so confused! > Perhaps rubygems was updated and so it "compiled it right this time"? > Note that there don't appear to be significant code changes in mongrel > from 1.1.3 to 1.1.4, though there could be, I'm not sure. It's probably > not even a mongrel problem. > > Maybe if somebody else who 'suspects' a memory leak could try my code > and see if it leaks theirs, that would be well appreciated. > > The good news is that "maybe" updating rubygems and recompiling mongrel > helps with memory leaks in rails apps. > > Thoughts? > -R > OS X PPC patchlevel 111 Did you upgraded rubygems at the time you updated mongrel from 1.1.3 to 1.1.4? Also, did you hit the same request/url during testing and recreating this bug? (just to be sure). I've seen lot of complains about "my mongrel is eating my memory", but as Erza commented in his Rack Rails [1], it seems lot of stuff at Rails needs attention and lot of fixes, since they eat memory like a hungry baby... and is not directly fault of Mongrel. Could be Ruby, rubygems, some plugin you're using, even Rails :-P If you cannot revert back to the version of rubygems you was using to test the exact same environment, there is nothing more we can do except: thank god you got that solved! :-P Maybe you can see bleakhouse [2] and give it a whirl? [1] http://brainspl.at/articles/2008/04/25/hey-rails-nice-rack [2] http://blog.evanweaver.com/articles/2008/04/06/bleakhouse-4/ -- Luis Lavena Multimedia systems - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams From rogerpack2005 at gmail.com Mon Apr 28 20:02:20 2008 From: rogerpack2005 at gmail.com (Roger Pack) Date: Mon, 28 Apr 2008 18:02:20 -0600 Subject: [Mongrel] leaking weirdness In-Reply-To: References: <1b85c2fbe14ffbc79ab5a1509255ddc3@ruby-forum.com> Message-ID: <966599840804281702h13e72590v7bafce54d6644155@mail.gmail.com> On Mon, Apr 28, 2008 at 2:23 PM, Evan Weaver wrote: > Maybe the upgrade installed fastthread? Could have. Prolly wouldn't have done much since I'm running patch 111. > Most likely though it's related to the Rails app itself, not Mongrel, > and isn't immediately reproducible. Cache freshness could have a large > effect on which codepaths get traversed. It happened even with a 'blank, vanilla' rails app. All I had to do was add an extra thread and run it in mongrel. rails 1.2.3 I believe the problem 'kicked off' no matter what page I requested--meaning that basically rails had to load. I wonder if the problem is that I started a thread BEFORE rails loaded, and so some of that 'non thread safeness' of Ruby came back to haunt me or something. It confused scopes or something. [1] > What did you add that Thread loop for? I don't understand. Originally it was a 'helper thread' that restarted my app if in production and any files were changed. Then I saw that leaks were happening, and replaced it with that one, to exaggerate the problem. and was successful :) I almost wish I could recreate it so I could kill these leaking bugs once and for all. I was this close from diving into eval.c and tearing into ruby's gc until I squashed it. So much for recreation I guess. As a side-note, from [1] we see that Ruby, at least the MRI of that post, had some sockets that would be cross-threaded. I still can't figure out why this doesn't occur with mongrel. Hmm. -R [1] http://64.233.167.104/search?q=cache:Ilkzvy47LgcJ:permalink.gmane.org/gmane.comp.lang.ruby.mongrel.general/245+zed+shaw+rails+thread+safety&hl=en&gl=us&ct=clnk&cd=1 From lists at ruby-forum.com Tue Apr 29 03:12:12 2008 From: lists at ruby-forum.com (Kirk Schafer) Date: Tue, 29 Apr 2008 09:12:12 +0200 Subject: [Mongrel] mongrel startup fails now: already initialized constant In-Reply-To: <6157d35f0b42389234242367d66e5971@ruby-forum.com> References: <6157d35f0b42389234242367d66e5971@ruby-forum.com> Message-ID: <8a17b227eaf20ff0f85d3fad1cc04f5f@ruby-forum.com> Sj Seo wrote: > Stephen Bannasch wrote: >> I updated to ruby 1.8.6p111 last night and updated some gems and now >> mongrel doesn't work: >> >> (group doesn't allow long quotes) >> >> $ cd test5 >> $ script/server >> => Booting Mongrel (use 'script/server webrick' to force WEBrick) >> => Rails application starting on http://0.0.0.0:3000 >> => Call with -d to detach >> => Ctrl-C to shutdown server >> ** Starting Mongrel listening at 0.0.0.0:3000 >> ** Starting Rails with development environment... >> ** Rails loaded. >> ** Loading any Rails specific GemPlugins >> Exiting >> /usr/local/lib/ruby/gems/1.8/gems/rails-1.2.5/lib/commands/servers/mongrel.rb:15: >> warning: already initialized constant OPTIONS >> >> > > ---------------- > > I had a similar issue, but now I have resolved it. > After I've installed some rails plugins on window xp, > mongrel server didn't work. > > As I ran the rake task, 'rake rails:update', > web server has worked. > > ## change to the root of rails project > $ cd test5 > > ## check the rake tasks > $ test5> rake --tasks > > ## run the 'rails:update' task > $ test5> rake rails:update > > If you have installed the ruby 1.8.6 from souce package, > maybe you must be re-install all the gem plugins(rails, rake, mysql, > ...). > > ref. > - > http://scoop.cheerfactory.co.uk/2007/11/02/upgrading-ruby-on-ubuntu-dapper/ > - > http://wiki.ajaxstart.com/usemodj/browse.screen?Ruby_1.8.6_Source_Install --------------------------- I realize this is an old post, so apologize for bumping it if that's what occurs. I encountered the same issue on a new Gentoo box after a "darcs get", whereby script/server started WEBrick but then out popped Mongrel..and then the error "warning: already initialized constant OPTIONS." This made no sense to me, and I landed here. The suggestion to run the rake tasks had no effect, but a mongrel_rails start did report a different error: /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require': no such file to load -- tzinfo (MissingSourceFile) Subsequently, only a single "gem install tzinfo" (your circumstances may vary) was required to allow both "script/server" and "mongrel_rails start" to function normally. So, an FYI for the community at large. -- Posted via http://www.ruby-forum.com/. From rogerpack2005 at gmail.com Tue Apr 29 22:30:15 2008 From: rogerpack2005 at gmail.com (Roger Pack) Date: Tue, 29 Apr 2008 20:30:15 -0600 Subject: [Mongrel] using less RAM Message-ID: <966599840804291930j7a70e7cahac8f075ecb008499@mail.gmail.com> in case anybody finds it useful, here's a link to Hongli's work on patching the GC so you can save like 30%/mongrel or so [if you use forked mongrels]. Might be useful if RAM is your bottleneck. http://izumi.plan99.net/blog/index.php/2008/01/14/making-ruby's-garbage-collector-copy-on-write-friendly-part-7/ From eimorton at gmail.com Tue Apr 29 22:31:17 2008 From: eimorton at gmail.com (Erik Morton) Date: Tue, 29 Apr 2008 22:31:17 -0400 Subject: [Mongrel] Mongrel hangs in production. gdb stack included Message-ID: <5E151013-6346-40E5-B1FB-588F4C6EB014@gmail.com> I am getting hanging Mongrels daily under light volume. Mongrel Web Server 1.1.4 Apache 2.2 w/mod_proxy RedHat EL4 2.6.9-5.ELsmp #1 SMP Wed Jan 5 19:30:39 EST 2005 i686 i686 i386 GNU/Linux ruby 1.8.5 (2006-08-25) [i686-linux] fastthread (1.0.1) Mysql 5.0.45 The mongrels are hanging with 0% CPU, their database connections are still being reported as open by mysql. Attaching gdb to the processes yields similar results (included below). I have no idea about what the next steps are in figuring out what is going on -- short of upgrading to ruby 1.8.6! Help and pointers are much appreciated. (gdb) whe #0 0x0058b7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 #1 0x0065ec93 in __read_nocancel () from /lib/tls/libc.so.6 #2 0x04ccc27f in vio_read (vio=0xfffffe00, buf=0xfffffe00
, size=-512) at viosocket.c:44 #3 0x04ccc356 in vio_read_buff (vio=0x9088ae0, buf=0xb5d471df "e Travel Plaza, located on I-90 eastbound between Interchanges 48A (Pembroke) and 48 (Batavia), at milepost 397. The renovation projects scheduled at Ulster and Pembroke travel plazas, in addition to "..., size=4412) at viosocket.c:95 #4 0x04ccd8f6 in my_real_read (net=0x913eb10, complen=0xbfed5e98) at net.c:804 #5 0x04ccdc30 in my_net_read (net=0x913eb10) at net.c:978 #6 0x04cc6b78 in net_safe_read (mysql=0x913eb10) at client.c:596 #7 0x04cc792f in cli_read_rows (mysql=0x913eb10, mysql_fields=0xad6e1a0, fields=59) at client.c:1343 #8 0x04cc9881 in mysql_store_result (mysql=0x913eb10) at client.c:2697 #9 0x04ca299f in store_result (obj=3076071240) at mysql.c:677 #10 0x0805bcb4 in rb_call0 (klass=3076091220, recv=3076071240, id=27825, oid=4294966784, argc=1, argv=0xbfed6160, body=0xb7596458, flags=0) at eval.c:5665 #11 0x0805c017 in rb_call (klass=3076091220, recv=3076071240, mid=27825, argc=1, argv=0xbfed6160, scope=0) at eval.c:6048 #12 0x08058127 in rb_eval (self=3076059900, n=0x113c) at ruby.h:654 #13 0x0805d99b in rb_yield_0 (val=6, self=3076059900, klass=0, flags=0, avalue=0) at eval.c:4987 #14 0x08058475 in rb_eval (self=3076059900, n=0x113c) at eval.c:3248 #15 0x08058952 in rb_eval (self=3076059900, n=0x113c) at eval.c:3624 #16 0x0805d99b in rb_yield_0 (val=6, self=3076059900, klass=0, flags=0, avalue=0) at eval.c:4987 #17 0x08058475 in rb_eval (self=3082341360, n=0x113c) at eval.c:3248 #18 0x0805b9ce in rb_call0 (klass=3082341260, recv=3082341360, id=53385, oid=4294966784, argc=0, argv=0x0, body=0xb7b931f8, flags=0) at eval.c:5954 #19 0x0805c017 in rb_call (klass=3082341260, recv=3082341360, mid=53385, argc=0, argv=0x0, scope=0) at eval.c:6048 #20 0x08058127 in rb_eval (self=3082341360, n=0x113c) at ruby.h:654 (gdb) whe #0 0x0058b7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 #1 0x0065ec93 in __read_nocancel () from /lib/tls/libc.so.6 #2 0x01d6d27f in vio_read (vio=0xfffffe00, buf=0xfffffe00
, size=-512) at viosocket.c:44 #3 0x01d6d356 in vio_read_buff (vio=0x93e7c30, buf=0xb6f00db4 "~\202\215^-?\213\201?QJ\237\203\201\230D?\232V \201M8\233\224?\201\021-o\217[\200?#?\211\236\200?\203??0\213\021wC?h \211\236j\234?\215\210F]?\232\207\006Q\b\236\220\205?Dv\231t\204? 8\220\224 \203?-\214\216\236\202?$\026\210?\201?\203Y?u\221?v???\217? jS??\215?]\206??\213?P?\235?\212\023DS\230?\210 at 8\210\223d\206o-?\215? \204?$R\210z\202?\203\017??\230\236v??\026\226", size=503704) at viosocket.c:95 #4 0x01d6e8f6 in my_real_read (net=0x93e71a8, complen=0xbff9efa8) at net.c:804 #5 0x01d6ec30 in my_net_read (net=0x93e71a8) at net.c:978 #6 0x01d67b78 in net_safe_read (mysql=0x93e71a8) at client.c:596 #7 0x01d6892f in cli_read_rows (mysql=0x93e71a8, mysql_fields=0xaec8f00, fields=51) at client.c:1343 #8 0x01d6a881 in mysql_store_result (mysql=0x93e71a8) at client.c:2697 #9 0x01d4399f in store_result (obj=3071130920) at mysql.c:677 #10 0x0805bcb4 in rb_call0 (klass=3071150400, recv=3071130920, id=27825, oid=4294966784, argc=1, argv=0xbff9f270, body=0xb70e0058, flags=0) at eval.c:5665 #11 0x0805c017 in rb_call (klass=3071150400, recv=3071130920, mid=27825, argc=1, argv=0xbff9f270, scope=0) at eval.c:6048 #12 0x08058127 in rb_eval (self=3071126200, n=0x7af98) at ruby.h:654 From wyhaines at gmail.com Wed Apr 30 08:12:53 2008 From: wyhaines at gmail.com (Kirk Haines) Date: Wed, 30 Apr 2008 06:12:53 -0600 Subject: [Mongrel] Mongrel hangs in production. gdb stack included In-Reply-To: <5E151013-6346-40E5-B1FB-588F4C6EB014@gmail.com> References: <5E151013-6346-40E5-B1FB-588F4C6EB014@gmail.com> Message-ID: On Tue, Apr 29, 2008 at 8:31 PM, Erik Morton wrote: > I am getting hanging Mongrels daily under light volume. > #8 0x04cc9881 in mysql_store_result (mysql=0x913eb10) at client.c:2697 > #8 0x01d6a881 in mysql_store_result (mysql=0x93e71a8) at client.c:2697 Whatever the problem, it relates to your app's db connection. Kirk Haines From dave at cheney.net Wed Apr 30 08:50:56 2008 From: dave at cheney.net (Dave Cheney) Date: Wed, 30 Apr 2008 22:50:56 +1000 Subject: [Mongrel] Mongrel hangs in production. gdb stack included In-Reply-To: <5E151013-6346-40E5-B1FB-588F4C6EB014@gmail.com> References: <5E151013-6346-40E5-B1FB-588F4C6EB014@gmail.com> Message-ID: <94A91EF2-5A28-4674-9975-4B3A204E7103@cheney.net> What does mysql> show innodb status\G mysql> show processlist; show during the hang, it may be a deadlock in the db. We had similar problems with large after_save hooks that would deadlock updating multiple tables. Cheers Dave On 30/04/2008, at 12:31 PM, Erik Morton wrote: > I am getting hanging Mongrels daily under light volume. > > Mongrel Web Server 1.1.4 > Apache 2.2 w/mod_proxy > RedHat EL4 2.6.9-5.ELsmp #1 SMP Wed Jan 5 19:30:39 EST 2005 i686 > i686 i386 GNU/Linux > ruby 1.8.5 (2006-08-25) [i686-linux] > fastthread (1.0.1) > Mysql 5.0.45 > > The mongrels are hanging with 0% CPU, their database connections are > still being reported as open by mysql. Attaching gdb to the > processes yields similar results (included below). I have no idea > about what the next steps are in figuring out what is going on -- > short of upgrading to ruby 1.8.6! > > Help and pointers are much appreciated. > > (gdb) whe > #0 0x0058b7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 > #1 0x0065ec93 in __read_nocancel () from /lib/tls/libc.so.6 > #2 0x04ccc27f in vio_read (vio=0xfffffe00, buf=0xfffffe00
0xfffffe00 out of bounds>, size=-512) at viosocket.c:44 > #3 0x04ccc356 in vio_read_buff (vio=0x9088ae0, > buf=0xb5d471df "e Travel Plaza, located on I-90 eastbound between > Interchanges 48A (Pembroke) and 48 (Batavia), at milepost 397. The > renovation projects scheduled at Ulster and Pembroke travel plazas, > in addition to "..., size=4412) at viosocket.c:95 > #4 0x04ccd8f6 in my_real_read (net=0x913eb10, complen=0xbfed5e98) > at net.c:804 > #5 0x04ccdc30 in my_net_read (net=0x913eb10) at net.c:978 > #6 0x04cc6b78 in net_safe_read (mysql=0x913eb10) at client.c:596 > #7 0x04cc792f in cli_read_rows (mysql=0x913eb10, > mysql_fields=0xad6e1a0, fields=59) at client.c:1343 > #8 0x04cc9881 in mysql_store_result (mysql=0x913eb10) at client.c: > 2697 > #9 0x04ca299f in store_result (obj=3076071240) at mysql.c:677 > #10 0x0805bcb4 in rb_call0 (klass=3076091220, recv=3076071240, > id=27825, oid=4294966784, argc=1, argv=0xbfed6160, body=0xb7596458, > flags=0) at eval.c:5665 > #11 0x0805c017 in rb_call (klass=3076091220, recv=3076071240, > mid=27825, argc=1, argv=0xbfed6160, scope=0) at eval.c:6048 > #12 0x08058127 in rb_eval (self=3076059900, n=0x113c) at ruby.h:654 > #13 0x0805d99b in rb_yield_0 (val=6, self=3076059900, klass=0, > flags=0, avalue=0) at eval.c:4987 > #14 0x08058475 in rb_eval (self=3076059900, n=0x113c) at eval.c:3248 > #15 0x08058952 in rb_eval (self=3076059900, n=0x113c) at eval.c:3624 > #16 0x0805d99b in rb_yield_0 (val=6, self=3076059900, klass=0, > flags=0, avalue=0) at eval.c:4987 > #17 0x08058475 in rb_eval (self=3082341360, n=0x113c) at eval.c:3248 > #18 0x0805b9ce in rb_call0 (klass=3082341260, recv=3082341360, > id=53385, oid=4294966784, argc=0, argv=0x0, body=0xb7b931f8, > flags=0) at eval.c:5954 > #19 0x0805c017 in rb_call (klass=3082341260, recv=3082341360, > mid=53385, argc=0, argv=0x0, scope=0) at eval.c:6048 > #20 0x08058127 in rb_eval (self=3082341360, n=0x113c) at ruby.h:654 > > > (gdb) whe > #0 0x0058b7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 > #1 0x0065ec93 in __read_nocancel () from /lib/tls/libc.so.6 > #2 0x01d6d27f in vio_read (vio=0xfffffe00, buf=0xfffffe00
0xfffffe00 out of bounds>, size=-512) at viosocket.c:44 > #3 0x01d6d356 in vio_read_buff (vio=0x93e7c30, > buf=0xb6f00db4 "~\202\215^-?\213\201?QJ\237\203\201\230D?\232V > \201M8\233\224?\201\021-o\217[\200?#?\211\236\200?\203??0\213\021wC?h > \211\236j\234?\215\210F]?\232\207\006Q\b\236\220\205?Dv\231t\204? > 8\220\224 \203?-\214\216\236\202?$\026\210?\201?\203Y?u\221?v???\217? > jS??\215?]\206??\213?P?\235?\212\023DS\230?\210 at 8\210\223d\206o-? > \215?\204?$R\210z\202?\203\017??\230\236v??\026\226", size=503704) > at viosocket.c:95 > #4 0x01d6e8f6 in my_real_read (net=0x93e71a8, complen=0xbff9efa8) > at net.c:804 > #5 0x01d6ec30 in my_net_read (net=0x93e71a8) at net.c:978 > #6 0x01d67b78 in net_safe_read (mysql=0x93e71a8) at client.c:596 > #7 0x01d6892f in cli_read_rows (mysql=0x93e71a8, > mysql_fields=0xaec8f00, fields=51) at client.c:1343 > #8 0x01d6a881 in mysql_store_result (mysql=0x93e71a8) at client.c: > 2697 > #9 0x01d4399f in store_result (obj=3071130920) at mysql.c:677 > #10 0x0805bcb4 in rb_call0 (klass=3071150400, recv=3071130920, > id=27825, oid=4294966784, argc=1, argv=0xbff9f270, body=0xb70e0058, > flags=0) at eval.c:5665 > #11 0x0805c017 in rb_call (klass=3071150400, recv=3071130920, > mid=27825, argc=1, argv=0xbff9f270, scope=0) at eval.c:6048 > #12 0x08058127 in rb_eval (self=3071126200, n=0x7af98) at ruby.h:654 > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users From eimorton at gmail.com Wed Apr 30 09:28:34 2008 From: eimorton at gmail.com (Erik Morton) Date: Wed, 30 Apr 2008 09:28:34 -0400 Subject: [Mongrel] Mongrel hangs in production. gdb stack included In-Reply-To: <94A91EF2-5A28-4674-9975-4B3A204E7103@cheney.net> References: <5E151013-6346-40E5-B1FB-588F4C6EB014@gmail.com> <94A91EF2-5A28-4674-9975-4B3A204E7103@cheney.net> Message-ID: <197640DD-71D3-466E-A2A8-80BE8BE7673D@gmail.com> It happens at night usually so I haven't had a chance to run show innodb status yet. I will say that once when it happened show processlist still showed all of the expected connections. I have no large hooks. Just a call to MyObject.find(:all, :include => :parent) that generates an SQL statement approximately 2,000-3,000 characters long. My max_packet is set to 16mb Erik On Apr 30, 2008, at 8:50 AM, Dave Cheney wrote: > What does > mysql> show innodb status\G > mysql> show processlist; > > show during the hang, it may be a deadlock in the db. We had similar > problems with large after_save hooks that would deadlock updating > multiple tables. > > Cheers > > Dave > > On 30/04/2008, at 12:31 PM, Erik Morton wrote: > >> I am getting hanging Mongrels daily under light volume. >> >> Mongrel Web Server 1.1.4 >> Apache 2.2 w/mod_proxy >> RedHat EL4 2.6.9-5.ELsmp #1 SMP Wed Jan 5 19:30:39 EST 2005 i686 >> i686 i386 GNU/Linux >> ruby 1.8.5 (2006-08-25) [i686-linux] >> fastthread (1.0.1) >> Mysql 5.0.45 >> >> The mongrels are hanging with 0% CPU, their database connections >> are still being reported as open by mysql. Attaching gdb to the >> processes yields similar results (included below). I have no idea >> about what the next steps are in figuring out what is going on -- >> short of upgrading to ruby 1.8.6! >> >> Help and pointers are much appreciated. >> >> (gdb) whe >> #0 0x0058b7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 >> #1 0x0065ec93 in __read_nocancel () from /lib/tls/libc.so.6 >> #2 0x04ccc27f in vio_read (vio=0xfffffe00, buf=0xfffffe00
> 0xfffffe00 out of bounds>, size=-512) at viosocket.c:44 >> #3 0x04ccc356 in vio_read_buff (vio=0x9088ae0, >> buf=0xb5d471df "e Travel Plaza, located on I-90 eastbound between >> Interchanges 48A (Pembroke) and 48 (Batavia), at milepost 397. The >> renovation projects scheduled at Ulster and Pembroke travel plazas, >> in addition to "..., size=4412) at viosocket.c:95 >> #4 0x04ccd8f6 in my_real_read (net=0x913eb10, complen=0xbfed5e98) >> at net.c:804 >> #5 0x04ccdc30 in my_net_read (net=0x913eb10) at net.c:978 >> #6 0x04cc6b78 in net_safe_read (mysql=0x913eb10) at client.c:596 >> #7 0x04cc792f in cli_read_rows (mysql=0x913eb10, >> mysql_fields=0xad6e1a0, fields=59) at client.c:1343 >> #8 0x04cc9881 in mysql_store_result (mysql=0x913eb10) at client.c: >> 2697 >> #9 0x04ca299f in store_result (obj=3076071240) at mysql.c:677 >> #10 0x0805bcb4 in rb_call0 (klass=3076091220, recv=3076071240, >> id=27825, oid=4294966784, argc=1, argv=0xbfed6160, body=0xb7596458, >> flags=0) at eval.c:5665 >> #11 0x0805c017 in rb_call (klass=3076091220, recv=3076071240, >> mid=27825, argc=1, argv=0xbfed6160, scope=0) at eval.c:6048 >> #12 0x08058127 in rb_eval (self=3076059900, n=0x113c) at ruby.h:654 >> #13 0x0805d99b in rb_yield_0 (val=6, self=3076059900, klass=0, >> flags=0, avalue=0) at eval.c:4987 >> #14 0x08058475 in rb_eval (self=3076059900, n=0x113c) at eval.c:3248 >> #15 0x08058952 in rb_eval (self=3076059900, n=0x113c) at eval.c:3624 >> #16 0x0805d99b in rb_yield_0 (val=6, self=3076059900, klass=0, >> flags=0, avalue=0) at eval.c:4987 >> #17 0x08058475 in rb_eval (self=3082341360, n=0x113c) at eval.c:3248 >> #18 0x0805b9ce in rb_call0 (klass=3082341260, recv=3082341360, >> id=53385, oid=4294966784, argc=0, argv=0x0, body=0xb7b931f8, >> flags=0) at eval.c:5954 >> #19 0x0805c017 in rb_call (klass=3082341260, recv=3082341360, >> mid=53385, argc=0, argv=0x0, scope=0) at eval.c:6048 >> #20 0x08058127 in rb_eval (self=3082341360, n=0x113c) at ruby.h:654 >> >> >> (gdb) whe >> #0 0x0058b7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 >> #1 0x0065ec93 in __read_nocancel () from /lib/tls/libc.so.6 >> #2 0x01d6d27f in vio_read (vio=0xfffffe00, buf=0xfffffe00
> 0xfffffe00 out of bounds>, size=-512) at viosocket.c:44 >> #3 0x01d6d356 in vio_read_buff (vio=0x93e7c30, >> buf=0xb6f00db4 "~\202\215^-?\213\201?QJ\237\203\201\230D?\232V >> \201M8\233\224?\201\021-o\217[\200?#?\211\236\200?\203??0\213\021wC? >> h\211\236j\234?\215\210F]?\232\207\006Q\b\236\220\205?Dv\231t\204? >> 8\220\224 \203?-\214\216\236\202?$\026\210?\201?\203Y?u\221?v??? >> \217?jS??\215?]\206??\213?P?\235?\212\023DS\230?\210 at 8\210\223d >> \206o-?\215?\204?$R\210z\202?\203\017??\230\236v??\026\226", >> size=503704) >> at viosocket.c:95 >> #4 0x01d6e8f6 in my_real_read (net=0x93e71a8, complen=0xbff9efa8) >> at net.c:804 >> #5 0x01d6ec30 in my_net_read (net=0x93e71a8) at net.c:978 >> #6 0x01d67b78 in net_safe_read (mysql=0x93e71a8) at client.c:596 >> #7 0x01d6892f in cli_read_rows (mysql=0x93e71a8, >> mysql_fields=0xaec8f00, fields=51) at client.c:1343 >> #8 0x01d6a881 in mysql_store_result (mysql=0x93e71a8) at client.c: >> 2697 >> #9 0x01d4399f in store_result (obj=3071130920) at mysql.c:677 >> #10 0x0805bcb4 in rb_call0 (klass=3071150400, recv=3071130920, >> id=27825, oid=4294966784, argc=1, argv=0xbff9f270, body=0xb70e0058, >> flags=0) at eval.c:5665 >> #11 0x0805c017 in rb_call (klass=3071150400, recv=3071130920, >> mid=27825, argc=1, argv=0xbff9f270, scope=0) at eval.c:6048 >> #12 0x08058127 in rb_eval (self=3071126200, n=0x7af98) at ruby.h:654 >> >> _______________________________________________ >> Mongrel-users mailing list >> Mongrel-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mongrel-users > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users From jgeiger at gmail.com Wed Apr 30 10:51:37 2008 From: jgeiger at gmail.com (Joey Geiger) Date: Wed, 30 Apr 2008 09:51:37 -0500 Subject: [Mongrel] Mongrel hangs in production. gdb stack included In-Reply-To: <197640DD-71D3-466E-A2A8-80BE8BE7673D@gmail.com> References: <5E151013-6346-40E5-B1FB-588F4C6EB014@gmail.com> <94A91EF2-5A28-4674-9975-4B3A204E7103@cheney.net> <197640DD-71D3-466E-A2A8-80BE8BE7673D@gmail.com> Message-ID: <466af3440804300751n50a18aa7r8501fa7f868b5f2b@mail.gmail.com> Are you rotating your log files? Might want to google "mongrel log rotate hang" as I believe there have been some issues brought up here in the past. On Wed, Apr 30, 2008 at 8:28 AM, Erik Morton wrote: > It happens at night usually so I haven't had a chance to run show innodb > status yet. I will say that once when it happened show processlist still > showed all of the expected connections. > > I have no large hooks. Just a call to MyObject.find(:all, :include => > :parent) that generates an SQL statement approximately 2,000-3,000 > characters long. My max_packet is set to 16mb > > Erik > > > On Apr 30, 2008, at 8:50 AM, Dave Cheney wrote: > > > What does > > mysql> show innodb status\G > > mysql> show processlist; > > > > show during the hang, it may be a deadlock in the db. We had similar > problems with large after_save hooks that would deadlock updating multiple > tables. > > > > Cheers > > > > Dave > > > > On 30/04/2008, at 12:31 PM, Erik Morton wrote: > > > > > > > I am getting hanging Mongrels daily under light volume. > > > > > > Mongrel Web Server 1.1.4 > > > Apache 2.2 w/mod_proxy > > > RedHat EL4 2.6.9-5.ELsmp #1 SMP Wed Jan 5 19:30:39 EST 2005 i686 i686 > i386 GNU/Linux > > > ruby 1.8.5 (2006-08-25) [i686-linux] > > > fastthread (1.0.1) > > > Mysql 5.0.45 > > > > > > The mongrels are hanging with 0% CPU, their database connections are > still being reported as open by mysql. Attaching gdb to the processes yields > similar results (included below). I have no idea about what the next steps > are in figuring out what is going on -- short of upgrading to ruby 1.8.6! > > > > > > Help and pointers are much appreciated. > > > > > > (gdb) whe > > > #0 0x0058b7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 > > > #1 0x0065ec93 in __read_nocancel () from /lib/tls/libc.so.6 > > > #2 0x04ccc27f in vio_read (vio=0xfffffe00, buf=0xfffffe00
0xfffffe00 out of bounds>, size=-512) at viosocket.c:44 > > > #3 0x04ccc356 in vio_read_buff (vio=0x9088ae0, > > > buf=0xb5d471df "e Travel Plaza, located on I-90 eastbound between > Interchanges 48A (Pembroke) and 48 (Batavia), at milepost 397. The > renovation projects scheduled at Ulster and Pembroke travel plazas, in > addition to "..., size=4412) at viosocket.c:95 > > > #4 0x04ccd8f6 in my_real_read (net=0x913eb10, complen=0xbfed5e98) at > net.c:804 > > > #5 0x04ccdc30 in my_net_read (net=0x913eb10) at net.c:978 > > > #6 0x04cc6b78 in net_safe_read (mysql=0x913eb10) at client.c:596 > > > #7 0x04cc792f in cli_read_rows (mysql=0x913eb10, > mysql_fields=0xad6e1a0, fields=59) at client.c:1343 > > > #8 0x04cc9881 in mysql_store_result (mysql=0x913eb10) at client.c:2697 > > > #9 0x04ca299f in store_result (obj=3076071240) at mysql.c:677 > > > #10 0x0805bcb4 in rb_call0 (klass=3076091220, recv=3076071240, id=27825, > oid=4294966784, argc=1, argv=0xbfed6160, body=0xb7596458, flags=0) at > eval.c:5665 > > > #11 0x0805c017 in rb_call (klass=3076091220, recv=3076071240, mid=27825, > argc=1, argv=0xbfed6160, scope=0) at eval.c:6048 > > > #12 0x08058127 in rb_eval (self=3076059900, n=0x113c) at ruby.h:654 > > > #13 0x0805d99b in rb_yield_0 (val=6, self=3076059900, klass=0, flags=0, > avalue=0) at eval.c:4987 > > > #14 0x08058475 in rb_eval (self=3076059900, n=0x113c) at eval.c:3248 > > > #15 0x08058952 in rb_eval (self=3076059900, n=0x113c) at eval.c:3624 > > > #16 0x0805d99b in rb_yield_0 (val=6, self=3076059900, klass=0, flags=0, > avalue=0) at eval.c:4987 > > > #17 0x08058475 in rb_eval (self=3082341360, n=0x113c) at eval.c:3248 > > > #18 0x0805b9ce in rb_call0 (klass=3082341260, recv=3082341360, id=53385, > oid=4294966784, argc=0, argv=0x0, body=0xb7b931f8, flags=0) at eval.c:5954 > > > #19 0x0805c017 in rb_call (klass=3082341260, recv=3082341360, mid=53385, > argc=0, argv=0x0, scope=0) at eval.c:6048 > > > #20 0x08058127 in rb_eval (self=3082341360, n=0x113c) at ruby.h:654 > > > > > > > > > (gdb) whe > > > #0 0x0058b7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 > > > #1 0x0065ec93 in __read_nocancel () from /lib/tls/libc.so.6 > > > #2 0x01d6d27f in vio_read (vio=0xfffffe00, buf=0xfffffe00
0xfffffe00 out of bounds>, size=-512) at viosocket.c:44 > > > #3 0x01d6d356 in vio_read_buff (vio=0x93e7c30, > > > buf=0xb6f00db4 > "~\202\215^-?\213\201?QJ\237\203\201\230D?\232V\201M8\233\224?\201\021-o\217[\200?#?\211\236\200?\203??0\213\021wC?h\211\236j\234?\215\210F]?\232\207\006Q\b\236\220\205?Dv\231t\204?8\220\224 > \203?-\214\216\236\202?$\026\210?\201?\203Y?u\221?v???\217?jS??\215?]\206??\213?P?\235?\212\023DS\230?\210 at 8\210\223d\206o-?\215?\204?$R\210z\202?\203\017??\230\236v??\026\226", > size=503704) > > > at viosocket.c:95 > > > #4 0x01d6e8f6 in my_real_read (net=0x93e71a8, complen=0xbff9efa8) at > net.c:804 > > > #5 0x01d6ec30 in my_net_read (net=0x93e71a8) at net.c:978 > > > #6 0x01d67b78 in net_safe_read (mysql=0x93e71a8) at client.c:596 > > > #7 0x01d6892f in cli_read_rows (mysql=0x93e71a8, > mysql_fields=0xaec8f00, fields=51) at client.c:1343 > > > #8 0x01d6a881 in mysql_store_result (mysql=0x93e71a8) at client.c:2697 > > > #9 0x01d4399f in store_result (obj=3071130920) at mysql.c:677 > > > #10 0x0805bcb4 in rb_call0 (klass=3071150400, recv=3071130920, id=27825, > oid=4294966784, argc=1, argv=0xbff9f270, body=0xb70e0058, flags=0) at > eval.c:5665 > > > #11 0x0805c017 in rb_call (klass=3071150400, recv=3071130920, mid=27825, > argc=1, argv=0xbff9f270, scope=0) at eval.c:6048 > > > #12 0x08058127 in rb_eval (self=3071126200, n=0x7af98) at ruby.h:654 > > > > > > _______________________________________________ > > > Mongrel-users mailing list > > > Mongrel-users at rubyforge.org > > > http://rubyforge.org/mailman/listinfo/mongrel-users > > > > > > > _______________________________________________ > > Mongrel-users mailing list > > Mongrel-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/mongrel-users > > > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users From eimorton at gmail.com Wed Apr 30 10:56:50 2008 From: eimorton at gmail.com (Erik Morton) Date: Wed, 30 Apr 2008 10:56:50 -0400 Subject: [Mongrel] Mongrel hangs in production. gdb stack included In-Reply-To: <466af3440804300751n50a18aa7r8501fa7f868b5f2b@mail.gmail.com> References: <5E151013-6346-40E5-B1FB-588F4C6EB014@gmail.com> <94A91EF2-5A28-4674-9975-4B3A204E7103@cheney.net> <197640DD-71D3-466E-A2A8-80BE8BE7673D@gmail.com> <466af3440804300751n50a18aa7r8501fa7f868b5f2b@mail.gmail.com> Message-ID: <78F5B1C0-8795-47D4-92FE-89DE65F4FBDB@gmail.com> No log rotation through ruby. I caught it just about an hour ago and show innodb status looked about as normal as a site under low volume could. No deadlocks. On Apr 30, 2008