[Mongrel] Unix Domain Sockets + Fork for improved scalability

Ezra Zygmuntowicz ezmobius at gmail.com
Tue May 6 00:30:49 EDT 2008


Hey Brian-

	This is pretty kick ass, I'll definitely take a look.

-Ezra	

On May 5, 2008, at 8:37 PM, Brian Weaver wrote:
> Hello All,
>
> Ticket #30 (http://mongrel.rubyforge.org/ticket/30)  has a set of
> changes to mongrel that I've written on behalf of
> my employer, Raritan Computer, Inc. Also, these changes are released
> under the existing Ruby license used for Mongrel.
>
> Raritan is using mongrel internally for product development and was
> not able to get the level of scalability desired due to limitations
> Mongrel/Rails/Ruby. In particular the single threaded nature of
> processing rails request is a serious bottleneck when one or more long
> running requests could occur. Originally we used mongrel cluster to
> spin up a small number of mongrel rails instance; Apache mod_proxy was
> used to hand off requests to the various instances of mongrel.
>
> Unfortunately a long running request could cause our entire web
> application to "hang".  As these requests are being processing one or
> more mongrel instances continued to accept and enqueue new connections
> it can't immediately service.
>
> Mongrel cluster didn't provide any features for dynamically sizing the
> number of mongrel_rails instances necessary to keep the application
> running smoothly. Also, mongrels nature of continuing to enqueue
> connections while processing a long running request was a problem for
> our application. In essence we needed a smarter mongrel that would
> know which instances of a "cluster" were busy or free; if none were
> free then it should spin up a new instance.
>
> The attached patch does just that. It's definitely tailored for a Unix
> derivative platform and depends on the functionality of the UNIXSocket
> class to pass file descriptors between instances. It works in a
> master-slave type of server setup. The "master" is the initial
> instance of mongrel rails which goes through its normal initialization
> gyrations before forking mulitple "slave" instances to do the actual
> rails processing. The server listens on the defined TCP/IP address and
> port accepting request (from Apache in our case), determines which
> child if any is available, and then hands off the file descriptor for
> processing.
>
> The number of children can dynamically grow and shrink depending on
> how busy the application is being kept. The patch has fixed our issue
> with having long running requests hang the web application.
>
> The patch isn't without any warts. Due to the way that mongrel
> preforms initialization I found it difficult to follow the startup
> logic. Because of the convoluted
> contortions that occur during startup I was unable to
> make deeper changes due to time constraints which means that the
> following problem exists.
>
> * If you start the server as root and the configuration causes a
> setuid/setgid call to a non-root user then during shutdown mongrel may
> not be able to remove its pid file.
>
> I'm not aware of any other lurking problem, but I wouldn't be
> surprised if one or more exists.
>
> If you have any further question just send me a note or post to the
> list. I'll stay subscribed for a few weeks at least. Also, for the
> maintainers: I've added a copyright notice for Raritan Computer on
> those files that had extensive changes. I believe that is the correct
> way to handle major changes, let me know if I'm incorrect in my
> understanding.
>
> Enjoy
>
> -- Brian Weaver
>
> *sigh* here is the rest of the text I'm required to include
>
> THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED
> WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
> MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
>
>
>
> -- 
>
> /* insert witty comment here */
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users




More information about the Mongrel-users mailing list