[Mongrel] Unix Domain Sockets + Fork for improved scalability

Ezra Zygmuntowicz ezmobius at gmail.com
Tue May 6 00:30:49 EDT 2008

Hey Brian-

	This is pretty kick ass, I'll definitely take a look.


On May 5, 2008, at 8:37 PM, Brian Weaver wrote:
> Hello All,
> Ticket #30 (http://mongrel.rubyforge.org/ticket/30)  has a set of
> changes to mongrel that I've written on behalf of
> my employer, Raritan Computer, Inc. Also, these changes are released
> under the existing Ruby license used for Mongrel.
> Raritan is using mongrel internally for product development and was
> not able to get the level of scalability desired due to limitations
> Mongrel/Rails/Ruby. In particular the single threaded nature of
> processing rails request is a serious bottleneck when one or more long
> running requests could occur. Originally we used mongrel cluster to
> spin up a small number of mongrel rails instance; Apache mod_proxy was
> used to hand off requests to the various instances of mongrel.
> Unfortunately a long running request could cause our entire web
> application to "hang".  As these requests are being processing one or
> more mongrel instances continued to accept and enqueue new connections
> it can't immediately service.
> Mongrel cluster didn't provide any features for dynamically sizing the
> number of mongrel_rails instances necessary to keep the application
> running smoothly. Also, mongrels nature of continuing to enqueue
> connections while processing a long running request was a problem for
> our application. In essence we needed a smarter mongrel that would
> know which instances of a "cluster" were busy or free; if none were
> free then it should spin up a new instance.
> The attached patch does just that. It's definitely tailored for a Unix
> derivative platform and depends on the functionality of the UNIXSocket
> class to pass file descriptors between instances. It works in a
> master-slave type of server setup. The "master" is the initial
> instance of mongrel rails which goes through its normal initialization
> gyrations before forking mulitple "slave" instances to do the actual
> rails processing. The server listens on the defined TCP/IP address and
> port accepting request (from Apache in our case), determines which
> child if any is available, and then hands off the file descriptor for
> processing.
> The number of children can dynamically grow and shrink depending on
> how busy the application is being kept. The patch has fixed our issue
> with having long running requests hang the web application.
> The patch isn't without any warts. Due to the way that mongrel
> preforms initialization I found it difficult to follow the startup
> logic. Because of the convoluted
> contortions that occur during startup I was unable to
> make deeper changes due to time constraints which means that the
> following problem exists.
> * If you start the server as root and the configuration causes a
> setuid/setgid call to a non-root user then during shutdown mongrel may
> not be able to remove its pid file.
> I'm not aware of any other lurking problem, but I wouldn't be
> surprised if one or more exists.
> If you have any further question just send me a note or post to the
> list. I'll stay subscribed for a few weeks at least. Also, for the
> maintainers: I've added a copyright notice for Raritan Computer on
> those files that had extensive changes. I believe that is the correct
> way to handle major changes, let me know if I'm incorrect in my
> understanding.
> Enjoy
> -- Brian Weaver
> *sigh* here is the rest of the text I'm required to include
> -- 
> /* insert witty comment here */
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users

More information about the Mongrel-users mailing list