[Mongrel] Why Rails + mongrel_cluster + load balancing doesn't work for us and the beginning of a solution

Paul Butcher paul at paulbutcher.com
Wed Sep 20 16:14:40 EDT 2006


Zed Shaw <zedshaw at zedshaw.com> wrote
> But, if your interface is time sensitive then why does it have actions
> that take too much time?  See the conundrum there?

I was kinda expecting that response, Zed, but I didn't want to rob you of
the pleasure of saying it ;-)

We are in fact using backgroundrb for a number of long-running actions. It's
fantastic, but it isn't ever going to solve the problem for us completely.
There are a whole bunch of reasons, which may be specific to our particular
application, but I doubt it.

First the less convincing arguments:

1) We're lazy (in a good way). Our application is used by hundreds of our
employees, 24 hours a day. We, correctly, invest a great deal of time making
sure that the things that they do over and over again are as efficient as
they possibly can be. We don't, however, believe that it's appropriate for
us to spend time optimising things which are only used by one or two guys
once or twice a day. It's not as if we have a lack of things to spend our
time on, and we'd rather spend that time on things that really matter,
rather than optimising things which only need to be optimised because the
performance characteristics of Rails mean that one poorly performing action
results in *all* actions performing poorly.

2) We're not infallible. Sometimes we screw up and release a version of the
software where one of our actions takes longer than it should. This is bad
if it only affects the action we screwed up, but if that's all it does then
it's not a disaster. If one slow action screws up *all* the actions,
however, that is a disaster.

3) Even if we spend all of the time we possibly can optimising actions,
different actions will always take different amounts of time to execute.
It's not as if there's "one true" execution time for an action. The bad
effects become apparent whenever you have any two actions which take
noticeably different amounts of time to execute. Yes, in the example that I
gave it was 1 second and 10 seconds. But the same basic effect would be
there if we were talking about 1ms and 10ms. 

Those arguments would be enough, I believe. But in our case there's one much
stronger argument which trumps all the above IMHO:

4) It's not possible to predict the time that an action, even a heavily
optimised one, will take. The vagaries of cacheing, swapping etc mean that
this is simply out of our hands. In our case, for example, many of our
actions involve fulltext searches of the database (we have no choice about
this - it's fundamental to the nature of what we do). The performance of a
fulltext search is *extremely* unpredictable (in MySQL anyway). There can be
occasions where the first time a particular search is done, it takes 10
seconds. The second time, because the database has "got the idea", it can
take a few milliseconds. Backgroundrb cannot solve this problem - if we sent
all of these actions to backgroundrb, pretty much *every* action would end
up being sent to backgroundrb and then we end up with a very similar load
balancing problem - it just becomes a problem of load balancing to
backgroundrb instead of to mongrel.

Make sense?

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

MSN: paul at paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher
LinkedIn: https://www.linkedin.com/in/paulbutcher




More information about the Mongrel-users mailing list