[Mongrel] production-izing a popular site
rsanheim at gmail.com
Fri Dec 29 04:24:08 EST 2006
On 12/28/06, Ezra Zygmuntowicz <ezmobius at gmail.com> wrote:
> On Dec 28, 2006, at 5:45 PM, Rob Sanheim wrote:
> > Lets say you have a site that is serving, oh, around 100k unique
> > visitors a day (plain ole' browser requests) - plus probably ~ 40k
> > uniques to feeds. Assume this site is 90% read like most of the web,
> > so the traffic looks like your typical news or portal site. There are
> > two web boxes behind a hardware load balancer, each doing apache 2.2.3
> > -> mongrel_proxy_balancer -> mongrel clusters. One db machine, but
> > that shouldn't be stressed at all due to heavy caching of the content.
> > Where to being with performance testing something like this? How many
> > mongrels per web box to start with? Any comparable sites in
> > production to compare it to (Ezra?)
> > thanks,
> > - Rob
> Hey Rob-
> I am hosting a site that does about 110k unique visitors per day and
> about 250k page views/day. I would say it is a typical rails app with
> about 85% read and then some write and user contributed uploads and
> such. This app is pretty well optimized and doesn't have any requests
> except for file uploads that take longer then about 1.5 seconds to
> complete with most requests going out at about 35-45req/sec. Pretty
> typical rails app.
> It is running on two of our slices which are Xen VM's where there is
> a hardware load balancer up front, then nginx proxying to mongrel
> cluster on each node. It runs 3 mongrels on each VM for a total of 6.
> The database is on another super beefy machine but is not dedicated
> entirely to this app and is shared by others. I think this will be a
> good place for you to start load testing, 3 - 5 mongrels on each of
> your app servers. First get a baseline for your hardware by using ab
> or httperf on a fresh rails app that just does render :text =>
> 'hello' to get the best case scenario for a rails request on your
> hardware. This is the fastest you can expect rails to serve a
> dynamic request on your boxes. It will probably be around 100-140req/
> sec with sessions turned on. Then do a few more of this same
> benchmark against one mongrel and a few of your rails actions that
> represent short and long requests for your app. Write these results
> down or save them as you go. This is important because I discovered
> that 3 mongrels performs *better* then 5 mongrels on our
> architecture. This will vary widely though over different os and
> hardware configurations.
> Now its time to add more mongrels until you hit a point of
> diminishing returns or you are confident in your ability to handle
> the traffic you expect plus some. ounds like you have two nice boxes
> for your app servers so start with 3 mongrels on one of the servers.
> Get this benchmark for the render :text action as well as the other
> actions you benched in your real app. At this point you should get
> more req/sec when you measure with concurrent users at 20-50. Then
> add one mongrel at a time and measure until adding one more mongrel
> does not buy you any more performance. This is your sweet spot where
> you are maxing out your proxy_balancers efficiency.
> Now that you hit the sweet spot for one app server setup the second
> one the same way and do some more benches at high concurrency. If you
> have good hardware load balancers you should get a good improvement
> with the two app servers over just one of them. In reality software
> load balancers don't multiply the perf of a single mongrel by the
> number of mongrels. Its more like adding another mongrel may get you
> 20-50% more until you hit the sweet spot and can't get anymore. But
> the good hardware load balancers are more efficient and get closer to
> 50-80% improvement when adding more nodes. Sometimes it's nice to go
> straight from the hardware load balancers right to mongrel and use an
> asset_host for static content. I don't think you will need this for
> the app you are talking about.
> I suspect you will end up with 4-6 mongrels per app server and this
> will leave you some overhead for what you need to serve. Another
> point of reference is the http://yakimaherald.com gets about 75-90k
> page views/day on 3 mongrels behind apache22 on an xserve.
> -- Ezra Zygmuntowicz
> -- Lead Rails Evangelist
> -- ez at engineyard.com
> -- Engine Yard, Serious Rails Hosting
> -- (866) 518-YARD (9273)
Ezra, you the man.
I'm gonna read this over a few more times when its not 3 am, and I'm
sure it will be very helpful with my current project.
thanks a lot,
More information about the Mongrel-users