[Mongrel] Non-Cache Handling for Bots
shawn at brmethod.com
Wed Jun 10 22:26:48 EDT 2009
"Serving up different results based on user agent may cause your site to be
perceived as deceptive and removed from the Google index."
From: mongrel-users-bounces at rubyforge.org
[mailto:mongrel-users-bounces at rubyforge.org] On Behalf Of Mike Papper
Sent: Wednesday, June 10, 2009 8:42 PM
To: mongrel-users at rubyforge.org
Subject: Re: [Mongrel] Non-Cache Handling for Bots
I am talking about Rails standard page-caching mechanism. Rails by
default puts full pages into public/... and if mongrel sees them
there, it serves them (without running rails dispatch et al). This is
fine for normal but not good for the bot user agents.
Here is a new solution:
1) set rails to cache in public/cache
2) Use Apache rewrite to serve these files directly (if found)
3) If not found, pass to mongrel which will not find the cached files
either since MONGREL ONLY LOOKS IN public for cached files. Mongrel
does not honor the config.action_controller.page_cache_directory rails
4) Rails processes the file and puts it into public/cache/...
...on the next request, apache serves from cache.
I am working on the reqwrite rules etc. for this.
On Jun 10, 2009, at 6:16 PM, mcr at simtone.net wrote:
> What kind of caches are you talking about?
> Are these full page caches? The kind that get stored into /public?
> My question is, can you instead of adding /robot to the URL when
> finds the robot, can you instead change Apache's DocumentRoot?
> It seems to be that this would prevent apache from finding the cached
> page. Also, if you actually point to another copy of your /public,
> could get the normal static pages...
> I think perhaps since you are talking about changing mongrel's caching
> behaviour that you aren't talking about the page caches that get
> into /public. (Well, I'm rusty on terminology here)
> Michael Richardson <mcr at simtone.net>
> Director -- Consumer Desktop Development, Simtone Corporation,
> Ottawa, Canada
> Personal: http://www.sandelman.ca/mcr/
> SIMtone Corporation fundamentally transforms computing into simple,
> secure, and very low-cost network-provisioned services pervasively
> accessible by everyone. Learn more at www.simtone.net and
Mongrel-users mailing list
Mongrel-users at rubyforge.org
More information about the Mongrel-users