[Mongrel] Non-Cache Handling for Bots

mcr at simtone.net mcr at simtone.net
Wed Jun 10 22:20:08 EDT 2009

>>>>> "Mike" == Mike Papper <bodaro at gmail.com> writes:
    Mike> I am talking about Rails standard page-caching mechanism. Rails by  default
    Mike> puts full pages into public/... and if mongrel sees them  there, it serves
    Mike> them (without running rails dispatch et al). This is  fine for normal but not
    Mike> good for the bot user agents.

right, so that's what I thought you were talking about.
Only, it's not mongrel that serves up the pages, but Apache, usually.
In your apache config, you have something like:

        # Rewrite all non-static requests to cluster
        RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
        RewriteRule ^/(.*)$ balancer://spartan_cluster%{REQUEST_URI} [P,QSA,L]

which basically serves up any files found in /public, otherwise, punts
to the mongrel.    I thought that rails put the files directly there for
apache to use/see. (there are caveats if your mongrel and apache do not
share the same file system, such as because they are on different machines)

If you are telling me that actually mongrel does this, it's news to me.

    Mike> Here is a new solution:
    Mike> 1) set rails to cache in public/cache
    Mike> 2) Use Apache rewrite to serve these files directly (if found)
    Mike> 3) If not found, pass to mongrel which will not find the cached files  either
    Mike> since MONGREL ONLY LOOKS IN public for cached files. Mongrel  does not honor
    Mike> the config.action_controller.page_cache_directory rails  setting
    Mike> 4) Rails processes the file and puts it into public/cache/...

    Mike> ...on the next request, apache serves from cache.

    Mike> I am working on the reqwrite rules etc. for this.

So, basically have apache pick a different cache location when it sees a

Michael Richardson <mcr at simtone.net>
Director -- Consumer Desktop Development, Simtone Corporation, Ottawa, Canada
Personal: http://www.sandelman.ca/mcr/ 

SIMtone Corporation fundamentally transforms computing into simple,
secure, and very low-cost network-provisioned services pervasively
accessible by everyone.  Learn more at www.simtone.net and www.SIMtoneVDU.com 

More information about the Mongrel-users mailing list