[Rubygems-developers] Updating source index is slow

[Eivind Eklund]

On Wed, Nov 10, 2004 at 08:42:10AM -0500, Jim Weirich wrote:
> On Wednesday 10 November 2004 07:39 am, Chad Fowler wrote:
> > You're right.  It grabs (and caches) all of the gem metadata.   Your
> > idea for a solution might be the most pragmatic (as opposed to
> > implementing rsync in ruby ;).
> 
> Do we use HEAD to get the time stamp of the file?  Then we would just need to 
> download it whenever it changes.

You already do something quite a bit like this; a GET is started, and
from this GET you retrieve the Content-Length.  If this match the cached
data, you use that; if not, you start a new GET and retrieve the file.

This is present from 0.8.0 and up; older clients will always download
the complete file.  I don't know your client distribution, but this may
be the primary source of the load described elsewhere in the thread.

Eivind.