[Rubygems-developers] Updating source index is slow

Richard Kilmer rich at infoether.com
Wed Nov 10 08:17:59 EST 2004


yaml.Z is a compressed file...that is the one downloaded by default (not the
source 'yaml' file).

I will look into downloading just the updated gemspecs and see if that can
be used to speed things up.  It would obviously be a cgi...wanted to stay
away from that...but CPU is cheeper than bandwidth right now!

-rich


On 11/10/04 8:11 AM, "Hugh Sasse Staff Elec Eng" <hgs at dmu.ac.uk> wrote:

> On Wed, 10 Nov 2004, Chad Fowler wrote:
> 
>>> I may be ignorant here -- I'm assuming the client is hitting
>>> http://gems.rubyforge.org/yaml
>>> 
>>> 
>> 
>> You're right.  It grabs (and caches) all of the gem metadata.   Your idea for
>          [...]
>> How big an issue does it seem to be?  For me, the gem repository doesn't
>> update (and therefore I don't have to redownload the yaml) enough to be a
>> serious drag, but I can see how as more gems are released it will get more
>> annoying on the client.
> 
> A quick glance through the code suggests that it doesn't try to use
> compression by default. HTTP headers like:
> 
>   "accept-encoding" => "gzip;q=1.0, " + "identity; q=0.5, " + "*;q=0 "
> 
> which we can handle with things in the standard library now, something like:
> 
>    if answer.key?("content-encoding")
>      case answer["content-encoding"]
>      when 'gzip'
>        gz = Zlib::GzipReader.new(StringIO.new(answer.read_body))
>        body = gz.read
>      [...]
>      end
>      [...]
>    end
> 
> [Cut, pasted and pruned from my hacked version of rubric]
>> 
>> Chad
> 
>          Hugh
> _______________________________________________
> Rubygems-developers mailing list
> Rubygems-developers at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rubygems-developers
> 




More information about the Rubygems-developers mailing list