[Rubygems-developers] Updating source index is slow

Gavin Sinclair gsinclair at soyabean.com.au
Wed Nov 10 08:24:08 EST 2004


On Wednesday, November 10, 2004, 11:39:17 PM, Chad wrote:


> On 09-Nov-04, at 9:47 PM, Patrick May wrote:

>>
>> On Tuesday, November 9, 2004, at 02:04  AM, Gavin Sinclair wrote:
>>
>>> Updating the source index from the rubyforge server is slow.  This
>>> could be sped up if there were a pure-Ruby rsync implementation
>>> available (for client and server).  Discuss.
>>
>> Does the rubygems client need the entire source index?  It seems to me
>> that one would be more likely to ask for a list of packages, then 
>> drill individually into the metadata for particular pages.
>>
>> I may be ignorant here -- I'm assuming the client is hitting 
>> http://gems.rubyforge.org/yaml
>>
>>

> You're right.  It grabs (and caches) all of the gem metadata.   Your
> idea for a solution might be the most pragmatic (as opposed to 
> implementing rsync in ruby ;).

Nice and pragmatic, but I think the client should have all the
metadata, all else being equal, to enable greater searches.
Especially in GUI clients.

Aside: I'm getting annoyed with being asked about dependencies one by
one.  The entire dependency tree for a gem is knowable upfront, so it
would be nice to OK them all at once.  That can only happen if the
client has the metadata.

I agree implementing rsync is not worth it.  But were it to exist by
some other means... :)

> How big an issue does it seem to be?  For me, the gem repository 
> doesn't update (and therefore I don't have to redownload the yaml) 
> enough to be a serious drag, but I can see how as more gems are 
> released it will get more annoying on the client.

I notice it a lot.  A substantial download every time a gem gets
added... (about 30 of them since RubyConf).

Gavin




More information about the Rubygems-developers mailing list