[Rubygems-developers] More work on the remote installer

[Chad Fowler]

On 01-Dec-04, at 9:01 AM, Jim Weirich wrote:

> I am continuing my work on the remote installer.  Recent changes 
> include :
>
> (1) proxy support added back to the RemoteSourceFetcher.  We lost proxy
> support when I switched to using an HTTP HEAD request.  We now have 
> that
> back.  In addition, there are several unit tests covering proxy 
> support in
> RemoteSourceFetcher.  The unit tests involve firing up two web services
> during the unit test.  I would appreciate if people try it and and 
> report any
> problems.
>

Worked great for me!

> (2) The YAML file retrieved from the remote source is "reduced" before 
> passing
> it to YAML.load.  I was having a terrible time with large YAML files 
> and the
> YAML loader crashing.  Reducing the size of the YAML works around this 
> bug,
> but does not eliminate the root case.  The YAML file is reduced by 
> removing
> the files list from all the gemspecs.  This reduced the size of the 
> YAML file
> by 60% (IIRC).  Since the files list is not used in the source info 
> cache, I
> don't anticipate any problems.  We might consider reducing the YAML
> file /before/ sending it over the wire to reduce bandwith usage.
>

Not a bad idea.  We're not currently serving the yaml dynamically, but 
I bet it wouldn't take much to change the reaper script.  Rich has so 
much free time these days, he could probably get it done today ;) (just 
kidding).  I'm thinking we can actually make this change on the server 
without breaking existing code.  Is that right as per your 
understanding?


> Next on my hit list ...
>
> (A) Fix the test_install method in test_remote_installer, which was 
> broken
> (and currently disabled) by the RemoteSourceFetcher changes.
>
> (B) Add support for a writable per-user source info cache for use when 
> the
> global cache cannot be written.j
>
> (C) Continued refactoring of the remote installer.
>

Great stuff, Jim.  This is definitely the area that needs the most help.

Chad