[Rubygems-developers] New Beta Gem with Better Memory Footprint (was: Bulk updates)
ryand-ruby at zenspider.com
Fri Jun 1 11:36:27 EDT 2007
On Jun 1, 2007, at 08:24 , Jim Weirich wrote:
> I've taken Ryan's changes (removing a copy step in the bulk download)
> and added a few more for a beta release of RubyGems.
> Here are the changes:
> * Bulk downloads no longer copy the YAML string before loading it.
> * Bulk downloads don't kick in until more than 500 gems need resyncing
> (the old limit was 50).
> * Users may set the bulk download threshhold in the command line (-B)
> or in the .gemrc file.
Is this going to thrash rubyforge.org with the number of connections?
> Here are the memory stats using Ruby 1.8.5 on Mac OSX:
> Old Version Bulk: 107Mb Real, 146Mb Virtual
> New Version Bulk: 98Mb Real, 130Mb Virtual
> New Version Incremental 36Mb Real, 63Mb Virtual
For the record, I got much better numbers than this. I'm not sure
what the differentiating factor is. I had 120Mb -> 73Mb for the first
I'm not sure how/why the incremental would be that much better. I
proposed sticking with bulk but letting go of the previous cache for
that server so it can be GC'd. I think this would be a good balance
between # of socket connections and memory consumption.
Also Jim, I just committed a change to properly skip those files with
the bad names. Please scoop that up as well.
More information about the Rubygems-developers