Gem file size limits

Evan Phoenix evan at phx.io
Tue Jan 17 18:10:32 EST 2012


I believe that rubygems.org needs to limit the max size of a .gem file which will be allowed.

This serves two purposes:  
1) It protects users from themselves. The top 19 of 20 gems sorted by size are all huge because they accidentally packaged all previous versions within themselves. This issue needs to be fixed on the gem build side also, but there is no reason to allow these gems.
2) Cost. Rubygems.org is becoming increasingly expensive to run and thus we need to begin thinking of ways to keep it mean and lean.

I think we can all agree that some kind of limit makes sense. At the moment, there is nothing from preventing a user from using rubygems.org as their personal backup and pushing terabytes in a .gem file. Clearly we can't operate if people do that.

So the natural question I have for all of you is: what makes sense as the size limit? To help you with this decision, here is some data for you to chew on:

1) The top 1000 gems, sorted by size: https://gist.github.com/1629309 
2) A histogram of gem sizes by megabyte: https://gist.github.com/1629435

You can see from the histogram that 96% of gems are less than one megabyte, and 98% are 3 megs or less. It seems like that fact should inform our decision.

To start the decision, let me throw out a starting point: 10 megs.

Looking at the biggest non-accidental gems, they're almost all jruby related and contain huge .jar files. We've pinged others about removing the impediment to pushing gems with maven deps and thusly devs would use that functionality rather than packaging the jars within the gems themselves.

Comments and Criticisms Required.

 - Evan

-- 
Evan Phoenix // evan at phx.io




More information about the RubyGems-Developers mailing list