[Rubygems-developers] Detecting server changes

Hugh Sasse Staff Elec Eng hgs at dmu.ac.uk
Mon Nov 29 07:13:47 EST 2004

On Fri, 26 Nov 2004, Jim Weirich wrote:

> There was some discussion a while back on the efficiency of detecting changes
> on the server.  I was doing a little experimentation and got the following
> results ...
         [interesting study trimmed]
Thanks for doing that study.
> All of the shortcuts (HEAD, IMS(no) and open-uri/sz) all take approximately
> the same amount of time, and all are much faster than downloading the entire
> compressed source.
> Based on that, I don't think there is a strong compeling reason based on these
> benchmarks to switch from the open-uri/sz technique that is in use currently.
> (although the size technique can fail with non-monotonically increasing file
> sizes, in which case the IMS technique looks attractive).

And the IMS one agrees more with the specs for how we should be
doing this, and if the web server changes then our assumptions may
break. But if we flag that up somewhere, we should be able to find
it if/when that happens months down the road.
> But, there is a problem with the current code.  On many systems, the cache is
> in an area that can only be written while running under sudo.  That means the
> cache is updated on installs, but not on general queries.  This forces the
> gem command to download the yaml source on every command unless a recent
> install has occurred.
> I'm suggesting that we store the cached results in the users's home directory
> (or appropriate directory on windows).  This allows a gem list command to
> update the cache without running under sudo.  We could act intelligently and
> only store in the home directory if the normal cache directory is write
> protected.  We could also check both the system-wide cache and the
> user-specific cache when comparing against the server.

More efficient, but less secure.  If gems is installed centrally
then any users can use it, even if they are not trusted to touch
system files.  If the data from home directories becomes trusted
then some of our students could DOS attack the gem database.  Unless
you are proposing that the user's own database may be updated from
the system one, but not the other way round.

Is it sensible to have a special process to do the database mods that
can be requested by normal users, so that their processes never
writes (possibly suspect) data to the database?  Maybe not root,
but user gem or rubygems (8 chars in a username is pretty std for
unix) so that this user owns the files rubygems depends on.

> Any thoughts before I dive into this?
> -- -- Jim Weirich    jim at weirichhouse.org
> http://onestepback.org


More information about the Rubygems-developers mailing list