[Rubygems-developers] Detecting server changes
jim at weirichhouse.org
Fri Nov 26 23:51:34 EST 2004
There was some discussion a while back on the efficiency of detecting changes
on the server. I was doing a little experimentation and got the following
user system total real
HEAD 0.020000 0.000000 0.020000 ( 0.115111)
IMS(no) 0.000000 0.000000 0.000000 ( 0.109897)
IMS(yes) 0.180000 0.020000 0.200000 ( 1.952789)
NMod 0.160000 0.000000 0.160000 ( 2.156582)
open-uri 0.290000 0.030000 0.320000 ( 2.112810)
open-uri/sz 0.000000 0.000000 0.000000 ( 0.146552)
HEAD -- Just request the headers.
IMS(no) -- Use If-Modified-Since (where the answer is no).
IMS(yes) -- Use If-Modified-Since (where the answer is yes).
NMod -- No IMS header (normal HTTP get)
open-uri -- Using open-uri straight up
open-uri/sz -- Using open-uri will the break on content length.
All of the shortcuts (HEAD, IMS(no) and open-uri/sz) all take approximately
the same amount of time, and all are much faster than downloading the entire
Based on that, I don't think there is a strong compeling reason based on these
benchmarks to switch from the open-uri/sz technique that is in use currently.
(although the size technique can fail with non-monotonically increasing file
sizes, in which case the IMS technique looks attractive).
But, there is a problem with the current code. On many systems, the cache is
in an area that can only be written while running under sudo. That means the
cache is updated on installs, but not on general queries. This forces the
gem command to download the yaml source on every command unless a recent
install has occurred.
I'm suggesting that we store the cached results in the users's home directory
(or appropriate directory on windows). This allows a gem list command to
update the cache without running under sudo. We could act intelligently and
only store in the home directory if the normal cache directory is write
protected. We could also check both the system-wide cache and the
user-specific cache when comparing against the server.
Any thoughts before I dive into this?
-- Jim Weirich jim at weirichhouse.org http://onestepback.org
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
More information about the Rubygems-developers