Bugs: Browse | Submit New | Admin

[#27154] Computed hash is sometimes too large.

Date:
2009-09-21 14:42
Priority:
3
Submitted By:
Ryan Riley (panesofglass)
Assigned To:
John Barnette (jbarnette)
Category:
`gem install` command
State:
Deleted
Summary:
Computed hash is sometimes too large.

Detailed description
The current hash algorithm in dependency.rb (line 138) and specification.rb (line 658) can sometimes create a hash that
is too large. This is also possible with Array#hash in MRI, which can also misbehave if one of the array elements returns
a large hash value:

class C
  def hash
    100000000000000000000
  end
end

[C.new].hash  # => in `hash': bignum too big to convert into 
`long' (RangeError)

REXML has a similar issue. http://redmine.ruby-
lang.org/issues/show/1883 tracks the issue, and MRI will be 
fixing the issue.

Suggested fixes are:

edit: c:/ruby/libs/ruby/site_ruby/1.8/rubygems/dependency.rb;C840659
File: dependency.rb
===================================================================
--- c:/ruby/libs/ruby/site_ruby/1.8/rubygems/dependency.rb;C840659  (server)    6/23/2009 1:21 PM
+++ c:/ruby/libs/ruby/site_ruby/1.8/rubygems/dependency.rb
@@ -112,7 +112,7 @@
   end
 
   def hash # :nodoc:
-    name.hash + type.hash + version_requirements.hash
+    name.hash ^ type.hash ^ version_requirements.hash
   end
 
 end
===================================================================
edit: c:/ruby/libs/ruby/site_ruby/1.8/rubygems/specification.rb;C908357
File: specification.rb
===================================================================
--- c:/ruby/libs/ruby/site_ruby/1.8/rubygems/specification.rb;C908357  (server)    6/23/2009 1:24 PM
+++ c:/ruby/libs/ruby/site_ruby/1.8/rubygems/specification.rb
@@ -661,9 +661,8 @@
     private :same_attributes?
 
     def hash # :nodoc:
-      @@attributes.inject(0) { |hash_code, (name, default_value)|
-        n = self.send(name).hash
-        hash_code + n
+      @@attributes.inject(612553) { |hash_code, (name, default_value)|
+        hash_code ^ self.send(name).hash
       }
     end
 
===================================================================

Add A Comment: Notepad

Please login


Followup

Message
Date: 2011-03-01 23:41
Sender: Bernard Lambeau

Hi Ryan,

I'm sorry if my posts seem aggressive, this is far from being
an objective in itself.

I'm far from being sure that "Upgrade ruby and the problem
is fixed" is the appropriate answer to a problem users encounter
in practice. Many users are much less skilled than you are and
are just afraid by obtrusive changes.

In my opinion, people upgrading ruby version should be a middle-term
community goal, backed up with well  documented good practices
like "ruby 1.8.7 p248 is considered as a must have, prior
versions are considered too buggy in practice".

Rubygems is certainly a good place to broadcast such messages,
... slowly (as an example I certainly support the recent broken
support of ruby 1.8.6 by influencing gems).

Recent major changes in rubygems are certainly required and welcome(d)
for long-term goals around ruby. Too many such changes will make
users' life a nightmare.

Anyway, maybe I'm just wrong. In that case, please apologize.
Date: 2011-03-01 22:35
Sender: Ryan Davis

Bernard,

Might I suggest reading a copy of Zig ZIgler's How to Win Friends
and Influence People? Because you're failing.

Your passive-agressive barb of a blog post seems to gloss over
the fact that this bug is a bug in ruby, not a bug in the libraries.
Not only that, but the bug has been fixed (and released) for
over a year. It is not in any author's best interest to maintain
hack after hack to deal with any and every incompatibility with
every version of ruby ever released.

We're calling #hash or an array of arrays of strings and ints.
If #hash returns a BigNum, that's a bug... in ruby. Upgrade ruby
and the problem is fixed.
Date: 2011-03-01 18:54
Sender: Bernard Lambeau

This bug seems to re-appear. requirement.rb should probably be
patched as proposed by Daniel Weaver to guarantee compatibility
with ruby 1.8.7-p176.

I've written a review with additional details about this bug
at: http://revision-zero.org/history-of-a-bug
Date: 2011-02-02 00:03
Sender: Eric Hodel

This is fixed.
Date: 2010-12-20 20:20
Sender: Daniel Weaver

I realize these are bugs in the underlying Ruby implementation
of hash, but it really helps us to have them fixed in gem.

This is the patch for requirement.rb that we currently need to
deploy to make bundler happy with our apps:

zlerntop:rubygems danielweaver$ diff -u requirement.rb.orig
requirement.rb
--- requirement.rb.orig	2010-12-20 12:13:40.000000000 -0800
+++ requirement.rb	2010-12-20 12:13:40.000000000 -0800
@@ -106,7 +106,7 @@
   end
 
   def hash # :nodoc:
-    requirements.hash
+    requirements.inject(0) { |h, r| h ^ r.first.hash ^ r.last.hash
}
   end
 
   def marshal_dump # :nodoc:
Date: 2010-12-07 04:06
Sender: George Mendoza

Hello again,

I got the bignum error again, but this time with cucumber-0.9.4.
These commands weren't able to install the gem:

  * bundle install
  * bundle install --local
  * sudo bundle install --local
  * sudo bundle install --local --system
  * sudo bundle install --system
  
However, I was able to work around the issue by

  1. installing the gem to my system manually (sudo gem install
cucumber --version "0.9.4"), and
  2. installing the bundle to the system (sudo bundle install
--system)

For step 2, trying "bundle install --local" worked
only if all the gems in the bundle are already installed. If
there were some gems from the bundle that are not installed,
running "sudo bundle install --system" was the only
way I could install them.

George Mendoza
Philippines
Date: 2010-12-01 03:52
Sender: George Mendoza

Hello,

Appreciate if anybody can re-open this ticket. I am also getting
the error when I try to install the dmitryv-backup gem via bundle.
Please let me know if I need to provide any other information
in order to fix this bug. Thanks!

Gemfile:
----------------------------------------
source :rubygems
source "http://gems.github.com"

#...

gem 'dmitryv-backup', '2.4.0', :require => "backup"

#...

----------------------------------------

Running:
----------------------------------------
bundle install --local
/usr/local/lib/site_ruby/1.8/rubygems/requirement.rb:109:in `hash':
bignum too big to convert into `long' (RangeError)
  from /usr/local/lib/site_ruby/1.8/rubygems/requirement.rb:109:in
`hash'
  from /usr/local/lib/site_ruby/1.8/rubygems/specification.rb:675:in
`hash'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/inde
x.rb:36:in `inject'
  from /usr/local/lib/site_ruby/1.8/rubygems/specification.rb:674:in
`each'
  from /usr/local/lib/site_ruby/1.8/rubygems/specification.rb:674:in
`inject'
  from /usr/local/lib/site_ruby/1.8/rubygems/specification.rb:674:in
`hash'
  from /usr/lib/ruby/1.8/tsort.rb:204:in `include?'
  from /usr/lib/ruby/1.8/tsort.rb:204:in
`each_strongly_connected_component_from'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:130:in `tsort_each_child'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:130:in `each'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:130:in `tsort_each_child'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:128:in `each'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:128:in `tsort_each_child'
  from /usr/lib/ruby/1.8/tsort.rb:203:in
`each_strongly_connected_component_from'
  from /usr/lib/ruby/1.8/tsort.rb:209:in
`each_strongly_connected_component_from'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:130:in `tsort_each_child'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:130:in `each'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:130:in `tsort_each_child'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:128:in `each'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:128:in `tsort_each_child'
  from /usr/lib/ruby/1.8/tsort.rb:203:in
`each_strongly_connected_component_from'
  from /usr/lib/ruby/1.8/tsort.rb:182:in
`each_strongly_connected_component'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:124:in `tsort_each_node'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:124:in `each'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:124:in `tsort_each_node'
  from /usr/lib/ruby/1.8/tsort.rb:180:in
`each_strongly_connected_component'
  from /usr/lib/ruby/1.8/tsort.rb:148:in `tsort_each'
  from /usr/lib/ruby/1.8/tsort.rb:135:in `tsort'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:107:in `sorted'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/spec
_set.rb:12:in `each'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/inst
aller.rb:44:in `run'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/inst
aller.rb:8:in `install'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/cli.
rb:225:in `install'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/vend
or/thor/task.rb:22:in `send'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/vend
or/thor/task.rb:22:in `run'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/vend
or/thor/invocation.rb:118:in `invoke_task'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/vend
or/thor.rb:246:in `dispatch'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/vend
or/thor/base.rb:389:in `start'
  from /usr/lib/ruby/gems/1.8/gems/bundler-1.0.7/bin/bundle:13
  from /usr/bin/bundle:19:in `load'
  from /usr/bin/bundle:19
----------------------------------------

Thanks,

George Mendoza
Philippines
Date: 2010-11-12 22:46
Sender: Ryan Davis

This ticket has been deemed stale and we're closing it in order
to catch up
with our ticket list. If you think it is still valid, please
reopen.
Date: 2010-10-18 21:08
Sender: Craig Cook

Hello,

I tried this patch, but I am still getting the error, at the
same place in the code.  The error is the same with or without
the patch to specification.rb; dependency.rb was already like
the patched part shown above.  This happened when installing
using bundler, executed via capistrano.

We  had been using hpricot, it wasn't needed, it was giving this
error first.  I took it out, but then the same  RangeError happened
with mysql.  I've seen it before also with nokogiri.  (Always
with native gems, though).

Here's the relevant part of the log:
----------- begin log ----------------------------
 ** [out :: 192.168.10.63] Installing mysql (2.8.1)
 ** [out :: 192.168.10.63] with native extensions
 ** [out :: 192.168.10.62]
/usr/lib/ruby/site_ruby/1.8/rubygems/requirement.rb:109:in `hash'
 ** [out :: 192.168.10.62] :
 ** [out :: 192.168.10.62] bignum too big to convert into `long'
 ** [out :: 192.168.10.62] (
 ** [out :: 192.168.10.62] RangeError
 ** [out :: 192.168.10.62] )
 ** [out :: 192.168.10.62] from
/usr/lib/ruby/site_ruby/1.8/rubygems/requirement.rb:109:in `hash'
 ** [out :: 192.168.10.62] from
/usr/lib/ruby/site_ruby/1.8/rubygems/specification.rb:675:in
`hash'
 ** [out :: 192.168.10.62] from /usr/lib/ruby/1.8/fileutils.rb:243:in
`inject'
 ** [out :: 192.168.10.62] from
/usr/lib/ruby/site_ruby/1.8/rubygems/specification.rb:674:in
`each'
 ** [out :: 192.168.10.62] from
/usr/lib/ruby/site_ruby/1.8/rubygems/specification.rb:674:in
`inject'
 ** [out :: 192.168.10.62] from
/usr/lib/ruby/site_ruby/1.8/rubygems/specification.rb:674:in
`hash'
 ** [out :: 192.168.10.62] from /usr/lib/ruby/1.8/tsort.rb:205:in
`include?'
 ** [out :: 192.168.10.62] from /usr/lib/ruby/1.8/tsort.rb:205:in
`each_strongly_connected_component_from'
 ** [out :: 192.168.10.62] ... 22 levels...
 ** [out :: 192.168.10.62] from
/usr/lib/ruby/gems/1.8/gems/bundler-1.0.2/lib/bundler/vendor/thor
/base.rb:389:in `start'
 ** [out :: 192.168.10.62] from
/usr/lib/ruby/gems/1.8/gems/bundler-1.0.2/bin/bundle:13
 ** [out :: 192.168.10.62] from /usr/bin/bundle:19:in `load'
 ** [out :: 192.168.10.62] from /usr/bin/bundle:19
----------- end cap output -------------------


running:
------------ system & ruby info ------------------
[ccook@ev2-stage ~]$ ruby -v
ruby 1.8.7 (2010-08-16 patchlevel 302) [i686-linux]
[ccook@ev2-stage ~]$ gem  -v
1.3.7
[ccook@ev2-stage ~]$ uname -a
Linux ev2-stage 2.6.18-194.11.4.el5xen #1 SMP Tue Sep 21 06:28:04
EDT 2010 i686 i686 i386 GNU/Linux
------ end system / ruby info  ----------

Gemfile:

----------- begin Gemfile -----------------
  source :gemcutter
             
  gem 'rails',          '2.3.8'    
  gem 'mysql',          '2.8.1'
  gem "geokit"
  gem "geokit-rails" 
  gem "builder"
  gem 'chronic'
  gem 'whenever'
  gem 'database_cleaner'       
  gem 'apn_on_rails'
  gem 'factory_girl'
  gem 'net-sftp'   
  gem 'will_paginate'    # dependency for squirrel & jqgrid
  gem 'nokogiri'           # only for
test/functional/auth_controller_test.rb
  
  gem 'cucumber'
  gem 'cucumber-rails'
  gem 'webrat'
  gem 'rspec'

---------- end Gemfile -----------------


So, is this the same issue? It sure looks like it to me.  I hope
not to have to dive into the innards of rubygems if  possible.
But we need to be able to deploy.  The app is working and passing
tests on development, but hard to say on staging as we can't
get it over there with cap.  I suppose it could all be done by
hand once anyway, but we (other dev on my team and I ) need to
be able to do this over again.

Thanks for your time in this!

Cheers,
Craig

Attached Files:

Name Description Download
No Files Currently Attached

Changes:

Field Old Value Date By
status_idClosed2011-03-01 22:35zenspider
close_date2011-03-01 22:352011-03-01 22:35zenspider
resolution_idAccepted2011-03-01 22:35zenspider
resolution_idNone2011-02-02 00:03drbrain
status_idOpen2011-02-02 00:03drbrain
artifact_group_idNone2011-02-02 00:03drbrain
close_date2011-02-02 00:032011-02-02 00:03drbrain
assigned_tonone2010-12-01 10:54zenspider
status_idClosed2010-12-01 10:54zenspider
close_date2010-11-12 22:462010-11-12 22:46zenspider
status_idOpen2010-11-12 22:46zenspider