[Rubygems-developers] [ rubygems-Bugs-14943 ] rubygems should use the same arch specific subdirs in lib for the native extensions as ruby

noreply at rubyforge.org noreply at rubyforge.org
Wed Jan 20 13:26:01 EST 2010


Bugs item #14943, was opened at 2007-10-22 10:57
You can respond by visiting: 
http://rubyforge.org/tracker/?func=detail&atid=575&aid=14943&group_id=126

Category: RubyGems installer (setup.rb)
Group: v0.9.5
Status: Open
Resolution: None
Priority: 1
Submitted By: Marcus Rueckert (darix)
Assigned to: Eric Hodel (drbrain)
Summary: rubygems should use the same arch specific subdirs in lib for the native extensions as ruby

Initial Comment:
quoting my my mail to ruby-core [1]
[[[
there is another feature that gem doesnt handle nicely atm:
with the standard ruby directory layout you can share the tree via nfs
for multiple architectures as the native extensions are in an arch
specific path. within an installed gem they are directly inside the
"lib" subdir. i wonder if gem should use the arch specific subdirs below
its "lib" subdir aswell.
]]]

Eric asked me in a follow up to open a ticket here. :)

Looking forward to the fix :)

[1] http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/12724

----------------------------------------------------------------------

>Comment By: James Tucker (raggi)
Date: 2010-01-20 18:26

Message:
I support the idea of putting a gem in an arch specific subdir.

I am strongly against this subdir being inside the gems installed path.

1. some platform specific gems pack different *ruby* code in their gemfiles for specific archs
2. some gems do not use the commonly accepted paths, or have multiple lib paths, i wouldn't want selection of the path to become arbitrary



----------------------------------------------------------------------

Comment By: Hongli Lai (hongli)
Date: 2010-01-20 16:27

Message:
I support this change. This would make it possible for multiple Ruby interpreters, possibly on multiple machines with different platforms, to share the same gem directory.

However I don't think Gem::Platform in its current form is adequate. While Gem::Platform is a good enough description of the platform, it isn't a good enough description of Ruby extension binary compatibility, which is what matters more. A few examples:
- OS X Snow Leopard broke the ABI by switching to x86_64 by default, but uname and therefore RUBY_PLATFORM still report i386 as platform.
- Ruby breaks API compatibility even between teeny releases, e.g. RSTRING_PTR vs RSTRING(x)->ptr. The former didn't exist until 1.8.6. On 1.8.7 and later it became required so that the latter doesn't work anymore. It doesn't look like the ABI changes but the Ruby core team doesn't seem to make any ABI guarantees. The switch from 1.8 to 1.9 most definitely breaks some ABI though.

For Phusion Passenger we use the following code to identify Ruby extension binary compatibility: http://pastie.org/786648

But even this doesn't tell us the entire story. For example it doesn't include the C++ ABI version, which can be a problem for Ruby extensions written in C++ like EventMachine. GCC breaks the C++ ABI regularly, sometimes even multiple times within the same minor release. For example GCC 3.1 broke the C++ ABI about 3 times if I recall correctly.

----------------------------------------------------------------------

Comment By: Daniel Berger (djberg96)
Date: 2009-02-02 04:56

Message:
Luis,

Right, modification to the extension builder would be required as far as I know.

Dan

----------------------------------------------------------------------

Comment By: Luis Lavena (luislavena)
Date: 2009-01-31 13:03

Message:
Sorry not to properly introduce my question.

Let's say I build a gem with VC6 (i386-mswin32), the gem platform is x86-mswin32-60

This means, my lib folder will be like this:

lib/my_lib.rb
lib/x86-mswin32-60/my_ext.so

my_lib.rb contains this:

require 'my_ext'

now, I'm in a IRB session:

require 'rubygems'
require 'my_lib'

It is supposed to work out.

But what about this:

require 'rubygems'
require 'my_ext'

Right now, since lib is added to LOAD_PATH on gem activation, there is no problem with that.

With the change, it indicates that:

If a gem contains a platform folder for libdir and the platform matches the current one, it should append that folder into LOAD_PATH.

That is the correct assumption?

I believe changes to Extension Builders bundled in rubygems will be required.


----------------------------------------------------------------------

Comment By: Daniel Berger (djberg96)
Date: 2009-01-27 17:52

Message:
Luis, I'm not sure what you mean by "require of the extension directly". Can you explain?

Dan

----------------------------------------------------------------------

Comment By: Luis Lavena (luislavena)
Date: 2009-01-27 17:01

Message:
So this will mean "lib/#{Gem::Platform}" will be added to the $LOAD_PATH if exist, correct?

What happens when users doing the require of the extension directly? it triggers the activation mechanism?

I kind of like this, but maybe I'm missing a drawback that will against us in the long run?


----------------------------------------------------------------------

Comment By: Daniel Berger (djberg96)
Date: 2009-01-26 20:24

Message:
For example, if you install a C extension gem called 'foo' on Windows you want it to install as:

C:/ruby/lib/ruby/gems/1.8/gems/foo/lib/i386-msvcr80/foo.so

And Rubygems should add this sitearch directory to its search path?

Is that about the sum of it?

Regards,

Dan

----------------------------------------------------------------------

Comment By: 7rans  (transami)
Date: 2007-11-06 10:27

Message:
After some trial and error with extensions, I
m starting to agree with this. It's more flexible.

----------------------------------------------------------------------

You can respond by visiting: 
http://rubyforge.org/tracker/?func=detail&atid=575&aid=14943&group_id=126


More information about the Rubygems-developers mailing list