[Ironruby-core] IronRuby's Marshal.dump doesn't work with CLR types, or ruby types backed by a CLR type

Orion Edwards Orion.Edwards at gallagher.co
Thu Oct 27 22:46:48 EDT 2011

I've made some changes and have a pull request on Github:


The changeset is 


What I did:

1. Ran load_spec and dump_spec under 
  - Lots and lots of things failed

2. Fixed all the failures (except for one, which I can't quite figure 
  - The majority of this work was around writing string encodings when 
marshalling. IronRuby now writes string encodings as per MRI 1.9.2
      - I didn't make Marshal.Read understand string encodings yet though. 
The dump-with-encoding loads the string with no error, it just won't pick 
up the encoding

  - IronRuby didn't handle loading of self-referential arrays, hashes and 
objects. I added some specs for these and fixed the Marshal.Load code
  - It appears the behaviour of calling Marshal.load passing a proc 
changed between ruby 1.8 and 1.9. It now does what MRI 1.9.2 does

3. Removed the code path from Marshal.load that was using .NET 
 - Most things worked fine, except the ruby Range CLR class which was 
using the .NET serialization interface. 
 - As the Ruby range class is immutable we can't rely on instance variable 
setting, and unfortunately range just gets dumped as Object (no special 
token) so it doesn't really fit into the Marshal.load structure :-(
 - I added a new interface: IRubySpecialMarshalling. the range CLR class 
implements this, and Marshal.load checks for it as an alternative way of 
setting instance data.
     - This has the bonus of letting us dump subclasses of  Range. 
Previously this did not work.

4. Added code and specs for basic marshalling of Exception types. 
Currently only Exception.message and Exception.backtrace are supported, 
but this covers the majority of use cases for me.
   - Added a special case in Marshal.Write to write the exception data
   - As Exception is like Range (can't set instance variables, can't use 
.NET serialization) It uses IRubySpecialMarshalling through a proxy 
object. The proxy object is required because I can't change the CLR 
exception class to include my interface -:-(

Some notes:

- I'm not particularly happy with the way IRubySpecialMarshalling is 
working, but I'm not sure of a better way to do it. 
- Using reflection to set Exception.message sucks, but the requirement to 
unmarshal self-referential objects basically means you cannot Marshal.load 
an object's attributes an object before you create the object itself. This 
is probably no big deal in CRuby :-(
- The single failing marshal test is "loads an Array with proc". I can't 
figure out what the logic behind calling the proc is as it gets really 
complicated with lots of nested types. If anyone could shed some light on 
it that would be great.
- Where encodings have aliases, IronRuby selects a string's encoding by 
asking windows what to use based on a Codepage number. Because MRI uses a 
different bit of logic, MRI and IronRuby can abitrarily select different 
aliases. EG:

s = "foo"
s.force_encoding "cp850" 
s.force_encoding "ibm850"  # alternative which has the same effect
puts s.encoding => "CP850" in MRI, "ibm850" in IronRuby

I don't suppose this matters much, but it did make the specs harder to 
write :-(

Anyway, any feedback would be much appreciated. Thanks, Orion

From:   Tomas Matousek <Tomas.Matousek at microsoft.com>
To:     "ironruby-core at rubyforge.org" <ironruby-core at rubyforge.org>
Date:   26/10/2011 03:42 p.m.
Subject:        Re: [Ironruby-core] IronRuby's Marshal.dump doesn't work 
with CLR types, or ruby types backed by a CLR type
Sent by:        ironruby-core-bounces at rubyforge.org

I think we should NOT serialize non-Ruby types for now. The implementation 
isn?t quite working so I?d prefer we delete all code that deals with 
ISerializable, clean up the marshaller and if .NET serialization is needed 
in future implement it fully and correctly.
Marshal.dump has to output exactly the same data as CRuby implementation 
(for Ruby objects). Otherwise you won?t be able to read data serialized by 
CRuby in IronRuby and vice versa.
From: ironruby-core-bounces at rubyforge.org [
mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of Orion Edwards
Sent: Tuesday, October 25, 2011 6:22 PM
To: ironruby-core at rubyforge.org
Subject: [Ironruby-core] IronRuby's Marshal.dump doesn't work with CLR 
types, or ruby types backed by a CLR type
Backstory: I'm trying to use DRb for some in-house utility code. DRb 
itself seems to work fine, but I found that when I misspelled a method 
name, instead of reporting back a NoMethodError, the IronRuby process 
crashed immediately to the console. 

This is using a relatively recent build of IronRuby from Github 

Steps to repro: 

e = RuntimeError.new 'xyz' 
dumped = Marshal.dump e 
e2 = Marshal.load dumped 

I would expect e2 to be equivalent to e, but instead the process crashes 
with this exception 

mscorlib:0:in `_InvokeConstructor': Exception has been thrown by the 
target of an invocation. (System::Reflection::TargetInvocationException) 
        from mscorlib:0:in `InvokeConstructor' 
        from mscorlib:0:in `Invoke' 
        from (ir):1:in `load' 
        from (ir):1 

Note: This also happens with any CLR type eg System::DateTime. 

I looked through IronRuby's marshalling code, and it appears that the 
behaviour of Marshal.dump and Marshal.load don't align properly. 

Marshal.dump is it's own self-contained set of code which essentially 
writes data to a BinaryStream. 
For ruby types, it ends up writing a series of values in a format that 
looks a lot like what I remember CRuby's marshal writing. 
For CLR types, this just writes the Type name and no instance data (clr 
objects don't have ruby instance variables after all) 

Marshal.load does 2 things: 
1. Reads any ruby instance variables out into an Attributes dictionary 
2. Uses reflection to find any non-public Constructor(SerializationInfo, 
StreamingContext) and invoke it. 

For ruby types, this finds the protected RubyClass(SerializationInfo, 
StreamingContext) ctor, which calls RubyOps.DeserializeObject, which in 
turn reads the attributes dictionary and it's all fine. 
For CLR types, this finds whatever constructor might exist for the CLR 
object, which does whatever it does for that type. 

Unfortunately because the data that is getting passed into the CLR 
deserialization constructor came from Marshal.dump which has no knowledge 
whatsoever of CLR serialization, the whole thing crashes. 

I'm no expert on CLR serialization, so I'd really appreciate some comments 
on this, as I'm not sure what to do here. 

As far as I can guess however, I can see two solutions: 

1. Implement Marshal.dump on top of the CLR serialization code so it 
matches Marshal.load and should therefore be able to handle CLR types too. 

2. Don't allow marshalling of CLR types, and put some special-case code 
into any Ruby types that are backed by CLR types (such as Exception) so 
these at least can be serialized? 

I'm going to have a crack at #1, but I'm not sure how successful this is. 

Again, any feedback would be greatly appreciated. 

Thanks, Orion_______________________________________________
Ironruby-core mailing list
Ironruby-core at rubyforge.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/ironruby-core/attachments/20111028/b7ff542b/attachment-0001.html>

More information about the Ironruby-core mailing list