[Ironruby-core] IronRuby's Marshal.dump doesn't work with CLR types, or ruby types backed by a CLR type
Orion.Edwards at gallagher.co
Thu Oct 27 22:46:48 EDT 2011
I've made some changes and have a pull request on Github:
The changeset is
What I did:
1. Ran load_spec and dump_spec under
- Lots and lots of things failed
2. Fixed all the failures (except for one, which I can't quite figure
- The majority of this work was around writing string encodings when
marshalling. IronRuby now writes string encodings as per MRI 1.9.2
- I didn't make Marshal.Read understand string encodings yet though.
The dump-with-encoding loads the string with no error, it just won't pick
up the encoding
- IronRuby didn't handle loading of self-referential arrays, hashes and
objects. I added some specs for these and fixed the Marshal.Load code
- It appears the behaviour of calling Marshal.load passing a proc
changed between ruby 1.8 and 1.9. It now does what MRI 1.9.2 does
3. Removed the code path from Marshal.load that was using .NET
- Most things worked fine, except the ruby Range CLR class which was
using the .NET serialization interface.
- As the Ruby range class is immutable we can't rely on instance variable
setting, and unfortunately range just gets dumped as Object (no special
token) so it doesn't really fit into the Marshal.load structure :-(
- I added a new interface: IRubySpecialMarshalling. the range CLR class
implements this, and Marshal.load checks for it as an alternative way of
setting instance data.
- This has the bonus of letting us dump subclasses of Range.
Previously this did not work.
4. Added code and specs for basic marshalling of Exception types.
Currently only Exception.message and Exception.backtrace are supported,
but this covers the majority of use cases for me.
- Added a special case in Marshal.Write to write the exception data
- As Exception is like Range (can't set instance variables, can't use
.NET serialization) It uses IRubySpecialMarshalling through a proxy
object. The proxy object is required because I can't change the CLR
exception class to include my interface -:-(
- I'm not particularly happy with the way IRubySpecialMarshalling is
working, but I'm not sure of a better way to do it.
- Using reflection to set Exception.message sucks, but the requirement to
unmarshal self-referential objects basically means you cannot Marshal.load
an object's attributes an object before you create the object itself. This
is probably no big deal in CRuby :-(
- The single failing marshal test is "loads an Array with proc". I can't
figure out what the logic behind calling the proc is as it gets really
complicated with lots of nested types. If anyone could shed some light on
it that would be great.
- Where encodings have aliases, IronRuby selects a string's encoding by
asking windows what to use based on a Codepage number. Because MRI uses a
different bit of logic, MRI and IronRuby can abitrarily select different
s = "foo"
s.force_encoding "ibm850" # alternative which has the same effect
puts s.encoding => "CP850" in MRI, "ibm850" in IronRuby
I don't suppose this matters much, but it did make the specs harder to
Anyway, any feedback would be much appreciated. Thanks, Orion
From: Tomas Matousek <Tomas.Matousek at microsoft.com>
To: "ironruby-core at rubyforge.org" <ironruby-core at rubyforge.org>
Date: 26/10/2011 03:42 p.m.
Subject: Re: [Ironruby-core] IronRuby's Marshal.dump doesn't work
with CLR types, or ruby types backed by a CLR type
Sent by: ironruby-core-bounces at rubyforge.org
I think we should NOT serialize non-Ruby types for now. The implementation
isn?t quite working so I?d prefer we delete all code that deals with
ISerializable, clean up the marshaller and if .NET serialization is needed
in future implement it fully and correctly.
Marshal.dump has to output exactly the same data as CRuby implementation
(for Ruby objects). Otherwise you won?t be able to read data serialized by
CRuby in IronRuby and vice versa.
From: ironruby-core-bounces at rubyforge.org [
mailto:ironruby-core-bounces at rubyforge.org] On Behalf Of Orion Edwards
Sent: Tuesday, October 25, 2011 6:22 PM
To: ironruby-core at rubyforge.org
Subject: [Ironruby-core] IronRuby's Marshal.dump doesn't work with CLR
types, or ruby types backed by a CLR type
Backstory: I'm trying to use DRb for some in-house utility code. DRb
itself seems to work fine, but I found that when I misspelled a method
name, instead of reporting back a NoMethodError, the IronRuby process
crashed immediately to the console.
This is using a relatively recent build of IronRuby from Github
Steps to repro:
e = RuntimeError.new 'xyz'
dumped = Marshal.dump e
e2 = Marshal.load dumped
I would expect e2 to be equivalent to e, but instead the process crashes
with this exception
mscorlib:0:in `_InvokeConstructor': Exception has been thrown by the
target of an invocation. (System::Reflection::TargetInvocationException)
from mscorlib:0:in `InvokeConstructor'
from mscorlib:0:in `Invoke'
from (ir):1:in `load'
Note: This also happens with any CLR type eg System::DateTime.
I looked through IronRuby's marshalling code, and it appears that the
behaviour of Marshal.dump and Marshal.load don't align properly.
Marshal.dump is it's own self-contained set of code which essentially
writes data to a BinaryStream.
For ruby types, it ends up writing a series of values in a format that
looks a lot like what I remember CRuby's marshal writing.
For CLR types, this just writes the Type name and no instance data (clr
objects don't have ruby instance variables after all)
Marshal.load does 2 things:
1. Reads any ruby instance variables out into an Attributes dictionary
2. Uses reflection to find any non-public Constructor(SerializationInfo,
StreamingContext) and invoke it.
For ruby types, this finds the protected RubyClass(SerializationInfo,
StreamingContext) ctor, which calls RubyOps.DeserializeObject, which in
turn reads the attributes dictionary and it's all fine.
For CLR types, this finds whatever constructor might exist for the CLR
object, which does whatever it does for that type.
Unfortunately because the data that is getting passed into the CLR
deserialization constructor came from Marshal.dump which has no knowledge
whatsoever of CLR serialization, the whole thing crashes.
I'm no expert on CLR serialization, so I'd really appreciate some comments
on this, as I'm not sure what to do here.
As far as I can guess however, I can see two solutions:
1. Implement Marshal.dump on top of the CLR serialization code so it
matches Marshal.load and should therefore be able to handle CLR types too.
2. Don't allow marshalling of CLR types, and put some special-case code
into any Ruby types that are backed by CLR types (such as Exception) so
these at least can be serialized?
I'm going to have a crack at #1, but I'm not sure how successful this is.
Again, any feedback would be greatly appreciated.
Ironruby-core mailing list
Ironruby-core at rubyforge.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Ironruby-core