[Ironruby-core] How our libraries stack up against the specs ...

John Lam (DLR) jflam at microsoft.com
Mon Oct 15 20:33:44 EDT 2007

I re-implemented parts of the Rubinius test suite harness to use features that *are* implemented in IronRuby. The good news is that we're using the specs as written without any changes at all.

The spec suite driver currently looks at Array, Hash and String only.

If I run the driver under MRI, I see 17 failures out of 743 specs; only one failure is expected. The other 16 failures are due the fact that I couldn't implement that part of the test suite harness without using features that IronRuby doesn't have today. So that's not a bad baseline start.

If I run the same spec suite under IronRuby, we see 457 failures out of 743, for a pass rate of 38%.  If we omit String, which is the least implemented of these types, our pass rate for Array and Hash rises to 58% (149 fail out of 353).

Some of the failures are obviously due to features that aren't implemented at all (mostly in String). Some of them are due to a class of language features that aren't implemented (e.g. taint). Some are due to known DLR bugs like the one that Martin is currently consumed with fixing. The remainder are real bugs in our implementation.

The spec harness is a fairly sophisticated Ruby program. The team should be proud of getting the language to this state so quickly!

I'm attaching the test dump to this mail for the curious. The shelveset that contains this stuff should be ready to review tomorrow and will be pushed out to SVN right after that.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: out.zip
Type: application/x-zip-compressed
Size: 20032 bytes
Desc: out.zip
Url : http://rubyforge.org/pipermail/ironruby-core/attachments/20071015/7e0b8d84/attachment-0001.bin 

More information about the Ironruby-core mailing list