[Ironruby-core] Deadlock in IronRuby

Orion Edwards Orion.Edwards at gallagher.co
Thu Sep 1 23:54:19 EDT 2011


I'm encountering what looks like a fairly textbook deadlock in the core 
IronRuby code when creating Classes: 

Thread 0:
  -> calls RubyContext.GetOrCreateClass(Type) - acquires 
RubyContext.ModuleCacheLock
  -> function body invokes RubyContext.GetOrCreateClassNoLock(Type)
  -> The class does not exist, and there are Clr Mixins present, so calls 
if(clrMixins != null) } using(ClassHierarchyLocker())...  - deadlocks 
trying to acquire RubyContext._classHierarchyLock

My Ruby code on this thread is calling .to_module on a CLR type

Thread 6:
 -> Calls RubyContext.GetQualifiedConstant(RubyScope, ConstantSiteCache, 
string[], bool)
 -> function body invokes using (context.ClassHierarchyLocker()) - 
acquires RubyContext._classHierarchyLock
 -> ... other intermediate calls ...
 -> calls RubyContext.GetOrCreateModule(NamespaceTracker) - deadlocks 
trying to acquire RubyContext.ModuleCacheLock

This thread is a background thread (pretty sure it's just a regular CLR 
threadpool thread) which is responding to an event fired by an external 
application.
Basically I have a .NET library which fires events, and I'm subscribing to 
those events from ruby. The events are normal and boring.
It doesn't look like any ruby code on this thread has run yet as it is not 
mentioned in the callstack. It appears to be trying to get a ruby module 
or class, perhaps so it can begin running some ruby code.

Basically, GetOrCreateClassNoLock is lying to us, and is acquiring a lock 
in spite of being called "NoLock". 

The code is complicated enough however that I'm not sure of the best way 
to resolve it, but the standard deadlock-resolving steps suggest one of 
these

1. Move the "expandMixins" code outside the "NoLock" method
  This is the most sensible option, but I've no idea where it could be 
moved to.

2. Collapse to a single lock (eg: Remove the separate moduleCacheLock and 
just use _classHierarchyLock for everything)
  note however there is also a seperate NamespaceCacheLock, 
ReferenceTypeInstanceDataLock, ValueTypeInstanceDataLock, and perhaps 
others which would also likely need collapsing.

3. Enforce a lock order such that you must always acquire 
_classHierarchyLock before acquiring the other locks
   (if we do this we may as well just collapse the locks as it would be 
simpler)

Is there any performance benefit to be gained from having 5 locks, or 
would it make more sense to just have a single "TypeSystem" lock ?
Hopefully someone with more knowledge of the IronRuby internals can give a 
better perspective on this.

Thanks a lot, Orion

Appendix:

Here's the output from SOS which illustrates it: I took a memory dump of 
IR running with -D command line parameter. 
I built the IronRuby executable off the github master code about 2 months 
ago, there are no modifications to the code

0:000> !syncblk
Index SyncBlock MonitorHeld Recursion Owning Thread Info  SyncBlock Owner
  112 08ab0780            3         1 08a55e38 148c   6   02608cc8 
IronRuby.Runtime.CheckedMonitor
  153 08ab0fd4            3         1 004d9d38 1970   0   0260a240 
System.Collections.Generic.Dictionary`2[[System.Type, 
mscorlib],[IronRuby.Builtins.RubyModule, IronRuby]]


Callstack for thread 0:

0:000> !clrstack
OS Thread Id: 0x1970 (0)
Child SP IP       Call Site
001fbd4c 77e200ed [GCFrame: 001fbd4c] 
001fbed8 77e200ed [GCFrame: 001fbed8] 
001fbef4 77e200ed [HelperMethodFrame: 001fbef4] 
System.Threading.Monitor.ReliableEnter(System.Object, Boolean ByRef)
001fbf44 6fcde0c4 System.Threading.Monitor.Enter(System.Object, Boolean 
ByRef)
001fbf54 5ea0b8ff IronRuby.Runtime.CheckedMonitor.Enter(Boolean ByRef) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\CheckedMonitor.cs @ 
32]
001fbf84 5e9f242b 
IronRuby.Runtime.RubyContext.GetOrCreateClassNoLock(System.Type) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyContext.cs @ 
876]
001fbfd4 5e9f2c40 
IronRuby.Runtime.RubyContext.GetOrCreateClass(System.Type) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyContext.cs @ 
815]
001fc008 5e9f0dea IronRuby.Runtime.RubyContext.GetModule(System.Type) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyContext.cs @ 
1355]
001fc018 0a3ad40d 
DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure, 
System.Runtime.CompilerServices.CallSite, IronRuby.Runtime.RubyScope, 
System.Object)
001fc038 099266c7 
DynamicClass.ℑℜ:wrap_method:133:1688:D:/Dev/TestFramework/lib/com_method_gen.rb(System.Runtime.CompilerServices.Closure, 
IronRuby.Runtime.BlockParam, System.Object, System.Object)
001fc078 5d5dd25f 
Microsoft.Scripting.Interpreter.LightLambda.Run3[[System.__Canon, 
mscorlib],[System.__Canon, mscorlib],[System.__Canon, 
mscorlib],[System.__Canon, mscorlib]](System.__Canon, System.__Canon, 
System.__Canon)
001fc0b4 5eaa1886 
IronRuby.Runtime.Calls.BlockDispatcher1.Invoke(IronRuby.Runtime.BlockParam, 
System.Object, IronRuby.Builtins.Proc, System.Object) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\Calls\BlockDispatchers.cs 
@ 133]
001fc0c8 5ea2067d IronRuby.Runtime.RubyOps.Yield1(System.Object, 
IronRuby.Builtins.Proc, System.Object, IronRuby.Runtime.BlockParam) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 336]
001fc100 5ea601e6 IronRuby.Runtime.BlockParam.Yield(System.Object, 
System.Object ByRef) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\BlockParam.cs @ 
205]
001fc110 5ecfe2a1 
IronRuby.Builtins.Enumerable+<>c__DisplayClassa.<Map>b__6(IronRuby.Runtime.BlockParam, 
System.Object, System.Object) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Libraries\Builtins\Enumerable.cs 
@ 117]
001fc12c 5eaa1886 
IronRuby.Runtime.Calls.BlockDispatcher1.Invoke(IronRuby.Runtime.BlockParam, 
System.Object, IronRuby.Builtins.Proc, System.Object) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\Calls\BlockDispatchers.cs 
@ 133]
001fc140 5ea2067d IronRuby.Runtime.RubyOps.Yield1(System.Object, 
IronRuby.Builtins.Proc, System.Object, IronRuby.Runtime.BlockParam) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 336]
001fc178 5ecea6eb 
IronRuby.Builtins.IListOps.Each(IronRuby.Runtime.BlockParam, 
System.Collections.IList) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Libraries\Extensions\IListOps.cs 
@ 885]
001fc190 05afe35f 
DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure, 
System.Runtime.CompilerServices.CallSite, System.Object, 
IronRuby.Builtins.Proc)
001fc1c8 5ecf2750 
IronRuby.Builtins.Enumerable.Each(IronRuby.Runtime.CallSiteStorage`1<System.Func`4<System.Runtime.CompilerServices.CallSite,System.Object,IronRuby.Builtins.Proc,System.Object>>, 
System.Object, IronRuby.Builtins.Proc) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Libraries\Builtins\Enumerable.cs 
@ 37]
.... elided ....
001fef74 5d61bb5c 
Microsoft.Scripting.Hosting.Shell.CommandLine.RunFile(Microsoft.Scripting.Hosting.ScriptSource) 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\CommandLine.cs 
@ 182]
001fefa4 5eaa641d IronRuby.Hosting.RubyCommandLine.RunFile(System.String) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Hosting\RubyCommandLine.cs 
@ 69]
001fefb8 5d61bc9c Microsoft.Scripting.Hosting.Shell.CommandLine.Run() 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\CommandLine.cs 
@ 144]
001fefc4 5d61bd4a 
Microsoft.Scripting.Hosting.Shell.CommandLine.Run(Microsoft.Scripting.Hosting.ScriptEngine, 
Microsoft.Scripting.Hosting.Shell.IConsole, 
Microsoft.Scripting.Hosting.Shell.ConsoleOptions) 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\CommandLine.cs 
@ 112]
001ff004 5d61d1da 
Microsoft.Scripting.Hosting.Shell.ConsoleHost.RunCommandLine() 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs 
@ 402]
001ff040 5d61d577 
Microsoft.Scripting.Hosting.Shell.ConsoleHost.ExecuteInternal() 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs 
@ 337]
001ff04c 5d61d987 Microsoft.Scripting.Hosting.Shell.ConsoleHost.Execute() 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs 
@ 316]
001ff05c 5d61c9da 
Microsoft.Scripting.Hosting.Shell.ConsoleHost.Run(System.String[]) 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs 
@ 213]
001ff09c 6eb253e5 Host.Main(System.String[])*** WARNING: Unable to verify 
checksum for ir.ni.exe
 [c:\Dev\ironlanguages-main\Languages\Ruby\Console\Program.cs @ 80]


Callstack for thread 6:

0:006> !clrstack
OS Thread Id: 0x148c (6)
Child SP IP       Call Site
0955e490 77e200ed [GCFrame: 0955e490] 
0955e5a8 77e200ed [GCFrame: 0955e5a8] 
0955e5c4 77e200ed [HelperMethodFrame_1OBJ: 0955e5c4] 
System.Threading.Monitor.ReliableEnter(System.Object, Boolean ByRef)
0955e61c 6fcde0c4 System.Threading.Monitor.Enter(System.Object, Boolean 
ByRef)*** WARNING: Unable to verify checksum for mscorlib.ni.dll

0955e62c 5e9f2f47 
IronRuby.Runtime.RubyContext.GetOrCreateModule(Microsoft.Scripting.Actions.NamespaceTracker)*** 
WARNING: Unable to verify checksum for IronRuby.ni.dll

0955e660 5e9f011d 
IronRuby.Runtime.RubyContext.TrackerToModule(System.Object) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyContext.cs @ 
1661]
0955e670 5ea15b5e 
IronRuby.Builtins.RubyModule.TryGetConstantNoAutoloadCheck(System.String, 
IronRuby.Runtime.ConstantStorage ByRef) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyModule.cs @ 
1132]
0955e698 5ea563a0 
IronRuby.Builtins.RubyModule+<>c__DisplayClass6.<TryResolveConstantNoAutoloadCheck>b__5(IronRuby.Builtins.RubyModule) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyModule.cs @ 
1102]
0955e6a4 5ea15096 
IronRuby.Builtins.RubyModule.ForEachDeclaredAncestor(System.Func`2<IronRuby.Builtins.RubyModule,Boolean>) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyModule.cs @ 
876]
0955e6b8 5ea18026 
IronRuby.Builtins.RubyClass.ForEachAncestor(System.Func`2<IronRuby.Builtins.RubyModule,Boolean>) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyClass.cs @ 
622]
0955e6c8 5ea15a6d 
IronRuby.Builtins.RubyModule.TryResolveConstantNoAutoloadCheck(Boolean, 
System.String, IronRuby.Runtime.ConstantStorage ByRef) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyModule.cs @ 
1102]
0955e6e8 5ea15967 
IronRuby.Builtins.RubyModule.TryLookupConstantNoLock(Boolean, Boolean, 
IronRuby.Runtime.RubyGlobalScope, System.String, 
IronRuby.Runtime.ConstantStorage ByRef) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyModule.cs @ 
1063]
0955e71c 5ea49915 
IronRuby.Runtime.RubyScope.TryResolveConstantNoLock(IronRuby.Runtime.RubyGlobalScope, 
System.String, IronRuby.Runtime.ConstantStorage ByRef) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyScope.cs @ 530]
0955e740 5ea1edd1 
IronRuby.Runtime.RubyOps.ResolveQualifiedConstant(IronRuby.Runtime.RubyScope, 
System.String[], IronRuby.Builtins.RubyModule, Boolean, 
IronRuby.Runtime.ConstantStorage ByRef, Boolean ByRef) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 997]
0955e7b0 5ea1f711 
IronRuby.Runtime.RubyOps.GetQualifiedConstant(IronRuby.Runtime.RubyScope, 
IronRuby.Runtime.ConstantSiteCache, System.String[], Boolean) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 811]
0955e808 5eaccd74 
Microsoft.Scripting.Interpreter.FuncCallInstruction`5[[System.__Canon, 
mscorlib],[System.__Canon, mscorlib],[System.__Canon, 
mscorlib],[System.Boolean, mscorlib],[System.__Canon, 
mscorlib]].Run(Microsoft.Scripting.Interpreter.InterpretedFrame)*** 
WARNING: Unable to verify checksum for Microsoft.Dynamic.ni.dll
 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Instructions\CallInstruction.Generated.cs 
@ 764]
0955e840 5d5ae299 
Microsoft.Scripting.Interpreter.Interpreter.Run(Microsoft.Scripting.Interpreter.InterpretedFrame) 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Interpreter.cs 
@ 126]
0955e870 5d5dd2c6 
Microsoft.Scripting.Interpreter.LightLambda.Run3[[System.__Canon, 
mscorlib],[System.__Canon, mscorlib],[System.__Canon, 
mscorlib],[System.__Canon, mscorlib]](System.__Canon, System.__Canon, 
System.__Canon) 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\LightLambda.Generated.cs 
@ 130]
0955e8ac 5eaa1886 
IronRuby.Runtime.Calls.BlockDispatcher1.Invoke(IronRuby.Runtime.BlockParam, 
System.Object, IronRuby.Builtins.Proc, System.Object) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\Calls\BlockDispatchers.cs 
@ 133]
0955e8c0 5ea2067d IronRuby.Runtime.RubyOps.Yield1(System.Object, 
IronRuby.Builtins.Proc, System.Object, IronRuby.Runtime.BlockParam) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 336]
0955e8f8 5ecea6eb 
IronRuby.Builtins.IListOps.Each(IronRuby.Runtime.BlockParam, 
System.Collections.IList)*** WARNING: Unable to verify checksum for 
IronRuby.Libraries.ni.dll
 
[c:\Dev\ironlanguages-main\Languages\Ruby\Libraries\Extensions\IListOps.cs 
@ 885]
0955e910 5d624a86 
Microsoft.Scripting.Interpreter.FuncCallInstruction`3[[System.__Canon, 
mscorlib],[System.__Canon, mscorlib],[System.__Canon, 
mscorlib]].Run(Microsoft.Scripting.Interpreter.InterpretedFrame) 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Instructions\CallInstruction.Generated.cs 
@ 716]
0955e93c 5d5ae299 
Microsoft.Scripting.Interpreter.Interpreter.Run(Microsoft.Scripting.Interpreter.InterpretedFrame) 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Interpreter.cs 
@ 126]
0955e96c 5d5dd649 
Microsoft.Scripting.Interpreter.LightLambda.Run4[[System.__Canon, 
mscorlib],[System.__Canon, mscorlib],[System.__Canon, 
mscorlib],[System.__Canon, mscorlib],[System.__Canon, 
mscorlib]](System.__Canon, System.__Canon, System.__Canon, System.__Canon) 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\LightLambda.Generated.cs 
@ 165]
0955e9ac 6f6ebfc2 
System.Dynamic.UpdateDelegates.UpdateAndExecute3[[System.__Canon, 
mscorlib],[System.__Canon, mscorlib],[System.__Canon, 
mscorlib],[System.__Canon, 
mscorlib]](System.Runtime.CompilerServices.CallSite, System.__Canon, 
System.__Canon, System.__Canon)*** WARNING: Unable to verify checksum for 
System.Core.ni.dll

0955ea08 5d62d2dd 
Microsoft.Scripting.Interpreter.DynamicInstruction`4[[System.__Canon, 
mscorlib],[System.__Canon, mscorlib],[System.__Canon, 
mscorlib],[System.__Canon, 
mscorlib]].Run(Microsoft.Scripting.Interpreter.InterpretedFrame) 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Instructions\DynamicInstructions.Generated.cs 
@ 193]
0955ea38 5d5ae299 
Microsoft.Scripting.Interpreter.Interpreter.Run(Microsoft.Scripting.Interpreter.InterpretedFrame) 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Interpreter.cs 
@ 126]
0955ea68 5d5dd649 
Microsoft.Scripting.Interpreter.LightLambda.Run4[[System.__Canon, 
mscorlib],[System.__Canon, mscorlib],[System.__Canon, 
mscorlib],[System.__Canon, mscorlib],[System.__Canon, 
mscorlib]](System.__Canon, System.__Canon, System.__Canon, System.__Canon) 
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\LightLambda.Generated.cs 
@ 165]
0955eaa8 5eaa1aea 
IronRuby.Runtime.Calls.BlockDispatcher2.Invoke(IronRuby.Runtime.BlockParam, 
System.Object, IronRuby.Builtins.Proc, System.Object, System.Object) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\Calls\BlockDispatchers.cs 
@ 229]
0955eac0 5ea20590 IronRuby.Runtime.RubyOps.Yield2(System.Object, 
System.Object, IronRuby.Builtins.Proc, System.Object, 
IronRuby.Runtime.BlockParam) 
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 362]
0955eafc 0a092f4a 
DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure, 
System.Runtime.CompilerServices.CallSite, System.Object, System.Object, 
MyApplication.SomeEventArgs)
0955eb20 6f6ebfc2 
System.Dynamic.UpdateDelegates.UpdateAndExecute3[[System.__Canon, 
mscorlib],[System.__Canon, mscorlib],[System.__Canon, 
mscorlib],[System.__Canon, 
mscorlib]](System.Runtime.CompilerServices.CallSite, System.__Canon, 
System.__Canon, System.__Canon)
0955eb7c 05afe257 DynamicClass._Scripting_(System.Object[], System.Object, 
MyApplication.SomeEventArgs)
0955eb90 098df5fd MyApplication.SomeClass.RaiseEvents(Boolean, 
System.Collections.Generic.IEnumerable`1<MyApplication.DataStruct>)
0955ebd8 098df45f MyApplication.SomeClass.OnEvent(UInt32, Boolean, 
Boolean, UInt32, MyApplication.DataStruct[])
0955ec28 098df3c9 DomainBoundILStubClass.IL_STUB_COMtoCLR(Int32, Int32, 
Int32, Int32, IntPtr)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/ironruby-core/attachments/20110902/e177c8a2/attachment-0001.html>


More information about the Ironruby-core mailing list