[Nitro] Store pool management

Alex Pooley alex at msgpad.com
Fri Sep 15 23:08:15 EDT 2006


Og was previously leaking stores via the store pool. This was most
visible with the SQLite adapter as you would simply run out of file
descriptors after a while! This has been fixed, but it involved a
significant change to the management of the store pool, and it's
consequential usage. This will probably only concern you if you are
involved with nitroproject development.

I'll detail the implications below, for further information please refer
to the patches and documentation within manager.rb in Og.

The Og manager controls a pool of connections when running in thread
safe mode. Previously you literally pulled a store out of the pool like
so:

ogmanager.store.do_something

The assumption was that the store would eventually be returned to the
store pool, but this never happened [1]. This was fine if you only ever
ran as many threads as stores in the pool, but this is an unreasonable
assumption [2]. Once you ran more threads than you had stores in the
pool, you would deadlock waiting on one of the threads to return a store
to the pool.

The new store access is a bit uglier:

ogmanager.run {|store| store.do_something}

Instead of pulling a store out of Og manager, you ask the manager to run
a block of code on a store of it's own choosing. This way, the manager
never lets the store out of it's sight and is able to control the usage
of the stores. Unfortunately it's still possible to leak the stores by
evaluating the block to the store and saving it to outside the block.
The pool will not deadlock, but you will no longer be thread safe if you
do this!

I've also modified the default :connection_count setting from 5 to just
1. This setting dictates how large the store pool should be. From my
testing using the new tc_manager.rb I found that a single connection was
faster with SQLite.

Og manager also appeared to have the ability to drop in and out of
thread safe mode(?). I'm not sure why you would want this? I gather it's
just legacy code from before a thread_safe version was built? As the
manager stands now, it is simply thread safe. Unless anyone objects I
will go ahead and remove all the thread_safe stuff from og.rb and the
few references to the Og.thread_safe in the nitro adapters. Speak now or
forever hold your peace :)

Alex.

[1] Actually, inside Og this never appeared to happen, though I've since
found where I think it was meant to happen in Nitro! That suprised me as
it's a coupling between Nitro and Og that is just not required.

[2] You may have noticed that your code still worked with more threads
than stores in the pool. The reason was that the store pool just kept
growing due to buggy code. If you ran with SQLite you eventually ran out
of file descriptors!

-- 
Alex Pooley (msgpad founder)
w: http://msgpad.com
e: alex at msgpad.com
b: http://alexpooley.com




More information about the Nitro-general mailing list