[Ruby-dbi-devel-new] Ruby/DBI spec and 0.4.0 development notes

[Erik Hollensbe]

I know I've left you guys in the dark for about a week regarding where  
0.4.0 is going, and I apologize for that. Work has been particularly  
demanding on a number of levels. Yep.... that's the best excuse I can  
offer. :)

Anyways, I wanted to make everyone aware of what's going on.

First, I have rubygems rolled and rakefile'd up, but the DBI core  
itself needs to be modified a tad to use the separate DBDs. Notably,  
DBI doesn't use rubygems anywhere (although I did add it in a few of  
the DBDs). Daniel Berger started some work on this process, but dbi.rb  
and the specific DBDs do not actually require rubygems themselves,  
meaning that if you have to perform non-gem installations of all the  
low-level drivers to work with the DBDs.

The other problem is, the DBD directory does not match the gem  
installation path of dbd/SQLite.rb, f.e.

I'm sure I could go through a lot of pain (and believe me, there is  
quite a bit of pain in the DBD include logic) to get the existing  
system working that handles case-insensitivity and a number of other  
odd cases, but frankly, I think this is a lot of overcomplicated code  
that serves no one especially well (f.e., Perl's DBI is case-sensitive  
when it comes to drivers) and de-complicating the logic here seems to  
be the best win for Rubygems support.

The plan for these gems is to go up as DBI 0.3.0 (development)  
supported gems and to allow people to track DBI that way, but I'm  
trying to find a good way to say "Achtung! Stop! This is development!"  
in a `gem install` ecosystem so people who use `gem search` or are  
simply not paying a lot of attention will install 0.2.0 (or perhaps,  
0.1.1) for a stable release.

Once I figure these two things out they will be available. For now,  
you can track the "development" branch of the git repository and build  
your own gems, but I'll be surprised if using them in code requires  
the right files ATM.

Second, I'm working on a little system to annotate the spec so I can  
review it and bring points of contention (and allow others to do the  
same) to the list. This is taking a bit longer than I wanted it to, so  
I may just go with a passworded wiki or something in the same vein.  
Please let me know what you guys think is best and easiest for you to  
work with.

The spec is critical. We're moving to distributing the DBDs separately  
and the OCI8 DBD is already that way. Before we can reliably guarantee  
our users DBDs that are uniform, we need a document and tests that  
ensure that. Ideally DBI's core documentation should be the base of  
the spec, not a copy of the code-specific portions. Look at Perl DBI's  
perldoc for a great example.

I would first like to move to an RFC-style requirements system (MUST,  
MUST NOT, SHOULD, SHOULD NOT, etc) for the spec. I think the spec is  
entirely too vague right now and it's allowing a lot of things through.

After the spec is done but (ideally) before any coding is done, I'm  
going to write 'general' DBD tests against the spec for our supported  
DBDs and DBI. This will hopefully enforce the spec better than words  
in a RDoc document.

As far as features are concerned, I would like to concentrate on type  
handling and ideally lifting a lot more code out of the DBDs and make  
canonical representations in DBI for type coercion a possibility.  
Right now a *lot* of duplicated mappings, coercion systems, and other  
type handling madness is in the DBDs and it makes them extremely hard  
to maintain in a uniform fashion. Abstracting this is one of the  
places that Perl's DBI really doesn't have an equally transferrable  
solution for (anyone familiar with perl knows the perils of its "type  
system") so we're gonna have to be a little more creative than usual.

After that's solved, I would like to direct our focus towards properly  
managing the expectations of database metadata calls (such as tables()  
and columns()) WRT to things like system tables and Pg schemas, and  
describing (in RFC terms) what information should be available here.

After these four things are handled, this would be a good stopping  
point for an 0.4.0 release, along with any 0.2.0 bugfixes that come  
along in that time. I'd talk about 0.6.0 but I don't have anything to  
say just yet. :)

Please feel welcome to argue this with me, especially if I have  
glossed over something you feel is a critical feature that needs to be  
addressed immediately.

Thanks,

-Erik