From ruby-pg at j-davis.com Wed Dec 5 17:18:49 2007 From: ruby-pg at j-davis.com (Jeff Davis) Date: Wed, 05 Dec 2007 14:18:49 -0800 Subject: [Ruby-pg-general] ruby-pg refactoring Message-ID: <1196893129.28804.65.camel@dogma.ljc.laika.com> I'm doing some major refactoring of ruby-pg. A lot of the code was inherited from ruby-postgres, and I think some of the approaches are wrong. The fundamental problem I think is that too much of the code is dedicated to type mapping and extra utility functions, all mixed in with the rest of the C code. There is little, if any reason to write that kind of code in C. And there is no reason at all that it needs to cover the underlying libpq functionality. I'm concerned that parts of libpq aren't being exposed because of too much magic. As an example, we have a lot of code that does all kinds of tricks for formatting, but we don't even have the ability to asynchronously execute a prepared statement... there's no use of PQsendPrepare()/PQsendPrepared () at all. Not to mention, there are all kinds of portability problems with the code that I'm trying to clean up. Limiting the C code is a great way to make it more portable. So what I'm doing is rewriting pg.c as a very raw wrapper around libpq (not a complete rewrite, of course, but it is major refactoring). All of libpq (and the large object interface) will be exposed. No non-libpq features will exist in pg.c. Now, for compatibility, I will write ruby code on top, as "postgres.rb". This will allow existing code to work, unmodified, as well as possible. I hope this will improve the stability, portability, and functionality of ruby-pg. I plan to continue maintaining the "compatibility" functions in C so that ruby-pg will function with very old versions of PostgreSQL, although some methods may raise exceptions. For instance, PQexecParams with binary formats won't work on 6.5.3, obviously, there's no way around that. I also think we may be able to sidestep some of the desires for type mapping by making ruby-pg a driver for ruby-dbi. I am branching rev 12 off into "raw_libpq" and working in there. Comments? Regards, Jeff Davis From ruby-pg at j-davis.com Fri Dec 7 13:55:18 2007 From: ruby-pg at j-davis.com (Jeff Davis) Date: Fri, 07 Dec 2007 10:55:18 -0800 Subject: [Ruby-pg-general] ruby-postgres doesn't compile against 8.3, ruby-pg does Message-ID: <1197053718.28804.125.camel@dogma.ljc.laika.com> ruby-postgres is the most widely used PostgreSQL binding for ruby, but it is unmaintained. ruby-postgres-0.7.1.tar.gz compiles fine ruby-postgres-20051221.tar.gz doesn't build ruby-postgres-20060405.tar.gz doesn't build ruby-postgres-20060406.tar.gz doesn't build The same problems exist for the gems, versions 0.7.1.2006.04.05 0.7.1.2006.04.06. This could mean that a lot (probably most) of ruby developers will have a problem upgrading to PostgreSQL 8.3. This is easy to fix (described below) but unfortunately ruby-postgres is unmaintained. I am working on my own fork of the project, ruby-pg, where I maintain patches against ruby-postgres. It can be found here: http://rubyforge.org/projects/ruby-pg I backpatched it to stable, so you can check it out here: svn co http://ruby-pg.rubyforge.org/svn/branches/stable and I also released it as a file in the project. The stable branch is very similar to snapshot 20060406 with a few updates of my own. It *should* work fine with applications that work with that snapshot release. Most of my work on the project has gone into a branch called raw_libpq (not currently stable), in which I'm rewriting a lot of the code. I'm trying to streamline the module and make it a fairly thin wrapper around libpq, that exposes *all* libpq functionality, with a ruby feel to it. For instance, I raise exceptions on errors, but those exception objects also hold the PGconn and PGresult objects so that you can access detailed information if needed. I would like this new branch to eventually replace the current ruby- postgres, which does a lot of type conversion magic in C, which ends up hiding libpq functionality and complicating the C code. Any type conversion magic should really be done in ruby; there's no reason to do it in C. I think that's a big part of the reason the current ruby- postgres is so cluttered. So, use ruby-pg instead. But please test it first, it hasn't received much attention outside of my environment. Regards, Jeff Davis ------------------------------------------------ PROBLEM DETAILS: The snapshot releases of ruby-postgres do not compile against PostgreSQL 8.3. This is due to a rather strange sequence in postgres.c: #ifndef HAVE_PG_ENCODING_TO_CHAR #define pg_encoding_to_char(x) "SQL_ASCII" #else extern char* pg_encoding_to_char(int); #endif Apparently, PostgreSQL 8.2 and below did export a function called pg_encoding_to_char, but did not declare it in any header. PostgreSQL 8.3 does declare it in the header, leading to a "conflicting types" compilation error. The quick fix I applied simply removes the declaration (the #else clause) from postgres.c (which has been renamed to pg.c in my fork). This leads to an "implicit declaration" compiler warning in 8.2 and before, but no errors or warnings compiling against 8.3. From ruby-pg at j-davis.com Thu Dec 13 16:55:22 2007 From: ruby-pg at j-davis.com (Jeff Davis) Date: Thu, 13 Dec 2007 13:55:22 -0800 Subject: [Ruby-pg-general] osx leopard In-Reply-To: <84A9F480-ECAA-4F23-A493-AB16447317B6@cisco.com> References: <84A9F480-ECAA-4F23-A493-AB16447317B6@cisco.com> Message-ID: <1197582922.28804.214.camel@dogma.ljc.laika.com> On Thu, 2007-12-13 at 15:25 -0500, Brett Eisenberg wrote: > Hi Jeff, > > I'm assuming it was you that updated the postgres gem......fyi, it's > broken on osx leopard. [ I CC'd the ruby-pg-general list ] Thank you for the detailed report. What version of PostgreSQL are you building against? Regards, Jeff Davis From ruby-pg at j-davis.com Thu Dec 13 17:38:32 2007 From: ruby-pg at j-davis.com (Jeff Davis) Date: Thu, 13 Dec 2007 14:38:32 -0800 Subject: [Ruby-pg-general] osx leopard In-Reply-To: <1197582922.28804.214.camel@dogma.ljc.laika.com> References: <84A9F480-ECAA-4F23-A493-AB16447317B6@cisco.com> <1197582922.28804.214.camel@dogma.ljc.laika.com> Message-ID: <1197585512.28804.219.camel@dogma.ljc.laika.com> On Thu, 2007-12-13 at 13:55 -0800, Jeff Davis wrote: > On Thu, 2007-12-13 at 15:25 -0500, Brett Eisenberg wrote: > > Hi Jeff, > > > > I'm assuming it was you that updated the postgres gem......fyi, it's > > broken on osx leopard. > > [ I CC'd the ruby-pg-general list ] > > Thank you for the detailed report. What version of PostgreSQL are you > building against? > This appears to be an issue with leopard. You have to set the environment variable ARCHFLAGS appropriately. For example: # export ARCHFLAGS=' -arch i386 ' # gem install --remote postgres Can you please try that and tell me whether it works? As I understand it, that's due to a problem with leopard that will hopefully be fixed. If there's something I should do in my build process to guard against that problem, than I will. Thank you for reporting the issue, I will alert others of the potential problem. Regards, Jeff Davis From ruby-pg at j-davis.com Mon Dec 31 15:08:13 2007 From: ruby-pg at j-davis.com (Jeff Davis) Date: Mon, 31 Dec 2007 12:08:13 -0800 Subject: [Ruby-pg-general] ruby-pg segfault In-Reply-To: <477808D3.8020803@mit.edu> References: <477808D3.8020803@mit.edu> Message-ID: <1199131693.10057.135.camel@dogma.ljc.laika.com> On Sun, 2007-12-30 at 21:08 +0000, Omari Stephens wrote: > PGresult#res_status does weird things. Under irb, this causes a segfault. > Under normal ruby, this causes weird exceptions. > > irb(main):004:0> conn = PGconn.new(nil, 5433, nil, nil, 'testational') > => # > irb(main):005:0> a = conn.exec("SELECT 4") > => # > irb(main):006:0> a.res_status > (irb):6: [BUG] Segmentation fault > ruby 1.8.6 (2007-09-24) [i486-linux] > > Aborted > > 21:01:32> [xsdg{whitespace}@~] > $ruby -rubygems -e 'require "pg"; PGconn.new(nil, 5433, nil, nil, > "testational").exec("SELECT 4").res_status' > -e:1:in `res_status': method `respond_to?' called on terminated object > (0xb7d9c570) (NotImplementedError) > from -e:1 > > 21:03:34> [xsdg{whitespace}@~] > $ruby -e 'require "rubygems"; require "pg"; PGconn.new(nil, 5433, nil, nil, > "testational").exec("SELECT 4").res_status' > -e:1:in `res_status': can't convert false into Integer (TypeError) > from -e:1 Hi, I apologize for the slow reply (out of town). Thank you for the report, I will fix this behavior. It appears that I declared the method to take zero arguments, but it actually expects one. You are using the 'pg' module. If you'd like your old code to work correctly, it would be best to use the 'postgres' module (still part of the ruby-pg gem). The 'postgres' module is intended for backwards compatibility, and only has a few fixes from the older unmaintained gem (I took over from another project named ruby-postgres). The 'pg' module is just starting to stabalize, so I'm sure there are more bugs. It should be stable relatively soon, and it will be more featureful, consistent, and portable. Let me know if you have any more problems. Regards, Jeff Davis From ruby-pg at j-davis.com Mon Dec 31 19:08:02 2007 From: ruby-pg at j-davis.com (Jeff Davis) Date: Mon, 31 Dec 2007 16:08:02 -0800 Subject: [Ruby-pg-general] ruby-pg "pg" API doesn't work as documented In-Reply-To: <4777D9DA.6050301@mit.edu> References: <4777D9DA.6050301@mit.edu> Message-ID: <1199146082.10057.138.camel@dogma.ljc.laika.com> On Sun, 2007-12-30 at 17:48 +0000, Omari Stephens wrote: > Hi. I installed the "postgres" gem (which pulled the 2007-12-22 > version). However, copying-and-pasting the example in the docs > embedded in pg.c fails: > irb(main):002:0> require 'rubygems' > => true > irb(main):003:0> require 'pg' > => true > irb(main):004:0> conn = PGconn.open('dbname' => 'test1') > TypeError: wrong argument type Hash (expected String) > from (irb):4:in `initialize' > from (irb):4:in `open' > from (irb):4 > I fixed this in /trunk, and I will release a new gem after a little testing. Thanks for the reports. Let me know if anything else is not working after I release a new gem. Regards, Jeff Davis From ruby-pg at j-davis.com Mon Dec 31 19:16:21 2007 From: ruby-pg at j-davis.com (Jeff Davis) Date: Mon, 31 Dec 2007 16:16:21 -0800 Subject: [Ruby-pg-general] whats the name of ruby-pg gem? In-Reply-To: References: Message-ID: <1199146581.10057.148.camel@dogma.ljc.laika.com> On Wed, 2007-12-26 at 18:17 +0800, Jon Hancock wrote: > Jeff, > I read that your ruby-pg project represents the current choice for > rails interaction with postgres. > > I have a few questions. > 1 - What's the name of the gem that this project maps to? gem doesn't > seem to know of a gem named "ruby-pg" so I guess thats not it. There > are a few other projects I see related to posgres so I don't want to > just say "gem install postgres" and assume its your latest ruby-pg code. The name of my project is "ruby-pg" and the name of the gem is "postgres". The reasons for this are: a) I supply both a 'postgres' module, intended to be backwards compatible with the old unmaintained gem by the same name; in addition to my new 'pg' module, which is not quite stable (but close). b) There are important fixes I made to the backwards compatible 'postgres' module, for example, support for building against PostgreSQL 8.3. If my project's gem was a different name, people would miss those important fixes, and perhaps be unable to work with PostgreSQL 8.3 and ruby. > 2 - Do you know how I'm supposed to get a proper install when I've > installed using EnterprsieDB 8.3-beta4? When I do run "sudo gem > install postgres", I get errors like:extconf.rb:6: command not found: > pg_config --includedir > pg_config is a postgresql tool to help the build process along. All it does is provide useful information about your postgresql install. Make sure you have some kind of pg_config in your PATH environment variable. I'm not sure exactly what EnterpriseDB does, or where it may put pg_config (assuming they didn't rename it). I will look into making the build process work somewhat sanely without pg_config (you might be able to hack the extconf.rb to make it work without too much effort). If you provide some more details that would be helpful. I'd like to make sure it works with EnterpriseDB also, so if you can keep testing I'd like to get this resolved. Thanks for the report. Regards, Jeff Davis