From walter at katipo.co.nz Thu Aug 16 18:23:02 2007 From: walter at katipo.co.nz (Walter McGinnis) Date: Fri, 17 Aug 2007 10:23:02 +1200 Subject: [Ruby-zoom-devel] Release for ZOOM Extended Services Message-ID: <0B27FD14-4156-4A5A-8FFF-B730502503B5@katipo.co.nz> Hi all, As you know I have committed code to add ZOOM Extended Services via ZOOM::Connection#packages. Thanks Nicolai for the good work! It's in trunk and also there is a 0.4.0 tag as well. Currently all previous tests pass. In addition, Nicolai added tests for the new code that doesn't rely on connecting to an actual Z39.50 server. These all pass. I also added tests that relied on a test Z39.50 server that I can't guarantee will be available. I have commented these out. They are meant to test the CRUD capabilities (or more accurately the CUD capabilities) of Extended Services. These commented out tests have two issues: 1. a reliable Z39.50 test server needs to be set up that is configured to handle Extended Services 2. the update and destroy tests were failing because the test records weren't being matched properly When creating the tests, I simply went with XML that would work with my other servers. Quite complex records for testing. In the process I set the record files up wrong and thus the tests failed. My recommendation is that when we determine the test Z39.50 test server we evaluate the appropriate test records at the time. This should fix the failing update and delete tests. It should be noted that my tests using real records via my acts_as_zoom Rails plugin all passed for update and delete. So I'm confidant the problem isn't with the new Extended Services code, but with my tests in zoom 0.4.0. So what do you people think is the way forward towards a release? Cheers, Walter From walter at katipo.co.nz Thu Aug 16 18:10:39 2007 From: walter at katipo.co.nz (Walter McGinnis) Date: Fri, 17 Aug 2007 10:10:39 +1200 Subject: [Ruby-zoom-devel] Took awhile, but I'm now on the list Message-ID: <3BDF50AE-EA5F-4650-BC5B-83BF0ED70870@katipo.co.nz> Good ole greylisting... Anyway, glad to get this list going. I'll post a separate email about zoom gem releases... Cheers, Walter From walter at katipo.co.nz Thu Aug 16 19:28:35 2007 From: walter at katipo.co.nz (Walter McGinnis) Date: Fri, 17 Aug 2007 11:28:35 +1200 Subject: [Ruby-zoom-devel] Release for ZOOM Extended Services In-Reply-To: <763570460708161603h2d256bc2g51983fbd4fb15518@mail.gmail.com> References: <0B27FD14-4156-4A5A-8FFF-B730502503B5@katipo.co.nz> <763570460708161603h2d256bc2g51983fbd4fb15518@mail.gmail.com> Message-ID: <42A70EC6-078E-459B-97F6-D916967CA288@katipo.co.nz> I'm glad that we have straightened things out. Great news about your tests. I'm not certain that having a reliable test server for Extended Services is necessary. Essentially if we can test successfully before building the gem for release, that seems sufficient (although potentially confusing for someone browsing the source). I would say updating the API reference to reflect Extended Services is a higher prioritiy. As far as my test server, I think the filtering of the XML record is being picky about the datestamp field. I have adjusted this in my records in the Kete source code, but I don't think the test records I put into zoom testing reflect this. Thanks everyone for their work on this. Cheers, Walter On Aug 17, 2007, at 11:03 AM, Jason Ronallo wrote: > I sent this to the list earlier but it didn't get through--probably > because of my (bad) habit of adding a "+list_name" to all mailing > lists. I tried to correct it but the change has not been reflected in > my address yet. jnr > ------------------------ > Hi, > First I wanted to apologize for my part in the miscommunication > over branching. > > On testing: All the unit tests for search passed. All the uncommented > tests for package passed. > > I set up a local test zebra server and tested extended services > create, update and delete. Simple grs.xml indexing of marcxml records. > I then wrote a test script (not a unit test per se) against that > database. I used the tests written by Walter but currently commented > out in trunk as an example. I successfully created, updated and > deleted the record. > > Walter, so the unit tests you wrote fail for your test server, but you > have a working system where extended services CUD works fine? You > suggest that record syntax might be at fault, and I could also see the > zebra setup causing problems. Are you using the same records and zebra > server for the tests as you are on your working system? > > How important is it to have a test server set up so that the CUD unit > tests can work before making a release? > > I don't know how long it will take me (especially since classes are > starting back up again) but I hope to do a few things here unless > someone gets to it first. I hope to set up a public zebra server to > index marcxml records with a very small register for testing, but I > don't know that I can guarantee uptime. I want to rewrite and add to > the CRUD unit tests that use that public server. I want to get a new > website and documentation in place to reflect the addition of extended > services support. But with classes starting I can't expect to do it > right away. > > --Jason > > > On 8/16/07, Walter McGinnis wrote: >> Hi all, >> >> As you know I have committed code to add ZOOM Extended Services via >> ZOOM::Connection#packages. Thanks Nicolai for the good work! It's >> in trunk and also there is a 0.4.0 tag as well. >> >> Currently all previous tests pass. In addition, Nicolai added tests >> for the new code that doesn't rely on connecting to an actual Z39.50 >> server. These all pass. I also added tests that relied on a test >> Z39.50 server that I can't guarantee will be available. I have >> commented these out. They are meant to test the CRUD capabilities >> (or more accurately the CUD capabilities) of Extended Services. >> >> These commented out tests have two issues: >> >> 1. a reliable Z39.50 test server needs to be set up that is >> configured to handle Extended Services >> 2. the update and destroy tests were failing because the test records >> weren't being matched properly >> >> When creating the tests, I simply went with XML that would work with >> my other servers. Quite complex records for testing. In the process >> I set the record files up wrong and thus the tests failed. My >> recommendation is that when we determine the test Z39.50 test server >> we evaluate the appropriate test records at the time. This should >> fix the failing update and delete tests. >> >> It should be noted that my tests using real records via my >> acts_as_zoom Rails plugin all passed for update and delete. So I'm >> confidant the problem isn't with the new Extended Services code, but >> with my tests in zoom 0.4.0. >> >> So what do you people think is the way forward towards a release? >> >> Cheers, >> Walter >> >> >> >> _______________________________________________ >> Ruby-zoom-devel mailing list >> Ruby-zoom-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/ruby-zoom-devel >> From jronallo at gmail.com Mon Aug 20 09:56:41 2007 From: jronallo at gmail.com (Jason Ronallo) Date: Mon, 20 Aug 2007 09:56:41 -0400 Subject: [Ruby-zoom-devel] Release for ZOOM Extended Services In-Reply-To: <42A70EC6-078E-459B-97F6-D916967CA288@katipo.co.nz> References: <0B27FD14-4156-4A5A-8FFF-B730502503B5@katipo.co.nz> <763570460708161603h2d256bc2g51983fbd4fb15518@mail.gmail.com> <42A70EC6-078E-459B-97F6-D916967CA288@katipo.co.nz> Message-ID: <763570460708200656h4a782be6nc4c72e81423d44c2@mail.gmail.com> If we won't have a permanent test server, do you think it'd be a good idea to check in an example zebra server config (default.idx, tab, etc.) that all tests are run against before any release? Then the tests could be written to test against a localhost zebra server on the default port. Where should something like this live (which directory?) --Jason On 8/16/07, Walter McGinnis wrote: > I'm glad that we have straightened things out. Great news about your > tests. > > I'm not certain that having a reliable test server for Extended > Services is necessary. Essentially if we can test successfully > before building the gem for release, that seems sufficient (although > potentially confusing for someone browsing the source). > > I would say updating the API reference to reflect Extended Services > is a higher prioritiy. > > As far as my test server, I think the filtering of the XML record is > being picky about the datestamp field. I have adjusted this in my > records in the Kete source code, but I don't think the test records I > put into zoom testing reflect this. > > Thanks everyone for their work on this. > > Cheers, > Walter > > > On Aug 17, 2007, at 11:03 AM, Jason Ronallo wrote: > > > I sent this to the list earlier but it didn't get through--probably > > because of my (bad) habit of adding a "+list_name" to all mailing > > lists. I tried to correct it but the change has not been reflected in > > my address yet. jnr > > ------------------------ > > Hi, > > First I wanted to apologize for my part in the miscommunication > > over branching. > > > > On testing: All the unit tests for search passed. All the uncommented > > tests for package passed. > > > > I set up a local test zebra server and tested extended services > > create, update and delete. Simple grs.xml indexing of marcxml records. > > I then wrote a test script (not a unit test per se) against that > > database. I used the tests written by Walter but currently commented > > out in trunk as an example. I successfully created, updated and > > deleted the record. > > > > Walter, so the unit tests you wrote fail for your test server, but you > > have a working system where extended services CUD works fine? You > > suggest that record syntax might be at fault, and I could also see the > > zebra setup causing problems. Are you using the same records and zebra > > server for the tests as you are on your working system? > > > > How important is it to have a test server set up so that the CUD unit > > tests can work before making a release? > > > > I don't know how long it will take me (especially since classes are > > starting back up again) but I hope to do a few things here unless > > someone gets to it first. I hope to set up a public zebra server to > > index marcxml records with a very small register for testing, but I > > don't know that I can guarantee uptime. I want to rewrite and add to > > the CRUD unit tests that use that public server. I want to get a new > > website and documentation in place to reflect the addition of extended > > services support. But with classes starting I can't expect to do it > > right away. > > > > --Jason > > > > > > On 8/16/07, Walter McGinnis wrote: > >> Hi all, > >> > >> As you know I have committed code to add ZOOM Extended Services via > >> ZOOM::Connection#packages. Thanks Nicolai for the good work! It's > >> in trunk and also there is a 0.4.0 tag as well. > >> > >> Currently all previous tests pass. In addition, Nicolai added tests > >> for the new code that doesn't rely on connecting to an actual Z39.50 > >> server. These all pass. I also added tests that relied on a test > >> Z39.50 server that I can't guarantee will be available. I have > >> commented these out. They are meant to test the CRUD capabilities > >> (or more accurately the CUD capabilities) of Extended Services. > >> > >> These commented out tests have two issues: > >> > >> 1. a reliable Z39.50 test server needs to be set up that is > >> configured to handle Extended Services > >> 2. the update and destroy tests were failing because the test records > >> weren't being matched properly > >> > >> When creating the tests, I simply went with XML that would work with > >> my other servers. Quite complex records for testing. In the process > >> I set the record files up wrong and thus the tests failed. My > >> recommendation is that when we determine the test Z39.50 test server > >> we evaluate the appropriate test records at the time. This should > >> fix the failing update and delete tests. > >> > >> It should be noted that my tests using real records via my > >> acts_as_zoom Rails plugin all passed for update and delete. So I'm > >> confidant the problem isn't with the new Extended Services code, but > >> with my tests in zoom 0.4.0. > >> > >> So what do you people think is the way forward towards a release? > >> > >> Cheers, > >> Walter > >> > >> > >> > >> _______________________________________________ > >> Ruby-zoom-devel mailing list > >> Ruby-zoom-devel at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/ruby-zoom-devel > >> > > _______________________________________________ > Ruby-zoom-devel mailing list > Ruby-zoom-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/ruby-zoom-devel > From walter at katipo.co.nz Mon Aug 20 18:54:05 2007 From: walter at katipo.co.nz (Walter McGinnis) Date: Tue, 21 Aug 2007 10:54:05 +1200 Subject: [Ruby-zoom-devel] Release for ZOOM Extended Services In-Reply-To: <763570460708200656h4a782be6nc4c72e81423d44c2@mail.gmail.com> References: <0B27FD14-4156-4A5A-8FFF-B730502503B5@katipo.co.nz> <763570460708161603h2d256bc2g51983fbd4fb15518@mail.gmail.com> <42A70EC6-078E-459B-97F6-D916967CA288@katipo.co.nz> <763570460708200656h4a782be6nc4c72e81423d44c2@mail.gmail.com> Message-ID: <22AAC9CA-1F41-410D-B9D0-24B2376C5CDB@katipo.co.nz> Kete actually includes a zebradb subdirectory with tab, conf, directories where the db files live, etc. under it's rails root (i.e. application root) and this works quite well. I think your idea sounds like a good plan. I would think putting a zebra directory under ruby-zoom/test/ would do the trick. Cheers, Walter On Aug 21, 2007, at 1:56 AM, Jason Ronallo wrote: > If we won't have a permanent test server, do you think it'd be a good > idea to check in an example zebra server config (default.idx, tab, > etc.) that all tests are run against before any release? Then the > tests could be written to test against a localhost zebra server on the > default port. Where should something like this live (which directory?) > > --Jason > > On 8/16/07, Walter McGinnis wrote: >> I'm glad that we have straightened things out. Great news about your >> tests. >> >> I'm not certain that having a reliable test server for Extended >> Services is necessary. Essentially if we can test successfully >> before building the gem for release, that seems sufficient (although >> potentially confusing for someone browsing the source). >> >> I would say updating the API reference to reflect Extended Services >> is a higher prioritiy. >> >> As far as my test server, I think the filtering of the XML record is >> being picky about the datestamp field. I have adjusted this in my >> records in the Kete source code, but I don't think the test records I >> put into zoom testing reflect this. >> >> Thanks everyone for their work on this. >> >> Cheers, >> Walter >> >> >> On Aug 17, 2007, at 11:03 AM, Jason Ronallo wrote: >> >>> I sent this to the list earlier but it didn't get through--probably >>> because of my (bad) habit of adding a "+list_name" to all mailing >>> lists. I tried to correct it but the change has not been >>> reflected in >>> my address yet. jnr >>> ------------------------ >>> Hi, >>> First I wanted to apologize for my part in the miscommunication >>> over branching. >>> >>> On testing: All the unit tests for search passed. All the >>> uncommented >>> tests for package passed. >>> >>> I set up a local test zebra server and tested extended services >>> create, update and delete. Simple grs.xml indexing of marcxml >>> records. >>> I then wrote a test script (not a unit test per se) against that >>> database. I used the tests written by Walter but currently commented >>> out in trunk as an example. I successfully created, updated and >>> deleted the record. >>> >>> Walter, so the unit tests you wrote fail for your test server, >>> but you >>> have a working system where extended services CUD works fine? You >>> suggest that record syntax might be at fault, and I could also >>> see the >>> zebra setup causing problems. Are you using the same records and >>> zebra >>> server for the tests as you are on your working system? >>> >>> How important is it to have a test server set up so that the CUD >>> unit >>> tests can work before making a release? >>> >>> I don't know how long it will take me (especially since classes are >>> starting back up again) but I hope to do a few things here unless >>> someone gets to it first. I hope to set up a public zebra server to >>> index marcxml records with a very small register for testing, but I >>> don't know that I can guarantee uptime. I want to rewrite and add to >>> the CRUD unit tests that use that public server. I want to get a new >>> website and documentation in place to reflect the addition of >>> extended >>> services support. But with classes starting I can't expect to do it >>> right away. >>> >>> --Jason >>> >>> >>> On 8/16/07, Walter McGinnis wrote: >>>> Hi all, >>>> >>>> As you know I have committed code to add ZOOM Extended Services via >>>> ZOOM::Connection#packages. Thanks Nicolai for the good work! It's >>>> in trunk and also there is a 0.4.0 tag as well. >>>> >>>> Currently all previous tests pass. In addition, Nicolai added >>>> tests >>>> for the new code that doesn't rely on connecting to an actual >>>> Z39.50 >>>> server. These all pass. I also added tests that relied on a test >>>> Z39.50 server that I can't guarantee will be available. I have >>>> commented these out. They are meant to test the CRUD capabilities >>>> (or more accurately the CUD capabilities) of Extended Services. >>>> >>>> These commented out tests have two issues: >>>> >>>> 1. a reliable Z39.50 test server needs to be set up that is >>>> configured to handle Extended Services >>>> 2. the update and destroy tests were failing because the test >>>> records >>>> weren't being matched properly >>>> >>>> When creating the tests, I simply went with XML that would work >>>> with >>>> my other servers. Quite complex records for testing. In the >>>> process >>>> I set the record files up wrong and thus the tests failed. My >>>> recommendation is that when we determine the test Z39.50 test >>>> server >>>> we evaluate the appropriate test records at the time. This should >>>> fix the failing update and delete tests. >>>> >>>> It should be noted that my tests using real records via my >>>> acts_as_zoom Rails plugin all passed for update and delete. So I'm >>>> confidant the problem isn't with the new Extended Services code, >>>> but >>>> with my tests in zoom 0.4.0. >>>> >>>> So what do you people think is the way forward towards a release? >>>> >>>> Cheers, >>>> Walter >>>> >>>> >>>> >>>> _______________________________________________ >>>> Ruby-zoom-devel mailing list >>>> Ruby-zoom-devel at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/ruby-zoom-devel >>>> >> >> _______________________________________________ >> Ruby-zoom-devel mailing list >> Ruby-zoom-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/ruby-zoom-devel >> > _______________________________________________ > Ruby-zoom-devel mailing list > Ruby-zoom-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/ruby-zoom-devel From ehs at pobox.com Tue Aug 21 13:16:46 2007 From: ehs at pobox.com (Ed Summers) Date: Tue, 21 Aug 2007 13:16:46 -0400 Subject: [Ruby-zoom-devel] erm, is this thing on? Message-ID: I feel like a true moron...I have been sending messages to the discussion list without being subscribed, and gmail has been quietly stashing away messages as spam. So my apologies for being incommunicado. I like the idea of including a zoom configuration with the test suite, and then starting up a server to run the tests against. I actually split off the live test from the vanilla test suite in the Rakefile so you can them explicitly with: rake live_test Embarrassingly yours, Ed From walter at katipo.co.nz Tue Aug 21 20:35:31 2007 From: walter at katipo.co.nz (Walter McGinnis) Date: Wed, 22 Aug 2007 12:35:31 +1200 Subject: [Ruby-zoom-devel] erm, is this thing on? In-Reply-To: References: Message-ID: <9995A6EC-C437-4CF1-B03D-B74D920C065D@katipo.co.nz> Cool, sounds like we are all agreed. So where is the codebase at as far as implementing the zebra config and new tests? I wonder if there is a feed for changes in a rubyforge repository... Cheers, Walter On Aug 22, 2007, at 5:16 AM, Ed Summers wrote: > I feel like a true moron...I have been sending messages to the > discussion list without being subscribed, and gmail has been quietly > stashing away messages as spam. So my apologies for being > incommunicado. > > I like the idea of including a zoom configuration with the test suite, > and then starting up a server to run the tests against. I actually > split off the live test from the vanilla test suite in the Rakefile so > you can them explicitly with: > > rake live_test > > Embarrassingly yours, > Ed > _______________________________________________ > Ruby-zoom-devel mailing list > Ruby-zoom-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/ruby-zoom-devel From jronallo at gmail.com Wed Aug 22 11:58:12 2007 From: jronallo at gmail.com (Jason Ronallo) Date: Wed, 22 Aug 2007 11:58:12 -0400 Subject: [Ruby-zoom-devel] test zebra database commit Message-ID: <763570460708220858w7ce3e6abk570dead766a9f716@mail.gmail.com> Hi, I did an initial commit of a simple zebra test server configuration and live_test unit tests. The test server indexes marcxml records. It tests create, update and delete. Some of the zebra configuration could be much simpler for our needs but I just left a lot from my own project testing in there. It inserts the records for Programming Ruby into zebra, checks for its presence. Then updates the record to include "Jason" instead of "David" in the 100$a field and tests that a record can be found with the author "Jason" and not the author "David." It then deletes the record and tests that no record of that id exists in the database. It's my first big commit to a shared project so please try it out and take a good look at it. All I can say is that it works for me. But please make sure it's really testing what I think it is. Zebra 2.0 must be installed and then simply running "rake live_test" ought to bring up the server on the default port 9999 (if a server is not already running on this port) and runs the unit tests. Currently the zebrasrv needs to be manually killed. Walter, let me know if this test needs to run on a different port to not conflict with your own testing. And if it doesn't work please revert trunk back to revision 63. :) --Jason From jronallo at gmail.com Wed Aug 22 12:31:08 2007 From: jronallo at gmail.com (Jason Ronallo) Date: Wed, 22 Aug 2007 12:31:08 -0400 Subject: [Ruby-zoom-devel] Rakefile changes Message-ID: <763570460708220931m4a00dd78tc6aa22902172a8e9@mail.gmail.com> Ed, I wanted to bring you attention to a change I made to get the tests to run for me. You were using "-r zoom" to load zoom, but that wasn't working. I commented out the ruby_opts lines (removing ", -r zoom" would have worked as well) and just included zoom as a rubygem in the tests. That's the only way I could get either one of them working. What I read said that gems could not be included via the "-r" switch. But maybe that code wasn't intending to include zoom as a gem but trying to use zoom from source? In which case that now needs fixed. Could it be a problem with my environment variables? Here's the error I got: /usr/bin/ruby1.8 -Ilib -r test/unit -I src -r zoom "/var/lib/gems/1.8/gems/rake-0.7.3/lib/rake/rake_test_loader.rb" "test/package_test.rb" "test/search_test.rb" /usr/bin/ruby1.8: no such file to load -- zoom (LoadError) rake aborted! Command failed with status (1): [/usr/bin/ruby1.8 -Ilib -r test/unit -I src...] --Jason From ehs at pobox.com Wed Aug 22 13:39:05 2007 From: ehs at pobox.com (Ed Summers) Date: Wed, 22 Aug 2007 13:39:05 -0400 Subject: [Ruby-zoom-devel] Rakefile changes In-Reply-To: <763570460708220931m4a00dd78tc6aa22902172a8e9@mail.gmail.com> References: <763570460708220931m4a00dd78tc6aa22902172a8e9@mail.gmail.com> Message-ID: The problem probably was you didn't 'build' before 'test' ... Thing work for me, in fact I've remove the 'require' statements in the individual test/*_test.rb files. //Ed From ehs at pobox.com Wed Aug 22 14:48:02 2007 From: ehs at pobox.com (Ed Summers) Date: Wed, 22 Aug 2007 14:48:02 -0400 Subject: [Ruby-zoom-devel] live_test and the release Message-ID: Nice work guys! The live_test is working for me know too. I could imagine some syntactic sugar--to make the api a bit more at home in rubyland...but perhaps acts_as_zoom will be enough. Should we distribute acts_as_zoom with ruby-zoom? //Ed From walter at katipo.co.nz Wed Aug 22 15:53:11 2007 From: walter at katipo.co.nz (Walter McGinnis) Date: Thu, 23 Aug 2007 07:53:11 +1200 Subject: [Ruby-zoom-devel] live_test and the release In-Reply-To: References: Message-ID: <35BF23C3-129C-43D5-B4F5-43CCE71416F3@katipo.co.nz> Yep, thanks Jason and Ed for getting the testing going. Much appreciated. I usually run multiple zebra instances (each Kete instance has "private" and "public" zebra databases) and tend to stay away from port 9999 because of this. So not a hassle for me. acts_as_zoom is one of my earliest pieces of ruby and could probably use some eyes on it. You can check it out from here: http://svn.kete.net.nz/projects/acts_as_zoom/trunk Or you can have a browse or put in tickets here: http://development.kete.net.nz/acts-as-zoom/repository/browse and http://development.kete.net.nz/acts-as-zoom/tickets It feels more a like a Rails plugin to me than a gem, but I'm open to opinions on it. I can set up commit rights for whomever would like them. Refactoring or extensions appreciated. I sort of punted on abstracting queries. I formulate my raw query in my search mechanism directly. I'm also open to moving any useful elements that are more general to ruby-zoom. All that being said, I'm taking a long weekend. So won't be able to do much until Monday, NZ time. Cheers, Walter On Aug 23, 2007, at 6:48 AM, Ed Summers wrote: > Nice work guys! The live_test is working for me know too. I could > imagine some syntactic sugar--to make the api a bit more at home in > rubyland...but perhaps acts_as_zoom will be enough. Should we > distribute acts_as_zoom with ruby-zoom? > > //Ed > _______________________________________________ > Ruby-zoom-devel mailing list > Ruby-zoom-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/ruby-zoom-devel From walter at katipo.co.nz Wed Aug 22 15:57:17 2007 From: walter at katipo.co.nz (Walter McGinnis) Date: Thu, 23 Aug 2007 07:57:17 +1200 Subject: [Ruby-zoom-devel] live_test and the release In-Reply-To: <35BF23C3-129C-43D5-B4F5-43CCE71416F3@katipo.co.nz> References: <35BF23C3-129C-43D5-B4F5-43CCE71416F3@katipo.co.nz> Message-ID: <0471B579-57D2-41E9-898E-830CD66A68D8@katipo.co.nz> And embarrassingly acts_as_zoom doesn't have any tests. For shame. Cheers, Walter On Aug 23, 2007, at 7:53 AM, Walter McGinnis wrote: > Yep, thanks Jason and Ed for getting the testing going. Much > appreciated. > > I usually run multiple zebra instances (each Kete instance has > "private" and "public" zebra databases) and tend to stay away from > port 9999 because of this. So not a hassle for me. > > acts_as_zoom is one of my earliest pieces of ruby and could > probably use some eyes on it. You can check it out from here: > > http://svn.kete.net.nz/projects/acts_as_zoom/trunk > > Or you can have a browse or put in tickets here: > > http://development.kete.net.nz/acts-as-zoom/repository/browse > > and > > http://development.kete.net.nz/acts-as-zoom/tickets > > It feels more a like a Rails plugin to me than a gem, but I'm open > to opinions on it. I can set up commit rights for whomever would > like them. Refactoring or extensions appreciated. I sort of > punted on abstracting queries. I formulate my raw query in my > search mechanism directly. I'm also open to moving any useful > elements that are more general to ruby-zoom. > > All that being said, I'm taking a long weekend. So won't be able > to do much until Monday, NZ time. > > Cheers, > Walter > > On Aug 23, 2007, at 6:48 AM, Ed Summers wrote: > >> Nice work guys! The live_test is working for me know too. I could >> imagine some syntactic sugar--to make the api a bit more at home in >> rubyland...but perhaps acts_as_zoom will be enough. Should we >> distribute acts_as_zoom with ruby-zoom? >> >> //Ed >> _______________________________________________ >> Ruby-zoom-devel mailing list >> Ruby-zoom-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/ruby-zoom-devel > From jronallo at gmail.com Mon Aug 27 10:25:27 2007 From: jronallo at gmail.com (Jason Ronallo) Date: Mon, 27 Aug 2007 10:25:27 -0400 Subject: [Ruby-zoom-devel] destroy connection at end of block Message-ID: <763570460708270725i3ab904fcy6c368b8e7398be80@mail.gmail.com> Hi, I've been playing more with zoom now and discovering something that seems to not be behaving properly. The API docs say that if passed a block Connection#open will automatically destroy the connection at the end of the block. I don't know that this is happening. If I connect to my zebra server with multiple instances of yaz-client a new zebrasrv process is started with each one. The process ends when I either close the connection or exit yaz-client. If I create a script which opens several connections, the connections are not destroyed at the end of the block. I can still see the generated zebrasrv processes running. They are only killed when the script ends. I'm not sure what to make of this really. Maybe something else is going on? Walter, if the connections are not being closed at the end of the block, I thought this might be important for you. Don't want too many resources taken up by dead connections. I looked at the C source and it looks like the data wrap struct is supposed to destroy the connection, but it doesn't seem to be happening. I poked trying to get it to worked, but not knowing any C I, of course, failed. I was able to get Connection#new to accept a block, though. Might be a feature we'd like to add on? --Jason From jronallo at gmail.com Mon Aug 27 10:57:58 2007 From: jronallo at gmail.com (Jason Ronallo) Date: Mon, 27 Aug 2007 10:57:58 -0400 Subject: [Ruby-zoom-devel] database_name warnings Message-ID: <763570460708270757j66883a44x8c8ef190563a28ec@mail.gmail.com> Since using the new version of zoom with extended services with my copy cataloging script, I was getting the following warnings as the script starts up: (eval):1: warning: method redefined; discarding old database_name (eval):2: warning: method redefined; discarding old database_name= (eval):3: warning: method redefined; discarding old set_database_name I had one use of database_name in my script (and all my own libraries), removed it, and still got the warnings. I found define_zoom_option (c, "databaseName"); twice at the end of rbzoompackage.c. Removing the second one got rid of the warnings. It passes all tests. Think it would be ok to commit that change? I've also got changes to package_live.rb that brings down the test zebrasrv on exit but only for *nix. The other little change I made was to use full zurl 'localhost:9999/test' for connections rather than also setting the database_name after the connection is created. --Jason From ehs at pobox.com Mon Aug 27 12:36:18 2007 From: ehs at pobox.com (Ed Summers) Date: Mon, 27 Aug 2007 12:36:18 -0400 Subject: [Ruby-zoom-devel] database_name warnings In-Reply-To: <763570460708270757j66883a44x8c8ef190563a28ec@mail.gmail.com> References: <763570460708270757j66883a44x8c8ef190563a28ec@mail.gmail.com> Message-ID: On 8/27/07, Jason Ronallo wrote: > I found define_zoom_option (c, "databaseName"); > twice at the end of rbzoompackage.c. Removing the second one got rid > of the warnings. It passes all tests. Think it would be ok to commit > that change? Nice catch! Please commit stuff like this without even asking. If you can just note these kind of changes in the Changes file -- if it's still there. > I've also got changes to package_live.rb that brings down the test > zebrasrv on exit but only for *nix. The other little change I made was > to use full zurl 'localhost:9999/test' for connections rather than > also setting the database_name after the connection is created. Awesome that would be very nice indeed. //Ed From walter at katipo.co.nz Mon Aug 27 15:32:03 2007 From: walter at katipo.co.nz (Walter McGinnis) Date: Tue, 28 Aug 2007 07:32:03 +1200 Subject: [Ruby-zoom-devel] destroy connection at end of block In-Reply-To: <763570460708270725i3ab904fcy6c368b8e7398be80@mail.gmail.com> References: <763570460708270725i3ab904fcy6c368b8e7398be80@mail.gmail.com> Message-ID: I've noticed this as well, but it's not new. In fact, IIRC, the Perl stuff is what I noticed this with. It is annoying. Not sure what to do about it. Cheers, Walter On Aug 28, 2007, at 2:25 AM, Jason Ronallo wrote: > Hi, > I've been playing more with zoom now and discovering something that > seems to not be behaving properly. The API docs say that if passed a > block Connection#open will automatically destroy the connection at the > end of the block. I don't know that this is happening. > > If I connect to my zebra server with multiple instances of yaz-client > a new zebrasrv process is started with each one. The process ends when > I either close the connection or exit yaz-client. > > If I create a script which opens several connections, the connections > are not destroyed at the end of the block. I can still see the > generated zebrasrv processes running. They are only killed when the > script ends. > > I'm not sure what to make of this really. Maybe something else is > going on? > > Walter, if the connections are not being closed at the end of the > block, I thought this might be important for you. Don't want too many > resources taken up by dead connections. > > I looked at the C source and it looks like the data wrap struct is > supposed to destroy the connection, but it doesn't seem to be > happening. I poked trying to get it to worked, but not knowing any C > I, of course, failed. > > I was able to get Connection#new to accept a block, though. Might be a > feature we'd like to add on? > > --Jason > _______________________________________________ > Ruby-zoom-devel mailing list > Ruby-zoom-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/ruby-zoom-devel From n.molesbenfell at gmail.com Mon Aug 27 19:15:29 2007 From: n.molesbenfell at gmail.com (Nicolai Moles-Benfell) Date: Tue, 28 Aug 2007 11:15:29 +1200 Subject: [Ruby-zoom-devel] destroy connection at end of block In-Reply-To: <763570460708270725i3ab904fcy6c368b8e7398be80@mail.gmail.com> References: <763570460708270725i3ab904fcy6c368b8e7398be80@mail.gmail.com> Message-ID: <460C1697-03E5-4A64-A91B-5AD2A4382C45@gmail.com> Hi, I should introduce my self, my name is Nicolai, I am the guy Walter found to extended the Ruby Zoom bridge to include the extended services features of YAZ. So any mistakes/bugs/admissions with the code are mine. I have experience with C and C++ but this was my first look at the Ruby C API, so I followed the existing Ruby Zoom modules for guidance, and looked at the Perl Zoom library for exposing the extended service features. I'd be happy to help out with future requests. On 28/08/2007, at 2:25 AM, Jason Ronallo wrote: > I looked at the C source and it looks like the data wrap struct is > supposed to destroy the connection, but it doesn't seem to be > happening. I poked trying to get it to worked, but not knowing any C > I, of course, failed. > The YAZ library eposes the methods ZOOM_connection_destroy, ZOOM_package_destroy which are being called by the Ruby garbage collector when destroying the Ruby connection or package objects... these methods can be exposed in Ruby or called explicitly at the end of a method that accepts a block... something like if (rb_block_given_p ()) { rb_yield(rb_connect); ZOOM_connect_destroy(cZoomConnect, NULL, ZOOM_connection_destroy, connection) return Qnil; } return rb_connection; Cheers, Nicolai. From jronallo at gmail.com Tue Aug 28 09:14:14 2007 From: jronallo at gmail.com (Jason Ronallo) Date: Tue, 28 Aug 2007 09:14:14 -0400 Subject: [Ruby-zoom-devel] destroy connection at end of block In-Reply-To: <460C1697-03E5-4A64-A91B-5AD2A4382C45@gmail.com> References: <763570460708270725i3ab904fcy6c368b8e7398be80@mail.gmail.com> <460C1697-03E5-4A64-A91B-5AD2A4382C45@gmail.com> Message-ID: <763570460708280614l6ea37d1aw9c1e7d72121650d5@mail.gmail.com> Hi, Nicolai, Thank you for your work on this. It's really great to have the extended services support now. > The YAZ library eposes the methods ZOOM_connection_destroy, > ZOOM_package_destroy which are being called by the Ruby garbage > collector when destroying the Ruby connection or package objects... > these methods can be exposed in Ruby or called explicitly at the end > of a method that accepts a block... I think this is an issue of a disconnect with garbage collection. Even explicitly calling garbage collection after the end of the block in Ruby does not destroy the connection. I wonder if it has to do with the NULL in the code below. From my reading of Ruby C-bindings where that NULL is is the position used for marking an object so in the next sweep it is collected. Without the object being marked I don't think ZOOM_connection_destroy is ever doing any work. How to do that I don't know. As soon as it mentioned allocating memory I backed away. :) Nicolai, would it be possible to write a mark method? That should insure that the connection is in fact closed at the end of a block, right? --Jason > something like > > if (rb_block_given_p ()) { > rb_yield(rb_connect); > ZOOM_connect_destroy(cZoomConnect, NULL, ZOOM_connection_destroy, > connection) > return Qnil; > } > > return rb_connection; From n.molesbenfell at gmail.com Fri Aug 31 03:48:31 2007 From: n.molesbenfell at gmail.com (Nicolai Moles-Benfell) Date: Fri, 31 Aug 2007 19:48:31 +1200 Subject: [Ruby-zoom-devel] destroy connection at end of block In-Reply-To: <763570460708280614l6ea37d1aw9c1e7d72121650d5@mail.gmail.com> References: <763570460708270725i3ab904fcy6c368b8e7398be80@mail.gmail.com> <460C1697-03E5-4A64-A91B-5AD2A4382C45@gmail.com> <763570460708280614l6ea37d1aw9c1e7d72121650d5@mail.gmail.com> Message-ID: <356B7204-3CDC-490A-B5B7-F7E1903DE493@gmail.com> Hi Jason, Sorry for the delay in my replay, I had been involved in a year long (death march) project which finished this afternoon. I think the simplest approach to implementing the mark function would be to follow a Dispose pattern, where a method (say Package#dispose, Connection#dispose) is called at the end of a block or optionally by the user, the mark method would look at a parameter set by the dispose method to determine whether the connection/package can be collected, and if so would free/destroy the package/connection. For people more familiar with Ruby, is a dispose method common to Ruby? is there a more commonly used name for such a method? I finally have some free time, so I'll do a basic implementation this weekend and see if it resolves the open connections. Cheers, Nicolai. On 29/08/2007, at 1:14 AM, Jason Ronallo wrote: > Hi, Nicolai, > > Thank you for your work on this. It's really great to have the > extended services support now. > >> The YAZ library eposes the methods ZOOM_connection_destroy, >> ZOOM_package_destroy which are being called by the Ruby garbage >> collector when destroying the Ruby connection or package objects... >> these methods can be exposed in Ruby or called explicitly at the end >> of a method that accepts a block... > > I think this is an issue of a disconnect with garbage collection. Even > explicitly calling garbage collection after the end of the block in > Ruby does not destroy the connection. > > I wonder if it has to do with the NULL in the code below. From my > reading of Ruby C-bindings where that NULL is is the position used for > marking an object so in the next sweep it is collected. Without the > object being marked I don't think ZOOM_connection_destroy is ever > doing any work. > > How to do that I don't know. As soon as it mentioned allocating memory > I backed away. :) Nicolai, would it be possible to write a mark > method? That should insure that the connection is in fact closed at > the end of a block, right? > > --Jason > > >> something like >> >> if (rb_block_given_p ()) { >> rb_yield(rb_connect); >> ZOOM_connect_destroy(cZoomConnect, NULL, >> ZOOM_connection_destroy, >> connection) >> return Qnil; >> } >> >> return rb_connection; > _______________________________________________ > Ruby-zoom-devel mailing list > Ruby-zoom-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/ruby-zoom-devel