From dan.bikle at gmail.com Wed Mar 4 16:30:06 2009 From: dan.bikle at gmail.com (dan.bikle at gmail.com) Date: Wed, 4 Mar 2009 13:30:06 -0800 (PST) Subject: [libxml-devel] Enclosing class/module 'cXMLHtmlParser' for.. Message-ID: <85187851-f369-42df-af82-70aa7983e11a@o2g2000prl.googlegroups.com> libxml-devel list, I posted a question over on rubyforge yesterday: http://rubyforge.org/forum/forum.php?thread_id=31903&forum_id=2129 I repost here: ruby-xml people, I just tried gem install libxml-ruby on a new installation of rails 2.2.2 on my leopard-mac. Here is a screen dump: mac3:/pt/c3/rho/rhosync maco$ rake gems:install rake gems:install (in /Users/maco/pt/customers3/rho/rhosync) gem install libxml-ruby Enclosing class/module 'cXMLHtmlParser' for module Options not known Enclosing class/module 'cXMLParser' for module Options not known No definition for rb_class_new_instance Enclosing class/module 'cXMLHtmlParser' for module Options not known Enclosing class/module 'cXMLParser' for module Options not known No definition for rb_class_new_instance Building native extensions. This could take a while... Successfully installed libxml-ruby-0.9.8 1 gem installed Installing ri documentation for libxml-ruby-0.9.8... Installing RDoc documentation for libxml-ruby-0.9.8... mac3:/pt/c3/rho/rhosync maco$ Notice the message: Enclosing class/module 'cXMLHtmlParser' for module Options not known Is it indicating a problem? It's not a complete sentence. I don't know what the author of this gem is trying to tell me. Clues anyone? --Dan From cfis at savagexi.com Wed Mar 4 17:34:28 2009 From: cfis at savagexi.com (Charlie Savage) Date: Wed, 04 Mar 2009 15:34:28 -0700 Subject: [libxml-devel] Enclosing class/module 'cXMLHtmlParser' for.. In-Reply-To: <85187851-f369-42df-af82-70aa7983e11a@o2g2000prl.googlegroups.com> References: <85187851-f369-42df-af82-70aa7983e11a@o2g2000prl.googlegroups.com> Message-ID: <49AF01F4.5050801@savagexi.com> > libxml-devel list, > > I posted a question over on rubyforge yesterday: > > http://rubyforge.org/forum/forum.php?thread_id=31903&forum_id=2129 > > > Is it indicating a problem? > It's not a complete sentence. > I don't know what the author of this gem is trying to tell me. Its an older version of Rdoc complaining. You can just ignore it. Alternatively, you can install a new version of Rdoc (gem install rdoc, then delete your existing rdoc installation). Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From cfis at savagexi.com Fri Mar 6 00:45:13 2009 From: cfis at savagexi.com (Charlie Savage) Date: Thu, 05 Mar 2009 22:45:13 -0700 Subject: [libxml-devel] libxml-ruby 0.9.9 with Ruby 1.9.1 support Message-ID: <49B0B869.7010507@savagexi.com> I'm happy to announce the release of libxml-ruby 0.9.9. This release is chock full of goodies: * Ruby 1.9.1 support (Charlie Savage) * Much reduced memory usage (Joe Khoobyar). * Better OS X 10.5 and MacPorts support (Joe Khoobyar). * Tons of bug fixes (see changelog) At this point, the libxml-ruby bindings are in great shape. They are fast, stable, have a clean api and are reasonably well documented (see the latest RDocs). Next up is the 1.0 release. So install the bindings, make sure the test suite passes, and run them with your applications. This is particulary true on OS X, where we've had our struggles recently. Also reports from the field on 1.9.1 would be greatly appreciated. Make sure to report any issues you find on RubyForge: http://rubyforge.org/tracker/?atid=1971&group_id=494&func=browse Thanks, Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From kevin.menard at snapmylife.com Fri Mar 6 11:14:35 2009 From: kevin.menard at snapmylife.com (Kevin Menard) Date: Fri, 6 Mar 2009 11:14:35 -0500 Subject: [libxml-devel] libxml-ruby 0.9.9 with Ruby 1.9.1 support In-Reply-To: <49B0B869.7010507@savagexi.com> References: <49B0B869.7010507@savagexi.com> Message-ID: <296445fb0903060814h3858b9f8ga1570dabbf9f65b7@mail.gmail.com> I just tried to install on MacOS X 10.5 with libxml2 installed via MacPorts and got the following: extconf failure: Your version of libxml2 is too old. Please upgrade. I last upgraded my MacPorts maybe a month back. I'll see if there's a newer libxml package, but this is something of a pain in the neck. -- Kevin 2009/3/6 Charlie Savage > I'm happy to announce the release of libxml-ruby 0.9.9. This release is > chock full of goodies: > > * Ruby 1.9.1 support (Charlie Savage) > * Much reduced memory usage (Joe Khoobyar). > * Better OS X 10.5 and MacPorts support (Joe Khoobyar). > * Tons of bug fixes (see changelog) > > At this point, the libxml-ruby bindings are in great shape. They are fast, > stable, have a clean api and are reasonably well documented (see the latest > RDocs). > > Next up is the 1.0 release. So install the bindings, make sure the test > suite passes, and run them with your applications. This is particulary true > on OS X, where we've had our struggles recently. Also reports from the > field on 1.9.1 would be greatly appreciated. > > Make sure to report any issues you find on RubyForge: > > http://rubyforge.org/tracker/?atid=1971&group_id=494&func=browse > > Thanks, > > Charlie > > > _______________________________________________ > libxml-devel mailing list > libxml-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/libxml-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.menard at snapmylife.com Fri Mar 6 11:18:23 2009 From: kevin.menard at snapmylife.com (Kevin Menard) Date: Fri, 6 Mar 2009 11:18:23 -0500 Subject: [libxml-devel] libxml-ruby 0.9.9 with Ruby 1.9.1 support In-Reply-To: <296445fb0903060814h3858b9f8ga1570dabbf9f65b7@mail.gmail.com> References: <49B0B869.7010507@savagexi.com> <296445fb0903060814h3858b9f8ga1570dabbf9f65b7@mail.gmail.com> Message-ID: <296445fb0903060818q405a86b1ieee3186d1667858f@mail.gmail.com> I finally finished the upgrade. Still the same problem. Looks like someone already reported it though, so sorry for the extra noise: http://rubyforge.org/tracker/index.php?func=detail&aid=24353&group_id=494&atid=1971 -- Kevin On Fri, Mar 6, 2009 at 11:14 AM, Kevin Menard wrote: > I just tried to install on MacOS X 10.5 with libxml2 installed via MacPorts > and got the following: > extconf failure: Your version of libxml2 is too old. Please upgrade. > > I last upgraded my MacPorts maybe a month back. I'll see if there's a > newer libxml package, but this is something of a pain in the neck. > > -- > Kevin > > 2009/3/6 Charlie Savage > >> I'm happy to announce the release of libxml-ruby 0.9.9. This release is >> chock full of goodies: >> >> * Ruby 1.9.1 support (Charlie Savage) >> * Much reduced memory usage (Joe Khoobyar). >> * Better OS X 10.5 and MacPorts support (Joe Khoobyar). >> * Tons of bug fixes (see changelog) >> >> At this point, the libxml-ruby bindings are in great shape. They are fast, >> stable, have a clean api and are reasonably well documented (see the latest >> RDocs). >> >> Next up is the 1.0 release. So install the bindings, make sure the test >> suite passes, and run them with your applications. This is particulary true >> on OS X, where we've had our struggles recently. Also reports from the >> field on 1.9.1 would be greatly appreciated. >> >> Make sure to report any issues you find on RubyForge: >> >> http://rubyforge.org/tracker/?atid=1971&group_id=494&func=browse >> >> Thanks, >> >> Charlie >> >> >> _______________________________________________ >> libxml-devel mailing list >> libxml-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/libxml-devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cfis at savagexi.com Fri Mar 6 11:53:56 2009 From: cfis at savagexi.com (Charlie Savage) Date: Fri, 06 Mar 2009 09:53:56 -0700 Subject: [libxml-devel] libxml-ruby 0.9.9 with Ruby 1.9.1 support In-Reply-To: <296445fb0903060818q405a86b1ieee3186d1667858f@mail.gmail.com> References: <49B0B869.7010507@savagexi.com> <296445fb0903060814h3858b9f8ga1570dabbf9f65b7@mail.gmail.com> <296445fb0903060818q405a86b1ieee3186d1667858f@mail.gmail.com> Message-ID: <49B15524.9050402@savagexi.com> Hi Kevin, Kevin Menard wrote: > I finally finished the upgrade. Still the same problem. Looks like > someone already reported it though, so sorry for the extra noise: > > http://rubyforge.org/tracker/index.php?func=detail&aid=24353&group_id=494&atid=1971 > No problem. Sure you are linking against MacPorts? The version of libxml2 in MacPorts should work fine. I think maybe you have to use --with-xml2-config. Joe? Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From kevin.menard at snapmylife.com Fri Mar 6 14:16:42 2009 From: kevin.menard at snapmylife.com (Kevin Menard) Date: Fri, 6 Mar 2009 14:16:42 -0500 Subject: [libxml-devel] libxml-ruby 0.9.9 with Ruby 1.9.1 support In-Reply-To: <49B15524.9050402@savagexi.com> References: <49B0B869.7010507@savagexi.com> <296445fb0903060814h3858b9f8ga1570dabbf9f65b7@mail.gmail.com> <296445fb0903060818q405a86b1ieee3186d1667858f@mail.gmail.com> <49B15524.9050402@savagexi.com> Message-ID: <296445fb0903061116r2874f9c8mca99002fa4c20053@mail.gmail.com> 2009/3/6 Charlie Savage > > > No problem. Sure you are linking against MacPorts? The version of libxml2 > in MacPorts should work fine. > > I think maybe you have to use --with-xml2-config. Joe? > I've been able to build every version of libxml-ruby up until 0.9.9, save for one version that broke on MacPorts (0.9.3?). So, I have no reason to believe otherwise. The problem I have is precisely that described in the other bug report. -- Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: From johnathan.conley at gmail.com Sat Mar 7 14:04:09 2009 From: johnathan.conley at gmail.com (Johnathan Conley) Date: Sat, 7 Mar 2009 11:04:09 -0800 (PST) Subject: [libxml-devel] libxml-ruby 0.9.9 with Ruby 1.9.1 support In-Reply-To: <296445fb0903061116r2874f9c8mca99002fa4c20053@mail.gmail.com> References: <49B0B869.7010507@savagexi.com> <296445fb0903060814h3858b9f8ga1570dabbf9f65b7@mail.gmail.com> <296445fb0903060818q405a86b1ieee3186d1667858f@mail.gmail.com> <49B15524.9050402@savagexi.com> <296445fb0903061116r2874f9c8mca99002fa4c20053@mail.gmail.com> Message-ID: Kevin, only thing I can think is to make sure that /opt/local/bin and / opt/local/sbin are exported as the first items in your PATH (as per the macports install guide) If libxml-ruby cannot find /opt/local/bin/xml2-config... then it may not work. For example, in my /etc/profile I have: export PATH=/opt/local/bin:/opt/local/sbin:$PATH export MANPATH=/opt/local/share/man:$MANPATH (if you change this, be sure to restart your terminal window) -jc On Mar 6, 1:16?pm, Kevin Menard wrote: > 2009/3/6 Charlie Savage > > > > > No problem. ?Sure you are linking against MacPorts? ?The version of libxml2 > > in MacPorts should work fine. > > > I think maybe you have to use --with-xml2-config. ?Joe? > > I've been able to build every version of libxml-ruby up until 0.9.9, save > for one version that broke on MacPorts (0.9.3?). ?So, I have no reason to > believe otherwise. ?The problem I have is precisely that described in the > other bug report. > > -- > Kevin > > _______________________________________________ > libxml-devel mailing list > libxml-de... at rubyforge.orghttp://rubyforge.org/mailman/listinfo/libxml-devel From cfis at savagexi.com Sat Mar 7 17:32:08 2009 From: cfis at savagexi.com (Charlie Savage) Date: Sat, 07 Mar 2009 15:32:08 -0700 Subject: [libxml-devel] libxml-ruby 0.9.9 with Ruby 1.9.1 support In-Reply-To: References: <49B0B869.7010507@savagexi.com> <296445fb0903060814h3858b9f8ga1570dabbf9f65b7@mail.gmail.com> <296445fb0903060818q405a86b1ieee3186d1667858f@mail.gmail.com> <49B15524.9050402@savagexi.com> <296445fb0903061116r2874f9c8mca99002fa4c20053@mail.gmail.com> Message-ID: <49B2F5E8.8070705@savagexi.com> > Kevin, only thing I can think is to make sure that /opt/local/bin and / > opt/local/sbin are exported as the first items in your PATH (as per > the macports install guide) > If libxml-ruby cannot find /opt/local/bin/xml2-config... then it may > not work. > > For example, in my /etc/profile I have: > export PATH=/opt/local/bin:/opt/local/sbin:$PATH > export MANPATH=/opt/local/share/man:$MANPATH > Fixed in 1.0.0. Problem was that on OS X ruby is compiled as a universal binary. Which means ruby extensions will be built as univeral binaries. So if you install libxml2 like this: sudo port install libxml2 It won't work, because you'll get a link error when building a ppc binary. What you should do is: sudo port install libxml2 +universal Alternatively, sudo -s ARCHFLAGS="-arch i386" gem install libxml-ruby See http://www.macruby.org/trac/wiki/Troubleshooting Anyway, you no longer have to worry about this with 1.0.0. Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From cfis at savagexi.com Sat Mar 7 17:36:40 2009 From: cfis at savagexi.com (Charlie Savage) Date: Sat, 07 Mar 2009 15:36:40 -0700 Subject: [libxml-devel] Announcing libxml-ruby 1.0.0 Message-ID: <49B2F6F8.1070500@savagexi.com> I'm happy to announce the release of libxml-ruby 1.0.0 (finally!). The only change in this release is better support for OS X. The bindings now work with both: * The default version of libxml2 (2.6.16) installed with OS X 10.5 * Newer versions installed via MacPorts Enjoy! Charlie --------------- == 1.0.0 / 2009-03-05 Charlie Savage * OS X (Charlie Savage). Update bindings to support the default installed version of libxml2 (2.6.16) on OS X 10.5 and the latest version available via MacPorts. == 0.9.9 / 2009-03-05 Charlie Savage * Ruby 1.9.1 support (Charlie Savage). libxml-ruby now compiles and runs on either 1.8.6 and 1.9.1. With 1.8.6 all tests should pass while on 1.9.1 all but for encoding tests pass. The port to Ruby 1.9.1 revealed two memory allocation bugs (one with dtds, one with nodes) which are now fixed. * Better OS X support (Joe Khoobyar). The default version of libxml2 on OS X 10.5 is fairly old, resulting in this link error: NSLinkModule() error dyld: Symbol not found: _htmlNewParserCtxt This can be fixed by using MacPorts to get a newer version of libxml2. To make use of MacPorts, the build script has been updated to use xml2-config. This can be fine-tuned using the new --with-xml2-config / --without-xml2-config options to extconf.rb (default is --without-xml2-config to match existing behavior). * Greatly reduced memory usage (Joe Khoobyar). See http://rubyforge.org/pipermail/libxml-devel/2009-February/001375.html. * Add Document#xhtml? and document#node_type methods (Joe Khoobyar) * Add XPath::Object#last (Joe Khoobyar) * Provide finer control over CDATA nodes on a parser by parser basis (Joe Khoobyar). * Bug fix - Namespaces were incorrectly merged with attributes in the new sax2 handler (Charlie Savage). * Bug fix - Support iterating over nodes and attributes even with blocks that call remove! (Charlie Savage) * Bug fix - If reader.node is NULL, return nil instead of crashing (Charlie Savage) * Bug fix - Dtd's owned by documents were freed twice in some circumstances (Joe Khoobyar). * Bug fix - Fix output escaping on attributes nodes (Joe Khoobyar). * Bug fix - Make sure IO objects are not garbage collected when used as parser sources (Charlie Savage). -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From kevin.menard at snapmylife.com Sat Mar 7 22:49:00 2009 From: kevin.menard at snapmylife.com (Kevin Menard) Date: Sat, 7 Mar 2009 22:49:00 -0500 Subject: [libxml-devel] libxml-ruby 0.9.9 with Ruby 1.9.1 support In-Reply-To: <49B2F5E8.8070705@savagexi.com> References: <49B0B869.7010507@savagexi.com> <296445fb0903060814h3858b9f8ga1570dabbf9f65b7@mail.gmail.com> <296445fb0903060818q405a86b1ieee3186d1667858f@mail.gmail.com> <49B15524.9050402@savagexi.com> <296445fb0903061116r2874f9c8mca99002fa4c20053@mail.gmail.com> <49B2F5E8.8070705@savagexi.com> Message-ID: <296445fb0903071949n537fe19tb5e0731604489a60@mail.gmail.com> That indeed did the trick. Thanks for the quick turnaround. -- Kevin 2009/3/7 Charlie Savage > Kevin, only thing I can think is to make sure that /opt/local/bin and / >> opt/local/sbin are exported as the first items in your PATH (as per >> the macports install guide) >> If libxml-ruby cannot find /opt/local/bin/xml2-config... then it may >> not work. >> >> For example, in my /etc/profile I have: >> export PATH=/opt/local/bin:/opt/local/sbin:$PATH >> export MANPATH=/opt/local/share/man:$MANPATH >> >> > Fixed in 1.0.0. > > Problem was that on OS X ruby is compiled as a universal binary. Which > means ruby extensions will be built as univeral binaries. > > So if you install libxml2 like this: > > sudo port install libxml2 > > It won't work, because you'll get a link error when building a ppc binary. > What you should do is: > > sudo port install libxml2 +universal > > Alternatively, > > sudo -s > ARCHFLAGS="-arch i386" gem install libxml-ruby > > See http://www.macruby.org/trac/wiki/Troubleshooting > > Anyway, you no longer have to worry about this with 1.0.0. > > Charlie > > > > _______________________________________________ > libxml-devel mailing list > libxml-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/libxml-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jacob.lauemoeller at iteray.com Sun Mar 8 03:10:02 2009 From: jacob.lauemoeller at iteray.com (=?UTF-8?Q?Jacob_Lauem=C3=B8ller?=) Date: Sun, 8 Mar 2009 08:10:02 +0100 Subject: [libxml-devel] Announcing libxml-ruby 1.0.0 In-Reply-To: <49B2F6F8.1070500@savagexi.com> References: <49B2F6F8.1070500@savagexi.com> Message-ID: Thank you! On 07/03/2009, at 23.36, Charlie Savage wrote: > I'm happy to announce the release of libxml-ruby 1.0.0 (finally!). > > The only change in this release is better support for OS X. The > bindings now work with both: > > * The default version of libxml2 (2.6.16) installed with OS X 10.5 > * Newer versions installed via MacPorts > > Enjoy! > > Charlie > > --------------- > == 1.0.0 / 2009-03-05 Charlie Savage > > * OS X (Charlie Savage). Update bindings to support the default > installed > version of libxml2 (2.6.16) on OS X 10.5 and the latest version > available > via MacPorts. > > > == 0.9.9 / 2009-03-05 Charlie Savage > > * Ruby 1.9.1 support (Charlie Savage). libxml-ruby now compiles and > runs on either > 1.8.6 and 1.9.1. With 1.8.6 all tests should pass while on 1.9.1 > all but > for encoding tests pass. The port to Ruby 1.9.1 revealed two memory > allocation bugs (one with dtds, one with nodes) which are now fixed. > > * Better OS X support (Joe Khoobyar). The default version of libxml2 > on OS X 10.5 is fairly old, resulting in this link error: > > NSLinkModule() error > dyld: Symbol not found: _htmlNewParserCtxt > > This can be fixed by using MacPorts to get a newer version of > libxml2. > To make use of MacPorts, the build script has been updated to use > xml2-config. > This can be fine-tuned using the new --with-xml2-config / --without- > xml2-config > options to extconf.rb (default is --without-xml2-config to match > existing behavior). > > * Greatly reduced memory usage (Joe Khoobyar). > See http://rubyforge.org/pipermail/libxml-devel/2009-February/001375.html > . > > * Add Document#xhtml? and document#node_type methods (Joe Khoobyar) > > * Add XPath::Object#last (Joe Khoobyar) > > * Provide finer control over CDATA nodes on a parser by parser basis > (Joe Khoobyar). > > * Bug fix - Namespaces were incorrectly merged with attributes in > the new sax2 > handler (Charlie Savage). > > * Bug fix - Support iterating over nodes and attributes even with > blocks > that call remove! (Charlie Savage) > > * Bug fix - If reader.node is NULL, return nil instead of crashing > (Charlie Savage) > > * Bug fix - Dtd's owned by documents were freed twice in some > circumstances (Joe Khoobyar). > > * Bug fix - Fix output escaping on attributes nodes (Joe Khoobyar). > > * Bug fix - Make sure IO objects are not garbage collected when used > as parser sources (Charlie Savage). > _______________________________________________ > libxml-devel mailing list > libxml-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/libxml-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 1933 bytes Desc: not available URL: From chris at octopod.info Sun Mar 8 06:18:29 2009 From: chris at octopod.info (Chris McGrath) Date: Sun, 8 Mar 2009 10:18:29 +0000 Subject: [libxml-devel] Announcing libxml-ruby 1.0.0 In-Reply-To: <49B2F6F8.1070500@savagexi.com> References: <49B2F6F8.1070500@savagexi.com> Message-ID: <23e3fe0f0903080318i2edc251dx881f0056eff58bd7@mail.gmail.com> 2009/3/7 Charlie Savage : > I'm happy to announce the release of libxml-ruby 1.0.0 (finally!). Great news! Thanks to you and everyone who contributed for all your hard work on this. Cheers, Chris From tom at compton.nu Mon Mar 9 11:24:55 2009 From: tom at compton.nu (Tom Hughes) Date: Mon, 09 Mar 2009 15:24:55 +0000 Subject: [libxml-devel] SEGV with libxml-ruby 1.0.0 Message-ID: <49B534C7.8030303@compton.nu> Having upgraded to libxml-ruby 1.0.0 yesterday I am now seeing repeatable crashes in the garbage collection. The end of the trace looks like: #0 rxml_attr_mark (xattr=0x0) at ruby_xml_attr.c:41 #1 0xb7ed6a15 in gc_mark_children (ptr=3050895040, lev=1) at gc.c:945 #2 0xb7ed6c49 in mark_locations_array (x=0xbfc32f90, n=39) at gc.c:629 #3 0xb7ed6e17 in garbage_collect () at gc.c:1366 #4 0xb7ed79c5 in ruby_xmalloc (size=48) at gc.c:103 #5 0xb71855fa in xmlNewPropInternal (node=0xa20a190, ns=0x0, name=0x84983b0 "k", value=0xa20a170 "created_by", eatname=0) at tree.c:1791 #6 0xb727ce48 in rxml_attr_initialize (argc=3, argv=0xbfc333a0, self=3050895040) at ruby_xml_attr.c:104 #7 0xb7eb8815 in call_cfunc (func=0xb727cd30 , recv=3050895040, len=34, argc=3, argv=0xb727d100) at eval.c:5691 #8 0xb7ec0f1d in rb_call0 (klass=3075364500, recv=3050895040, id=2961, oid=2961, argc=3, argv=0xbfc333a0, body=0xb74e561c, flags=0) at eval.c:5846 #9 0xb7ec11d8 in rb_call (klass=3075364500, recv=3050895040, mid=2961, argc=3, argv=0xbfc333a0, scope=1, self=6) at eval.c:6093 #10 0xb7ec14e7 in rb_obj_call_init (obj=3050895040, argc=3, argv=0xbfc333a0) at eval.c:7625 #11 0xb7eef43a in rb_class_new_instance (argc=3, argv=0xbfc333a0, klass=3075364500) at object.c:1594 #12 0xb726cc63 in rxml_attributes_attribute_set (self=3050895060, name=3050895080, value=3050895220) at ruby_xml_attributes.c:176 #13 0xb726f5e3 in rxml_node_property_set (self=3050895120, name=3050895080, value=3050895220) at ruby_xml_node.c:1106 As you can see it is being asked to mark an attribute, but the attribute pointer it is given is null. Any ideas? Tom -- Tom Hughes (tom at compton.nu) http://www.compton.nu/ From cfis at savagexi.com Mon Mar 9 12:29:05 2009 From: cfis at savagexi.com (Charlie Savage) Date: Mon, 09 Mar 2009 10:29:05 -0600 Subject: [libxml-devel] SEGV with libxml-ruby 1.0.0 In-Reply-To: <49B534C7.8030303@compton.nu> References: <49B534C7.8030303@compton.nu> Message-ID: <49B543D1.3030601@savagexi.com> Hi Tom, > Having upgraded to libxml-ruby 1.0.0 yesterday I am now seeing > repeatable crashes in the garbage collection. The end of the trace looks > like: > > #0 rxml_attr_mark (xattr=0x0) at ruby_xml_attr.c:41 > #1 0xb7ed6a15 in gc_mark_children (ptr=3050895040, lev=1) at gc.c:945 > #2 0xb7ed6c49 in mark_locations_array (x=0xbfc32f90, n=39) at gc.c:629 > #3 0xb7ed6e17 in garbage_collect () at gc.c:1366 > #4 0xb7ed79c5 in ruby_xmalloc (size=48) at gc.c:103 > #5 0xb71855fa in xmlNewPropInternal (node=0xa20a190, ns=0x0, > name=0x84983b0 "k", value=0xa20a170 "created_by", eatname=0) at tree.c:1791 > #6 0xb727ce48 in rxml_attr_initialize (argc=3, argv=0xbfc333a0, > self=3050895040) at ruby_xml_attr.c:104 > #7 0xb7eb8815 in call_cfunc (func=0xb727cd30 , > recv=3050895040, len=34, argc=3, argv=0xb727d100) at eval.c:5691 > #8 0xb7ec0f1d in rb_call0 (klass=3075364500, recv=3050895040, id=2961, > oid=2961, argc=3, argv=0xbfc333a0, body=0xb74e561c, flags=0) at eval.c:5846 > #9 0xb7ec11d8 in rb_call (klass=3075364500, recv=3050895040, mid=2961, > argc=3, argv=0xbfc333a0, scope=1, self=6) at eval.c:6093 > #10 0xb7ec14e7 in rb_obj_call_init (obj=3050895040, argc=3, > argv=0xbfc333a0) at eval.c:7625 > #11 0xb7eef43a in rb_class_new_instance (argc=3, argv=0xbfc333a0, > klass=3075364500) at object.c:1594 > #12 0xb726cc63 in rxml_attributes_attribute_set (self=3050895060, > name=3050895080, value=3050895220) at ruby_xml_attributes.c:176 > #13 0xb726f5e3 in rxml_node_property_set (self=3050895120, > name=3050895080, value=3050895220) at ruby_xml_node.c:1106 > > As you can see it is being asked to mark an attribute, but the attribute > pointer it is given is null. Yeah, this should be easy to track down. Is it 1.9.1 by chance? Do you have a test case? Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From tom at compton.nu Mon Mar 9 12:41:38 2009 From: tom at compton.nu (Tom Hughes) Date: Mon, 09 Mar 2009 16:41:38 +0000 Subject: [libxml-devel] SEGV with libxml-ruby 1.0.0 In-Reply-To: <49B534C7.8030303@compton.nu> References: <49B534C7.8030303@compton.nu> Message-ID: <49B546C2.3050702@compton.nu> Tom Hughes wrote: > Having upgraded to libxml-ruby 1.0.0 yesterday I am now seeing > repeatable crashes in the garbage collection. The end of the trace looks > like: > > #0 rxml_attr_mark (xattr=0x0) at ruby_xml_attr.c:41 > #1 0xb7ed6a15 in gc_mark_children (ptr=3050895040, lev=1) at gc.c:945 > #2 0xb7ed6c49 in mark_locations_array (x=0xbfc32f90, n=39) at gc.c:629 > #3 0xb7ed6e17 in garbage_collect () at gc.c:1366 > #4 0xb7ed79c5 in ruby_xmalloc (size=48) at gc.c:103 > #5 0xb71855fa in xmlNewPropInternal (node=0xa20a190, ns=0x0, > name=0x84983b0 "k", value=0xa20a170 "created_by", eatname=0) at tree.c:1791 One of my colleagues thinks he has spotted the problem: "the attribute is being allocated (rxml_attr_alloc), which sets the data pointer to NULL. almost immediately, the initialise method is called (rxml_attr_initialize) where the self object has that NULL data pointer. when it gets to xmlNewProp it triggers the GC which tries to mark the current object, which hasn't finished initialising yet... Making rxml_attr_mark() return immediately if xattr is null seems to have stopped it segving anyway - the mark routine for nodes is already doing that in fact. Tom -- Tom Hughes (tom at compton.nu) http://www.compton.nu/ From tom at compton.nu Mon Mar 9 13:21:29 2009 From: tom at compton.nu (Tom Hughes) Date: Mon, 09 Mar 2009 17:21:29 +0000 Subject: [libxml-devel] SEGV with libxml-ruby 1.0.0 In-Reply-To: <49B543D1.3030601@savagexi.com> References: <49B534C7.8030303@compton.nu> <49B543D1.3030601@savagexi.com> Message-ID: <49B55019.5030805@compton.nu> Charlie Savage wrote: >> As you can see it is being asked to mark an attribute, but the >> attribute pointer it is given is null. > > Yeah, this should be easy to track down. Is it 1.9.1 by chance? Do you > have a test case? No, it's 1.8.6 actually. I don't have a simple test case I'm afraid - it was happening on the openstreetmap code base against our live database... Tom -- Tom Hughes (tom at compton.nu) http://www.compton.nu/ From tom at compton.nu Mon Mar 9 13:52:09 2009 From: tom at compton.nu (Tom Hughes) Date: Mon, 09 Mar 2009 17:52:09 +0000 Subject: [libxml-devel] libxml-ruby 1.0.0 fails to install with libxml 2.6.27 Message-ID: <49B55749.8090400@compton.nu> There is some code in ext/libxml/ruby_xml_html_parser_context.c that defines a couple of functions for older versions of libxml that are included in newer versions. The comment says the functions were added to libxml in 2.6.27 but the code is included for all versions <= 2.6.27 so if you happen to have exactly 2.6.27 on your system then you get a double definition. Tom -- Tom Hughes (tom at compton.nu) http://www.compton.nu/ From cfis at savagexi.com Mon Mar 9 13:52:49 2009 From: cfis at savagexi.com (Charlie Savage) Date: Mon, 09 Mar 2009 11:52:49 -0600 Subject: [libxml-devel] SEGV with libxml-ruby 1.0.0 In-Reply-To: <49B546C2.3050702@compton.nu> References: <49B534C7.8030303@compton.nu> <49B546C2.3050702@compton.nu> Message-ID: <49B55771.6000503@savagexi.com> >> #0 rxml_attr_mark (xattr=0x0) at ruby_xml_attr.c:41 >> #1 0xb7ed6a15 in gc_mark_children (ptr=3050895040, lev=1) at gc.c:945 >> #2 0xb7ed6c49 in mark_locations_array (x=0xbfc32f90, n=39) at gc.c:629 >> #3 0xb7ed6e17 in garbage_collect () at gc.c:1366 >> #4 0xb7ed79c5 in ruby_xmalloc (size=48) at gc.c:103 >> #5 0xb71855fa in xmlNewPropInternal (node=0xa20a190, ns=0x0, >> name=0x84983b0 "k", value=0xa20a170 "created_by", eatname=0) at >> tree.c:1791 > > One of my colleagues thinks he has spotted the problem: > > "the attribute is being allocated (rxml_attr_alloc), which sets the data > pointer to NULL. almost immediately, the initialise method is called > (rxml_attr_initialize) where the self object has that NULL data > pointer. when it gets to xmlNewProp it triggers the GC which tries to > mark the current object, which hasn't finished initialising yet... This is an interesting one, and the diagnosis is correct. It is caused by having libxml use ruby's memory allocator. 1. Create a new ruby attribute object 2. Initialize it 3. Call libxml xmlNewNsProp 4. It asks ruby for memory 5. Ruby runs a gc since it has no memory 6. The newly created, but not initialized attribute has its mark function called The reason for that change was it greatly reduced libxml-ruby's memory usage. > Making rxml_attr_mark() return immediately if xattr is null seems to > have stopped it segving anyway - the mark routine for nodes is already > doing that in fact. Yes, that would work. But I wonder if there are other cases of something similar happening. I think you're seeing it because you probably have a lot higher load on your server then what we test with. I tried to duplicate the issue but didn't succeed: def test_high_allocations node = XML::Node.new('test') 1.upto(100000) do |i| name = "attr_#{i}" XML::Attr.new(node, name, i.to_s) end assert(true) end Anyway, I added the check and went back to using libxml's internal memory allocator. Fixes in 1.1.0 which I just uploaded. Let me know how it goes. Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From cfis at savagexi.com Mon Mar 9 13:54:49 2009 From: cfis at savagexi.com (Charlie Savage) Date: Mon, 09 Mar 2009 11:54:49 -0600 Subject: [libxml-devel] Announcing libxml-ruby 1.1.0 Message-ID: <49B557E9.80204@savagexi.com> Well, that didn't last quite as long as I had hoped. A new version of libxml-ruby, version 1.1.0, is now available. It includes two changes: * Fix bug caused by the mark function being called on partially initialized attributes. * Revert back to libxml2's internal memory manager. These changes hopefully fix the issue reported earlier today by Tom. Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From joe at collectivex.com Mon Mar 9 14:17:51 2009 From: joe at collectivex.com (Joe Khoobyar) Date: Mon, 09 Mar 2009 14:17:51 -0400 Subject: [libxml-devel] SEGV with libxml-ruby 1.0.0 In-Reply-To: <49B55771.6000503@savagexi.com> References: <49B534C7.8030303@compton.nu> <49B546C2.3050702@compton.nu> <49B55771.6000503@savagexi.com> Message-ID: <49B55D4F.3020504@collectivex.com> What a shame. I think we should investigate alternatives, but high memory usage is better than crashing. :'( I should point out that we also have high load but we haven't experienced any segfaulting - so it may be configuration dependent, but not necessarily.... I'll try and take a look at this when I get the chance to see if we can find a way to have our cake and eat it too. - Joe Khoobyar Charlie Savage wrote: >>> #0 rxml_attr_mark (xattr=0x0) at ruby_xml_attr.c:41 >>> #1 0xb7ed6a15 in gc_mark_children (ptr=3050895040, lev=1) at gc.c:945 >>> #2 0xb7ed6c49 in mark_locations_array (x=0xbfc32f90, n=39) at gc.c:629 >>> #3 0xb7ed6e17 in garbage_collect () at gc.c:1366 >>> #4 0xb7ed79c5 in ruby_xmalloc (size=48) at gc.c:103 >>> #5 0xb71855fa in xmlNewPropInternal (node=0xa20a190, ns=0x0, >>> name=0x84983b0 "k", value=0xa20a170 "created_by", eatname=0) at >>> tree.c:1791 >> >> One of my colleagues thinks he has spotted the problem: >> >> "the attribute is being allocated (rxml_attr_alloc), which sets the data >> pointer to NULL. almost immediately, the initialise method is called >> (rxml_attr_initialize) where the self object has that NULL data >> pointer. when it gets to xmlNewProp it triggers the GC which tries to >> mark the current object, which hasn't finished initialising yet... > > This is an interesting one, and the diagnosis is correct. It is > caused by having libxml use ruby's memory allocator. > > 1. Create a new ruby attribute object > 2. Initialize it > 3. Call libxml xmlNewNsProp > 4. It asks ruby for memory > 5. Ruby runs a gc since it has no memory > 6. The newly created, but not initialized attribute has its mark > function called > > The reason for that change was it greatly reduced libxml-ruby's memory > usage. > >> Making rxml_attr_mark() return immediately if xattr is null seems to >> have stopped it segving anyway - the mark routine for nodes is >> already doing that in fact. > > Yes, that would work. But I wonder if there are other cases of > something similar happening. > > I think you're seeing it because you probably have a lot higher load > on your server then what we test with. I tried to duplicate the issue > but didn't succeed: > > def test_high_allocations > node = XML::Node.new('test') > > 1.upto(100000) do |i| > name = "attr_#{i}" > XML::Attr.new(node, name, i.to_s) > end > assert(true) > end > > Anyway, I added the check and went back to using libxml's internal > memory allocator. > > Fixes in 1.1.0 which I just uploaded. Let me know how it goes. > > Charlie > ------------------------------------------------------------------------ > > _______________________________________________ > libxml-devel mailing list > libxml-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/libxml-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From cfis at savagexi.com Mon Mar 9 14:46:23 2009 From: cfis at savagexi.com (Charlie Savage) Date: Mon, 09 Mar 2009 12:46:23 -0600 Subject: [libxml-devel] SEGV with libxml-ruby 1.0.0 In-Reply-To: <49B55D4F.3020504@collectivex.com> References: <49B534C7.8030303@compton.nu> <49B546C2.3050702@compton.nu> <49B55771.6000503@savagexi.com> <49B55D4F.3020504@collectivex.com> Message-ID: <49B563FF.6050801@savagexi.com> > What a shame. I think we should investigate alternatives, but high > memory usage is better than crashing. :'( > My thinking on this is go back through all the mark and free functions and see which other ones theoretically could have the problem. Then having done that review, give it another try. > I should point out that we also have high load but we haven't > experienced any segfaulting - so it may be configuration dependent, but > not necessarily.... Probably so - I ran a test script to create 10k attributes to force the GC to happen, but couldn't reproduce it. But the stack trace is pretty clear on what's happening. Could be the Ruby version, or the OS, or the configuration. > I'll try and take a look at this when I get the chance to see if we can > find a way to have our cake and eat it too. Yeah, see above. I think is probably doable. Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From cfis at savagexi.com Mon Mar 9 15:03:23 2009 From: cfis at savagexi.com (Charlie Savage) Date: Mon, 09 Mar 2009 13:03:23 -0600 Subject: [libxml-devel] libxml-ruby 1.0.0 fails to install with libxml 2.6.27 In-Reply-To: <49B55749.8090400@compton.nu> References: <49B55749.8090400@compton.nu> Message-ID: <49B567FB.2000502@savagexi.com> > There is some code in ext/libxml/ruby_xml_html_parser_context.c that > defines a couple of functions for older versions of libxml that are > included in newer versions. Right, those were added for OS X compatibility, since it has version 2.6.16. They actually exist in older libxml versions, but are not exposed in the header files. Thus my ugly hack to just copy them over to ruby_xml_html_parser_context.c. > The comment says the functions were added to libxml in 2.6.27 but the > code is included for all versions <= 2.6.27 so if you happen to have > exactly 2.6.27 on your system then you get a double definition. Yup, should be < not <=. I take it you are using 2.6.27? Fix checked into trunk. Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From tom at compton.nu Mon Mar 9 16:16:44 2009 From: tom at compton.nu (Tom Hughes) Date: Mon, 09 Mar 2009 20:16:44 +0000 Subject: [libxml-devel] libxml-ruby 1.0.0 fails to install with libxml 2.6.27 In-Reply-To: <49B567FB.2000502@savagexi.com> References: <49B55749.8090400@compton.nu> <49B567FB.2000502@savagexi.com> Message-ID: <49B5792C.3090901@compton.nu> Charlie Savage wrote: >> The comment says the functions were added to libxml in 2.6.27 but the >> code is included for all versions <= 2.6.27 so if you happen to have >> exactly 2.6.27 on your system then you get a double definition. > > Yup, should be < not <=. I take it you are using 2.6.27? Well one of our machines was, yes. That seems to be the latest version in Etch. Tom -- Tom Hughes (tom at compton.nu) http://www.compton.nu/ From danj at 3skel.com Tue Mar 10 06:48:42 2009 From: danj at 3skel.com (Dan Janowski) Date: Tue, 10 Mar 2009 06:48:42 -0400 Subject: [libxml-devel] Announcing libxml-ruby 1.1.0 In-Reply-To: <49B557E9.80204@savagexi.com> References: <49B557E9.80204@savagexi.com> Message-ID: <02A81774-2918-4164-A81B-760054D21511@3skel.com> Maybe I missed something, but doc.encoding takes/returns a number instead of an encoding string. not sure how that change from 0.8.x happens. On Mar 9, 2009, at 13:54, Charlie Savage wrote: > Well, that didn't last quite as long as I had hoped. > > A new version of libxml-ruby, version 1.1.0, is now available. It > includes two changes: > > * Fix bug caused by the mark function being called on partially > initialized attributes. > > * Revert back to libxml2's internal memory manager. > > These changes hopefully fix the issue reported earlier today by Tom. > > Charlie > _______________________________________________ > libxml-devel mailing list > libxml-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/libxml-devel From dsisnero at gmail.com Tue Mar 10 12:10:33 2009 From: dsisnero at gmail.com (Dominic Sisneros) Date: Tue, 10 Mar 2009 10:10:33 -0600 Subject: [libxml-devel] subclassing reader Message-ID: I am trying to build a library of pull parser based mixins for LibXML::XML::Reader and rexml/pull_parser but I cannot subclass LibXML::XML::Reader and get this mixins included. How can I do this without reopening LibXML::XML::Reader? for instance: this doesn't work require 'libxml' module Helpers def testing 'this should work' end end class SimpleReader < LibXML::XML::Reader include Helpers end if $0 == __FILE__ reader = SimpleReader.string('> C:\Documents and Settings\dominic e sisneros\My Documents\programming\ruby\simple_reader\spec>ruby include_test.rb include_test.rb:21: undefined method `testing' for # (NoMethodError) but if I do module LibXML module XML class Reader -------------- next part -------------- An HTML attachment was scrubbed... URL: From dsisnero at gmail.com Tue Mar 10 12:12:58 2009 From: dsisnero at gmail.com (Dominic Sisneros) Date: Tue, 10 Mar 2009 10:12:58 -0600 Subject: [libxml-devel] Fwd: subclassing reader, sent it too early Message-ID: I am trying to build a library of pull parser based mixins for LibXML::XML::Reader and rexml/pull_parser but I cannot subclass LibXML::XML::Reader and get this mixins included. How can I do this without reopening LibXML::XML::Reader? for instance: this doesn't work require 'libxml' module Helpers def testing 'this should work' end end class SimpleReader < LibXML::XML::Reader include Helpers end if $0 == __FILE__ reader = SimpleReader.string('> C:\Documents and Settings\dominic e sisneros\My Documents\programming\ruby\simple_reader\spec>ruby include_test.rb include_test.rb:21: undefined method `testing' for # (NoMethodError) but if I do module LibXML module XML class Reader include Helpers end end end it works -------------- next part -------------- An HTML attachment was scrubbed... URL: From cfis at savagexi.com Tue Mar 10 12:52:59 2009 From: cfis at savagexi.com (Charlie Savage) Date: Tue, 10 Mar 2009 10:52:59 -0600 Subject: [libxml-devel] Announcing libxml-ruby 1.1.0 In-Reply-To: <02A81774-2918-4164-A81B-760054D21511@3skel.com> References: <49B557E9.80204@savagexi.com> <02A81774-2918-4164-A81B-760054D21511@3skel.com> Message-ID: <49B69AEB.4080707@savagexi.com> > Maybe I missed something, but doc.encoding takes/returns a number > instead of an encoding string. not sure how that change from 0.8.x happens. They were all changed a few releases back to the libxml encoding constants defined here: http://xmlsoft.org/html/libxml-encoding.html Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From cfis at savagexi.com Tue Mar 10 14:43:37 2009 From: cfis at savagexi.com (Charlie Savage) Date: Tue, 10 Mar 2009 12:43:37 -0600 Subject: [libxml-devel] Announcing libxml-ruby 1.1.1 Message-ID: <49B6B4D9.2010606@savagexi.com> A new version of libxml-ruby, version 1.1.1, is now available. It includes one minor change: * Fix - Only include extra html parser context methods for versions of libxml2 older than 2.6.27. Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From cfis at savagexi.com Tue Mar 10 19:48:38 2009 From: cfis at savagexi.com (Charlie Savage) Date: Tue, 10 Mar 2009 17:48:38 -0600 Subject: [libxml-devel] Mark libxml-ruby as 1.9.1 compatible Message-ID: <49B6FC56.40805@savagexi.com> Hi everyone, The site http://isitruby19.com/ lists which gems are 1.9.1 compatible. Now that libxml-ruby is 1.9.1 compatible, can anyone who has experience with 1.9.1 go vote yes (and if you find any bugs then please submit them to ruby forge). Thanks, Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From cfis at savagexi.com Wed Mar 11 23:11:22 2009 From: cfis at savagexi.com (Charlie Savage) Date: Wed, 11 Mar 2009 21:11:22 -0600 Subject: [libxml-devel] libxml-ruby 1.x Released: Ruby Now A Serious XML Player Message-ID: <49B87D5A.9090907@savagexi.com> http://www.rubyinside.com/libxml-ruby-1x-released-ruby-now-a-serious-xml-player-1630.html Charlie Savage -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From joe at collectivex.com Wed Mar 11 23:25:52 2009 From: joe at collectivex.com (Joe Khoobyar) Date: Wed, 11 Mar 2009 23:25:52 -0400 Subject: [libxml-devel] libxml-ruby 1.x Released: Ruby Now A Serious XML Player In-Reply-To: <49B87D5A.9090907@savagexi.com> References: <49B87D5A.9090907@savagexi.com> Message-ID: <49B880C0.8020509@collectivex.com> Yee-Haw! Getting some press! *Joe Khoobyar * Chief Technical Officer & Lead Developer CollectiveX - /Groups that work!/ mobile: 585.245.2902 email: joe at collectivex.com web: www.collectivex.com www.groupsites.com The third-rate mind is only happy when it is thinking with the majority. The second-rate mind is only happy when it is thinking with the minority. The first-rate mind is only happy when it is thinking. ? A.A. Milne Charlie Savage wrote: > http://www.rubyinside.com/libxml-ruby-1x-released-ruby-now-a-serious-xml-player-1630.html > > > Charlie Savage > ------------------------------------------------------------------------ > > _______________________________________________ > libxml-devel mailing list > libxml-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/libxml-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From cfis at savagexi.com Sat Mar 14 17:28:47 2009 From: cfis at savagexi.com (Charlie Savage) Date: Sat, 14 Mar 2009 15:28:47 -0600 Subject: [libxml-devel] Announcing libxml-ruby 1.1.1 Message-ID: <49BC218F.50404@savagexi.com> Changes: * Added XML::Node#inner_xml helper method. * Fixed segmentation that could occur when calling the mark function on a previously freed node Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From luc at honk-honk.com Sat Mar 14 18:00:23 2009 From: luc at honk-honk.com (Luc Heinrich) Date: Sat, 14 Mar 2009 23:00:23 +0100 Subject: [libxml-devel] Announcing libxml-ruby 1.1.1 In-Reply-To: <49BC218F.50404@savagexi.com> References: <49BC218F.50404@savagexi.com> Message-ID: <5E644432-DE53-4EEC-9B24-ECED5FD2D673@honk-honk.com> On 14 mars 09, at 22:28, Charlie Savage wrote: > Changes: I think you meant 1.1.2 ;) -- Luc Heinrich - luc at honk-honk.com From cfis at savagexi.com Sat Mar 14 18:27:14 2009 From: cfis at savagexi.com (Charlie Savage) Date: Sat, 14 Mar 2009 16:27:14 -0600 Subject: [libxml-devel] Announcing libxml-ruby 1.1.1 In-Reply-To: <5E644432-DE53-4EEC-9B24-ECED5FD2D673@honk-honk.com> References: <49BC218F.50404@savagexi.com> <5E644432-DE53-4EEC-9B24-ECED5FD2D673@honk-honk.com> Message-ID: <49BC2F42.8030405@savagexi.com> > >> Changes: > > I think you meant 1.1.2 ;) Right. Oops. Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From jimmy at deefa.com Sun Mar 15 07:38:58 2009 From: jimmy at deefa.com (James Healy) Date: Sun, 15 Mar 2009 22:38:58 +1100 Subject: [libxml-devel] Reader and entity not defined Message-ID: <20090315113858.GA17807@deefa.com> Hi, I need to read XML files from our suppliers, who generally use the ONIX spec when generating their files. These files contain book metadata, and from a high level have a fairly simple make up: 1 header section and 1 or more product sections. My approach so far as been to use Reader to avoid loading the entire files into memory, and for the most part this works fine. The only place it falls down is when a file contains an entity that isn't & < > or a numeric one. Here's a sample file that uses the – entity: http://gist.github.com/79386 And here's a contrived example that uses Reader to extract the Header and Product records: http://gist.github.com/79387. If you run this, it outputs the following nonfatal error and doesn't return the full text of the Product node: ~/git/onix.git master$ ruby examples/entities.rb
HarperCollins Publishers Australian Booksellers Association 20081106
Error: Entity 'ndash' not defined at examples/../data/entities.xml:28. -- I have 2 questions: - The ONIX DTD has a definition for a range of entities, including – Can I get libxml/Reader to recognise them? - Failing that, can I get reader to just return entities unmodified instead of exiting with an error? I've tried passing various options to the Reader constructor (like XML::Parser::Options::NOENT) to no avail. Cheers -- James Healy Sun, 15 Mar 2009 22:18:44 +1100 From stephen.bannasch at deanbrook.org Mon Mar 16 16:11:59 2009 From: stephen.bannasch at deanbrook.org (Stephen Bannasch) Date: Mon, 16 Mar 2009 16:11:59 -0400 Subject: [libxml-devel] fyi: bug in libxml backend for Rails new xml_mini backend Message-ID: FYI: I've got a bug in a Rails lighthouse ticket http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/2258-libxml-xml_mini-backend-producing-incorrect-hashes Showing a case where the libxml backend produces an incorrect hash. Would be great to get this fixed before the next release of Rails (and day or hour ;-) -- specifically because I've got a benchmark showing using libxml as the backend for Hash#.from_xml is 25x faster than REXML. See: http://github.com/stepheneb/rails_hash_from_xml/tree/master From cfis at savagexi.com Mon Mar 16 17:11:58 2009 From: cfis at savagexi.com (Charlie Savage) Date: Mon, 16 Mar 2009 15:11:58 -0600 Subject: [libxml-devel] fyi: bug in libxml backend for Rails new xml_mini backend In-Reply-To: References: Message-ID: <49BEC09E.4060604@savagexi.com> Stephen, And where is the Hash#to_xml method defined? Charlie Stephen Bannasch wrote: > FYI: > > I've got a bug in a Rails lighthouse ticket > > http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/2258-libxml-xml_mini-backend-producing-incorrect-hashes > > > Showing a case where the libxml backend produces an incorrect hash. > > Would be great to get this fixed before the next release of Rails (and > day or hour ;-) -- specifically because I've got a benchmark showing > using libxml as the backend for Hash#.from_xml is 25x faster than REXML. > > See: http://github.com/stepheneb/rails_hash_from_xml/tree/master > _______________________________________________ > libxml-devel mailing list > libxml-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/libxml-devel -- Charlie Savage http://cfis.savagexi.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From stephen.bannasch at deanbrook.org Mon Mar 16 18:31:52 2009 From: stephen.bannasch at deanbrook.org (Stephen Bannasch) Date: Mon, 16 Mar 2009 18:31:52 -0400 Subject: [libxml-devel] fyi: bug in libxml backend for Rails new xml_mini backend In-Reply-To: <49BEC09E.4060604@savagexi.com> References: <49BEC09E.4060604@savagexi.com> Message-ID: At 3:11 PM -0600 3/16/09, Charlie Savage wrote: >Stephen, > >And where is the Hash#to_xml method defined? It's part of what an activesupport xml_mini backend does. Take a look in the rails source code here: activesupport/lib/active_support/xml_mini Which you can see here: http://github.com/rails/rails/tree/18eb80ccc7e932f9a6c00462ceaeea648631b120/activesupport/lib/active_support/xml_mini line 40 in libxml.rb is the start of the Hash.from_xml method implementation for the libxml backend. Here's the test I'm working on: http://github.com/stepheneb/rails_hash_from_xml/blob/823a8b6b4c2bad04a27e9131082d816e8c19990e/test/test_rexml_equality.rb libxml is failing two test right now: [rails_hash_from_xml (master)]$ ruby -I$RAILS_SOURCE/activesupport/lib test/test_rexml_equality.rb Testing xml_mini backends: Nokogiri LibXML Loaded suite test/test_rexml_equality Started ....F.............F. Finished in 0.165816 seconds. 1) Failure: test_children_with_non_adjacent_comparing_LibXML_to_rexml(TestRexmlEquality) [test/test_rexml_equality.rb:151]: <{"root"=> {"__content__"=>"\n good\n \n morning\n ", "products"=>{"__content__"=>"\n hello everyone\n "}}}> expected but was <{"root"=> {"__content__"=>"\n morning\n ", "products"=>{"__content__"=>"\n hello everyone\n "}}}>. 2) Failure: test_two_lists_in_a_collection_comparing_LibXML_to_rexml(TestRexmlEquality) [test/test_rexml_equality.rb:151]: <{"collection"=> {"list"=> {"definition"=> [{"word"=>"Evaporation", "definition"=> "Evaporation is the process of changing from a liquid to a gas, for example, water changing from a liquid into water vapor."}, {"word"=>"Condensation", "definition"=> "Condensation is the process of changing from a gas to a liquid, for example, water changing from water vapor into a liquid."}], "object"=>[{"id"=>"123"}, {"id"=>"456"}]}}}> expected but was <{"collection"=> {"list"=> {"object"=> [{"id"=>"123"}, {"id"=>"456"}, {"word"=>"Evaporation", "definition"=> "Evaporation is the process of changing from a liquid to a gas, for example, water changing from a liquid into water vapor."}, {"word"=>"Condensation", "definition"=> "Condensation is the process of changing from a gas to a liquid, for example, water changing from water vapor into a liquid."}]}}}>. From cfis at savagexi.com Mon Mar 16 19:39:55 2009 From: cfis at savagexi.com (Charlie Savage) Date: Mon, 16 Mar 2009 17:39:55 -0600 Subject: [libxml-devel] fyi: bug in libxml backend for Rails new xml_mini backend In-Reply-To: References: <49BEC09E.4060604@savagexi.com> Message-ID: <49BEE34B.9050604@savagexi.com> On the first test, its a bug in the activesupport code: def to_hash(hash={}) if text? raise LibXML::XML::Error if content.length >= LIB_XML_LIMIT hash[CONTENT_ROOT] = content # !!!!! INCORRECT !!!!! else This is the xml: good hello everyone morning Look at the sequence of nodes as they are processed: root text (child) element (products) text (hello everyone) text (morning) When the text node (morning) is processed: hash[CONTENT_ROOT] = content Thus overwriting the text "good" with "morning" You want something more like this: hash[CONTENT_ROOT] = (hash[CONTENT_ROOT] || '') + content Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From cfis at savagexi.com Mon Mar 16 19:52:35 2009 From: cfis at savagexi.com (Charlie Savage) Date: Mon, 16 Mar 2009 17:52:35 -0600 Subject: [libxml-devel] fyi: bug in libxml backend for Rails new xml_mini backend In-Reply-To: References: <49BEC09E.4060604@savagexi.com> Message-ID: <49BEE643.1080602@savagexi.com> And on the second test case: The error is here: def children_array_to_hash(hash={}) hash[child.name] = map do |child| returning({}) { |sub_hash| child.children_to_hash(sub_hash) } end hash end There is no check to see if the element name has changed from object to definition.... Charlie -- Charlie Savage http://cfis.savagexi.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From stephen.bannasch at deanbrook.org Mon Mar 16 23:50:31 2009 From: stephen.bannasch at deanbrook.org (Stephen Bannasch) Date: Mon, 16 Mar 2009 23:50:31 -0400 Subject: [libxml-devel] bus error cause by bad XML Message-ID: In working on the rails xml_mini backend tests I found a test generating a bus error when using libxml-ruby-1.1.2. Here's a stand-alone adaptation of the test that runs on the just released Rails 2.3.2: file: test_attack_xml.rb require 'rubygems' require 'test/unit' require 'activesupport' require 'active_support/xml_mini' class TestAttackXml < Test::Unit::TestCase include ActiveSupport def test_exception_thrown_on_expansion_attack assert_raise SyntaxError do attack_xml = <<-EOT ]> &a; EOT hash = XmlMini.with_backend("LibXML") { Hash.from_xml(attack_xml) } end end end Here's the result running on MacOS 10.5.6 $ ruby test_attack_xml_with_libxml.rb Loaded suite test_attack_xml Started /Library/Ruby/Gems/1.8/gems/libxml-ruby-1.1.2/lib/libxml/node.rb:100: [BUG] Bus Error ruby 1.8.6 (2008-03-03) [universal-darwin9.0] Abort trap From aaron.patterson at gmail.com Mon Mar 16 23:59:34 2009 From: aaron.patterson at gmail.com (Aaron Patterson) Date: Mon, 16 Mar 2009 20:59:34 -0700 Subject: [libxml-devel] bus error cause by bad XML In-Reply-To: References: Message-ID: <6959e1680903162059l43dffbfere1ceec34cf521fd0@mail.gmail.com> On Mon, Mar 16, 2009 at 8:50 PM, Stephen Bannasch wrote: > In working on the rails xml_mini backend tests I found a test generating a > bus error when using libxml-ruby-1.1.2. > > Here's a stand-alone adaptation of the test that runs on the just released > Rails 2.3.2: > > file: ?test_attack_xml.rb > > require 'rubygems' > require 'test/unit' > require 'activesupport' > require 'active_support/xml_mini' > > class TestAttackXml < Test::Unit::TestCase > ?include ActiveSupport > > ?def test_exception_thrown_on_expansion_attack > ? ?assert_raise SyntaxError do > ? ? ?attack_xml = <<-EOT > ? ? ? > ? ? ? ? ? ? ? > ? ? ? ? > ? ? ? ? > ? ? ? ? > ? ? ? ? > ? ? ? ? > ? ? ? ? > ? ? ?]> > ? ? ? > ? ? ?&a; > ? ? ? > ? ? ?EOT > ? ? ?hash = XmlMini.with_backend("LibXML") { Hash.from_xml(attack_xml) } > ? ?end > ?end > end > > Here's the result running on MacOS 10.5.6 > > $ ruby test_attack_xml_with_libxml.rb > Loaded suite test_attack_xml > Started > /Library/Ruby/Gems/1.8/gems/libxml-ruby-1.1.2/lib/libxml/node.rb:100: [BUG] > Bus Error > ruby 1.8.6 (2008-03-03) [universal-darwin9.0] > > Abort trap This works fine for me. What version of libxml2 are you running? Use this command to find out: ruby -rubygems -e'require "libxml"; puts LibXML::XML::LIBXML_VERSION' Also, it would be helpful if you could reproduce the problem outside of rails. -- Aaron Patterson http://tenderlovemaking.com/ From cfis at savagexi.com Tue Mar 17 00:23:13 2009 From: cfis at savagexi.com (Charlie Savage) Date: Mon, 16 Mar 2009 22:23:13 -0600 Subject: [libxml-devel] bus error cause by bad XML In-Reply-To: References: Message-ID: <49BF25B1.60105@savagexi.com> Hi Stephen, > Here's the result running on MacOS 10.5.6 > > $ ruby test_attack_xml_with_libxml.rb > Loaded suite test_attack_xml > Started > /Library/Ruby/Gems/1.8/gems/libxml-ruby-1.1.2/lib/libxml/node.rb:100: > [BUG] Bus Error > ruby 1.8.6 (2008-03-03) [universal-darwin9.0] > > Abort trap Can you get a stack trace? Testing on both Windows and Fedora 10 shows (not within Rails, I extracted the test case out, see below): Fatal error: Detected an entity reference loop at :1. Fatal error: Detected an entity reference loop at :1. Fatal error: Detected an entity reference loop at :1. Fatal error: Detected an entity reference loop at :1. Fatal error: Detected an entity reference loop at :1. Fatal error: Detected an entity reference loop at :12. C:/Development/src/libxml-ruby/test/new_main.rb:15:in `parse': Fatal error: Detected an entity reference loop at :12. (LibXML::XML::Error) from C:/Development/src/libxml-ruby/test/new_main.rb:15:in `from_xml' from C:/Development/src/libxml-ruby/test/new_main.rb:210 I assume that is what you are after? Charlie ------------------- require 'pp' require 'libxml' class Object def returning(value) yield(value) value end end class Hash def self.from_xml(xml) LibXML::XML.default_keep_blanks = false doc = LibXML::XML::Parser.string(xml.strip).parse typecast_xml_value(unrename_keys(doc)) end def typecast_xml_value(value) case value.class.to_s when 'Hash' if value['type'] == 'array' child_key, entries = value.detect { |k,v| k != 'type' } # child_key is throwaway if entries.nil? || (c = value['__content__'] && c.blank?) [] else case entries.class.to_s # something weird with classes not matching here. maybe singleton methods breaking is_a? when "Array" entries.collect { |v| typecast_xml_value(v) } when "Hash" [typecast_xml_value(entries)] else raise "can't typecast #{entries.inspect}" end end elsif value.has_key?("__content__") content = value["__content__"] if parser = XML_PARSING[value["type"]] if parser.arity == 2 XML_PARSING[value["type"]].call(content, value) else XML_PARSING[value["type"]].call(content) end else content end elsif value['type'] == 'string' && value['nil'] != 'true' "" # blank or nil parsed values are represented by nil elsif value.blank? || value['nil'] == 'true' nil # If the type is the only element which makes it then # this still makes the value nil, except if type is # a XML node(where type['value'] is a Hash) elsif value['type'] && value.size == 1 && !value['type'].is_a?(::Hash) nil else xml_value = value.inject({}) do |h,(k,v)| h[k] = typecast_xml_value(v) h end # Turn { :files => { :file => # } into { :files => # } so it is compatible with # how multipart uploaded files from HTML appear xml_value["file"].is_a?(StringIO) ? xml_value["file"] : xml_value end when 'Array' value.map! { |i| typecast_xml_value(i) } case value.length when 0 then nil when 1 then value.first else value end when 'String' value else raise "can't typecast #{value.class.name} - #{value.inspect}" end end end module LibXML module Conversions module Document def to_hash root.to_hash end end module Node CONTENT_ROOT = '__content__' LIB_XML_LIMIT = 30000000 # Hardcoded LibXML limit # Convert XML document to hash # # hash:: # Hash to merge the converted element into. def to_hash(hash={}) if text? raise LibXML::XML::Error if content.length >= LIB_XML_LIMIT hash[CONTENT_ROOT] = (hash[CONTENT_ROOT] || '') + content else sub_hash = insert_name_into_hash(hash, name) attributes_to_hash(sub_hash) if array? children_array_to_hash(sub_hash) elsif yaml? children_yaml_to_hash(sub_hash) else children_to_hash(sub_hash) end end hash end protected # Insert name into hash # # hash:: # Hash to merge the converted element into. # name:: # name to to merge into hash def insert_name_into_hash(hash, name) sub_hash = {} if hash[name] if !hash[name].kind_of? Array hash[name] = [hash[name]] end hash[name] << sub_hash else hash[name] = sub_hash end sub_hash end # Insert children into hash # # hash:: # Hash to merge the children into. def children_to_hash(hash={}) each { |child| child.to_hash(hash) } attributes_to_hash(hash) hash end # Convert xml attributes to hash # # hash:: # Hash to merge the attributes into def attributes_to_hash(hash={}) each_attr { |attr| hash[attr.name] = attr.value } hash end # Convert array into hash # # hash:: # Hash to merge the array into def children_array_to_hash(hash={}) hash[child.name] = map do |child| returning({}) { |sub_hash| child.children_to_hash(sub_hash) } end hash end # Convert yaml into hash # # hash:: # Hash to merge the yaml into def children_yaml_to_hash(hash = {}) hash[CONTENT_ROOT] = content unless content.blank? hash end # Check if child is of type array def array? child? && child.next? && child.name == child.next.name end # Check if child is of type yaml def yaml? attributes.collect{|x| x.value}.include?('yaml') end end end end LibXML::XML::Document.send(:include, LibXML::Conversions::Document) LibXML::XML::Node.send(:include, LibXML::Conversions::Node) attack_xml = <<-EOT ]> &a; EOT hash = Hash.from_xml(attack_xml) -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From stephen.bannasch at deanbrook.org Tue Mar 17 10:53:12 2009 From: stephen.bannasch at deanbrook.org (Stephen Bannasch) Date: Tue, 17 Mar 2009 10:53:12 -0400 Subject: [libxml-devel] bus error cause by bad XML In-Reply-To: <6959e1680903162059l43dffbfere1ceec34cf521fd0@mail.gmail.com> References: <6959e1680903162059l43dffbfere1ceec34cf521fd0@mail.gmail.com> Message-ID: >On Mon, Mar 16, 2009 at 8:50 PM, Stephen Bannasch > wrote: >> In working on the rails xml_mini backend tests I found a test generating a > > bus error when using libxml-ruby-1.1.2. > >This works fine for me. What version of libxml2 are you running? Use >this command to find out: > > ruby -rubygems -e'require "libxml"; puts LibXML::XML::LIBXML_VERSION' $ ruby -rubygems -e'require "libxml"; puts LibXML::XML::LIBXML_VERSION' 2.6.16 >Also, it would be helpful if you could reproduce the problem outside of rails. libxml dies on line 66 in this test outside of rails: http://gist.github.com/80576 I simplified what Charlie did. From stephen.bannasch at deanbrook.org Tue Mar 17 11:02:11 2009 From: stephen.bannasch at deanbrook.org (Stephen Bannasch) Date: Tue, 17 Mar 2009 11:02:11 -0400 Subject: [libxml-devel] bus error cause by bad XML In-Reply-To: <49BF25B1.60105@savagexi.com> References: <49BF25B1.60105@savagexi.com> Message-ID: At 10:23 PM -0600 3/16/09, Charlie Savage wrote: >Hi Stephen, > >>Here's the result running on MacOS 10.5.6 >> >>$ ruby test_attack_xml_with_libxml.rb >>Loaded suite test_attack_xml >>Started >>/Library/Ruby/Gems/1.8/gems/libxml-ruby-1.1.2/lib/libxml/node.rb:100: >>[BUG] Bus Error >>ruby 1.8.6 (2008-03-03) [universal-darwin9.0] >> >>Abort trap > >Can you get a stack trace? simplified libxml_attack.rb = http://gist.github.com/80576 What can I do to get a stack trace? This is what I get now: $ ruby ~/Desktop/libxml_attack.rb /Library/Ruby/Gems/1.8/gems/libxml-ruby-1.1.2/lib/libxml/node.rb:100: [BUG] Bus Error ruby 1.8.6 (2008-03-03) [universal-darwin9.0] Abort trap From cfis at savagexi.com Tue Mar 17 14:06:24 2009 From: cfis at savagexi.com (Charlie Savage) Date: Tue, 17 Mar 2009 12:06:24 -0600 Subject: [libxml-devel] bus error cause by bad XML In-Reply-To: References: <49BF25B1.60105@savagexi.com> Message-ID: <49BFE6A0.9080607@savagexi.com> > > What can I do to get a stack trace? I'll let people more knowledgeable than I am on OS X anser. But at a minimum, compile the bindings with debug info (-g). And then maybe use this: http://developer.apple.com/DOCUMENTATION/Darwin/Reference/ManPages/man8/stackshot.8.html The other way is to use gdb to load ruby, and then run the script. Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From flaz at UFL.EDU Tue Mar 17 13:49:53 2009 From: flaz at UFL.EDU (Francesco Lazzarino) Date: Tue, 17 Mar 2009 13:49:53 -0400 Subject: [libxml-devel] validate against multiple xml schema Message-ID: <9FF60C3E-3999-4623-8B01-EE3075418634@ufl.edu> validation question: given a schema that has an xsd:any element. Is there a way to specify multiple schemas to validate against the outer document and any inner documents? currently I only see the validate_schema method thanks in advance, -franco Francesco Lazzarino Florida Digital Archive flaz at ufl.edu From cfis at savagexi.com Tue Mar 17 16:14:39 2009 From: cfis at savagexi.com (Charlie Savage) Date: Tue, 17 Mar 2009 14:14:39 -0600 Subject: [libxml-devel] validate against multiple xml schema In-Reply-To: <9FF60C3E-3999-4623-8B01-EE3075418634@ufl.edu> References: <9FF60C3E-3999-4623-8B01-EE3075418634@ufl.edu> Message-ID: <49C004AF.8090906@savagexi.com> Hi Franco, > validation question: > > given a schema that has an xsd:any element. Is there a way to specify > multiple schemas to validate against the outer document and any inner > documents? > > currently I only see the validate_schema method Good question. Do you know if libxml2 supports this? Maybe ask on the libxml2 mailing list? If so, then the bindings could also. Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From cfis at savagexi.com Tue Mar 17 17:01:43 2009 From: cfis at savagexi.com (Charlie Savage) Date: Tue, 17 Mar 2009 15:01:43 -0600 Subject: [libxml-devel] bus error cause by bad XML In-Reply-To: References: <6959e1680903162059l43dffbfere1ceec34cf521fd0@mail.gmail.com> Message-ID: <49C00FB7.6080508@savagexi.com> > libxml dies on line 66 in this test outside of rails: > > http://gist.github.com/80576 Yup, works here. Line 66 is a weird place for it to die: attributes_to_hash(hash) That's just a ruby method call. From your report yesterday, seems more like its at line 75: each_attr { |attr| hash[attr.name] = attr.value } With the access to attributes in C code. Since I can't reproduce on Linux or Windows, I'll have to go dig up an OS X box. Let me know if you have any luck generating a stack trace, that hopefully will clear up the mystery. Charlie -- Charlie Savage http://cfis.savagexi.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From joe at collectivex.com Tue Mar 17 17:18:35 2009 From: joe at collectivex.com (Joe Khoobyar) Date: Tue, 17 Mar 2009 17:18:35 -0400 Subject: [libxml-devel] bus error cause by bad XML In-Reply-To: <49C00FB7.6080508@savagexi.com> References: <6959e1680903162059l43dffbfere1ceec34cf521fd0@mail.gmail.com> <49C00FB7.6080508@savagexi.com> Message-ID: <49C013AB.1000106@collectivex.com> I've got an OS X box, I can try it out if you want. *Joe Khoobyar * Chief Technical Officer & Lead Developer CollectiveX - /Groups that work!/ mobile: 585.245.2902 email: joe at collectivex.com web: www.collectivex.com www.groupsites.com The third-rate mind is only happy when it is thinking with the majority. The second-rate mind is only happy when it is thinking with the minority. The first-rate mind is only happy when it is thinking. ? A.A. Milne Charlie Savage wrote: >> libxml dies on line 66 in this test outside of rails: >> >> http://gist.github.com/80576 > > Yup, works here. > > Line 66 is a weird place for it to die: > > attributes_to_hash(hash) > > That's just a ruby method call. From your report yesterday, seems > more like its at line 75: > > each_attr { |attr| hash[attr.name] = attr.value } > > With the access to attributes in C code. > > Since I can't reproduce on Linux or Windows, I'll have to go dig up an > OS X box. Let me know if you have any luck generating a stack trace, > that hopefully will clear up the mystery. > > Charlie > > ------------------------------------------------------------------------ > > _______________________________________________ > libxml-devel mailing list > libxml-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/libxml-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen.bannasch at deanbrook.org Tue Mar 17 22:38:02 2009 From: stephen.bannasch at deanbrook.org (Stephen Bannasch) Date: Tue, 17 Mar 2009 22:38:02 -0400 Subject: [libxml-devel] bus error cause by bad XML In-Reply-To: <49C013AB.1000106@collectivex.com> References: <6959e1680903162059l43dffbfere1ceec34cf521fd0@mail.gmail.com> <49C00FB7.6080508@savagexi.com> <49C013AB.1000106@collectivex.com> Message-ID: At 5:18 PM -0400 3/17/09, Joe Khoobyar wrote: >I've got an OS X box, I can try it out if you want. Joe, That would be great -- I haven't had time to get back to the problem. From stephen.bannasch at deanbrook.org Wed Mar 18 09:14:13 2009 From: stephen.bannasch at deanbrook.org (Stephen Bannasch) Date: Wed, 18 Mar 2009 09:14:13 -0400 Subject: [libxml-devel] bus error cause by bad XML In-Reply-To: <49BFE6A0.9080607@savagexi.com> References: <49BF25B1.60105@savagexi.com> <49BFE6A0.9080607@savagexi.com> Message-ID: At 12:06 PM -0600 3/17/09, Charlie Savage wrote: >>What can I do to get a stack trace? > >I'll let people more knowledgeable than I am on OS X anser. But at >a minimum, compile the bindings with debug info (-g). And then >maybe use this: Would I do that with an invocation like this somehow: sudo gem install libxml-ruby -- --with-compile-flags=-g I also have the libxml-ruby gem code checked out -- should I modify the rake task that builds the extensions: ext = Config::CONFIG["DLEXT"] task :extensions => ["ext/libxml/libxml_ruby.#{ext}"] file "ext/libxml/libxml_ruby.#{ext}" => ["ext/libxml/Makefile"] + FileList["ext/libxml/*.{c,h}"].to_a do |t| Dir.chdir("ext/libxml") { sh "make" } end >http://developer.apple.com/DOCUMENTATION/Darwin/Reference/ManPages/man8/stackshot.8.html That's interesting -- I'm going to see how it works. >The other way is to use gdb to load ruby, and then run the script. I've seen instructions elsewhere for doing that. From Hans.Marmolin at klockholm.se Wed Mar 18 12:39:53 2009 From: Hans.Marmolin at klockholm.se (Hans Marmolin) Date: Wed, 18 Mar 2009 17:39:53 +0100 Subject: [libxml-devel] Installation problem Message-ID: <82C1F265-3CDC-478D-AB1D-53CAB7DE309D@klockholm.se> have some installation problem trying to use libxml-ruby omn my Mac Leopard has installed the gem with sudo gem install -r libxml-ruby got some strange error messages 1 gem installed Installing ri documentation for libxml-ruby-1.1.2... Enclosing class/module 'mLibXML' for module XML not known ............. etc A large set of these kind of messages, but I assumed these concerned the ri documention Wrote a small test application and got the error messages Unimplemented block at /SourceCache/libxml2/libxml2-17.7.6/libxml2/ xmlreader.c:1578 trying to read inner xml and outer xml Anyone that had have the same problem ? How to fix it ? I have not made any thing about the gem dependencies (and C bindings ?) as I assumed that this already was done on my mac Any help would be appreciated ____________________________________________ Hans Marmolin Prof Cognitive Ergonomics St: Larsgatan 50, 58224 Link?ping, Sweden Phone: +46708371202 From cfis at savagexi.com Wed Mar 18 22:16:49 2009 From: cfis at savagexi.com (Charlie Savage) Date: Wed, 18 Mar 2009 20:16:49 -0600 Subject: [libxml-devel] Installation problem In-Reply-To: <82C1F265-3CDC-478D-AB1D-53CAB7DE309D@klockholm.se> References: <82C1F265-3CDC-478D-AB1D-53CAB7DE309D@klockholm.se> Message-ID: <49C1AB11.8050607@savagexi.com> Hi Hans, > A large set of these kind of messages, but I assumed these concerned the > ri documention Yes. Newer versions of rdoc fix them. > Wrote a small test application and got the error messages > Unimplemented block at > /SourceCache/libxml2/libxml2-17.7.6/libxml2/xmlreader.c:1578 > trying to read inner xml and outer xml > > Anyone that had have the same problem ? > How to fix it ? Haven't see that one. Looks like a libxml2 issue to me, but hard to tell without seeing your test program. What version of libxml2 do you have installed? Maybe upgrade it using MacPorts? > I have not made any thing about the gem dependencies (and C bindings ?) > as I assumed that this already was done on my mac Not sure what you mean. Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From hans.marmolin at klockholm.se Thu Mar 19 02:35:08 2009 From: hans.marmolin at klockholm.se (Hans Marmolin) Date: Thu, 19 Mar 2009 07:35:08 +0100 Subject: [libxml-devel] Installation problem In-Reply-To: <49C1AB11.8050607@savagexi.com> References: <82C1F265-3CDC-478D-AB1D-53CAB7DE309D@klockholm.se> <49C1AB11.8050607@savagexi.com> Message-ID: <4C93FFB0-CC03-4362-B8F2-732A337D117E@klockholm.se> Thanks for your fast answer I use ibxml-ruby 1.1.2 and use the libxml2 that already was installed on my mac last changed july 2006 I have though about upgrading but do not know if it is really necessery and do not know how to do it. I do not have mac port My test sweet is like this reader = XML::Reader.string("123") reader.read puts '*****name '+reader.name.to_s puts '*****value '+reader.value.to_s puts '*****key '+reader['id'].to_s puts '*****node_type '+reader.node_type.to_s puts '*****read_string '+reader.read_string.to_s puts '*****next name '+reader.name.to_s reader.next_sibling puts '*****next sibing name '+reader.name.to_s r=reader.expand puts '*****'+r.to_s puts '*****expand next name '+reader.name.to_s end and this is the result *****name foo *****value *****key *****node_type 1 Unimplemented block at /SourceCache/libxml2/libxml2-17.7.6/libxml2/ xmlreader.c:1578 *****read_inner_xml Unimplemented block at /SourceCache/libxml2/libxml2-17.7.6/libxml2/ xmlreader.c:1594 *****read_outer_xml *****next name foo *****next sibing name foo ***** 1 2 3 *****expand next name foo Thanks for any help Do you know any tutorial or how to or examples of using XML::reader and libxml-ruby ____________________________________________ Hans Marmolin Prof Cognitive Ergonomics St: Larsgatan 50, 58224 Link?ping, Sweden Phone: +46708371202 19 mar 2009 kl. 03.16 skrev Charlie Savage: > Hi Hans, > >> A large set of these kind of messages, but I assumed these >> concerned the ri documention > > Yes. Newer versions of rdoc fix them. > >> Wrote a small test application and got the error messages >> Unimplemented block at /SourceCache/libxml2/libxml2-17.7.6/libxml2/ >> xmlreader.c:1578 >> trying to read inner xml and outer xml >> Anyone that had have the same problem ? >> How to fix it ? > > Haven't see that one. Looks like a libxml2 issue to me, but hard to > tell without seeing your test program. What version of libxml2 do > you have installed? Maybe upgrade it using MacPorts? > >> I have not made any thing about the gem dependencies (and C >> bindings ?) as I assumed that this already was done on my mac > > Not sure what you mean. > > Charlie > _______________________________________________ > libxml-devel mailing list > libxml-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/libxml-devel From jimmy at deefa.com Sat Mar 21 11:55:47 2009 From: jimmy at deefa.com (James Healy) Date: Sun, 22 Mar 2009 02:55:47 +1100 Subject: [libxml-devel] XML::Reader and segfauly in 1.1.2 Message-ID: <20090321155547.GO1560@deefa.com> Hi, the following script triggers a segfault for me under Linux, both i386 and amd64. --- require 'rubygems' gem "libxml-ruby", "1.1.2" require 'libxml' filename = File.join(File.dirname(__FILE__), "bookwise.xml") reader = LibXML::XML::Reader.string(File.read(filename)) while reader.read if reader.node_type == LibXML::XML::Reader::TYPE_ELEMENT if reader.name == "Header" puts reader.read_outer_xml elsif reader.name == "Product" puts reader.read_outer_xml end end end --- I'm basically attempting to grab the full text of all Header and Product nodes in the input file. My actual code doesn't just puts the elements, this is simply the simplest script I could make that triggers it. Occasionally, instead of a segfault I get a fatal error, "Opening and ending tag mismatch", despite the fact the input file is valid and well formed. The bookwise.xml file I'm testing with is available at http://gir.deefa.com/bookwise.xml I am running libxml-ruby 1.1.2 and libxml 2.7.3 on MRI 1.8.7. I have also tested with libxml 2.6.32, and get the same result. Am I use the Reader bindings correctly? If I'm not, I'd appreciate a few pointers. -- James Healy Sun, 22 Mar 2009 02:14:22 +1100 From cfis at savagexi.com Sun Mar 22 00:16:49 2009 From: cfis at savagexi.com (Charlie Savage) Date: Sat, 21 Mar 2009 22:16:49 -0600 Subject: [libxml-devel] XML::Reader and segfauly in 1.1.2 In-Reply-To: <20090321155547.GO1560@deefa.com> References: <20090321155547.GO1560@deefa.com> Message-ID: <49C5BBB1.9080608@savagexi.com> Hi James, Thanks for the test case and report. > require 'rubygems' > gem "libxml-ruby", "1.1.2" > require 'libxml' > > filename = File.join(File.dirname(__FILE__), "bookwise.xml") > reader = LibXML::XML::Reader.string(File.read(filename)) > > while reader.read > > if reader.node_type == LibXML::XML::Reader::TYPE_ELEMENT > if reader.name == "Header" > puts reader.read_outer_xml > elsif reader.name == "Product" > puts reader.read_outer_xml > end > end > > end > --- > > I'm basically attempting to grab the full text of all Header and Product > nodes in the input file. My actual code doesn't just puts the elements, > this is simply the simplest script I could make that triggers it. You might want to use XPath instead for this. > Occasionally, instead of a segfault I get a fatal error, "Opening and > ending tag mismatch", despite the fact the input file is valid and well > formed. So I tested this with libxml-ruby 1.1.2, ruby 1.8.6 (patch level 287) on Windows Vista and Fedora 10 and could not reproduce. > Am I use the Reader bindings correctly? Yes. >If I'm not, I'd appreciate a few pointers. Since I can't reproduce the problem, can you compile the bindings with debug information and get a stack trace? Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From jh at chabran.fr Tue Mar 24 05:53:36 2009 From: jh at chabran.fr (JH. Chabran) Date: Tue, 24 Mar 2009 02:53:36 -0700 (PDT) Subject: [libxml-devel] Schema Validation issue "No declaration for elemen (...)" Message-ID: Hi, I'm using libxml-ruby 1.1.3, with ruby 1.9.1 on OSX. Reading stuff works nicely, but when it comes to schema validation I'm having the following issue. I have one XSD you can find here : http://pastie.org/private/ibjtrwso8gckh1lz9sdda The document I try to validate is : --- World --- My problem is when I try to validate it, I'm punched back with : "Error: No declaration for element emph at /Users/jh/Sources/(...)/ 2.xml:4" At first I thought it was a mistake I could have made in the schema, but after trying to validate it with other tools, it seems my schema is correct. Moreover, I can't validate the w3school example you can find here : http://www.w3schools.com/Schema/schema_example.asp , I get the following error ( using the named types approach ) : http://pastie.org/425227 Here is the ruby code I use to validate : schema_document = XML::Document.file(path) @schema = XML::Schema.document(schema_document) XML::Document.file(document_path(document)).validate(@schema) Thanks, jhc_