From jimmy at deefa.com Wed Jun 3 11:34:08 2009 From: jimmy at deefa.com (James Healy) Date: Thu, 4 Jun 2009 01:34:08 +1000 Subject: [libxml-devel] XML::Reader and segfauly in 1.1.2 In-Reply-To: <49C5BBB1.9080608@savagexi.com> References: <20090321155547.GO1560@deefa.com> <49C5BBB1.9080608@savagexi.com> Message-ID: <20090603153408.GC1560@deefa.com> Hi Charlie, It's been some time since my original post, bhowever I recently found some time to look at this properly. I've attached 2 test scripts, both which processthis XML file: http://gir.deefa.com/bookwise.xml. They contain a thin wrapper around XML::Reader designed to iterate over the nodes of interest in the file. ok.rb runs without a problem, segfault.rb triggers a segfault. The scripts are identical, with the exception of a single uncommented puts line in segfault.rb It's possilbe I'm using expand() incorrectly. If that's the case, it'd be nice to get an error that isn't a segfault. The final attachment is copy of the backtrace from gdb. I'm running libxml-ruby 1.1.3 and libxml2 2.7.3 on Debian. cheers -- James Healy Thu, 04 Jun 2009 01:22:12 +1000 Charlie Savage wrote: > Hi James, > > Thanks for the test case and report. > >> require 'rubygems' >> gem "libxml-ruby", "1.1.2" >> require 'libxml' >> >> filename = File.join(File.dirname(__FILE__), "bookwise.xml") >> reader = LibXML::XML::Reader.string(File.read(filename)) >> >> while reader.read >> >> if reader.node_type == LibXML::XML::Reader::TYPE_ELEMENT >> if reader.name == "Header" puts reader.read_outer_xml >> elsif reader.name == "Product" >> puts reader.read_outer_xml >> end >> end >> >> end >> --- >> >> I'm basically attempting to grab the full text of all Header and Product >> nodes in the input file. My actual code doesn't just puts the elements, >> this is simply the simplest script I could make that triggers it. > > You might want to use XPath instead for this. > >> Occasionally, instead of a segfault I get a fatal error, "Opening and >> ending tag mismatch", despite the fact the input file is valid and well >> formed. > > So I tested this with libxml-ruby 1.1.2, ruby 1.8.6 (patch level 287) on > Windows Vista and Fedora 10 and could not reproduce. > >> Am I use the Reader bindings correctly? > > Yes. > >> If I'm not, I'd appreciate a few pointers. > > Since I can't reproduce the problem, can you compile the bindings with > debug information and get a stack trace? > > Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: ok.rb Type: application/x-ruby Size: 805 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: segfault.rb Type: application/x-ruby Size: 804 bytes Desc: not available URL: -------------- next part -------------- (gdb) backtrace #0 0xb7dc37f6 in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0xb7dc6d38 in *__GI_abort () at abort.c:88 #2 0xb7dfa065 in __libc_message (do_abort=2, fmt=0xb7ebda68 "*** glibc detected *** %s: %s: 0x%s ***\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:173 #3 0xb7e04215 in malloc_printerr (action=2, str=0xb7ebdb60 "double free or corruption (!prev)", ptr=0xa65d9c8) at malloc.c:5994 #4 0xb7e05aec in *__GI___libc_free (mem=0xa65d9c8) at malloc.c:3625 #5 0xb7874205 in xmlTextReaderFreeNode (reader=0xa658eb8, cur=0xa65d9c8) at xmlreader.c:474 #6 0xb7874f6e in xmlTextReaderRead__internal_alias (reader=0xa658eb8) at xmlreader.c:1400 #7 0xb78e2415 in rxml_reader_read (self=3084323980) at ruby_xml_reader.c:418 #8 0xb7f7cf02 in call_cfunc (func=0xb78e23e0 , recv=3084323980, len=6, argc=0, argv=0x7ea3) at eval.c:5752 #9 0xb7f88333 in rb_call0 (klass=3082073500, recv=3084323980, id=7657, oid=7657, argc=0, argv=0x0, body=0xb7b4ab38, flags=0) at eval.c:5904 #10 0xb7f88470 in rb_call (klass=3082073500, recv=3084323980, mid=7657, argc=0, argv=0x0, scope=0, self=3084364640) at eval.c:6151 #11 0xb7f82b69 in rb_eval (self=3084364640, n=) at eval.c:3492 #12 0xb7f8557c in rb_eval (self=3084364640, n=) at eval.c:3142 #13 0xb7f881fe in rb_call0 (klass=3084462080, recv=3084364640, id=10985, oid=10985, argc=0, argv=0x0, body=0xb7d85150, flags=0) at eval.c:6055 #14 0xb7f88470 in rb_call (klass=3084462080, recv=3084364640, mid=10985, argc=0, argv=0x0, scope=2, self=3084364640) at eval.c:6151 #15 0xb7f82cd8 in rb_eval (self=3084364640, n=) at eval.c:3513 #16 0xb7f83db4 in rb_eval (self=3084364640, n=) at eval.c:3677 #17 0xb7f82a82 in rb_eval (self=3084364640, n=) at eval.c:3486 #18 0xb7f85535 in rb_eval (self=3084364640, n=) at eval.c:3177 #19 0xb7f881fe in rb_call0 (klass=3084462080, recv=3084364640, id=4001, oid=4001, argc=0, argv=0x0, body=0xb7d85948, flags=0) at eval.c:6055 #20 0xb7f88470 in rb_call (klass=3084462080, recv=3084364640, mid=4001, argc=0, argv=0x0, scope=0, self=3084478800) at eval.c:6151 #21 0xb7f82b69 in rb_eval (self=3084478800, n=) at eval.c:3492 #22 0xb7f85ff1 in rb_eval (self=3084478800, n=) at eval.c:3222 #23 0xb7f94817 in ruby_exec_internal () at eval.c:1641 #24 0xb7f94862 in ruby_exec () at eval.c:1661 #25 0xb7f9489f in ruby_run () at eval.c:1671 #26 0x080486ed in main (argc=Cannot access memory at address 0x7ea3) at main.c:48 From jimmy at deefa.com Thu Jun 4 03:07:52 2009 From: jimmy at deefa.com (James Healy) Date: Thu, 4 Jun 2009 17:07:52 +1000 Subject: [libxml-devel] XML::Reader and encodings Message-ID: <20090604070752.GH1560@deefa.com> Hi, I'm trying to get a grip on how XML::Reader behaves when processing input files with varied encodings. When processing an XML I can call Reader#encoding and I get an integer that seems to correctly indicate the encoding of the file (by matching constants in the XML::Encoding::* space). I also notice that I can pass an :encoding option to the string(), file() and io() class methods, although in my testing modifying this option has no effect. What is this option for? Finally, I've tested parsing ISO-8859-1, UTF-8 and UTF-16 files. When I grab a node from the ISO-8859-1 and UTF-16 files, I get a string that is UTF-8 encoded. Does libxml transparently convert all input files to utf-8? It's fine if it does, I'm just trying to get the behaviour straight in my own head! -- James Healy Thu, 04 Jun 2009 16:52:21 +1000 From transfire at gmail.com Sun Jun 7 10:22:06 2009 From: transfire at gmail.com (Trans) Date: Sun, 7 Jun 2009 07:22:06 -0700 (PDT) Subject: [libxml-devel] CSS selectors Message-ID: Any chance of adding CSS selector support to say Node#find or equivalent method? I've written a data-driven template system that needs them, and as it stands that means I have to use Nokogiri on the backend. But I'd like to support LibXML-Ruby too. Thanks, T. From transfire at gmail.com Tue Jun 9 14:11:47 2009 From: transfire at gmail.com (Trans) Date: Tue, 9 Jun 2009 11:11:47 -0700 (PDT) Subject: [libxml-devel] Two Methods for LibXML::XML::XPath::Object Message-ID: I'd like to suggest two new methods for LibXML::XML::XPath::Object. The first is simply a #to_s method that returns a concat string of all nodes. Eg. xml = XML::Document.string("12") bes = xml.find('b') bes.to_s #=> "12" Currently #to_s is the same as #inspect, which isn't useful. The second is to add #find, so that we can do sub-searches on the nodes. Eg. xml = XML::Document.string(%{12}) bes = xml.find('b') bes.to_s #=> "12" qbs = bes.find("[@id='q']") qbs.to_s #=> "2" Thanks, Tom From transfire at gmail.com Tue Jun 9 15:11:29 2009 From: transfire at gmail.com (Trans) Date: Tue, 9 Jun 2009 12:11:29 -0700 (PDT) Subject: [libxml-devel] #find not matching the current context? Message-ID: I can't figure out how to do this. require 'libxml' xml = LibXML::XML::Document.string <<-EOS dummydummy EOS nodes = xml.find(".//*[@class='q']") nodes.to_a => [dummy, dummy] x = nodes[0] => dummy So far so good. x.find(".//*[@id='a']").to_a => [] no match? I expect it to be: => [dummy] Why isn't it matching against the current context? I can't seem to find any way to make it do that. Thanks, Tom From davidv at spindance.com Tue Jun 9 13:22:22 2009 From: davidv at spindance.com (David van Geest) Date: Tue, 9 Jun 2009 13:22:22 -0400 Subject: [libxml-devel] installing libxml-ruby on Windows Message-ID: <000b01c9e926$da00bbc0$4700000a@spindance.inc> Hi, I'm trying to install libxml-ruby on Windows, and am having a problem. It seems that this has happened to several other people but the issue has never been resolved (at least the solution has never been posted). The command line output is below, but here's the gist. Apparently the inflate() method can't be found in zlib. I see zlib1.dll in C:\ruby\bin\, but just to be sure I downloaded the zlib DLL and copied it to C:\Windows\System. Got the same result, so I explicitly specified the path to the library with the following command: C:\ruby\src\ruby-1.8.6-p111\ext\zlib>ruby extconf.rb --with-zlib-lib=c:\Files\in stallers\zlib\ --with-zlib-include=c:\Files\installers\zlib\include The only difference there was that it couldn't find the deflateReset() method. Anyone have any ideas? Anyone installed libxml-ruby on Windows successfully? Thanks, -David van Geest SpinDance, Inc. >gem install roxml Install required dependency libxml-ruby? [Yn] y Select which gem to install for your platform (i386-mswin32) 1. libxml-ruby 1.1.3 (x86-mswin32-60) 2. libxml-ruby 1.1.3 (ruby) 3. libxml-ruby 1.1.2 (ruby) 4. libxml-ruby 1.1.2 (x86-mswin32-60) 5. Skip this gem 6. Cancel installation > 3 Building native extensions. This could take a while... ERROR: While executing gem ... (Gem::Installer::ExtensionBuildError) ERROR: Failed to build gem native extension. ruby extconf.rb install roxml checking for socket() in socket.lib... no checking for gethostbyname() in nsl.lib... no checking for atan() in m.lib... yes checking for inflate() in z.lib... no checking for inflate() in zlib.lib... no checking for inflate() in zlib1.lib... no *** extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. Provided configuration options: --with-opt-dir --without-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --srcdir=. --curdir --ruby=c:/ruby/bin/ruby --with-iconv-dir --without-iconv-dir --with-iconv-include --without-iconv-include=${iconv-dir}/include --with-iconv-lib --without-iconv-lib=${iconv-dir}/lib --with-zlib-dir --without-zlib-dir --with-zlib-include --without-zlib-include=${zlib-dir}/include --with-zlib-lib --without-zlib-lib=${zlib-dir}/lib --with-socketlib --without-socketlib --with-nsllib --without-nsllib --with-mlib --without-mlib --with-zlib --without-zlib --with-zliblib --without-zliblib --with-zlib1lib --without-zlib1lib extconf failure: need zlib From davidv at spindance.com Tue Jun 9 15:28:48 2009 From: davidv at spindance.com (David van Geest) Date: Tue, 9 Jun 2009 15:28:48 -0400 Subject: [libxml-devel] installing libxml-ruby on Windows Message-ID: <001401c9e938$8322dbf0$4700000a@spindance.inc> Hi, I'm trying to install libxml-ruby on Windows, and am having a problem. It seems that this has happened to several other people but the issue has never been resolved (at least the solution has never been posted). The command line output is below, but here's the gist. Apparently the inflate() method can't be found in zlib. I see zlib1.dll in C:\ruby\bin\, but just to be sure I downloaded the zlib DLL and copied it to C:\Windows\System. Got the same result, so I explicitly specified the path to the library with the following command: C:\ruby\src\ruby-1.8.6-p111\ext\zlib>ruby extconf.rb --with-zlib-lib=c:\Files\in stallers\zlib\ --with-zlib-include=c:\Files\installers\zlib\include The only difference there was that it couldn't find the deflateReset() method. Anyone have any ideas? Anyone installed libxml-ruby on Windows successfully? Thanks, -David van Geest SpinDance, Inc. >gem install roxml Install required dependency libxml-ruby? [Yn] y Select which gem to install for your platform (i386-mswin32) 1. libxml-ruby 1.1.3 (x86-mswin32-60) 2. libxml-ruby 1.1.3 (ruby) 3. libxml-ruby 1.1.2 (ruby) 4. libxml-ruby 1.1.2 (x86-mswin32-60) 5. Skip this gem 6. Cancel installation > 3 Building native extensions. This could take a while... ERROR: While executing gem ... (Gem::Installer::ExtensionBuildError) ERROR: Failed to build gem native extension. ruby extconf.rb install roxml checking for socket() in socket.lib... no checking for gethostbyname() in nsl.lib... no checking for atan() in m.lib... yes checking for inflate() in z.lib... no checking for inflate() in zlib.lib... no checking for inflate() in zlib1.lib... no *** extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. Provided configuration options: --with-opt-dir --without-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --srcdir=. --curdir --ruby=c:/ruby/bin/ruby --with-iconv-dir --without-iconv-dir --with-iconv-include --without-iconv-include=${iconv-dir}/include --with-iconv-lib --without-iconv-lib=${iconv-dir}/lib --with-zlib-dir --without-zlib-dir --with-zlib-include --without-zlib-include=${zlib-dir}/include --with-zlib-lib --without-zlib-lib=${zlib-dir}/lib --with-socketlib --without-socketlib --with-nsllib --without-nsllib --with-mlib --without-mlib --with-zlib --without-zlib --with-zliblib --without-zliblib --with-zlib1lib --without-zlib1lib extconf failure: need zlib From davidv at spindance.com Wed Jun 10 10:43:59 2009 From: davidv at spindance.com (David van Geest) Date: Wed, 10 Jun 2009 10:43:59 -0400 Subject: [libxml-devel] installing libxml-ruby on Windows In-Reply-To: <001401c9e938$8322dbf0$4700000a@spindance.inc> References: <001401c9e938$8322dbf0$4700000a@spindance.inc> Message-ID: <000b01c9e9d9$e44e94e0$4700000a@spindance.inc> Still trying to install libxml-ruby on Windows XP, but I'm switching tactics from the previous post. I've downloaded the libxml-ruby-1.1.3-x86-mswin32-60.gem from rubyforge. Switching to my download directory and running gem install, I get the following trace. Any ideas on what might be going wrong? C:\Files\installers>gem install libxml-ruby-1.1.3-x86-mswin32-60.gem Building native extensions. This could take a while... ERROR: While executing gem ... (RuntimeError) Error instaling libxml-ruby-1.1.3-x86-mswin32-60.gem: ERROR: Failed to build gem native extension. rake RUBYARCHDIR=c:/ruby/lib/ruby/gems/1.8/gems/libxml-ruby-1.1.3-x86-mswin32-60 /lib RUBYLIBDIR=c:/ruby/lib/ruby/gems/1.8/gems/libxml-ruby-1.1.3-x86-mswin32-60/ lib extension rake aborted! Don't know how to build task 'extension' (See full trace by running task with --trace) (in c:/ruby/lib/ruby/gems/1.8/gems/libxml-ruby-1.1.3-x86-mswin32-60/ext/mingw) Gem files will remain installed in c:/ruby/lib/ruby/gems/1.8/gems/libxml-ruby-1. 1.3-x86-mswin32-60 for inspection. Results logged to c:/ruby/lib/ruby/gems/1.8/gems/libxml-ruby-1.1.3-x86-mswin32-6 0/ext/mingw/gem_make.out Thanks, -David From davidv at spindance.com Wed Jun 10 10:56:16 2009 From: davidv at spindance.com (David van Geest) Date: Wed, 10 Jun 2009 10:56:16 -0400 Subject: [libxml-devel] installing libxml-ruby on Windows References: <001401c9e938$8322dbf0$4700000a@spindance.inc> Message-ID: <000c01c9e9db$9b24ce40$4700000a@spindance.inc> Problem solved. Apparently there's a bug in gem. Running gem update --system and then re-attempting the libxml-ruby install is successful. -David van Geest Spindance, Inc. -----Original Message----- From: David van Geest [mailto:davidv at spindance.com] Sent: Wednesday, June 10, 2009 10:44 AM To: 'libxml-devel at rubyforge.org' Cc: 'shaneh at spindance.com' Subject: RE: [libxml-devel] installing libxml-ruby on Windows Still trying to install libxml-ruby on Windows XP, but I'm switching tactics from the previous post. I've downloaded the libxml-ruby-1.1.3-x86-mswin32-60.gem from rubyforge. Switching to my download directory and running gem install, I get the following trace. Any ideas on what might be going wrong? C:\Files\installers>gem install libxml-ruby-1.1.3-x86-mswin32-60.gem Building native extensions. This could take a while... ERROR: While executing gem ... (RuntimeError) Error instaling libxml-ruby-1.1.3-x86-mswin32-60.gem: ERROR: Failed to build gem native extension. rake RUBYARCHDIR=c:/ruby/lib/ruby/gems/1.8/gems/libxml-ruby-1.1.3-x86-mswin32-60 /lib RUBYLIBDIR=c:/ruby/lib/ruby/gems/1.8/gems/libxml-ruby-1.1.3-x86-mswin32-60/ lib extension rake aborted! Don't know how to build task 'extension' (See full trace by running task with --trace) (in c:/ruby/lib/ruby/gems/1.8/gems/libxml-ruby-1.1.3-x86-mswin32-60/ext/mingw) Gem files will remain installed in c:/ruby/lib/ruby/gems/1.8/gems/libxml-ruby-1. 1.3-x86-mswin32-60 for inspection. Results logged to c:/ruby/lib/ruby/gems/1.8/gems/libxml-ruby-1.1.3-x86-mswin32-6 0/ext/mingw/gem_make.out Thanks, -David From transfire at gmail.com Mon Jun 15 11:24:22 2009 From: transfire at gmail.com (Trans) Date: Mon, 15 Jun 2009 11:24:22 -0400 Subject: [libxml-devel] Two Methods for LibXML::XML::XPath::Object In-Reply-To: <296445fb0906150428i3e8f2c39p62de94da64077ff7@mail.gmail.com> References: <296445fb0906150428i3e8f2c39p62de94da64077ff7@mail.gmail.com> Message-ID: <4b6f054f0906150824m75c4696bx44c5b954e379434a@mail.gmail.com> On Mon, Jun 15, 2009 at 7:28 AM, Kevin Menard wrote: > I'm not a dev on the project, but I think you should be opening enhancement > tickets so these suggestions don't get lost on the mailing list. Done. Thanks. -Trans