From transfire at gmail.com Wed Mar 5 10:19:41 2008 From: transfire at gmail.com (Trans) Date: Wed, 5 Mar 2008 07:19:41 -0800 (PST) Subject: [libxml-devel] Random crashes In-Reply-To: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> Message-ID: Seeing that no one else has responded, I take as my cue to raise a red flag. I haven't talked to Dan about this, but I'm going to go out on a limb and say, that after following this project for many years now -- not to mention personally working to revitalize it TWICE, I've come to the conclusion that there may be something fundamentally wrong with its design. For some reason there have always been segfaults, and no one can seem to get rid of them all. Unfortunately, I'm not a C coder, so I can't really assess this from a technical standpoint, but only as as the "hands-off" project manger/assistant-manager role I've maintained, and as a user. While the library works fine in many cases, there seem to be enough problem cases that the library can't pick up general traction. Perhaps others can fill in on the technical difficulties and the breadth of these issues. While problems could be arising from libXML2 itself I suspect that it is not generally the case, which makes me wonder how libxml-ruby is actually designed under the hood. Wouldn't it makes sense simply to map the libxml2 api to the Ruby C extension essentially one for one, and then write everything else as a pure-Ruby "Ruby-way" wrapper around that? Maybe this is how it is designed, but I can't really imagine it because I would expect it to work fairly flawlessly if that were the case. But like I said I'm not a C coder, so I can say for sure. I just wanted to put my thought out there and get some feed back on this. So I think at this point, the questions must be asked: How serious are these continuing issues? Are they fixable? And should the current code be abandoned and this project completely re-written from the ground up instead? -- Or am I interpreting the error reports and the otherwise relative silence on this list incorrectly? T. From keith at oreilly.com Wed Mar 5 10:30:37 2008 From: keith at oreilly.com (Keith Fahlgren) Date: Wed, 05 Mar 2008 07:30:37 -0800 Subject: [libxml-devel] Random crashes In-Reply-To: References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> Message-ID: <47CEBC9D.90503@oreilly.com> On 3/5/08 7:19 AM, Trans wrote: > While problems could be arising from libXML2 itself I suspect that it > is not generally the case, which makes me wonder how libxml-ruby is > actually designed under the hood. Just for the record, I've never had these problems with the Python libxml2 binding 'lxml' (http://codespeak.net/lxml/). HTH, Keith From saurabhnanda at gmail.com Wed Mar 5 14:52:33 2008 From: saurabhnanda at gmail.com (Saurabh Nanda) Date: Thu, 6 Mar 2008 01:22:33 +0530 Subject: [libxml-devel] Random crashes In-Reply-To: References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> Message-ID: <794f042d0803051152x7d23cd4en5b6f34ceac64b535@mail.gmail.com> Trans, Thanks for replying. Although your answer doesn't help solve my problem with libxml-ruby it does give me a sense that it's going to take long to get this fixed. At least I can start rewriting my schemas in Relax NG now, so that I can meet my application production rollout deadline. Saurabh. -- http://nandz.blogspot.com http://foodieforlife.blogspot.com From sean at chittenden.org Wed Mar 5 15:43:01 2008 From: sean at chittenden.org (Sean Chittenden) Date: Wed, 5 Mar 2008 12:43:01 -0800 Subject: [libxml-devel] Random crashes In-Reply-To: References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> Message-ID: <3F00FAF9-B562-445E-B673-1CBA2850C7AE@chittenden.org> > Seeing that no one else has responded I don't know if it's been nixed or not, but the is_shadow concept is fundamentally broken in any sophisticated workload. Every operation needs to return a copy. I initially did the shadow bits for speed reasons (object thrash), which satisfied my workload requirements at the time, but it's proven to be a really bad idea in practice. If I had cycles/interest in rewriting things, I'd take aim at anything that returns a wrapped xml object and would return a copy of the object instead. What I believe is happening is the GC is coming along, invalidating portions of the XML document structure, and leaves a dangling pointer. I don't know if that's what Dan fixed in the latest rev or not, but that's the only design thing that I'm aware of that _needs_ to get addressed. Is this still feasible? -sc -- Sean Chittenden sean at chittenden.org http://sean.chittenden.org/ From marc at bloodnok.com Wed Mar 5 10:55:27 2008 From: marc at bloodnok.com (Marc Munro) Date: Wed, 05 Mar 2008 07:55:27 -0800 Subject: [libxml-devel] Random crashes In-Reply-To: References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> Message-ID: <1204732527.17720.10.camel@bloodnok.com> As someone who has experienced a good number of these crashes, who has attempted to track them down, and who has at least some C experience, I would like to make a recommendation. From my own reading and limited understanding of the code I don't think there is anything fundamentally wrong. At least not systemically: there may be specific interfaces that need more work but I think Dan's new memory model stuff is, on the whole, wonderful. There clearly have been problems. I haven't hit any since upgrading to the latest svn version about 2 weeks ago but I'm not yet feeling confident that we're there yet. The big problem from my point of view is that debugging is too damn difficult, and I think is a fault with ruby 1.8 rather than with libxml. If there is work to be prioritised right now, I would recommend making libxml and libxsl work with ruby 1.9. Ruby 1.9 is much friendlier to valgrind which is the ideal tool to use to find memory faults. I intend to run my app under ruby 1.9 as soon as I'm able, and I will happily run it using valgrind to help track down the remaining issues. Respectfully. __ Marc On Wed, 2008-03-05 at 07:19 -0800, Trans wrote: > Seeing that no one else has responded, I take as my cue to raise a red > flag. I haven't talked to Dan about this, but I'm going to go out on a > limb and say, that after following this project for many years now -- > not to mention personally working to revitalize it TWICE, I've come to > the conclusion that there may be something fundamentally wrong with > its design. For some reason there have always been segfaults, and no > one can seem to get rid of them all. Unfortunately, I'm not a C coder, > so I can't really assess this from a technical standpoint, but only as > as the "hands-off" project manger/assistant-manager role I've > maintained, and as a user. While the library works fine in many cases, > there seem to be enough problem cases that the library can't pick up > general traction. Perhaps others can fill in on the technical > difficulties and the breadth of these issues. > > While problems could be arising from libXML2 itself I suspect that it > is not generally the case, which makes me wonder how libxml-ruby is > actually designed under the hood. Wouldn't it makes sense simply to > map the libxml2 api to the Ruby C extension essentially one for one, > and then write everything else as a pure-Ruby "Ruby-way" wrapper > around that? Maybe this is how it is designed, but I can't really > imagine it because I would expect it to work fairly flawlessly if that > were the case. But like I said I'm not a C coder, so I can say for > sure. I just wanted to put my thought out there and get some feed back > on this. > > So I think at this point, the questions must be asked: How serious are > these continuing issues? Are they fixable? And should the current code > be abandoned and this project completely re-written from the ground up > instead? -- Or am I interpreting the error reports and the otherwise > relative silence on this list incorrectly? > > T. > _______________________________________________ > libxml-devel mailing list > libxml-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/libxml-devel > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20080305/b45c20d0/attachment.bin From danj.3skel at gmail.com Fri Mar 7 05:47:34 2008 From: danj.3skel at gmail.com (Dan) Date: Fri, 7 Mar 2008 05:47:34 -0500 Subject: [libxml-devel] Random crashes In-Reply-To: <3F00FAF9-B562-445E-B673-1CBA2850C7AE@chittenden.org> References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> <3F00FAF9-B562-445E-B673-1CBA2850C7AE@chittenden.org> Message-ID: <20080307054734.37e6be88@Nokia-N810-51-3> Hi, Sorry I have been off the radar, some personal crap has been keeping me occupied. The memory model changes have been largely effective and creates a good one to one coupling between ruby object and libxml entities. The issue that sean is talking about is no longer the case. I have removed Reference counting everywhere I have been and multiple recall of an entity will return the same ruby object. I do not believe the libxml2 library is at all the problem. My greatest frustration has been the the GC and debugging and memory tracking. Basically the GC sucks and the way it is written and that it lacks any instrumentation for debugging makes for nothing but trouble. I have worked with Marc directly on a number of faults and have been largely successful, but the time cost is high, and I cannot make this my life's work. There are components that have not been evaled or rewritten yet and the problems that people are be experiencing may be the result of mixing those features. May or may not, since I have not looked. I have contemplated abandoning support, but I would rather not. Being the only coder makes it harder as ultimately every bug is my problem. But that is not the issue, ruby 1.8 is. I have considered moving to rubinius to see if their compatibilty api is better than the original. But that may yield nothing as it could work fine there. 1.9 I have not considered, but valgrind is sort of a white elephant since the GC screws things up without a trace before the real problem occurs. I am open to suggestions, but I can't do this part on my own. Dan On Wed, 5 Mar 2008 12:43:01 -0800 Sean Chittenden wrote: > > Seeing that no one else has responded > > I don't know if it's been nixed or not, but the is_shadow concept is > fundamentally broken in any sophisticated workload. Every operation > needs to return a copy. I initially did the shadow bits for speed > reasons (object thrash), which satisfied my workload requirements at > the time, but it's proven to be a really bad idea in practice. If I > had cycles/interest in rewriting things, I'd take aim at anything > that returns a wrapped xml object and would return a copy of the > object instead. What I believe is happening is the GC is coming > along, invalidating portions of the XML document structure, and > leaves a dangling pointer. I don't know if that's what Dan fixed in > the latest rev or not, but that's the only design thing that I'm > aware of that _needs_ to get addressed. Is this still feasible? -sc From transfire at gmail.com Fri Mar 7 06:52:39 2008 From: transfire at gmail.com (Trans) Date: Fri, 7 Mar 2008 03:52:39 -0800 (PST) Subject: [libxml-devel] Random crashes In-Reply-To: <20080307054734.37e6be88@Nokia-N810-51-3> References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> <3F00FAF9-B562-445E-B673-1CBA2850C7AE@chittenden.org> <20080307054734.37e6be88@Nokia-N810-51-3> Message-ID: On Mar 7, 5:47?am, Dan wrote: > Hi, > > Sorry I have been off the radar, some personal crap has been keeping me > occupied. > > The memory model changes have been largely effective and creates a good > one to one coupling between ruby object and libxml entities. The issue > that sean is talking about is no longer the case. I have removed > Reference counting everywhere I have been and multiple recall of an > entity will return the same ruby object. I do not believe the libxml2 > library is at all the problem. My greatest frustration has been the the > GC and debugging and memory tracking. Basically the GC sucks and the > way it is written and that it lacks any instrumentation for debugging > makes for nothing but trouble. I have worked with Marc directly on a > number of faults and have been largely successful, but the time cost is > high, and I cannot make this my life's work. > > There are components that have not been evaled or rewritten yet and the > problems that people are be experiencing may be the result of mixing > those features. May or may not, since I have not looked. > > I have contemplated abandoning support, but I would rather not. Being > the only coder makes it harder as ultimately every bug is my problem. > But that is not the issue, ruby 1.8 is. I have considered moving to > rubinius to see if their compatibilty api is better than the original. > But that may yield nothing as it could work fine there. 1.9 I have not > considered, but valgrind is sort of a white elephant since the GC > screws things up without a trace before the real problem occurs. > > I am open to suggestions, but I can't do this part on my own. Hey Dan, what do you think of forwarding this to ruby-core mailing list? I'm suspect the best input on the matter might come from there. T. From danj.3skel at gmail.com Fri Mar 7 09:36:25 2008 From: danj.3skel at gmail.com (Dan Janowski) Date: Fri, 7 Mar 2008 09:36:25 -0500 Subject: [libxml-devel] Random crashes In-Reply-To: References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> <3F00FAF9-B562-445E-B673-1CBA2850C7AE@chittenden.org> <20080307054734.37e6be88@Nokia-N810-51-3> Message-ID: My experience is that -core is not very helpful and it may well sound like any number of other complaints they have received. I have my doubts there is anything they can really offer. Dan > Hey Dan, what do you think of forwarding this to ruby-core mailing > list? I'm suspect the best input on the matter might come from there. > > T. From sean at chittenden.org Fri Mar 7 16:27:38 2008 From: sean at chittenden.org (Sean Chittenden) Date: Fri, 7 Mar 2008 13:27:38 -0800 Subject: [libxml-devel] Random crashes In-Reply-To: <20080307054734.37e6be88@Nokia-N810-51-3> References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> <3F00FAF9-B562-445E-B673-1CBA2850C7AE@chittenden.org> <20080307054734.37e6be88@Nokia-N810-51-3> Message-ID: > I have contemplated abandoning support, but I would rather not. Being > the only coder makes it harder as ultimately every bug is my problem. > But that is not the issue, ruby 1.8 is. I have considered moving to > rubinius to see if their compatibilty api is better than the original. > But that may yield nothing as it could work fine there. 1.9 I have not > considered, but valgrind is sort of a white elephant since the GC > screws things up without a trace before the real problem occurs. Hey Dan. Check out this post: http://sean.chittenden.org/news/2008/01/12/ I think that trick may be infinitely more useful in debugging the kind of class of problem you're describing. Your call for developer support is not falling on deft ears and I can sympathize with the burden of support after a piece of software is written. #{job} + #{real_life} + #{oss_time} ? Guess which gets squeezed first. The economics of open source suck on the development side of things. :-/ -sc -- Sean Chittenden sean at chittenden.org http://sean.chittenden.org/ From jburgess777 at googlemail.com Fri Mar 7 16:47:22 2008 From: jburgess777 at googlemail.com (Jon Burgess) Date: Fri, 07 Mar 2008 21:47:22 +0000 Subject: [libxml-devel] Random crashes In-Reply-To: References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> <3F00FAF9-B562-445E-B673-1CBA2850C7AE@chittenden.org> <20080307054734.37e6be88@Nokia-N810-51-3> Message-ID: <1204926442.4215.78.camel@localhost.localdomain> On Fri, 2008-03-07 at 13:27 -0800, Sean Chittenden wrote: > > I have contemplated abandoning support, but I would rather not. Being > > the only coder makes it harder as ultimately every bug is my problem. > > But that is not the issue, ruby 1.8 is. I have considered moving to > > rubinius to see if their compatibilty api is better than the original. > > But that may yield nothing as it could work fine there. 1.9 I have not > > considered, but valgrind is sort of a white elephant since the GC > > screws things up without a trace before the real problem occurs. > > Hey Dan. Check out this post: > > http://sean.chittenden.org/news/2008/01/12/ > > I think that trick may be infinitely more useful in debugging the kind > of class of problem you're describing. A little known feature of glibc is mtrace which does pretty much the same thing. See man mtrace / info mtrace for details. Jon From marc at bloodnok.com Mon Mar 10 12:12:55 2008 From: marc at bloodnok.com (Marc Munro) Date: Mon, 10 Mar 2008 09:12:55 -0700 Subject: [libxml-devel] Random crashes In-Reply-To: References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> <3F00FAF9-B562-445E-B673-1CBA2850C7AE@chittenden.org> <20080307054734.37e6be88@Nokia-N810-51-3> Message-ID: <1205165575.25829.2.camel@bloodnok.com> Here's my latest random crash. Please let me know if I can do anything more to help track this down. marc:[import]$ gdb ruby GNU gdb 6.7.1-debian Copyright (C) 2007 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i486-linux-gnu"... (no debugging symbols found) Using host libthread_db library "/lib/libthread_db.so.1". (gdb) run ./gnumeric2dat.rb dd_ab.gnumeric Starting program: /usr/bin/ruby ./gnumeric2dat.rb dd_ab.gnumeric (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) [Thread debugging using libthread_db enabled] (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) [New Thread 0xb7ce78c0 (LWP 28199)] (no debugging symbols found) (no debugging symbols found) Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xb7ce78c0 (LWP 28199)] 0xb7c81813 in ruby_xml_parser_mark (rxp=0x809d980) at ruby_xml_parser.c:951 951 if (!NIL_P(((rx_io_data *)rxp->data)->io)) (gdb) bt #0 0xb7c81813 in ruby_xml_parser_mark (rxp=0x809d980) at ruby_xml_parser.c:951 #1 0xb7eed5a5 in ?? () from /usr/lib/libruby1.8.so.1.8 #2 0x0809d980 in ?? () #3 0x00000000 in ?? () (gdb) __ Marc -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://rubyforge.org/pipermail/libxml-devel/attachments/20080310/ecbecf22/attachment-0001.bin From sean at chittenden.org Mon Mar 10 12:41:06 2008 From: sean at chittenden.org (Sean Chittenden) Date: Mon, 10 Mar 2008 09:41:06 -0700 Subject: [libxml-devel] Random crashes In-Reply-To: <1204926442.4215.78.camel@localhost.localdomain> References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> <3F00FAF9-B562-445E-B673-1CBA2850C7AE@chittenden.org> <20080307054734.37e6be88@Nokia-N810-51-3> <1204926442.4215.78.camel@localhost.localdomain> Message-ID: >> Hey Dan. Check out this post: >> >> http://sean.chittenden.org/news/2008/01/12/ >> >> I think that trick may be infinitely more useful in debugging the >> kind >> of class of problem you're describing. > > A little known feature of glibc is mtrace which does pretty much the > same thing. See man mtrace / info mtrace for details. That assumes you want to be on a platform that uses glibc. :) -sc -- Sean Chittenden sean at chittenden.org http://sean.chittenden.org/ From sean at chittenden.org Mon Mar 10 12:47:02 2008 From: sean at chittenden.org (Sean Chittenden) Date: Mon, 10 Mar 2008 09:47:02 -0700 Subject: [libxml-devel] Random crashes In-Reply-To: <1205165575.25829.2.camel@bloodnok.com> References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> <3F00FAF9-B562-445E-B673-1CBA2850C7AE@chittenden.org> <20080307054734.37e6be88@Nokia-N810-51-3> <1205165575.25829.2.camel@bloodnok.com> Message-ID: <50AFAE35-74EA-44F5-8555-96A49FB5533B@chittenden.org> > Please let me know if I can do anything more to help track this down. I know what this trace means, but not where the bug occurred. In a previous garbage collection sweep, the GC mark routine failed to mark an object and it was GC'ed - resulting in dangling pointers (which macro's such as NIL_P() don't handle very well). Before you dereferenced rxp->data->io, the GC cleaned up either rxp->data->io, rxp->data, or rxp. It's possible that memory location was recycled into a different object which can lead to further "interesting" behavior. If the shadow malloc/copy problems have been resolved, then this sounds like a mark/sweep issue where a node isn't being mark'ed where it needs to be. -sc > marc:[import]$ gdb ruby > GNU gdb 6.7.1-debian > Copyright (C) 2007 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "i486-linux-gnu"... > (no debugging symbols found) > Using host libthread_db library "/lib/libthread_db.so.1". > (gdb) run ./gnumeric2dat.rb dd_ab.gnumeric > Starting program: /usr/bin/ruby ./gnumeric2dat.rb dd_ab.gnumeric > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > [Thread debugging using libthread_db enabled] > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > [New Thread 0xb7ce78c0 (LWP 28199)] > (no debugging symbols found) > (no debugging symbols found) > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0xb7ce78c0 (LWP 28199)] > 0xb7c81813 in ruby_xml_parser_mark (rxp=0x809d980) at > ruby_xml_parser.c:951 > 951 if (!NIL_P(((rx_io_data *)rxp->data)->io)) > (gdb) bt > #0 0xb7c81813 in ruby_xml_parser_mark (rxp=0x809d980) at > ruby_xml_parser.c:951 > #1 0xb7eed5a5 in ?? () from /usr/lib/libruby1.8.so.1.8 > #2 0x0809d980 in ?? () > #3 0x00000000 in ?? () > (gdb) -- Sean Chittenden sean at chittenden.org http://sean.chittenden.org/ From danj.3skel at gmail.com Mon Mar 10 19:50:40 2008 From: danj.3skel at gmail.com (Dan) Date: Mon, 10 Mar 2008 19:50:40 -0400 Subject: [libxml-devel] Random crashes In-Reply-To: <50AFAE35-74EA-44F5-8555-96A49FB5533B@chittenden.org> References: <794f042d0802290549p75a32773se73b72b893a7f773@mail.gmail.com> <3F00FAF9-B562-445E-B673-1CBA2850C7AE@chittenden.org> <20080307054734.37e6be88@Nokia-N810-51-3> <1205165575.25829.2.camel@bloodnok.com> <50AFAE35-74EA-44F5-8555-96A49FB5533B@chittenden.org> Message-ID: <20080310195040.65b11ccd@Nokia-N810-51-3> The IO component of the extension has not been carefully examined in the new mem model. It is quite possible that the mark is not propogating to the io instances. This I can look at (it seems concrete). Dan On Mon, 10 Mar 2008 09:47:02 -0700 Sean Chittenden wrote: > > Please let me know if I can do anything more to help track this down. > > I know what this trace means, but not where the bug occurred. > > In a previous garbage collection sweep, the GC mark routine failed to > mark an object and it was GC'ed - resulting in dangling pointers > (which macro's such as NIL_P() don't handle very well). Before you > dereferenced rxp->data->io, the GC cleaned up either rxp->data->io, > rxp->data, or rxp. It's possible that memory location was recycled > into a different object which can lead to further "interesting" > behavior. If the shadow malloc/copy problems have been resolved, then > this sounds like a mark/sweep issue where a node isn't being mark'ed > where it needs to be. > > -sc > > > marc:[import]$ gdb ruby > > GNU gdb 6.7.1-debian > > Copyright (C) 2007 Free Software Foundation, Inc. > > License GPLv3+: GNU GPL version 3 or later > > > > This is free software: you are free to change and redistribute it. > > There is NO WARRANTY, to the extent permitted by law. Type "show > > copying" > > and "show warranty" for details. > > This GDB was configured as "i486-linux-gnu"... > > (no debugging symbols found) > > Using host libthread_db library "/lib/libthread_db.so.1". > > (gdb) run ./gnumeric2dat.rb dd_ab.gnumeric > > Starting program: /usr/bin/ruby ./gnumeric2dat.rb dd_ab.gnumeric > > (no debugging symbols found) > > (no debugging symbols found) > > (no debugging symbols found) > > (no debugging symbols found) > > [Thread debugging using libthread_db enabled] > > (no debugging symbols found) > > (no debugging symbols found) > > (no debugging symbols found) > > (no debugging symbols found) > > [New Thread 0xb7ce78c0 (LWP 28199)] > > (no debugging symbols found) > > (no debugging symbols found) > > > > Program received signal SIGSEGV, Segmentation fault. > > [Switching to Thread 0xb7ce78c0 (LWP 28199)] > > 0xb7c81813 in ruby_xml_parser_mark (rxp=0x809d980) at > > ruby_xml_parser.c:951 > > 951 if (!NIL_P(((rx_io_data *)rxp->data)->io)) > > (gdb) bt > > #0 0xb7c81813 in ruby_xml_parser_mark (rxp=0x809d980) at > > ruby_xml_parser.c:951 > > #1 0xb7eed5a5 in ?? () from /usr/lib/libruby1.8.so.1.8 > > #2 0x0809d980 in ?? () > > #3 0x00000000 in ?? () > > (gdb) > > > -- > Sean Chittenden > sean at chittenden.org > http://sean.chittenden.org/ > > _______________________________________________ > libxml-devel mailing list > libxml-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/libxml-devel From stephen.bannasch at deanbrook.org Wed Mar 26 00:12:39 2008 From: stephen.bannasch at deanbrook.org (Stephen Bannasch) Date: Wed, 26 Mar 2008 00:12:39 -0400 Subject: [libxml-devel] xml/libxml won't load in OS X 10.5.2 Leopard In-Reply-To: <94abb038-a936-406f-8223-fc18ea0eb66a@q33g2000hsh.googlegroups.com> References: <47C5AF40.2050505@revasystems.com> <94abb038-a936-406f-8223-fc18ea0eb66a@q33g2000hsh.googlegroups.com> Message-ID: At 6:59 PM -0800 2/27/08, Trans wrote: >On Feb 27, 1:43 pm, Calvin Bascom wrote: >> I ran into this as well installing the 0.5.3 gem on both CentOS and >> Fedora platforms. To get it working I had to rebuild the .so file >> manually after the gem install: >> > > cd /usr/local/lib/ruby/gems/1.8/gems/libxml-ruby-0.5.3 > > ruby xml/libxml/extconf.rb >> make >> make install >> make distclean >> >> The make install step copied the libxml_so.so file to the correct >> directory whereas the gem installation didn't for some reason. Don't >> know what causes this, but it at least worked for me after doing the above. > >I'm glad someone has tried that new gem out finally -- this is a >"beta" release (obviously). > >In contrast to setup.rb, the Gem is not suppose to copy the .so, >rather the loadpath should point to it. (At least that's what Eric >Hodel told me was the proper way). Interestingly, you used make >install to copy the .so over into the proper lib/ directory. > >I figured out the problem with the gem. In the gemspec, the extensions >field isn't getting set properly, so it can't find the extconf.rb. >Will fix shortly. I just installed the 0.5.3 gem on Leopard 10.5.2 yesterday and it still has the same problem -- but the solution described above doesn't work ... On my system the gem is located here: /Library/Ruby/Gems/1.8/gems/libxml-ruby-0.5.3 The location of extconf.rb is here: ./ext/libxml/extconf.rb Executing that file produces this error: $ ruby ext/libxml/extconf.rb checking for socket() in -lsocket... *** ext/libxml/extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. From transfire at gmail.com Wed Mar 26 07:30:32 2008 From: transfire at gmail.com (Trans) Date: Wed, 26 Mar 2008 07:30:32 -0400 Subject: [libxml-devel] xml/libxml won't load in OS X 10.5.2 Leopard In-Reply-To: References: <47C5AF40.2050505@revasystems.com> <94abb038-a936-406f-8223-fc18ea0eb66a@q33g2000hsh.googlegroups.com> Message-ID: <4b6f054f0803260430j34d65114mff213c116d45cd90@mail.gmail.com> On Wed, Mar 26, 2008 at 12:12 AM, Stephen Bannasch wrote: > At 6:59 PM -0800 2/27/08, Trans wrote: > >On Feb 27, 1:43 pm, Calvin Bascom wrote: > >> I ran into this as well installing the 0.5.3 gem on both CentOS and > >> Fedora platforms. To get it working I had to rebuild the .so file > >> manually after the gem install: > >> > > > cd /usr/local/lib/ruby/gems/1.8/gems/libxml-ruby-0.5.3 > > > ruby xml/libxml/extconf.rb > >> make > >> make install > >> make distclean > >> > >> The make install step copied the libxml_so.so file to the correct > >> directory whereas the gem installation didn't for some reason. Don't > >> know what causes this, but it at least worked for me after doing the above. > > > >I'm glad someone has tried that new gem out finally -- this is a > >"beta" release (obviously). > > > >In contrast to setup.rb, the Gem is not suppose to copy the .so, > >rather the loadpath should point to it. (At least that's what Eric > >Hodel told me was the proper way). Interestingly, you used make > >install to copy the .so over into the proper lib/ directory. > > > >I figured out the problem with the gem. In the gemspec, the extensions > >field isn't getting set properly, so it can't find the extconf.rb. > >Will fix shortly. > > I just installed the 0.5.3 gem on Leopard 10.5.2 yesterday and it still has the same problem -- but the solution described above doesn't work ... Just released 0.5.4 that should fix this problem. > On my system the gem is located here: > > /Library/Ruby/Gems/1.8/gems/libxml-ruby-0.5.3 > > The location of extconf.rb is here: > > ./ext/libxml/extconf.rb > > Executing that file produces this error: > > $ ruby ext/libxml/extconf.rb > checking for socket() in -lsocket... *** ext/libxml/extconf.rb failed *** > Could not create Makefile due to some reason, probably lack of > necessary libraries and/or headers. Check the mkmf.log file for more > details. You may need configuration options. Does the mkmf.log provide any clues? T. From luc at honk-honk.com Wed Mar 26 08:58:00 2008 From: luc at honk-honk.com (Luc Heinrich) Date: Wed, 26 Mar 2008 13:58:00 +0100 Subject: [libxml-devel] Version and library bugs Message-ID: <191022A3-1F00-4B19-9E2C-164A05927D45@honk-honk.com> Greetings, Just saw the mention of 0.5.4 in another post and thought I'd take a look. I noticed to obvious problems: 1. in ext/libxml/version.h, the version is still 0.5.2.2. 2. in lib/libxml.rb, XML::Attr#to_h is wrong: it should return the hash (to_a methods work because Array#<< returns the array itself, hash insertions return the inserted value). ## patch ############ --- lib/libxml.old.rb 2008-01-27 18:01:44.000000000 +0100 +++ lib/libxml.rb 2008-03-26 13:53:35.000000000 +0100 @@ -95,7 +95,7 @@ end def to_a - inject([]) do |ary,a| ary << [a.name, a.value] end + inject([]) do |ary,a| ary << [a.name, a.value]; h end end def to_s ## end of patch ##### -- Luc Heinrich - luc at honk-honk.com From transfire at gmail.com Wed Mar 26 10:02:08 2008 From: transfire at gmail.com (Trans) Date: Wed, 26 Mar 2008 07:02:08 -0700 (PDT) Subject: [libxml-devel] Version and library bugs In-Reply-To: <191022A3-1F00-4B19-9E2C-164A05927D45@honk-honk.com> References: <191022A3-1F00-4B19-9E2C-164A05927D45@honk-honk.com> Message-ID: On Mar 26, 8:58 am, Luc Heinrich wrote: > Greetings, > > Just saw the mention of 0.5.4 in another post and thought I'd take a > look. I noticed to obvious problems: > > 1. in ext/libxml/version.h, the version is still 0.5.2.2. Thanks for the report. I fixed this and just re-released 0.5.4. On the topic... I have forgotten... What is version.h being used for? Is it necessary? T. From luc at honk-honk.com Wed Mar 26 10:17:52 2008 From: luc at honk-honk.com (Luc Heinrich) Date: Wed, 26 Mar 2008 15:17:52 +0100 Subject: [libxml-devel] Version and library bugs In-Reply-To: References: <191022A3-1F00-4B19-9E2C-164A05927D45@honk-honk.com> Message-ID: On 26 mars 08, at 15:02, Trans wrote: > On the topic... I have forgotten... What is version.h being used for? It is used in ext/libxml/ruby_xml_parser.c to provide the XML::Parser::VERSION and XML::Parser::VERNUM constants. Why these constants are in the XML::Parser class and not in the XML module, that I don't know... :) -- Luc Heinrich - luc at honk-honk.com From sean at chittenden.org Wed Mar 26 11:19:55 2008 From: sean at chittenden.org (Sean Chittenden) Date: Wed, 26 Mar 2008 08:19:55 -0700 Subject: [libxml-devel] Version and library bugs In-Reply-To: References: <191022A3-1F00-4B19-9E2C-164A05927D45@honk-honk.com> Message-ID: <3F17CE5B-5717-4122-A985-5EB9DDB64C75@chittenden.org> >> On the topic... I have forgotten... What is version.h being used for? > > It is used in ext/libxml/ruby_xml_parser.c to provide the > XML::Parser::VERSION and XML::Parser::VERNUM constants. Why these > constants are in the XML::Parser class and not in the XML module, that > I don't know... :) Come to think of it, neither do I! ... If my feeble memory serves me less well than it used it, it's because I wrote ruby_xml_parser.c first and I used its namespace to stash those constants while I was doing the API/constants mappings. :~] -sc -- Sean Chittenden sean at chittenden.org http://sean.chittenden.org/ From stephen.bannasch at deanbrook.org Wed Mar 26 08:45:48 2008 From: stephen.bannasch at deanbrook.org (Stephen Bannasch) Date: Wed, 26 Mar 2008 08:45:48 -0400 Subject: [libxml-devel] xml/libxml won't load in OS X 10.5.2 Leopard In-Reply-To: <4b6f054f0803260430j34d65114mff213c116d45cd90@mail.gmail.com> References: <47C5AF40.2050505@revasystems.com> <94abb038-a936-406f-8223-fc18ea0eb66a@q33g2000hsh.googlegroups.com> <4b6f054f0803260430j34d65114mff213c116d45cd90@mail.gmail.com> Message-ID: At 7:30 AM -0400 3/26/08, Trans wrote: >Just released 0.5.4 that should fix this problem. > >> On my system the gem is located here: >> >> /Library/Ruby/Gems/1.8/gems/libxml-ruby-0.5.3 >> >> The location of extconf.rb is here: >> >> ./ext/libxml/extconf.rb >> > > Executing that file produces this error: >> >> $ ruby ext/libxml/extconf.rb >> checking for socket() in -lsocket... *** ext/libxml/extconf.rb failed *** >> Could not create Makefile due to some reason, probably lack of >> necessary libraries and/or headers. Check the mkmf.log file for more >> details. You may need configuration options. > >Does the mkmf.log provide any clues? Couldn't find one anywhere in: /Library/Ruby/Gems/1.8/gems/libxml-ruby-0.5.3 Version 0.5.4 installed and is working -- thanks! From luc at honk-honk.com Wed Mar 26 13:58:54 2008 From: luc at honk-honk.com (Luc Heinrich) Date: Wed, 26 Mar 2008 18:58:54 +0100 Subject: [libxml-devel] Version and library bugs In-Reply-To: <191022A3-1F00-4B19-9E2C-164A05927D45@honk-honk.com> References: <191022A3-1F00-4B19-9E2C-164A05927D45@honk-honk.com> Message-ID: On 26 mars 08, at 13:58, Luc Heinrich wrote: > 2. in lib/libxml.rb, XML::Attr#to_h is wrong: it should return the > hash (to_a methods work because Array#<< returns the array itself, > hash insertions return the inserted value). Er, sorry, the previous patch was completely bogus, this is the correct one: --- libxml.old.rb 2008-01-27 18:01:44.000000000 +0100 +++ libxml.rb 2008-03-26 18:57:56.000000000 +0100 @@ -91,7 +91,7 @@ alias :each :each_sibling def to_h - inject({}) do |h,a| h[a.name] = a.value end + inject({}) do |h,a| h[a.name] = a.value; h end end def to_a -- Luc Heinrich - luc at honk-honk.com From transfire at gmail.com Fri Mar 28 11:17:45 2008 From: transfire at gmail.com (Trans) Date: Fri, 28 Mar 2008 08:17:45 -0700 (PDT) Subject: [libxml-devel] Next release Message-ID: <6b25fac1-f446-4fdb-9cea-e2f125201088@8g2000hsu.googlegroups.com> Fixed two minor bugs since last release, a typo in extconf.rb and the #to_h in libxml.rb. I'm going to make a new release. But if anyone has any important patches they'd like to get in there first please bring them up now. T. From deckard1 at gmail.com Mon Mar 31 07:50:45 2008 From: deckard1 at gmail.com (Ben Lam) Date: Mon, 31 Mar 2008 12:50:45 +0100 Subject: [libxml-devel] xmlXPathFreeNodeSet segfault Message-ID: <6be2e9980803310450p3c296d03w612833d9593df40@mail.gmail.com> Hi All, I'm debugging a rails app that's using libxml-ruby to parse Amazon AWS xml documents. I'm getting a SegFault on calling GC.start after calling XML::Parser#parse on document, but this only occurs at some random time in the future, say after 350 or more iterations of the code. The juicy part of the gdb stack trace: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 46912503504000 (LWP 12822)] 0x00002aaaaebb9b72 in xmlXPathFreeNodeSet () from /usr/lib/libxml2.so.2 #0 0x00002aaaaebb9b72 in xmlXPathFreeNodeSet () from /usr/lib/libxml2.so.2 #1 0x00002aaaaebbae18 in xmlXPathFreeObject () from /usr/lib/libxml2.so.2 #2 0x000000000042c9b8 in garbage_collect () at gc.c:1209 --- snip --- And the offending code: xmlparser = XML::Parser.string(str) @doc = xmlparser.parse GC.start The reason for the GC.start right after the parse is that I found that Ruby was not cleaning up memory properly after XML::Parser had done its thing, leading mongrel to climb to glorious heights of memory usage. Adding this GC.start stopped cold the memory growth, but, after parsing about 350-360 documents over time, Ruby decides to take a vacation to SegFault land and never return. Am I asleep at the wheel here with that GC.start call? In order to reproduce this segfault, I'm using apache's ab to hammer my site, which is parsing the same Amazon XML document. I'm using libxml-ruby 0.5.3 with XML::Parser version 0.5.2.2 on an x86_64 build of ubuntu on 2.6.15-51-amd64-server kernel. Thanks, Ben Lam -- From seangeo at gmail.com Mon Mar 31 20:48:06 2008 From: seangeo at gmail.com (Sean Geoghegan) Date: Tue, 1 Apr 2008 11:18:06 +1030 Subject: [libxml-devel] Create a Node with a namespace Message-ID: Hi I have been trying to figure out how to programmatically create a Node with a namespace but without any luck. I have tried this: node = XML::Node.new('test') node.namespace = 'http://example.org' But that just results in a segfault. I have tried creating XML::NS objects but there doesn't seem to be a way to set any of it's attribute. The only way I can figure out how to do it is: node = XML::Node.new('test') node['xmlns'] = 'http://example.org' Which results in the following output: Which is valid, but it means that I either have to assign and manage all the namespace prefixes myself or I have to specify a new local namespace for each element in a different namespace in the document, which is a bit ugly. So, is there a better way? Sean From noreply at rubyforge.org Fri Mar 21 19:03:19 2008 From: noreply at rubyforge.org (noreply at rubyforge.org) Date: Fri, 21 Mar 2008 19:03:19 -0400 (EDT) Subject: [libxml-devel] [ libxml-Bugs-19022 ] 0.5.3 Gem fails on Macintosh with "LoadError: no such file to load -- libxml_so" Message-ID: <20080321230319.E46DB18586B3@rubyforge.org> Bugs item #19022, was opened at 2008-03-21 16:03 You can respond by visiting: http://rubyforge.org/tracker/?func=detail&atid=1971&aid=19022&group_id=494 Category: General Group: v0.5 Status: Open Resolution: None Priority: 3 Submitted By: Paul Legato (plegato) Assigned to: Nobody (None) Summary: 0.5.3 Gem fails on Macintosh with "LoadError: no such file to load -- libxml_so" Initial Comment: Attempting to load gem version 0.5.3 fails on my Mac with the following: LoadError: no such file to load -- libxml_so from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require' from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in `require' from /opt/local/lib/ruby/gems/1.8/gems/libxml-ruby-0.5.3/lib/libxml.rb:4 from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:32:in `gem_original_require' from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:32:in `require' from /opt/local/lib/ruby/gems/1.8/gems/libxml-ruby-0.5.3/lib/xml/libxml.rb:4 (Mac OS doesn't use .so files.) It works fine if I uninstall the 0.5.3 gem and force-revert to 0.5.2.0. ---------------------------------------------------------------------- You can respond by visiting: http://rubyforge.org/tracker/?func=detail&atid=1971&aid=19022&group_id=494