<div dir="ltr">Thank you so much Charlie.<div><br></div><div>I am for sure going to switch my code to stop abusing find and instead iterate over the children. Good catch.</div><div><br></div><div>If I can find some spare time I am going to give XMLReader a go and if I can work up the courage I will also try to patch the C code to free those objects.</div>
<div><br></div><div>Matt Margolis<br><br><div class="gmail_quote">2008/8/16 Charlie Savage <span dir="ltr"><<a href="mailto:cfis@savagexi.com">cfis@savagexi.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="Ih2E3d"><br>
<br>
Matthew Margolis wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I don't mind setting nodes = nil before calling GC.start (read some other threads so I think I understand why I have to do that) but I do mind the speed hit, so if you think there is a way around that I would love to know more. <br>
</blockquote>
<br></div>
It would all be in the C code. Something like:<br>
<br>
xml_document:<br>
* for each xpath_object returned, store a pointer to it (st_hash, key is a pointer, value is the object)<br>
* Update api to include register_xpath_object, unregister_xpath_object<br>
* When freed, iterate over xpath objects, and any left tell them to free their underlying C object (not the ruby object)<br>
<br>
xpath_object<br>
* On creation, call document.register_xpath_object<br>
* On freeding, call document.unregister_xpath_object<br>
* Add api, called by document, called free_xpath_object (or some such)<div class="Ih2E3d"><br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
My general calling pattern is 1. Document#find_first to get the most top level element I am interested in 2. top_level_element#find for each of its direct children. When I find each child I then recurse down and load that children's children.<br>
</blockquote>
<br></div>
No need to call find for that. Just iterate over the children directly - will be faster. node.children I think it is....<div class="Ih2E3d"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
So yes I am walking the entire tree which will create a bunch of objects. When only grabbing the top level element in my test program I am still seeing a big spike in memory. I looked at the XPath Object code and it looks to me like this case is the one I am going to match when trying to find the topmost element of interest.<br>
<br>
case XPATH_NODESET:<br>
rval = Data_Wrap_Struct(cXMLXPathObject,<br>
ruby_xml_xpath_object_mark,<br>
ruby_xml_xpath_object_free,<br>
xpop);<br>
<br>
I am not familiar with Data_Wrap_Struct(part of Ruby?) so I don't know if it could potentially create lots of objects.<br>
</blockquote>
<br></div>
Yes, this wraps just the return object. The xpop object looks like this:<br>
<br>
<a href="http://xmlsoft.org/XSLT/object.gif" target="_blank">http://xmlsoft.org/XSLT/object.gif</a><br>
<br>
Its iterating over xpop->nodesetval I was mentioning.<div class="Ih2E3d"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I will look at the XMLReader tests to try to get a better feel for if it will meet my needs. Thank you for the suggestion.<br>
</blockquote>
<br></div>
Sure - sounds like that will be your best bet. Seems perfect for what you need.<br><font color="#888888">
<br>
Charlie<br>
</font><br>_______________________________________________<br>
libxml-devel mailing list<br>
<a href="mailto:libxml-devel@rubyforge.org">libxml-devel@rubyforge.org</a><br>
<a href="http://rubyforge.org/mailman/listinfo/libxml-devel" target="_blank">http://rubyforge.org/mailman/listinfo/libxml-devel</a><br></blockquote></div><br></div></div>