From doug.arogos at gmail.com Mon Jul 2 14:13:50 2007 From: doug.arogos at gmail.com (Doug Smith) Date: Mon, 2 Jul 2007 11:13:50 -0700 Subject: [Ferret-talk] Strange intermittent no search results problem Message-ID: <42d8808f0707021113kcfb1426w827437938d88e1e9@mail.gmail.com> Hello, First the specs: ruby 1.8.6, rails 1.2.3, ferret 0.11.4, mongrel 1.0.1, mongrel_cluster 0.2.1 And the model's aaf config: # Ferret search engine acts_as_ferret({:fields => {:name => {:boost => 10}, :summary => {:boost => 2}, :body => {:boost => 1, :store => :no}, :question => {:boost => 1, :store => :no}, :answer => {:boost => 1, :store => :no}, :status_id => {:boost => 1}, :category_id => {:boost => 1}, :published => {:boost => 1}}, :store_class_name => true, :remote => true, :ferret => { :analyzer => Ferret::Analysis:: StandardAnalyzer.new([]) } } ) I've been wrestling with a very strange intermittent problem for a few days now. I'm running 3 mongrels behind an Apache 2.2 proxy balancer, and the aaf DRb server in production mode. I've been manually load testing a search where the query string contains a string, and also filter by fields category_id and status_id. I change the category or status and re-submit the search several times. After some random number of searches, if I send a search that should return some number of results, it will return no results on 1 or 2 of the mongrels, but the correct results on the remaining mongrel(s). So it's like this: refresh - no results, refresh - no results, refresh - 2 results, over & over in that pattern. Once this state has occurred, if I restart the DRb server, all will be well until I rapidly submit several more queries enough times to make this happen again. I have tested this both on our RedHat-based staging server and my local OSX dev machine with the same exact config, and I get the same results. Through logging I have confirmed that the same query is received by all the mongrels. To factor out browser-related issues, I've also tested by hitting the web server with a little looping ruby script, and it also breaks that way. I'm not sure where to debug from here -- can anyone point me in the right direction? Since a restart of the DRb server fixes the problem for a bit, I'm thinking it may be aaf related, but I don't know how to isolate this any further so as to determine the exact cause. Thanks, Doug -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070702/0767241b/attachment.html From kraemer at webit.de Tue Jul 3 03:22:10 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 3 Jul 2007 09:22:10 +0200 Subject: [Ferret-talk] Strange intermittent no search results problem In-Reply-To: <42d8808f0707021113kcfb1426w827437938d88e1e9@mail.gmail.com> References: <42d8808f0707021113kcfb1426w827437938d88e1e9@mail.gmail.com> Message-ID: <20070703072210.GB4305@cordoba.webit.de> On Mon, Jul 02, 2007 at 11:13:50AM -0700, Doug Smith wrote: [..] > > After some random number of searches, if I send a search that should return > some number of results, it will return no results on 1 or 2 of the mongrels, > but the correct results on the remaining mongrel(s). So it's like this: > refresh - no results, refresh - no results, refresh - 2 results, over & over > in that pattern. Once this state has occurred, if I restart the DRb server, > all will be well until I rapidly submit several more queries enough times to > make this happen again. [..] > > I'm not sure where to debug from here -- can anyone point me in the right > direction? Since a restart of the DRb server fixes the problem for a bit, > I'm thinking it may be aaf related, but I don't know how to isolate this any > further so as to determine the exact cause. Strange. The next step would be to find out if the query reaches the DRb server all the time, or if it gets lost somewhere in between. You should see the DRb method calls in ferret_server.log . Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From w.monk at hotmail.co.uk Tue Jul 3 08:29:15 2007 From: w.monk at hotmail.co.uk (William Monk) Date: Tue, 3 Jul 2007 14:29:15 +0200 Subject: [Ferret-talk] problems with acts_as_ferret Message-ID: <9735192a00b74bd8968e19763ecf35bc@ruby-forum.com> Hi, I have i am trying to add a search feature to a ruby on rails blog, so ive decided to use ferret. So far i have had quite a few problems with it, from following a few tutorials i didnt really understand... i am at the point where i can make a search and it returns the score of the result. I want it to also show the title of the post and i think i have implemented it correctly but it doesn't. This is the code in my search.rhtml: <% @results.each_with_index do |result, index| %> <%= result[:title] %> Score: <%= result[:score] %>

<% end %> And this is the controller: def search @query = params[:q] @total, @results = Post.find_storage_by_contents(@query, :page => (params[:page]||1)) @pages = pages_for(@total) end This is the code from post.rb: def self.find_storage_by_contents(query, options = {}) # Get the index that acts_as_ferret created for us index = self.aaf_index.ferret_index results = [] # search_each is the core search function from Ferret, which Acts_as_ferret hides total_hits = index.search_each(query, options) do |doc, score| result = {} # Store each field in a hash which we can reference in our views result[:title] = index[doc][:title] # We can even put the score in the hash, nice! result[:score] = score results.push result end return block_given? ? total_hits : [total_hits, results] end there is probably something i have missed before this is able to work, if u have any ideas please help! thanks, Will -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Tue Jul 3 09:32:55 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 3 Jul 2007 15:32:55 +0200 Subject: [Ferret-talk] problems with acts_as_ferret In-Reply-To: <9735192a00b74bd8968e19763ecf35bc@ruby-forum.com> References: <9735192a00b74bd8968e19763ecf35bc@ruby-forum.com> Message-ID: <20070703133255.GE4305@cordoba.webit.de> Hi! Why do you try doing this the hard way? There's really no reason to use Ferret directly for a blog search once you have acts_as_ferret in your Rails app. Here's the simplest possible example: Model: class Post acts_as_ferret :fields => { :title => {}, :content => {} } end Controller: def search @results = Post.find_by_contents(query) end View: For implementing paging across your result set, please check out http://www.railsenvy.com/2007/2/19/acts-as-ferret-tutorial#pagination Regarding your initial problem with missing titles - acts_as_ferret by default stores no attributes but the id of your records in the ferret index, to keep index size small. To get around this specify :store => :yes in the field options for your title field: acts_as_ferret :fields => { :title => { :store => :yes }, :content => {} } However as I said above, just go the easy way... Jens On Tue, Jul 03, 2007 at 02:29:15PM +0200, William Monk wrote: > Hi, > > I have i am trying to add a search feature to a ruby on rails blog, so > ive decided to use ferret. So far i have had quite a few problems with > it, from following a few tutorials i didnt really understand... i am at > the point where i can make a search and it returns the score of the > result. I want it to also show the title of the post and i think i have > implemented it correctly but it doesn't. > > This is the code in my search.rhtml: > > <% @results.each_with_index do |result, index| %> > <%= result[:title] %> > Score: <%= result[:score] %>

> <% end %> > > And this is the controller: > > def search > @query = params[:q] > @total, @results = Post.find_storage_by_contents(@query, :page => > (params[:page]||1)) > @pages = pages_for(@total) > end > > This is the code from post.rb: > > def self.find_storage_by_contents(query, options = {}) > # Get the index that acts_as_ferret created for us > index = self.aaf_index.ferret_index > results = [] > > # search_each is the core search function from Ferret, which > Acts_as_ferret hides > total_hits = index.search_each(query, options) do |doc, score| > result = {} > > # Store each field in a hash which we can reference in our views > result[:title] = index[doc][:title] > > > # We can even put the score in the hash, nice! > result[:score] = score > > results.push result > end > return block_given? ? total_hits : [total_hits, results] > end > > there is probably something i have missed before this is able to work, > if u have any ideas please help! thanks, Will > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From w.monk at hotmail.co.uk Tue Jul 3 10:12:09 2007 From: w.monk at hotmail.co.uk (William Monk) Date: Tue, 3 Jul 2007 16:12:09 +0200 Subject: [Ferret-talk] problems with acts_as_ferret In-Reply-To: <20070703133255.GE4305@cordoba.webit.de> References: <9735192a00b74bd8968e19763ecf35bc@ruby-forum.com> <20070703133255.GE4305@cordoba.webit.de> Message-ID: <5582a5cd8675e4843c869303066e995a@ruby-forum.com> Hi, Thanks for your really fast reply. I am quite new to rails and couldnt find anything which was pointing me in the right direction apart froms some complicated tutorials. I have done what you have said, but i am not sure what the form should look like to give results, this is what i am using at the moment, plus everything you told me: but i get an error: TypeError in BlogController#search wrong argument type Symbol (expected Data) im not sure if this is to do with the posts being indexed incorectly, but if you could help again that would be great! Thanks Will -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Tue Jul 3 11:35:13 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 3 Jul 2007 17:35:13 +0200 Subject: [Ferret-talk] problems with acts_as_ferret In-Reply-To: <5582a5cd8675e4843c869303066e995a@ruby-forum.com> References: <9735192a00b74bd8968e19763ecf35bc@ruby-forum.com> <20070703133255.GE4305@cordoba.webit.de> <5582a5cd8675e4843c869303066e995a@ruby-forum.com> Message-ID: <20070703153513.GF4305@cordoba.webit.de> On Tue, Jul 03, 2007 at 04:12:09PM +0200, William Monk wrote: > > Hi, > > Thanks for your really fast reply. I am quite new to rails and couldnt > find anything which was pointing me in the right direction apart froms > some complicated tutorials. I have done what you have said, but i am not > sure what the form should look like to give results, this is what i am > using at the moment, plus everything you told me: > > > > > > but i get an error: > > TypeError in BlogController#search > > wrong argument type Symbol (expected Data) > > im not sure if this is to do with the posts being indexed incorectly, > but if you could help again that would be great! Thanks Will I had a small typo in the controller, it should read find_by_contents(params[:query]) Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From doug.arogos at gmail.com Tue Jul 3 11:59:45 2007 From: doug.arogos at gmail.com (Doug Smith) Date: Tue, 3 Jul 2007 08:59:45 -0700 Subject: [Ferret-talk] Strange intermittent no search results problem In-Reply-To: <20070703072210.GB4305@cordoba.webit.de> References: <42d8808f0707021113kcfb1426w827437938d88e1e9@mail.gmail.com> <20070703072210.GB4305@cordoba.webit.de> Message-ID: <42d8808f0707030859q57e8f3d4i589e9d3e68b263b8@mail.gmail.com> Hi Jens, I really appreciate your excellent support of this plugin. On 7/3/07, Jens Kraemer wrote: > [...] > Strange. The next step would be to find out if the query reaches the DRb > server all the time, or if it gets lost somewhere in between. You should > see the DRb method calls in ferret_server.log . Yes, the query does always reach the DRb server. Here is a portion of the ferret_server.log with my comments prefixed with ###DAS: ###DAS: Works fine #method_missing(:find_id_by_contents, ["Article", "a* category_id:17 status_id:1", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "a* category_id:17 status_id:1", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "a* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "a* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "a* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "a* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17 status_id:100", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17 status_id:100", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17 status_id:100", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17 status_id:100", {:limit=>10, :offset=>0}]) ### DAS: This query returns no results, correctly #method_missing(:find_id_by_contents, ["Article", "* category_id:17 status_id:101", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17 status_id:101", {:limit=>10, :offset=>0}]) ### DAS: Begins cycle -- first query returns 2 results correctly, next two return nothing (3 mongrels) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) #method_missing(:find_id_by_contents, ["Article", "* category_id:17", {:limit=>10, :offset=>0}]) It does seem to only fail when doing a complete wildcard search, filtered by a field, after switching back & forth on another field a time or two. Very weird. Thanks again, Doug From kraemer at webit.de Tue Jul 3 12:20:58 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 3 Jul 2007 18:20:58 +0200 Subject: [Ferret-talk] Strange intermittent no search results problem In-Reply-To: <42d8808f0707030859q57e8f3d4i589e9d3e68b263b8@mail.gmail.com> References: <42d8808f0707021113kcfb1426w827437938d88e1e9@mail.gmail.com> <20070703072210.GB4305@cordoba.webit.de> <42d8808f0707030859q57e8f3d4i589e9d3e68b263b8@mail.gmail.com> Message-ID: <20070703162058.GG4305@cordoba.webit.de> On Tue, Jul 03, 2007 at 08:59:45AM -0700, Doug Smith wrote: > Hi Jens, > > I really appreciate your excellent support of this plugin. :-) [..] > It does seem to only fail when doing a complete wildcard search, > filtered by a field, after switching back & forth on another field a > time or two. You could try that failing sequence of queries without the DRb server, and/or with only one mongrel instance or from the console, or even with a plain Ferret::I instance to find out if aaf's DRb stuff, aaf itself, or Ferret itself cause that, and if it's multi-threading-related. > Very weird. Indeed. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From hakita at gmail.com Tue Jul 3 12:28:37 2007 From: hakita at gmail.com (Pierre-Yves Landanger) Date: Tue, 3 Jul 2007 18:28:37 +0200 Subject: [Ferret-talk] Highlight slowness In-Reply-To: References: Message-ID: Paul Lynch wrote: > Has anyone else found that using ferret's highlighting slows searches > down significantly? I am seeing that it more than doubles the search > time on my system. I am returning up to 500 results at once, so the > slow down is quite noticeable (probably adding about .7 seconds for > searches with large result sets.) > --Paul Hello Paul, I got the same problem. highlighting performance are based on the tolerance you selected. Example for me : QUERY~1 is fast. < 10 ms QUERY~0.9 is slower. < 50 ms QUERY~0.8 is slow. = 1 sec QUERY~0.7 is very slow. > 2 sec ... If anyone have a solution to speed it up, i would appreciate too. -- Posted via http://www.ruby-forum.com/. From hakita at gmail.com Tue Jul 3 13:02:26 2007 From: hakita at gmail.com (Landanger Pierre-Yves) Date: Tue, 3 Jul 2007 19:02:26 +0200 Subject: [Ferret-talk] Highlight slowness In-Reply-To: References: Message-ID: <16ce626110eae532356cfcad72a9da33@ruby-forum.com> Here how i do highlighting in my model : acts_as_ferret :fields => { :content => {:store => :yes}, :title => {:store => :yes}} results = list.find_by_contents("QUERY~7", options) In my view : <% results[0].highlight("QUERY~7", :field => :content,... ) -%> Is there a way to make it faster without changing the tolerance factor ? ( ~7 in this example ) -- Posted via http://www.ruby-forum.com/. From doug.arogos at gmail.com Tue Jul 3 14:02:37 2007 From: doug.arogos at gmail.com (Doug Smith) Date: Tue, 3 Jul 2007 11:02:37 -0700 Subject: [Ferret-talk] Strange intermittent no search results problem In-Reply-To: <20070703162058.GG4305@cordoba.webit.de> References: <42d8808f0707021113kcfb1426w827437938d88e1e9@mail.gmail.com> <20070703072210.GB4305@cordoba.webit.de> <42d8808f0707030859q57e8f3d4i589e9d3e68b263b8@mail.gmail.com> <20070703162058.GG4305@cordoba.webit.de> Message-ID: <42d8808f0707031102n71c918a0g13335f75bcdca0ee@mail.gmail.com> On 7/3/07, Jens Kraemer wrote: > On Tue, Jul 03, 2007 at 08:59:45AM -0700, Doug Smith wrote: > > It does seem to only fail when doing a complete wildcard search, > > filtered by a field, after switching back & forth on another field a > > time or two. > > You could try that failing sequence of queries without the DRb server, > and/or with only one mongrel instance or from the console, or even with > a plain Ferret::I instance to find out if aaf's DRb stuff, aaf itself, > or Ferret itself cause that, and if it's multi-threading-related. Ok, I wrote a test script, and between every test I stopped apache, mongrel, ferret, and deleted the indexes to make sure each one was clean. Here are the results: With DRb, 3 mongrels => fails With DRb, 2 mongrels => fails With DRb, 1 mongrel => passes Without DRb, 3 mongrels => passes So, it's the combination of DRb and multiple mongrels that always fails. Let me know how else I can test to help debug this. Thanks, Doug From doug.arogos at gmail.com Tue Jul 3 14:34:47 2007 From: doug.arogos at gmail.com (Doug Smith) Date: Tue, 3 Jul 2007 11:34:47 -0700 Subject: [Ferret-talk] Strange intermittent no search results problem In-Reply-To: <42d8808f0707031102n71c918a0g13335f75bcdca0ee@mail.gmail.com> References: <42d8808f0707021113kcfb1426w827437938d88e1e9@mail.gmail.com> <20070703072210.GB4305@cordoba.webit.de> <42d8808f0707030859q57e8f3d4i589e9d3e68b263b8@mail.gmail.com> <20070703162058.GG4305@cordoba.webit.de> <42d8808f0707031102n71c918a0g13335f75bcdca0ee@mail.gmail.com> Message-ID: <42d8808f0707031134qcba5eb3i861c0e16e9ce989@mail.gmail.com> On 7/3/07, Doug Smith wrote: > Ok, I wrote a test script, and between every test I stopped apache, > mongrel, ferret, and deleted the indexes to make sure each one was > clean. Here are the results: > > With DRb, 3 mongrels => fails > With DRb, 2 mongrels => fails > With DRb, 1 mongrel => passes > Without DRb, 3 mongrels => passes And one more factor: If I change this line in my controller from this: results = Article.find_by_contents(q, options) to this: results = Article.multi_search(q, [], options) It never fails, always passes under all conditions. The reason I found this is that a previous project that is in production won't fail, but it uses multi_search instead of find_by_contents like I'm using now. Thanks, Doug From miguel.carvalho at excentric.pt Tue Jul 3 15:25:53 2007 From: miguel.carvalho at excentric.pt (Miguel Carvalho) Date: Tue, 3 Jul 2007 21:25:53 +0200 Subject: [Ferret-talk] How index works! Message-ID: <5019c7c35cef6b165d3883cbfa3dca43@ruby-forum.com> Hi, i've a project in wich i have 2 different rails apps accessing the same DB. The backoffice, as usual, changes data. The frontoffice has a search capabilities with acts_as_ferret (paginated) for search. Maybe this is a newbie question but, when i delete index and restart front app all the articles are indexed, but the new one's (via backoffice) are not searchable. Does acts_as_ferret indexes on change ops, like insert,delete, update? Is that the reason why i can't see the new ones? Thanks in advance, Miguel -- Posted via http://www.ruby-forum.com/. From w.monk at hotmail.co.uk Tue Jul 3 19:23:01 2007 From: w.monk at hotmail.co.uk (William Monk) Date: Wed, 4 Jul 2007 01:23:01 +0200 Subject: [Ferret-talk] problems with acts_as_ferret In-Reply-To: <20070703153513.GF4305@cordoba.webit.de> References: <9735192a00b74bd8968e19763ecf35bc@ruby-forum.com> <20070703133255.GE4305@cordoba.webit.de> <5582a5cd8675e4843c869303066e995a@ruby-forum.com> <20070703153513.GF4305@cordoba.webit.de> Message-ID: Jens Kraemer wrote: > I had a small typo in the controller, it should read > > find_by_contents(params[:query]) Thanks a lot Jens, you have been a massive help for getting search to work, and now it works perfectly. Thanks! Will -- Posted via http://www.ruby-forum.com/. From wmorgan-ferret at masanjin.net Tue Jul 3 22:40:46 2007 From: wmorgan-ferret at masanjin.net (William Morgan) Date: Tue, 03 Jul 2007 19:40:46 -0700 Subject: [Ferret-talk] error at index.c:901 Message-ID: <1183516795-sup-3601@south> Hi Dave & co, I've got a couple Sup users who encounter the following error fairly consistently, even with 0.11.4. After they see this, the index is not recoverable and they have to rebuild from scratch. Any ideas? --- cut here --- /var/lib/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:726:in `initialize': IO Error occured at :93 in xraise (IOError) Error occured in index.c:901 - sis_find_segments_file Error reading the segment infos. Store listing was from /var/lib/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:726:in `new' from /var/lib/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:726:in `ensure_reader_open' from /var/lib/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:733:in `ensure_searcher_open' from /var/lib/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:755:in `do_search' from /var/lib/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:334:in `search' from /usr/lib/ruby/1.8/monitor.rb:238:in `synchronize' from /var/lib/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:333:in `search' from ./lib/sup/index.rb:312:in `load_entry_for_id' ... 33 levels... --- cut here --- Thanks! -- William From kraemer at webit.de Wed Jul 4 03:30:25 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 4 Jul 2007 09:30:25 +0200 Subject: [Ferret-talk] How index works! In-Reply-To: <5019c7c35cef6b165d3883cbfa3dca43@ruby-forum.com> References: <5019c7c35cef6b165d3883cbfa3dca43@ruby-forum.com> Message-ID: <20070704073025.GH4305@cordoba.webit.de> On Tue, Jul 03, 2007 at 09:25:53PM +0200, Miguel Carvalho wrote: > Hi, i've a project in wich i have 2 different rails apps accessing the > same DB. > > The backoffice, as usual, changes data. > > The frontoffice has a search capabilities with acts_as_ferret > (paginated) for search. > > Maybe this is a newbie question but, when i delete index and restart > front app all the articles are indexed, but the new one's (via > backoffice) are not searchable. > > Does acts_as_ferret indexes on change ops, like insert,delete, update? > Is that the reason why i can't see the new ones? Exactly. You should use aaf in your backoffice application, too. Keep in mind to use the same acts_as_ferret settings (fields, analyzer and such) as you do in your frontend app. And use the DRb server to avoid collisions when both applications want to access the index. In the end it doesn't matter where you run your DRb server (frontend or backoffice), but to me it seems good to keep the index in the backoffice app since that's where the data is created/changed. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Wed Jul 4 03:41:37 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 4 Jul 2007 09:41:37 +0200 Subject: [Ferret-talk] Strange intermittent no search results problem In-Reply-To: <42d8808f0707031134qcba5eb3i861c0e16e9ce989@mail.gmail.com> References: <42d8808f0707021113kcfb1426w827437938d88e1e9@mail.gmail.com> <20070703072210.GB4305@cordoba.webit.de> <42d8808f0707030859q57e8f3d4i589e9d3e68b263b8@mail.gmail.com> <20070703162058.GG4305@cordoba.webit.de> <42d8808f0707031102n71c918a0g13335f75bcdca0ee@mail.gmail.com> <42d8808f0707031134qcba5eb3i861c0e16e9ce989@mail.gmail.com> Message-ID: <20070704074137.GI4305@cordoba.webit.de> On Tue, Jul 03, 2007 at 11:34:47AM -0700, Doug Smith wrote: > On 7/3/07, Doug Smith wrote: > > Ok, I wrote a test script, and between every test I stopped apache, > > mongrel, ferret, and deleted the indexes to make sure each one was > > clean. Here are the results: > > > > With DRb, 3 mongrels => fails > > With DRb, 2 mongrels => fails > > With DRb, 1 mongrel => passes > > Without DRb, 3 mongrels => passes > > And one more factor: > > If I change this line in my controller from this: > > results = Article.find_by_contents(q, options) > > to this: > > results = Article.multi_search(q, [], options) > > It never fails, always passes under all conditions. ok, I guess now it's up to me to find the bug ;-) Using multi_search as a workaround should have no side effects. I filed a ticket so I won't forget - http://projects.jkraemer.net/acts_as_ferret/ticket/151 Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Wed Jul 4 03:50:05 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 4 Jul 2007 09:50:05 +0200 Subject: [Ferret-talk] Highlight slowness In-Reply-To: <16ce626110eae532356cfcad72a9da33@ruby-forum.com> References: <16ce626110eae532356cfcad72a9da33@ruby-forum.com> Message-ID: <20070704075005.GJ4305@cordoba.webit.de> On Tue, Jul 03, 2007 at 07:02:26PM +0200, Landanger Pierre-Yves wrote: > Here how i do highlighting > > > in my model : > acts_as_ferret :fields => { :content => {:store => :yes}, :title => > {:store => :yes}} > > results = list.find_by_contents("QUERY~7", options) > > > In my view : > <% results[0].highlight("QUERY~7", :field => :content,... ) -%> > > Is there a way to make it faster without changing the tolerance factor ? > ( ~7 in this example ) I could imagine that doing the highlighting right inside find_by_contents could speed up things quite a bit, maybe something like: results = list.find_by_contents("QUERY~7", :highlight => { :field => :content }) I'd happily apply a patch for this, or just file it as a feature request in aaf's Trac ;-) Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From miguel.carvalho at excentric.pt Wed Jul 4 05:46:14 2007 From: miguel.carvalho at excentric.pt (Miguel Carvalho) Date: Wed, 4 Jul 2007 11:46:14 +0200 Subject: [Ferret-talk] How index works! In-Reply-To: <20070704073025.GH4305@cordoba.webit.de> References: <5019c7c35cef6b165d3883cbfa3dca43@ruby-forum.com> <20070704073025.GH4305@cordoba.webit.de> Message-ID: Hum, tks. Anyway, it's not desirable for us (for now) to have that implemented in the backoffice. Is there a way of force reindex, for isntance, in a rails cron? Regards, Miguel Jens Kraemer wrote: > On Tue, Jul 03, 2007 at 09:25:53PM +0200, Miguel Carvalho wrote: >> backoffice) are not searchable. >> >> Does acts_as_ferret indexes on change ops, like insert,delete, update? >> Is that the reason why i can't see the new ones? > > Exactly. You should use aaf in your backoffice application, too. Keep in > mind to use the same acts_as_ferret settings (fields, analyzer and such) > as you do in your frontend app. And use the DRb server to avoid > collisions when both applications want to access the index. > > In the end it doesn't matter where you run your DRb server (frontend or > backoffice), but to me it seems good to keep the index in the backoffice > app since that's where the data is created/changed. > > Jens > > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Jul 4 06:00:14 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 4 Jul 2007 12:00:14 +0200 Subject: [Ferret-talk] How index works! In-Reply-To: References: <5019c7c35cef6b165d3883cbfa3dca43@ruby-forum.com> <20070704073025.GH4305@cordoba.webit.de> Message-ID: <20070704100014.GK4305@cordoba.webit.de> On Wed, Jul 04, 2007 at 11:46:14AM +0200, Miguel Carvalho wrote: > Hum, tks. > > Anyway, it's not desirable for us (for now) to have that implemented in > the backoffice. Is there a way of force reindex, for isntance, in a > rails cron? of course, just call Model.rebuild_index Note that unless you use the DRb server this will truncate the index used for searching and rebuilt it in place. With the DRb server the new index is built in the background so searches won't be affected. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From miguel.carvalho at excentric.pt Wed Jul 4 06:10:30 2007 From: miguel.carvalho at excentric.pt (Miguel Carvalho) Date: Wed, 4 Jul 2007 12:10:30 +0200 Subject: [Ferret-talk] How index works! In-Reply-To: <20070704100014.GK4305@cordoba.webit.de> References: <5019c7c35cef6b165d3883cbfa3dca43@ruby-forum.com> <20070704073025.GH4305@cordoba.webit.de> <20070704100014.GK4305@cordoba.webit.de> Message-ID: <006498498fb2b9dde29979503e635264@ruby-forum.com> Ok, tks. I've not configured or controlled the server in anyway. Just installed the gems as in http://www.railsenvy.com/2007/2/19/acts-as-ferret-tutorial . I believe you're talking about controlling the server as in http://projects.jkraemer.net/acts_as_ferret/wiki/DrbServer . I'm not sure of the differences. Can you direct me to an article that explains that? Thanks, Miguel Jens Kraemer wrote: > On Wed, Jul 04, 2007 at 11:46:14AM +0200, Miguel Carvalho wrote: >> Hum, tks. >> >> Anyway, it's not desirable for us (for now) to have that implemented in >> the backoffice. Is there a way of force reindex, for isntance, in a >> rails cron? > > of course, just call Model.rebuild_index > Note that unless you use the DRb server this will truncate the index > used for searching and rebuilt it in place. With the DRb server the new > index is built in the background so searches won't be affected. > > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From david.wennergren at gmail.com Wed Jul 4 08:05:01 2007 From: david.wennergren at gmail.com (David Wennergren) Date: Wed, 4 Jul 2007 14:05:01 +0200 Subject: [Ferret-talk] Inconsistent results when using wild card queries Message-ID: We get some unexpected results when using wild card queries. We're using aaf and Ferret 0.11.4 For exampel, when seraching on a part of a collegues name (kristofer) and limiting it to a specific source_id: Query: source_id:25 AND kri* Result: 2 documents. None of them containg the word kristofer, but other matching words, as "kring" and "kring?" (swedish) Query: source_id:25 AND kris* Result: 0 documents. Query: source_id:25 AND krist* Result: 12 document. Works as expected. The index contains in total about 200 000 documents and I've tried rebuilding and optimizing with no result. Has anyone else experienced something similar? Any ideas how to fix it? Thanks! /David Wennergren -- Posted via http://www.ruby-forum.com/. From brodaigh at gmail.com Wed Jul 4 09:19:29 2007 From: brodaigh at gmail.com (crissy crissy) Date: Wed, 4 Jul 2007 15:19:29 +0200 Subject: [Ferret-talk] problems after gem update Message-ID: <545733c44dddc5c71f5abd7320b2f8fe@ruby-forum.com> Hi, I had ferret working wonderfully but i am regretting doing a gem update today. I had alot of trouble first installing it and was really happy to have it working. The upgrade was from 0.3.1 to 0.4.0. The problem is; in the rails console, >> @results = Book.find_by_contents("peter pan") => # Its indexing only null values! and in the app(here's the stack), ArgumentError in SearchController#search wrong number of arguments (1 for 0) RAILS_ROOT: C:/INSTAN~1/rails_apps/clean/config/.. Application Trace | Framework Trace | Full Trace C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:220:in `ferret_enabled?' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:220:in `reindex_model' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:219:in `each' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:219:in `reindex_model' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:217:in `step' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:217:in `reindex_model' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/abstract/database_statements.rb:59:in `transaction' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/transactions.rb:95:in `transaction' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:216:in `reindex_model' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:47:in `rebuild_index' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:46:in `each' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:46:in `rebuild_index' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:23:in `ensure_index_exists' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/local_index.rb:9:in `initialize' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/class_methods.rb:233:in `new' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/class_methods.rb:233:in `create_index_instance' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/class_methods.rb:25:in `aaf_index' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/class_methods.rb:82:in `find_id_by_contents' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/class_methods.rb:134:in `ar_find_by_contents' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/class_methods.rb:128:in `find_records_lazy_or_not' C:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.0/lib/class_methods.rb:54:in `find_by_contents' #{RAILS_ROOT}/app/models/book.rb:100:in `full_text_search' #{RAILS_ROOT}/app/controllers/search_controller.rb:12:in `search' Help very much appreciated) -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Jul 4 09:32:27 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 4 Jul 2007 15:32:27 +0200 Subject: [Ferret-talk] problems after gem update In-Reply-To: <545733c44dddc5c71f5abd7320b2f8fe@ruby-forum.com> References: <545733c44dddc5c71f5abd7320b2f8fe@ruby-forum.com> Message-ID: <20070704133227.GL4305@cordoba.webit.de> On Wed, Jul 04, 2007 at 03:19:29PM +0200, crissy crissy wrote: > Hi, I had ferret working wonderfully but i am regretting doing a gem > update today. > I had alot of trouble first installing it and was really happy to have > it working. The upgrade was from 0.3.1 to 0.4.0. So you're using acts_as_Ferret 0.4.0 now. What version of Ferret are you running? I'd suggest 0.11.x . Rebuilding your index might help also. How do the pieces of code where you use acts_as_ferret look like? I'm not sure but there may have been some API changes from 0.3.1 to 0.4 . cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From brodaigh at gmail.com Wed Jul 4 10:17:54 2007 From: brodaigh at gmail.com (crissy crissy) Date: Wed, 4 Jul 2007 16:17:54 +0200 Subject: [Ferret-talk] problems after gem update In-Reply-To: <20070704133227.GL4305@cordoba.webit.de> References: <545733c44dddc5c71f5abd7320b2f8fe@ruby-forum.com> <20070704133227.GL4305@cordoba.webit.de> Message-ID: <571a7635e95c80dc05be896a4feaeaac@ruby-forum.com> Jens Kraemer wrote: > On Wed, Jul 04, 2007 at 03:19:29PM +0200, crissy crissy wrote: >> Hi, I had ferret working wonderfully but i am regretting doing a gem >> update today. >> I had alot of trouble first installing it and was really happy to have >> it working. The upgrade was from 0.3.1 to 0.4.0. > > So you're using acts_as_Ferret 0.4.0 now. What version of Ferret are you > running? I'd suggest 0.11.x . > > Rebuilding your index might help also. How do the pieces of code where > you use acts_as_ferret look like? I'm not sure but there may have been > some API changes from 0.3.1 to 0.4 . > > cheers, > Jens > > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa Hi Jens, Thankyou for your reply. I'm running ferret 0.11.4 The code is just your stock standard AAF stuff(mostly stolen from the railenvy tutorial;-) def search @query = params[:query] if @query.blank? flash[:error] = "Query was blank" redirect_to(:action => 'index') else @total, @books = Book.full_text_search(@query, :page => (params[:page]||1)) @pages = pages_for(@total) render_cloud @total_hits = Book.total_hits(@query) end end and in the Book Model; def self.full_text_search(q, options = {}) return nil if q.nil? or q=="" default_options = {:limit => 10, :page => 1} options = default_options.merge options # get the offset based on what page we're on options[:offset] = options[:limit] * (options.delete(:page).to_i-1) # now do the query with our options results = Book.find_by_contents(q, options)#, :conditions => conditions) return [results.total_hits, results] end I'm at a loss as to what to do. -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Thu Jul 5 03:35:41 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 5 Jul 2007 09:35:41 +0200 Subject: [Ferret-talk] How index works! In-Reply-To: <006498498fb2b9dde29979503e635264@ruby-forum.com> References: <5019c7c35cef6b165d3883cbfa3dca43@ruby-forum.com> <20070704073025.GH4305@cordoba.webit.de> <20070704100014.GK4305@cordoba.webit.de> <006498498fb2b9dde29979503e635264@ruby-forum.com> Message-ID: <20070705073541.GM4305@cordoba.webit.de> On Wed, Jul 04, 2007 at 12:10:30PM +0200, Miguel Carvalho wrote: > Ok, tks. > > I've not configured or controlled the server in anyway. Just installed > the gems as in > http://www.railsenvy.com/2007/2/19/acts-as-ferret-tutorial . > > I believe you're talking about controlling the server as in > http://projects.jkraemer.net/acts_as_ferret/wiki/DrbServer . > > I'm not sure of the differences. > > Can you direct me to an article that explains that? http://projects.jkraemer.net/acts_as_ferret/wiki/DrbServer pretty much explains this, I thought ;-) Basically the DRb server serves as a central resource all index access (search and updates) goes through. It therefore avoids locking and index corruption problems that may occur when multiple processes try to update the index at the same time (this may be the case if your live setup consists of more than one mongrel or fastcgi handler). The DRb server is part of acts_as_ferret, and you may choose to use it or not depending on RAILS_ENV, i.e. specify only a config for the production env in config/ferret_server.yml so aaf won't need the server in dev and test environments, but only in production. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From levent at leventali.com Thu Jul 5 04:45:25 2007 From: levent at leventali.com (Levent) Date: Thu, 5 Jul 2007 10:45:25 +0200 Subject: [Ferret-talk] Problem with setting up remote indexing In-Reply-To: <9665a881ac13df4843a68e1a9e059cf1@ruby-forum.com> References: <54d4de205945b182657d5ad14682ad65@ruby-forum.com> <20070331084445.GA10093@cordoba.webit.de> <20070401110859.GD14587@cordoba.webit.de> <50839b5a986b917ebebb5d802d57331f@ruby-forum.com> <20070402114545.GI10734@cordoba.webit.de> <9665a881ac13df4843a68e1a9e059cf1@ruby-forum.com> Message-ID: <34a87533486e81b3b1e6c218a0b2845f@ruby-forum.com> Harman Sandjaja wrote: >> ok, this is fixed now in trunk, too. > > Yup I updated already. > > Thanks again! =) When will this be released? cheers -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Thu Jul 5 04:55:20 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 5 Jul 2007 10:55:20 +0200 Subject: [Ferret-talk] Problem with setting up remote indexing In-Reply-To: <34a87533486e81b3b1e6c218a0b2845f@ruby-forum.com> References: <54d4de205945b182657d5ad14682ad65@ruby-forum.com> <20070331084445.GA10093@cordoba.webit.de> <20070401110859.GD14587@cordoba.webit.de> <50839b5a986b917ebebb5d802d57331f@ruby-forum.com> <20070402114545.GI10734@cordoba.webit.de> <9665a881ac13df4843a68e1a9e059cf1@ruby-forum.com> <34a87533486e81b3b1e6c218a0b2845f@ruby-forum.com> Message-ID: <20070705085520.GA11808@cordoba.webit.de> On Thu, Jul 05, 2007 at 10:45:25AM +0200, Levent wrote: > Harman Sandjaja wrote: > >> ok, this is fixed now in trunk, too. > > > > Yup I updated already. > > > > Thanks again! =) > > When will this be released? I hope to release 0.4.1 by sunday. jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From syrius.ml at no-log.org Thu Jul 5 05:05:36 2007 From: syrius.ml at no-log.org (syrius.ml at no-log.org) Date: Thu, 05 Jul 2007 11:05:36 +0200 Subject: [Ferret-talk] Inconsistent results when using wild card queries In-Reply-To: (David Wennergren's message of "Wed, 4 Jul 2007 14:05:01 +0200") References: Message-ID: <87r6nngred.87ps37gred@87odirgred.message.id> David Wennergren writes: Hi, > Has anyone else experienced something similar? Any ideas how to fix it? Unfortunatly i've also experienced that kind of weirdness. And most of the time it as to do with accentuation. i'm unable to match a single ? if I search for *?* (while it works with wordwith?) If i search for e it highlights single e, but it doesn't for single a... Sorry to say that, but at the moment I'm considering using another search enigne. (since I also have very weird unresolved issues with highlighting) I'm looking at xapian at the moment. -- From miguel.carvalho at excentric.pt Thu Jul 5 05:34:53 2007 From: miguel.carvalho at excentric.pt (Miguel Carvalho) Date: Thu, 5 Jul 2007 11:34:53 +0200 Subject: [Ferret-talk] How index works! In-Reply-To: <20070705073541.GM4305@cordoba.webit.de> References: <5019c7c35cef6b165d3883cbfa3dca43@ruby-forum.com> <20070704073025.GH4305@cordoba.webit.de> <20070704100014.GK4305@cordoba.webit.de> <006498498fb2b9dde29979503e635264@ruby-forum.com> <20070705073541.GM4305@cordoba.webit.de> Message-ID: <0fa3d93f868d8069b6b2999c3e04480a@ruby-forum.com> Jens, thanks a lot for your help. Best Regards, Miguel Jens Kraemer wrote: > On Wed, Jul 04, 2007 at 12:10:30PM +0200, Miguel Carvalho wrote: >> >> Can you direct me to an article that explains that? > > http://projects.jkraemer.net/acts_as_ferret/wiki/DrbServer pretty much > explains this, I thought ;-) > > Basically the DRb server serves as a central resource all index access > (search and updates) goes through. It therefore avoids locking and index > corruption problems that may occur when multiple processes try to update > the index at the same time (this may be the case if your live setup > consists of more than one mongrel or fastcgi handler). > > The DRb server is part of acts_as_ferret, and you may choose to use it > or not depending on RAILS_ENV, i.e. specify only a config for the > production env in config/ferret_server.yml so aaf won't need the server > in dev and test environments, but only in production. > > > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Thu Jul 5 05:43:15 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 5 Jul 2007 11:43:15 +0200 Subject: [Ferret-talk] Inconsistent results when using wild card queries In-Reply-To: <87r6nngred.87ps37gred@87odirgred.message.id> References: <87r6nngred.87ps37gred@87odirgred.message.id> Message-ID: <20070705094315.GE11808@cordoba.webit.de> On Thu, Jul 05, 2007 at 11:05:36AM +0200, syrius.ml at no-log.org wrote: > David Wennergren writes: > > Hi, > > > Has anyone else experienced something similar? Any ideas how to fix it? > > Unfortunatly i've also experienced that kind of weirdness. And most of > the time it as to do with accentuation. > i'm unable to match a single ? if I search for *?* (while it works > with wordwith?) I don't know if this is acceptable for you in terms of result exactness, but you might consider replacing accentuated chars with their ascii-counterparts during analysis. > If i search for e it highlights single e, but it doesn't for single > a... wild guess - maybe this is because a is a stopword and e isn't? In general highlighting 'e' works, as does highlighting 'a', as long as you use an analyzer with empty stopword list: require 'ferret' include Ferret i = I.new :analyzer => Analysis::StandardAnalyzer.new([]) i << 'A tree in the woods' i << 'Some sentence with e' i.highlight 'a', 0, :field => :id # => ["A tree in the woods"] i.highlight 'e', 1, :field => :id # => ["Some sentence with e"] Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From syrius.ml at no-log.org Thu Jul 5 06:19:52 2007 From: syrius.ml at no-log.org (syrius.ml at no-log.org) Date: Thu, 05 Jul 2007 12:19:52 +0200 Subject: [Ferret-talk] Inconsistent results when using wild card queries In-Reply-To: <20070705094315.GE11808@cordoba.webit.de> (Jens Kraemer's message of "Thu, 5 Jul 2007 11:43:15 +0200") References: <87r6nngred.87ps37gred@87odirgred.message.id> <20070705094315.GE11808@cordoba.webit.de> Message-ID: <87d4z7go7y.87bqergo7y@87abubgo7y.message.id> Jens Kraemer writes: >> > Has anyone else experienced something similar? Any ideas how to fix it? >> >> Unfortunatly i've also experienced that kind of weirdness. And most of >> the time it as to do with accentuation. >> i'm unable to match a single ? if I search for *?* (while it works >> with wordwith?) > > I don't know if this is acceptable for you in terms of result exactness, > but you might consider replacing accentuated chars with their > ascii-counterparts during analysis. Thanks for your quick answers Jens. It could be acceptable, but the highlighting problems I've discovered are stopping me from doing any further development. Unfortunatly I don't have time to fix them myself and Dave seems very busy. :( sorry if it sounds like whinging :) Cheers -- From kraemer at webit.de Thu Jul 5 06:30:50 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 5 Jul 2007 12:30:50 +0200 Subject: [Ferret-talk] Inconsistent results when using wild card queries In-Reply-To: <87d4z7go7y.87bqergo7y@87abubgo7y.message.id> References: <87r6nngred.87ps37gred@87odirgred.message.id> <20070705094315.GE11808@cordoba.webit.de> <87d4z7go7y.87bqergo7y@87abubgo7y.message.id> Message-ID: <20070705103050.GF11808@cordoba.webit.de> On Thu, Jul 05, 2007 at 12:19:52PM +0200, syrius.ml at no-log.org wrote: > Jens Kraemer writes: > > >> > Has anyone else experienced something similar? Any ideas how to fix it? > >> > >> Unfortunatly i've also experienced that kind of weirdness. And most of > >> the time it as to do with accentuation. > >> i'm unable to match a single ? if I search for *?* (while it works > >> with wordwith?) > > > > I don't know if this is acceptable for you in terms of result exactness, > > but you might consider replacing accentuated chars with their > > ascii-counterparts during analysis. > > Thanks for your quick answers Jens. > It could be acceptable, but the highlighting problems I've discovered > are stopping me from doing any further development. > Unfortunatly I don't have time to fix them myself and Dave seems very > busy. :( if you really like to switch, did you consider acts_as_solr? it's API is much like aaf's. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From syrius.ml at no-log.org Thu Jul 5 10:55:59 2007 From: syrius.ml at no-log.org (syrius.ml at no-log.org) Date: Thu, 05 Jul 2007 16:55:59 +0200 Subject: [Ferret-talk] Inconsistent results when using wild card queries In-Reply-To: <20070705103050.GF11808@cordoba.webit.de> (Jens Kraemer's message of "Thu, 5 Jul 2007 12:30:50 +0200") References: <87r6nngred.87ps37gred@87odirgred.message.id> <20070705094315.GE11808@cordoba.webit.de> <87d4z7go7y.87bqergo7y@87abubgo7y.message.id> <20070705103050.GF11808@cordoba.webit.de> Message-ID: <87wsxegb7e.87vecygb7e@87tzsigb7e.message.id> Jens Kraemer writes: > if you really like to switch, did you consider acts_as_solr? it's API is > much like aaf's. I certainly would if I was ok to use java. :) (but i'm not) at the moment, I'm considering hyperestraier and xapian. If there were a python api + rails plugin (and also as much features as ferret) that would be perfect :) I haven't really looked/tested yet :) -- From kraemer at webit.de Thu Jul 5 11:01:45 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 5 Jul 2007 17:01:45 +0200 Subject: [Ferret-talk] Inconsistent results when using wild card queries In-Reply-To: <87wsxegb7e.87vecygb7e@87tzsigb7e.message.id> References: <87r6nngred.87ps37gred@87odirgred.message.id> <20070705094315.GE11808@cordoba.webit.de> <87d4z7go7y.87bqergo7y@87abubgo7y.message.id> <20070705103050.GF11808@cordoba.webit.de> <87wsxegb7e.87vecygb7e@87tzsigb7e.message.id> Message-ID: <20070705150145.GJ11808@cordoba.webit.de> On Thu, Jul 05, 2007 at 04:55:59PM +0200, syrius.ml at no-log.org wrote: > Jens Kraemer writes: > > > if you really like to switch, did you consider acts_as_solr? it's API is > > much like aaf's. > > I certainly would if I was ok to use java. :) (but i'm not) afair you need no Java skills to get Solr running, however you'll need some spare server resources, that's for sure ;-) > at the moment, I'm considering hyperestraier and xapian. > If there were a python api + rails plugin (and also as much features > as ferret) that would be perfect :) Solr has an http interface, so talking to it from python would be no big deal. Otherwise you could now, possibly being the first user of xapian in a rails app, start your very own acts_as_xapian ;-) Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From ed.temp.01 at gmail.com Thu Jul 5 14:58:55 2007 From: ed.temp.01 at gmail.com (Ed Ed) Date: Thu, 5 Jul 2007 20:58:55 +0200 Subject: [Ferret-talk] ferret svn down for a few days, everything OK? Message-ID: I've been unable to check the svn repository for a week or more, I get "Can't connect to host 'svn.davebalmain.com': Connection refused". Anyone know what's going on? -- Posted via http://www.ruby-forum.com/. From ruby at jaredjackson.com Thu Jul 5 16:11:50 2007 From: ruby at jaredjackson.com (Jared Jackson) Date: Thu, 5 Jul 2007 22:11:50 +0200 Subject: [Ferret-talk] Ferret Analysis Message-ID: Is it possible to query an index and retrieve, for example, the 10 most commonly found words in the index? -- Posted via http://www.ruby-forum.com/. From patcito at gmail.com Thu Jul 5 19:16:54 2007 From: patcito at gmail.com (Patrick Aljord) Date: Thu, 5 Jul 2007 18:16:54 -0500 Subject: [Ferret-talk] how to search date with ferret and acts_as_ferret? Message-ID: <6b6419750707051616mc706e8cp944d73d2e715edca@mail.gmail.com> Hey all, How can I search date with ferret? Do I need to submit them in a special format? Right now I have in my Item model: acts_as_ferret :fields => [:created_on] with eg created_on = Thu May 11 17:00:00 -0500 2006 now when I do Item.find_by_content("created_on:<#{Time.now}") I get nil, I'm probably doing something wrong. Any idea how to fix it? thanx in advance Pat From patcito at gmail.com Thu Jul 5 20:46:21 2007 From: patcito at gmail.com (Patrick Aljord) Date: Thu, 5 Jul 2007 19:46:21 -0500 Subject: [Ferret-talk] how to search date with ferret and acts_as_ferret? In-Reply-To: <6b6419750707051616mc706e8cp944d73d2e715edca@mail.gmail.com> References: <6b6419750707051616mc706e8cp944d73d2e715edca@mail.gmail.com> Message-ID: <6b6419750707051746k484cb8e5l85edd5b49603fb71@mail.gmail.com> ok found it. acts_as_ferret :fields => [:created_on => {:index => :untokenized_omit_norms, :term_vector => :no}] def created_on self.created_on.to_i end From ktemme at gmail.com Thu Jul 5 21:25:48 2007 From: ktemme at gmail.com (Karsten Temme) Date: Fri, 6 Jul 2007 03:25:48 +0200 Subject: [Ferret-talk] Strange error when running as DRb Message-ID: I'm getting an error that seems to be unique when trying to run ferret as a DRb in conjunction with acts_as_ferret. Locally, I can run in production mode with ferret running as a DRb and everything works great. When I transfer my project to my host, things begin to fall apart. Running ferret remotely as a DRb appears to work just fine, but 'mongrel_rails start' produces: ** Starting Rails with production environment... /home/223/data/rubygems/gems/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:263:in `load_missing_constant': uninitialized constant Ferret::Store (NameError) from /home/223/data/rubygems/gems/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:452:in `const_missing'..... + more levels Has anyone encountered a similar Ferret::Store error? Thanks, Karsten -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Jul 6 03:50:32 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 6 Jul 2007 09:50:32 +0200 Subject: [Ferret-talk] ferret svn down for a few days, everything OK? In-Reply-To: References: Message-ID: <20070706075032.GK11808@cordoba.webit.de> On Thu, Jul 05, 2007 at 08:58:55PM +0200, Ed Ed wrote: > I've been unable to check the svn repository for a week or more, I get > > "Can't connect to host 'svn.davebalmain.com': Connection refused". svn checkout svn://davebalmain.com/ferret/trunk ferret works fine for me here. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Fri Jul 6 04:15:30 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 6 Jul 2007 10:15:30 +0200 Subject: [Ferret-talk] Ferret Analysis In-Reply-To: References: Message-ID: <20070706081529.GL11808@cordoba.webit.de> On Thu, Jul 05, 2007 at 10:11:50PM +0200, Jared Jackson wrote: > Is it possible to query an index and retrieve, for example, the 10 most > commonly found words in the index? http://ferret.davebalmain.com/api/classes/Ferret/Index/TermEnum.html gets quite close to this - it lets you enumerate all terms of a specific field including their frequency. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From matthew.langham at indiginox.com Fri Jul 6 05:49:54 2007 From: matthew.langham at indiginox.com (Matthew Langham) Date: Fri, 6 Jul 2007 11:49:54 +0200 Subject: [Ferret-talk] Ferret::FileNotFoundError - delete Message-ID: <9f16b21a2c67c88d5d2c5ce39fa3102e@ruby-forum.com> Hi, we are using ferret and acts_as_ferret in a single server setup. We have 2 models that use acts_as_ferret. We are running into problems where a create or update of the models cause a ferret error Ferret::FileNotFoundError occured at :117 in xpop_context Error occured in fs_store.c:329 - fs_open_input Where the file that needs to be opened isn't there. The error occurs in /lib/ferret/index.rb:285:in 'delete' Stopping the server and restarting with an index rebuild solves the problem - but it occurs again after a short time. And this occurs with both models. Ferret: 0.11.4 AAF Plugin Any ideas? Thanks Matthew -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Jul 6 06:00:27 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 6 Jul 2007 12:00:27 +0200 Subject: [Ferret-talk] Ferret::FileNotFoundError - delete In-Reply-To: <9f16b21a2c67c88d5d2c5ce39fa3102e@ruby-forum.com> References: <9f16b21a2c67c88d5d2c5ce39fa3102e@ruby-forum.com> Message-ID: <20070706100027.GN11808@cordoba.webit.de> On Fri, Jul 06, 2007 at 11:49:54AM +0200, Matthew Langham wrote: > Hi, > > we are using ferret and acts_as_ferret in a single server setup. We have > 2 models that use acts_as_ferret. > > We are running into problems where a create or update of the models > cause a ferret error > > Ferret::FileNotFoundError occured at :117 in xpop_context > Error occured in fs_store.c:329 - fs_open_input I didn't ever have this problem myself (0.11.4 / Ruby 1.8.5 / Debian). But afair several people have reported that going back to 0.11.3 fixed this for them. Cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From matthew.langham at indiginox.com Fri Jul 6 06:04:37 2007 From: matthew.langham at indiginox.com (Matthew Langham) Date: Fri, 6 Jul 2007 12:04:37 +0200 Subject: [Ferret-talk] Ferret::FileNotFoundError - delete In-Reply-To: <20070706100027.GN11808@cordoba.webit.de> References: <9f16b21a2c67c88d5d2c5ce39fa3102e@ruby-forum.com> <20070706100027.GN11808@cordoba.webit.de> Message-ID: <98a4c3f95edf5cd2e30ec0d0e9169660@ruby-forum.com> Jens Kraemer wrote: > On Fri, Jul 06, 2007 at 11:49:54AM +0200, Matthew Langham wrote: >> Hi, >> >> we are using ferret and acts_as_ferret in a single server setup. We have >> 2 models that use acts_as_ferret. >> >> We are running into problems where a create or update of the models >> cause a ferret error >> >> Ferret::FileNotFoundError occured at :117 in xpop_context >> Error occured in fs_store.c:329 - fs_open_input > > I didn't ever have this problem myself (0.11.4 / Ruby 1.8.5 / Debian). > But afair several people have reported that going back to 0.11.3 fixed > this for them. > Thanks Jens, we'll try that. Matthew > Cheers, > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From neongrau at gmail.com Fri Jul 6 12:41:57 2007 From: neongrau at gmail.com (neongrau __) Date: Fri, 6 Jul 2007 18:41:57 +0200 Subject: [Ferret-talk] ferret (0.11.4-mswin32) w/ acts as ferret Message-ID: <9384f413d92a644fb57e869fa7893122@ruby-forum.com> i'm indexing a table from some legacy db via a nightly executed script/runner command (LegacyInfobase.rebuild_index). but very often newer documents don't show up in search results for quite some days. when running it manually from console for diagnosis it mostly stops with this error (but not always): [This was fast-saved 1 times. Some information is lost] IOError: IO Error occured: couldn't rename file "./script/../config/../config/../index/production/legacy_in fobase\_3d4.tmp" to "./script/../config/../config/../index/production/legacy_inf obase\_3d4.cfs": is it required to manually delete the existing index files before a rebuild? -- Posted via http://www.ruby-forum.com/. From ferret-talk at stuartsierra.com Fri Jul 6 15:52:08 2007 From: ferret-talk at stuartsierra.com (Stuart Sierra) Date: Fri, 06 Jul 2007 15:52:08 -0400 Subject: [Ferret-talk] Highlighting returns entire field in Ferret trunk? Message-ID: <468E9D68.1080001@stuartsierra.com> Hello, I'm using Ferret trunk (#770) with Kyle's large file support patch: http://ferret.davebalmain.com/trac/ticket/215 I'm having trouble with Ferret::Search::Searcher#highlight returning too much text around the query term. Regardless of what I set :excerpt_length to, highlight returns the entire contents of the field up to the first occurrence of the search term. Any suggestions? -Stuart From curtis.hatter at insightbb.com Fri Jul 6 23:18:09 2007 From: curtis.hatter at insightbb.com (Mitchell Curtis Hatter) Date: Fri, 6 Jul 2007 23:18:09 -0400 Subject: [Ferret-talk] Extending/Modifying QueryParser Message-ID: <104F7D9B-C0C9-44C8-8C94-11C3F9B944A7@insightbb.com> Hi, I've implemented synonym searching in my rails application but have an idea I'd like to implement but can't figure out how to do. The idea is that I'd like to give the end user the choice on whether to search for the synonym of a word or not. Preferably by extending the query language to parse a construct similar to '%word1' and then have the word turned into a or list (i.e., word1|word2|word3|...). Currently, the query parser constantly calls SynonymTokenFilter to get synonyms for each token. Is there a way I can go about achieving this functionality? Here's an overview of what I've done so far: My model classes in my rails app use acts_as_ferret with a call that looks like: acts_as_ferret( :fields => [:body], :store_class_name => true, :ferret => { :or_default => false, :analyzer => SynonymAnalyzer.new(WordnetSynonymEngine.new, []) } ) I created a SynonymAnalyzer and SynonymTokenFilter: class SynonymAnalyzer < Ferret::Analysis::Analyzer include Ferret::Analysis def initialize(synonym_engine, stop_words = FULL_ENGLISH_STOP_WORDS, lower = true) @synonym_engine = synonym_engine @lower = lower @stop_words = stop_words end def token_stream(field, str) ts = StandardTokenizer.new(str) ts = LowerCaseFilter.new(ts) if @lower ts = StopFilter.new(ts, @stop_words) ts = SynonymTokenFilter.new(ts, @synonym_engine) end end class SynonymTokenFilter < Ferret::Analysis::TokenStream include Ferret::Analysis def initialize(token_stream, synonym_engine) @token_stream = token_stream @synonym_stack = [] @synonym_engine = synonym_engine end def text=(text) @token_stream.text = text end def next return @synonym_stack.pop if @synonym_stack.size > 0 if token = @token_stream.next add_synonyms_to_stack(token) unless token.nil? end return token end private def add_synonyms_to_stack(token) synonyms = @synonym_engine.get_synonyms(token.text) return if synonyms.nil? synonyms.each do |s| @synonym_stack.push( Token.new(s, token.start, token.end, 0)) end end end FInally a WordnetSynonymEngine that queries my wordnet index I created: class WordnetSynonymEngine include Ferret::Search def initialize(index_name = "wordnet") @searcher = Searcher.new("#{RAILS_ROOT}/index/#{ENV ['RAILS_ENV']}/#{index_name}") end def get_synonyms(word) @searcher.search_each(TermQuery.new(:word, word)) do |doc_id, score| return @searcher[doc_id][:syn] end return nil end end It works great except that I'd really like that ability to only run tokens through the SynonymTokenFilter when they are prepended by an unescaped % sign. Also, if anyone is interested I can post the code for turning the wordnet prolog database into a ferret database (primarily recoding the java lucene program that did the same thing to ruby and ferret). Thanks, Curtis From ahfeel at gmail.com Mon Jul 9 04:11:27 2007 From: ahfeel at gmail.com (=?utf-8?Q?J=c3=a9r=c3=a9mie_Bordier?=) Date: Mon, 9 Jul 2007 10:11:27 +0200 Subject: [Ferret-talk] Strange document loss Message-ID: <8b2679edea0046cfd0fb12db58f584ab@ruby-forum.com> Hi all :-) I'm experiencing strange document loss with Ferret 0.11.4 / Ruby 1.8.4. The index handles around 3000 documents, and sometimes during a query_update operation, the delete is done but not the add. That's very random and happens after a lot of queries (many K). I've tried to look into Ferret's C code, everything looks like going well, the add *LOOKS LIKE* being done, so it's probably deeper in the internals.. Does anybody had this problem too ? Is there any known solution ? :/ Cheers, J?r?mie -- J?r?mie -ahFeel- BORDIER http://www.unixaumonde.com -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Mon Jul 9 04:27:29 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Mon, 9 Jul 2007 10:27:29 +0200 Subject: [Ferret-talk] Strange document loss In-Reply-To: <8b2679edea0046cfd0fb12db58f584ab@ruby-forum.com> References: <8b2679edea0046cfd0fb12db58f584ab@ruby-forum.com> Message-ID: On 2007-07-09, at 10:11 AM, J?r?mie Bordier wrote: > I've tried to look into Ferret's C code, everything looks like going > well, the add *LOOKS LIKE* being done, so it's probably deeper in the > internals.. > > Does anybody had this problem too ? > Is there any known solution ? :/ hey .. i've have a similar problem but did not investigate further.. i got some 50k records indexed.. and sometimes specific documents are gone from the index.. i'm currently doing a reindex every now and then.. but i need to check this behavior .. i try to take a look today, maybe we have the same problem .. and to be more precisely.. i'm not sure if the documents are really gone from the index, but they are no longer a part of search results.. Ben From bk at benjaminkrause.com Mon Jul 9 05:34:37 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Mon, 9 Jul 2007 11:34:37 +0200 Subject: [Ferret-talk] Problems with sorting of search results Message-ID: Hey .. maybe someone got a hint for me .. I've got a problem with the sorting of search results .. strangely enough, the sorting is fine on my mac development environment, but not okay on my linux live system. I got a field in my index called hierarchy. It contains long strings like: Living together > Family > Brother-In-Law Living together > Family > Marriage > Wife Language and Communication > Language > Language Family > Finnish Now i want the results to be sorted by that hierarchy field, but it seems to be sorted randomly .. i tried both :sort => "hierarchy" as well as :sort => SortField.new(:hierarchy, :type => :string) i get differently sorted results, using the SortField or the string parameter, but neither are in the correct order. As i said, the sorting on my local development machine is fine, it's sorted the way i would expect it .. I use ferret 0.11.4 with ruby 1.8.5 on both machines. anyone got an idea or similar problems? Ben From ahfeel at gmail.com Mon Jul 9 05:57:06 2007 From: ahfeel at gmail.com (=?utf-8?Q?J=c3=a9r=c3=a9mie_Bordier?=) Date: Mon, 9 Jul 2007 11:57:06 +0200 Subject: [Ferret-talk] Strange document loss In-Reply-To: References: <8b2679edea0046cfd0fb12db58f584ab@ruby-forum.com> Message-ID: Interesting... That's very strange, what kind of usage do you do of Ferret ? most reading / searching operations ? We have a lot of writes in our requests, which isn't common in ferret's usage (ppl generally use only act_as_ferret). We are able to reproduce these issues with a copy of our index and a log of queries to replay, and i've asked D. Balmain for some help, but he doesn't answer actually. It looks like it's not just a search bug, because a strings index/* | grep myword doesn't get any result. Waiting on your answer, and _happy_ to see we're not alone with that, Cheers, J?r?mie -- J?r?mie -ahFeel- BORDIER http://www.unixaumonde.com -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Mon Jul 9 06:10:27 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Mon, 9 Jul 2007 12:10:27 +0200 Subject: [Ferret-talk] Strange document loss In-Reply-To: References: <8b2679edea0046cfd0fb12db58f584ab@ruby-forum.com> Message-ID: Hey .. > Interesting... That's very strange, what kind of usage do you do of > Ferret ? most reading / searching operations ? We have a lot of writes > in our requests, which isn't common in ferret's usage (ppl > generally use > only act_as_ferret). I'm running omdb.org .. and we have a lot of indexing requests, acutally we have a backgroundrb server accepting indexing requests and queue'ing them... so we're basically using a mechanism similar to acts_as_ferrets index-server. but our implementation is based on ferret and has nothing to do with acts_as_ferret (beside the fact, that Jens write large parts of the indexing/searching code) ;-) > We are able to reproduce these issues with a copy of our index and > a log > of queries to replay, and i've asked D. Balmain for some help, but he > doesn't answer actually. What i'm experiencing is, that a certain important record is missing after a few days. You can add casts/crew member to movies on omdb, and the Job "Director of Photography" is part of the ferret index.. but for some reason, it's gone after a few days.. and you're no longer able to select "director of photography" e.g. in such a dialog: http://www.omdb.org/movie/1832/cast/edit_crew > Waiting on your answer, and _happy_ to see we're not alone with that, as i said, i will try to investigate further in that topic.. it's about time to identify a few problems with the ferret indexing.. Ben From kraemer at webit.de Mon Jul 9 10:40:53 2007 From: kraemer at webit.de (Jens Kraemer) Date: Mon, 9 Jul 2007 16:40:53 +0200 Subject: [Ferret-talk] Problems with sorting of search results In-Reply-To: References: Message-ID: <20070709144053.GA23935@cordoba.webit.de> On Mon, Jul 09, 2007 at 11:34:37AM +0200, Benjamin Krause wrote: > Hey .. > > maybe someone got a hint for me .. I've got a > problem with the sorting of search results .. > strangely enough, the sorting is fine on my mac > development environment, but not okay on my > linux live system. > > I got a field in my index called hierarchy. It > contains long strings like: > > Living together > Family > Brother-In-Law > Living together > Family > Marriage > Wife > Language and Communication > Language > Language Family > Finnish are you sure this field is untokenized? Maybe you changed this field lately to be untokenized and have to rebuild the index on the live system? Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From bk at benjaminkrause.com Mon Jul 9 13:12:12 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Mon, 9 Jul 2007 19:12:12 +0200 Subject: [Ferret-talk] Problems with sorting of search results In-Reply-To: References: Message-ID: <88A2BD1D-0955-4619-A76E-5AC2E50C445F@benjaminkrause.com> On 2007-07-09, at 11:34 AM, Benjamin Krause wrote: > maybe someone got a hint for me .. I've got a > problem with the sorting of search results .. > strangely enough, the sorting is fine on my mac > development environment, but not okay on my > linux live system. Hey .. for some reason, the field_infos were gone (or weren't created) on my live index. The ferret-browser is a cool tool to verify the settings for each field. So if you're experiencing sorting problems, double check if your field is really :untokenized, before posting to a ml ;-) Ben From kraemer at webit.de Tue Jul 10 04:15:28 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 10 Jul 2007 10:15:28 +0200 Subject: [Ferret-talk] Extending/Modifying QueryParser In-Reply-To: <104F7D9B-C0C9-44C8-8C94-11C3F9B944A7@insightbb.com> References: <104F7D9B-C0C9-44C8-8C94-11C3F9B944A7@insightbb.com> Message-ID: <20070710081528.GR11808@cordoba.webit.de> On Fri, Jul 06, 2007 at 11:18:09PM -0400, Mitchell Curtis Hatter wrote: > Hi, > > I've implemented synonym searching in my rails application but have > an idea I'd like to implement but can't figure out how to do. The > idea is that I'd like to give the end user the choice on whether to > search for the synonym of a word or not. Preferably by extending the > query language to parse a construct similar to '%word1' and then have > the word turned into a or list (i.e., word1|word2|word3|...). > > Currently, the query parser constantly calls SynonymTokenFilter to > get synonyms for each token. Is there a way I can go about achieving > this functionality? You have to extend Ferret's Query Parser to achieve this. If you don't want to mess around with the grammar stuff the parser code is generated from, you could also preprocess user queries to modify them accordingly before giving them to the QueryParser. Can get complicated, too ;-) Atm you're doing the synonym stuff twice, once at indexing time and once when Queries are parsed. Because of the insertion of synonyms in the index at indexing time, adding synonyms to Queries is not really needed any more. So you don't really want to specify your SynonymAnalyzer for aaf as the analyzer to use for indexing and searching (aaf doesn't support different analyzers for indexing/searching bec. in general it's a good idea to use the same analyzer in both cases). If you used plain Ferret and wanted Synonyms everywhere or in a specific field, but for ALL queries, you could use your Analyzer at indexing time, but not for Query parsing. In your case, using your WordnetEngine in a customized QueryParser or a custom query preprocessor would be the better way. > Here's an overview of what I've done so far: [..] That's really cool stuff, would you mind posting this to Ferret's Wiki so other people can more easily find it? If you included the WordnetSynonymEngine that would be even better :-) Cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Tue Jul 10 04:17:38 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 10 Jul 2007 10:17:38 +0200 Subject: [Ferret-talk] ferret (0.11.4-mswin32) w/ acts as ferret In-Reply-To: <9384f413d92a644fb57e869fa7893122@ruby-forum.com> References: <9384f413d92a644fb57e869fa7893122@ruby-forum.com> Message-ID: <20070710081738.GS11808@cordoba.webit.de> On Fri, Jul 06, 2007 at 06:41:57PM +0200, neongrau __ wrote: > i'm indexing a table from some legacy db via a nightly executed > script/runner command (LegacyInfobase.rebuild_index). > > but very often newer documents don't show up in search results for quite > some days. when running it manually from console for diagnosis it mostly > stops with this error (but not always): > > > [This was fast-saved 1 times. Some information is lost] > IOError: IO Error occured: > couldn't rename file > "./script/../config/../config/../index/production/legacy_in > fobase\_3d4.tmp" to > "./script/../config/../config/../index/production/legacy_inf > obase\_3d4.cfs": > > is it required to manually delete the existing index files before a > rebuild? no, usually not. But make sure nobody else uses the index while you're rebuilding it. Use the DRb server if you can't ensure this, it takes care then. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From schulte.eric at gmail.com Tue Jul 10 10:05:40 2007 From: schulte.eric at gmail.com (Eric Schulte) Date: Tue, 10 Jul 2007 10:05:40 -0400 Subject: [Ferret-talk] Article score calculations for Boolean and MultiTerm Queries, and customization options Message-ID: Hi, I have some questions about the way that documents are scored by the Boolean and MultiTerm Queries, and about possible options for custom scoring articles. I am working on a project experimenting with different methods of automatically generating queries and the scoring mechanisms behind Lucene and Ferret have been perplexing us. >From looking at the Lucene explanation at ( http://lucene.zones.apache.org:8080/hudson/job/Lucene-Nightly/javadoc/org/apache/lucene/search/Similarity.html#formula_coord) and through using the explain function in Ferret it seems that the score calculation for a boolean query is (in latex) score = ( querynorm \times fieldnorm ) \sum_{term \in query}{ idf_{term}^{2} tf_{term} boost_{term}} and the calculation for the score of a document matching a MultiTerm Query is score = ( querynorm \times fieldnorm ) idf_{terms \in query}^{2} \sum_{term \in query}{tf_{term} boost_{term}} I would like to implement something much simpler like score = \sum_{term \in query}{tf_{term} boost_{term}} however I'm not incredibly familiar with C, and frankly looking at the scoring calculation in C inside ferret terrified me. Would the pure ruby version of ferret be a good place to try to make these changes? The latest version of that code that I can find is 0.9.4 or so. What would you recommend? Also, do you know why Lucene (and Ferret) use idf squared instead of just idf, that seems like a weird choice to me. Another sticking point is that the method of calculating idf for the MultiTerm queries (the idf of the sum of the df for every term in the query) didn't seem to make sense. For example with a query with many common words it is possible that the sum of your df's could be greater than the number of documents in the index. Many Thanks! Eric ps. let me know if the latex equations is too obtuse, and I will try to find another way to express sums in email -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070710/889e436a/attachment.html From curtis.hatter at insightbb.com Tue Jul 10 14:12:59 2007 From: curtis.hatter at insightbb.com (Mitchell Curtis Hatter) Date: Tue, 10 Jul 2007 14:12:59 -0400 Subject: [Ferret-talk] Extending/Modifying QueryParser In-Reply-To: <20070710081528.GR11808@cordoba.webit.de> References: <104F7D9B-C0C9-44C8-8C94-11C3F9B944A7@insightbb.com> <20070710081528.GR11808@cordoba.webit.de> Message-ID: <1D7BBE99-C134-4CA0-B10A-27E9C96395BD@insightbb.com> > You have to extend Ferret's Query Parser to achieve this. If you don't > want to mess around with the grammar stuff the parser code is > generated > from, you could also preprocess user queries to modify them > accordingly > before giving them to the QueryParser. Can get complicated, too ;-) I do not enjoy writing parsers, and am not especially good at it. I think first I'll check out the grammar for the parser and see if I can modify that. Perhaps creating a SynonymQuery class? I did consider preprocesing user queries and then just grouping the resulting or'd query in parens: 'rabbit %{ferret}' would parse to 'rabbit (ferret|"black-footed ferret"|etc|etc)' but I'm sure there are situations where that would not be good but it's an option. > > So you don't really want to specify your SynonymAnalyzer for aaf as > the > analyzer to use for indexing and searching (aaf doesn't support > different analyzers for indexing/searching bec. in general it's a good > idea to use the same analyzer in both cases). Thanks, I was looking at aaf wondering how I could specific a different analyzer to use for searches. I didn't find anything that would really let me get a hold of the QueryParser to change the analyzer used. Glad I wasn't just missing it. > > If you used plain Ferret and wanted Synonyms everywhere or in a > specific > field, but for ALL queries, you could use your Analyzer at indexing > time, > but not for Query parsing. In your case, using your WordnetEngine in a > customized QueryParser or a custom query preprocessor would be the > better way. Since this isn't for anything but fun right now (at work I'm stuck using Oracle's full text engine which has its own set of problems) first I'll try modifying the QueryParser grammar to account for a new query type. My C is not very good so hopefully won't have to do much, but I like that solution better then having to write a pre-processor for queries. > > That's really cool stuff, would you mind posting this to Ferret's Wiki > so other people can more easily find it? If you included the > WordnetSynonymEngine that would be even better :-) > > Cheers, > Jens Thanks, I've posted it to the Ferret wiki. It's quite long but I hope that's not a problem. I included the wordnetSynonymEngine and created a YAMLSynonymEngine just to show how it can be pluggable. Thanks for the tips I'll see what I can accomplish, Curtis From danwick at gmail.com Tue Jul 10 22:01:31 2007 From: danwick at gmail.com (Dan Wick) Date: Wed, 11 Jul 2007 04:01:31 +0200 Subject: [Ferret-talk] Install errors Message-ID: <699e971ed58f0c4b046b062492561e50@ruby-forum.com> I've been trying to install Ferret on RedHat 7.3, Ruby 1.8 (let me know if more details would be helpful) using the gem install. I've tried every version >10 and it installs, but I get errors: Building native extensions. This could take a while... In file included from q_filtered_query.c:1: search.h:683: array size missing in 'comparables' make: *** [q_filtered_query.o] Error 1 In file included from q_filtered_query.c:1: search.h:683: array size missing in 'comparables' (Yes, I get it twice.) It then goes on to successfully install, but when I go to test in the Ruby console I get an error: /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in 'gem_original_require': no such file to load -- ferret_ext (LoadError) Any ideas on what might be going on? I've had it running on Win32 locally, but I can't get it going on my staging server. Thanks. -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Jul 11 01:56:18 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 11 Jul 2007 07:56:18 +0200 Subject: [Ferret-talk] Install errors In-Reply-To: <699e971ed58f0c4b046b062492561e50@ruby-forum.com> References: <699e971ed58f0c4b046b062492561e50@ruby-forum.com> Message-ID: <20070711055618.GA30151@cordoba.webit.de> Hi! Actually your installation has failed, but unfortunately gem doesn't notice that the compilation failed and pretends everything is ok. I guess it has something to do with your build environment on that machine. What does 'gcc --version' give you on that box? What exact Ruby version do you have? Were you able to install other gems containing native extensions (i.e., Mongrel, Hpricot, RMagick)? Jens On Wed, Jul 11, 2007 at 04:01:31AM +0200, Dan Wick wrote: > I've been trying to install Ferret on RedHat 7.3, Ruby 1.8 (let me know > if more details would be helpful) using the gem install. I've tried > every version >10 and it installs, but I get errors: > > Building native extensions. This could take a while... > In file included from q_filtered_query.c:1: > search.h:683: array size missing in 'comparables' > make: *** [q_filtered_query.o] Error 1 > In file included from q_filtered_query.c:1: > search.h:683: array size missing in 'comparables' > (Yes, I get it twice.) > > It then goes on to successfully install, but when I go to test in the > Ruby console I get an error: > > /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in > 'gem_original_require': no such file to load -- ferret_ext (LoadError) > > Any ideas on what might be going on? I've had it running on Win32 > locally, but I can't get it going on my staging server. > > Thanks. > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From starburger234 at yahoo.de Wed Jul 11 06:27:14 2007 From: starburger234 at yahoo.de (Star Burger) Date: Wed, 11 Jul 2007 12:27:14 +0200 Subject: [Ferret-talk] Query with special characters crashes under Windows XP Message-ID: <463c7f948c7387ff72fd0e19fe0c4c42@ruby-forum.com> Hi, Index queries with special characters (e.g. German Umlauts but also other European ones) make the Ruby process run for quite a long time and crash finally. I'm using Rails on Windows XP with Ferret 0.11.4_mswin_32. I could isolate the problem in the following simple controller action (so there's no web browser form field involved): def test index = Ferret::I.new index << "Portim?o" p index.search "Portim?o" end The statement index.search "Portim?o" causes the crash. There is no entry in ferret_index.log or any other feedback. My renvironment contains the following: ENV['LANG'] = 'de_DE.UTF-8 at euro' ENV['LC_TIME'] = 'C' require 'acts_as_ferret' What am I doing wrong? Thanks for your help, Starburger -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Jul 11 06:32:38 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 11 Jul 2007 12:32:38 +0200 Subject: [Ferret-talk] Query with special characters crashes under Windows XP In-Reply-To: <463c7f948c7387ff72fd0e19fe0c4c42@ruby-forum.com> References: <463c7f948c7387ff72fd0e19fe0c4c42@ruby-forum.com> Message-ID: <20070711103238.GB8763@cordoba.webit.de> Hi! Afair this issue has come up a few times now, always on Windows machines... Is there some windows equivalent to unix locale settings that might be set wrong (i.e. to something not UTF8)? Jens On Wed, Jul 11, 2007 at 12:27:14PM +0200, Star Burger wrote: > Hi, > > Index queries with special characters (e.g. German Umlauts but also > other European ones) make the Ruby process run for quite a long time and > crash finally. > > I'm using Rails on Windows XP with Ferret 0.11.4_mswin_32. > > I could isolate the problem in the following simple controller action > (so there's no web browser form field involved): > > def test > index = Ferret::I.new > index << "Portim?o" > p index.search "Portim?o" > end > > > The statement index.search "Portim?o" causes the crash. There is no > entry in ferret_index.log or any other feedback. > > > My renvironment contains the following: > > ENV['LANG'] = 'de_DE.UTF-8 at euro' > ENV['LC_TIME'] = 'C' > require 'acts_as_ferret' > > What am I doing wrong? > > Thanks for your help, > > Starburger > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From starburger234 at yahoo.de Wed Jul 11 06:39:05 2007 From: starburger234 at yahoo.de (Star Burger) Date: Wed, 11 Jul 2007 12:39:05 +0200 Subject: [Ferret-talk] Query with special characters crashes under Windows XP In-Reply-To: <20070711103238.GB8763@cordoba.webit.de> References: <463c7f948c7387ff72fd0e19fe0c4c42@ruby-forum.com> <20070711103238.GB8763@cordoba.webit.de> Message-ID: Jens Kraemer wrote: > Hi! > > Afair this issue has come up a few times now, always on Windows > machines... Is there a hope that this problem will be gone under Suse Linux? This will be the production env. Starburger -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Jul 11 06:41:19 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 11 Jul 2007 12:41:19 +0200 Subject: [Ferret-talk] Query with special characters crashes under Windows XP In-Reply-To: References: <463c7f948c7387ff72fd0e19fe0c4c42@ruby-forum.com> <20070711103238.GB8763@cordoba.webit.de> Message-ID: <20070711104119.GC8763@cordoba.webit.de> On Wed, Jul 11, 2007 at 12:39:05PM +0200, Star Burger wrote: > Jens Kraemer wrote: > > Hi! > > > > Afair this issue has come up a few times now, always on Windows > > machines... > > Is there a hope that this problem will be gone under Suse Linux? This > will be the production env. Definitely. I don't have one at hand but here (Ubuntu) your snippet caused no problem. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From danwick at gmail.com Wed Jul 11 07:29:35 2007 From: danwick at gmail.com (Dan Wick) Date: Wed, 11 Jul 2007 13:29:35 +0200 Subject: [Ferret-talk] Install errors In-Reply-To: <20070711055618.GA30151@cordoba.webit.de> References: <699e971ed58f0c4b046b062492561e50@ruby-forum.com> <20070711055618.GA30151@cordoba.webit.de> Message-ID: <73380e0b780d14db7702a7dcb4129bdd@ruby-forum.com> >What does 'gcc --version' give you on that box? 2.96 >What exact Ruby version do you have? Ruby 1.8.4 (2005-12-24) [i686-linux] >Were you able to install other gems containing native extensions (i.e., Mongrel, Hpricot, RMagick)? I successfully installed and have been running Mongrel with no problems. Anything else I should check? -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Jul 11 08:15:13 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 11 Jul 2007 14:15:13 +0200 Subject: [Ferret-talk] Install errors In-Reply-To: <73380e0b780d14db7702a7dcb4129bdd@ruby-forum.com> References: <699e971ed58f0c4b046b062492561e50@ruby-forum.com> <20070711055618.GA30151@cordoba.webit.de> <73380e0b780d14db7702a7dcb4129bdd@ruby-forum.com> Message-ID: <20070711121513.GD8763@cordoba.webit.de> On Wed, Jul 11, 2007 at 01:29:35PM +0200, Dan Wick wrote: > >What does 'gcc --version' give you on that box? > 2.96 now *that* is really old ;-) Even old stable Debian Sarge has 3.3.5 (where Ferret compiles, with small tweaks in the Makefile). I guess a newer version of gcc will solve your problem. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From andreas.korth at gmx.net Wed Jul 11 08:24:33 2007 From: andreas.korth at gmx.net (Andreas Korth) Date: Wed, 11 Jul 2007 14:24:33 +0200 Subject: [Ferret-talk] Query scoring - WTF? Message-ID: <670991A3-4A9E-4F4D-B1E6-27B166C28AA1@gmx.net> Hi! I thought I understood Ferret's query scoring and how to tweak results using boost values. What I currently experience however, leaves me completely baffled. Perhaps someone can shed some light on the scoring algorithm, because asking Ferret to "explain" the score for a particular document isn't as informative as I thought. Actually, it confuses me even more. Here's what I got: I'm indexing locations (addresses) in Ferret using the following fields: street, zipcode, district, city, county, state, country_code Addresses are stored in different precisions, i.e. not all of the fields contain values depending on the location's accuracy. Here are two examples: 1. Berlin, Germany: country_code: de city: Berlin 2. The district 'Berlin' in a town called 'Seedorf': country_code: de city: Seedorf district: Berlin When querying for "berlin, de", document #2 is ranked higher (probably due to its natural position in the index). Since I want the less accurate locations to rank higher, I added boost values. In the example above, assume that city has a boost of 8 and district has a boost of 7. With this little adjustment the first document should rank higher since the term 'berlin' appears in the city field. As you might suspect, this is not what happens. And I consider this a bug. Then I went and set the document boost to be 8 for a countries and 1 for streets. This doesn't help either. The ranking of other results change slightly but nothing seems to be consistent with the boost settings. Perhaps the boost settings and the results are related in some way. But it's definitely not a logical relation. I'm thankful for any hint on how to achieve a proper ranking. Thanks! Andy From kraemer at webit.de Wed Jul 11 08:36:54 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 11 Jul 2007 14:36:54 +0200 Subject: [Ferret-talk] Query scoring - WTF? In-Reply-To: <670991A3-4A9E-4F4D-B1E6-27B166C28AA1@gmx.net> References: <670991A3-4A9E-4F4D-B1E6-27B166C28AA1@gmx.net> Message-ID: <20070711123654.GE8763@cordoba.webit.de> Hi! I tried to reproduce this however changing the sorting with modifying boosts works perfectly for me: require 'rubygems' require 'ferret' include Ferret fi = Index::FieldInfos.new fi.add_field :country_code fi.add_field :city, :boost => 8 fi.add_field :district, :boost => 7 i = Ferret::I.new :field_infos => fi i << { :country_code => 'de', :city => 'Berlin' } i << { :country_code => 'de', :city => 'Seedorf', :district => 'Berlin' } i.search_each 'berlin, de' do |hit,score| puts "#{i[hit][:country_code]} #{i[hit][:district]} #{i[hit][:city]} Score: #{score}" end this outputs de Berlin Score: 0.841327428817749 de Berlin Seedorf Score: 0.740611553192139 Swapping the boost values (city:7, district:8) also changes the result sorting. Any more info on other circumstances that might cause your problems? Jens On Wed, Jul 11, 2007 at 02:24:33PM +0200, Andreas Korth wrote: > Hi! > > I thought I understood Ferret's query scoring and how to tweak > results using boost values. What I currently experience however, > leaves me completely baffled. > > Perhaps someone can shed some light on the scoring algorithm, because > asking Ferret to "explain" the score for a particular document isn't > as informative as I thought. Actually, it confuses me even more. > > Here's what I got: > > I'm indexing locations (addresses) in Ferret using the following fields: > > street, zipcode, district, city, county, state, country_code > > Addresses are stored in different precisions, i.e. not all of the > fields contain values depending on the location's accuracy. Here are > two examples: > > 1. Berlin, Germany: > > country_code: de > city: Berlin > > 2. The district 'Berlin' in a town called 'Seedorf': > > country_code: de > city: Seedorf > district: Berlin > > When querying for "berlin, de", document #2 is ranked higher > (probably due to its natural position in the index). Since I want the > less accurate locations to rank higher, I added boost values. In the > example above, assume that city has a boost of 8 and district has a > boost of 7. > > With this little adjustment the first document should rank higher > since the term 'berlin' appears in the city field. As you might > suspect, this is not what happens. And I consider this a bug. > > Then I went and set the document boost to be 8 for a countries and 1 > for streets. This doesn't help either. > > The ranking of other results change slightly but nothing seems to be > consistent with the boost settings. Perhaps the boost settings and > the results are related in some way. But it's definitely not a > logical relation. > > I'm thankful for any hint on how to achieve a proper ranking. > > Thanks! > Andy > > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From andreas.korth at gmx.net Wed Jul 11 09:40:38 2007 From: andreas.korth at gmx.net (Andreas Korth) Date: Wed, 11 Jul 2007 15:40:38 +0200 Subject: [Ferret-talk] Query scoring - WTF? In-Reply-To: <20070711123654.GE8763@cordoba.webit.de> References: <670991A3-4A9E-4F4D-B1E6-27B166C28AA1@gmx.net> <20070711123654.GE8763@cordoba.webit.de> Message-ID: <0BEE1512-250B-4446-A81B-1C15BE56198B@gmx.net> Hi Jens, thanks a lot for reminding me that distilling a simple test case can help clearing up things quickly. I was buried so deep in my own code that I couldn't see the obvious. Turns out there is a problem with a custom analyzer of mine. It works OK and passed all tests but it seems that Ferret isn't using the same analyzer for searching and indexing although I've arranged for it. Or so I thought. I still haven't found the culprit but you put me on the right track anyway. Thanks, Andy On 11.07.2007, at 14:36, Jens Kraemer wrote: > require 'rubygems' > require 'ferret' > > include Ferret > > fi = Index::FieldInfos.new > fi.add_field :country_code > fi.add_field :city, :boost => 8 > fi.add_field :district, :boost => 7 > i = Ferret::I.new :field_infos => fi > > i << { :country_code => 'de', :city => 'Berlin' } > i << { :country_code => 'de', :city => 'Seedorf', :district => > 'Berlin' } > > i.search_each 'berlin, de' do |hit,score| > puts "#{i[hit][:country_code]} #{i[hit][:district]} #{i[hit] > [:city]} Score: #{score}" > end > > this outputs > de Berlin Score: 0.841327428817749 > de Berlin Seedorf Score: 0.740611553192139 > > Swapping the boost values (city:7, district:8) also changes the result > sorting. > > Any more info on other circumstances that might cause your problems? From andreas.korth at gmx.net Wed Jul 11 10:12:35 2007 From: andreas.korth at gmx.net (Andreas Korth) Date: Wed, 11 Jul 2007 16:12:35 +0200 Subject: [Ferret-talk] Query scoring - WTF? In-Reply-To: <0BEE1512-250B-4446-A81B-1C15BE56198B@gmx.net> References: <670991A3-4A9E-4F4D-B1E6-27B166C28AA1@gmx.net> <20070711123654.GE8763@cordoba.webit.de> <0BEE1512-250B-4446-A81B-1C15BE56198B@gmx.net> Message-ID: <012D59EE-68E4-43FA-A368-4204CE3649BE@gmx.net> On 11.07.2007, at 15:40, Andreas Korth wrote: > Turns out there is a problem with a custom analyzer of mine. It works > OK and passed all tests but it seems that Ferret isn't using the same > analyzer for searching and indexing although I've arranged for it. Or > so I thought. Here are three more questions related to the problem. The problem is definitely an analyzer mismatch but I can't really put my finger on it. 1. Is it required to pass the field_infos everytime the index is opened, or is it sufficient if the index is once created via FieldInfos#create_index? In other words: are the field infos stored in the index? 2. The analyzer to be used for both reading and writing is passed to Index.new() via the :analyzer parameter. Correct? This is what I do and I even set the analyzer explicitly using Index#add_document(doc, analyzer). 3. For a given Index, how can I determine which analyzer is currently used for any given field, both for reading and writing? Cheers, Andy From kraemer at webit.de Wed Jul 11 10:32:22 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 11 Jul 2007 16:32:22 +0200 Subject: [Ferret-talk] Query scoring - WTF? In-Reply-To: <012D59EE-68E4-43FA-A368-4204CE3649BE@gmx.net> References: <670991A3-4A9E-4F4D-B1E6-27B166C28AA1@gmx.net> <20070711123654.GE8763@cordoba.webit.de> <0BEE1512-250B-4446-A81B-1C15BE56198B@gmx.net> <012D59EE-68E4-43FA-A368-4204CE3649BE@gmx.net> Message-ID: <20070711143222.GF8763@cordoba.webit.de> On Wed, Jul 11, 2007 at 04:12:35PM +0200, Andreas Korth wrote: > > On 11.07.2007, at 15:40, Andreas Korth wrote: > > > Turns out there is a problem with a custom analyzer of mine. It works > > OK and passed all tests but it seems that Ferret isn't using the same > > analyzer for searching and indexing although I've arranged for it. Or > > so I thought. > > Here are three more questions related to the problem. The problem is > definitely an analyzer mismatch but I can't really put my finger on it. > > 1. Is it required to pass the field_infos everytime the index is > opened, or is it sufficient if the index is once created via > FieldInfos#create_index? In other words: are the field infos stored > in the index? yes. > 2. The analyzer to be used for both reading and writing is passed to > Index.new() via the :analyzer parameter. Correct? This is what I do > and I even set the analyzer explicitly using Index#add_document(doc, > analyzer). correct. > 3. For a given Index, how can I determine which analyzer is currently > used for any given field, both for reading and writing? I don't know any way to get this information. You can use process_query to see what the query parser generates from your query string (which involves analyzing it). To see what gets indexed, you could use the ferret_browser Dave introduced with the latest release to inspect your index. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From butvan at inbox.ru Thu Jul 12 06:54:54 2007 From: butvan at inbox.ru (Andrew Butvin) Date: Thu, 12 Jul 2007 12:54:54 +0200 Subject: [Ferret-talk] is it possible to get id of the matched field? Message-ID: i want to create link on the matched field: f.e. i have person and phone models. class Person < ActiveRecord::Base has_many :phones acts_as_ferret :fields => [:phone_numbers] def phone_numbers phones.collect{|phone| phone.number} end end search result's page lists out people with matched phone_numbers highlighted. i want to create link on matched phone_number to get edit_phone page, for instance. So I have to get the id of the phone with matched phone_number. is there any opportunity to do it without another search in (additional) phones index? -- Posted via http://www.ruby-forum.com/. From mmangino at elevatedrails.com Thu Jul 12 23:33:47 2007 From: mmangino at elevatedrails.com (Mike Mangino) Date: Fri, 13 Jul 2007 05:33:47 +0200 Subject: [Ferret-talk] More sorting problems with untokenized index Message-ID: I'm having problems sorting on untokenized fields. I have one field that sorts fine, but there are others that seem to sort on a different field. Here's the index description: acts_as_ferret :remote=>true,:fields=>{:name=>{:boost=>2},:name_for_sort=>{:index => :untokenized}, :city=>{:boost=>2}, :city_for_sort=>{:index=>:untokenized}, :state=>{:boost=>2}, :state_for_sort=>{:index=>:untokenized}, :tag_list=>{:boost=>0},:tag_list_for_sort=>{:boost=>0}, :date_summary=>{:boost=>1}, :date_for_range=>{:boost=>0}, :start_date=>{:boost=>0}} When I sort on name_for_sort it works fine. City_for_sort however causes problems. Here is a random offset. There are 16,000 records, so I wouldn't expect so much disparity: >> Event.find_by_contents("marathon",:sort=>"city_for_sort",:offset => 100).map(&:city_for_sort) => ["laguna hills", "burlington", "buffalo", "sun valley", "ottawa", "alexandria", "green bay", "cleveland", "aurora denver lakewood", "corpus christi"] and a later batch: >> Event.find_by_contents("marathon",:sort=>"city_for_sort",:offset => 400).map(&:city_for_sort) => ["ocean shores", "austin", "boca raton", "sauvie island", "crested butte", "austin", "portland", "avery", "leadville", "houston"] Notice that name works: >> Event.find_by_contents("marathon",:sort=>"name_for_sort",:offset => 400).map(&:name_for_sort) => ["Columbus Marathon", "Columbus Marathon", "Columbus Marathon", "Columbus Marathon", "Columbus Marathon", "Columbus Marathon Relay", "Columbus Marathon Relay", "Columbus Marathon Relay", "Comcast Baltimore Marathon", "Comcast Baltimore Marathon"] Notice however that it appears to be sorting on a range column, even when we ask for city_for_sort: >> Event.find_by_contents("marathon",:sort=>"city_for_sort",:offset => 400).map(&:date_for_range) => ["20060709", "20060708", "20060708", "20060704", "20060704", "20060704", "20060704", "20060702", "20060701", "20060701"] Does anyone have an idea what could cause this? I've rebuild the index several times and it doesn't help. I've also noticed that the default field list doesn't include the columns: using index in script/../config/../config/../index/development/event default field list: [:state, :start_date, :name, :tag_list, :city, :tag_list_for_sort, :date_for_range, :date_summary] When I look at ferret-browser, it does show the city_for_sort column. I can browse the values in order and its parameters match those of name_for_sort which works. I'm completely stumped. -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Jul 13 04:06:38 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 13 Jul 2007 10:06:38 +0200 Subject: [Ferret-talk] More sorting problems with untokenized index In-Reply-To: References: Message-ID: <20070713080638.GB30252@cordoba.webit.de> Hi! Just to rule out the possibility of aaf being the culprit here, could you try your queries using Ferret directly on the index? Shut down your app and the DRb server before, just to be sure ;-) Then you could also try to use the sort API instead of string sort (check out the Sort and SortField classes in Ferret's API). The fact that your untokenized fields do not appear in the default field list is ok, the default field list lists the field used by aaf for searching (when no specific fieldnames are used in your queries) and excludes untokenized fields (searching these fields could lead to less search results than you expected, if you are interested in why this is the case - this has been discussed on this list a few months ago). Your date_for_range and tag_list_for_sort fields should also be untokenized, however that won't solve your problem I guess. Jens On Fri, Jul 13, 2007 at 05:33:47AM +0200, Mike Mangino wrote: > I'm having problems sorting on untokenized fields. I have one field that > sorts fine, but there are others that seem to sort on a different field. > Here's the index description: > > > acts_as_ferret > :remote=>true,:fields=>{:name=>{:boost=>2},:name_for_sort=>{:index => > :untokenized}, > :city=>{:boost=>2}, :city_for_sort=>{:index=>:untokenized}, > :state=>{:boost=>2}, :state_for_sort=>{:index=>:untokenized}, > :tag_list=>{:boost=>0},:tag_list_for_sort=>{:boost=>0}, > :date_summary=>{:boost=>1}, > :date_for_range=>{:boost=>0}, > :start_date=>{:boost=>0}} > > When I sort on name_for_sort it works fine. > > City_for_sort however causes problems. Here is a random offset. There > are 16,000 records, so I wouldn't expect so much disparity: > > >> Event.find_by_contents("marathon",:sort=>"city_for_sort",:offset => 100).map(&:city_for_sort) > => ["laguna hills", "burlington", "buffalo", "sun valley", "ottawa", > "alexandria", "green bay", "cleveland", "aurora denver lakewood", > "corpus christi"] > > > and a later batch: > > >> Event.find_by_contents("marathon",:sort=>"city_for_sort",:offset => 400).map(&:city_for_sort) > => ["ocean shores", "austin", "boca raton", "sauvie island", "crested > butte", "austin", "portland", "avery", "leadville", "houston"] > > > Notice that name works: > > >> Event.find_by_contents("marathon",:sort=>"name_for_sort",:offset => 400).map(&:name_for_sort) > => ["Columbus Marathon", "Columbus Marathon", "Columbus Marathon", > "Columbus Marathon", "Columbus Marathon", "Columbus Marathon Relay", > "Columbus Marathon Relay", "Columbus Marathon Relay", "Comcast Baltimore > Marathon", "Comcast Baltimore Marathon"] > > > Notice however that it appears to be sorting on a range column, even > when we ask for city_for_sort: > > >> Event.find_by_contents("marathon",:sort=>"city_for_sort",:offset => 400).map(&:date_for_range) > => ["20060709", "20060708", "20060708", "20060704", "20060704", > "20060704", "20060704", "20060702", "20060701", "20060701"] > > > > Does anyone have an idea what could cause this? I've rebuild the index > several times and it doesn't help. > > I've also noticed that the default field list doesn't include the > columns: > > using index in script/../config/../config/../index/development/event > default field list: [:state, :start_date, :name, :tag_list, :city, > :tag_list_for_sort, :date_for_range, :date_summary] > > When I look at ferret-browser, it does show the city_for_sort column. I > can browse the values in order and its parameters match those of > name_for_sort which works. > > I'm completely stumped. > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Fri Jul 13 07:50:33 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 13 Jul 2007 13:50:33 +0200 Subject: [Ferret-talk] acts_as_ferret and capistrano In-Reply-To: <636b87ad7f3c5c7b8f7cb3e3f9d20220@ruby-forum.com> References: <20070628132941.GS7326@cordoba.webit.de> <636b87ad7f3c5c7b8f7cb3e3f9d20220@ruby-forum.com> Message-ID: <20070713115033.GE30252@cordoba.webit.de> For the time being, the Palmtree project seems to contain Cap 2.0 recipes for Ferret: http://rubyforge.org/projects/palmtree/ Jens On Fri, Jun 29, 2007 at 02:05:07AM +0200, Jonathan Viney wrote: > > I usually create my index directory once in shared/index and symlink > > this to current/index in an after_update_code task. > > Sounds pretty good. It would be nice to include all the necessary > capistrano tasks in the plugin. > > >> supposed to be using DRb. The "unless options[:remote]" block in > >> act_methods.rb (line 136) should extend down to logger.debug five lines > >> later to avoid the call to find_last_index_version. > > > > good point, I just committed this. > > > > Cheers, > -Jonathan. > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From mmangino at elevatedrails.com Fri Jul 13 08:51:00 2007 From: mmangino at elevatedrails.com (Mike Mangino) Date: Fri, 13 Jul 2007 14:51:00 +0200 Subject: [Ferret-talk] More sorting problems with untokenized index In-Reply-To: <20070713080638.GB30252@cordoba.webit.de> References: <20070713080638.GB30252@cordoba.webit.de> Message-ID: <398d69651cd3a159e91d640e21eb65c7@ruby-forum.com> Jens Kraemer wrote: > Hi! > > Just to rule out the possibility of aaf being the culprit here, could > you try your queries using Ferret directly on the index? Shut down your > app and the DRb server before, just to be sure ;-) > Sure > Then you could also try to use the sort API instead of string sort > (check out the Sort and SortField classes in Ferret's API). Good suggestion. Sort with a string is still broken when I use ferret directly. Sort with auto type is as well: total_hits = fidx.search_each("marathon",:sort=>Ferret::Search::Sort.new([Ferret::Search::SortField.new(:city_for_sort)]),:offset=>400) do |hit,score| ?> doc = fidx[hit] >> results << doc[:id] >> end => 1887 >> Event.find(results).map(&:city_for_sort) => ["ocean shores", "austin", "boca raton", "sauvie island", "crested butte", "austin", "portland", "avery", "leadville", "houston"] However, when I specify a string sort, that seems to fix it: >> Event.find(results).map(&:city_for_sort) => ["bellevue", "baton rouge", "basalt", "bend", "bellevue", "bedford", "bend", "berlin", "berlin", "beijing, china"] >> total_hits = fidx.search_each("marathon",:sort=>Ferret::Search::Sort.new([Ferret::Search::SortField.new(:city_for_sort,:type=>:string)]),:offset=>400) do |hit,score| ?> doc = fidx[hit] >> results << doc[:id] >> end => 1887 There were some fields with the text "0". I wonder if it was guessing the wrong type of index? I cleaned up that data and I'm rebuilding the index now. [snip] > > > Jens > -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Jul 13 08:55:47 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 13 Jul 2007 14:55:47 +0200 Subject: [Ferret-talk] More sorting problems with untokenized index In-Reply-To: <398d69651cd3a159e91d640e21eb65c7@ruby-forum.com> References: <20070713080638.GB30252@cordoba.webit.de> <398d69651cd3a159e91d640e21eb65c7@ruby-forum.com> Message-ID: <20070713125547.GJ30252@cordoba.webit.de> On Fri, Jul 13, 2007 at 02:51:00PM +0200, Mike Mangino wrote: > Jens Kraemer wrote: [..] > However, when I specify a string sort, that seems to fix it: > > >> Event.find(results).map(&:city_for_sort) > => ["bellevue", "baton rouge", "basalt", "bend", "bellevue", "bedford", > "bend", "berlin", "berlin", "beijing, china"] > >> total_hits = fidx.search_each("marathon",:sort=>Ferret::Search::Sort.new([Ferret::Search::SortField.new(:city_for_sort,:type=>:string)]),:offset=>400) do |hit,score| > ?> doc = fidx[hit] > >> results << doc[:id] > >> end > => 1887 > > > There were some fields with the text "0". I wonder if it was guessing > the wrong type of index? I cleaned up that data and I'm rebuilding the > index now. That sounds like a really good explanation to me. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From chenhui0594 at hotmail.com Fri Jul 13 10:49:19 2007 From: chenhui0594 at hotmail.com (Chen Hui) Date: Fri, 13 Jul 2007 16:49:19 +0200 Subject: [Ferret-talk] nike company Message-ID: <3badb7371144b9f024cf67621f4c1d41@ruby-forum.com> liyan lnternational trade Co., Ltd. is lied in the city which is general headquarters producing base of China of Nike Company ----- Putian of Fujian . We can supply all kinds of Nike brand sport shoes ( running shoes basketball shoes footwear shoes, etc.).mainly Nike , adidas,timberland,gucci,prada,Luie Vuitton and puma shoes . We can provide Nike Jordan series, Airmax and Shox series. For example: nike airmax 95, airmax 97,airmax 2003, shox TL,shox R4,Shox NZ, Air force one(AF1),Dunk, Kobe,James, Jordan I-XIX. All the shoes are packed in original boxes with retro cards, and the tags and style code number are 100% correct.The products have sold well in the places,such as America, Europe,Middle East, Southest Asia,etc.Warmly welcome customers from home and abroad contact with us by phone,e-mail and on line negotiating. We'd like to establish a long term business relationship with you. The company insists that " Customer the highest, Quality first " .With high-standard quality, a satisfied delivery and service, rational price show products and cooperative will of us to you. We always take the principle of "Based on People with Intelligence and Customers First ",with the integrity and credibility as the concept of management.All the friends from all over the world are wecome to contact us and establish a long and stabile business relation with a mutual benefit and development. CONTACT US: SITE: www.lovbiz.com E M S: chenhui0594 at hotmail.com -- Posted via http://www.ruby-forum.com/. From mmangino at elevatedrails.com Fri Jul 13 11:14:22 2007 From: mmangino at elevatedrails.com (Mike Mangino) Date: Fri, 13 Jul 2007 17:14:22 +0200 Subject: [Ferret-talk] More sorting problems with untokenized index In-Reply-To: <20070713125547.GJ30252@cordoba.webit.de> References: <20070713080638.GB30252@cordoba.webit.de> <398d69651cd3a159e91d640e21eb65c7@ruby-forum.com> <20070713125547.GJ30252@cordoba.webit.de> Message-ID: Jens Kraemer wrote: > On Fri, Jul 13, 2007 at 02:51:00PM +0200, Mike Mangino wrote: >> Jens Kraemer wrote: > > [..] > >> >> >> There were some fields with the text "0". I wonder if it was guessing >> the wrong type of index? I cleaned up that data and I'm rebuilding the >> index now. > > That sounds like a really good explanation to me. If only it were true :) >> Event.find(:all).select {|e| /^[0-9]+$/.match(e.city_for_sort)} => [] >> Event.find(:all).select {|e| /^[0-9\.]+$/.match(e.city_for_sort)} => [] >> but the problem still exists. According to http://ferret.davebalmain.com/trac/browser/trunk/c/src/sort.c, it looks like that should fix it. When I use the Sort and SortField I get the DRB error I reported previously because it can't marshall those objects: Event.find_by_contents("marathon",:sort=>Ferret::Search::arch::SortField.new(:city_for_sort)]),:offset=>400).map(&:city_for_sort) DRb::DRbConnError: DRb::DRbServerNotFound Is there an easy fix for this? > > Jens > -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Jul 13 11:29:49 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 13 Jul 2007 17:29:49 +0200 Subject: [Ferret-talk] More sorting problems with untokenized index In-Reply-To: References: <20070713080638.GB30252@cordoba.webit.de> <398d69651cd3a159e91d640e21eb65c7@ruby-forum.com> <20070713125547.GJ30252@cordoba.webit.de> Message-ID: <20070713152949.GK30252@cordoba.webit.de> On Fri, Jul 13, 2007 at 05:14:22PM +0200, Mike Mangino wrote: > Jens Kraemer wrote: > > On Fri, Jul 13, 2007 at 02:51:00PM +0200, Mike Mangino wrote: > >> Jens Kraemer wrote: > > > > [..] > > > >> > >> > >> There were some fields with the text "0". I wonder if it was guessing > >> the wrong type of index? I cleaned up that data and I'm rebuilding the > >> index now. > > > > That sounds like a really good explanation to me. > > If only it were true :) > > >> Event.find(:all).select {|e| /^[0-9]+$/.match(e.city_for_sort)} > => [] > >> Event.find(:all).select {|e| /^[0-9\.]+$/.match(e.city_for_sort)} > => [] > >> > > but the problem still exists. > > According to > http://ferret.davebalmain.com/trac/browser/trunk/c/src/sort.c, it looks > like that should fix it. > > When I use the Sort and SortField I get the DRB error I reported > previously because it can't marshall those objects: > > Event.find_by_contents("marathon",:sort=>Ferret::Search::arch::SortField.new(:city_for_sort)]),:offset=>400).map(&:city_for_sort) > > DRb::DRbConnError: DRb::DRbServerNotFound > > > Is there an easy fix for this? Do you already use acts_as_ferret's trunk? If yes and this problem still exists, your best bet is to extend local_index.rb and add a custom search method that constructs your sort objects based on additional parameters (that are no sort objects, to avoid the drb probs) you hand it over. This method will be then reachable via the ferret_index property of your model class. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From mmangino at elevatedrails.com Fri Jul 13 12:05:30 2007 From: mmangino at elevatedrails.com (Mike Mangino) Date: Fri, 13 Jul 2007 18:05:30 +0200 Subject: [Ferret-talk] More sorting problems with untokenized index In-Reply-To: <20070713152949.GK30252@cordoba.webit.de> References: <20070713080638.GB30252@cordoba.webit.de> <398d69651cd3a159e91d640e21eb65c7@ruby-forum.com> <20070713125547.GJ30252@cordoba.webit.de> <20070713152949.GK30252@cordoba.webit.de> Message-ID: Jens Kraemer wrote: > On Fri, Jul 13, 2007 at 05:14:22PM +0200, Mike Mangino wrote: >> >> index now. >> >> >> DRb::DRbConnError: DRb::DRbServerNotFound >> >> >> Is there an easy fix for this? > > Do you already use acts_as_ferret's trunk? > > If yes and this problem still exists, your best bet is to extend > local_index.rb and add a custom search method that constructs your sort > objects based on additional parameters (that are no sort objects, to > avoid the drb probs) you hand it over. This method will be then > reachable via the ferret_index property of your model class. > Okay. I did this previously. Here is my change: def find_id_by_contents(query, options = {}) if (sort=options[:sort]) sort = [sort] unless sort.is_a?(Array) sort_fields = sort.map do |field| term,direction,sort_type = field.split(/\s+/) direction ||= "asc" sort_type ||= "auto" Ferret::Search::SortField.new(term,:reverse=>direction.match(/desc/i),:type=>sort_type.to_sym) end options[:sort]=Ferret::Search::Sort.new(sort_fields) end ... That fixes the sort ordering and allows you to specify the type in the sort. I can roll that up into a patch if you would like that for inclusion. Mike > > Jens > > > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Jul 13 15:06:47 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 13 Jul 2007 21:06:47 +0200 Subject: [Ferret-talk] More sorting problems with untokenized index In-Reply-To: References: <20070713080638.GB30252@cordoba.webit.de> <398d69651cd3a159e91d640e21eb65c7@ruby-forum.com> <20070713125547.GJ30252@cordoba.webit.de> <20070713152949.GK30252@cordoba.webit.de> Message-ID: <20070713190647.GA9774@cordoba.webit.de> On Fri, Jul 13, 2007 at 06:05:30PM +0200, Mike Mangino wrote: > Jens Kraemer wrote: > > On Fri, Jul 13, 2007 at 05:14:22PM +0200, Mike Mangino wrote: > >> >> index now. > >> > >> > >> DRb::DRbConnError: DRb::DRbServerNotFound > >> > >> > >> Is there an easy fix for this? > > > > Do you already use acts_as_ferret's trunk? > > > > If yes and this problem still exists, your best bet is to extend > > local_index.rb and add a custom search method that constructs your sort > > objects based on additional parameters (that are no sort objects, to > > avoid the drb probs) you hand it over. This method will be then > > reachable via the ferret_index property of your model class. > > > > Okay. I did this previously. Here is my change: > > def find_id_by_contents(query, options = {}) > if (sort=options[:sort]) > sort = [sort] unless sort.is_a?(Array) > sort_fields = sort.map do |field| > term,direction,sort_type = field.split(/\s+/) > direction ||= "asc" > sort_type ||= "auto" > Ferret::Search::SortField.new(term,:reverse=>direction.match(/desc/i),:type=>sort_type.to_sym) > end > options[:sort]=Ferret::Search::Sort.new(sort_fields) > end > ... > > That fixes the sort ordering and allows you to specify the type in the > sort. > > I can roll that up into a patch if you would like that for inclusion. yes, please post it to acts_as_ferret's trac :-) thanks, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From mmangino at elevatedrails.com Fri Jul 13 15:27:34 2007 From: mmangino at elevatedrails.com (Mike Mangino) Date: Fri, 13 Jul 2007 21:27:34 +0200 Subject: [Ferret-talk] More sorting problems with untokenized index In-Reply-To: <20070713190647.GA9774@cordoba.webit.de> References: <20070713080638.GB30252@cordoba.webit.de> <398d69651cd3a159e91d640e21eb65c7@ruby-forum.com> <20070713125547.GJ30252@cordoba.webit.de> <20070713152949.GK30252@cordoba.webit.de> <20070713190647.GA9774@cordoba.webit.de> Message-ID: <636f50c4b2e8e8d9ff4faff49c1ce183@ruby-forum.com> > yes, please post it to acts_as_ferret's trac :-) > http://projects.jkraemer.net/acts_as_ferret/ticket/155 > > thanks, > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa Mike -- Posted via http://www.ruby-forum.com/. From golak.sarangi at gmail.com Sat Jul 14 07:14:38 2007 From: golak.sarangi at gmail.com (golak Sarangi) Date: Sat, 14 Jul 2007 16:44:38 +0530 Subject: [Ferret-talk] performance bottleneck Message-ID: <3854b3a40707140414p757e25c0q7d16b404b86ddad6@mail.gmail.com> I have got my database in Mysql. I used ferret to index a table with 10 million rows. On limiting the selection of data to 1000 initial retrieval, it takes 200 seconds but for the whole table it took more than four hours and after which i had to close my indexing application. I used the StandardAnalyser for it. There is no problem from the database side as retrieval of all the data in the table into a document takes very less time. So please suggest me ways to overcome it. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070714/4332ddd8/attachment.html From andreas.korth at gmx.net Sat Jul 14 10:40:31 2007 From: andreas.korth at gmx.net (Andreas Korth) Date: Sat, 14 Jul 2007 16:40:31 +0200 Subject: [Ferret-talk] performance bottleneck In-Reply-To: <3854b3a40707140414p757e25c0q7d16b404b86ddad6@mail.gmail.com> References: <3854b3a40707140414p757e25c0q7d16b404b86ddad6@mail.gmail.com> Message-ID: <287BCA16-4922-490D-BE36-83B7C39E9593@gmx.net> Hi, here's what I did to speed up index rebuilds: 1. Database ------------------ Use raw SQL to retrieve objects from the DB with ModelClass.connection.select_all() instead of Model.find_by_sql(). This will prevent ActiveRecord objects from being instantiated for every row, which is fairly expensive. Instead of ActiveRecord objects, select_all() returns hashes with string keys. You need to access the values with object['name'] instead of object.name. Another consequence is that helper methods such as def full_name last_name + ' ' + first_name end are not available anymore. You have to use SQL for doing this: "SELECT ..., CONCAT(first_name, ' ', last_name) AS full_name, ...". Another important point are associations. If you index objects with a has_many association, the objects themselves are loaded in a batch statement but the associateded objects are loaded separately. Assume, for example, you index Articles which have many Comments. If the Articles are loaded in batches of 1000, there will be another 1000 statements for loading the comments. With ActiveRecord find() you can use the :include option as a remedy. With raw SQL you have to use joins and grouping to get all the data you need in one row. Hint: have a look at the group functions that your RDBMS supports. Select only the columns that you need for indexing. Instead of "SELECT products.*" use "SELECT product.name, product.description,...". This is especially important if your table includes BLOB or CLOB columns. Bottom line: Brush up your SQL skills and do as much work as possible inside the database. ActiveRecord is definitely not made for batch processing, especially when dealing with millions of rows. 2. Analyzers ------------------ Use fast analyzers if possible. StandardAnalyzer is fine for most purposes but it includes HyphenFilter and StopFilter which are slow. Consider using lightweight LetterAnalyzer or WhiteSpaceAnalyzer or even no analyzer at all for particular fields. You can set individual analyzers for different fields using PerFieldAnalyzer. Don't be reluctant to write your own custom analyzer. I'm using an analyzer written in pure Ruby based on regular expressions. For its particular purpose, it's as fast as StandardAnalyzer but yields better results. Text processing in Ruby is actually pretty fast. 3. Ferret ------------ Tweak the Index parameters for rebuilds. See the Ferret documentation on what you can do. Here's an example i = Ferret::Index::Index.new( ..., :max_buffer_memory => 0x8000000, :max_buffered_docs => 0x10000, :merge_factor => 0x10, :term_index_interval => 0x40, :doc_skip_interval => 0x8 ) Experiment with these values, but be careful! Setting them arbitrarily can cause Ferret to crash. Start with the values above and try to tweak one at time. This is the last measure in optimization. Don't bother fiddling with these parameters unless you get steps 1. and 2. right. Hope this helps. Cheers. Andreas On 14.07.2007, at 13:14, golak Sarangi wrote: > I have got my database in Mysql. I used ferret to index a table > with 10 million rows. On limiting the selection of data to 1000 > initial retrieval, it takes 200 seconds but for the whole table it > took more than four hours and after which i had to close my > indexing application. I used the StandardAnalyser for it. There is > no problem from the database side as retrieval of all the data in > the table into a document takes very less time. So please suggest > me ways to overcome it. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From sjtechie at gmail.com Mon Jul 16 15:06:11 2007 From: sjtechie at gmail.com (Ikai Lan) Date: Mon, 16 Jul 2007 21:06:11 +0200 Subject: [Ferret-talk] Ferret::FileNotFoundError - delete In-Reply-To: <98a4c3f95edf5cd2e30ec0d0e9169660@ruby-forum.com> References: <9f16b21a2c67c88d5d2c5ce39fa3102e@ruby-forum.com> <20070706100027.GN11808@cordoba.webit.de> <98a4c3f95edf5cd2e30ec0d0e9169660@ruby-forum.com> Message-ID: Any luck with this? I am seeing this problem too with 0.11.4/Ubuntu 7.04/Ruby 1.8.5. I downgraded to 0.11.3 ... we'll see if this helps. Ikai -- Posted via http://www.ruby-forum.com/. From hechengcai at tom.com Tue Jul 17 05:48:18 2007 From: hechengcai at tom.com (Chengcai He) Date: Tue, 17 Jul 2007 11:48:18 +0200 Subject: [Ferret-talk] two acts_as_ferret questions! Message-ID: hi, guys: two acts_as_ferret questions! 1, ruby script/plugin install svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret sh: svn: command not found i can not install acts_as_ferret to my Rails project. what's wrong? i'm useing rhel 4! 2, i downloaded the acts_as_ferret-0.4.0.tgz at http://rubyforge.org/frs/?group_id=2916&release_id=10617 and uploaded to the /testapp/vendor/plugins/acts_as_ferret folder, when i run the commond: RAILS_ENV=development vendor/plugins/acts_as_ferret/script/ferret_start RAILS_ENV=development vendor/plugins/acts_as_ferret/script/ferret_start /usr/local/ruby/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/runner.rb:45: (eval):23: You have a nil object when you didn't expect it! (NoMethodError) You might have expected an instance of Array. The error occurred while evaluating nil.[] from /usr/local/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `eval' from /usr/local/ruby/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/runner.rb:45 from /usr/local/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require' from /usr/local/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require' from script/runner:3 i'm totally fonfused by the acts_as_ferret! how can i download the right version of acts_as_ferret? -- Posted via http://www.ruby-forum.com/. From hechengcai at tom.com Tue Jul 17 05:49:54 2007 From: hechengcai at tom.com (Chengcai He) Date: Tue, 17 Jul 2007 11:49:54 +0200 Subject: [Ferret-talk] two acts_as_ferret questions! In-Reply-To: References: Message-ID: <3d2cc78ba7bdce9002d91eca722f9c7e@ruby-forum.com> and here's the ruby & rails version ruby -v ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-linux] rails -v Rails 1.2.3 i do not know why! -- Posted via http://www.ruby-forum.com/. From flo at andersground.net Tue Jul 17 06:11:55 2007 From: flo at andersground.net (Florian Gilcher) Date: Tue, 17 Jul 2007 12:11:55 +0200 Subject: [Ferret-talk] #more_like_this : strange behaviour Message-ID: <469C95EB.9060301@andersground.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I have the following Problem: I created a fairly simple sample project to try out acts_as_ferret and present the results. The test set is relatively easy: I have extracts from 6 Wikipedia-Articles about several Topics, which are copied into a model that has two fields: title and text. This works quite well, until I try to use #more_like_this, which returns all of the other articles, even if they have nothing to do with the active article. I debugged a bit and found out that the query build by #more_like_this is nothing more then "-id:". (so the _result_ is correct) To try that out on the console, I used: entry = Entry.find(1) entry.more_like_this(:field_names => ['text']) Either I'm doing something entirely wrong or there is a bug. ;) Before filing a ticket, I want to rule out the first case. Ferret version is 0.11.4, aaf version is the current stable version (although trunk didn't work as well). I uploaded the demo project together with a dump of the Database to: Project: http://putstuff.putfile.com/95477/8752808 Dump: http://putstuff.putfile.com/95479/6169502 Thanks in advance. Florian Gilcher P.S.: There is another minor bug. Altough #more_like_this does set a default option for :field_names (line #35), this option leads to a crash in #retrieve_terms. The default option is nil and #retrieve_terms thus tries to call #each on nil. (line #113) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGnJXo8RlGMqQ8m7oRAoAqAJ4g3oqoLk8XB61tCm+hUJlKdfz0UQCgmoSi /t3GM4u/N10/S1DVyzUUE48= =wocT -----END PGP SIGNATURE----- From kraemer at webit.de Tue Jul 17 06:33:59 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 17 Jul 2007 12:33:59 +0200 Subject: [Ferret-talk] two acts_as_ferret questions! In-Reply-To: References: Message-ID: <20070717103358.GQ30252@cordoba.webit.de> On Tue, Jul 17, 2007 at 11:48:18AM +0200, Chengcai He wrote: > hi, guys: > > two acts_as_ferret questions! > 1, ruby script/plugin install > svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret > sh: svn: command not found > i can not install acts_as_ferret to my Rails project. > what's wrong? i'm useing rhel 4! looks like you don't have a subversion client installed. > 2, i downloaded the acts_as_ferret-0.4.0.tgz at > http://rubyforge.org/frs/?group_id=2916&release_id=10617 > and uploaded to the /testapp/vendor/plugins/acts_as_ferret folder, > when i run the commond: > RAILS_ENV=development vendor/plugins/acts_as_ferret/script/ferret_start > > RAILS_ENV=development vendor/plugins/acts_as_ferret/script/ferret_start > /usr/local/ruby/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/runner.rb:45: > (eval):23: You have a nil object when you didn't expect it! > (NoMethodError) did you install ferret? Acts_as_ferret is just a plugin using the Ferret search engine. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Tue Jul 17 07:47:10 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 17 Jul 2007 13:47:10 +0200 Subject: [Ferret-talk] #more_like_this : strange behaviour In-Reply-To: <469C95EB.9060301@andersground.net> References: <469C95EB.9060301@andersground.net> Message-ID: <20070717114710.GS30252@cordoba.webit.de> Hi, first of all, 6 documents is not really a corpus to judge the usability of more_like_this - by default it will only consider terms occuring in at least 5 documents to be of any relevance (:min_doc_freq option). So if you have very different documents where the only common words are filtered out as noise words, you'll end up without any terms to use for finding similar documents, which would lead to the query you mentioned. However more_like_this should indeed return an empty result set in this case ;-) Besides that, you should store term vectors (give :term_vector => :yes for the fields you want to use more_like_this on in your call to acts_as_ferret), this will speed up the search for relevant terms. Jens On Tue, Jul 17, 2007 at 12:11:55PM +0200, Florian Gilcher wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > > I have the following Problem: > > I created a fairly simple sample project to try out acts_as_ferret and > present the results. > > The test set is relatively easy: I have extracts from 6 > Wikipedia-Articles about several Topics, which are copied into a model > that has two fields: title and text. This works quite well, until I try > to use #more_like_this, which returns all of the other articles, even if > they have nothing to do with the active article. I debugged a bit and > found out that the query build by #more_like_this is nothing more then > "-id:". > (so the _result_ is correct) > > To try that out on the console, I used: > > entry = Entry.find(1) > entry.more_like_this(:field_names => ['text']) > > Either I'm doing something entirely wrong or there is a bug. ;) Before > filing a ticket, I want to rule out the first case. > > Ferret version is 0.11.4, aaf version is the current stable version > (although trunk didn't work as well). > > I uploaded the demo project together with a dump of the Database to: > > Project: http://putstuff.putfile.com/95477/8752808 > Dump: http://putstuff.putfile.com/95479/6169502 > > Thanks in advance. > Florian Gilcher > > P.S.: There is another minor bug. Altough #more_like_this does set a > default option for :field_names (line #35), this option leads to a crash > in #retrieve_terms. The default option is nil and #retrieve_terms thus > tries to call #each on nil. (line #113) > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.3 (Darwin) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iD8DBQFGnJXo8RlGMqQ8m7oRAoAqAJ4g3oqoLk8XB61tCm+hUJlKdfz0UQCgmoSi > /t3GM4u/N10/S1DVyzUUE48= > =wocT > -----END PGP SIGNATURE----- > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From flo at andersground.net Tue Jul 17 08:07:14 2007 From: flo at andersground.net (Florian Gilcher) Date: Tue, 17 Jul 2007 14:07:14 +0200 Subject: [Ferret-talk] #more_like_this : strange behaviour In-Reply-To: <20070717114710.GS30252@cordoba.webit.de> References: <469C95EB.9060301@andersground.net> <20070717114710.GS30252@cordoba.webit.de> Message-ID: <469CB0F2.2090100@andersground.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I am aware of the fact that the corpus is a bit small (but nicer for presentation purposes), but it surprised me that I found no way (even when playing with the parameters) to get at least 1 common word from the the set. (it wasn't intended to be usable, but presentable) I will play around a bit more and add some documents. Thanks for the hints. Greetings Florian Gilcher Jens Kraemer wrote: > Hi, > > first of all, 6 documents is not really a corpus to judge the usability > of more_like_this - by default it will only consider terms occuring in > at least 5 documents to be of any relevance (:min_doc_freq option). So > if you have very different documents where the only common words are > filtered out as noise words, you'll end up without any terms to use > for finding similar documents, which would lead to the query you > mentioned. > > However more_like_this should indeed return an empty result set in this > case ;-) > > Besides that, you should store term vectors (give :term_vector => :yes > for the fields you want to use more_like_this on in your call to > acts_as_ferret), this will speed up the search for relevant terms. > > > Jens > > > On Tue, Jul 17, 2007 at 12:11:55PM +0200, Florian Gilcher wrote: > Hi, > > I have the following Problem: > > I created a fairly simple sample project to try out acts_as_ferret and > present the results. > > The test set is relatively easy: I have extracts from 6 > Wikipedia-Articles about several Topics, which are copied into a model > that has two fields: title and text. This works quite well, until I try > to use #more_like_this, which returns all of the other articles, even if > they have nothing to do with the active article. I debugged a bit and > found out that the query build by #more_like_this is nothing more then > "-id:". > (so the _result_ is correct) > > To try that out on the console, I used: > > entry = Entry.find(1) > entry.more_like_this(:field_names => ['text']) > > Either I'm doing something entirely wrong or there is a bug. ;) Before > filing a ticket, I want to rule out the first case. > > Ferret version is 0.11.4, aaf version is the current stable version > (although trunk didn't work as well). > > I uploaded the demo project together with a dump of the Database to: > > Project: http://putstuff.putfile.com/95477/8752808 > Dump: http://putstuff.putfile.com/95479/6169502 > > Thanks in advance. > Florian Gilcher > > P.S.: There is another minor bug. Altough #more_like_this does set a > default option for :field_names (line #35), this option leads to a crash > in #retrieve_terms. The default option is nil and #retrieve_terms thus > tries to call #each on nil. (line #113) _______________________________________________ Ferret-talk mailing list Ferret-talk at rubyforge.org http://rubyforge.org/mailman/listinfo/ferret-talk >> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGnLDa8RlGMqQ8m7oRAvfwAJ9Tf3n8doy/EzkDS/Q4Mgf+WNTZZwCeMCnu 75or+J8oDXojyqO4oUzt3IY= =uhKz -----END PGP SIGNATURE----- From hechengcai at tom.com Tue Jul 17 09:34:29 2007 From: hechengcai at tom.com (Chengcai He) Date: Tue, 17 Jul 2007 15:34:29 +0200 Subject: [Ferret-talk] two acts_as_ferret questions! In-Reply-To: <20070717103358.GQ30252@cordoba.webit.de> References: <20070717103358.GQ30252@cordoba.webit.de> Message-ID: <96880dd26c5a17a3f9de448905cec677@ruby-forum.com> thanks to Jens Kraemer: I have installed the ferret. gem install ferret! -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Tue Jul 17 17:03:32 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Tue, 17 Jul 2007 23:03:32 +0200 Subject: [Ferret-talk] acts_as_ferret 0.4.1 Message-ID: <20070717210332.GI21849@thunder.jkraemer.net> Hi folks, I just released version 0.4.1 fixing several bugs and bringing the DRb server's index versioning feature to the stable branch. Please see [1] for more changes. Cheers, Jens [1] http://projects.jkraemer.net/acts_as_ferret/query?status=closed&milestone=0.4.1 -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From hechengcai at tom.com Tue Jul 17 22:23:21 2007 From: hechengcai at tom.com (Chengcai He) Date: Wed, 18 Jul 2007 04:23:21 +0200 Subject: [Ferret-talk] acts_as_ferret 0.4.1 In-Reply-To: <20070717210332.GI21849@thunder.jkraemer.net> References: <20070717210332.GI21849@thunder.jkraemer.net> Message-ID: <00c05ef0ba9b0dfc8b8c744422e85614@ruby-forum.com> Hello, man, how to download it? 2, i downloaded the acts_as_ferret-0.4.0.tgz at http://rubyforge.org/frs/?group_id=2916&release_id=10617 and uploaded to the /testapp/vendor/plugins/acts_as_ferret folder, when i run the commond: RAILS_ENV=development vendor/plugins/acts_as_ferret/script/ferret_start RAILS_ENV=development vendor/plugins/acts_as_ferret/script/ferret_start /usr/local/ruby/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/runner.rb:45: (eval):23: You have a nil object when you didn't expect it! (NoMethodError) You might have expected an instance of Array. The error occurred while evaluating nil.[] from /usr/local/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `eval' from /usr/local/ruby/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/runner.rb:45 from /usr/local/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require' from /usr/local/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require' from script/runner:3 -- Posted via http://www.ruby-forum.com/. From hechengcai at tom.com Tue Jul 17 22:58:53 2007 From: hechengcai at tom.com (Chengcai He) Date: Wed, 18 Jul 2007 04:58:53 +0200 Subject: [Ferret-talk] DRb Server can not run! Message-ID: I'm doing the next. 1, rails hecc 2, cd hecc 3, script/plugin install svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret 4, RAILS_ENV=production vendor/plugins/acts_as_ferret/script/ferret_start vendor/plugins/acts_as_ferret/script/ferret_start:21:in `require': no such file to load -- vendor/plugins/acts_as_ferret/script/../config/boot (LoadError) from vendor/plugins/acts_as_ferret/script/ferret_start:21 ferret_start content 20 ENV['FERRET_USE_LOCAL_INDEX'] = 'true' 21 require File.dirname(__FILE__) + '/../config/boot' 22 require RAILS_ROOT + '/config/environment' I do not know why! -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Wed Jul 18 02:08:18 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Wed, 18 Jul 2007 08:08:18 +0200 Subject: [Ferret-talk] DRb Server can not run! In-Reply-To: References: Message-ID: <20070718060818.GA17572@thunder.jkraemer.net> On Wed, Jul 18, 2007 at 04:58:53AM +0200, Chengcai He wrote: > I'm doing the next. > > 1, rails hecc > 2, cd hecc > 3, script/plugin install > svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret > > 4, RAILS_ENV=production > vendor/plugins/acts_as_ferret/script/ferret_start use the start script that has been copied to script/ by the installer: RAILS_ENV=production script/ferret_start Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From xinyin at sjtu.edu.cn Wed Jul 18 03:36:32 2007 From: xinyin at sjtu.edu.cn (Sinnyn Lee) Date: Wed, 18 Jul 2007 09:36:32 +0200 Subject: [Ferret-talk] How to use fixnum in RangeQuery? Message-ID: <29f21317151bf83a57e934d07293d5e4@ruby-forum.com> As far as i know about the RangeQuery is that it need fixed-length number in query options such as RangeQuery.new(:count, :>= => '00', :<= => '99'). But is there a mechanism to use a fixnum in the options instead of filling the left with 0? Regards. -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Wed Jul 18 04:33:49 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Wed, 18 Jul 2007 10:33:49 +0200 Subject: [Ferret-talk] How to use fixnum in RangeQuery? In-Reply-To: <29f21317151bf83a57e934d07293d5e4@ruby-forum.com> References: <29f21317151bf83a57e934d07293d5e4@ruby-forum.com> Message-ID: <20070718083349.GB17572@thunder.jkraemer.net> Hi! On Wed, Jul 18, 2007 at 09:36:32AM +0200, Sinnyn Lee wrote: > As far as i know about the RangeQuery is that it need fixed-length > number in query options such as RangeQuery.new(:count, :>= => '00', :<= > => '99'). > But is there a mechanism to use a fixnum in the options instead of > filling the left with 0? No, you have to use strings. You could write a wrapper around RangeQuery that accepts Fixnums and does the left padding to the width of your largest field value of course. cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From hechengcai at tom.com Wed Jul 18 11:41:46 2007 From: hechengcai at tom.com (Chengcai He) Date: Wed, 18 Jul 2007 17:41:46 +0200 Subject: [Ferret-talk] Strange search result with conditions in find_by_contents Message-ID: <50fce5369644be50ab58968019d66fa6@ruby-forum.com> Hi, guys: Strange search result with conditions in find_by_contents! first of all, i've installed the acts_as_ferret to my project vender folder by 'ruby script/plugin install svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret' in my SearchController def searchforum if !params[:doSearch].nil? if params[:searchTerms].nil? || params[:searchTerms] == "" flash[:notice] = 'Please enter some words to search on.' else @conditions = " 1 = 1"; if !params[:dateRange].nil? && params[:dateRange] != "" @conditions += " and creationDate >= " + params[:dateRange] end if !params[:forumID].nil? && params[:forumID] != "" @conditions += " and forum_id = " + params[:forumID] end # @total, @topics = Topic.full_text_search(params[:searchTerms], :page => (params[:page]||1)) @total, @topics = Topic.full_text_search(params[:searchTerms], {:page => (params[:page]||1)}, {:conditions => @conditions}) @pages = pages_for(@total) end end end in my model Topic: def self.full_text_search(q, options = {}, find_options = {}) return nil if q.nil? or q=="" default_options = {:limit => 10, :page => 1} options = default_options.merge options # get the offset based on what page we're on options[:offset] = options[:limit] * (options.delete(:page).to_i-1) # now do the query with our options results = Topic.find_by_contents(q, options, find_options) puts options puts find_options return [results.total_hits, results] end -------------------------------- i display 10 results in one page. In SearchController, if i run this code: @total, @topics = Topic.full_text_search(params[:searchTerms], :page => (params[:page]||1)) and input a search term, it will get 16 result messages. when i click to the next page, it display the next 6 results, and the total_hits is still 16. but if i run this code in SearchController: @total, @topics = Topic.full_text_search(params[:searchTerms], {:page => (params[:page]||1)}, {:conditions => @conditions}) and input the same search term, it will also get 16 result messages. but when i click to the next page, it display the next 6 results, and the total_hits changes to 6 messages, and only display one page. speak in shortly: searching with conditions in find_by_contents, it got 16 total_hits, it display the first 10 result messages in first page, when i click to the next page, it display the next 6 results, but the total_hits changes to 6, and only got one page to display. i don't know why. -- Posted via http://www.ruby-forum.com/. From hechengcai at tom.com Wed Jul 18 11:44:50 2007 From: hechengcai at tom.com (Chengcai He) Date: Wed, 18 Jul 2007 17:44:50 +0200 Subject: [Ferret-talk] Strange search result with conditions in find_by_content In-Reply-To: <50fce5369644be50ab58968019d66fa6@ruby-forum.com> References: <50fce5369644be50ab58968019d66fa6@ruby-forum.com> Message-ID: <5acb1ef844ad4cc4314721195db2f81a@ruby-forum.com> and in the ApplicationController def pages_for(size, options = {}) default_options = {:per_page => 10} options = default_options.merge options pages = Paginator.new self, size, options[:per_page], (params[:page]||1) return pages end thanks in advance! -- Posted via http://www.ruby-forum.com/. From cwaters at networkchemistry.com Wed Jul 18 12:07:11 2007 From: cwaters at networkchemistry.com (Waters, Chris) Date: Wed, 18 Jul 2007 12:07:11 -0400 Subject: [Ferret-talk] Ferret doesn't work with Luke Message-ID: <8D2B87EFB35A4341B53B86D3DE6F4CD50234D03E@mse1be1.mse1.mailstreet.com> Hi, Does anyone know why the indexes created by Ferret can't be opened by Luke (http://www.getopt.org/luke/)? When I do development with Clucene I use Luke all the time to understand what is going on in the index. It is especially useful when trying to diagnose analyzer issues. When I try to open a Ferret index with Luke I get the message "Invalid or corrupted index". I thought that Ferret used Lucene's file format and so Luke should be able to open them. Thanks, Chris Waters From ferret-talk at stuartsierra.com Wed Jul 18 13:39:41 2007 From: ferret-talk at stuartsierra.com (Stuart Sierra) Date: Wed, 18 Jul 2007 13:39:41 -0400 Subject: [Ferret-talk] Ferret doesn't work with Luke In-Reply-To: <8D2B87EFB35A4341B53B86D3DE6F4CD50234D03E@mse1be1.mse1.mailstreet.com> References: <8D2B87EFB35A4341B53B86D3DE6F4CD50234D03E@mse1be1.mse1.mailstreet.com> Message-ID: <469E505D.7050402@stuartsierra.com> I believe Ferret has diverged from Lucene to the point that their indexes are not compatible. ferret-browser, included in the latest Ferret gems, may display some of the same information as Luke. -Stuart Sierra Waters, Chris wrote: > Hi, > > Does anyone know why the indexes created by Ferret can't be opened by > Luke (http://www.getopt.org/luke/)? When I do development with Clucene I > use Luke all the time to understand what is going on in the index. It is > especially useful when trying to diagnose analyzer issues. > > When I try to open a Ferret index with Luke I get the message "Invalid > or corrupted index". I thought that Ferret used Lucene's file format and > so Luke should be able to open them. > > Thanks, > > Chris Waters > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From jk at jkraemer.net Wed Jul 18 14:04:03 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Wed, 18 Jul 2007 20:04:03 +0200 Subject: [Ferret-talk] Strange search result with conditions in find_by_contents In-Reply-To: <50fce5369644be50ab58968019d66fa6@ruby-forum.com> References: <50fce5369644be50ab58968019d66fa6@ruby-forum.com> Message-ID: <20070718180403.GG22943@thunder.jkraemer.net> Hi! On Wed, Jul 18, 2007 at 05:41:46PM +0200, Chengcai He wrote: [..] > > speak in shortly: searching with conditions in find_by_contents, it got > 16 total_hits, it display the first 10 result messages in first page, > when i click to the next page, it display the next 6 results, but the > total_hits changes to 6, and only got one page to display. The reason for the odd behaviour you experience is as follows: Aaf first queries ferret for your search results, it gets a number of ids matching your query (and your paging). Now it uses this set of ids to run a query against the DB, combining it with your own conditions. So in the end there is no simple way for aaf to determine the total number of hits (without running the query more than once). Additionally it might happen that you get less results than expected, since Ferret is delivering :limit hits, but these might well be cut down to less than :limit by your active record conditions. There are several ways to work around this: - Get the total_hits separately by calling Model.find_by_contents without the paging options. Doesn't solve the second problem I mentioned above. - Use :limit and :offset as part of your find_options, so the paging takes place in the database. Might work okay, didn't ever try it. - Put the additional data in your index, too, and do not use active_record conditions at all. Just append your additional query criteria to the query entered by the user. - If you find a way how aaf could deliver consistent results in this scenario with any other method I can't think of right now, submit a patch :-) Btw, do never ever directly append request parameters to conditions like this: > @conditions = " 1 = 1"; > if !params[:dateRange].nil? && params[:dateRange] != "" > @conditions += " and creationDate >= " + params[:dateRange] > end > if !params[:forumID].nil? && params[:forumID] != "" > @conditions += " and forum_id = " + params[:forumID] > end but use something like conditions = [ 'creationDate >= ? AND forum_id = ?', params[:dateRange], forum_id ] to avoid exploitation of your app via sql injection. In newer Rails there's a Hash based notation for the conditions, too (that might be easier to assemble in your case). cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From cwaters at networkchemistry.com Wed Jul 18 19:00:49 2007 From: cwaters at networkchemistry.com (Waters, Chris) Date: Wed, 18 Jul 2007 19:00:49 -0400 Subject: [Ferret-talk] Ferret doesn't work with Luke In-Reply-To: <469E505D.7050402@stuartsierra.com> References: <8D2B87EFB35A4341B53B86D3DE6F4CD50234D03E@mse1be1.mse1.mailstreet.com> <469E505D.7050402@stuartsierra.com> Message-ID: <8D2B87EFB35A4341B53B86D3DE6F4CD50234D150@mse1be1.mse1.mailstreet.com> Thanks. Ferret-browser is exactly what I needed. Any idea where the index formats diverged? It is a pity because it removes the ability to use a much larger set of tools for managing Ferret indexes, and also the possibility of writing tools in ruby to manipulate lucene indexes. Regards, Chris. -----Original Message----- From: ferret-talk-bounces at rubyforge.org [mailto:ferret-talk-bounces at rubyforge.org] On Behalf Of Stuart Sierra Sent: Wednesday, July 18, 2007 10:40 AM To: ferret-talk at rubyforge.org Subject: Re: [Ferret-talk] Ferret doesn't work with Luke I believe Ferret has diverged from Lucene to the point that their indexes are not compatible. ferret-browser, included in the latest Ferret gems, may display some of the same information as Luke. -Stuart Sierra Waters, Chris wrote: > Hi, > > Does anyone know why the indexes created by Ferret can't be opened by > Luke (http://www.getopt.org/luke/)? When I do development with Clucene I > use Luke all the time to understand what is going on in the index. It is > especially useful when trying to diagnose analyzer issues. > > When I try to open a Ferret index with Luke I get the message "Invalid > or corrupted index". I thought that Ferret used Lucene's file format and > so Luke should be able to open them. > > Thanks, > > Chris Waters > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk _______________________________________________ Ferret-talk mailing list Ferret-talk at rubyforge.org http://rubyforge.org/mailman/listinfo/ferret-talk From hechengcai at tom.com Wed Jul 18 23:40:45 2007 From: hechengcai at tom.com (Chengcai He) Date: Thu, 19 Jul 2007 05:40:45 +0200 Subject: [Ferret-talk] Strange search result with conditions in find_by_content In-Reply-To: <20070718180403.GG22943@thunder.jkraemer.net> References: <50fce5369644be50ab58968019d66fa6@ruby-forum.com> <20070718180403.GG22943@thunder.jkraemer.net> Message-ID: one more question: in the Model acts_as_ferret({ :fields => {:username => {:store => :yes, :boost => 30}, :subject => {:store => :yes, :boost => 20}, :body => {:store => :yes, :boost => 10}}, :remote => true }, { :analyzer => Ferret::Analysis::RegExpAnalyzer.new(/./, false) }) def username return self.user.login end search as: @total, @topics = Topic.full_text_search(params[:searchTerms], {:page => (params[:page]||1)}, {:conditions => @conditions}) if i input a query term to search, i will got the result: @total = 14, @topics.length = 12 but when i input the user login to search the topics posted by the user, it got: @total = 14, @topics.length = 0. It just display the paginate things, but none of the result topic displayed! I'm totally lost! -- Posted via http://www.ruby-forum.com/. From hechengcai at tom.com Wed Jul 18 23:57:01 2007 From: hechengcai at tom.com (Chengcai He) Date: Thu, 19 Jul 2007 05:57:01 +0200 Subject: [Ferret-talk] Strange search result with conditions in find_by_content In-Reply-To: References: <50fce5369644be50ab58968019d66fa6@ruby-forum.com> <20070718180403.GG22943@thunder.jkraemer.net> Message-ID: <3af565b648580001e8f388dda0f948f2@ruby-forum.com> search terms @total @topics.length ???? 12 12 ?? 14 12 hecc 16 0 -------------------------------------- 'hecc' is the user login name. -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Thu Jul 19 04:17:54 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 19 Jul 2007 10:17:54 +0200 Subject: [Ferret-talk] Ferret doesn't work with Luke In-Reply-To: <8D2B87EFB35A4341B53B86D3DE6F4CD50234D150@mse1be1.mse1.mailstreet.com> References: <8D2B87EFB35A4341B53B86D3DE6F4CD50234D03E@mse1be1.mse1.mailstreet.com> <469E505D.7050402@stuartsierra.com> <8D2B87EFB35A4341B53B86D3DE6F4CD50234D150@mse1be1.mse1.mailstreet.com> Message-ID: <20070719081753.GZ30252@cordoba.webit.de> On Wed, Jul 18, 2007 at 07:00:49PM -0400, Waters, Chris wrote: > Thanks. Ferret-browser is exactly what I needed. > > Any idea where the index formats diverged? It is a pity because it > removes the ability to use a much larger set of tools for managing > Ferret indexes, and also the possibility of writing tools in ruby to > manipulate lucene indexes. afaik problems started with UTF8 content, which Java serializes in a way different from the rest of the world. Then Dave decided to give up on index compatibility since it was useless for everybody not only storing ASCII content anyway and made changes to the index format to enhance performance and ease the implementation of some features, but I'm not sure what or when exactly this was. The ml archive could be of help to find out more. cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From starburger234 at yahoo.de Thu Jul 19 10:12:18 2007 From: starburger234 at yahoo.de (Star Burger) Date: Thu, 19 Jul 2007 16:12:18 +0200 Subject: [Ferret-talk] Query with special characters crashes under Windows XP In-Reply-To: <20070711104119.GC8763@cordoba.webit.de> References: <463c7f948c7387ff72fd0e19fe0c4c42@ruby-forum.com> <20070711103238.GB8763@cordoba.webit.de> <20070711104119.GC8763@cordoba.webit.de> Message-ID: Yeah! No problem with Suse 10. Jens Kraemer wrote: > > Definitely. I don't have one at hand but here (Ubuntu) your snippet > caused no problem. > > Jens > > -- Posted via http://www.ruby-forum.com/. From chelsea.robb at gmail.com Thu Jul 19 11:29:42 2007 From: chelsea.robb at gmail.com (Chelsea Robb) Date: Thu, 19 Jul 2007 17:29:42 +0200 Subject: [Ferret-talk] Acts_As_Ferret only returns results when searching for "*" Message-ID: <9317eaadaca2d44c5af259468525e251@ruby-forum.com> Hey, I'm probably just missing something really obvious but I've been stuck on this problem for a while now and I'm not getting anywhere. I have installed the acts_as_ferret gem, as well as ferret according to the tutorial on RailsEnvy. My model, controller, and environment.rb files have all been modified as per that turotial. However any attempts to use the find_by_contents on my model returns 0 results unless I am doing Article.find_by_contents("*"), with Article being my model. Searching for * will return all records from the database. In the Article model I have the "acts_as_ferret" with no fields specified as I was under the impression that this would index all fields. I've also tried specifying specific fields which still yielded the same error. Has anyone else run into this problem? -- Posted via http://www.ruby-forum.com/. From jjm at codewell.com Thu Jul 19 15:30:17 2007 From: jjm at codewell.com (Jeff Mallatt) Date: Thu, 19 Jul 2007 15:30:17 -0400 Subject: [Ferret-talk] highlighting from multiple indexes Message-ID: <7.0.1.0.2.20070719152943.03660210@codewell.com> Hi. I'm searching multiple indexes by passing an array of paths to Index::IndexReader.new(). I get several hits back, and can determine the document id of each. Now I want to fetch highlighted excerpts from a field in one of the matched documents. The problem I'm having is that Index::IndexReader doesn't have a highlight() method. And, while Index::Index does have a highlight() method, its new() method doesn't allow for an array of paths. What am I missing, here? Thanks in advance. From bryan at team46.com Thu Jul 19 17:34:19 2007 From: bryan at team46.com (Bryan Wilkerson) Date: Thu, 19 Jul 2007 23:34:19 +0200 Subject: [Ferret-talk] partial term query by default Message-ID: <09d93240a21b1853ab31effa209016ab@ruby-forum.com> My customers seem to expect that the world of search is dictated by Google. As such they expect all queries except phrase queries to include partial term hits. In other words, searching on "test" also returns "testing". Is there a quick option to enable this or anyway more elegant than gsub'ing the query to add asterisks around each term (except those in phrases of course)? Thanks, -bryan -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Fri Jul 20 02:53:00 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Fri, 20 Jul 2007 08:53:00 +0200 Subject: [Ferret-talk] Acts_As_Ferret only returns results when searching for "*" In-Reply-To: <9317eaadaca2d44c5af259468525e251@ruby-forum.com> References: <9317eaadaca2d44c5af259468525e251@ruby-forum.com> Message-ID: <20070720065300.GA15455@thunder.jkraemer.net> Hi! Please have a look in your log files (development.log and ferret_index.log), maybe we can see what's going on there. Jens On Thu, Jul 19, 2007 at 05:29:42PM +0200, Chelsea Robb wrote: > Hey, I'm probably just missing something really obvious but I've been > stuck on this problem for a while now and I'm not getting anywhere. > > I have installed the acts_as_ferret gem, as well as ferret according to > the tutorial on RailsEnvy. My model, controller, and environment.rb > files have all been modified as per that turotial. However any attempts > to use the find_by_contents on my model returns 0 results unless I am > doing Article.find_by_contents("*"), with Article being my model. > Searching for * will return all records from the database. > > In the Article model I have the "acts_as_ferret" with no fields > specified as I was under the impression that this would index all > fields. I've also tried specifying specific fields which still yielded > the same error. > > Has anyone else run into this problem? > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From kylenordman at gmail.com Fri Jul 20 04:03:07 2007 From: kylenordman at gmail.com (Kyle Nord) Date: Fri, 20 Jul 2007 10:03:07 +0200 Subject: [Ferret-talk] Cacti - Monitor Ferret Server Stats? Message-ID: I'm wondering if someone can point me in a direction to monitor the ferret drb server while using cacti. Is it possible to run multiple instances of ferret drb on the same machine, just on different ports? Will it still update the index properly, or will you get collision? New to moving over to the drb server. Thanks. -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Jul 20 04:28:00 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 20 Jul 2007 10:28:00 +0200 Subject: [Ferret-talk] partial term query by default In-Reply-To: <09d93240a21b1853ab31effa209016ab@ruby-forum.com> References: <09d93240a21b1853ab31effa209016ab@ruby-forum.com> Message-ID: <20070720082800.GE30252@cordoba.webit.de> On Thu, Jul 19, 2007 at 11:34:19PM +0200, Bryan Wilkerson wrote: > > My customers seem to expect that the world of search is dictated by > Google. As such they expect all queries except phrase queries to > include partial term hits. In other words, searching on "test" also > returns "testing". With a Stemmer included in your analyzer 'testing' would actually be indexed as 'test' and therefore get found with a query for 'test'. Of course this won't help if you expect to find 'tree' when searching for 'tr', in this case you'll have to build wildcard queries from the terms of the query. Here's an example for a custom analyzer using a StemFilter: http://www.ruby-forum.com/topic/90606#new Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Fri Jul 20 04:30:36 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 20 Jul 2007 10:30:36 +0200 Subject: [Ferret-talk] highlighting from multiple indexes In-Reply-To: <7.0.1.0.2.20070719152943.03660210@codewell.com> References: <7.0.1.0.2.20070719152943.03660210@codewell.com> Message-ID: <20070720083036.GF30252@cordoba.webit.de> On Thu, Jul 19, 2007 at 03:30:17PM -0400, Jeff Mallatt wrote: > Hi. > > I'm searching multiple indexes by passing an array of paths to > Index::IndexReader.new(). I get several hits back, and can determine > the document id of each. > > Now I want to fetch highlighted excerpts from a field in one of the > matched documents. The problem I'm having is that Index::IndexReader > doesn't have a highlight() method. And, while Index::Index does have > a highlight() method, its new() method doesn't allow for an array of paths. > > What am I missing, here? The Searcher class :-) Create a searcher from your IndexReader and use it's highlight method. Cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Fri Jul 20 04:43:10 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 20 Jul 2007 10:43:10 +0200 Subject: [Ferret-talk] Cacti - Monitor Ferret Server Stats? In-Reply-To: References: Message-ID: <20070720084310.GH30252@cordoba.webit.de> On Fri, Jul 20, 2007 at 10:03:07AM +0200, Kyle Nord wrote: > I'm wondering if someone can point me in a direction to monitor the > ferret drb server while using cacti. > > Is it possible to run multiple instances of ferret drb on the same > machine, just on different ports? Will it still update the index > properly, or will you get collision? New to moving over to the drb > server. The whole point of the DRb server is to only have one process accessing the index - so running multiple DRb's on the same index would be quite counter-productive ;-) cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Fri Jul 20 04:45:53 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 20 Jul 2007 10:45:53 +0200 Subject: [Ferret-talk] Strange search result with conditions in find_by_content In-Reply-To: References: <50fce5369644be50ab58968019d66fa6@ruby-forum.com> <20070718180403.GG22943@thunder.jkraemer.net> Message-ID: <20070720084553.GI30252@cordoba.webit.de> On Thu, Jul 19, 2007 at 05:40:45AM +0200, Chengcai He wrote: > one more question: > in the Model > acts_as_ferret({ :fields => {:username => {:store => :yes, :boost => > 30}, :subject => {:store => :yes, :boost => 20}, :body => {:store => > :yes, :boost => 10}}, :remote => true }, { :analyzer => > Ferret::Analysis::RegExpAnalyzer.new(/./, false) }) > > def username > return self.user.login > end > > search as: > @total, @topics = Topic.full_text_search(params[:searchTerms], {:page => > (params[:page]||1)}, {:conditions => @conditions}) > > if i input a query term to search, i will got the result: @total = 14, > @topics.length = 12 > but when i input the user login to search the topics posted by the user, > it got: > @total = 14, @topics.length = 0. It just display the paginate things, > but none of the result topic displayed! > > I'm totally lost! Look at your log files, see what queries ferret runs against the index, and see what sql statements are issued to your DB. I guess without the conditions everything is ok? cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kylenordman at gmail.com Fri Jul 20 04:49:18 2007 From: kylenordman at gmail.com (Kyle Nord) Date: Fri, 20 Jul 2007 10:49:18 +0200 Subject: [Ferret-talk] Cacti - Monitor Ferret Server Stats? In-Reply-To: <20070720084310.GH30252@cordoba.webit.de> References: <20070720084310.GH30252@cordoba.webit.de> Message-ID: Jens Kraemer wrote: > On Fri, Jul 20, 2007 at 10:03:07AM +0200, Kyle Nord wrote: >> I'm wondering if someone can point me in a direction to monitor the >> ferret drb server while using cacti. >> >> Is it possible to run multiple instances of ferret drb on the same >> machine, just on different ports? Will it still update the index >> properly, or will you get collision? New to moving over to the drb >> server. > > The whole point of the DRb server is to only have one process accessing > the index - so running multiple DRb's on the same index would be quite > counter-productive ;-) > > cheers, > Jens > > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa I should have actually re-phrased that, I apologize. Long term solution thinking here... What happens when the load gets too high on the box the index is currently on, is there a way to scale it horizontally? Besides using certain app servers to access certain indexes (ie: photos for searching on app003 --> search003), and profiles app002 --> search 002. Not sure how else to explain that better... But hopefully it's clear enough =) -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Jul 20 05:17:10 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 20 Jul 2007 11:17:10 +0200 Subject: [Ferret-talk] Cacti - Monitor Ferret Server Stats? In-Reply-To: References: <20070720084310.GH30252@cordoba.webit.de> Message-ID: <20070720091710.GJ30252@cordoba.webit.de> On Fri, Jul 20, 2007 at 10:49:18AM +0200, Kyle Nord wrote: [..] > > I should have actually re-phrased that, I apologize. > > Long term solution thinking here... What happens when the load gets too > high on the box the index is currently on, is there a way to scale it > horizontally? Besides using certain app servers to access certain > indexes (ie: photos for searching on app003 --> search003), and profiles > app002 --> search 002. Ah ok, I see :-) First of all - have you already checked if load will ever be too high on a dedicated search box with your expected traffic? You know, premature optimization is the root of all evil ;-) Actually the one-process-per-index thing is only necessary for writing to the index, you can have multiple searchers open on the same physical index. So if you find a way to give multiple servers access to the same physical index (i.e. via NAS) you could run searches from multiple machines, but write access would have to be restricted to one machine. Shared file systems like NFS or samba aren't a good idea for sharing the index. Of course you could also just duplicate your index across machines for faster searching, but then you'll have to take care of syncing it with the master every now and then of course. The stock DRb server doesn't have built in support for any of these distribution scenarios. If searches really are equally distributed between the various indexes putting them on separate machines might be your best option, imho. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From alain.ravet+ferret at gmail.com Fri Jul 20 11:40:58 2007 From: alain.ravet+ferret at gmail.com (Alain Ravet) Date: Fri, 20 Jul 2007 17:40:58 +0200 Subject: [Ferret-talk] acts_as_ferret 0.4.1 In-Reply-To: <00c05ef0ba9b0dfc8b8c744422e85614@ruby-forum.com> References: <20070717210332.GI21849@thunder.jkraemer.net> <00c05ef0ba9b0dfc8b8c744422e85614@ruby-forum.com> Message-ID: > Hello, man, how to download it? ./script/plugin install svn://projects.jkraemer.net/acts_as_ferret/tags/0.4.1/plugin/acts_as_ferret Alain From alain.ravet+ferret at gmail.com Fri Jul 20 11:52:22 2007 From: alain.ravet+ferret at gmail.com (Alain Ravet) Date: Fri, 20 Jul 2007 17:52:22 +0200 Subject: [Ferret-talk] why/when does the DRb server rebuild the index (when I don't ask)? Message-ID: Hi, I'm using DRb server, and sometimes it starts rebuilding the index for no - apparent - reason (no model nor DB schema change). As this blocks the computer for more that an hour, I must fix this problem before I put the system in production. Question: - how can I find WHY the server triggered an index rebuild? - how can I avoid it? Thanks in advance. Alain From kylenordman at gmail.com Fri Jul 20 13:30:44 2007 From: kylenordman at gmail.com (Kyle Nord) Date: Fri, 20 Jul 2007 19:30:44 +0200 Subject: [Ferret-talk] Cacti - Monitor Ferret Server Stats? In-Reply-To: <20070720091710.GJ30252@cordoba.webit.de> References: <20070720084310.GH30252@cordoba.webit.de> <20070720091710.GJ30252@cordoba.webit.de> Message-ID: <8b8ca0e5a6de46a53af138ae36921f01@ruby-forum.com> Jens Kraemer wrote: > On Fri, Jul 20, 2007 at 10:49:18AM +0200, Kyle Nord wrote: > [..] >> >> I should have actually re-phrased that, I apologize. >> >> Long term solution thinking here... What happens when the load gets too >> high on the box the index is currently on, is there a way to scale it >> horizontally? Besides using certain app servers to access certain >> indexes (ie: photos for searching on app003 --> search003), and profiles >> app002 --> search 002. > > Ah ok, I see :-) > > First of all - have you already checked if load will ever be too high on > a dedicated search box with your expected traffic? You know, premature > optimization is the root of all evil ;-) > > Actually the one-process-per-index thing is only necessary for writing > to the index, you can have multiple searchers open on the same physical > index. So if you find a way to give multiple servers access to the same > physical index (i.e. via NAS) you could run searches from multiple > machines, but write access would have to be restricted to one machine. > > Shared file systems like NFS or samba aren't a good idea for > sharing the index. > > Of course you could also just duplicate your index across machines for > faster searching, but then you'll have to take care of syncing it with > the master every now and then of course. > > The stock DRb server doesn't have built in support for any of these > distribution scenarios. > > If searches really are equally distributed between the various indexes > putting them on separate machines might be your best option, imho. > > Jens > > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa Thanks so much for the reply, that's all I needed to know! I actually use to use backgroundrb to access the search results before, and render it with javascript (so kind of an ajax style search) but I had a huge bottleneck with the drb server. So I'm hoping this new acts_as_ferret with 0.11.4 ferret will be the ticket, and so far it looks as if it is =). Once I get the stats, I'll fill you in on our load. Currently about 15,000 unique searches a day (peaking between 5-8pm PST), hits load at 1.8.. thats with a sun x4100. So during those peak hours if I could offload some of the processing to other machines, that would be the best. Maybe just cron an rsync job for the index every now and again. BTW: very impressed with the fact that if the drb server cannot be connected to it defaults to the local index (and if that does not exist it crashes). But that is a great fail safe, so if you have an rsync job with the index and just copy it to your app servers every 4am, seems to be a great "nothing shared" tactic. -- Posted via http://www.ruby-forum.com/. From tennisbum2002 at hotmail.com Fri Jul 20 23:22:39 2007 From: tennisbum2002 at hotmail.com (Aryk Grosz) Date: Sat, 21 Jul 2007 05:22:39 +0200 Subject: [Ferret-talk] Ferret not building index successfully all of a sudden Message-ID: I've been using acts_as_ferret for a couple of months now. Everything was going fine. Suddenly, however, when I try to rebuild my index, only 1 of the 2 models I index gets indexed correctly. Looking in the ferret_server logs, it says that both were successfully rebuilt. However, one of the indexes has no data in the folder. The other was index correctly. The odd thing is that this only happens on my production environment where I have a DRB server. I think it might have something to do with this DRB folder but Im not sure. Again, it throws no errors, and says the rebuild was successful. I haven't changed any code relating to my search or the indexing fields either. Do you guys have any ideas what happened? -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Sat Jul 21 12:44:52 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Sat, 21 Jul 2007 18:44:52 +0200 Subject: [Ferret-talk] Online Version of the Ferret API Message-ID: Hey ... while davebalmain.com is not renewed (and who knows when that is going happen), i've put the 0.11.4 API rdocs online at: http://api.omdb-beta.org/ferret/index.html Ben From matthew.langham at indiginox.com Sun Jul 22 14:42:46 2007 From: matthew.langham at indiginox.com (Matthew Langham) Date: Sun, 22 Jul 2007 20:42:46 +0200 Subject: [Ferret-talk] Strange Connection refused error acts_as_ferret server Message-ID: <539f1611631fb2658b80aac6dd3015c1@ruby-forum.com> I am having a strange problem starting the acts_as_ferret Drb server on a particular Linux machine (Debian Etch). The same setup works fine on my Mac (OSX). Without the Rails application running (yes, I've checked several times) I get the following error if I try to start the ferret server and one of my models contains :remote => true for acts_as_ferret: >> script/ferret_start /usr/lib/ruby/1.8/drb/drb.rb:736:in `open': druby://localhost:3009 - # (DRb::DRbConnError) from /usr/lib/ruby/1.8/drb/drb.rb:729:in `each' from /usr/lib/ruby/1.8/drb/drb.rb:729:in `open' from /usr/lib/ruby/1.8/drb/drb.rb:1189:in `initialize' from /usr/lib/ruby/1.8/drb/drb.rb:1169:in `new' from /usr/lib/ruby/1.8/drb/drb.rb:1169:in `open' from /usr/lib/ruby/1.8/drb/drb.rb:1085:in `method_missing' from /usr/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend' from /usr/lib/ruby/1.8/drb/drb.rb:1084:in `method_missing' ... 6 levels... from /usr/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/runner.rb:39 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require' from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `require' from script/runner:3 << If I remove the :remote => true bit then I can start the Ferret server with no problems. I just don't understand why the model entry is causing this - especially as the application isn't running. ferret 0.11.3 I'm betting this is in the "Duh" section - but I just don't see it. Thanks Matthew -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Mon Jul 23 02:33:51 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Mon, 23 Jul 2007 08:33:51 +0200 Subject: [Ferret-talk] Strange Connection refused error acts_as_ferret server In-Reply-To: <539f1611631fb2658b80aac6dd3015c1@ruby-forum.com> References: <539f1611631fb2658b80aac6dd3015c1@ruby-forum.com> Message-ID: <20070723063351.GM15455@thunder.jkraemer.net> On Sun, Jul 22, 2007 at 08:42:46PM +0200, Matthew Langham wrote: > I am having a strange problem starting the acts_as_ferret Drb server on > a particular Linux machine (Debian Etch). The same setup works fine on > my Mac (OSX). > > Without the Rails application running (yes, I've checked several times) > I get the following error if I try to start the ferret server and one of > my models contains :remote => true for acts_as_ferret: > > >> > script/ferret_start > /usr/lib/ruby/1.8/drb/drb.rb:736:in `open': druby://localhost:3009 - > # > (DRb::DRbConnError) This looks like the DRb server itself tries to connect to the DRb server - normally aaf should detect when it is running *in* the DRb server, but sometimes this doesn't seem to work. Do you use the latest aaf (0.4.1)? There have been some fixes in this area since 0.4.0. Please have a look in your development log, watch out for a line starting with 'Asked for a remote server'. Cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From portie at bitterbal.org Mon Jul 23 14:18:53 2007 From: portie at bitterbal.org (Izit Izit) Date: Mon, 23 Jul 2007 20:18:53 +0200 Subject: [Ferret-talk] AAF association not workin Message-ID: <0d5af68d6a26e96c9a45452d62e90ce2@ruby-forum.com> Hi, I have configured my model as follows: class Product < ActiveRecord::Base acts_as_ferret :fields => [:description,:label_description,:label_free,:product_id_supplier,:description_supplier,:supplier_description] belongs_to :supplier def supplier_description return "#{self.supplier.description}" end In the development log I can see the supplier_description being added to the index: creating doc for class: Product, id: 24710306 Adding field label_free with value '67CM' to index Adding field product_id_supplier with value '177-175-078 middel' to index Supplier Load (0.000359) SELECT * FROM suppliers WHERE (suppliers.`id` = 19) Adding field supplier_description with value 'Haans' to index Adding field description with value 'MAND RATTAN MIDDEL 67CM' to index Adding field description_supplier with value '' to index Adding field label_description with value 'MAND RATTAN MIDDEL' to index But when I do a Product.find_by_contents("Haans") I get nothing back. => # The development log shows: Query: Haans total hits: 0, results delivered: 0 Does anybody know why my association does not work. Other searches on fields in the Product model work fine. -- Posted via http://www.ruby-forum.com/. From james.herdman at gmail.com Mon Jul 23 14:34:31 2007 From: james.herdman at gmail.com (James Herdman) Date: Mon, 23 Jul 2007 20:34:31 +0200 Subject: [Ferret-talk] Acts_As_Ferret: disable_ferret and Callbacks Message-ID: <875b9c87521c773132de43812739ac1f@ruby-forum.com> Does disable_ferret monkey with callbacks at all? I have a model in my application that has associated models which perform callbacks after their validation (they're all instantiated around the same time). I recently added a conditional call to disable ferret provided a flag and I'm finding these callbacks aren't quite working right. Here's a code sample: class Model < ActiveRecord::Base has_one :associated_model # This model will fill in some fields upon it's own validation def save_with_activation_handling! if active? # This part works self.activated_at = Time.now else disable_ferret :once self.activated_at = nil end save_without_activation_handling! end alias_method_chain :save!, :activation_handling end Any ideas? Thank you, James Herdman -- Posted via http://www.ruby-forum.com/. From tennisbum2002 at hotmail.com Mon Jul 23 16:07:06 2007 From: tennisbum2002 at hotmail.com (Aryk Grosz) Date: Mon, 23 Jul 2007 22:07:06 +0200 Subject: [Ferret-talk] Ferret indexing problem through rake tasks Message-ID: <6025e10e9982adc6fb055934af456e78@ruby-forum.com> I'm having a weird problem. Recently, on all my environments, one of my two indexed models is not returning any results to acts_as_ferret. However, the folder is being created for it and Im not receiving any errors. On my two models, I call the following: Book.rebuild_index User.rebuild_index When I call those commands through either my live app "ruby script/server" or through the console "ruby script/console," it works fine and I can search on the Book model. However, when I run those exact commands through a rake task (with :environment set), it creates the indexes successfully, but it will return NO results for the Book model. However, User will continue to work. Looking in the log files, it shows that the rebuilds were successful. Im guessing there is some kind of linking problems here. Anyone have any suggestions? Thanks in advance. - Aryk -- Posted via http://www.ruby-forum.com/. From mbklein at gmail.com Mon Jul 23 16:20:48 2007 From: mbklein at gmail.com (Michael Klein) Date: Mon, 23 Jul 2007 22:20:48 +0200 Subject: [Ferret-talk] Determining the version of Ferret used to create an index Message-ID: Hello, I keep getting "IndexError: index XXXXXX out of string" when I try to use the index (with XXXXXX changing depending on what version of Ferret I try to use to load the index). The data from which I created the index has been lost, and I'm hoping to retrieve whatever I can out of the index. Is there a way I can determine from a file header or something which version of Ferret was used to create the index so I can install the correct gem? Thanks! Michael -- Posted via http://www.ruby-forum.com/. From martin.xus at gmail.com Mon Jul 23 23:19:57 2007 From: martin.xus at gmail.com (Xu Martin) Date: Tue, 24 Jul 2007 05:19:57 +0200 Subject: [Ferret-talk] Why can't find index file sometimes? Message-ID: <2649bcda29f84605138b8f720b51979a@ruby-forum.com> I'm using ferret and acts_as_ferret,i can success save record,but sometimes when i update record,index not found error occurred(not always).I don't know why:( [quote] File Not Found Error occured at :117 in xpop_context Error occured in fs_store.c:329 - fs_open_input tried to open "/home/dev/www/index/development/rcu/_12q.cfs" but it doesn't exist: [/quote] Here is the trace [quote] /usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:285:in `delete' /usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:285:in `<<' /usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:8:in `synchrolock' /usr/local/lib/ruby/1.8/monitor.rb:238:in `synchronize' /usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:8:in `synchrolock' /usr/local/lib/ruby/gems/1.8/gems/ferret-0.11.4/lib/ferret/index.rb:267:in `<<' #{RAILS_ROOT}/vendor/plugins/acts_as_ferret/lib/local_index.rb:157:in `<<' #{RAILS_ROOT}/vendor/plugins/acts_as_ferret/lib/instance_methods.rb:73:in `ferret_update' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/callbacks.rb:333:in `send' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/callbacks.rb:333:in `callback' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/callbacks.rb:330:in `each' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/callbacks.rb:330:in `callback' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/callbacks.rb:268:in `update_without_timestamps' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/timestamp.rb:48:in `update' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/base.rb:1789:in `create_or_update_without_callbacks' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/callbacks.rb:242:in `create_or_update' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/base.rb:1545:in `save_without_validation' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/validations.rb:752:in `save_without_transactions' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/transactions.rb:129:in `save' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:59:in `transaction' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/transactions.rb:95:in `transaction' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/transactions.rb:121:in `transaction' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/transactions.rb:129:in `save' #{RAILS_ROOT}/vendor/rails/activerecord/lib/active_record/validations.rb:773:in `update_attribute' [/quote] -- Posted via http://www.ruby-forum.com/. From portie at bitterbal.org Tue Jul 24 02:18:59 2007 From: portie at bitterbal.org (Izit Izit) Date: Tue, 24 Jul 2007 08:18:59 +0200 Subject: [Ferret-talk] AAF association not workin In-Reply-To: <0d5af68d6a26e96c9a45452d62e90ce2@ruby-forum.com> References: <0d5af68d6a26e96c9a45452d62e90ce2@ruby-forum.com> Message-ID: I got it working but in a very strange way. If I deleted the index directory and AAF recreated it automatically on the first search my search did not work. BUT if I do Model.rebuild_index to rebuild the index my search DOES work. Izit Izit wrote: > Hi, > > I have configured my model as follows: > > class Product < ActiveRecord::Base > > acts_as_ferret :fields => > [:description,:label_description,:label_free,:product_id_supplier,:description_supplier,:supplier_description] > > belongs_to :supplier > > > def supplier_description > return "#{self.supplier.description}" > end > > In the development log I can see the supplier_description being added to > the index: > > creating doc for class: Product, id: 24710306 > Adding field label_free with value '67CM' to index > Adding field product_id_supplier with value '177-175-078 middel' to > index > Supplier Load (0.000359) SELECT * FROM suppliers WHERE > (suppliers.`id` = 19) > Adding field supplier_description with value 'Haans' to index > Adding field description with value 'MAND RATTAN MIDDEL 67CM' to index > Adding field description_supplier with value '' to index > Adding field label_description with value 'MAND RATTAN MIDDEL' to index > > > But when I do a Product.find_by_contents("Haans") I get nothing back. > > => # > > The development log shows: > > Query: Haans > total hits: 0, results delivered: 0 > > Does anybody know why my association does not work. Other searches on > fields in the Product model work fine. -- Posted via http://www.ruby-forum.com/. From isha.kakodkar at gmail.com Tue Jul 24 03:33:47 2007 From: isha.kakodkar at gmail.com (isha kakodkar) Date: Tue, 24 Jul 2007 00:33:47 -0700 Subject: [Ferret-talk] Act as Ferret supports Conditional search? Message-ID: <87b412ce0707240033y3f72a1bcr6a77e74f92512b3b@mail.gmail.com> Hi all, Im a newbie to ferret.I have installed the act_as_ferret gem and i actually want to search for some content in the model,but per user. My model has data,user_id.So within the data column i have to serach for a word if it exists for a given user. So can i achieve that ? presently i know that it just searches the "data" column irrespective of the user given. Can someone help me with this? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070724/5c3f59fe/attachment.html From jk at jkraemer.net Tue Jul 24 04:47:49 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Tue, 24 Jul 2007 10:47:49 +0200 Subject: [Ferret-talk] Act as Ferret supports Conditional search? In-Reply-To: <87b412ce0707240033y3f72a1bcr6a77e74f92512b3b@mail.gmail.com> References: <87b412ce0707240033y3f72a1bcr6a77e74f92512b3b@mail.gmail.com> Message-ID: <20070724084749.GB26963@thunder.jkraemer.net> On Tue, Jul 24, 2007 at 12:33:47AM -0700, isha kakodkar wrote: > Hi all, > Im a newbie to ferret.I have installed the act_as_ferret gem and i > actually want to search for some content in the model,but per user. > My model has data,user_id.So within the data column i have to serach for a > word if it exists for a given user. > So can i achieve that ? presently i know that it just searches the "data" > column irrespective of the user given. > Can someone help me with this? Index the user_id field (as untokenized) and append " user_id:#{user.id}" to the query. cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From bk at benjaminkrause.com Tue Jul 24 12:21:30 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Tue, 24 Jul 2007 18:21:30 +0200 Subject: [Ferret-talk] Why can't find index file sometimes? In-Reply-To: <2649bcda29f84605138b8f720b51979a@ruby-forum.com> References: <2649bcda29f84605138b8f720b51979a@ruby-forum.com> Message-ID: <7F616174-6708-4A57-8C07-5C645057D50B@benjaminkrause.com> On 2007-07-24, at 5:19 AM, Xu Martin wrote: > I'm using ferret and acts_as_ferret,i can success save record,but > sometimes when i update record,index not found error occurred(not > always).I don't know why:( do you use the ferret indexing server? If not, use it :-) Most of the time these errors occurs, if you have more than one indexwriter open at the same time.. Ben From isha.kakodkar at gmail.com Wed Jul 25 02:39:15 2007 From: isha.kakodkar at gmail.com (isha kakodkar) Date: Tue, 24 Jul 2007 23:39:15 -0700 Subject: [Ferret-talk] Act as Ferret supports Conditional search? In-Reply-To: <20070724084749.GB26963@thunder.jkraemer.net> References: <87b412ce0707240033y3f72a1bcr6a77e74f92512b3b@mail.gmail.com> <20070724084749.GB26963@thunder.jkraemer.net> Message-ID: <87b412ce0707242339oe3acb4n6ec9d474a30b27e3@mail.gmail.com> thanx..i actually did it using the :conditions with find_by_contents. But when i do find_id_by_contents,which searches from the filesystem,can i search within data based on this index value?and not just the full data field? On 7/24/07, Jens Kraemer wrote: > > On Tue, Jul 24, 2007 at 12:33:47AM -0700, isha kakodkar wrote: > > Hi all, > > Im a newbie to ferret.I have installed the act_as_ferret gem and i > > actually want to search for some content in the model,but per user. > > My model has data,user_id.So within the data column i have to serach for > a > > word if it exists for a given user. > > So can i achieve that ? presently i know that it just searches the > "data" > > column irrespective of the user given. > > Can someone help me with this? > > Index the user_id field (as untokenized) and append " user_id:#{user.id}" > to the query. > > cheers, > Jens > > > -- > Jens Kr?mer > http://www.jkraemer.net/ - Blog > http://www.omdb.org/ - The new free film database > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070724/3ec92088/attachment.html From kraemer at webit.de Wed Jul 25 05:36:25 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 25 Jul 2007 11:36:25 +0200 Subject: [Ferret-talk] Acts_As_Ferret: disable_ferret and Callbacks In-Reply-To: <875b9c87521c773132de43812739ac1f@ruby-forum.com> References: <875b9c87521c773132de43812739ac1f@ruby-forum.com> Message-ID: <20070725093625.GA27557@cordoba.webit.de> On Mon, Jul 23, 2007 at 08:34:31PM +0200, James Herdman wrote: > Does disable_ferret monkey with callbacks at all? I have a model in my > application that has associated models which perform callbacks after > their validation (they're all instantiated around the same time). I > recently added a conditional call to disable ferret provided a flag and > I'm finding these callbacks aren't quite working right. what exactly does this mean? Is Ferret not disabled, or are your callbacks never called? Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Wed Jul 25 05:40:06 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 25 Jul 2007 11:40:06 +0200 Subject: [Ferret-talk] Ferret indexing problem through rake tasks In-Reply-To: <6025e10e9982adc6fb055934af456e78@ruby-forum.com> References: <6025e10e9982adc6fb055934af456e78@ruby-forum.com> Message-ID: <20070725094006.GB27557@cordoba.webit.de> Hi! On Mon, Jul 23, 2007 at 10:07:06PM +0200, Aryk Grosz wrote: > I'm having a weird problem. Recently, on all my environments, one of my > two indexed models is not returning any results to acts_as_ferret. > However, the folder is being created for it and Im not receiving any > errors. Did you change anything in your setup before this started happening (maybe an aaf upgrade)? What version of aaf do you use? > On my two models, I call the following: > > Book.rebuild_index > User.rebuild_index > > When I call those commands through either my live app "ruby > script/server" or through the console "ruby script/console," it works > fine and I can search on the Book model. > > However, when I run those exact commands through a rake task (with > :environment set), it creates the indexes successfully, but it will > return NO results for the Book model. However, User will continue to > work. Looking in the log files, it shows that the rebuilds were > successful. Im guessing there is some kind of linking problems here. When you say aaf creates the indexes successfully, does that mean that you checked them with plain Ferret and could retrieve documents from the books index? cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Wed Jul 25 05:43:42 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 25 Jul 2007 11:43:42 +0200 Subject: [Ferret-talk] why/when does the DRb server rebuild the index (when I don't ask)? In-Reply-To: References: Message-ID: <20070725094342.GC27557@cordoba.webit.de> Hi! First of all, what version of aaf do you use? On Fri, Jul 20, 2007 at 05:52:22PM +0200, Alain Ravet wrote: > Hi, > > I'm using DRb server, and sometimes it starts rebuilding the index for > no - apparent - reason (no model nor DB schema change). As this blocks > the computer for more that an hour, I must fix this problem before I > put the system in production. > > Question: > - how can I find WHY the server triggered an index rebuild? > - how can I avoid it? The only time the index is rebuilt automatically by the server is when there is no index. Everything else I'd consider a bug. Does the server really rebuild without any external activity, i.e. deploying/restarting of application or drb server? Anything interesting in the log files? Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From portie at bitterbal.org Wed Jul 25 07:04:01 2007 From: portie at bitterbal.org (Izit Izit) Date: Wed, 25 Jul 2007 13:04:01 +0200 Subject: [Ferret-talk] DRb not starting Message-ID: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> Hi, I have my Model as follows: class Mutation < ActiveRecord::Base acts_as_ferret ({:fields => {:description=>{}, :product_id=>{:index => :untokenized}, :product_description=>{}, :product_label_description=>{}, :product_label_free=>{}, :product_product_id_supplier=>{}, :product_description_supplier=>{}, :supplier_description=>{}, :pub_date_sort => {:index => :untokenized_omit_norms, :term_vector => :no}, :location_id => {:index => :untokenized}, :tab => {:index => :untokenized}},:remote => false}) THis will do the ferret locally without the DRB server. Works ok. If I change this to: class Mutation < ActiveRecord::Base acts_as_ferret ({:fields => {:description=>{}, :product_id=>{:index => :untokenized}, :product_description=>{}, :product_label_description=>{}, :product_label_free=>{}, :product_product_id_supplier=>{}, :product_description_supplier=>{}, :supplier_description=>{}, :pub_date_sort => {:index => :untokenized_omit_norms, :term_vector => :no}, :location_id => {:index => :untokenized}, :tab => {:index => :untokenized}},:remote => true}) I get the following error when starting ferret server (and also when starting mongrel) /opt/local/lib/ruby/1.8/drb/drb.rb:736:in `open': druby://localhost:9010 - # (DRb::DRbConnError) from /opt/local/lib/ruby/1.8/drb/drb.rb:729:in `each' from /opt/local/lib/ruby/1.8/drb/drb.rb:729:in `open' from /opt/local/lib/ruby/1.8/drb/drb.rb:1189:in `initialize' from /opt/local/lib/ruby/1.8/drb/drb.rb:1169:in `new' from /opt/local/lib/ruby/1.8/drb/drb.rb:1169:in `open' from /opt/local/lib/ruby/1.8/drb/drb.rb:1085:in `method_missing' from /opt/local/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend' from /opt/local/lib/ruby/1.8/drb/drb.rb:1084:in `method_missing' ... 29 levels... from /opt/local/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/runner.rb:39 from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require' from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in `require' from script/runner:3 When I change the :remote back to false, both Mongrel as DRB start without an error message. -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Jul 25 08:39:52 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 25 Jul 2007 14:39:52 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> Message-ID: <20070725123952.GD27557@cordoba.webit.de> Hi! Do you use the latest acts_as_ferret (0.4.1)? Jens On Wed, Jul 25, 2007 at 01:04:01PM +0200, Izit Izit wrote: > Hi, > > I have my Model as follows: > > class Mutation < ActiveRecord::Base > acts_as_ferret ({:fields => {:description=>{}, > :product_id=>{:index => :untokenized}, > :product_description=>{}, > :product_label_description=>{}, > :product_label_free=>{}, > :product_product_id_supplier=>{}, > :product_description_supplier=>{}, > :supplier_description=>{}, > :pub_date_sort => {:index => :untokenized_omit_norms, :term_vector => > :no}, > :location_id => {:index => :untokenized}, > :tab => {:index => :untokenized}},:remote => false}) > > THis will do the ferret locally without the DRB server. Works ok. > > If I change this to: > > class Mutation < ActiveRecord::Base > acts_as_ferret ({:fields => {:description=>{}, > :product_id=>{:index => :untokenized}, > :product_description=>{}, > :product_label_description=>{}, > :product_label_free=>{}, > :product_product_id_supplier=>{}, > :product_description_supplier=>{}, > :supplier_description=>{}, > :pub_date_sort => {:index => :untokenized_omit_norms, :term_vector => > :no}, > :location_id => {:index => :untokenized}, > :tab => {:index => :untokenized}},:remote => true}) > > I get the following error when starting ferret server (and also when > starting mongrel) > > /opt/local/lib/ruby/1.8/drb/drb.rb:736:in `open': druby://localhost:9010 > - # > (DRb::DRbConnError) > from /opt/local/lib/ruby/1.8/drb/drb.rb:729:in `each' > from /opt/local/lib/ruby/1.8/drb/drb.rb:729:in `open' > from /opt/local/lib/ruby/1.8/drb/drb.rb:1189:in `initialize' > from /opt/local/lib/ruby/1.8/drb/drb.rb:1169:in `new' > from /opt/local/lib/ruby/1.8/drb/drb.rb:1169:in `open' > from /opt/local/lib/ruby/1.8/drb/drb.rb:1085:in `method_missing' > from /opt/local/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend' > from /opt/local/lib/ruby/1.8/drb/drb.rb:1084:in `method_missing' > ... 29 levels... > from > /opt/local/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/runner.rb:39 > from > /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in > `gem_original_require' > from > /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in > `require' > from script/runner:3 > > > When I change the :remote back to false, both Mongrel as DRB start > without an error message. > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From portie at bitterbal.org Wed Jul 25 08:55:10 2007 From: portie at bitterbal.org (Izit Izit) Date: Wed, 25 Jul 2007 14:55:10 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <20070725123952.GD27557@cordoba.webit.de> References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> Message-ID: I am running revision 217 of the AAF plugin Strange thing is that I have another model where remote => true is configured but this does not give an issue, only this specific model generates the problem as soon as remote is set to true. The other model looks like this: acts_as_ferret ({:fields => [:description,:label_description,:label_free,:product_id_supplier,:description_supplier,:supplier_description] , :remote => true}) Jens Kraemer wrote: > Hi! > > > Do you use the latest acts_as_ferret (0.4.1)? > > Jens > > > On Wed, Jul 25, 2007 at 01:04:01PM +0200, Izit Izit wrote: >> :product_product_id_supplier=>{}, >> >> :no}, >> from /opt/local/lib/ruby/1.8/drb/drb.rb:729:in `open' >> /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in >> Posted via http://www.ruby-forum.com/. >> _______________________________________________ >> Ferret-talk mailing list >> Ferret-talk at rubyforge.org >> http://rubyforge.org/mailman/listinfo/ferret-talk >> > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From portie at bitterbal.org Wed Jul 25 08:57:58 2007 From: portie at bitterbal.org (Izit Izit) Date: Wed, 25 Jul 2007 14:57:58 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> Message-ID: <64c3a0c968303820228a6787784e67c7@ruby-forum.com> Some more info. What I also find strange is that when I start Mongrel it shows: => Booting Mongrel (use 'script/server webrick' to force WEBrick) => Rails application starting on http://0.0.0.0:3000 => Call with -d to detach => Ctrl-C to shutdown server ** Starting Mongrel listening at 0.0.0.0:3000 ** Starting Rails with development environment... Using AdapterExtensions Asked for a remote server ? false, ENV["FERRET_USE_LOCAL_INDEX"] is nil, looks like we are not the server Will use local index. using index in script/../config/../index/development/mutation default field list: [:supplier_description, :description, :product_description, :pub_date_sort, :product_label_description, :product_label_free, :product_product_id_supplier, :product_description_supplier] ** Rails loaded. ** Loading any Rails specific GemPlugins ** Signals ready. TERM => stop. USR2 => restart. INT => stop (no restart). ** Rails signals registered. HUP => reload (without restart). It might not work well. ** Mongrel available at 0.0.0.0:3000 ** Use CTRL-C to stop. Why is it refering to AAF during startup. It only does this for the Model I have the issue with, other models with AAF configured do not show up during Mongrel startup. I have also check it with webrick and some occurs. Izit Izit wrote: > I am running revision 217 of the AAF plugin > > Strange thing is that I have another model where remote => true is > configured but this does not give an issue, only this specific model > generates the problem as soon as remote is set to true. > > The other model looks like this: > > acts_as_ferret ({:fields => > [:description,:label_description,:label_free,:product_id_supplier,:description_supplier,:supplier_description] > , :remote => true}) > > > > > Jens Kraemer wrote: >> Hi! >> >> >> Do you use the latest acts_as_ferret (0.4.1)? >> >> Jens >> >> >> On Wed, Jul 25, 2007 at 01:04:01PM +0200, Izit Izit wrote: >>> :product_product_id_supplier=>{}, >>> >>> :no}, >>> from /opt/local/lib/ruby/1.8/drb/drb.rb:729:in `open' >>> /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in >>> Posted via http://www.ruby-forum.com/. >>> _______________________________________________ >>> Ferret-talk mailing list >>> Ferret-talk at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/ferret-talk >>> >> >> -- >> Jens Kr?mer >> webit! Gesellschaft f?r neue Medien mbH >> Schnorrstra?e 76 | 01069 Dresden >> Telefon +49 351 46766-0 | Telefax +49 351 46766-66 >> kraemer at webit.de | www.webit.de >> >> Amtsgericht Dresden | HRB 15422 >> GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From portie at bitterbal.org Wed Jul 25 09:41:43 2007 From: portie at bitterbal.org (Izit Izit) Date: Wed, 25 Jul 2007 15:41:43 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <64c3a0c968303820228a6787784e67c7@ruby-forum.com> References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> Message-ID: AFter some debugging, I got it working: acts_as_ferret (:fields => {:description=>{}, :product_id=>{:index => :untokenized}, :product_description=>{}, :product_label_description=>{}, :product_label_free=>{}, :product_product_id_supplier=>{}, :product_description_supplier=>{}, :supplier_description=>{}, :pub_date_sort => {:index => :untokenized_omit_norms, :term_vector => :no}, :location_id => {:index => :untokenized}, :tab => {:index => :untokenized}},:remote=>true) Small difference in usage of {} Izit Izit wrote: > Some more info. > > What I also find strange is that when I start Mongrel it shows: > > => Booting Mongrel (use 'script/server webrick' to force WEBrick) > => Rails application starting on http://0.0.0.0:3000 > => Call with -d to detach > => Ctrl-C to shutdown server > ** Starting Mongrel listening at 0.0.0.0:3000 > ** Starting Rails with development environment... > Using AdapterExtensions > Asked for a remote server ? false, ENV["FERRET_USE_LOCAL_INDEX"] is nil, > looks like we are not the server > Will use local index. > using index in script/../config/../index/development/mutation > default field list: [:supplier_description, :description, > :product_description, :pub_date_sort, :product_label_description, > :product_label_free, :product_product_id_supplier, > :product_description_supplier] > ** Rails loaded. > ** Loading any Rails specific GemPlugins > ** Signals ready. TERM => stop. USR2 => restart. INT => stop (no > restart). > ** Rails signals registered. HUP => reload (without restart). It might > not work well. > ** Mongrel available at 0.0.0.0:3000 > ** Use CTRL-C to stop. > > Why is it refering to AAF during startup. It only does this for the > Model I have the issue with, other models with AAF configured do not > show up during Mongrel startup. I have also check it with webrick and > some occurs. > > > > Izit Izit wrote: >> I am running revision 217 of the AAF plugin >> >> Strange thing is that I have another model where remote => true is >> configured but this does not give an issue, only this specific model >> generates the problem as soon as remote is set to true. >> >> The other model looks like this: >> >> acts_as_ferret ({:fields => >> [:description,:label_description,:label_free,:product_id_supplier,:description_supplier,:supplier_description] >> , :remote => true}) >> >> >> >> >> Jens Kraemer wrote: >>> Hi! >>> >>> >>> Do you use the latest acts_as_ferret (0.4.1)? >>> >>> Jens >>> >>> >>> On Wed, Jul 25, 2007 at 01:04:01PM +0200, Izit Izit wrote: >>>> :product_product_id_supplier=>{}, >>>> >>>> :no}, >>>> from /opt/local/lib/ruby/1.8/drb/drb.rb:729:in `open' >>>> /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:27:in >>>> Posted via http://www.ruby-forum.com/. >>>> _______________________________________________ >>>> Ferret-talk mailing list >>>> Ferret-talk at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/ferret-talk >>>> >>> >>> -- >>> Jens Kr?mer >>> webit! Gesellschaft f?r neue Medien mbH >>> Schnorrstra?e 76 | 01069 Dresden >>> Telefon +49 351 46766-0 | Telefax +49 351 46766-66 >>> kraemer at webit.de | www.webit.de >>> >>> Amtsgericht Dresden | HRB 15422 >>> GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Jul 25 09:44:13 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 25 Jul 2007 15:44:13 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <64c3a0c968303820228a6787784e67c7@ruby-forum.com> References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> Message-ID: <20070725134413.GE27557@cordoba.webit.de> On Wed, Jul 25, 2007 at 02:57:58PM +0200, Izit Izit wrote: > Some more info. > > What I also find strange is that when I start Mongrel it shows: > > => Booting Mongrel (use 'script/server webrick' to force WEBrick) > => Rails application starting on http://0.0.0.0:3000 > => Call with -d to detach > => Ctrl-C to shutdown server > ** Starting Mongrel listening at 0.0.0.0:3000 > ** Starting Rails with development environment... > Using AdapterExtensions > Asked for a remote server ? false, ENV["FERRET_USE_LOCAL_INDEX"] is nil, > looks like we are not the server > Will use local index. > using index in script/../config/../index/development/mutation > default field list: [:supplier_description, :description, > :product_description, :pub_date_sort, :product_label_description, > :product_label_free, :product_product_id_supplier, > :product_description_supplier] > ** Rails loaded. > ** Loading any Rails specific GemPlugins > ** Signals ready. TERM => stop. USR2 => restart. INT => stop (no > restart). > ** Rails signals registered. HUP => reload (without restart). It might > not work well. > ** Mongrel available at 0.0.0.0:3000 > ** Use CTRL-C to stop. > > Why is it refering to AAF during startup. It only does this for the > Model I have the issue with, other models with AAF configured do not > show up during Mongrel startup. I have also check it with webrick and > some occurs. This output should go to the log file normally, not sure why you see it on stdout. However it looks like you didn't specify :remote => true in this case (that's the "Asked for a remote server?" line). Interesting would be the output and log files from starting the DRb server. Most of the time these problems stem from the DRb server not realizing that it *is* the DRb server, resulting in aaf inside the DRb server trying to connect to another (of course non-existent) DRb server. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Wed Jul 25 09:45:18 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 25 Jul 2007 15:45:18 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> Message-ID: <20070725134518.GF27557@cordoba.webit.de> On Wed, Jul 25, 2007 at 03:41:43PM +0200, Izit Izit wrote: > AFter some debugging, I got it working: > > acts_as_ferret (:fields => {:description=>{}, > :product_id=>{:index => :untokenized}, > :product_description=>{}, > :product_label_description=>{}, > :product_label_free=>{}, > :product_product_id_supplier=>{}, > :product_description_supplier=>{}, > :supplier_description=>{}, > :pub_date_sort => {:index => :untokenized_omit_norms, :term_vector => > :no}, > :location_id => {:index => :untokenized}, > :tab => {:index => :untokenized}},:remote=>true) > > Small difference in usage of {} Argh :-) -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From portie at bitterbal.org Wed Jul 25 13:38:52 2007 From: portie at bitterbal.org (Izit Izit) Date: Wed, 25 Jul 2007 19:38:52 +0200 Subject: [Ferret-talk] svn down Message-ID: <942a5904e6a125d25405a811dfb104d8@ruby-forum.com> I tried to checkout AAF from the repository but get: Fetching external item into 'trunk/vendor/plugins/acts_as_ferret' svn: Can't connect to host 'projects.jkraemer.net': Connection refused Anyone else experiencing the same ? -- Posted via http://www.ruby-forum.com/. From jmckible at gmail.com Wed Jul 25 14:50:49 2007 From: jmckible at gmail.com (Jordan McKible) Date: Wed, 25 Jul 2007 20:50:49 +0200 Subject: [Ferret-talk] svn down In-Reply-To: <942a5904e6a125d25405a811dfb104d8@ruby-forum.com> References: <942a5904e6a125d25405a811dfb104d8@ruby-forum.com> Message-ID: I'm also having trouble with the plugin install via svn. Trac seems to be up, so perhaps there's another address that will work. I will play with it. -- Posted via http://www.ruby-forum.com/. From james.herdman at gmail.com Wed Jul 25 19:46:39 2007 From: james.herdman at gmail.com (James Herdman) Date: Thu, 26 Jul 2007 01:46:39 +0200 Subject: [Ferret-talk] Acts_As_Ferret: disable_ferret and Callbacks In-Reply-To: <20070725093625.GA27557@cordoba.webit.de> References: <875b9c87521c773132de43812739ac1f@ruby-forum.com> <20070725093625.GA27557@cordoba.webit.de> Message-ID: Jens Kraemer wrote: > On Mon, Jul 23, 2007 at 08:34:31PM +0200, James Herdman wrote: >> Does disable_ferret monkey with callbacks at all? I have a model in my >> application that has associated models which perform callbacks after >> their validation (they're all instantiated around the same time). I >> recently added a conditional call to disable ferret provided a flag and >> I'm finding these callbacks aren't quite working right. > > what exactly does this mean? Is Ferret not disabled, or are your > callbacks never called? The callbacks weren't firing. It doesn't matter though. I've since approached the problem from a different angle. -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Thu Jul 26 02:32:32 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 26 Jul 2007 08:32:32 +0200 Subject: [Ferret-talk] svn down In-Reply-To: References: <942a5904e6a125d25405a811dfb104d8@ruby-forum.com> Message-ID: <20070726063232.GK26963@thunder.jkraemer.net> On Wed, Jul 25, 2007 at 08:50:49PM +0200, Jordan McKible wrote: > I'm also having trouble with the plugin install via svn. Trac seems to > be up, so perhaps there's another address that will work. I will play > with it. SVN is up again. Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Thu Jul 26 02:36:14 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 26 Jul 2007 08:36:14 +0200 Subject: [Ferret-talk] Act as Ferret supports Conditional search? In-Reply-To: <87b412ce0707242339oe3acb4n6ec9d474a30b27e3@mail.gmail.com> References: <87b412ce0707240033y3f72a1bcr6a77e74f92512b3b@mail.gmail.com> <20070724084749.GB26963@thunder.jkraemer.net> <87b412ce0707242339oe3acb4n6ec9d474a30b27e3@mail.gmail.com> Message-ID: <20070726063614.GL26963@thunder.jkraemer.net> On Tue, Jul 24, 2007 at 11:39:15PM -0700, isha kakodkar wrote: > thanx..i actually did it using the :conditions with find_by_contents. > But when i do find_id_by_contents,which searches from the filesystem,can i > search within data based on this index value?and not just the full data > field? yes, every attribute you index is stored in it's own Ferret field, and you can run queries (or parts of queries) against just one Ferret field by specifying the field name like I did in the example below. Jens > > On 7/24/07, Jens Kraemer wrote: > > > >On Tue, Jul 24, 2007 at 12:33:47AM -0700, isha kakodkar wrote: > >> Hi all, > >> Im a newbie to ferret.I have installed the act_as_ferret gem and i > >> actually want to search for some content in the model,but per user. > >> My model has data,user_id.So within the data column i have to serach for > >a > >> word if it exists for a given user. > >> So can i achieve that ? presently i know that it just searches the > >"data" > >> column irrespective of the user given. > >> Can someone help me with this? > > > >Index the user_id field (as untokenized) and append " user_id:#{user.id}" > >to the query. > > > >cheers, > >Jens > > > > > >-- > >Jens Kr?mer > >http://www.jkraemer.net/ - Blog > >http://www.omdb.org/ - The new free film database > >_______________________________________________ > >Ferret-talk mailing list > >Ferret-talk at rubyforge.org > >http://rubyforge.org/mailman/listinfo/ferret-talk > > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From ed.temp.01 at gmail.com Thu Jul 26 02:43:36 2007 From: ed.temp.01 at gmail.com (Ed Ed) Date: Thu, 26 Jul 2007 08:43:36 +0200 Subject: [Ferret-talk] Ferret - current status? Message-ID: <69c1b1491fcaa44e7799dd5e1639674d@ruby-forum.com> Hi guys, Having committed a fairly large project to ferret I'm a little concerned that ferret svn has been essentially unavailable for weeks (pretty much every time I try I get "can't connect") and more so now that davebalmain.com has gone off the air. Without meaning to pry, does anyone know whether existing problems in ferret are likely to get fixed? (I can get crashes quite easily with a combination of highlighting and proximity searches over longish distances) and whether, or if, there will be another release? -- Posted via http://www.ruby-forum.com/. From portie at bitterbal.org Thu Jul 26 03:14:36 2007 From: portie at bitterbal.org (Izit Izit) Date: Thu, 26 Jul 2007 09:14:36 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <20070725134518.GF27557@cordoba.webit.de> References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> <20070725134518.GF27557@cordoba.webit.de> Message-ID: The same problem is back again. I moved my development app to production (dev = OS X, prod = Linux). started drb server and same problem occurs again. Jens Kraemer wrote: > On Wed, Jul 25, 2007 at 03:41:43PM +0200, Izit Izit wrote: >> :pub_date_sort => {:index => :untokenized_omit_norms, :term_vector => >> :no}, >> :location_id => {:index => :untokenized}, >> :tab => {:index => :untokenized}},:remote=>true) >> >> Small difference in usage of {} > > Argh :-) > > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Thu Jul 26 03:29:06 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 26 Jul 2007 09:29:06 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> <20070725134518.GF27557@cordoba.webit.de> Message-ID: <20070726072905.GP26963@thunder.jkraemer.net> On Thu, Jul 26, 2007 at 09:14:36AM +0200, Izit Izit wrote: > The same problem is back again. I moved my development app to production > (dev = OS X, prod = Linux). started drb server and same problem occurs > again. So what does the DRb server log when it's started? Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Thu Jul 26 03:31:12 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 26 Jul 2007 09:31:12 +0200 Subject: [Ferret-talk] Ferret - current status? In-Reply-To: <69c1b1491fcaa44e7799dd5e1639674d@ruby-forum.com> References: <69c1b1491fcaa44e7799dd5e1639674d@ruby-forum.com> Message-ID: <20070726073112.GQ26963@thunder.jkraemer.net> On Thu, Jul 26, 2007 at 08:43:36AM +0200, Ed Ed wrote: > Hi guys, > > Having committed a fairly large project to ferret I'm a little concerned > that ferret svn has been essentially unavailable for weeks (pretty much > every time I try I get "can't connect") and more so now that > davebalmain.com has gone off the air. > > Without meaning to pry, does anyone know whether existing problems in > ferret are likely to get fixed? (I can get crashes quite easily with a > combination of highlighting and proximity searches over longish > distances) and whether, or if, there will be another release? I just mailed Dave with pretty much the same questions - I'll keep you up to date about this. Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From portie at bitterbal.org Thu Jul 26 03:38:49 2007 From: portie at bitterbal.org (Izit Izit) Date: Thu, 26 Jul 2007 09:38:49 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <20070726072905.GP26963@thunder.jkraemer.net> References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> <20070725134518.GF27557@cordoba.webit.de> <20070726072905.GP26963@thunder.jkraemer.net> Message-ID: <3f338318f54b3c9e276e616e90810261@ruby-forum.com> cli command outputs: /usr/lib/ruby/1.8/drb/drb.rb:736:in `open': druby://localhost:9010 - # (DRb::DRbConnError) from /usr/lib/ruby/1.8/drb/drb.rb:729:in `each' from /usr/lib/ruby/1.8/drb/drb.rb:729:in `open' from /usr/lib/ruby/1.8/drb/drb.rb:1189:in `initialize' from /usr/lib/ruby/1.8/drb/drb.rb:1169:in `new' from /usr/lib/ruby/1.8/drb/drb.rb:1169:in `open' from /usr/lib/ruby/1.8/drb/drb.rb:1085:in `method_missing' from /usr/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend' from /usr/lib/ruby/1.8/drb/drb.rb:1084:in `method_missing' ... 30 levels... from /usr/lib64/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/runner.rb:39 from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require' from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require' from script/runner:3 Jens Kraemer wrote: > On Thu, Jul 26, 2007 at 09:14:36AM +0200, Izit Izit wrote: >> The same problem is back again. I moved my development app to production >> (dev = OS X, prod = Linux). started drb server and same problem occurs >> again. > > So what does the DRb server log when it's started? > > Jens > -- > Jens Kr?mer > http://www.jkraemer.net/ - Blog > http://www.omdb.org/ - The new free film database -- Posted via http://www.ruby-forum.com/. From portie at bitterbal.org Thu Jul 26 03:42:24 2007 From: portie at bitterbal.org (Izit Izit) Date: Thu, 26 Jul 2007 09:42:24 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <3f338318f54b3c9e276e616e90810261@ruby-forum.com> References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> <20070725134518.GF27557@cordoba.webit.de> <20070726072905.GP26963@thunder.jkraemer.net> <3f338318f54b3c9e276e616e90810261@ruby-forum.com> Message-ID: <28255c44bd85f5f33f22ccb95fd339b5@ruby-forum.com> Some experimenting: In my problem model I put remote to false. Started ferret drb server. When startup was done changed remote to true in the problem model. I now stopped ferret drb server. On the stop cli command the drb server started building the index of the problem model. Not sure if it is related but a bit strange I thought. :-) Izit Izit wrote: > cli command outputs: > > /usr/lib/ruby/1.8/drb/drb.rb:736:in `open': druby://localhost:9010 - > # > (DRb::DRbConnError) > from /usr/lib/ruby/1.8/drb/drb.rb:729:in `each' > from /usr/lib/ruby/1.8/drb/drb.rb:729:in `open' > from /usr/lib/ruby/1.8/drb/drb.rb:1189:in `initialize' > from /usr/lib/ruby/1.8/drb/drb.rb:1169:in `new' > from /usr/lib/ruby/1.8/drb/drb.rb:1169:in `open' > from /usr/lib/ruby/1.8/drb/drb.rb:1085:in `method_missing' > from /usr/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend' > from /usr/lib/ruby/1.8/drb/drb.rb:1084:in `method_missing' > ... 30 levels... > from > /usr/lib64/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/runner.rb:39 > from > /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in > `gem_original_require' > from > /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require' > from script/runner:3 > > > Jens Kraemer wrote: >> On Thu, Jul 26, 2007 at 09:14:36AM +0200, Izit Izit wrote: >>> The same problem is back again. I moved my development app to production >>> (dev = OS X, prod = Linux). started drb server and same problem occurs >>> again. >> >> So what does the DRb server log when it's started? >> >> Jens >> -- >> Jens Kr?mer >> http://www.jkraemer.net/ - Blog >> http://www.omdb.org/ - The new free film database -- Posted via http://www.ruby-forum.com/. From portie at bitterbal.org Thu Jul 26 03:57:32 2007 From: portie at bitterbal.org (Izit Izit) Date: Thu, 26 Jul 2007 09:57:32 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <28255c44bd85f5f33f22ccb95fd339b5@ruby-forum.com> References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> <20070725134518.GF27557@cordoba.webit.de> <20070726072905.GP26963@thunder.jkraemer.net> <3f338318f54b3c9e276e616e90810261@ruby-forum.com> <28255c44bd85f5f33f22ccb95fd339b5@ruby-forum.com> Message-ID: Maybe this is a way to the solution: Observers. for my problem model 'Mutation' I also have configured an observer. With the observer defined and remote set to true in the model, DRB server does not start. If I remove the observer statement, the DRB server starts without a problem. This may also explain why mongrel has the AAF log messages during startup because it is accessing the Model because of the observer statement. Not sure how to fix it though. Izit Izit wrote: > Some experimenting: > > In my problem model I put remote to false. Started ferret drb server. > When startup was done changed remote to true in the problem model. I now > stopped ferret drb server. On the stop cli command the drb server > started building the index of the problem model. > > Not sure if it is related but a bit strange I thought. :-) > > Izit Izit wrote: >> cli command outputs: >> >> /usr/lib/ruby/1.8/drb/drb.rb:736:in `open': druby://localhost:9010 - >> # >> (DRb::DRbConnError) >> from /usr/lib/ruby/1.8/drb/drb.rb:729:in `each' >> from /usr/lib/ruby/1.8/drb/drb.rb:729:in `open' >> from /usr/lib/ruby/1.8/drb/drb.rb:1189:in `initialize' >> from /usr/lib/ruby/1.8/drb/drb.rb:1169:in `new' >> from /usr/lib/ruby/1.8/drb/drb.rb:1169:in `open' >> from /usr/lib/ruby/1.8/drb/drb.rb:1085:in `method_missing' >> from /usr/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend' >> from /usr/lib/ruby/1.8/drb/drb.rb:1084:in `method_missing' >> ... 30 levels... >> from >> /usr/lib64/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/runner.rb:39 >> from >> /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in >> `gem_original_require' >> from >> /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require' >> from script/runner:3 >> >> >> Jens Kraemer wrote: >>> On Thu, Jul 26, 2007 at 09:14:36AM +0200, Izit Izit wrote: >>>> The same problem is back again. I moved my development app to production >>>> (dev = OS X, prod = Linux). started drb server and same problem occurs >>>> again. >>> >>> So what does the DRb server log when it's started? >>> >>> Jens >>> -- >>> Jens Kr?mer >>> http://www.jkraemer.net/ - Blog >>> http://www.omdb.org/ - The new free film database -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Thu Jul 26 04:19:44 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 26 Jul 2007 10:19:44 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: References: <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> <20070725134518.GF27557@cordoba.webit.de> <20070726072905.GP26963@thunder.jkraemer.net> <3f338318f54b3c9e276e616e90810261@ruby-forum.com> <28255c44bd85f5f33f22ccb95fd339b5@ruby-forum.com> Message-ID: <20070726081944.GC17382@cordoba.webit.de> On Thu, Jul 26, 2007 at 09:57:32AM +0200, Izit Izit wrote: > Maybe this is a way to the solution: Observers. > > for my problem model 'Mutation' I also have configured an observer. With > the observer defined and remote set to true in the model, DRB server > does not start. If I remove the observer statement, the DRB server > starts without a problem. > > This may also explain why mongrel has the AAF log messages during > startup because it is accessing the Model because of the observer > statement. Yeah this problem is known - when the model gets loaded during rails startup (the DRb server runs in a full Rails environment) and therefore before the server itself is started, aaf inside the model has no way to know it is running inside the server and so tries to connect to the server which of course isn't there yet. However recent aaf (0.4.1 or trunk) should not have this problem, it uses an environment variable set in script/ferret_start to tell aaf it shouldn't try to connect to DRb. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From portie at bitterbal.org Thu Jul 26 04:31:17 2007 From: portie at bitterbal.org (Izit Izit) Date: Thu, 26 Jul 2007 10:31:17 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <20070726081944.GC17382@cordoba.webit.de> References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> <20070725134518.GF27557@cordoba.webit.de> <20070726072905.GP26963@thunder.jkraemer.net> <3f338318f54b3c9e276e616e90810261@ruby-forum.com> <28255c44bd85f5f33f22ccb95fd339b5@ruby-forum.com> <20070726081944.GC17382@cordoba.webit.de> Message-ID: <914cc00d595e047c37ad9c9ac37c98e2@ruby-forum.com> I got it working now but starting the DRB server as follows: FERRET_USE_LOCAL_INDEX=1 RAILS_ENV=production script/ferret_start > > Yeah this problem is known - when the model gets loaded during rails > startup (the DRb server runs in a full Rails environment) and therefore > before the server itself is started, aaf inside the model has no way to > know it is running inside the server and so tries to connect to the > server which of course isn't there yet. However recent aaf (0.4.1 or > trunk) should not have this problem, it uses an environment variable set > in script/ferret_start to tell aaf it shouldn't try to connect to DRb. > > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Thu Jul 26 04:42:32 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 26 Jul 2007 10:42:32 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <914cc00d595e047c37ad9c9ac37c98e2@ruby-forum.com> References: <64c3a0c968303820228a6787784e67c7@ruby-forum.com> <20070725134518.GF27557@cordoba.webit.de> <20070726072905.GP26963@thunder.jkraemer.net> <3f338318f54b3c9e276e616e90810261@ruby-forum.com> <28255c44bd85f5f33f22ccb95fd339b5@ruby-forum.com> <20070726081944.GC17382@cordoba.webit.de> <914cc00d595e047c37ad9c9ac37c98e2@ruby-forum.com> Message-ID: <20070726084232.GB589@thunder.jkraemer.net> On Thu, Jul 26, 2007 at 10:31:17AM +0200, Izit Izit wrote: > I got it working now but starting the DRB server as follows: > > FERRET_USE_LOCAL_INDEX=1 RAILS_ENV=production script/ferret_start yep, and that's exactly what recent versions of the ferret_start script do: ENV['FERRET_USE_LOCAL_INDEX'] = 'true' require File.dirname(__FILE__) + '/../config/boot' require RAILS_ROOT + '/config/environment' ... Are you sure you have that first line in your ferret_start script? Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From portie at bitterbal.org Thu Jul 26 04:49:37 2007 From: portie at bitterbal.org (Izit Izit) Date: Thu, 26 Jul 2007 10:49:37 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <20070726084232.GB589@thunder.jkraemer.net> References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> <20070725134518.GF27557@cordoba.webit.de> <20070726072905.GP26963@thunder.jkraemer.net> <3f338318f54b3c9e276e616e90810261@ruby-forum.com> <28255c44bd85f5f33f22ccb95fd339b5@ruby-forum.com> <20070726081944.GC17382@cordoba.webit.de> <914cc00d595e047c37ad9c9ac37c98e2@ruby-forum.com> <20070726084232.GB589@thunder.jkraemer.net> Message-ID: <1575320cf292f6de2a0acb5ecec73ac8@ruby-forum.com> The ferret_start in RAILS_ROOT/script directory does not have those lines but the vendor/plugin/acts_as_ferret/script/ferret_start does. I have done a fresh svn install of the plugin this morning. Jens Kraemer wrote: > On Thu, Jul 26, 2007 at 10:31:17AM +0200, Izit Izit wrote: >> I got it working now but starting the DRB server as follows: >> >> FERRET_USE_LOCAL_INDEX=1 RAILS_ENV=production script/ferret_start > > yep, and that's exactly what recent versions of the ferret_start script > do: > > ENV['FERRET_USE_LOCAL_INDEX'] = 'true' > require File.dirname(__FILE__) + '/../config/boot' > require RAILS_ROOT + '/config/environment' > ... > > Are you sure you have that first line in your ferret_start script? > > Jens > > > -- > Jens Kr?mer > http://www.jkraemer.net/ - Blog > http://www.omdb.org/ - The new free film database -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Thu Jul 26 05:22:28 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 26 Jul 2007 11:22:28 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <1575320cf292f6de2a0acb5ecec73ac8@ruby-forum.com> References: <20070725134518.GF27557@cordoba.webit.de> <20070726072905.GP26963@thunder.jkraemer.net> <3f338318f54b3c9e276e616e90810261@ruby-forum.com> <28255c44bd85f5f33f22ccb95fd339b5@ruby-forum.com> <20070726081944.GC17382@cordoba.webit.de> <914cc00d595e047c37ad9c9ac37c98e2@ruby-forum.com> <20070726084232.GB589@thunder.jkraemer.net> <1575320cf292f6de2a0acb5ecec73ac8@ruby-forum.com> Message-ID: <20070726092228.GE17382@cordoba.webit.de> On Thu, Jul 26, 2007 at 10:49:37AM +0200, Izit Izit wrote: > The ferret_start in RAILS_ROOT/script directory does not have those > lines but the vendor/plugin/acts_as_ferret/script/ferret_start does. > > I have done a fresh svn install of the plugin this morning. hm, svn install is supposed to copy those files from vendor/plugin/acts_as_ferret/script/ to script/. Might be that it fails if the target file already exists - I'll fix that so a warning is issued in this case. Or maybe permissions on script/ferret_start just didn't allow overwriting it? Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From portie at bitterbal.org Thu Jul 26 05:43:53 2007 From: portie at bitterbal.org (Izit Izit) Date: Thu, 26 Jul 2007 11:43:53 +0200 Subject: [Ferret-talk] DRb not starting In-Reply-To: <20070726092228.GE17382@cordoba.webit.de> References: <1242e837302d9fe66c70f4c5355a8ff8@ruby-forum.com> <20070725123952.GD27557@cordoba.webit.de> <64c3a0c968303820228a6787784e67c7@ruby-forum.com> <20070725134518.GF27557@cordoba.webit.de> <20070726072905.GP26963@thunder.jkraemer.net> <3f338318f54b3c9e276e616e90810261@ruby-forum.com> <28255c44bd85f5f33f22ccb95fd339b5@ruby-forum.com> <20070726081944.GC17382@cordoba.webit.de> <914cc00d595e047c37ad9c9ac37c98e2@ruby-forum.com> <20070726084232.GB589@thunder.jkraemer.net> <1575320cf292f6de2a0acb5ecec73ac8@ruby-forum.com> <20070726092228.GE17382@cordoba.webit.de> Message-ID: permissions are ok. thanks for all the help Jens Kraemer wrote: > On Thu, Jul 26, 2007 at 10:49:37AM +0200, Izit Izit wrote: >> The ferret_start in RAILS_ROOT/script directory does not have those >> lines but the vendor/plugin/acts_as_ferret/script/ferret_start does. >> >> I have done a fresh svn install of the plugin this morning. > > hm, svn install is supposed to copy those files from > vendor/plugin/acts_as_ferret/script/ to script/. Might be that it fails > if the target file already exists - I'll fix that so a warning is issued > in this case. Or maybe permissions on script/ferret_start just didn't > allow overwriting it? > > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From golak.sarangi at gmail.com Thu Jul 26 10:01:18 2007 From: golak.sarangi at gmail.com (golak Sarangi) Date: Thu, 26 Jul 2007 19:31:18 +0530 Subject: [Ferret-talk] doubts in ferret Message-ID: <3854b3a40707260701k6f4f1eb2pc3341fd8de7ff8b1@mail.gmail.com> I am using ferret to build a search application for my site. I used stemming analyzer to build the index. When i searched "market" i get hits but on searching "marketing" i get no hits,while there are fields containing the word marketing. I am using stemming analyzer even while searching. Is the problem with the analyzer? Or am I missing out something -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070726/7d426aa8/attachment.html From andreas.korth at gmx.net Thu Jul 26 13:25:23 2007 From: andreas.korth at gmx.net (Andreas Korth) Date: Thu, 26 Jul 2007 19:25:23 +0200 Subject: [Ferret-talk] doubts in ferret In-Reply-To: <3854b3a40707260701k6f4f1eb2pc3341fd8de7ff8b1@mail.gmail.com> References: <3854b3a40707260701k6f4f1eb2pc3341fd8de7ff8b1@mail.gmail.com> Message-ID: <0E9F8477-E7B0-4BEB-A83A-E8FAA1A44581@gmx.net> On 26.07.2007, at 16:01, golak Sarangi wrote: > I am using ferret to build a search application for my site. I used > stemming analyzer to build the index. When i searched "market" i > get hits but on searching "marketing" i get no hits,while there are > fields containing the word marketing. I am using stemming analyzer > even while searching. Is the problem with the analyzer? Or am I > missing out something Doubt in yourself before you doubt in Ferret ;) Check if your stemming analyzer is actually used for both indexing and searching. Start with an empty index and add a simple document. Open an IndexReader on the index and examine the terms using IndexReader#terms (). The index should only contain stemmed terms. Make sure the stemming analyzer is used for searching as well. Call Index#process_query() with a query of your choice. This returns the parsed (and hence analyzed) query. Here, too, all terms should be stemmed. If either one doesn't work as expected, you have the wrong analyzer in place. HTH, Andy From nappin713 at yahoo.com Thu Jul 26 19:39:45 2007 From: nappin713 at yahoo.com (Raymond O'Connor) Date: Fri, 27 Jul 2007 01:39:45 +0200 Subject: [Ferret-talk] Fuzzy Searches with Multiple Terms Message-ID: I'm having trouble getting fuzzy searches to work with multiple search terms. Let's say searching 'foo' returns results, 'bar' returns results and 'foo bar' returns results. When I do this fou~ bar~ it returns nothing. This, however, works fine: fou~ bar I want the fuzzy search to apply to each word, and this seems to do the trick: fou+bar~ but I wanted to check if anyone can confirm this is correct because I thought the '+' meant that the term following '+' was required. Thanks -- Posted via http://www.ruby-forum.com/. From rick.ceo at lacharite.ca Fri Jul 27 00:56:34 2007 From: rick.ceo at lacharite.ca (Rick Lacharite) Date: Fri, 27 Jul 2007 06:56:34 +0200 Subject: [Ferret-talk] Mongrel won't start Message-ID: I have a win xp box running instant rails. I installed ferret (the win version) and acts_as_ferret seemingly succesfully. I then added "require 'acts_as_ferret'" to the environment.rb file of my project. However, I cannot get webrick/mongrel to start. Can anyone suggest a solution? Rick -- Posted via http://www.ruby-forum.com/. From myowntribe at yahoo.com Fri Jul 27 01:50:18 2007 From: myowntribe at yahoo.com (Cass Amino) Date: Fri, 27 Jul 2007 07:50:18 +0200 Subject: [Ferret-talk] Problem with installing acts_as_ferret plugin Message-ID: Hi all, when I type this: ruby script/plugin install svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret from my rails project the plugin is not getting installed, its simply going back to my rails application. I don't find the plugin in the vendor/plugin directory I am using Instant Rails for your information. Could anyone please let me know whats going wrong... Cheers Cass -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Jul 27 04:11:35 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 27 Jul 2007 10:11:35 +0200 Subject: [Ferret-talk] Problem with installing acts_as_ferret plugin In-Reply-To: References: Message-ID: <20070727081135.GB12090@cordoba.webit.de> On Fri, Jul 27, 2007 at 07:50:18AM +0200, Cass Amino wrote: > Hi all, > > when I type this: > > ruby script/plugin install > svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret > > from my rails project the plugin is not getting installed, its simply > going back to my rails application. > > I don't find the plugin in the vendor/plugin directory > > > I am using Instant Rails for your information. Could anyone please let > me know whats going wrong... Do yo have subversion installed? What happens if you do svn ls svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret on the command line? Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Fri Jul 27 04:13:12 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 27 Jul 2007 10:13:12 +0200 Subject: [Ferret-talk] Mongrel won't start In-Reply-To: References: Message-ID: <20070727081312.GC12090@cordoba.webit.de> On Fri, Jul 27, 2007 at 06:56:34AM +0200, Rick Lacharite wrote: > I have a win xp box running instant rails. I installed ferret (the win > version) and acts_as_ferret seemingly succesfully. I then added "require > 'acts_as_ferret'" to the environment.rb file of my project. However, I > cannot get webrick/mongrel to start. Can anyone suggest a solution? Hardly without any further info. Did you install aaf as a plugin or via ruby gems? the require is not needed if aaf resides in vendor/plugins inside your application. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From myowntribe at yahoo.com Fri Jul 27 04:37:41 2007 From: myowntribe at yahoo.com (Cass Amino) Date: Fri, 27 Jul 2007 10:37:41 +0200 Subject: [Ferret-talk] Problem with installing acts_as_ferret plugin In-Reply-To: <20070727081135.GB12090@cordoba.webit.de> References: <20070727081135.GB12090@cordoba.webit.de> Message-ID: <87ba8340451d1d2b9ea3a77eacaae1a8@ruby-forum.com> Hi Jens, I presume I have subversion installed as I didn't get any problems while installing has_many_friends plugin. When I type this at the command line: svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret it says: svn is not recognized as an internal or external command, operational program or batch file but if I type the full command: ruby script/plugin install svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret it simply returns the command prompt. I am totally new to ruby and to programming itself so its quite confusing... wonder if i need to install subversion as you asked? Please help Cass Jens Kraemer wrote: > On Fri, Jul 27, 2007 at 07:50:18AM +0200, Cass Amino wrote: >> I don't find the plugin in the vendor/plugin directory >> >> >> I am using Instant Rails for your information. Could anyone please let >> me know whats going wrong... > > Do yo have subversion installed? What happens if you do > svn ls > svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret > on the command line? > > > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Jul 27 04:44:45 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 27 Jul 2007 10:44:45 +0200 Subject: [Ferret-talk] Problem with installing acts_as_ferret plugin In-Reply-To: <87ba8340451d1d2b9ea3a77eacaae1a8@ruby-forum.com> References: <20070727081135.GB12090@cordoba.webit.de> <87ba8340451d1d2b9ea3a77eacaae1a8@ruby-forum.com> Message-ID: <20070727084445.GD12090@cordoba.webit.de> On Fri, Jul 27, 2007 at 10:37:41AM +0200, Cass Amino wrote: > Hi Jens, > > I presume I have subversion installed as I didn't get any problems while > installing has_many_friends plugin. > > When I type this at the command line: > > svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret > > it says: svn is not recognized as an internal or external command, > operational > program or batch file > > but if I type the full command: > > ruby script/plugin install > svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret > > it simply returns the command prompt. > > I am totally new to ruby and to programming itself so its quite > confusing... I'm not sure about the way these things work under windows, maybe script/plugin install can work even without having an svn executable anywhere. Sometimes the svn:// protocol makes problems with restrictive firewalls blocking outgoing connections. There's an export of the current stable version (0.4.1) attached to the wiki start page: http://projects.jkraemer.net/acts_as_ferret/ Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From myowntribe at yahoo.com Fri Jul 27 05:09:30 2007 From: myowntribe at yahoo.com (Cass Amino) Date: Fri, 27 Jul 2007 11:09:30 +0200 Subject: [Ferret-talk] Problem with installing acts_as_ferret plugin In-Reply-To: <20070727084445.GD12090@cordoba.webit.de> References: <20070727081135.GB12090@cordoba.webit.de> <87ba8340451d1d2b9ea3a77eacaae1a8@ruby-forum.com> <20070727084445.GD12090@cordoba.webit.de> Message-ID: Hi Jens Thanks for the reply, I downloaded the export from the wiki and copied the acts_as_ferret plugin into my vendor/plugin directory, but now I have a new problem, my webrick server for the project aint starting, it syas something about ruby_gems/custom_require.rb:32:in 'gem_original_require', If I remove the plugin from the vendor directory the server is starting normally. What could I have missed? Cheers Cass Jens Kraemer wrote: > On Fri, Jul 27, 2007 at 10:37:41AM +0200, Cass Amino wrote: >> operational >> confusing... > I'm not sure about the way these things work under windows, maybe > script/plugin install can work even without having an svn executable > anywhere. > > Sometimes the svn:// protocol makes problems with restrictive firewalls > blocking outgoing connections. There's an export of the current stable > version (0.4.1) attached to the wiki start page: > http://projects.jkraemer.net/acts_as_ferret/ > > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Jul 27 06:02:51 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 27 Jul 2007 12:02:51 +0200 Subject: [Ferret-talk] Problem with installing acts_as_ferret plugin In-Reply-To: References: <20070727081135.GB12090@cordoba.webit.de> <87ba8340451d1d2b9ea3a77eacaae1a8@ruby-forum.com> <20070727084445.GD12090@cordoba.webit.de> Message-ID: <20070727100251.GE12090@cordoba.webit.de> On Fri, Jul 27, 2007 at 11:09:30AM +0200, Cass Amino wrote: > Hi Jens > > Thanks for the reply, I downloaded the export from the wiki and copied > the acts_as_ferret plugin into my vendor/plugin directory, but now I > have a new problem, my webrick server for the project aint starting, it > syas something about ruby_gems/custom_require.rb:32:in > 'gem_original_require', > > If I remove the plugin from the vendor directory the server is starting > normally. > > What could I have missed? wild guess - did you install the ferret gem? Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From myowntribe at yahoo.com Fri Jul 27 06:12:03 2007 From: myowntribe at yahoo.com (Cass Amino) Date: Fri, 27 Jul 2007 12:12:03 +0200 Subject: [Ferret-talk] Problem with installing acts_as_ferret plugin In-Reply-To: <20070727100251.GE12090@cordoba.webit.de> References: <20070727081135.GB12090@cordoba.webit.de> <87ba8340451d1d2b9ea3a77eacaae1a8@ruby-forum.com> <20070727084445.GD12090@cordoba.webit.de> <20070727100251.GE12090@cordoba.webit.de> Message-ID: Jens Kraemer wrote: > On Fri, Jul 27, 2007 at 11:09:30AM +0200, Cass Amino wrote: >> >> What could I have missed? > > wild guess - did you install the ferret gem? > > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa Hi Jen, thanks for the reply, I just reinstalled ferret gem and its working fine. Thanks a lot for all the guidance. Cheers Cass -- Posted via http://www.ruby-forum.com/. From isha.kakodkar at gmail.com Fri Jul 27 07:13:39 2007 From: isha.kakodkar at gmail.com (isha kakodkar) Date: Fri, 27 Jul 2007 04:13:39 -0700 Subject: [Ferret-talk] Act as Ferret supports Conditional search? In-Reply-To: <20070726063614.GL26963@thunder.jkraemer.net> References: <87b412ce0707240033y3f72a1bcr6a77e74f92512b3b@mail.gmail.com> <20070724084749.GB26963@thunder.jkraemer.net> <87b412ce0707242339oe3acb4n6ec9d474a30b27e3@mail.gmail.com> <20070726063614.GL26963@thunder.jkraemer.net> Message-ID: <87b412ce0707270413y17dca908n10e869f935231f4@mail.gmail.com> Thats cool...I could get that working too.Thanks a lot..1 more problem i have before getting it to completion... I currently have 2 models,chat having a "Has_many" relationship with messages model. A chat has a name and messages have message content.So when i search a keyword,i want 2 go against both these fields. So if i want 2 do single index on chat module.but for both the fields...how do i go about doing it? so i have chat.rb model and message.rb model. In chat.rb acts_as_ferret :store_class_name => true, :fields => { :name , :msg } def msg return "#{self.message.find(:all,:select=>'msg')}" end msg is supposed to be a array of messages for one chat...So can i index on all elements of the msg array within chat module? And thereafter whenever a msg gets inserted in message model,will its contents be automatically indexed? I tried doing t vice-versa by putting index on messages.It works.But in case of row pagination doesnt come out well... if many messages have the keyword,it sends me those many msgIds and they all belong to just 1 chat....total hits become more and actual records i would show are less...So extra page numbers gets displayed. A better option for me would be the first approach..So can you help me on it? On 7/25/07, Jens Kraemer wrote: > > On Tue, Jul 24, 2007 at 11:39:15PM -0700, isha kakodkar wrote: > > thanx..i actually did it using the :conditions with find_by_contents. > > But when i do find_id_by_contents,which searches from the filesystem,can > i > > search within data based on this index value?and not just the full data > > field? > > yes, every attribute you index is stored in it's own Ferret field, and > you can run queries (or parts of queries) against just one Ferret field > by specifying the field name like I did in the example below. > > Jens > > > > > On 7/24/07, Jens Kraemer wrote: > > > > > >On Tue, Jul 24, 2007 at 12:33:47AM -0700, isha kakodkar wrote: > > >> Hi all, > > >> Im a newbie to ferret.I have installed the act_as_ferret gem and i > > >> actually want to search for some content in the model,but per user. > > >> My model has data,user_id.So within the data column i have to serach > for > > >a > > >> word if it exists for a given user. > > >> So can i achieve that ? presently i know that it just searches the > > >"data" > > >> column irrespective of the user given. > > >> Can someone help me with this? > > > > > >Index the user_id field (as untokenized) and append " user_id:#{user.id > }" > > >to the query. > > > > > >cheers, > > >Jens > > > > > > > > >-- > > >Jens Kr?mer > > >http://www.jkraemer.net/ - Blog > > >http://www.omdb.org/ - The new free film database > > >_______________________________________________ > > >Ferret-talk mailing list > > >Ferret-talk at rubyforge.org > > >http://rubyforge.org/mailman/listinfo/ferret-talk > > > > > > _______________________________________________ > > Ferret-talk mailing list > > Ferret-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/ferret-talk > > -- > Jens Kr?mer > http://www.jkraemer.net/ - Blog > http://www.omdb.org/ - The new free film database > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070727/79b2f0ab/attachment-0001.html From portie at bitterbal.org Fri Jul 27 08:18:18 2007 From: portie at bitterbal.org (Izit Izit) Date: Fri, 27 Jul 2007 14:18:18 +0200 Subject: [Ferret-talk] AAF index updated when record is deleted? Message-ID: Got it all running nicely now for a few days but noticed something funny. I have a table were lots of records get added, deleted every day. I can see the added records being added by AAF to the index but I notices some "ghost" results when doing searches. The search returns record ID-s that do not exist anymore because they were deleted. Does AAF also remove entries from the index when records are deleted in the model ? regards -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Fri Jul 27 15:54:22 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Fri, 27 Jul 2007 21:54:22 +0200 Subject: [Ferret-talk] AAF index updated when record is deleted? In-Reply-To: References: Message-ID: <20070727195422.GG8712@thunder.jkraemer.net> On Fri, Jul 27, 2007 at 02:18:18PM +0200, Izit Izit wrote: > Got it all running nicely now for a few days but noticed something > funny. I have a table were lots of records get added, deleted every day. > I can see the added records being added by AAF to the index but I > notices some "ghost" results when doing searches. The search returns > record ID-s that do not exist anymore because they were deleted. > > Does AAF also remove entries from the index when records are deleted in > the model ? That depends :-) If you call record.destroy, aaf will notice, if you use record.delete, it won't (as no callbacks are called in this case). cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From rick.ceo at lacharite.ca Fri Jul 27 19:38:30 2007 From: rick.ceo at lacharite.ca (Rick Lacharite) Date: Sat, 28 Jul 2007 01:38:30 +0200 Subject: [Ferret-talk] Mongrel won't start In-Reply-To: <20070727081312.GC12090@cordoba.webit.de> References: <20070727081312.GC12090@cordoba.webit.de> Message-ID: Jens Kraemer wrote: > On Fri, Jul 27, 2007 at 06:56:34AM +0200, Rick Lacharite wrote: >> I have a win xp box running instant rails. I installed ferret (the win >> version) and acts_as_ferret seemingly succesfully. I then added "require >> 'acts_as_ferret'" to the environment.rb file of my project. However, I >> cannot get webrick/mongrel to start. Can anyone suggest a solution? > > Hardly without any further info. > > Did you install aaf as a plugin or via ruby gems? the require is not > needed if aaf resides in vendor/plugins inside your application. > > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa Jens, I installed aaf as a gem. I couldn't get the plugin to install. Rick -- Posted via http://www.ruby-forum.com/. From rick.ceo at lacharite.ca Fri Jul 27 19:44:07 2007 From: rick.ceo at lacharite.ca (Rick Lacharite) Date: Sat, 28 Jul 2007 01:44:07 +0200 Subject: [Ferret-talk] Mongrel won't start In-Reply-To: References: <20070727081312.GC12090@cordoba.webit.de> Message-ID: <64fb44b4a70dbb1dcd589be8e23eb81e@ruby-forum.com> Rick Lacharite wrote: > Jens Kraemer wrote: >> On Fri, Jul 27, 2007 at 06:56:34AM +0200, Rick Lacharite wrote: >>> I have a win xp box running instant rails. I installed ferret (the win >>> version) and acts_as_ferret seemingly succesfully. I then added "require >>> 'acts_as_ferret'" to the environment.rb file of my project. However, I >>> cannot get webrick/mongrel to start. Can anyone suggest a solution? >> >> Hardly without any further info. >> >> Did you install aaf as a plugin or via ruby gems? the require is not >> needed if aaf resides in vendor/plugins inside your application. >> >> Jens >> >> -- >> Jens Kr?mer >> webit! Gesellschaft f?r neue Medien mbH >> Schnorrstra?e 76 | 01069 Dresden >> Telefon +49 351 46766-0 | Telefax +49 351 46766-66 >> kraemer at webit.de | www.webit.de >> >> Amtsgericht Dresden | HRB 15422 >> GF Sven Haubold, Hagen Malessa > Jens, I installed aaf as a gem. I couldn't get the plugin to install. This was the output when I tried to start webrick" G:\emporium>ruby script/server => Booting Mongrel (use 'script/server webrick' to force WEBrick) => Rails application starting on http://0.0.0.0:3000 => Call with -d to detach => Ctrl-C to shutdown server ** Starting Mongrel listening at 0.0.0.0:3000 ** Starting Rails with development environment... Exiting G:/InstantRails/ruby/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/servers/mon grel.rb:15: warning: already initialized constant OPTIONS G:/InstantRails/ruby/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/commands/servers/mon grel.rb:18: undefined method `options' for []:Array (NoMethodError) from G:/InstantRails/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require .rb:32:in `gem_original_require' from G:/InstantRails/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require .rb:32:in `require' from G:/InstantRails/ruby/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib /active_support/dependencies.rb:495:in `require' from G:/InstantRails/ruby/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib /active_support/dependencies.rb:342:in `new_constants_in' from G:/InstantRails/ruby/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib /active_support/dependencies.rb:495:in `require' from G:/InstantRails/ruby/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/command s/server.rb:39 from G:/InstantRails/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require .rb:27:in `gem_original_require' from G:/InstantRails/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require .rb:27:in `require' from script/server:3 -- Posted via http://www.ruby-forum.com/. From myowntribe at yahoo.com Sat Jul 28 01:48:28 2007 From: myowntribe at yahoo.com (Cass Amino) Date: Sat, 28 Jul 2007 07:48:28 +0200 Subject: [Ferret-talk] Mongrel won't start In-Reply-To: References: Message-ID: Rick Lacharite wrote: > I have a win xp box running instant rails. I installed ferret (the win > version) and acts_as_ferret seemingly succesfully. I then added "require > 'acts_as_ferret'" to the environment.rb file of my project. However, I > cannot get webrick/mongrel to start. Can anyone suggest a solution? > > Rick Hi Rick I had the same problem but after I reinstalled ferret with 'gem install ferret' command, mongrel started normally. If you havent installed ferret do this: gem install ferret and then start the server, it might work. And if you have installed acts_as_ferret locally in your project in the vendor/plugin directory you need not have to type 'require acts_as_ferret' in environment.rb Cheers Cass -- Posted via http://www.ruby-forum.com/. From myowntribe at yahoo.com Sat Jul 28 01:57:35 2007 From: myowntribe at yahoo.com (Cass Amino) Date: Sat, 28 Jul 2007 07:57:35 +0200 Subject: [Ferret-talk] Problem with acts_as_ferret demo Message-ID: Hi Jens, I copied the demo folder into my rails apps and also created the ferret_production and ferret_test databases. when I start the server and type this in the browser: localhost:3000/content I get the following error: NameError in ContentController#new uninitialized constant Ferret::Search::BooleanClause RAILS_ROOT: ./script/../config/.. Application Trace | Framework Trace | Full Trace F:/InstantRails/ruby/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:263:in `load_missing_constant' F:/InstantRails/ruby/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:452:in `const_missing' #{RAILS_ROOT}/vendor/plugins/acts_as_ferret/lib/a Could you please tell me what's going wrong? Cheers Cass -- Posted via http://www.ruby-forum.com/. From portie at bitterbal.org Sat Jul 28 03:20:26 2007 From: portie at bitterbal.org (Izit Izit) Date: Sat, 28 Jul 2007 09:20:26 +0200 Subject: [Ferret-talk] AAF index updated when record is deleted? In-Reply-To: <20070727195422.GG8712@thunder.jkraemer.net> References: <20070727195422.GG8712@thunder.jkraemer.net> Message-ID: <937733ceff8b1925d4d61a31c372e0c6@ruby-forum.com> Oh silly silly me :-) I had a delete_all in stead of destroy_all. All is fine now. thanks Jens Kraemer wrote: > On Fri, Jul 27, 2007 at 02:18:18PM +0200, Izit Izit wrote: >> Got it all running nicely now for a few days but noticed something >> funny. I have a table were lots of records get added, deleted every day. >> I can see the added records being added by AAF to the index but I >> notices some "ghost" results when doing searches. The search returns >> record ID-s that do not exist anymore because they were deleted. >> >> Does AAF also remove entries from the index when records are deleted in >> the model ? > > That depends :-) > If you call record.destroy, aaf will notice, if you use record.delete, > it won't (as > no callbacks are called in this case). > > cheers, > Jens > > > -- > Jens Kr?mer > http://www.jkraemer.net/ - Blog > http://www.omdb.org/ - The new free film database -- Posted via http://www.ruby-forum.com/. From rick.ceo at lacharite.ca Sat Jul 28 12:27:40 2007 From: rick.ceo at lacharite.ca (Rick Lacharite) Date: Sat, 28 Jul 2007 18:27:40 +0200 Subject: [Ferret-talk] Mongrel won't start In-Reply-To: References: Message-ID: Cass Amino wrote: > Rick Lacharite wrote: >> I have a win xp box running instant rails. I installed ferret (the win >> version) and acts_as_ferret seemingly succesfully. I then added "require >> 'acts_as_ferret'" to the environment.rb file of my project. However, I >> cannot get webrick/mongrel to start. Can anyone suggest a solution? >> >> Rick > > Hi Rick > > I had the same problem but after I reinstalled ferret with 'gem install > ferret' command, mongrel started normally. > > If you havent installed ferret do this: > > gem install ferret > > and then start the server, it might work. > > And if you have installed acts_as_ferret locally in your project in the > vendor/plugin directory you need not have to type 'require > acts_as_ferret' in environment.rb > > Cheers > > Cass Thanks Cass, That seems to have worked. I need to do some testing to check it out completely. Rick -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Sun Jul 29 06:33:53 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Sun, 29 Jul 2007 12:33:53 +0200 Subject: [Ferret-talk] Memory leak in PerFieldAnalyzer Message-ID: <98F12D98-D791-46B3-A753-E98186C127FA@benjaminkrause.com> Hello everyone, we've recently discovered a memory leak in the PerFieldAnalyzer. If you use the PerFieldAnalyzer (which you acutally should), you should switch to a pure ruby version of that analyzer. The C version of the Analyzer is consuming memory on every analyzing request. You can find an example script to verify the leak[1]. Furthermore we've added a workaround, building a PerFieldAnalyzer in pure ruby[2]. You can read more about the leak on the omdb tech blog[3]. Benjamin --- [1] http://pastie.caboo.se/83195 [2] http://pastie.caboo.se/83194 [3] http://blog.omdb-beta.org/2007/7/29/tracking-down-a-memory-leak- in-ferret-0-11-4 From jk at jkraemer.net Sun Jul 29 07:08:19 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Sun, 29 Jul 2007 13:08:19 +0200 Subject: [Ferret-talk] Memory leak in PerFieldAnalyzer In-Reply-To: <98F12D98-D791-46B3-A753-E98186C127FA@benjaminkrause.com> References: <98F12D98-D791-46B3-A753-E98186C127FA@benjaminkrause.com> Message-ID: <20070729110819.GJ8712@thunder.jkraemer.net> On Sun, Jul 29, 2007 at 12:33:53PM +0200, Benjamin Krause wrote: > Hello everyone, > > we've recently discovered a memory leak in the > PerFieldAnalyzer. If you use the PerFieldAnalyzer > (which you acutally should), you should switch > to a pure ruby version of that analyzer. The C > version of the Analyzer is consuming memory > on every analyzing request. > > You can find an example script to verify the > leak[1]. Furthermore we've added a > workaround, building a PerFieldAnalyzer > in pure ruby[2]. You can read more about the > leak on the omdb tech blog[3]. Great work! And now included in acts_as_ferret's trunk :-) I put your new PerFieldAnalyzer in the Ferret::Analysis namespace so it hides the original implementation and no changes to existing code are needed. cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From eimorton at gmail.com Sun Jul 29 14:34:26 2007 From: eimorton at gmail.com (Erik Morton) Date: Sun, 29 Jul 2007 14:34:26 -0400 Subject: [Ferret-talk] RDig and AAF playing together Message-ID: <0B92FEA3-EF29-4FA2-A383-492C69CEE0E3@gmail.com> I have a site with two indexes. Index A is created offline by RDig and queried from the web via RDig (specifically, RDig.searcher.search). Index B is managed by AAF with :remote => true. Simple enough. However, I need to query both indexes from RDig. Usually this is ok, as I modified RDig to accept an array of search_paths with an element for index A and index B. However, when Index B is updated by AAF, RDig.searcher.search will not "see" the changes to Index B until I restart Mongrel (or restart script/console). If I query Index B directly through ClassB.find_by_contents("myfield:my_value") I see the updated results immediately with no restart. I know that RDig creates a single IndexReader for class. Does the IndexReader cache the segments files in memory? Does anyone have any ideas? Thanks in advance for your help! Erik From kraemer at webit.de Mon Jul 30 03:38:33 2007 From: kraemer at webit.de (Jens Kraemer) Date: Mon, 30 Jul 2007 09:38:33 +0200 Subject: [Ferret-talk] RDig and AAF playing together In-Reply-To: <0B92FEA3-EF29-4FA2-A383-492C69CEE0E3@gmail.com> References: <0B92FEA3-EF29-4FA2-A383-492C69CEE0E3@gmail.com> Message-ID: <20070730073833.GA2198@cordoba.webit.de> On Sun, Jul 29, 2007 at 02:34:26PM -0400, Erik Morton wrote: > I have a site with two indexes. Index A is created offline by RDig > and queried from the web via RDig (specifically, > RDig.searcher.search). Index B is managed by AAF with :remote => > true. Simple enough. However, I need to query both indexes from RDig. > Usually this is ok, as I modified RDig to accept an array of > search_paths with an element for index A and index B. > > However, when Index B is updated by AAF, RDig.searcher.search will > not "see" the changes to Index B until I restart Mongrel (or restart > script/console). If I query Index B directly through > ClassB.find_by_contents("myfield:my_value") I see the updated results > immediately with no restart. > > I know that RDig creates a single IndexReader for class. Does the > IndexReader cache the segments files in memory? Yes. > Does anyone have any ideas? You can check if your reader still 'sees' the most recent version of the index with the latest? method, and re-open it accordingly. You might have to hack RDig a bit to allow opening a new reader, but this shouldn't be too hard. Cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From eimorton at gmail.com Mon Jul 30 08:25:59 2007 From: eimorton at gmail.com (Erik Morton) Date: Mon, 30 Jul 2007 08:25:59 -0400 Subject: [Ferret-talk] RDig and AAF playing together In-Reply-To: <20070730073833.GA2198@cordoba.webit.de> References: <0B92FEA3-EF29-4FA2-A383-492C69CEE0E3@gmail.com> <20070730073833.GA2198@cordoba.webit.de> Message-ID: <034C55C4-B496-42B6-A3BC-F4E9564C61DD@gmail.com> Thanks Jens, that makes sense. I started with the following addition to RDig::Searcher # Returns true if RDig's IndexReader has the latest index loaded. False otherwise. def latest? @ferret_searcher.reader.latest? end I fired up two script/console instances. In the first I called ClassB.rebuild_index, and in the second console I called RDig.searcher.latest? and received the following seg fault. >> RDig.searcher.latest? ./script/../config/../config/../vendor/gems/rdig-0.3.4/lib/rdig/ search.rb:36: [BUG] Bus Error ruby 1.8.4 (2005-12-24) [i686-darwin8.6.1] Did I break a Ferret rule of some kind by having a reader looking at the version of an index that is being rebuilt? Thanks again. Erik On Jul 30, 2007, at 3:38 AM, Jens Kraemer wrote: > On Sun, Jul 29, 2007 at 02:34:26PM -0400, Erik Morton wrote: >> I have a site with two indexes. Index A is created offline by RDig >> and queried from the web via RDig (specifically, >> RDig.searcher.search). Index B is managed by AAF with :remote => >> true. Simple enough. However, I need to query both indexes from RDig. >> Usually this is ok, as I modified RDig to accept an array of >> search_paths with an element for index A and index B. >> >> However, when Index B is updated by AAF, RDig.searcher.search will >> not "see" the changes to Index B until I restart Mongrel (or restart >> script/console). If I query Index B directly through >> ClassB.find_by_contents("myfield:my_value") I see the updated results >> immediately with no restart. >> >> I know that RDig creates a single IndexReader for class. Does the >> IndexReader cache the segments files in memory? > > Yes. > >> Does anyone have any ideas? > > You can check if your reader still 'sees' the most recent version > of the > index with the latest? method, and re-open it accordingly. You might > have to hack RDig a bit to allow opening a new reader, but this > shouldn't be too hard. > > Cheers, > Jens > > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From isha.kakodkar at gmail.com Mon Jul 30 08:32:43 2007 From: isha.kakodkar at gmail.com (isha kakodkar) Date: Mon, 30 Jul 2007 05:32:43 -0700 Subject: [Ferret-talk] indexing only the changed values Message-ID: <87b412ce0707300532m16ca253eie34f24d3deb7a52d@mail.gmail.com> Hi all, i have model A which has a field indexed from model B. model A belongs to model B. So whenever i insert a row in model 'A', a query is fired to the field from model 'B' even though the data was not changed for the field in model B. Can i somehow avoid these extra queries,or rather query the data and index it,only if the data has been changed>? e.g model A { message chat Name } model B{ chat Name } For 1 chatName there are 10 messages.so i want to index chatName once per 10 messages and not 10 times.How do i do this? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070730/0cc8044b/attachment.html From kraemer at webit.de Mon Jul 30 08:58:12 2007 From: kraemer at webit.de (Jens Kraemer) Date: Mon, 30 Jul 2007 14:58:12 +0200 Subject: [Ferret-talk] RDig and AAF playing together In-Reply-To: <034C55C4-B496-42B6-A3BC-F4E9564C61DD@gmail.com> References: <0B92FEA3-EF29-4FA2-A383-492C69CEE0E3@gmail.com> <20070730073833.GA2198@cordoba.webit.de> <034C55C4-B496-42B6-A3BC-F4E9564C61DD@gmail.com> Message-ID: <20070730125812.GE2198@cordoba.webit.de> On Mon, Jul 30, 2007 at 08:25:59AM -0400, Erik Morton wrote: > Thanks Jens, that makes sense. I started with the following addition > to RDig::Searcher > > # Returns true if RDig's IndexReader has the latest > index loaded. False otherwise. > def latest? > @ferret_searcher.reader.latest? > end > > I fired up two script/console instances. In the first I called > ClassB.rebuild_index, and in the second console I called > RDig.searcher.latest? and received the following seg fault. > > >> RDig.searcher.latest? > ./script/../config/../config/../vendor/gems/rdig-0.3.4/lib/rdig/ > search.rb:36: [BUG] Bus Error > ruby 1.8.4 (2005-12-24) [i686-darwin8.6.1] > > Did I break a Ferret rule of some kind by having a reader looking at > the version of an index that is being rebuilt? Yes. An index rebuild begins with deleting the old index, which will cause index readers that were opened on the now removed index to fail this way. So latest? is only good to detect additions/deletions of documents. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From eimorton at gmail.com Mon Jul 30 09:18:33 2007 From: eimorton at gmail.com (Erik Morton) Date: Mon, 30 Jul 2007 09:18:33 -0400 Subject: [Ferret-talk] RDig and AAF playing together In-Reply-To: <20070730125812.GE2198@cordoba.webit.de> References: <0B92FEA3-EF29-4FA2-A383-492C69CEE0E3@gmail.com> <20070730073833.GA2198@cordoba.webit.de> <034C55C4-B496-42B6-A3BC-F4E9564C61DD@gmail.com> <20070730125812.GE2198@cordoba.webit.de> Message-ID: <3B2EA537-079A-4FD3-BDE8-05482F0E202F@gmail.com> It's strange, I'm actually getting the Bus Error anytime I call latest? on RDig's index reader. The index is no longer being rebuilt. It's interesting because the following lines were commented out of my version of RDig: # if @ferret_searcher and !@ferret_searcher.reader.latest? # # reopen searcher # @ferret_searcher.close # @ferret_searcher = nil # end So this has obviously happened before. I must have commented these lines out myself :-/ On linux I get the following: >> RDig.searcher.ferret_searcher.reader.latest? (irb):5: [BUG] Segmentation fault ruby 1.8.4 (2005-12-24) [i386-linux] Aborted Erik On Jul 30, 2007, at 8:58 AM, Jens Kraemer wrote: > On Mon, Jul 30, 2007 at 08:25:59AM -0400, Erik Morton wrote: >> Thanks Jens, that makes sense. I started with the following addition >> to RDig::Searcher >> >> # Returns true if RDig's IndexReader has the latest >> index loaded. False otherwise. >> def latest? >> @ferret_searcher.reader.latest? >> end >> >> I fired up two script/console instances. In the first I called >> ClassB.rebuild_index, and in the second console I called >> RDig.searcher.latest? and received the following seg fault. >> >>>> RDig.searcher.latest? >> ./script/../config/../config/../vendor/gems/rdig-0.3.4/lib/rdig/ >> search.rb:36: [BUG] Bus Error >> ruby 1.8.4 (2005-12-24) [i686-darwin8.6.1] >> >> Did I break a Ferret rule of some kind by having a reader looking at >> the version of an index that is being rebuilt? > > Yes. > An index rebuild begins with deleting the old index, which will > cause index readers that were opened on the now removed index to fail > this way. So latest? is only good to detect additions/deletions of > documents. > > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From bk at benjaminkrause.com Mon Jul 30 09:26:26 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Mon, 30 Jul 2007 15:26:26 +0200 Subject: [Ferret-talk] indexing only the changed values In-Reply-To: <87b412ce0707300532m16ca253eie34f24d3deb7a52d@mail.gmail.com> References: <87b412ce0707300532m16ca253eie34f24d3deb7a52d@mail.gmail.com> Message-ID: <5B51C217-620D-4EC8-9691-99F1F868176C@benjaminkrause.com> Hey.. Ferret does not allow you to update information in the index. You can either add information or remove information. There is no SQL-like 'UPDATE' statement. Ferret or AAF - whatever you use - need all information, if you want to index something. And AAF does not keep track of changes in the model, so it will not allow you to add a flag like :only_index_dependencies_if_model_really_changed :) So if you want to avoid your extra queries, build some caching inside your model classes. This has nothing to do with Ferret. E.g. try storing your data in a memcache with a short TTL or think of any other caching mechanism that will work for your model, regardless of Ferret. Ben On 2007-07-30, at 2:32 PM, isha kakodkar wrote: > Hi all, > i have model A which has a field indexed from model B. model A > belongs to model B. > So whenever i insert a row in model 'A', a query is fired to the > field from model 'B' even though the data was not changed for the > field in model B. > Can i somehow avoid these extra queries,or rather query the data > and index it,only if the data has been changed>? > e.g model A { > message > chat Name > } > > > model B{ > chat Name > } > > > For 1 chatName there are 10 messages.so i want to index chatName > once per 10 messages and not 10 times.How do i do this? > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From kraemer at webit.de Mon Jul 30 09:35:50 2007 From: kraemer at webit.de (Jens Kraemer) Date: Mon, 30 Jul 2007 15:35:50 +0200 Subject: [Ferret-talk] RDig and AAF playing together In-Reply-To: <3B2EA537-079A-4FD3-BDE8-05482F0E202F@gmail.com> References: <0B92FEA3-EF29-4FA2-A383-492C69CEE0E3@gmail.com> <20070730073833.GA2198@cordoba.webit.de> <034C55C4-B496-42B6-A3BC-F4E9564C61DD@gmail.com> <20070730125812.GE2198@cordoba.webit.de> <3B2EA537-079A-4FD3-BDE8-05482F0E202F@gmail.com> Message-ID: <20070730133550.GA30135@cordoba.webit.de> On Mon, Jul 30, 2007 at 09:18:33AM -0400, Erik Morton wrote: > It's strange, I'm actually getting the Bus Error anytime I call > latest? on RDig's index reader. The index is no longer being rebuilt. > It's interesting because the following lines were commented out of my > version of RDig: > # if @ferret_searcher and !@ferret_searcher.reader.latest? > # # reopen searcher > # @ferret_searcher.close > # @ferret_searcher = nil > # end > So this has obviously happened before. I must have commented these > lines out myself :-/ > > On linux I get the following: > >> RDig.searcher.ferret_searcher.reader.latest? > (irb):5: [BUG] Segmentation fault > ruby 1.8.4 (2005-12-24) [i386-linux] Ah yes :-) If your reader looks at two sub-readers for different indexes (as it seems to do, if I got your first mail right) you'll have to call latest? on each of the sub readers to get around this. I do the same in acts_as_ferret's MultiIndex class. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From cwaters at networkchemistry.com Mon Jul 30 13:27:41 2007 From: cwaters at networkchemistry.com (Waters, Chris) Date: Mon, 30 Jul 2007 13:27:41 -0400 Subject: [Ferret-talk] Ferret crashing ruby while highlighting Message-ID: <8D2B87EFB35A4341B53B86D3DE6F4CD5023DA6AF@mse1be1.mse1.mailstreet.com> Hi, I am having a problem where ferret.so is crashing which highlighting search results. The query string is "25,42,53,80,88,119,135,139,389,445,464,563,593,636,1441,1442,1444,1515, 1519,1527,1537,1546,3389". In the debugger I can see that the crash happens in search.c on line 907. The reason is that the excerpt->end field is -1. I think the reason is because in this code from search.c: for (e_start = e_end = 0; e_start < mv->size; e_start++) { const int start_offset = matches[e_start].start_offset; if (e_start > e_end) { running_score = 0.0; e_end = e_start; } while (e_end < mv->size && (matches[e_end].end_offset <= start_offset + excerpt_len)) { running_score += matches[e_end].score; e_end++; } pq_push(excerpt_pq, excerpt_new(e_start, e_end - 1, running_score)); /* - 0.1 so that earlier matches take priority */ running_score -= matches[e_start].score; } e_end is never being incremented beyond 0, so e_end-1 is stored as the excerpt. However I am not sure why that is. Anyway, thought this bug report might be useful. Regards, Chris Waters CTO, PhD Network Chemistry, Inc Email: chris.waters at networkchemistry.com Phone: (650) 858-3125 www.networkchemistry.com www.projectwishbone.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070730/79bb0d44/attachment-0001.html From ed.temp.01 at gmail.com Mon Jul 30 14:16:43 2007 From: ed.temp.01 at gmail.com (Ed --) Date: Mon, 30 Jul 2007 20:16:43 +0200 Subject: [Ferret-talk] Memory leak in PerFieldAnalyzer In-Reply-To: <98F12D98-D791-46B3-A753-E98186C127FA@benjaminkrause.com> References: <98F12D98-D791-46B3-A753-E98186C127FA@benjaminkrause.com> Message-ID: Benjamin Krause wrote: > Hello everyone, > > we've recently discovered a memory leak in the > PerFieldAnalyzer. If you use the PerFieldAnalyzer > (which you acutally should), you should switch > to a pure ruby version of that analyzer. The C > version of the Analyzer is consuming memory > on every analyzing request. Is this relevant to those of us who do not explicitly use PerFieldAnalyzer? i.e. is it used behind the scenes unless you say otherwise? Ed -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Mon Jul 30 14:20:37 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Mon, 30 Jul 2007 20:20:37 +0200 Subject: [Ferret-talk] Memory leak in PerFieldAnalyzer In-Reply-To: References: <98F12D98-D791-46B3-A753-E98186C127FA@benjaminkrause.com> Message-ID: <20070730182037.GM8712@thunder.jkraemer.net> On Mon, Jul 30, 2007 at 08:16:43PM +0200, Ed -- wrote: > Benjamin Krause wrote: > > Hello everyone, > > > > we've recently discovered a memory leak in the > > PerFieldAnalyzer. If you use the PerFieldAnalyzer > > (which you acutally should), you should switch > > to a pure ruby version of that analyzer. The C > > version of the Analyzer is consuming memory > > on every analyzing request. > > Is this relevant to those of us who do not explicitly use > PerFieldAnalyzer? i.e. is it used behind the scenes unless you say > otherwise? no, PerFieldAnalyzer is only used if you tell Ferret to do so. Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From isha.kakodkar at gmail.com Tue Jul 31 02:55:47 2007 From: isha.kakodkar at gmail.com (isha kakodkar) Date: Mon, 30 Jul 2007 23:55:47 -0700 Subject: [Ferret-talk] indexing only the changed values In-Reply-To: <5B51C217-620D-4EC8-9691-99F1F868176C@benjaminkrause.com> References: <87b412ce0707300532m16ca253eie34f24d3deb7a52d@mail.gmail.com> <5B51C217-620D-4EC8-9691-99F1F868176C@benjaminkrause.com> Message-ID: <87b412ce0707302355w7ffce9afs4a45b3eb8631a359@mail.gmail.com> ok thanks... Can i specify :group clause while searching....I want to apply a groupBy clause in sql to get the resultset and index it... Can i do it with Ferret? On 7/30/07, Benjamin Krause wrote: > > Hey.. > > Ferret does not allow you to update information in > the index. You can either add information or remove > information. There is no SQL-like 'UPDATE' > statement. Ferret or AAF - whatever you use - need > all information, if you want to index something. And > AAF does not keep track of changes in the model, > so it will not allow you to add a flag like > :only_index_dependencies_if_model_really_changed :) > > So if you want to avoid your extra queries, build some > caching inside your model classes. This has nothing to > do with Ferret. E.g. try storing your data in a memcache > with a short TTL or think of any other caching mechanism > that will work for your model, regardless of Ferret. > > > Ben > > > > On 2007-07-30, at 2:32 PM, isha kakodkar wrote: > > > Hi all, > > i have model A which has a field indexed from model B. model A > > belongs to model B. > > So whenever i insert a row in model 'A', a query is fired to the > > field from model 'B' even though the data was not changed for the > > field in model B. > > Can i somehow avoid these extra queries,or rather query the data > > and index it,only if the data has been changed>? > > e.g model A { > > message > > chat Name > > } > > > > > > model B{ > > chat Name > > } > > > > > > For 1 chatName there are 10 messages.so i want to index chatName > > once per 10 messages and not 10 times.How do i do this? > > _______________________________________________ > > Ferret-talk mailing list > > Ferret-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/ferret-talk > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070730/752e7e39/attachment.html From isha.kakodkar at gmail.com Tue Jul 31 05:12:23 2007 From: isha.kakodkar at gmail.com (isha kakodkar) Date: Tue, 31 Jul 2007 02:12:23 -0700 Subject: [Ferret-talk] Group by clause Message-ID: <87b412ce0707310212s3a072a9bv6d1711611e7da907@mail.gmail.com> Hi Does acts_as_ferret support a :group clause? For e.g any rails options like :select or :group etc? or is it that it supports only few of such options?Like it supports :include Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070731/a74fb666/attachment.html From kraemer at webit.de Tue Jul 31 05:31:41 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 31 Jul 2007 11:31:41 +0200 Subject: [Ferret-talk] Group by clause In-Reply-To: <87b412ce0707310212s3a072a9bv6d1711611e7da907@mail.gmail.com> References: <87b412ce0707310212s3a072a9bv6d1711611e7da907@mail.gmail.com> Message-ID: <20070731093141.GG30135@cordoba.webit.de> On Tue, Jul 31, 2007 at 02:12:23AM -0700, isha kakodkar wrote: > Hi > Does acts_as_ferret support a :group clause? > For e.g any rails options like :select or :group etc? or is it that it > supports only few of such options?Like it supports :include you can put whatever AR options you like into the second argument hash of find_by_contents. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From john at digitalpulp.com Tue Jul 31 17:17:53 2007 From: john at digitalpulp.com (John Bachir) Date: Tue, 31 Jul 2007 17:17:53 -0400 Subject: [Ferret-talk] When to restart the DRb server? Message-ID: Under what circumstances does the DRb server need to be restarted? When new models are being indexed? When a model changes? Thanks, John From bk at benjaminkrause.com Tue Jul 31 18:03:54 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Wed, 1 Aug 2007 00:03:54 +0200 Subject: [Ferret-talk] When to restart the DRb server? In-Reply-To: References: Message-ID: On 2007-07-31, at 11:17 PM, John Bachir wrote: > Under what circumstances does the DRb server need to be restarted? > When new models are being indexed? When a model changes? Hey John, if you change your model classes, the backgroundrb server should be restarted. This includes migrations or any other code that is relevant for the indexing process. But if you use capistrano, there is a capistrano task available for acts_as_ferret[1]. Ben [1] http://projects.jkraemer.net/acts_as_ferret/browser/trunk/plugin/ acts_as_ferret/lib/ferret_cap_tasks.rb