From dyzz at mail.ustc.edu.cn Sat Sep 1 09:59:14 2007 From: dyzz at mail.ustc.edu.cn (Zhuang Austin) Date: Sat, 1 Sep 2007 15:59:14 +0200 Subject: [Ferret-talk] Can lucene use index generated by ferret? Message-ID: <9f1bc5e7a1fc0df703a721a60b110918@ruby-forum.com> I found it quite fun to generate index by ferret. However, I had to use lucene now. Can I use the index that I've already done by ferret? ps. Lucene 2.2.0 vs Ferret 0.11.4 Thx! -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Sat Sep 1 10:12:57 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Sat, 1 Sep 2007 16:12:57 +0200 Subject: [Ferret-talk] Can lucene use index generated by ferret? In-Reply-To: <9f1bc5e7a1fc0df703a721a60b110918@ruby-forum.com> References: <9f1bc5e7a1fc0df703a721a60b110918@ruby-forum.com> Message-ID: On 2007-09-01, at 3:59 PM, Zhuang Austin wrote: > I found it quite fun to generate index by ferret. However, I had to > use lucene now. Can I use the index that I've already done by ferret? > ps. Lucene 2.2.0 vs Ferret 0.11.4 no, its not possible .. see http://ferret.davebalmain.com/trac/wiki/FAQ% 3AIndexing#CanIuseanindexthatwasbuildwithLucene The Ferret and the Lucene index is not compatible, at least since Ferret Version 0.10 Ben From dyzz at mail.ustc.edu.cn Sat Sep 1 11:13:39 2007 From: dyzz at mail.ustc.edu.cn (Zhuang Austin) Date: Sat, 1 Sep 2007 17:13:39 +0200 Subject: [Ferret-talk] Can lucene use index generated by ferret? In-Reply-To: References: <9f1bc5e7a1fc0df703a721a60b110918@ruby-forum.com> Message-ID: Got it... Thx Benjamin Krause wrote: > On 2007-09-01, at 3:59 PM, Zhuang Austin wrote: > >> I found it quite fun to generate index by ferret. However, I had to >> use lucene now. Can I use the index that I've already done by ferret? >> ps. Lucene 2.2.0 vs Ferret 0.11.4 > > no, its not possible .. see > > http://ferret.davebalmain.com/trac/wiki/FAQ% > 3AIndexing#CanIuseanindexthatwasbuildwithLucene > > The Ferret and the Lucene index is not compatible, at least since > Ferret Version 0.10 > > Ben -- Posted via http://www.ruby-forum.com/. From hakita at gmail.com Sat Sep 1 19:41:27 2007 From: hakita at gmail.com (Hakita Hakita) Date: Sun, 2 Sep 2007 01:41:27 +0200 Subject: [Ferret-talk] ferret acts_as_ferret and performance In-Reply-To: <20070831220635.GE11981@thunder.jkraemer.net> References: <20070831220635.GE11981@thunder.jkraemer.net> Message-ID: Good idea Once again, Thank you Jens -- Posted via http://www.ruby-forum.com/. From hakita at gmail.com Sat Sep 1 20:10:32 2007 From: hakita at gmail.com (Hakita Hakita) Date: Sun, 2 Sep 2007 02:10:32 +0200 Subject: [Ferret-talk] ferret acts_as_ferret and performance In-Reply-To: References: <20070831220635.GE11981@thunder.jkraemer.net> Message-ID: This is going really faster now ! As i am indexing thousands of rss feeds, it now takes one hour to retrieve and index it all ( several hours before ) -- Posted via http://www.ruby-forum.com/. From testmyeclipse at ua.fm Sun Sep 2 06:03:59 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Sun, 2 Sep 2007 12:03:59 +0200 Subject: [Ferret-talk] ferret for professionals Message-ID: Hello, I'm trying to setup ferret search engine(what i did successfully for searching english phrases, words) But i doesn't work for UTF-8 symbols!!! I tried to find the solution for this problem: 1) i add to enviroment.rb followinf lines: ENV['LANG'] = 'de_DE.UTF-8 at euro' ENV['LC_TIME'] = 'C' require 'acts_as_ferret' 2) next i using scaffold added few posts(my model, that i index this way: acts_as_ferret :fields => [ :name, :post ] English phrases searching well, but for german it get empty result array. I don't know what to do, so I asked to professional, who do this!! Thanks for your replies! -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Sun Sep 2 07:28:03 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Sun, 2 Sep 2007 13:28:03 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: References: Message-ID: <28E3C370-FF00-4738-BAEB-F168CF394925@benjaminkrause.com> Hey .. i just set the followings: ENV['LC_CTYPE'] = 'en_US.UTF-8' Ferret.locale = "en_US.UTF-8" see http://bugs.omdb.org/browser/trunk/config/environment.rb searching for utf-8 works great, i've indexed characters in german, english, asian languages, hebrew and other.. Ben On 2007-09-02, at 12:03 PM, Igor K. wrote: > Hello, > > I'm trying to setup ferret search engine(what i did successfully for > searching english phrases, words) > > But i doesn't work for UTF-8 symbols!!! > > I tried to find the solution for this problem: > 1) i add to enviroment.rb followinf lines: > ENV['LANG'] = 'de_DE.UTF-8 at euro' > ENV['LC_TIME'] = 'C' > require 'acts_as_ferret' > 2) next i using scaffold added few posts(my model, that i index this > way: > acts_as_ferret :fields => [ :name, :post ] > > English phrases searching well, but for german it get empty result > array. > > I don't know what to do, so I asked to professional, who do this!! > > Thanks for your replies! > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From jk at jkraemer.net Sun Sep 2 15:48:07 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Sun, 2 Sep 2007 21:48:07 +0200 Subject: [Ferret-talk] ferret acts_as_ferret and performance In-Reply-To: References: <20070831220635.GE11981@thunder.jkraemer.net> Message-ID: <20070902194806.GA25296@thunder.jkraemer.net> On Sun, Sep 02, 2007 at 02:10:32AM +0200, Hakita Hakita wrote: > This is going really faster now ! > As i am indexing thousands of rss feeds, it now takes one hour to > retrieve and index it all ( several hours before ) glad to hear that :-) Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From testmyeclipse at ua.fm Mon Sep 3 04:08:19 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Mon, 3 Sep 2007 10:08:19 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <28E3C370-FF00-4738-BAEB-F168CF394925@benjaminkrause.com> References: <28E3C370-FF00-4738-BAEB-F168CF394925@benjaminkrause.com> Message-ID: Benjamin Krause wrote: > Hey .. > > i just set the followings: > > ENV['LC_CTYPE'] = 'en_US.UTF-8' > Ferret.locale = "en_US.UTF-8" > > see http://bugs.omdb.org/browser/trunk/config/environment.rb > > searching for utf-8 works great, i've indexed characters > in german, english, asian languages, hebrew and other.. > > Ben Thak you for your reply, but even this can't help me, please check my sources in attachment. Here is 2 controlles - add(for adding new posts) and search Thanks Attachments: http://www.ruby-forum.com/attachment/212/ferrettest2.rar -- Posted via http://www.ruby-forum.com/. From matthew.langham at indiginox.com Mon Sep 3 09:42:07 2007 From: matthew.langham at indiginox.com (Matthew Langham) Date: Mon, 3 Sep 2007 15:42:07 +0200 Subject: [Ferret-talk] MySQL has gone away Message-ID: Hi, in a production environment we are running into "MySQL server has gone away" errors (see below) when we call rebuild_index in aaf. This happens sporadically. Any suggestions for what might be wrong or workarounds? I guess we try a reconnect before we call rebuild_index? Matthew ActiveRecord::StatementInvalid (Mysql::Error: MySQL server has gone away: SELECT count(*) AS count_all FROM bands ): (druby://localhost:3009) /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/abstract_adapter.rb:128:in `log' (druby://localhost:3009) /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/mysql_adapter.rb:243:in `execute' (druby://localhost:3009) /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/mysql_adapter.rb:399:in `select' (druby://localhost:3009) /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/abstract/database_statements.rb:13:in `select_one' (druby://localhost:3009) /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/abstract/database_statements.rb:19:in `select_value' (druby://localhost:3009) /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/calculations.rb:212:in `execute_simple_calculation' (druby://localhost:3009) /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/calculations.rb:121:in `calculate' (druby://localhost:3009) /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/calculations.rb:117:in `catch' (druby://localhost:3009) /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/calculations.rb:117:in `calculate' (druby://localhost:3009) /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/calculations.rb:45:in `count' (druby://localhost:3009) /var/www/web1/oms/current/script/../config/../vendor/plugins/acts_as_ferret/lib/ferret_extensions.rb:22:in `index_model' (druby://localhost:3009) /var/www/web1/oms/current/script/../config/../vendor/plugins/acts_as_ferret/lib/ferret_extensions.rb:9:in `index_models' (druby://localhost:3009) /var/www/web1/oms/current/script/../config/../vendor/plugins/acts_as_ferret/lib/ferret_extensions.rb:9:in `each' (druby://localhost:3009) /var/www/web1/oms/current/script/../config/../vendor/plugins/acts_as_ferret/lib/ferret_extensions.rb:9:in `index_models' (druby://localhost:3009) /var/www/web1/oms/current/script/../config/../vendor/plugins/acts_as_ferret/lib/ferret_server.rb:93:in `rebuild_index' (druby://localhost:3009) /var/www/web1/oms/current/script/../config/../vendor/plugins/acts_as_ferret/lib/ferret_server.rb:113:in `with_class' (druby://localhost:3009) /var/www/web1/oms/current/script/../config/../vendor/plugins/acts_as_ferret/lib/ferret_server.rb:88:in `rebuild_index' /vendor/plugins/acts_as_ferret/lib/remote_index.rb:16:in `send' /vendor/plugins/acts_as_ferret/lib/remote_index.rb:16:in `method_missing' /vendor/plugins/acts_as_ferret/lib/class_methods.rb:15:in `rebuild_index' -- Posted via http://www.ruby-forum.com/. From matthew.langham at indiginox.com Mon Sep 3 09:52:27 2007 From: matthew.langham at indiginox.com (Matthew Langham) Date: Mon, 3 Sep 2007 15:52:27 +0200 Subject: [Ferret-talk] MySQL has gone away In-Reply-To: References: Message-ID: <134eb7e5dc8f96072e9c344eb1a0da1b@ruby-forum.com> I need to add that we are running the integrated aaf Ferret Drb Server, so I guess it is the Drb Server that is losing the connection? How can I get it to reconnect without throwing the error? -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Mon Sep 3 10:11:34 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Mon, 3 Sep 2007 16:11:34 +0200 Subject: [Ferret-talk] MySQL has gone away In-Reply-To: <134eb7e5dc8f96072e9c344eb1a0da1b@ruby-forum.com> References: <134eb7e5dc8f96072e9c344eb1a0da1b@ruby-forum.com> Message-ID: On 2007-09-03, at 3:52 PM, Matthew Langham wrote: > I need to add that we are running the integrated aaf Ferret Drb > Server, > so I guess it is the Drb Server that is losing the connection? How > can I > get it to reconnect without throwing the error? Hey .. we've got the same error with our non-aaf backgroundrb solution .. that's what we're doing: begin index_object( record ) rescue ActiveRecord::RecordNotFound @logger.warn "Object gone" rescue ActiveRecord::StatementInvalid @logger.warn "MySQL is gone.." clazz.connection.reconnect! enqueue_object( record ) end So basically just catch the StatementInvalid Exception and try again after you reconnected to the database.. I guess Jens can add something like that to AAF quite easily :) Ben From kraemer at webit.de Mon Sep 3 10:22:25 2007 From: kraemer at webit.de (Jens Kraemer) Date: Mon, 3 Sep 2007 16:22:25 +0200 Subject: [Ferret-talk] MySQL has gone away In-Reply-To: References: Message-ID: <20070903142225.GW7512@cordoba.webit.de> Hi! On Mon, Sep 03, 2007 at 03:42:07PM +0200, Matthew Langham wrote: > Hi, > > in a production environment we are running into "MySQL server has gone > away" errors (see below) when we call rebuild_index in aaf. This happens > sporadically. Does it happen after a long time of inactivity on the DRb server's side? If yes, a look at mysql's wait_timeout setting, and ActiveRecord::Base.verification_timeout, wich should be less than the former, might help. Setting ActiveRecord::Base.allow_concurrency to true solved this problem for me once, however I'm not sure why and it was not ferret-related. Cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From bk at benjaminkrause.com Mon Sep 3 10:27:33 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Mon, 3 Sep 2007 16:27:33 +0200 Subject: [Ferret-talk] MySQL has gone away In-Reply-To: <20070903142225.GW7512@cordoba.webit.de> References: <20070903142225.GW7512@cordoba.webit.de> Message-ID: <7103A74E-3E66-4AAC-B1C2-9D1F0625F3F9@benjaminkrause.com> > > Setting ActiveRecord::Base.allow_concurrency to true solved this > problem for me once, however I'm not sure why and it was not > ferret-related. We set the following .. but still - the sql connection is lost from time to time.. ActiveRecord::Base.allow_concurrency = true ActiveRecord::Base.verification_timeout = 10 Ben From julioody at gmail.com Mon Sep 3 18:22:54 2007 From: julioody at gmail.com (Julio Cesar Ody) Date: Tue, 4 Sep 2007 08:22:54 +1000 Subject: [Ferret-talk] MySQL has gone away In-Reply-To: <7103A74E-3E66-4AAC-B1C2-9D1F0625F3F9@benjaminkrause.com> References: <20070903142225.GW7512@cordoba.webit.de> <7103A74E-3E66-4AAC-B1C2-9D1F0625F3F9@benjaminkrause.com> Message-ID: Are you running Ubuntu? If yes, try installing the libmysqlclient-dev package (latest one you have), then run "sudo gem install mysql". You get my meaning. If you're not running Ubuntu, get the mysql client development headers, which are required by the mysql gem, then install the gem. I had a similar issue to yours, then I installed the C based extensions. Problem went away. On 9/4/07, Benjamin Krause wrote: > > > > Setting ActiveRecord::Base.allow_concurrency to true solved this > > problem for me once, however I'm not sure why and it was not > > ferret-related. > > We set the following .. but still - the sql connection is lost from time > to time.. > > ActiveRecord::Base.allow_concurrency = true > ActiveRecord::Base.verification_timeout = 10 > > Ben > > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From chris at drinkingbird.net Mon Sep 3 22:26:43 2007 From: chris at drinkingbird.net (Chris --) Date: Tue, 4 Sep 2007 04:26:43 +0200 Subject: [Ferret-talk] AAF not searching associated record data Message-ID: <8e1e3b29b0e3d972296510f74d9e9432@ruby-forum.com> I'm using AAF to search through an AR model, with fields from associated tables included through helper methods in the AR class. The search is working perfectly for the model's fields, but the data from the associated records aren't included, it seems. Pastie: http://pastie.caboo.se/93667 I've made sure it's not just a matter of rebuilding the index, and I'm at a loss as to why this isn't working. -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Tue Sep 4 03:34:05 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Tue, 4 Sep 2007 09:34:05 +0200 Subject: [Ferret-talk] MySQL has gone away In-Reply-To: References: <20070903142225.GW7512@cordoba.webit.de> <7103A74E-3E66-4AAC-B1C2-9D1F0625F3F9@benjaminkrause.com> Message-ID: <85C232E3-5B92-4FE5-AEE4-B255C9ABA563@benjaminkrause.com> > I had a similar issue to yours, then I installed the C based > extensions. Problem went away. Julio, i got the mysql bindings (version 2.7) installed. But there is a newer version (2.7.3). I will update them and take a look. Thanks for the suggestion :) Ben From kraemer at webit.de Tue Sep 4 03:47:14 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 4 Sep 2007 09:47:14 +0200 Subject: [Ferret-talk] AAF not searching associated record data In-Reply-To: <8e1e3b29b0e3d972296510f74d9e9432@ruby-forum.com> References: <8e1e3b29b0e3d972296510f74d9e9432@ruby-forum.com> Message-ID: <20070904074714.GZ7512@cordoba.webit.de> On Tue, Sep 04, 2007 at 04:26:43AM +0200, Chris -- wrote: > I'm using AAF to search through an AR model, with fields from associated > tables included through helper methods in the AR class. > > The search is working perfectly for the model's fields, but the data > from the associated records aren't included, it seems. > > Pastie: http://pastie.caboo.se/93667 > > I've made sure it's not just a matter of rebuilding the index, and I'm > at a loss as to why this isn't working. Your code looks like it should work, what's in your log files when such a model is saved? Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Tue Sep 4 03:57:05 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 4 Sep 2007 09:57:05 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: References: Message-ID: <20070904075705.GB7512@cordoba.webit.de> On Sun, Sep 02, 2007 at 12:03:59PM +0200, Igor K. wrote: > Hello, > > I'm trying to setup ferret search engine(what i did successfully for > searching english phrases, words) > > But i doesn't work for UTF-8 symbols!!! > > I tried to find the solution for this problem: > 1) i add to enviroment.rb followinf lines: > ENV['LANG'] = 'de_DE.UTF-8 at euro' > ENV['LC_TIME'] = 'C' > require 'acts_as_ferret' > 2) next i using scaffold added few posts(my model, that i index this > way: > acts_as_ferret :fields => [ :name, :post ] > > English phrases searching well, but for german it get empty result > array. Does this apply for searching via a web form, or in unit tests? Maybe non-ascii characters in your queries get garbled before ferret even gets to see them? The same applies for the content you save. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From testmyeclipse at ua.fm Tue Sep 4 05:28:11 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Tue, 4 Sep 2007 11:28:11 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <20070904075705.GB7512@cordoba.webit.de> References: <20070904075705.GB7512@cordoba.webit.de> Message-ID: <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> Jens Kraemer wrote: > On Sun, Sep 02, 2007 at 12:03:59PM +0200, Igor K. wrote: >> ENV['LC_TIME'] = 'C' >> require 'acts_as_ferret' >> 2) next i using scaffold added few posts(my model, that i index this >> way: >> acts_as_ferret :fields => [ :name, :post ] >> > Does this apply for searching via a web form, or in unit tests? Maybe > non-ascii characters in your queries get garbled before ferret even gets > to see them? No, i use UTF8 in database, controllers, environment.rb When i do search like Post.find(:all, :conditions => ['name like ?', 'SOME-UTF8-TEXT']) it founds. Have anybody looked at my test project in attachment? -- Posted via http://www.ruby-forum.com/. From andreas.korth at gmail.com Tue Sep 4 07:33:18 2007 From: andreas.korth at gmail.com (Andreas Korth) Date: Tue, 4 Sep 2007 13:33:18 +0200 Subject: [Ferret-talk] MySQL has gone away In-Reply-To: <7103A74E-3E66-4AAC-B1C2-9D1F0625F3F9@benjaminkrause.com> References: <20070903142225.GW7512@cordoba.webit.de> <7103A74E-3E66-4AAC-B1C2-9D1F0625F3F9@benjaminkrause.com> Message-ID: <417C45B8-7F5E-4D4F-B6B1-5E7B28A41D72@gmx.net> On 03.09.2007, at 16:27, Benjamin Krause wrote: >> >> Setting ActiveRecord::Base.allow_concurrency to true solved this >> problem for me once, however I'm not sure why and it was not >> ferret-related. > > ActiveRecord::Base.allow_concurrency = true > ActiveRecord::Base.verification_timeout = 10 ActiveRecord just isn't reentrant. Period ;) There are supposedly ways to make it work using allow_concurrency, reconnect and whatever. But... > .. but still - the sql connection is lost from time to time.. Exactly. I rebuild my index in a separate process and directory. When it's done, I tell the index server via DRb that a new index is available and it'll switch to the new directory. Works like a charm. Cheers, Andy From golak.sarangi at gmail.com Tue Sep 4 09:03:20 2007 From: golak.sarangi at gmail.com (golak Sarangi) Date: Tue, 4 Sep 2007 18:33:20 +0530 Subject: [Ferret-talk] can boost change dynamically?? Message-ID: <3854b3a40709040603u6ce46d1cud5145985b8addad2@mail.gmail.com> Hi, I have got a peculiar requirement in my project. I need to apply some boost to different fields of my index. But the problem is the boost that needs to be applied is dynamic. It changes for everyrow of data.I am storing that in the index itself while building it. Is there anyway by which i can add the boost to search query while searching. at present am retreiving the whole set of searched result and then manipulating with the score to get the output. but by that way it gets too slow. can any one help me. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070904/1277060e/attachment.html From kraemer at webit.de Tue Sep 4 09:08:11 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 4 Sep 2007 15:08:11 +0200 Subject: [Ferret-talk] can boost change dynamically?? In-Reply-To: <3854b3a40709040603u6ce46d1cud5145985b8addad2@mail.gmail.com> References: <3854b3a40709040603u6ce46d1cud5145985b8addad2@mail.gmail.com> Message-ID: <20070904130811.GE7512@cordoba.webit.de> On Tue, Sep 04, 2007 at 06:33:20PM +0530, golak Sarangi wrote: > Hi, > > I have got a peculiar requirement in my project. I need to apply some boost > to different fields of my index. But the problem is the boost that needs to > be applied is dynamic. It changes for everyrow of data.I am storing that in > the index itself while building it. Is there anyway by which i can add the > boost to search query while searching. at present am retreiving the whole > set of searched result and then manipulating with the score to get the > output. but by that way it gets too slow. can any one help me. did you search the archives for this? Imho this topic came up a few days (or weeks?) ago. Short answer: References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> Message-ID: <20070904135115.GF7512@cordoba.webit.de> On Tue, Sep 04, 2007 at 11:28:11AM +0200, Igor K. wrote: > Jens Kraemer wrote: > > On Sun, Sep 02, 2007 at 12:03:59PM +0200, Igor K. wrote: > >> ENV['LC_TIME'] = 'C' > >> require 'acts_as_ferret' > >> 2) next i using scaffold added few posts(my model, that i index this > >> way: > >> acts_as_ferret :fields => [ :name, :post ] > >> > > > Does this apply for searching via a web form, or in unit tests? Maybe > > non-ascii characters in your queries get garbled before ferret even gets > > to see them? > > No, i use UTF8 in database, controllers, environment.rb > When i do search like Post.find(:all, :conditions => ['name like ?', > 'SOME-UTF8-TEXT']) it founds. > > Have anybody looked at my test project in attachment? Yes, I did. After changing the line setting ENV['LANG'] to 'en_US.UTF-8' in config/environment.rb (because I don't have the de_DE.UTF-8 at euro locale installed on my system) it works perfectly. Before it didn't, and I couldn't even enter german umlauts on the rails console. So you should make sure the locale you set LANG to also exists on your system. 'dpkg-reconfigure locales' can be used on Debian/Ubuntu to check which locales you have and enable more if you like. cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From testmyeclipse at ua.fm Tue Sep 4 10:40:15 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Tue, 4 Sep 2007 16:40:15 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <20070904135115.GF7512@cordoba.webit.de> References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> Message-ID: <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> > Yes, I did. After changing the line setting ENV['LANG'] > to 'en_US.UTF-8' in config/environment.rb (because I don't have > the de_DE.UTF-8 at euro locale installed on my system) it works perfectly. > > Before it didn't, and I couldn't even enter german umlauts on the rails > console. > > So you should make sure the locale you set LANG to also exists on your > system. 'dpkg-reconfigure locales' can be used on Debian/Ubuntu to check > which locales you have and enable more if you like. > > Thank you for your replies, I will check it! Maybe it depends on operating system? I have Win XP Home SP2 russian. Where i can get a list of locations for other countries('en_US.UTF-8', de_DE.UTF-8)??? Thanks -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Tue Sep 4 10:44:10 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 4 Sep 2007 16:44:10 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> Message-ID: <20070904144410.GH7512@cordoba.webit.de> On Tue, Sep 04, 2007 at 04:40:15PM +0200, Igor K. wrote: > > > Yes, I did. After changing the line setting ENV['LANG'] > > to 'en_US.UTF-8' in config/environment.rb (because I don't have > > the de_DE.UTF-8 at euro locale installed on my system) it works perfectly. > > > > Before it didn't, and I couldn't even enter german umlauts on the rails > > console. > > > > So you should make sure the locale you set LANG to also exists on your > > system. 'dpkg-reconfigure locales' can be used on Debian/Ubuntu to check > > which locales you have and enable more if you like. > > > > > Thank you for your replies, > > I will check it! > > Maybe it depends on operating system? I have Win XP Home SP2 russian. > Where i can get a list of locations for other countries('en_US.UTF-8', > de_DE.UTF-8)??? imho you should be in a utf8 environment by default then - however I don't have a windows to check this. Maybe for starters just start irb and have a look at ENV['LANG'] ? Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From chris at drinkingbird.net Tue Sep 4 18:38:47 2007 From: chris at drinkingbird.net (Chris --) Date: Wed, 5 Sep 2007 00:38:47 +0200 Subject: [Ferret-talk] AAF not searching associated record data In-Reply-To: <20070904074714.GZ7512@cordoba.webit.de> References: <8e1e3b29b0e3d972296510f74d9e9432@ruby-forum.com> <20070904074714.GZ7512@cordoba.webit.de> Message-ID: The problem seems to have resolved itself. I just went through and deleted the index, restarted everything, and it's behaving now. I should have checked the logs, while they existed (was working on an SVN branch that's now been merged, and the log files lost in its removal). -- Posted via http://www.ruby-forum.com/. From testmyeclipse at ua.fm Wed Sep 5 02:57:43 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Wed, 5 Sep 2007 08:57:43 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <20070904144410.GH7512@cordoba.webit.de> References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> Message-ID: > > imho you should be in a utf8 environment by default then - however I > don't have a windows to check this. Maybe for starters just start irb > and have a look at ENV['LANG'] ? > > Jens > Hello, Thanks for replies :) you really helps me I have tried to change: - windows locale to Russia - environment.rb ENV['LC_CTYPE'] = 'ru_RU.UTF-8' Ferret.locale = "ru_RU.UTF-8" - and what was important to change database locale from UTF8-unicode to UTF8-general. But as i espect it was no finish of problems: - because it index data incorrect, for example some data came to index db. The other problem is when - some objects it can find and some not(russian objects), when i search by '*' i can get all objects, but with '**' i get only part of them. what version of ferret-server, acts-as-ferret, locale in MySQL database do you have? Thanks -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Sep 5 04:28:16 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 5 Sep 2007 10:28:16 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> Message-ID: <20070905082816.GA8344@cordoba.webit.de> On Wed, Sep 05, 2007 at 08:57:43AM +0200, Igor K. wrote: > > > > imho you should be in a utf8 environment by default then - however I > > don't have a windows to check this. Maybe for starters just start irb > > and have a look at ENV['LANG'] ? > > > > Jens > > > Hello, > Thanks for replies :) you really helps me > I have tried to change: > - windows locale to Russia > - environment.rb > ENV['LC_CTYPE'] = 'ru_RU.UTF-8' > Ferret.locale = "ru_RU.UTF-8" > - and what was important to change database locale from UTF8-unicode to > UTF8-general. > > But as i espect it was no finish of problems: > - because it index data incorrect, for example some data came to index > db. The other problem is when - some objects it can find and some > not(russian objects), Have you checked with the Ferret browser to make sure it really indexed incorret values? Have you rebuilt your index after changing these locale things? > when i search by '*' i can get all objects, but with '**' i get only > part of them. I'm not sure what Ferret does with a query like '**'... > what version of ferret-server, acts-as-ferret, locale in MySQL database > do you have? I used the aaf from inside your app when checking it out, and ferret 0.11.4. Here's a small checklist I use for making sure everything is UTF-8: HTTP/HTML content-type delivered by web server should be 'Content-Type: text/html; charset=utf-8' html content-type meta tag should be '' Mysql settings: In your application's console, execute the following: r = Post.connection.execute "SHOW VARIABLES LIKE 'character%'" r.each {|r|puts r} This should result in something like this: character_set_client utf8 character_set_connection utf8 character_set_database utf8 If your output differs, try the following: set 'encoding: utf8' in environment.rb (this affects the character_set_connection and character_set_client values) set 'default-character-set=utf8' in the mysqld section of the mysql configuration (/etc/mysql/my.cnf on linux). After restart of the server, newly created databases will be utf8 by default, and new tables in these databases will inherit this setting. Maybe it's possible to change the character set of existing databases/tables, too, however your data will have to be converted, too. The per database setting imho is only a default setting applied to new tables. with all these settings in place, everything should be fine on the UTF-8 front :-) Cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From neongrau at gmail.com Wed Sep 5 04:53:59 2007 From: neongrau at gmail.com (neongrau __) Date: Wed, 5 Sep 2007 10:53:59 +0200 Subject: [Ferret-talk] aaf index rebuild problems on windows Message-ID: <47a683031249a1d57c2ff57a23c0e056@ruby-forum.com> hello list! i'm still having massive problems with index rebuilds. after i recently switched to the DRb mode i was hoping to get rid of the problems with locked files and such. but even with DRb and the latest stable version of acts_as_ferret my index rebuilds are failing C:\Informer>ruby script/console production Loading production environment. >> LegacyInfobase.rebuild_index IOError: IO Error occured: couldn't create InStream C:/Informer/script/../config/../index/production/legacy _infobase/rebuild\_uu.fdt: could this be the "\" in front of the file name thats causing the problem? -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Sep 5 05:14:47 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 5 Sep 2007 11:14:47 +0200 Subject: [Ferret-talk] aaf index rebuild problems on windows In-Reply-To: <47a683031249a1d57c2ff57a23c0e056@ruby-forum.com> References: <47a683031249a1d57c2ff57a23c0e056@ruby-forum.com> Message-ID: <20070905091446.GD8344@cordoba.webit.de> On Wed, Sep 05, 2007 at 10:53:59AM +0200, neongrau __ wrote: > hello list! > > i'm still having massive problems with index rebuilds. > after i recently switched to the DRb mode i was hoping to get rid of the > problems with locked files and such. > > but even with DRb and the latest stable version of acts_as_ferret my > index rebuilds are failing > > C:\Informer>ruby script/console production > Loading production environment. > >> LegacyInfobase.rebuild_index > IOError: IO Error occured: > couldn't create InStream > C:/Informer/script/../config/../index/production/legacy > _infobase/rebuild\_uu.fdt: > > > could this be the "\" in front of the file name thats causing the > problem? Strange. The '\' seems to be appended by Ferret, aaf just tells it to rebuild the index in the rebuild directory, without trailing (back)slash. However I think if this were the problem, more people would have already discovered this. Does rebuilding the index without DRb, just from the console, without the application running, work? cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From testmyeclipse at ua.fm Wed Sep 5 05:16:20 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Wed, 5 Sep 2007 11:16:20 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <20070905082816.GA8344@cordoba.webit.de> References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> <20070905082816.GA8344@cordoba.webit.de> Message-ID: Thank you very much for answers, but i still have problems > HTTP/HTML > content-type delivered by web server > should be 'Content-Type: text/html; charset=utf-8' > html content-type meta tag > should be ' content="text/html;charset=utf-8" />' for this i did: application.rhtml application.rb before_filter :set_charset def set_charset if request.xhr? headers["Content-Type"] = "text/javascript; charset=utf-8" else headers["Content-Type"] = "text/html; charset=utf-8" end end > > Mysql settings: > In your application's console, execute the following: > r = Post.connection.execute "SHOW VARIABLES LIKE 'character%'" > r.each {|r|puts r} > This should result in something like this: > > character_set_client > utf8 > character_set_connection > utf8 > character_set_database > utf8 > Here i gen results: ["character_set_client", "latin1"] ["character_set_connection", "latin1"] ["character_set_database", "utf8"] ["character_set_filesystem", "binary"] ["character_set_results", "latin1"] ["character_set_server", "latin1"] ["character_set_system", "utf8"] ["character_sets_dir", "C:\\InstantRails\\mysql\\share\\charsets\\"] > If your output differs, try the following: > > set 'encoding: utf8' in environment.rb > (this affects the character_set_connection and character_set_client > values) > in environement.rb i add: ENV['LC_CTYPE'] = 'ru_RU.UTF8' ENV['LANG'] = 'ru_RU.UTF8' $KCODE = 'u' require 'acts_as_ferret' > set 'default-character-set=utf8' in the mysqld section of the mysql > configuration (/etc/mysql/my.cnf on linux). After restart of the > server, > newly created databases will be utf8 by default, and new tables in > these > databases will inherit this setting. Maybe it's possible to change the > character set of existing databases/tables, too, however your data > will > have to be converted, too. The per database setting imho is only a > default setting applied to new tables. > What else to do? Thanks -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Sep 5 05:19:04 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 5 Sep 2007 11:19:04 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> <20070905082816.GA8344@cordoba.webit.de> Message-ID: <20070905091904.GE8344@cordoba.webit.de> On Wed, Sep 05, 2007 at 11:16:20AM +0200, Igor K. wrote: > > Here i gen results: > ["character_set_client", "latin1"] > ["character_set_connection", "latin1"] > ["character_set_database", "utf8"] > ["character_set_filesystem", "binary"] > ["character_set_results", "latin1"] > ["character_set_server", "latin1"] > ["character_set_system", "utf8"] > ["character_sets_dir", "C:\\InstantRails\\mysql\\share\\charsets\\"] > > > If your output differs, try the following: > > > > set 'encoding: utf8' in environment.rb > > (this affects the character_set_connection and character_set_client > > values) Sorry, my mistake. You need to set encoding: utf8 in database.yml for your db connections. Looking at the output above this should fix your problem. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From testmyeclipse at ua.fm Wed Sep 5 05:32:35 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Wed, 5 Sep 2007 11:32:35 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <20070905091904.GE8344@cordoba.webit.de> References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> <20070905082816.GA8344@cordoba.webit.de> <20070905091904.GE8344@cordoba.webit.de> Message-ID: <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> > Sorry, my mistake. You need to set > encoding: utf8 > in database.yml for your db connections. No problem, anyway you really helps me :) now i got ["character_set_client", "utf8"] ["character_set_connection", "utf8"] ["character_set_database", "utf8"] ["character_set_filesystem", "binary"] ["character_set_results", "utf8"] ["character_set_server", "latin1"] ["character_set_system", "utf8"] ["character_sets_dir", "C:\\InstantRails\\mysql\\share\\charsets\\"] is it okay -> ["character_set_server", "latin1"]??? > Looking at the output above this should fix your problem. > -- Posted via http://www.ruby-forum.com/. From testmyeclipse at ua.fm Wed Sep 5 05:33:57 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Wed, 5 Sep 2007 11:33:57 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> <20070905082816.GA8344@cordoba.webit.de> <20070905091904.GE8344@cordoba.webit.de> <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> Message-ID: <48f66d2215ac48d1632d4f5b8989831b@ruby-forum.com> and how to start Ferret browser ??? -- Posted via http://www.ruby-forum.com/. From neongrau at gmail.com Wed Sep 5 05:43:45 2007 From: neongrau at gmail.com (neongrau __) Date: Wed, 5 Sep 2007 11:43:45 +0200 Subject: [Ferret-talk] aaf index rebuild problems on windows In-Reply-To: <20070905091446.GD8344@cordoba.webit.de> References: <47a683031249a1d57c2ff57a23c0e056@ruby-forum.com> <20070905091446.GD8344@cordoba.webit.de> Message-ID: <1db6295d52708f29ebc931d68c36f4c8@ruby-forum.com> will try this now. for the moment the only way to rebuild my index (tested on XP and 2003 server) is to delete or rename the models index dir, call ".find_by_content" and wait until the index is created. most of the time i even have to restart the DRb before that because otherwise windows complains about the dir being in use. but that causes a downtime of the search function for at least 20 minutes. -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Sep 5 05:46:49 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 5 Sep 2007 11:46:49 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> <20070905082816.GA8344@cordoba.webit.de> <20070905091904.GE8344@cordoba.webit.de> <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> Message-ID: <20070905094649.GA12765@cordoba.webit.de> On Wed, Sep 05, 2007 at 11:32:35AM +0200, Igor K. wrote: > > Sorry, my mistake. You need to set > > encoding: utf8 > > in database.yml for your db connections. > > No problem, anyway you really helps me :) > > now i got > ["character_set_client", "utf8"] > ["character_set_connection", "utf8"] > ["character_set_database", "utf8"] > ["character_set_filesystem", "binary"] > ["character_set_results", "utf8"] > ["character_set_server", "latin1"] > ["character_set_system", "utf8"] > ["character_sets_dir", "C:\\InstantRails\\mysql\\share\\charsets\\"] > > > is it okay -> ["character_set_server", "latin1"]??? yeah, as long as your app's database is set to utf8. ferret-browser is started with ferret-browser path/to/index Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From testmyeclipse at ua.fm Wed Sep 5 06:00:03 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Wed, 5 Sep 2007 12:00:03 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <20070905094649.GA12765@cordoba.webit.de> References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> <20070905082816.GA8344@cordoba.webit.de> <20070905091904.GE8344@cordoba.webit.de> <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> <20070905094649.GA12765@cordoba.webit.de> Message-ID: > ferret-browser is started with > > ferret-browser path/to/index > Seems to be that ferret works incorrect please check my sources and screenshot 22.gif. ps: when i start server i get an warning C:/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.1/lib/ferret_server.rb:123: warning: parenthesize argument(s) for future version -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Sep 5 06:06:05 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 5 Sep 2007 12:06:05 +0200 Subject: [Ferret-talk] aaf index rebuild problems on windows In-Reply-To: <1db6295d52708f29ebc931d68c36f4c8@ruby-forum.com> References: <47a683031249a1d57c2ff57a23c0e056@ruby-forum.com> <20070905091446.GD8344@cordoba.webit.de> <1db6295d52708f29ebc931d68c36f4c8@ruby-forum.com> Message-ID: <20070905100605.GB12765@cordoba.webit.de> On Wed, Sep 05, 2007 at 11:43:45AM +0200, neongrau __ wrote: > will try this now. > > for the moment the only way to rebuild my index (tested on XP and 2003 > server) is to delete or rename the models index dir, call > ".find_by_content" and wait until the index is created. why Model.rebuild_index doesn't work? > most of the time i even have to restart the DRb before that because > otherwise windows complains about the dir being in use. Omg, please do yourself a favour and switch to a platform with a file system that actually works. I think all your problems would go away if you tried it out on Linux or MacOSX. > but that causes a downtime of the search function for at least 20 > minutes. Yeah. The drb server gets around this by doing the rebuild in a separate directory. Cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From testmyeclipse at ua.fm Wed Sep 5 06:06:50 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Wed, 5 Sep 2007 12:06:50 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> <20070905082816.GA8344@cordoba.webit.de> <20070905091904.GE8344@cordoba.webit.de> <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> <20070905094649.GA12765@cordoba.webit.de> Message-ID: > > Seems to be that ferret works incorrect > please check my sources and screenshot 22.gif. > > ps: when i start server i get an warning > C:/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.1/lib/ferret_server.rb:123: > warning: parenthesize argument(s) for future version here ia attachment Attachments: http://www.ruby-forum.com/attachment/228/ferrettest3.rar -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Sep 5 06:07:52 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 5 Sep 2007 12:07:52 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: References: <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> <20070905082816.GA8344@cordoba.webit.de> <20070905091904.GE8344@cordoba.webit.de> <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> <20070905094649.GA12765@cordoba.webit.de> Message-ID: <20070905100752.GC12765@cordoba.webit.de> On Wed, Sep 05, 2007 at 12:00:03PM +0200, Igor K. wrote: > > ferret-browser is started with > > > > ferret-browser path/to/index > > > > Seems to be that ferret works incorrect in general I don't think so :-) > please check my sources and screenshot 22.gif. where? > ps: when i start server i get an warning > C:/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.1/lib/ferret_server.rb:123: > warning: parenthesize argument(s) for future version nothing to worry about... Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Wed Sep 5 06:10:32 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 5 Sep 2007 12:10:32 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: References: <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> <20070905082816.GA8344@cordoba.webit.de> <20070905091904.GE8344@cordoba.webit.de> <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> <20070905094649.GA12765@cordoba.webit.de> Message-ID: <20070905101032.GD12765@cordoba.webit.de> On Wed, Sep 05, 2007 at 12:06:50PM +0200, Igor K. wrote: > > > > Seems to be that ferret works incorrect > > please check my sources and screenshot 22.gif. > > > > ps: when i start server i get an warning > > C:/ruby/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.1/lib/ferret_server.rb:123: > > warning: parenthesize argument(s) for future version > > here ia attachment > > Attachments: > http://www.ruby-forum.com/attachment/228/ferrettest3.rar can't see what's wrong with the screenshot (besides the missing images for yes/no). did you check the index contents? Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From testmyeclipse at ua.fm Wed Sep 5 06:28:53 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Wed, 5 Sep 2007 12:28:53 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <20070905101032.GD12765@cordoba.webit.de> References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> <20070905082816.GA8344@cordoba.webit.de> <20070905091904.GE8344@cordoba.webit.de> <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> <20070905094649.GA12765@cordoba.webit.de> <20070905101032.GD12765@cordoba.webit.de> Message-ID: <186aff8a28a1c58759136334399bb6a5@ruby-forum.com> > can't see what's wrong with the screenshot (besides the missing images > for yes/no). > did you check the index contents? What do you mean? please check another attachment, why i can see only id field? Attachments: http://www.ruby-forum.com/attachment/229/333.GIF -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Wed Sep 5 06:34:32 2007 From: kraemer at webit.de (Jens Kraemer) Date: Wed, 5 Sep 2007 12:34:32 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <186aff8a28a1c58759136334399bb6a5@ruby-forum.com> References: <20070905082816.GA8344@cordoba.webit.de> <20070905091904.GE8344@cordoba.webit.de> <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> <20070905094649.GA12765@cordoba.webit.de> <20070905101032.GD12765@cordoba.webit.de> <186aff8a28a1c58759136334399bb6a5@ruby-forum.com> Message-ID: <20070905103432.GE12765@cordoba.webit.de> On Wed, Sep 05, 2007 at 12:28:53PM +0200, Igor K. wrote: > > > can't see what's wrong with the screenshot (besides the missing images > > for yes/no). > > > > did you check the index contents? > > What do you mean? > > please check another attachment, why i can see only id field? ah, of course. aaf by default only stores the id in the index, the rest of the data is just indexed (so records can be found, but you cannot retrieve their original contents from the index). use something like acts_as_ferret :fields => { :name => { :store => :yes }, :body => { :store => :yes } } then you'll see the data in ferret-browser. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From neongrau at gmail.com Wed Sep 5 06:35:27 2007 From: neongrau at gmail.com (neongrau __) Date: Wed, 5 Sep 2007 12:35:27 +0200 Subject: [Ferret-talk] aaf index rebuild problems on windows In-Reply-To: <20070905100605.GB12765@cordoba.webit.de> References: <47a683031249a1d57c2ff57a23c0e056@ruby-forum.com> <20070905091446.GD8344@cordoba.webit.de> <1db6295d52708f29ebc931d68c36f4c8@ruby-forum.com> <20070905100605.GB12765@cordoba.webit.de> Message-ID: <712d9f971f63154ed960f3baf971b883@ruby-forum.com> Jens Kraemer wrote: >> for the moment the only way to rebuild my index (tested on XP and 2003 >> server) is to delete or rename the models index dir, call >> ".find_by_content" and wait until the index is created. > > why Model.rebuild_index doesn't work? that would work as well after the restart, but the find will start the rebuild itself when there is no index. so i just hit the search button in the browser and wait for the rebuild to finish. >> most of the time i even have to restart the DRb before that because >> otherwise windows complains about the dir being in use. > > Omg, please do yourself a favour and switch to a platform with a file > system that actually works. I think all your problems would go away if > you tried it out on Linux or MacOSX. trust me i would have ditched windows years ago. but thats not so easy if you have a customer base that is using only using windows. at least i'm on a mac now myself with windows running only in vmware. but for now i have to get it to work on the windows machines. then i'm going to set it up on a linux vm. -- Posted via http://www.ruby-forum.com/. From testmyeclipse at ua.fm Wed Sep 5 06:57:08 2007 From: testmyeclipse at ua.fm (Igor K.) Date: Wed, 5 Sep 2007 12:57:08 +0200 Subject: [Ferret-talk] ferret for professionals In-Reply-To: <20070905103432.GE12765@cordoba.webit.de> References: <20070904075705.GB7512@cordoba.webit.de> <08ee5c28ad11eba1b8a827da3350c764@ruby-forum.com> <20070904135115.GF7512@cordoba.webit.de> <76b4ff5f31f5ce5000e00ba5569a7ae2@ruby-forum.com> <20070904144410.GH7512@cordoba.webit.de> <20070905082816.GA8344@cordoba.webit.de> <20070905091904.GE8344@cordoba.webit.de> <8dd05df13009525c8c3465f1bfb5f63e@ruby-forum.com> <20070905094649.GA12765@cordoba.webit.de> <20070905101032.GD12765@cordoba.webit.de> <186aff8a28a1c58759136334399bb6a5@ruby-forum.com> <20070905103432.GE12765@cordoba.webit.de> Message-ID: <474d4ab1a83786d440047bdbb9b39fcc@ruby-forum.com> Ohhhh :), it is not working anyway, please check attachment. in console as you can see i searching by symbol that one word in database starts. i don't know what to do? locale in system - russian Attachments: http://www.ruby-forum.com/attachment/230/1a.rar -- Posted via http://www.ruby-forum.com/. From golak.sarangi at gmail.com Wed Sep 5 07:29:36 2007 From: golak.sarangi at gmail.com (golak Sarangi) Date: Wed, 5 Sep 2007 16:59:36 +0530 Subject: [Ferret-talk] can boost change dynamically?? Message-ID: <3854b3a40709050429i1e01a750s338606b3614d45b@mail.gmail.com> On Tue, Sep 04, 2007 at 06:33:20PM +0530, golak Sarangi wrote: >* Hi, *>* *>* I have got a peculiar requirement in my project. I need to apply some boost *>* to different fields of my index. But the problem is the boost that needs to *>* be applied is dynamic. It changes for everyrow of data.I am storing that in *>* the index itself while building it. Is there anyway by which i can add the *>* boost to search query while searching. at present am retreiving the whole *>* set of searched result and then manipulating with the score to get the *>* output. but by that way it gets too slow. can any one help me. * did you search the archives for this? Imho this topic came up a few days (or weeks?) ago. Short answer: References: <3854b3a40709050429i1e01a750s338606b3614d45b@mail.gmail.com> Message-ID: <20070905114124.GF12765@cordoba.webit.de> On Wed, Sep 05, 2007 at 04:59:36PM +0530, golak Sarangi wrote: > calculation that ferret does to give me the output score. Although > the output that i get is relevant enough but its like a black hole > from outside. I tried to search for it from the C extension of ferret > but lost track > of it at the msea_search function. Could you please through some > light on the calculation of score when am using boost for document as > well as individual fields separately. have a look at Ferret's explain function, this will show you how the score of a result is calculated. Jens PS: your mail was pretty hard to read, please try to quote previous text correctly. http://www.netmeister.org/news/learn2quote.html -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From ed.temp.01 at gmail.com Wed Sep 5 12:40:17 2007 From: ed.temp.01 at gmail.com (Ed --) Date: Wed, 5 Sep 2007 18:40:17 +0200 Subject: [Ferret-talk] MySQL has gone away In-Reply-To: <417C45B8-7F5E-4D4F-B6B1-5E7B28A41D72@gmx.net> References: <20070903142225.GW7512@cordoba.webit.de> <7103A74E-3E66-4AAC-B1C2-9D1F0625F3F9@benjaminkrause.com> <417C45B8-7F5E-4D4F-B6B1-5E7B28A41D72@gmx.net> Message-ID: <0a49c1a8c126205dd2746cad1f8c0403@ruby-forum.com> Andreas Korth wrote: > > I rebuild my index in a separate process and directory. When it's > done, I tell the index server via DRb that a new index is available > and it'll switch to the new directory. Works like a charm. Do you use backgroundrb for this?. I'm coming up against similar problems with long running processes which need an ActiveRecord connection. I briefly tried to use fork() to background it but that seemed to give me problems with acts-as-authenticated. -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Wed Sep 5 17:09:53 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Wed, 5 Sep 2007 23:09:53 +0200 Subject: [Ferret-talk] MySQL has gone away In-Reply-To: References: <134eb7e5dc8f96072e9c344eb1a0da1b@ruby-forum.com> Message-ID: <20070905210953.GR18053@thunder.jkraemer.net> On Mon, Sep 03, 2007 at 04:11:34PM +0200, Benjamin Krause wrote: > > On 2007-09-03, at 3:52 PM, Matthew Langham wrote: > > > I need to add that we are running the integrated aaf Ferret Drb > > Server, > > so I guess it is the Drb Server that is losing the connection? How > > can I > > get it to reconnect without throwing the error? > > Hey .. > > we've got the same error with our non-aaf backgroundrb solution .. > > that's what we're doing: > > begin > index_object( record ) > rescue ActiveRecord::RecordNotFound > @logger.warn "Object gone" > rescue ActiveRecord::StatementInvalid > @logger.warn "MySQL is gone.." > clazz.connection.reconnect! > enqueue_object( record ) > end > > So basically just catch the StatementInvalid Exception and > try again after you reconnected to the database.. > > I guess Jens can add something like that to AAF quite easily :) Okay, I just committed this. According to my limited testing it works as intended - close db connection with Model.connection.disconnect, and it will reconnect again. Please give it a try and report the outcome here :-) cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From karlshea at gmail.com Wed Sep 5 19:58:05 2007 From: karlshea at gmail.com (Karl Shea) Date: Thu, 6 Sep 2007 01:58:05 +0200 Subject: [Ferret-talk] AAF and DRb with highlighting Message-ID: How would I change this method in order to get highlighting working with DRb? I've given up on searching on Google, I'm getting no results that are actually helpful. def self.find_storage_by_contents(query, options = {}) # Get the index that acts_as_ferret created for us index = self.aaf_index.ferret_index results = [] default_options = {:limit => 10, :page => 1} options = default_options.merge options options[:offset] = options[:limit] * (options[:page].to_i - 1) # search_each is the core search function from Ferret, which Acts_as_ferret hides total_hits = index.search_each(query, options) do |hit, score| doc = index[hit] result = {} article = Article.find(doc[:id]) # Store each field in a hash which we can reference in our views result[:headline_highlight] = index.highlight(query, hit, :field => :headline, :pre_tag => "", :post_tag => "", :num_excerpts => 1) result[:body_highlight] = index.highlight(query, hit, :field => :body, :pre_tag => "", :post_tag => "", :num_excerpts => 1) result[:article] = article result[:score] = score results.push result end unless query.nil? return block_given? ? total_hits : [total_hits, results] end -- Posted via http://www.ruby-forum.com/. From karlshea at gmail.com Wed Sep 5 20:36:27 2007 From: karlshea at gmail.com (Karl Shea) Date: Thu, 6 Sep 2007 02:36:27 +0200 Subject: [Ferret-talk] AAF and DRb with highlighting In-Reply-To: References: Message-ID: Ok, I think I've actually got it working: def self.find_storage_by_contents(query, options = {}) results = [] default_options = {:limit => 10, :page => 1} options = default_options.merge options options[:offset] = options[:limit] * (options[:page].to_i - 1) search = find_by_contents(query, options) total_hits = search.total_hits search.each do |hit| result = {} # Store each field in a hash which we can reference in our views result[:headline_highlight] = hit.highlight(query, :field => :headline, :pre_tag => "", :post_tag => "", :num_excerpts => 1) result[:body_highlight] = hit.highlight(query, :field => :body, :pre_tag => "", :post_tag => "", :num_excerpts => 1) result[:article] = hit results.push result end unless query.nil? return block_given? ? total_hits : [total_hits, results] end Is there anything in there that I'm not doing correctly? -- Posted via http://www.ruby-forum.com/. From golak.sarangi at gmail.com Thu Sep 6 04:33:23 2007 From: golak.sarangi at gmail.com (golak Sarangi) Date: Thu, 6 Sep 2007 14:03:23 +0530 Subject: [Ferret-talk] (no subject) Message-ID: <3854b3a40709060133h60273603t1318b9fbe64e65e4@mail.gmail.com> On Wed, Sep 05, 2007 at 04:59:36PM +0530, golak Sarangi wrote: > calculation that ferret does to give me the output score. Although > the output that i get is relevant enough but its like a black hole > from outside. I tried to search for it from the C extension of ferret > but lost track > of it at the msea_search function. Could you please through some > light on the calculation of score when am using boost for document as > well as individual fields separately. have a look at Ferret's explain function, this will show you how the score of a result is calculated. Jens Hi jens I tried the explain function but it generates a segmentation fault and the whole application crashes. I am facing one more problem. I am creating index on my data base. while creating index i am adding boost on the document. but it show segmentation fault on Free BSD while the index gets gerated in a Ms-windows for the same code. Am i doing something wrong. I am using ferret 0.10.9 on both machine. I am using ruby 1.8.4 for windows while 1.8.5 for linux. is that causing problem. I cant change the ruby version on linux machine as thatis the server. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070906/ec80ff69/attachment.html From kraemer at webit.de Thu Sep 6 05:21:19 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 6 Sep 2007 11:21:19 +0200 Subject: [Ferret-talk] (no subject) In-Reply-To: <3854b3a40709060133h60273603t1318b9fbe64e65e4@mail.gmail.com> References: <3854b3a40709060133h60273603t1318b9fbe64e65e4@mail.gmail.com> Message-ID: <20070906092119.GI12765@cordoba.webit.de> Hi, please switch to Ferret 0.11.4 first. Jens On Thu, Sep 06, 2007 at 02:03:23PM +0530, golak Sarangi wrote: > On Wed, Sep 05, 2007 at 04:59:36PM +0530, golak Sarangi wrote: > > calculation that ferret does to give me the output score. Although > > the output that i get is relevant enough but its like a black hole > > from outside. I tried to search for it from the C extension of ferret > > but lost track > > of it at the msea_search function. Could you please through some > > light on the calculation of score when am using boost for document as > > well as individual fields separately. > > have a look at Ferret's explain function, this will show you how the > score of a result is calculated. > > Jens > > > Hi jens > I tried the explain function but it generates a segmentation fault and the > whole application crashes. > I am facing one more problem. I am creating index on my data base. while > creating index i am adding boost on the document. but it show segmentation > fault on Free BSD while the index gets gerated in a Ms-windows for the same > code. Am i doing something wrong. > I am using ferret 0.10.9 on both machine. I am using ruby 1.8.4 for windows > while 1.8.5 for linux. is that causing problem. I cant change the ruby > version on linux machine as thatis the server. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From golak.sarangi at gmail.com Thu Sep 6 05:41:54 2007 From: golak.sarangi at gmail.com (golak Sarangi) Date: Thu, 6 Sep 2007 15:11:54 +0530 Subject: [Ferret-talk] Ferret 0.11.4 issue Message-ID: <3854b3a40709060241j79cba49ax3a144b66d6f09cd5@mail.gmail.com> Hi, please switch to Ferret 0.11.4 first. Jens On Thu, Sep 06, 2007 at 02:03:23PM +0530, golak Sarangi wrote: > On Wed, Sep 05, 2007 at 04:59:36PM +0530, golak Sarangi wrote: > > calculation that ferret does to give me the output score. Although > > the output that i get is relevant enough but its like a black hole > > from outside. I tried to search for it from the C extension of ferret > > but lost track > > of it at the msea_search function. Could you please through some > > light on the calculation of score when am using boost for document as > > well as individual fields separately. > > have a look at Ferret's explain function, this will show you how the > score of a result is calculated. > > Jens > > > Hi jens > I tried the explain function but it generates a segmentation fault and the > whole application crashes. > I am facing one more problem. I am creating index on my data base. while > creating index i am adding boost on the document. but it show segmentation > fault on Free BSD while the index gets gerated in a Ms-windows for the same > code. Am i doing something wrong. > I am using ferret 0.10.9 on both machine. I am using ruby 1.8.4 for windows > while 1.8.5 for linux. is that causing problem. I cant change the ruby > version on linux machine as thatis the server. I had tried 0.11.4 but the problem with it was it generated an "error rb_w32_write could not be located" while the older version did not. so now i am using 0.10.9 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070906/c0c9e2a7/attachment-0001.html From andreas.korth at gmail.com Thu Sep 6 06:36:00 2007 From: andreas.korth at gmail.com (Andreas Korth) Date: Thu, 6 Sep 2007 12:36:00 +0200 Subject: [Ferret-talk] MySQL has gone away In-Reply-To: <0a49c1a8c126205dd2746cad1f8c0403@ruby-forum.com> References: <20070903142225.GW7512@cordoba.webit.de> <7103A74E-3E66-4AAC-B1C2-9D1F0625F3F9@benjaminkrause.com> <417C45B8-7F5E-4D4F-B6B1-5E7B28A41D72@gmx.net> <0a49c1a8c126205dd2746cad1f8c0403@ruby-forum.com> Message-ID: <8CD83534-E85C-4A6B-BD48-B5B39024047A@gmx.net> On 05.09.2007, at 18:40, Ed -- wrote: >> I rebuild my index in a separate process and directory. When it's >> done, I tell the index server via DRb that a new index is available >> and it'll switch to the new directory. Works like a charm. > > Do you use backgroundrb for this?. I'm coming up against similar > problems with long running processes which need an ActiveRecord > connection. I briefly tried to use fork() to background it but that > seemed to give me problems with acts-as-authenticated. I use fork without problems. Here's a snippet from my controller which triggers the rebuild: def exec_command(command) dbconfig = ActiveRecord::Base.remove_connection fork do begin ActiveRecord::Base.establish_connection(dbconfig) `cd #{RAILS_ROOT} && script/runner -e #{ENV['RAILS_ENV']} '#{index_class}.#{command}'` ensure ActiveRecord::Base.remove_connection end end ActiveRecord::Base.establish_connection(dbconfig) end As you can see, I'm actually running a shell script which starts a whole new Rails process via script/runner. Note that ActiveRecord::Base.allow_concurrency is NOT set to true. I don't remember why this remove/establish connection was necessary, but it's definitely needed. Cheers, Andy From mindtonic at yahoo.com Thu Sep 6 07:12:12 2007 From: mindtonic at yahoo.com (Jay Sanders) Date: Thu, 6 Sep 2007 13:12:12 +0200 Subject: [Ferret-talk] acts_as_ferret connection was refused Message-ID: <00798128a4630d14a683648e248e8a55@ruby-forum.com> Hello, I tried to install the acts_as_ferret plugin: ruby script/plugin install svn://projects.jkraemer.net/acts_as_ferret/tags/stable/acts_as_ferret but I get the response svn: Can't connect to host 'projects.jkraemer.net': connection refused. Everything ok over there or do I need to use a different route? Thanks for your help! Jay Sanders -- Posted via http://www.ruby-forum.com/. From mindtonic at yahoo.com Thu Sep 6 08:07:13 2007 From: mindtonic at yahoo.com (Jay Sanders) Date: Thu, 6 Sep 2007 14:07:13 +0200 Subject: [Ferret-talk] acts_as_ferret connection was refused In-Reply-To: <00798128a4630d14a683648e248e8a55@ruby-forum.com> References: <00798128a4630d14a683648e248e8a55@ruby-forum.com> Message-ID: Sorry, got the gem installed and it works fine. I do have another question, when I load the console, I get the following warning: /usr/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.1/lib/ferret_server.rb:123: warning: parenthesize argument(s) for future version any idea what this is about? Thanks! -- Posted via http://www.ruby-forum.com/. From neongrau at gmail.com Thu Sep 6 08:37:09 2007 From: neongrau at gmail.com (neongrau __) Date: Thu, 6 Sep 2007 14:37:09 +0200 Subject: [Ferret-talk] aaf index rebuild problems on windows In-Reply-To: <20070905100605.GB12765@cordoba.webit.de> References: <47a683031249a1d57c2ff57a23c0e056@ruby-forum.com> <20070905091446.GD8344@cordoba.webit.de> <1db6295d52708f29ebc931d68c36f4c8@ruby-forum.com> <20070905100605.GB12765@cordoba.webit.de> Message-ID: <3ee57b11319121e0061f0ef93bc31fc8@ruby-forum.com> hi jens! just a short question... Jens Kraemer wrote: > > Yeah. The drb server gets around this by doing the rebuild in a separate > directory. > how exactly does this work? - the index gets built in a temp dir named "rebuild" - when indexing is complete the "rebuild" gets renamed to a timestamp like "20070906122218" now does the search automatically use the index with the latest timestamp? by now i have 3 timestamped dirs with indexes and the old index directly in that models dir. how many old versions are kept? or is this again some stupid behavior by windows? gr?sse ralf -- Posted via http://www.ruby-forum.com/. From radu at rdconcept.ro Thu Sep 6 14:03:20 2007 From: radu at rdconcept.ro (Marcelo Barbudas) Date: Thu, 6 Sep 2007 20:03:20 +0200 Subject: [Ferret-talk] multi_search with dfferent model conditions Message-ID: <85f8bda77df7ddd7d56cfb8b5005677b@ruby-forum.com> Hi, Is there a way to pass multi_search separate conditions for each model? -- R. -- Posted via http://www.ruby-forum.com/. From moigg2000 at yahoo.fr Thu Sep 6 17:53:39 2007 From: moigg2000 at yahoo.fr (Guillaume Differenthink) Date: Thu, 6 Sep 2007 23:53:39 +0200 Subject: [Ferret-talk] Find by contents and missing ferret_score method Message-ID: Hi, i m using acts as ferret via the find_by_contents, and then i my trying to get the score of each results... But the score of ferret_score method is missing... I ve got the @total_hits in the array, but not the @score one :/ What s going on ? Is this method disappear in acts_as_ferret ? thanks Guillaume. -- Posted via http://www.ruby-forum.com/. From doug.arogos at gmail.com Thu Sep 6 18:24:11 2007 From: doug.arogos at gmail.com (Doug Smith) Date: Thu, 6 Sep 2007 15:24:11 -0700 Subject: [Ferret-talk] Find by contents and missing ferret_score method In-Reply-To: References: Message-ID: <42d8808f0709061524n53ce9e3ds993f849110e3657e@mail.gmail.com> You're looking for "ferret_score". Each object returned by find_by_contents has that method available. Thanks, Doug On 9/6/07, Guillaume Differenthink wrote: > Hi, > > i m using acts as ferret via the find_by_contents, and then i my trying > to get the score of each results... > But the score of ferret_score method is missing... > > I ve got the @total_hits in the array, but not the @score one :/ > > What s going on ? > Is this method disappear in acts_as_ferret ? > > thanks > > Guillaume. > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From moigg2000 at yahoo.fr Thu Sep 6 18:45:51 2007 From: moigg2000 at yahoo.fr (Guillaume Differenthink) Date: Fri, 7 Sep 2007 00:45:51 +0200 Subject: [Ferret-talk] Find by contents and missing ferret_score method In-Reply-To: <42d8808f0709061524n53ce9e3ds993f849110e3657e@mail.gmail.com> References: <42d8808f0709061524n53ce9e3ds993f849110e3657e@mail.gmail.com> Message-ID: Well my problem is that there is not :/ I can assure you !!! i do : @videos = Video.find_by_contents(mystuff) and then if i do for video in @video video.ferret_score end i get : undefined method `ferret_score' for # any clues ? Thanks for you reply -- Posted via http://www.ruby-forum.com/. From doug.arogos at gmail.com Thu Sep 6 19:49:34 2007 From: doug.arogos at gmail.com (Doug Smith) Date: Thu, 6 Sep 2007 16:49:34 -0700 Subject: [Ferret-talk] Find by contents and missing ferret_score method In-Reply-To: References: <42d8808f0709061524n53ce9e3ds993f849110e3657e@mail.gmail.com> Message-ID: <42d8808f0709061649v1b619a98yba87cbe223957794@mail.gmail.com> Well, the very similar code works for me. What version of things are you using? I'm using: Ruby 1.8.6 Rails 1.2.3 Ferret 0.11.4 acts_as_ferret 0.4.1 Thanks, Doug On 9/6/07, Guillaume Differenthink wrote: > Well my problem is that there is not :/ > > I can assure you !!! > i do : > > @videos = Video.find_by_contents(mystuff) > > and then if i do > for video in @video > video.ferret_score > end > > i get : undefined method `ferret_score' for # > > any clues ? > > Thanks for you reply > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From moigg2000 at yahoo.fr Fri Sep 7 03:20:18 2007 From: moigg2000 at yahoo.fr (Guillaume Differenthink) Date: Fri, 7 Sep 2007 09:20:18 +0200 Subject: [Ferret-talk] Find by contents and missing ferret_score method In-Reply-To: <42d8808f0709061649v1b619a98yba87cbe223957794@mail.gmail.com> References: <42d8808f0709061524n53ce9e3ds993f849110e3657e@mail.gmail.com> <42d8808f0709061649v1b619a98yba87cbe223957794@mail.gmail.com> Message-ID: <99d6b04daffa50ea175add71aa12d196@ruby-forum.com> FOUND THE PROBLEM !!! Do not use acts_as_ferret gem !!!!.. thanks doug.. Doug Smith wrote: > Well, the very similar code works for me. What version of things are > you using? I'm using: > > Ruby 1.8.6 > Rails 1.2.3 > Ferret 0.11.4 > acts_as_ferret 0.4.1 > > Thanks, > > Doug -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Sep 7 05:04:42 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 7 Sep 2007 11:04:42 +0200 Subject: [Ferret-talk] Find by contents and missing ferret_score method In-Reply-To: <99d6b04daffa50ea175add71aa12d196@ruby-forum.com> References: <42d8808f0709061524n53ce9e3ds993f849110e3657e@mail.gmail.com> <42d8808f0709061649v1b619a98yba87cbe223957794@mail.gmail.com> <99d6b04daffa50ea175add71aa12d196@ruby-forum.com> Message-ID: <20070907090442.GN12765@cordoba.webit.de> On Fri, Sep 07, 2007 at 09:20:18AM +0200, Guillaume Differenthink wrote: > FOUND THE PROBLEM !!! > > Do not use acts_as_ferret gem !!!!.. this should work with the gem, too, if you have the most recent version (0.4.1) of it installed. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Fri Sep 7 05:12:57 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 7 Sep 2007 11:12:57 +0200 Subject: [Ferret-talk] multi_search with dfferent model conditions In-Reply-To: <85f8bda77df7ddd7d56cfb8b5005677b@ruby-forum.com> References: <85f8bda77df7ddd7d56cfb8b5005677b@ruby-forum.com> Message-ID: <20070907091257.GO12765@cordoba.webit.de> On Thu, Sep 06, 2007 at 08:03:20PM +0200, Marcelo Barbudas wrote: > Hi, > > Is there a way to pass multi_search separate conditions for each model? Not yet, but it should not be too hard to implement. Maybe create a ticket at projects.jkraemer.net/acts_as_ferret. A patch would be even better, of course :-) Cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Fri Sep 7 05:15:34 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 7 Sep 2007 11:15:34 +0200 Subject: [Ferret-talk] aaf index rebuild problems on windows In-Reply-To: <3ee57b11319121e0061f0ef93bc31fc8@ruby-forum.com> References: <47a683031249a1d57c2ff57a23c0e056@ruby-forum.com> <20070905091446.GD8344@cordoba.webit.de> <1db6295d52708f29ebc931d68c36f4c8@ruby-forum.com> <20070905100605.GB12765@cordoba.webit.de> <3ee57b11319121e0061f0ef93bc31fc8@ruby-forum.com> Message-ID: <20070907091534.GP12765@cordoba.webit.de> On Thu, Sep 06, 2007 at 02:37:09PM +0200, neongrau __ wrote: > hi jens! > > just a short question... > > Jens Kraemer wrote: > > > > Yeah. The drb server gets around this by doing the rebuild in a separate > > directory. > > > > how exactly does this work? > > - the index gets built in a temp dir named "rebuild" > - when indexing is complete the "rebuild" gets renamed to a timestamp > like "20070906122218" Exactly. > now does the search automatically use the index with the latest > timestamp? Yes. the DRb server switches over to the new index as soon as it's ready. > by now i have 3 timestamped dirs with indexes and the old index directly > in that models dir. how many old versions are kept? Aaf doesn't destroy old indexes, you have to do this yourself. You can also remove the old index directly in that models dir, aaf won't use it if there is any other index version present. > or is this again some stupid behavior by windows? not this time :-) cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From kraemer at webit.de Fri Sep 7 05:17:28 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 7 Sep 2007 11:17:28 +0200 Subject: [Ferret-talk] acts_as_ferret connection was refused In-Reply-To: References: <00798128a4630d14a683648e248e8a55@ruby-forum.com> Message-ID: <20070907091728.GQ12765@cordoba.webit.de> On Thu, Sep 06, 2007 at 02:07:13PM +0200, Jay Sanders wrote: > Sorry, got the gem installed and it works fine. I do have another > question, when I load the console, I get the following warning: > > /usr/lib/ruby/gems/1.8/gems/acts_as_ferret-0.4.1/lib/ferret_server.rb:123: > warning: parenthesize argument(s) for future version > > any idea what this is about? just a warning for lazy programmers to use more parantheses :-) problems with svn most often are caused by firewalls preventing access to port 3960, which the svnserver uses. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From bmihelac at mihelac.org Fri Sep 7 05:59:19 2007 From: bmihelac at mihelac.org (Bojan Mihelac) Date: Fri, 7 Sep 2007 11:59:19 +0200 Subject: [Ferret-talk] Is it possible to work without DRB Message-ID: Hi, I am looking for solution to work with ferret without DRB. I would like to turn off ferret indexing from application and use existing ferret index only for searching (read only), while I would create indexes once a day from script. Is it possible and how to turn indexing off and would ferret still complain if 2 mongrels try to open index file. thanks, Bojan -- Posted via http://www.ruby-forum.com/. From moigg2000 at yahoo.fr Fri Sep 7 10:21:56 2007 From: moigg2000 at yahoo.fr (Guillaume Differenthink) Date: Fri, 7 Sep 2007 16:21:56 +0200 Subject: [Ferret-talk] Custom Analyser .. where to put it ?? Message-ID: <41583cdffa9de1d213806123a4bf35ef@ruby-forum.com> Hi, I m trying to use a custom analyser to add my french stop words... i m reading the tutorial at : http://projects.jkraemer.net/acts_as_ferret/wiki/AdvancedUsage My problem is that i ve no idea where to put my custom Analyser class like : class GermanStemmingAnalyzer < Ferret::Analysis::Analyzer include Ferret::Analysis def initialize(stop_words = FULL_GERMAN_STOP_WORDS) @stop_words = stop_words end def token_stream(field, str) StemFilter.new(StopFilter.new(LowerCaseFilter.new(StandardTokenizer.new(str)), @stop_words), 'de') end end Any clue ? Thanks a lot Guillaume. -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Fri Sep 7 10:49:06 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Fri, 7 Sep 2007 16:49:06 +0200 Subject: [Ferret-talk] Is it possible to work without DRB In-Reply-To: References: Message-ID: <07779A0E-8EFD-4B72-A603-CD0A402563D3@benjaminkrause.com> Hey .. are you talking about ferret, or acts_as_ferret? its no problem with ferret, with acts_as_ferret, there is a flag on your model, see Model.enable_ferret and Model.disable_ferret Not sure if you can still search, but i guess that might be possible, as this might only disable the after_save hooks .. give it a try :) Ben On 2007-09-07, at 11:59 AM, Bojan Mihelac wrote: > Hi, > I am looking for solution to work with ferret without DRB. I would > like > to turn off ferret indexing from application and use existing ferret > index only for searching (read only), while I would create indexes > once > a day from script. Is it possible and how to turn indexing off and > would > ferret still complain if 2 mongrels try to open index file. > > thanks, > Bojan > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From bk at benjaminkrause.com Fri Sep 7 10:52:52 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Fri, 7 Sep 2007 16:52:52 +0200 Subject: [Ferret-talk] Custom Analyser .. where to put it ?? In-Reply-To: <41583cdffa9de1d213806123a4bf35ef@ruby-forum.com> References: <41583cdffa9de1d213806123a4bf35ef@ruby-forum.com> Message-ID: <02BC2556-A2AD-4E26-B2F0-8206CD07577F@benjaminkrause.com> Hey .. you mean where to place it in your directory structure? I place them in lib/ but any place is fine.. maybe in model/ .. just make sure it's in the load_path of rails. :-) Ben On 2007-09-07, at 4:21 PM, Guillaume Differenthink wrote: > Hi, > > I m trying to use a custom analyser to add my french stop words... i m > reading the tutorial at : > http://projects.jkraemer.net/acts_as_ferret/wiki/AdvancedUsage > > My problem is that i ve no idea where to put my custom Analyser class > like : > > class GermanStemmingAnalyzer < Ferret::Analysis::Analyzer > include Ferret::Analysis > def initialize(stop_words = FULL_GERMAN_STOP_WORDS) > @stop_words = stop_words > end > def token_stream(field, str) > StemFilter.new(StopFilter.new(LowerCaseFilter.new > (StandardTokenizer.new(str)), > @stop_words), 'de') > end > end > > Any clue ? > > Thanks a lot > > Guillaume. > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From kraemer at webit.de Fri Sep 7 10:55:03 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 7 Sep 2007 16:55:03 +0200 Subject: [Ferret-talk] Custom Analyser .. where to put it ?? In-Reply-To: <41583cdffa9de1d213806123a4bf35ef@ruby-forum.com> References: <41583cdffa9de1d213806123a4bf35ef@ruby-forum.com> Message-ID: <20070907145503.GA6338@cordoba.webit.de> Hi, #{RAILS_ROOT}/lib is a good place for things like this. If you name your file correctly, i.e. german_stemming_analyzer.rb, Rails will auto-load it when you use the class name. Jens On Fri, Sep 07, 2007 at 04:21:56PM +0200, Guillaume Differenthink wrote: > Hi, > > I m trying to use a custom analyser to add my french stop words... i m > reading the tutorial at : > http://projects.jkraemer.net/acts_as_ferret/wiki/AdvancedUsage > > My problem is that i ve no idea where to put my custom Analyser class > like : > > class GermanStemmingAnalyzer < Ferret::Analysis::Analyzer > include Ferret::Analysis > def initialize(stop_words = FULL_GERMAN_STOP_WORDS) > @stop_words = stop_words > end > def token_stream(field, str) > StemFilter.new(StopFilter.new(LowerCaseFilter.new(StandardTokenizer.new(str)), > @stop_words), 'de') > end > end > > Any clue ? > > Thanks a lot > > Guillaume. > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From moigg2000 at yahoo.fr Fri Sep 7 11:27:03 2007 From: moigg2000 at yahoo.fr (Guillaume Differenthink) Date: Fri, 7 Sep 2007 17:27:03 +0200 Subject: [Ferret-talk] Custom Analyser .. where to put it ?? In-Reply-To: <20070907145503.GA6338@cordoba.webit.de> References: <41583cdffa9de1d213806123a4bf35ef@ruby-forum.com> <20070907145503.GA6338@cordoba.webit.de> Message-ID: <5444996b760cf4668155fffbc68dfc63@ruby-forum.com> thanks, i ll try :) Jens Kraemer wrote: > Hi, > > #{RAILS_ROOT}/lib is a good place for things like this. If you name your > file correctly, i.e. german_stemming_analyzer.rb, Rails will auto-load > it when you use the class name. > > Jens > -- Posted via http://www.ruby-forum.com/. From bkelly01 at yahoo.com Fri Sep 7 14:18:52 2007 From: bkelly01 at yahoo.com (Brandon Kelly) Date: Fri, 7 Sep 2007 20:18:52 +0200 Subject: [Ferret-talk] Ferret DRB - can add/edit index, but can't search Message-ID: I'm trying to use the Ferret DRB server to avoid concurrency issues when using multiple mongrels. I can successfully add and edit data on my index via the DRB server, however, when I search the index, I get the following error: DRb::DRbConnError (DRb::DRbServerNotFound): /usr/lib/ruby/1.8/drb/drb.rb:1647:in `current_server' /usr/lib/ruby/1.8/drb/drb.rb:1709:in `to_id' /usr/lib/ruby/1.8/drb/drb.rb:1045:in `initialize' /usr/lib/ruby/1.8/drb/drb.rb:639:in `make_proxy' /usr/lib/ruby/1.8/drb/drb.rb:556:in `dump' /usr/lib/ruby/1.8/drb/drb.rb:600:in `send_request' /usr/lib/ruby/1.8/drb/drb.rb:599:in `send_request' /usr/lib/ruby/1.8/drb/drb.rb:903:in `send_request' /usr/lib/ruby/1.8/drb/drb.rb:1191:in `send_message' /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing' /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open' /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing' /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend' /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing' /vendor/plugins/acts_as_ferret/lib/remote_index.rb:20:in `find_id_by_contents' /vendor/plugins/acts_as_ferret/lib/class_methods.rb:120:in `find_id_by_contents' /vendor/plugins/acts_as_ferret/lib/class_methods.rb:176:in `ar_find_by_contents' /vendor/plugins/acts_as_ferret/lib/class_methods.rb:170:in `find_records_lazy_or_not' /vendor/plugins/acts_as_ferret/lib/class_methods.rb:86:in `find_by_contents' /app/models/article.rb:104:in `full_text_search' /app/controllers/search_controller.rb:93:in `index' /usr/lib/ruby/gems/1.8/gems/actionpack-1.13.3/lib/action_controller/base.rb:1095:in `perform_action_without_filters' /usr/lib/ruby/gems/1.8/gems/actionpack-1.13.3/lib/action_controller/filters.rb:632:in `call_filter' /usr/lib/ruby/gems/1.8/gems/actionpack-1.13.3/lib/action_controller/filters.rb:619:in `perform_action_without_benchmark' /usr/lib/ruby/gems/1.8/gems/actionpack-1.13.3/lib/action_controller/benchmarking.rb:66:in `perform_action_without_rescue' /usr/lib/ruby/1.8/benchmark.rb:293:in `measure' /usr/lib/ruby/gems/1.8/gems/actionpack-1.13.3/lib/action_controller/benchmarking.rb:66:in `perform_action_without_rescue' /usr/lib/ruby/gems/1.8/gems/actionpack-1.13.3/lib/action_controller/rescue.rb:83:in `perform_action' /usr/lib/ruby/gems/1.8/gems/actionpack-1.13.3/lib/action_controller/base.rb:430:in `process_without_filters' /usr/lib/ruby/gems/1.8/gems/actionpack-1.13.3/lib/action_controller/filters.rb:624:in `process_without_session_management_support' /usr/lib/ruby/gems/1.8/gems/actionpack-1.13.3/lib/action_controller/session_management.rb:114:in `process' /usr/lib/ruby/gems/1.8/gems/actionpack-1.13.3/lib/action_controller/base.rb:330:in `process' /usr/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/dispatcher.rb:41:in `dispatch' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/rails.rb:78:in `process' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/rails.rb:76:in `process' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel.rb:618:in `process_client' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel.rb:617:in `process_client' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel.rb:736:in `run' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel.rb:736:in `run' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel.rb:720:in `run' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/configurator.rb:271:in `run' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/configurator.rb:270:in `run' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/bin/mongrel_rails:127:in `run' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/command.rb:211:in `run' /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/bin/mongrel_rails:243 /usr/bin/mongrel_rails:16 ----------------- When I'm adding/editing data in the index, my ferret_server.log file shows the following: #method_missing(:add, ["Article", {:synopsis=>"Analysis and Projections", :status=>1, :player_names=>"", :team_names=>"", :publisher=>"", :title=>"Article Title", :db_state=>0, :id=>1, :article_published_date=>1189115220}]) #method_missing(:add, ["Article", {:synopsis=>"Analysis and Projections", :status=>1, :player_names=>"", :team_names=>"", :publisher=>"", :title=>"Article Title 1", :db_state=>0, :id=>1, :article_published_date=>1189115220}]) #method_missing(:add, ["Player", {:last_name=>"Smith", :first_name=>"Walter", :db_state=>0, :id=>1}]) #method_missing(:add, ["Team", {:name=>"Kalispell HS", :db_state=>0, :id=>13}]) ----------------- I'm running the following environment: Linux version 2.6.16.29-xen_3.0.3.0 mongrel (1.0.1) mongrel_cluster (1.0.2, 0.2.1) ferret (0.11.4) acts_as_ferret stable plugin (as of 9/7/2007) I have no problem running this application on my local development environment... and I had no problems running in this hosting environment before I started using Ferret DRB. Here are some example sorts and/or filters I'm using in my searches (not sure if this matters to Ferret DRB or not): Ferret::Search::SortField.new(:article_published_date, :reverse => true) Ferret::Search::Sort.new([Ferret::Search::SortField::SCORE, date_sort]) Ferret::Search::RangeQuery.new(:article_published_date, :>= => UserSystem::APP_DEFAULTS[:days_for_new].days.ago.utc.to_i.to_s, :<= => Time.now.utc.to_i.to_s) Ferret::Search::Sort.new([Ferret::Search::SortField.new(:article_published_date, :reverse => true)]) Any ideas why my searches wouldn't be working? -- Posted via http://www.ruby-forum.com/. From bmihelac at mihelac.org Fri Sep 7 18:05:12 2007 From: bmihelac at mihelac.org (Bojan Mihelac) Date: Sat, 8 Sep 2007 00:05:12 +0200 Subject: [Ferret-talk] Is it possible to work without DRB In-Reply-To: <07779A0E-8EFD-4B72-A603-CD0A402563D3@benjaminkrause.com> References: <07779A0E-8EFD-4B72-A603-CD0A402563D3@benjaminkrause.com> Message-ID: Benjamin Krause wrote: > Hey .. > > are you talking about ferret, or acts_as_ferret? > > its no problem with ferret, with acts_as_ferret, there is a flag on > your model, see > > Model.enable_ferret and Model.disable_ferret > > Not sure if you can still search, but i guess that might be possible, > as this might only disable the after_save hooks .. give it a try :) > > Ben Hi Benjamin, I was thinking about disabling ferret from acts_as_ferret. It does help to Model.disable_ferret (in environment.rb if anyone is interested) and it is possible to search. Thank you! Bojan -- Posted via http://www.ruby-forum.com/. From bkelly01 at yahoo.com Sat Sep 8 09:44:36 2007 From: bkelly01 at yahoo.com (Brandon Kelly) Date: Sat, 8 Sep 2007 15:44:36 +0200 Subject: [Ferret-talk] Ferret DRB - can add/edit index, but can't search In-Reply-To: References: Message-ID: <81a917d248c36bdc947fb3d99fb52d89@ruby-forum.com> I wanted to note the Ruby and Rails versions that I'm running, in case that helps. > Linux version 2.6.16.29-xen_3.0.3.0 + Ruby 1.8.4 (2005-12-24 [i386-linux] + Rails 1.2.3 > mongrel (1.0.1) > mongrel_cluster (1.0.2, 0.2.1) > ferret (0.11.4) > acts_as_ferret stable plugin (as of 9/7/2007) -- Posted via http://www.ruby-forum.com/. From fausto.galli at email.it Sat Sep 8 15:24:18 2007 From: fausto.galli at email.it (Mix Mix) Date: Sat, 8 Sep 2007 21:24:18 +0200 Subject: [Ferret-talk] create index as other user/group, how? Message-ID: hi, i'm running a mongrel cluster plus apache and i've a user project:project which access the files. actually i run mongrel as root because otherwise it doesn't work, i don't know why, maybe because it can't access the /usr directory... but with this when something is added ferret create its indexes as root:root, how can i change and set it to create them as project:project? is it possible? i've set also the index directory with chmod -R u+s index && chmod -R g+s index, but they are still created as root... :( thanks -- Posted via http://www.ruby-forum.com/. From matthew.langham at indiginox.com Sun Sep 9 05:14:10 2007 From: matthew.langham at indiginox.com (Matthew Langham) Date: Sun, 9 Sep 2007 11:14:10 +0200 Subject: [Ferret-talk] Share monit.rc entry for Ferret Drb? Message-ID: <7a0aedceacb44f417299d0a34683a3e7@ruby-forum.com> Hi, we're using monit to watch over our mongrels and I want to add an entry for the ferret Drb server. Would anyone care to share their montit.rc entry? Thanks Matthew -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Sun Sep 9 06:34:57 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Sun, 9 Sep 2007 12:34:57 +0200 Subject: [Ferret-talk] create index as other user/group, how? In-Reply-To: References: Message-ID: <20070909103457.GE16997@thunder.jkraemer.net> On Sat, Sep 08, 2007 at 09:24:18PM +0200, Mix Mix wrote: > hi, i'm running a mongrel cluster plus apache and i've a user > project:project which access the files. actually i run mongrel as root > because otherwise it doesn't work, i don't know why, maybe because it > can't access the /usr directory... but with this when something is added > ferret create its indexes as root:root, how can i change and set it to > create them as project:project? is it possible? i've set also the index > directory with chmod -R u+s index && chmod -R g+s index, but they are > still created as root... :( You really should fix your mongrel setup, so it runs as a non-privileged user. Running web servers (or any servers reachable from the outside) as root is a major security issue. To do so, check if the places mongrel needs to write to (dir where PID files go, dir where log files go), are writable by that user, i.e. do 'su - project' and then try to create a file there. If that doesn't work, either change these locations to somewhere else, i.e. somewhere in the home directory of your project user, or make them writable by the project user. cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Sun Sep 9 06:43:27 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Sun, 9 Sep 2007 12:43:27 +0200 Subject: [Ferret-talk] Ferret DRB - can add/edit index, but can't search In-Reply-To: References: Message-ID: <20070909104327.GF16997@thunder.jkraemer.net> Hi Brandon, transmitting objects from Ferret's API across the DRb connection is always a bit tricky. If these don't serialize correctly, DRb only transmits a stub and then tries to talk back to the original object via a DRb connection from the server to your web server process, which results in the message you see. Sort and SortFields should work, but I'm not sure if I introducted the one or the other after the 0.4.1 release. Imho, Sort objects can simply be replaced by an array holding the SortFields. For the RangeQueries and other query objects that fail the way you describe, please try using FQL instead to see if this fixes the problem. cheers, Jens On Fri, Sep 07, 2007 at 08:18:52PM +0200, Brandon Kelly wrote: > I'm trying to use the Ferret DRB server to avoid concurrency issues when > using multiple mongrels. I can successfully add and edit data on my > index via the DRB server, however, when I search the index, I get the > following error: > > DRb::DRbConnError (DRb::DRbServerNotFound): > /usr/lib/ruby/1.8/drb/drb.rb:1647:in `current_server' [..] > ----------------- > > I'm running the following environment: > > Linux version 2.6.16.29-xen_3.0.3.0 > mongrel (1.0.1) > mongrel_cluster (1.0.2, 0.2.1) > ferret (0.11.4) > acts_as_ferret stable plugin (as of 9/7/2007) > > > I have no problem running this application on my local development > environment... and I had no problems running in this hosting environment > before I started using Ferret DRB. > > Here are some example sorts and/or filters I'm using in my searches (not > sure if this matters to Ferret DRB or not): > > Ferret::Search::SortField.new(:article_published_date, :reverse => true) > > Ferret::Search::Sort.new([Ferret::Search::SortField::SCORE, date_sort]) > > Ferret::Search::RangeQuery.new(:article_published_date, :>= => > UserSystem::APP_DEFAULTS[:days_for_new].days.ago.utc.to_i.to_s, :<= => > Time.now.utc.to_i.to_s) > > Ferret::Search::Sort.new([Ferret::Search::SortField.new(:article_published_date, > :reverse => true)]) > > > Any ideas why my searches wouldn't be working? > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From bk at benjaminkrause.com Sun Sep 9 16:23:14 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Sun, 09 Sep 2007 22:23:14 +0200 Subject: [Ferret-talk] create index as other user/group, how? In-Reply-To: <20070909103457.GE16997@thunder.jkraemer.net> References: <20070909103457.GE16997@thunder.jkraemer.net> Message-ID: <46E45632.5020708@benjaminkrause.com> Hey .. > You really should fix your mongrel setup, so it runs as a non-privileged > user. Running web servers (or any servers reachable from the outside) as > root is a major security issue. > i totally agree .. take a look at mongrel_cluster to start/stop mongrel.. you can set a user and group like that: user: rails cwd: /var/www/localhost/rails/current port: "8010" environment: production group: apache address: 127.0.0.1 pid_file: log/mongrel.pid servers: 6 Ben From moigg2000 at yahoo.fr Sun Sep 9 18:09:03 2007 From: moigg2000 at yahoo.fr (Guillaume Differenthink) Date: Mon, 10 Sep 2007 00:09:03 +0200 Subject: [Ferret-talk] Custom Analyser .. where to put it ?? In-Reply-To: <5444996b760cf4668155fffbc68dfc63@ruby-forum.com> References: <41583cdffa9de1d213806123a4bf35ef@ruby-forum.com> <20070907145503.GA6338@cordoba.webit.de> <5444996b760cf4668155fffbc68dfc63@ruby-forum.com> Message-ID: Hi, i ve tried, but without success.... here is what i did, please tell me if something is wrong : In my class: acts_as_ferret({ :fields => { :nom => {}, :description => {:boost => 0}, :logiciel_nom => {}, :logiciel_id => {}, :difficulte_id => {}, :systeme_nom => {}, :fai_nom => {}, :fai_id =>{}, :lversion_id=>{}, :site_nom => {}, :siteutilise_id => {}, :nom_for_sort => {:index => :untokenized}, :note => {:index => :untokenized}, :visions_count => {:index => :untokenized}, :nb_vu => {:index => :untokenized}, :date_sort => {:index => :untokenized} }}, :analyzer => FrenchStemmingAnalyzer.new) My FrenchStemmingAnalyzer.rb (in /lib).. class FrenchStemmingAnalyzer < Ferret::Analysis::Analyzer include Ferret::Analysis def initialize(stop_words = FULL_FRENCH_STOP_WORDS) @stop_words = stop_words end def token_stream(field, str) StemFilter.new(StopFilter.new(LowerCaseFilter.new(StandardTokenizer.new(str)), @stop_words), 'fr') end end there is no error, i erased the /index/ form previous ferret analysis.. i restarted, did a search... but stil same results that before, even using only stop words... I do a fuzzy search with ferret (via find_by_content), could it be the cause of the problem ? Or maybe there is an error on the code ? Thanks you for helping, Guillaume. -- Posted via http://www.ruby-forum.com/. From john at johnleach.co.uk Sun Sep 9 19:17:21 2007 From: john at johnleach.co.uk (John Leach) Date: Mon, 10 Sep 2007 00:17:21 +0100 Subject: [Ferret-talk] MultiTerm query segfault - remote denial of service Message-ID: <1189379841.8534.33.camel@localhost.localdomain> Hi ferreteers, I came across a segfault in the query parser. It had already been reported[1] and fixed[2], but it can lead to a denial of service. If you use Ferret anywhere that you allow users to execute queries, those users can crash your Ruby process with a specially crafted query. I'm sure you're all using Monit or something for your Rails or drb processes, so they'll get restarted, but it's still not good. This was quite serious for a number of my sites (not to mention slowing development of a current app) so I applied the fix to the released 0.11.4 source and repackaged it as 0.11.4.1. Obviously this isn't in any way official, but it works for me and I'm sharing here for anyone else affected. Gem, tgz and zip here: http://johnleach.co.uk/downloads/ruby/ferret/ferret-0.11.4.1/ and just the patch (derived from Dave's changeset to trunk) here: http://johnleach.co.uk/downloads/ruby/ferret/ferret-0.11.4-fix-multiterm-segfault.patch The patch is against the release source, as the subversion repository seems to be down atm. John. [1] http://ferret.davebalmain.com/trac/ticket/208 [2] http://ferret.davebalmain.com/trac/changeset/773 -- high-profile mailing list advertising space exploitation: http://www.brightbox.co.uk - UK Rails Xen Hosting From bkelly01 at yahoo.com Sun Sep 9 23:13:41 2007 From: bkelly01 at yahoo.com (Brandon Kelly) Date: Mon, 10 Sep 2007 05:13:41 +0200 Subject: [Ferret-talk] Ferret DRB - can add/edit index, but can't search In-Reply-To: <20070909104327.GF16997@thunder.jkraemer.net> References: <20070909104327.GF16997@thunder.jkraemer.net> Message-ID: <2c26151bbde66ead532d570d00864413@ruby-forum.com> Jens, Thanks for your help. I ripped out the Sort, SortField and RangeQuery objects, and sure enough - it worked. I'll try your suggestions tomorrow -- using the sort fields and FQL. Brandon Jens Kraemer wrote: > Hi Brandon, > > transmitting objects from Ferret's API across the DRb connection is > always a bit tricky. If these don't serialize correctly, DRb only > transmits a stub and then tries to talk back to the original object > via a DRb connection from the server to your web server process, which > results in the message you see. > > Sort and SortFields should work, but I'm not sure if I introducted the > one or the other after the 0.4.1 release. Imho, Sort objects can simply > be replaced by an array holding the SortFields. > > For the RangeQueries and other query objects that fail the way you > describe, please try using FQL instead to see if this fixes the problem. > > cheers, > Jens -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Mon Sep 10 03:37:36 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Mon, 10 Sep 2007 09:37:36 +0200 Subject: [Ferret-talk] MultiTerm query segfault - remote denial of service In-Reply-To: <1189379841.8534.33.camel@localhost.localdomain> References: <1189379841.8534.33.camel@localhost.localdomain> Message-ID: cool.. thank you :) On 2007-09-10, at 1:17 AM, John Leach wrote: > Hi ferreteers, > > I came across a segfault in the query parser. It had already been > reported[1] and fixed[2], but it can lead to a denial of service. > > If you use Ferret anywhere that you allow users to execute queries, > those users can crash your Ruby process with a specially crafted > query. > > I'm sure you're all using Monit or something for your Rails or drb > processes, so they'll get restarted, but it's still not good. > > This was quite serious for a number of my sites (not to mention > slowing > development of a current app) so I applied the fix to the released > 0.11.4 source and repackaged it as 0.11.4.1. > > Obviously this isn't in any way official, but it works for me and I'm > sharing here for anyone else affected. Gem, tgz and zip here: > > http://johnleach.co.uk/downloads/ruby/ferret/ferret-0.11.4.1/ > > and just the patch (derived from Dave's changeset to trunk) here: > > http://johnleach.co.uk/downloads/ruby/ferret/ferret-0.11.4-fix- > multiterm-segfault.patch > > The patch is against the release source, as the subversion repository > seems to be down atm. > > John. > > [1] http://ferret.davebalmain.com/trac/ticket/208 > [2] http://ferret.davebalmain.com/trac/changeset/773 > > > -- > high-profile mailing list advertising space exploitation: > http://www.brightbox.co.uk - UK Rails Xen Hosting > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From fausto.galli at email.it Mon Sep 10 11:28:13 2007 From: fausto.galli at email.it (Mix Mix) Date: Mon, 10 Sep 2007 17:28:13 +0200 Subject: [Ferret-talk] create index as other user/group, how? In-Reply-To: <46E45632.5020708@benjaminkrause.com> References: <20070909103457.GE16997@thunder.jkraemer.net> <46E45632.5020708@benjaminkrause.com> Message-ID: <3c73a52d72b90a9a1d16830ece70c89d@ruby-forum.com> Benjamin Krause wrote: > Hey .. >> You really should fix your mongrel setup, so it runs as a non-privileged >> user. Running web servers (or any servers reachable from the outside) as >> root is a major security issue. >> > i totally agree .. take a look at mongrel_cluster to start/stop > mongrel.. you can set > a user and group like that: Ok, fixed :) Time ago i tried to set another user and group, but the site didn't work anymore, now i've tried and discovered the problem, there were some session files created by root which the server couldn't access, changed the owner and now works... thanks :) -- Posted via http://www.ruby-forum.com/. From chris at nonfatal.com Mon Sep 10 12:22:41 2007 From: chris at nonfatal.com (Chris Williams) Date: Mon, 10 Sep 2007 18:22:41 +0200 Subject: [Ferret-talk] IO errors with 2 processes Message-ID: Hi All, I'm using acts as ferret and getting a lot of these errors: ---- Error occured in fs_store.c:324 - fs_open_input couldn't create InStream /var/rails/projectblue/website/public/../config/../index/production/content_page/_347i.fdt: ---- and also some of these errors: ---- uninitialized constant Ferret::Index::Index::LockError [RAILS_ROOT]/vendor/rails/activesupport/lib/active_support/dependencies.rb:477:in `const_missing' ---- Now I've got a daemon I'm using to spider some webpages which is an independant process from the actual webserver. I thought I'd be able to get away with it, since the daemon is the only thing that writes to the database (the web server just does reads.) Do you think this is causing the problem? Any solutions? Thanks, Chris. -- Posted via http://www.ruby-forum.com/. From hakita at gmail.com Mon Sep 10 13:00:00 2007 From: hakita at gmail.com (Hakita Hakita) Date: Mon, 10 Sep 2007 19:00:00 +0200 Subject: [Ferret-talk] Asked for a remote server ? nil, ENV["FERRET_USE_LOCAL_IN... Message-ID: <9eebca078ef4baade48e2c3984a3aa86@ruby-forum.com> Hello, I know this subject as been talked many time, but i can not figure out how to get rid of this... ENV : ruby 1.8.4, ferret 0.11.4, acts_as_ferret 0.4.1 under win32 Error : Asked for a remote server ? nil, ENV["FERRET_USE_LOCAL_INDEX"] is nil, looks like we are not the server Will use local index. My application is working find, with 4 mongrels, but i got this message on every request... I tryed to add the following at the end of my environment.rb file, without sucess. FERRET_USE_LOCAL_INDEX=1 # didn't do anything FERRET_USE_LOCAL_INDEX=1 RAILS_ENV=production script/ferret_start # ruby did not start (exception) There is a "require 'acts_as_ferret'" at the beginning of my environment.rb I got the same problem with a Fedoracore5, using ferret 0.10.13 and acts_as_ferret 0.4.1 Thank for you precious help Jens -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Mon Sep 10 13:12:04 2007 From: kraemer at webit.de (Jens Kraemer) Date: Mon, 10 Sep 2007 19:12:04 +0200 Subject: [Ferret-talk] IO errors with 2 processes In-Reply-To: References: Message-ID: <20070910171204.GI1891@cordoba.webit.de> On Mon, Sep 10, 2007 at 06:22:41PM +0200, Chris Williams wrote: > Hi All, > I'm using acts as ferret and getting a lot of these errors: > ---- > Error occured in fs_store.c:324 - fs_open_input > couldn't create InStream > /var/rails/projectblue/website/public/../config/../index/production/content_page/_347i.fdt: > > ---- > and also some of these errors: > ---- > uninitialized constant Ferret::Index::Index::LockError > [RAILS_ROOT]/vendor/rails/activesupport/lib/active_support/dependencies.rb:477:in > `const_missing' > ---- > > Now I've got a daemon I'm using to spider some webpages which is an > independant process from the actual webserver. I thought I'd be able to > get away with it, since the daemon is the only thing that writes to the > database (the web server just does reads.) > > Do you think this is causing the problem? Any solutions? most probably it is. When you create (rebuild) a new index where the old one was, searches will fail because the old index isn't there anymore. Try building the new index in a separate directory, or use the DRb server which exactly does this. Cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From gonesail at gmail.com Mon Sep 10 17:19:22 2007 From: gonesail at gmail.com (alan kemp) Date: Mon, 10 Sep 2007 17:19:22 -0400 Subject: [Ferret-talk] ferret index file(s) Message-ID: I am running version 3.2 of ferret and would like to upgrade to a more current version. will my existing 3.2 index files be compatible? thanks, AK -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070910/742890f2/attachment.html From bk at benjaminkrause.com Mon Sep 10 17:35:34 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Mon, 10 Sep 2007 23:35:34 +0200 Subject: [Ferret-talk] ferret index file(s) In-Reply-To: References: Message-ID: <46E5B8A6.4080203@benjaminkrause.com> alan kemp schrieb: > I am running version 3.2 of ferret and would like to upgrade to a more > current version. will my existing 3.2 index files be compatible? Alan, are you sure about the version number? the current version of ferret is 0.11.4 .. so i'm not sure you've got the right number .. check it via "gem list ferret". Ben From gonesail at gmail.com Mon Sep 10 20:10:57 2007 From: gonesail at gmail.com (alan kemp) Date: Mon, 10 Sep 2007 20:10:57 -0400 Subject: [Ferret-talk] ferret index file(s) In-Reply-To: <46E5B8A6.4080203@benjaminkrause.com> References: <46E5B8A6.4080203@benjaminkrause.com> Message-ID: ok it is actually version 0.3.2 On 9/10/07, Benjamin Krause wrote: > > alan kemp schrieb: > > I am running version 3.2 of ferret and would like to upgrade to a more > > current version. will my existing 3.2 index files be compatible? > Alan, > > are you sure about the version number? the current version of ferret is > 0.11.4 .. so i'm not sure you've got the right number .. > > check it via "gem list ferret". > > Ben > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070910/ea7adf26/attachment.html From kraemer at webit.de Tue Sep 11 02:23:24 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 11 Sep 2007 08:23:24 +0200 Subject: [Ferret-talk] ferret index file(s) In-Reply-To: References: <46E5B8A6.4080203@benjaminkrause.com> Message-ID: <20070911062323.GJ1891@cordoba.webit.de> On Mon, Sep 10, 2007 at 08:10:57PM -0400, alan kemp wrote: > ok it is actually version 0.3.2 You'll have to rebuild your indexes, as 0.3.2 still used the lucene compatible index format. cheers, Jens > > On 9/10/07, Benjamin Krause wrote: > > > > alan kemp schrieb: > > > I am running version 3.2 of ferret and would like to upgrade to a more > > > current version. will my existing 3.2 index files be compatible? > > Alan, > > > > are you sure about the version number? the current version of ferret is > > 0.11.4 .. so i'm not sure you've got the right number .. > > > > check it via "gem list ferret". > > > > Ben > > > > _______________________________________________ > > Ferret-talk mailing list > > Ferret-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/ferret-talk > > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From 2828628 at gmail.com Tue Sep 11 03:12:04 2007 From: 2828628 at gmail.com (Ken Wei) Date: Tue, 11 Sep 2007 09:12:04 +0200 Subject: [Ferret-talk] MySQL has gone away In-Reply-To: <8CD83534-E85C-4A6B-BD48-B5B39024047A@gmx.net> References: <20070903142225.GW7512@cordoba.webit.de> <7103A74E-3E66-4AAC-B1C2-9D1F0625F3F9@benjaminkrause.com> <417C45B8-7F5E-4D4F-B6B1-5E7B28A41D72@gmx.net> <0a49c1a8c126205dd2746cad1f8c0403@ruby-forum.com> <8CD83534-E85C-4A6B-BD48-B5B39024047A@gmx.net> Message-ID: <106ec4d9dba5c3838d9a82abbe85777d@ruby-forum.com> I did a hack in the abstract connection code to enable the auto-reconnection property. Attachments: http://www.ruby-forum.com/attachment/264/active_record_fix.rb -- Posted via http://www.ruby-forum.com/. From chris at nonfatal.com Tue Sep 11 05:37:20 2007 From: chris at nonfatal.com (Chris Williams) Date: Tue, 11 Sep 2007 11:37:20 +0200 Subject: [Ferret-talk] IO errors with 2 processes In-Reply-To: <20070910171204.GI1891@cordoba.webit.de> References: <20070910171204.GI1891@cordoba.webit.de> Message-ID: <4caffcb38f294009e9f4025515e1f1b0@ruby-forum.com> Thanks for the reply, I'll do it. -- Posted via http://www.ruby-forum.com/. From chris at nonfatal.com Tue Sep 11 05:40:32 2007 From: chris at nonfatal.com (Chris Williams) Date: Tue, 11 Sep 2007 11:40:32 +0200 Subject: [Ferret-talk] aaf requires drb? Message-ID: <1f2cbe970e2849dcd21b976a4b7b4137@ruby-forum.com> Hi, I've just read some an article (http://www.jroller.com/BrightCandle/entry/oversights_in_rails_plugins_really.) This article (and the comment,) seem to imply that drb is required if running under a fastcgi kind of deployment. Is this true? Thanks, Chris. ps this is for a different server then my previous question. -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Tue Sep 11 06:09:27 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 11 Sep 2007 12:09:27 +0200 Subject: [Ferret-talk] aaf requires drb? In-Reply-To: <1f2cbe970e2849dcd21b976a4b7b4137@ruby-forum.com> References: <1f2cbe970e2849dcd21b976a4b7b4137@ruby-forum.com> Message-ID: <20070911100927.GO1891@cordoba.webit.de> On Tue, Sep 11, 2007 at 11:40:32AM +0200, Chris Williams wrote: > Hi, > I've just read some an article > (http://www.jroller.com/BrightCandle/entry/oversights_in_rails_plugins_really.) > > This article (and the comment,) seem to imply that drb is required if > running under a fastcgi kind of deployment. Is this true? Yes. Inspired by this post, I'll make the :remote => true option the default in production env from version 0.5 on, allowing to turn it off when needed. cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From csuml at yahoo.co.uk Tue Sep 11 07:45:05 2007 From: csuml at yahoo.co.uk (Paul Keeble) Date: Tue, 11 Sep 2007 13:45:05 +0200 Subject: [Ferret-talk] Newcomer perceived problems with AAF/Ferret Message-ID: <3adc8c99aadaf8062ec809bcfd59459f@ruby-forum.com> I blogged about some of the problems with aaf in production (http://www.jroller.com/BrightCandle) yesterday but inspired by the poitive response I thought I would share the perceived problems and discuss some of the potential solutions to help newcomers and make aaf work as if by magic just like Rails does. All of these problems boil down to one simple problem, running acts as ferret in a production environment simply is not consistent with the rails behaviour. Rails is single threaded system, resulting in multiple processes on a single server. Even the basic deployment scenario of Mongrel + Pen/Pound/Balance + Rails you have multiple processes. AAF in this case breaks without Drb, I don't think that how you develop should be different to how its run in production. Rails doesn't need a thing changing to work and that is the root cause of the production problem with aaf/ferret. 1) Each model needs remote => true adding to it for remoting to work. This is a breach of the DRY principle. Solution: Get rid of this requirement, if the config is there then use it, if its not then don't. 2) aaf needs to be installed as a plugin to get the start/stop scripts. Solution: Add startFerret, stopFerret Drb scripts to the gem so that even if it is not installed as a plugin the gem is still useable. 3) Drb server is a central point of failure. Solution 1: Lucence locks the file while it works which effectively allows many threads to write but it serialises them. This should be the case with aaf as well, at least then basic parrallism will work without having to setup up Drb or any other config settings. Sure you *could* get fancy with partial locking schemes but right now what I think I care about is it at least work. Drb should be the next performance step from Solution 1, improving performance at the cost of reliability. Solution 2: Support clustered behaviour of the Drb servers, write to all read from one just like a database. Solution 3: Allow the index to be stored in the database and use transactions to manage the updates (Just as compass does to lucene) Solution 4: You could write a multicast clustered behaviour system to make only one of the rails servers write to the file while all the others read from it. Probably more complex a solution than is needed. 4) AAF fails unexpectedly if you just use it without Drb. Solution: Produce a warning if more than one thread/process seems to be updating the file at any one time. Although bare in mind Solution 1 for problem (3) solves this in a different way and is a better solution overall. Thoughts? -- Posted via http://www.ruby-forum.com/. From gonesail at gmail.com Tue Sep 11 08:09:28 2007 From: gonesail at gmail.com (alan kemp) Date: Tue, 11 Sep 2007 08:09:28 -0400 Subject: [Ferret-talk] ferret index file(s) In-Reply-To: <20070911062323.GJ1891@cordoba.webit.de> References: <46E5B8A6.4080203@benjaminkrause.com> <20070911062323.GJ1891@cordoba.webit.de> Message-ID: ok thanks. does anyone know if there is a reasonable way to rebuild the indexes with Ferret or some Ferret utility? also is the threading and performance better with version 0.11.4? 0.3.2 seems to take a lot of CPU time for searches!! cheers, Alan On 9/11/07, Jens Kraemer wrote: > > On Mon, Sep 10, 2007 at 08:10:57PM -0400, alan kemp wrote: > > ok it is actually version 0.3.2 > > You'll have to rebuild your indexes, as 0.3.2 still used the lucene > compatible index format. > > cheers, > Jens > > > > > On 9/10/07, Benjamin Krause wrote: > > > > > > alan kemp schrieb: > > > > I am running version 3.2 of ferret and would like to upgrade to a > more > > > > current version. will my existing 3.2 index files be compatible? > > > Alan, > > > > > > are you sure about the version number? the current version of ferret > is > > > 0.11.4 .. so i'm not sure you've got the right number .. > > > > > > check it via "gem list ferret". > > > > > > Ben > > > > > > _______________________________________________ > > > Ferret-talk mailing list > > > Ferret-talk at rubyforge.org > > > http://rubyforge.org/mailman/listinfo/ferret-talk > > > > > > _______________________________________________ > > Ferret-talk mailing list > > Ferret-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/ferret-talk > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070911/2f6fef99/attachment-0001.html From csuml at yahoo.co.uk Tue Sep 11 08:37:34 2007 From: csuml at yahoo.co.uk (Paul Keeble) Date: Tue, 11 Sep 2007 14:37:34 +0200 Subject: [Ferret-talk] SVN installation problem in ferret In-Reply-To: References: <0b0ab9620fe28242561075f164436b62@ruby-forum.com> <20070819174021.GQ27074@thunder.jkraemer.net> Message-ID: <353787afbf315440d6c901d841fc15d5@ruby-forum.com> Since AAF is basically useless in a production environment without the plugin being installed (ie the gem is of no use) would it be possible to get the plugin exposed over http? -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Tue Sep 11 08:40:29 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Tue, 11 Sep 2007 14:40:29 +0200 Subject: [Ferret-talk] Newcomer perceived problems with AAF/Ferret In-Reply-To: <3adc8c99aadaf8062ec809bcfd59459f@ruby-forum.com> References: <3adc8c99aadaf8062ec809bcfd59459f@ruby-forum.com> Message-ID: <59323D2D-237E-4EF1-B82E-9C0966131F74@benjaminkrause.com> Hey ... thanks for your suggestions .. as a quick background, i'm one of the authors of omdb.org[2], Jens and I wrote the search backend for omdb, some parts of that backend is now part of AAF .. But omdb.org still has its own search logic.. basically because searching is a complex task. I don't see a plugin, that automagically resolves all the search related topic.. however some of your points are valid. i'll give my 2cent to your suggestions, even though, omdb.org does not use AAF :-) > 1) Each model needs remote => true adding to it for remoting to work. > This is a breach of the DRY principle. > Solution: Get rid of this requirement, if the config is there then use > it, if its not then don't. thats true.. basically you just need :remote => false in testing.. at least thats how omdb is doing it .. > 3) Drb server is a central point of failure. > Solution 1: Lucence locks the file while it works which effectively > allows many threads to write but it serialises them. This should be > the > case with aaf as well, at least then basic parrallism will work > without > having to setup up Drb or any other config settings. Sure you *could* > get fancy with partial locking schemes but right now what I think I > care > about is it at least work. Drb should be the next performance step > from > Solution 1, improving performance at the cost of reliability. omdb uses a queue to serialize all indexing requests. We also have two indizes, one offline index (new index requests will be added to that), and on online index (used for searching). As soon as our queue is empty or we processed 256 indexing requests, we'll sync online and offline index. This eliminates all of our search/indexing problems and is running very stable. Afaik, AAF is using backgroundrb for searching, this is something i would change, as i don't see a problem having the mongrel processes using a R/O access to the index (like the Searcher class). > Solution 3: Allow the index to be stored in the database and use > transactions to manage the updates (Just as compass does to lucene) That's an interesting point, and Jens and I even talked about that some month ago. Basically it is possible to write a new StorageEngine for Ferret (it just supports Memory and FS storage right now), but never got the time to do it. I'm fine with the online/offline index concept in the filesystem. This can even be distributed to several servers via rsync. But a db-backend is an option. > > 4) AAF fails unexpectedly if you just use it without Drb. > Solution: Produce a warning if more than one thread/process seems > to be > updating the file at any one time. Although bare in mind Solution 1 > for > problem (3) solves this in a different way and is a better solution > overall. I agree with your point from earlier on, I would simply eliminiate the non-drb option :-) Just use non-drb for testing .. btw, the whole omdb/ferret integration is open source, so feel free to take a look[2]. Ben [1] http://www.omdb.org [2] http://bugs.omdb.org/browser/branches/2007.1/lib/omdb/ferret From gonesail at gmail.com Tue Sep 11 10:01:59 2007 From: gonesail at gmail.com (alan kemp) Date: Tue, 11 Sep 2007 10:01:59 -0400 Subject: [Ferret-talk] ferret index file(s) In-Reply-To: References: <46E5B8A6.4080203@benjaminkrause.com> <20070911062323.GJ1891@cordoba.webit.de> Message-ID: to be MORE specific: I would like to convert the Ferret index files to version 0.11.4 from version 0.3.2 ... I want to do this by reading the index file values and outputting them to a file ... then importing them with the newer version and writing out the new index files. is this a good idea? On 9/11/07, alan kemp wrote: > > ok thanks. > does anyone know if there is a reasonable way to rebuild the indexes with > Ferret or some Ferret utility? > also is the threading and performance better with version 0.11.4? > 0.3.2 seems to take a lot of CPU time for searches!! > > cheers, > Alan > > > On 9/11/07, Jens Kraemer wrote: > > > > On Mon, Sep 10, 2007 at 08:10:57PM -0400, alan kemp wrote: > > > ok it is actually version 0.3.2 > > > > You'll have to rebuild your indexes, as 0.3.2 still used the lucene > > compatible index format. > > > > cheers, > > Jens > > > > > > > > On 9/10/07, Benjamin Krause wrote: > > > > > > > > alan kemp schrieb: > > > > > I am running version 3.2 of ferret and would like to upgrade to a > > more > > > > > current version. will my existing 3.2 index files be compatible? > > > > Alan, > > > > > > > > are you sure about the version number? the current version of ferret > > is > > > > 0.11.4 .. so i'm not sure you've got the right number .. > > > > > > > > check it via "gem list ferret". > > > > > > > > Ben > > > > > > > > _______________________________________________ > > > > Ferret-talk mailing list > > > > Ferret-talk at rubyforge.org > > > > http://rubyforge.org/mailman/listinfo/ferret-talk > > > > > > > > > _______________________________________________ > > > Ferret-talk mailing list > > > Ferret-talk at rubyforge.org > > > http://rubyforge.org/mailman/listinfo/ferret-talk > > > > -- > > Jens Kr?mer > > webit! Gesellschaft f?r neue Medien mbH > > Schnorrstra?e 76 | 01069 Dresden > > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > > kraemer at webit.de | www.webit.de > > > > Amtsgericht Dresden | HRB 15422 > > GF Sven Haubold, Hagen Malessa > > _______________________________________________ > > Ferret-talk mailing list > > Ferret-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/ferret-talk > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070911/5163af04/attachment.html From csuml at yahoo.co.uk Tue Sep 11 10:11:16 2007 From: csuml at yahoo.co.uk (Paul Keeble) Date: Tue, 11 Sep 2007 16:11:16 +0200 Subject: [Ferret-talk] ferret index file(s) In-Reply-To: References: <46E5B8A6.4080203@benjaminkrause.com> <20070911062323.GJ1891@cordoba.webit.de> Message-ID: The normal way would be to rebuild your index completely, from the source data. That eliminates the need to do any conversion. A simple call to Model.rebuild_index should do the job. alan kemp wrote: > to be MORE specific: I would like to convert the Ferret index files to > version 0.11.4 from version 0.3.2 ... I want to do this by reading the > index > file values and outputting them to a file ... then importing them with > the > newer version and writing out the new index files. is this a good idea? -- Posted via http://www.ruby-forum.com/. From neeraj.jsr at gmail.com Tue Sep 11 10:23:34 2007 From: neeraj.jsr at gmail.com (Raj Singh) Date: Tue, 11 Sep 2007 16:23:34 +0200 Subject: [Ferret-talk] How to search with range when I am using AAF Message-ID: <61e4ab6efd143b2014c62bcd969a872f@ruby-forum.com> acts_as_ferret :fields => { :name => {:}, :desc => {}, :start_date => {} } def start_date self.start_datetime.strftime("%Y%m%d") end Now that I am strong start_date in YYYYMMDD format, I want to search for all event in between 20070506 and 20070809 What will my query look like when I am using aaf. This one doesn't work Event.find_id_by_contents("start_date:[20070506 20070809]) I know how it works with native Ferret but can't seem to integrate with aaf. -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Tue Sep 11 11:02:56 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Tue, 11 Sep 2007 17:02:56 +0200 Subject: [Ferret-talk] ferret index file(s) In-Reply-To: References: <46E5B8A6.4080203@benjaminkrause.com> <20070911062323.GJ1891@cordoba.webit.de> Message-ID: On 2007-09-11, at 4:01 PM, alan kemp wrote: > to be MORE specific: I would like to convert the Ferret index files > to version 0.11.4 from version 0.3.2 ... I want to do this by > reading the index file values and outputting them to a file ... > then importing them with the newer version and writing out the new > index files. is this a good idea? alan, no, it's not :-) Just run the same process, that generated the data in the first place.. You won't get all information you need from the ferret index.. and no-one here will be able to help you on a ferret 0.3.2 question :) I guess you're using ferret without AAF? If you use AAF, its simple to reindex everything.. just like Paul said .. Ben From doug.arogos at gmail.com Tue Sep 11 11:08:38 2007 From: doug.arogos at gmail.com (Doug Smith) Date: Tue, 11 Sep 2007 08:08:38 -0700 Subject: [Ferret-talk] How to search with range when I am using AAF In-Reply-To: <61e4ab6efd143b2014c62bcd969a872f@ruby-forum.com> References: <61e4ab6efd143b2014c62bcd969a872f@ruby-forum.com> Message-ID: <42d8808f0709110808x1eb656b2i1e62c3711f11eac6@mail.gmail.com> For what it's worth, a similar query works for me: Article.find_by_contents("published_on:[2007-06-04 2007-09-14]") My published_on field is a DATE in the database, and I don't convert it at all. I'm using Ferret 0.11.4 and the latest aaf from the svn trunk. Hope that helps, Doug On 9/11/07, Raj Singh wrote: > acts_as_ferret :fields => { > :name => {:}, > :desc => {}, > :start_date => {} > } > > > def start_date > self.start_datetime.strftime("%Y%m%d") > end > > > Now that I am strong start_date in YYYYMMDD format, I want to search for > all event in between 20070506 and 20070809 > > What will my query look like when I am using aaf. This one doesn't work > Event.find_id_by_contents("start_date:[20070506 20070809]) > > I know how it works with native Ferret but can't seem to integrate with > aaf. > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From doug.arogos at gmail.com Tue Sep 11 11:41:14 2007 From: doug.arogos at gmail.com (Doug Smith) Date: Tue, 11 Sep 2007 08:41:14 -0700 Subject: [Ferret-talk] aaf requires drb? In-Reply-To: <20070911100927.GO1891@cordoba.webit.de> References: <1f2cbe970e2849dcd21b976a4b7b4137@ruby-forum.com> <20070911100927.GO1891@cordoba.webit.de> Message-ID: <42d8808f0709110841r4d2e15c2teacf0d0321bcae27@mail.gmail.com> I just tried to post this comment several times in response to this blog post and it keeps getting rejected as spam. I'm posting it here to provide an alternate opinion to the one shared in the blog. ------------ I'm a user of acts_as_ferret, and completely disagree with your post. Aaf/ferret is a very cool full-text search solution that works very reliably in Rails when used as the online manual suggests. Jens and the team on the mailing list provide better support for it than any other open source plugin I've ever used on any platform. The gem vs plugin thing is debatable. I much prefer plugins over gems, since I keep all the plugins in my project's svn repository. Deployment is much simplified, history is maintained in svn, and I don't have to worry about gem versions on various servers. Overall, your rant shows a lack of respect for the work that has gone into aaf/ferret, and an ignorance of the benefits of using it in its current form. Certainly, nothing is perfect, and everything requires constant improvement. But, we've had great success with aaf and will continue to use it happily. ------------- Thanks, Doug On 9/11/07, Jens Kraemer wrote: > On Tue, Sep 11, 2007 at 11:40:32AM +0200, Chris Williams wrote: > > Hi, > > I've just read some an article > > (http://www.jroller.com/BrightCandle/entry/oversights_in_rails_plugins_really) > > > > This article (and the comment,) seem to imply that drb is required if > > running under a fastcgi kind of deployment. Is this true? > > Yes. Inspired by this post, I'll make the :remote => true option the > default in production env from version 0.5 on, allowing to turn it off > when needed. > > cheers, > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From bkelly01 at yahoo.com Tue Sep 11 12:01:25 2007 From: bkelly01 at yahoo.com (Brandon Kelly) Date: Tue, 11 Sep 2007 18:01:25 +0200 Subject: [Ferret-talk] Ferret DRB - can add/edit index, but can't search In-Reply-To: <2c26151bbde66ead532d570d00864413@ruby-forum.com> References: <20070909104327.GF16997@thunder.jkraemer.net> <2c26151bbde66ead532d570d00864413@ruby-forum.com> Message-ID: Just to close out this thread -- After I converted the Sort, SortFields to :sort = ['field_one DESC', field_two'] and RangeQueries to FQL - Ferret DRB worked fine. The simpler approach is usually the right approach. Thanks Jens - for your suggestions. - Brandon Brandon Kelly wrote: > Jens, > > Thanks for your help. I ripped out the Sort, SortField and RangeQuery > objects, and sure enough - it worked. I'll try your suggestions > tomorrow -- using the sort fields and FQL. > > > Brandon > -- Posted via http://www.ruby-forum.com/. From csuml at yahoo.co.uk Tue Sep 11 16:16:23 2007 From: csuml at yahoo.co.uk (Paul Keeble) Date: Tue, 11 Sep 2007 22:16:23 +0200 Subject: [Ferret-talk] aaf requires drb? In-Reply-To: <42d8808f0709110841r4d2e15c2teacf0d0321bcae27@mail.gmail.com> References: <1f2cbe970e2849dcd21b976a4b7b4137@ruby-forum.com> <20070911100927.GO1891@cordoba.webit.de> <42d8808f0709110841r4d2e15c2teacf0d0321bcae27@mail.gmail.com> Message-ID: You didn't mention in your post the "alternative" view - only that you believe its great but with no counter argument to the points I raised. Can you be more specific where I am getting it wrong so that I can learn. I'm coming to ruby, rails and aaf(ferret) with a brand new set of eyes, I am a noobie in every sense and if I see it wrong then I want to be told as such. The capistrano issues I raised for example were for the most part resolveable and now all they need to do is needed is capture that in the official documentation. I comment on it not to rubbish but to begin the process of fixing the behaviour in a cluster and production behaviour. Should my comments and solutions be acceptable I am willing to put my coding time where my mouth is and implement the solutions, because I NEED it for my app to be able to work properly. Its not like any of the other solutions are good either. Gem V plugin is always going to rage, I simply can't get the plugin to work with a svn:// url so I don't have the option to use it, which means I can't run aaf at all because I have to have a basic cluster because rails requires it! As I said in my post if the gem is going to be published it should come with the necessary tools to get a Drb server started, it seems like a simple enough change (I hope) and one that will allow me to reenable my ferret support in my app. As for rejection as spam I don't know what is going wrong there, I don't even have the spam filter turned on, are you doing the maths question? -- Posted via http://www.ruby-forum.com/. From doug.arogos at gmail.com Tue Sep 11 17:18:02 2007 From: doug.arogos at gmail.com (Doug Smith) Date: Tue, 11 Sep 2007 14:18:02 -0700 Subject: [Ferret-talk] aaf requires drb? In-Reply-To: References: <1f2cbe970e2849dcd21b976a4b7b4137@ruby-forum.com> <20070911100927.GO1891@cordoba.webit.de> <42d8808f0709110841r4d2e15c2teacf0d0321bcae27@mail.gmail.com> Message-ID: <42d8808f0709111418x62b3ccdeyc2de449a2b334bd1@mail.gmail.com> Hi Paul, please see below: On 9/11/07, Paul Keeble wrote: > You didn't mention in your post the "alternative" view - only that you > believe its great but with no counter argument to the points I raised. Your general tone was that there was something hidden and that the docs don't tell the truth about the DRb requirement. That's just false. The top page of the doc says to use DRb in production environments: http://projects.jkraemer.net/acts_as_ferret/wiki Single point of failure might be the only valid point you make, but people are typically using monit to make sure that if the process fails, it automatically restarts. (That's what we do, and it works great.) > Gem V plugin is always going to rage, I simply can't get the plugin to > work with a svn:// url so I don't have the option to use it, which means > I can't run aaf at all because I have to have a basic cluster because > rails requires it! As I said in my post if the gem is going to be > published it should come with the necessary tools to get a Drb server > started, it seems like a simple enough change (I hope) and one that will > allow me to reenable my ferret support in my app. Have you tried using piston? That's a great way to get plugins for Rails, and it does support the svn:// URL: http://piston.rubyforge.org > As for rejection as spam I don't know what is going wrong there, I don't > even have the spam filter turned on, are you doing the maths question? I did -- several times. Each time the message said something like "your post has been flagged by our spam protection and will not be posted". Thanks, Doug From chris at nonfatal.com Tue Sep 11 17:26:42 2007 From: chris at nonfatal.com (Chris Williams) Date: Tue, 11 Sep 2007 23:26:42 +0200 Subject: [Ferret-talk] aaf requires drb? In-Reply-To: <42d8808f0709111418x62b3ccdeyc2de449a2b334bd1@mail.gmail.com> References: <1f2cbe970e2849dcd21b976a4b7b4137@ruby-forum.com> <20070911100927.GO1891@cordoba.webit.de> <42d8808f0709110841r4d2e15c2teacf0d0321bcae27@mail.gmail.com> <42d8808f0709111418x62b3ccdeyc2de449a2b334bd1@mail.gmail.com> Message-ID: <9f6e83253f63dd4fe3b841ae1f14728d@ruby-forum.com> Doug Smith wrote: > Your general tone was that there was something hidden and that the > docs don't tell the truth about the DRb requirement. That's just > false. The top page of the doc says to use DRb in production > environments: > > http://projects.jkraemer.net/acts_as_ferret/wiki > It doesn't actually say it's required though, just that it's supported. Chris. -- Posted via http://www.ruby-forum.com/. From doug.arogos at gmail.com Tue Sep 11 17:31:04 2007 From: doug.arogos at gmail.com (Doug Smith) Date: Tue, 11 Sep 2007 14:31:04 -0700 Subject: [Ferret-talk] aaf requires drb? In-Reply-To: <9f6e83253f63dd4fe3b841ae1f14728d@ruby-forum.com> References: <1f2cbe970e2849dcd21b976a4b7b4137@ruby-forum.com> <20070911100927.GO1891@cordoba.webit.de> <42d8808f0709110841r4d2e15c2teacf0d0321bcae27@mail.gmail.com> <42d8808f0709111418x62b3ccdeyc2de449a2b334bd1@mail.gmail.com> <9f6e83253f63dd4fe3b841ae1f14728d@ruby-forum.com> Message-ID: <42d8808f0709111431u63a3633ei44aee45c2983bdb5@mail.gmail.com> On 9/11/07, Chris Williams wrote: > > It doesn't actually say it's required though, just that it's supported. True, it says: "DRb Server for centralized index access in production environments" It doesn't say "required", true, but I hardly think it's worth charging the open-source code contributor(s) with "lying". Thanks, Doug From csuml at yahoo.co.uk Tue Sep 11 18:09:47 2007 From: csuml at yahoo.co.uk (Paul Keeble) Date: Wed, 12 Sep 2007 00:09:47 +0200 Subject: [Ferret-talk] aaf requires drb? In-Reply-To: <42d8808f0709111431u63a3633ei44aee45c2983bdb5@mail.gmail.com> References: <1f2cbe970e2849dcd21b976a4b7b4137@ruby-forum.com> <20070911100927.GO1891@cordoba.webit.de> <42d8808f0709110841r4d2e15c2teacf0d0321bcae27@mail.gmail.com> <42d8808f0709111418x62b3ccdeyc2de449a2b334bd1@mail.gmail.com> <9f6e83253f63dd4fe3b841ae1f14728d@ruby-forum.com> <42d8808f0709111431u63a3633ei44aee45c2983bdb5@mail.gmail.com> Message-ID: <5a745e1749e91857219593ab09ddf3fb@ruby-forum.com> > It doesn't say "required", true, but I hardly think it's worth > charging the open-source code contributor(s) with "lying". My tone is irrelevent, that is just me ranting. What is important is the technical problems I faced. Consider the fact that someone doesn't have any documentation, what would you want the code API to do? I think documenting it is a patch gap to a larger problem, one where the behaviour is not inline with the expected nor is it desireable from an ease of use perspective. Do you not agree that it would be nice not to have to use remote => true in every model AND more to the point not have to do different things in dev and production because of Drb? -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Tue Sep 11 18:16:23 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Wed, 12 Sep 2007 00:16:23 +0200 Subject: [Ferret-talk] SVN installation problem in ferret In-Reply-To: <353787afbf315440d6c901d841fc15d5@ruby-forum.com> References: <0b0ab9620fe28242561075f164436b62@ruby-forum.com> <20070819174021.GQ27074@thunder.jkraemer.net> <353787afbf315440d6c901d841fc15d5@ruby-forum.com> Message-ID: <20070911221623.GE3257@thunder.jkraemer.net> On Tue, Sep 11, 2007 at 02:37:34PM +0200, Paul Keeble wrote: > Since AAF is basically useless in a production environment without the > plugin being installed (ie the gem is of no use) would it be possible to > get the plugin exposed over http? Yeah that's already on my list. There also is a tarball of the latest stable (0.4.1) available at [0] just unpack into vendor/plugins, and copy over script/ferret_st* to script/, and config/ferret_server.yml to config/ (or use the plugin's install.rb for this). Btw, I just checked and the gem contains the scripts and config, it just lacked an installer to copy them into a project. I already created one, and updated the docs at [1] for the time until the next release. Cheers, Jens [0] http://projects.jkraemer.net/acts_as_ferret/attachment/wiki/WikiStart/acts_as_ferret_0.4.1.tar.gz [1] http://projects.jkraemer.net/acts_as_ferret/wiki/WikiStart#System-wideinstallation -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Tue Sep 11 18:51:34 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Wed, 12 Sep 2007 00:51:34 +0200 Subject: [Ferret-talk] Newcomer perceived problems with AAF/Ferret In-Reply-To: <3adc8c99aadaf8062ec809bcfd59459f@ruby-forum.com> References: <3adc8c99aadaf8062ec809bcfd59459f@ruby-forum.com> Message-ID: <20070911225134.GF3257@thunder.jkraemer.net> Hi! On Tue, Sep 11, 2007 at 01:45:05PM +0200, Paul Keeble wrote: > I blogged about some of the problems with aaf in production > (http://www.jroller.com/BrightCandle) yesterday but inspired by the > poitive response I thought I would share the perceived problems and > discuss some of the potential solutions to help newcomers and make aaf > work as if by magic just like Rails does. > > All of these problems boil down to one simple problem, running acts as > ferret in a production environment simply is not consistent with the > rails behaviour. Rails is single threaded system, resulting in multiple > processes on a single server. Even the basic deployment scenario of > Mongrel + Pen/Pound/Balance + Rails you have multiple processes. AAF in > this case breaks without Drb, I don't think that how you develop should > be different to how its run in production. Rails doesn't need a thing > changing to work and that is the root cause of the production problem > with aaf/ferret. Putting an app live is always something that needs special attention, and the DRb server is by far not the only thing you'll need to set up properly on your live site. Throw in memcached, monit, mongrel_cluster, log rotation, load balancing, cron jobs and suddenly it's not that much *additional* work anymore. > 1) Each model needs remote => true adding to it for remoting to work. > This is a breach of the DRY principle. > Solution: Get rid of this requirement, if the config is there then use > it, if its not then don't. I completely agree with you. The reason it is like that is more or less backwards compatiblity and the decision to make the DRb server an option, and not the default in the beginning, when it still was nothing more than an experiment. Plus nobody complained about it until now ;-) As I already wrote, I'll change that soon. > 2) aaf needs to be installed as a plugin to get the start/stop scripts. > Solution: Add startFerret, stopFerret Drb scripts to the gem so that > even if it is not installed as a plugin the gem is still useable. good point, however I think an install script that adds these scripts and a template config file to a given rails project would be ok, too. > 3) Drb server is a central point of failure. > Solution 1: Lucence locks the file while it works which effectively > allows many threads to write but it serialises them. This should be the > case with aaf as well, at least then basic parrallism will work without > having to setup up Drb or any other config settings. Sure you *could* > get fancy with partial locking schemes but right now what I think I care > about is it at least work. Drb should be the next performance step from > Solution 1, improving performance at the cost of reliability. Ferret's inter process locking doesn't work. Therefore Ferret is thread safe, but not multi-process safe. And I'm afraid that's unlikely to change anytime soon (unless you'd like to give it a try). I'm not sure where exactly on the way from Java to Ruby to C/Ruby this capability got lost... DRb based solutions have proven to be stable enough, so I don't consider this a real issue. Yes, you need to start the server in production environments and not in dev mode (unless you feel like it, nothing stops you from using the DRb in dev mode, too), but I think that's something people can handle. Of course we could provide more support for this, i.e. monit configs and cap tasks for controlling the DRb server. For starters, I changed the Wiki page to make really clear that the DRb server is required for multi-process environments. Another point why I wouldn't want to invest time in file based index access is that with multiple physical servers, file based index access won't be worth anything and DRb is your only option again. > Solution 2: Support clustered behaviour of the Drb servers, write to all > read from one just like a database. I guess you mean 'write to one and read from all'? Interesting and kind of what Ben does at omdb. > Solution 3: Allow the index to be stored in the database and use > transactions to manage the updates (Just as compass does to lucene) This is definitely an option if somebody wrote a database backend for Ferret. Performance would be interesting. Never heard about that compass thing, do you know how it performs when compared with Lucene's file system storage? > Solution 4: You could write a multicast clustered behaviour system to > make only one of the rails servers write to the file while all the > others read from it. Probably more complex a solution than is needed. Of course we could do that ;-) In general I see aaf more as a general-purpose fulltext search plugin which lets you get up and running fast, than the last and best solution to every problem that might get solved with Ferret. For sure there are applications that profit from having their own, tailored search implementation around Ferret. However in these cases aaf may still be a good starting point from where a custom solution may evolve. > 4) AAF fails unexpectedly if you just use it without Drb. > Solution: Produce a warning if more than one thread/process seems to be > updating the file at any one time. Although bare in mind Solution 1 for > problem (3) solves this in a different way and is a better solution > overall. Imho this isn't an issue anymore once remote mode is the default. cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From hakita at gmail.com Wed Sep 12 07:47:28 2007 From: hakita at gmail.com (Hakita Hakita) Date: Wed, 12 Sep 2007 13:47:28 +0200 Subject: [Ferret-talk] Asked for a remote server ? nil, ENV["FERRET_USE_LOCAL_I In-Reply-To: <9eebca078ef4baade48e2c3984a3aa86@ruby-forum.com> References: <9eebca078ef4baade48e2c3984a3aa86@ruby-forum.com> Message-ID: OK, Adding ',:remote => false' at the end of my acts_as_ferret definition, in my model did resolve the issue. -- Posted via http://www.ruby-forum.com/. From golak.sarangi at gmail.com Wed Sep 12 07:56:44 2007 From: golak.sarangi at gmail.com (golak Sarangi) Date: Wed, 12 Sep 2007 17:26:44 +0530 Subject: [Ferret-talk] ported index from one machine to another In-Reply-To: <3854b3a40709120325s7b2d2c03l904779756ea0b45e@mail.gmail.com> References: <3854b3a40709120325s7b2d2c03l904779756ea0b45e@mail.gmail.com> Message-ID: <3854b3a40709120456s1707d6adud77a4f6a3929871b@mail.gmail.com> > Hi, > I built an index on a free bsd machine using ferret and then ported the > index to another machine. While searching from that copied index the > searcher generates a file not found error. I have set all the permission > right and the files do exist in the directories.on creating index from > local machine the searcher finds the index. could any body please tell what > might be the problem. Is it a problem with my environments or the problem > lies in ferret. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070912/97b6610e/attachment.html From golak.sarangi at gmail.com Wed Sep 12 06:25:11 2007 From: golak.sarangi at gmail.com (golak Sarangi) Date: Wed, 12 Sep 2007 15:55:11 +0530 Subject: [Ferret-talk] ported index from one machine to another Message-ID: <3854b3a40709120325s7b2d2c03l904779756ea0b45e@mail.gmail.com> Hi, I built an index on a free bsd machine using ferret and then ported the index to another machine. While searching from that copied index the searcher generates a file not found error. I have set all the permission right and the files do exist in the directories.on creating index from local machine the searcher finds the index. could any body please tell what might be the problem. Is it a problem with my environments or the problem lies in ferret. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070912/80f128af/attachment-0001.html From eimorton at gmail.com Wed Sep 12 13:37:55 2007 From: eimorton at gmail.com (Erik Morton) Date: Wed, 12 Sep 2007 13:37:55 -0400 Subject: [Ferret-talk] Can't disable ferret when running in Mongrel Message-ID: <9B271646-9B28-46DE-92FB-DDCCA8936591@gmail.com> In my environment.rb I say the following: MyModel.disable_ferret puts MyModel.ferret_enabled? The console works as expected: script/console false >> MyModel.ferret_enabled? => false I can then edit an instance, ferret remains disabled and the changes are not immediately indexed. However, when I start mongrel Ferret is initially disabled, but then somehow becomes enabled again after the first request. I don't explicitly enable ferret anywhere. Here is my ferret config for the model in question: acts_as_ferret( { :remote => true, :index_dir => "#{AAF_INDEX}/ supplier", :fields => { :supplier => { :store => :yes }, :name => {:store => :yes}, :private_description => {}, :title => {}, :ur l => { :store => :yes }, :description => { :boost => 10.0 }, :su pplier_id => { :store => :yes }, :supplier_name => { :boost => 10.0 }, :ke ywords => { :boost => 10.0 }, :data => {}, :extract => {}, :city => {}, :st ate => {} } } ) Does anyone have any idea what could reenable ferret? Thanks, Erik From jk at jkraemer.net Wed Sep 12 14:06:45 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Wed, 12 Sep 2007 20:06:45 +0200 Subject: [Ferret-talk] ported index from one machine to another In-Reply-To: <3854b3a40709120325s7b2d2c03l904779756ea0b45e@mail.gmail.com> References: <3854b3a40709120325s7b2d2c03l904779756ea0b45e@mail.gmail.com> Message-ID: <20070912180645.GJ3257@thunder.jkraemer.net> On Wed, Sep 12, 2007 at 03:55:11PM +0530, golak Sarangi wrote: > Hi, > I built an index on a free bsd machine using ferret and then ported the > index to another machine. While searching from that copied index the > searcher generates a file not found error. I have set all the permission > right and the files do exist in the directories.on creating index from local > machine the searcher finds the index. could any body please tell what might > be the problem. Is it a problem with my environments or the problem lies in > ferret. are you sure you're using the same version of ferret everywhere? What does the error look like? Cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Wed Sep 12 14:10:45 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Wed, 12 Sep 2007 20:10:45 +0200 Subject: [Ferret-talk] Can't disable ferret when running in Mongrel In-Reply-To: <9B271646-9B28-46DE-92FB-DDCCA8936591@gmail.com> References: <9B271646-9B28-46DE-92FB-DDCCA8936591@gmail.com> Message-ID: <20070912181045.GK3257@thunder.jkraemer.net> On Wed, Sep 12, 2007 at 01:37:55PM -0400, Erik Morton wrote: > In my environment.rb I say the following: > > MyModel.disable_ferret > puts MyModel.ferret_enabled? > > The console works as expected: > > script/console > false > >> MyModel.ferret_enabled? > => false > > I can then edit an instance, ferret remains disabled and the changes > are not immediately indexed. > > However, when I start mongrel Ferret is initially disabled, but then > somehow becomes enabled again after the first request. I don't > explicitly enable ferret anywhere. is this happening in development or production mode? This might be an issue with the class reloading done in development mode, where the class is reloaded, but environment.rb isn't parsed again. Cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From eimorton at gmail.com Wed Sep 12 15:14:21 2007 From: eimorton at gmail.com (Erik Morton) Date: Wed, 12 Sep 2007 15:14:21 -0400 Subject: [Ferret-talk] Can't disable ferret when running in Mongrel In-Reply-To: <20070912181045.GK3257@thunder.jkraemer.net> References: <9B271646-9B28-46DE-92FB-DDCCA8936591@gmail.com> <20070912181045.GK3257@thunder.jkraemer.net> Message-ID: <158BFD92-069F-4D30-8C96-998F2B236BDE@gmail.com> That was the problem. Sounds obvious now. Thanks again. Erik On Sep 12, 2007, at 2:10 PM, Jens Kraemer wrote: > On Wed, Sep 12, 2007 at 01:37:55PM -0400, Erik Morton wrote: >> In my environment.rb I say the following: >> >> MyModel.disable_ferret >> puts MyModel.ferret_enabled? >> >> The console works as expected: >> >> script/console >> false >>>> MyModel.ferret_enabled? >> => false >> >> I can then edit an instance, ferret remains disabled and the changes >> are not immediately indexed. >> >> However, when I start mongrel Ferret is initially disabled, but then >> somehow becomes enabled again after the first request. I don't >> explicitly enable ferret anywhere. > > is this happening in development or production mode? This might be an > issue with the class reloading done in development mode, where the > class > is reloaded, but environment.rb isn't parsed again. > > Cheers, > Jens > > -- > Jens Kr?mer > http://www.jkraemer.net/ - Blog > http://www.omdb.org/ - The new free film database > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From jk at jkraemer.net Wed Sep 12 15:45:43 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Wed, 12 Sep 2007 21:45:43 +0200 Subject: [Ferret-talk] Can't disable ferret when running in Mongrel In-Reply-To: <158BFD92-069F-4D30-8C96-998F2B236BDE@gmail.com> References: <9B271646-9B28-46DE-92FB-DDCCA8936591@gmail.com> <20070912181045.GK3257@thunder.jkraemer.net> <158BFD92-069F-4D30-8C96-998F2B236BDE@gmail.com> Message-ID: <20070912194543.GO3257@thunder.jkraemer.net> On Wed, Sep 12, 2007 at 03:14:21PM -0400, Erik Morton wrote: > That was the problem. Sounds obvious now. Thanks again. Ok. Now how do you deal with this? I'd probably do something like this in application.rb: before_filter :disable_ferret if RAILS_ENV == 'development' def disable_ferret MyModel.disable_ferret end Not nice but should do until aaf gets a proper config mechanism that allows setting variables in environment.rb that are read whenever acts_as_ferret is called. While I think of it - maybe a simple :readonly => true switch for the acts_as_ferret method would be even more straightforward instead of calling MyModel.disable_ferret in environment.rb? Cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From neeraj.jsr at gmail.com Wed Sep 12 18:03:31 2007 From: neeraj.jsr at gmail.com (Raj Singh) Date: Thu, 13 Sep 2007 00:03:31 +0200 Subject: [Ferret-talk] How to search with range when I am using AAF In-Reply-To: <42d8808f0709110808x1eb656b2i1e62c3711f11eac6@mail.gmail.com> References: <61e4ab6efd143b2014c62bcd969a872f@ruby-forum.com> <42d8808f0709110808x1eb656b2i1e62c3711f11eac6@mail.gmail.com> Message-ID: <4b7e06cfb0fbef28dc3c7b11599f5318@ruby-forum.com> Doug: You are right. For some mysterious reason I wasn't getting the result. It's all good now. Thanks. Doug Smith wrote: > For what it's worth, a similar query works for me: > > Article.find_by_contents("published_on:[2007-06-04 2007-09-14]") > > My published_on field is a DATE in the database, and I don't convert > it at all. I'm using Ferret 0.11.4 and the latest aaf from the svn > trunk. > > Hope that helps, > > Doug -- Posted via http://www.ruby-forum.com/. From neeraj.jsr at gmail.com Wed Sep 12 18:08:22 2007 From: neeraj.jsr at gmail.com (Raj Singh) Date: Thu, 13 Sep 2007 00:08:22 +0200 Subject: [Ferret-talk] Highlighting isue in description Message-ID: I have read the highlighting section at http://www.railsenvy.com/2007/2/19/acts-as-ferret-tutorial#highlighting It mentions that for highlight to work I must have {:store => :yes}. However it also mentions that only "small" fields should be stored. My question is how small is small enough. I have a site where events information is maintained. I have events title and events description. On an average the description is of 60 words and around 300 characters. When search is performed I want to display the excerpt of description in highlighted fashion. For that to work I need to {:store => :yes} the description field. What is the recommended practice. How others are displaying the description highlight. I am leaning towards storing the whole description. Is 60 words too many words? -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Thu Sep 13 03:24:26 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 13 Sep 2007 09:24:26 +0200 Subject: [Ferret-talk] Highlighting isue in description In-Reply-To: References: Message-ID: <20070913072426.GU1891@cordoba.webit.de> On Thu, Sep 13, 2007 at 12:08:22AM +0200, Raj Singh wrote: > I have read the highlighting section at > http://www.railsenvy.com/2007/2/19/acts-as-ferret-tutorial#highlighting > > It mentions that for highlight to work I must have {:store => :yes}. > However it also mentions that only "small" fields should be stored. > > My question is how small is small enough. that depends on how large you want your index to grow, and how long the indexing may take ;-) > I have a site where events information is maintained. I have events > title and events description. On an average the description is of 60 > words and around 300 characters. That shouldn't be a problem. To save space with large field contents you may also use :store => :compressed, however with 300 characters that shouldn't be necessary. Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From bakki.kudva at gmail.com Thu Sep 13 15:32:14 2007 From: bakki.kudva at gmail.com (Bakki Kudva) Date: Thu, 13 Sep 2007 15:32:14 -0400 Subject: [Ferret-talk] refreshing indexes? Message-ID: I am new to ferret and am just reading about it in the O'reilly shortcuts as well as other web resources. My app is a Rails app and so I am looking into acts_as_ferret as well. There are some questions for which I couldn't find answers in the material I have read so far so I'd appreciate any help on these from the list. A bit of a background. My app will have 10,000 - 50,000 documents. These are 'owned' by users and the groups they belong to (like a file system). So the queries will have to be restricted to those documents owned by the groups the user belongs to. I have a group_id in each record and the session has the group_id's of the groups the user belongs to in an array. Question 1: If I add acts_as_ferret :fields => [:group_id, :document_text] can I do an activerecord search like... Document.find(:all, :conditions => ['group_id in (?)', session[:group_ids]) ? Question 2: These documents are dynamic, so they are deleted, updated, edited etc. How do I handle indexing in these circumstances? Delete all and recreate -or- is there a way to just delete an index by document_id and recreate it just for that document. I am going to try this in the app but before I took the plunge I thought I'd ask first. -tia, bakki PS: the app will be used by separate client companies, so on one server we will host the app in multiple virtual hosts(separate rails app instance for each). Can I share a drb for all clients or do I need one per client. From bk at benjaminkrause.com Thu Sep 13 16:16:57 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Thu, 13 Sep 2007 22:16:57 +0200 Subject: [Ferret-talk] refreshing indexes? In-Reply-To: References: Message-ID: <027CDC7A-7360-451F-9832-4D172F87C2C1@benjaminkrause.com> Hey.. > Question 1: > If I add acts_as_ferret :fields => [:group_id, :document_text] can I > do an activerecord search like... > Document.find(:all, :conditions => ['group_id in (?)', session > [:group_ids]) ? yes, you can. Check the FQL. But i wouldn't trust the ferret index, if you plan to use this to fetch security-related objects. > Question 2: > These documents are dynamic, so they are deleted, updated, edited etc. > How do I handle indexing in these circumstances? Delete all and > recreate -or- is there a way to just delete an index by document_id > and recreate it just for that document. use acts_as_ferret or check the way acts_as_ferret implements after_save/after_destroy hooks[1]. But basically, if you use AAF, you don't need to think about that :) If you don't use AAF, you need to implement hooks (like an observer) on save/destroy, check [2] for an example (including cache sweepers). > I am going to try this in the app but before I took the plunge I > thought I'd ask first. good idea :) Hope that helps .. Ben [1] - http://projects.jkraemer.net/acts_as_ferret/browser/trunk/ plugin/acts_as_ferret/lib/act_methods.rb [2] - http://bugs.omdb.org/browser/trunk/app/models/observers/ search_observer.rb From bakki.kudva at gmail.com Thu Sep 13 16:29:09 2007 From: bakki.kudva at gmail.com (Bakki Kudva) Date: Thu, 13 Sep 2007 16:29:09 -0400 Subject: [Ferret-talk] refreshing indexes? In-Reply-To: <027CDC7A-7360-451F-9832-4D172F87C2C1@benjaminkrause.com> References: <027CDC7A-7360-451F-9832-4D172F87C2C1@benjaminkrause.com> Message-ID: Ben, Thank you very much. That cleared things up for me and I am in the process of adding ferret to my project. However I am not clear on how to use the Drb server when you have multiple instances of the app. It is not the more common deployment scheme where you have multiple app servers or mongrels but share the same data. Our app will be shared but the database and the resources are private to each customer. So can we use Drb to listen on separate ports for serve different customers? -bakki On 9/13/07, Benjamin Krause wrote: > Hey.. > > > Question 1: > > If I add acts_as_ferret :fields => [:group_id, :document_text] can I > > do an activerecord search like... > > Document.find(:all, :conditions => ['group_id in (?)', session > > [:group_ids]) ? > > yes, you can. Check the FQL. But i wouldn't trust the ferret index, if > you plan to use this to fetch security-related objects. > > > Question 2: > > These documents are dynamic, so they are deleted, updated, edited etc. > > How do I handle indexing in these circumstances? Delete all and > > recreate -or- is there a way to just delete an index by document_id > > and recreate it just for that document. > > use acts_as_ferret or check the way acts_as_ferret implements > after_save/after_destroy hooks[1]. But basically, if you use AAF, > you don't need to think about that :) > If you don't use AAF, you need to implement hooks (like an > observer) on save/destroy, check [2] for an example (including > cache sweepers). > > > I am going to try this in the app but before I took the plunge I > > thought I'd ask first. > > good idea :) Hope that helps .. > > Ben > > [1] - http://projects.jkraemer.net/acts_as_ferret/browser/trunk/ > plugin/acts_as_ferret/lib/act_methods.rb > [2] - http://bugs.omdb.org/browser/trunk/app/models/observers/ > search_observer.rb > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From bk at benjaminkrause.com Thu Sep 13 16:38:38 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Thu, 13 Sep 2007 22:38:38 +0200 Subject: [Ferret-talk] refreshing indexes? In-Reply-To: References: <027CDC7A-7360-451F-9832-4D172F87C2C1@benjaminkrause.com> Message-ID: > > Thank you very much. That cleared things up for me and I am in the > process of adding ferret to my project. However I am not clear on how > to use the Drb server when you have multiple instances of the app. It > is not the more common deployment scheme where you have multiple app > servers or mongrels but share the same data. Our app will be shared > but the database and the resources are private to each customer. So > can we use Drb to listen on separate ports for serve different > customers? yes you can, check this [1]. Each app will have its own config file (including a port setting) for the ferret drb server. You should definitely use the drb server when using ferret. Ben [1] http://projects.jkraemer.net/acts_as_ferret/browser/trunk/plugin/ acts_as_ferret/config/ferret_server.yml From bk at benjaminkrause.com Thu Sep 13 16:50:35 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Thu, 13 Sep 2007 22:50:35 +0200 Subject: [Ferret-talk] refreshing indexes? In-Reply-To: References: <027CDC7A-7360-451F-9832-4D172F87C2C1@benjaminkrause.com> Message-ID: <69A35CCA-2096-4C0B-A8BC-9FFF6ABFC2D7@benjaminkrause.com> on a 2nd thought.. if you want one app with different databases, you should think about using different indexes for each customer. but i don't think you can do this with aaf out of the box .. so you just need one drb server and a way to select the right index to index to. Ben On 2007-09-13, at 10:38 PM, Benjamin Krause wrote: >> >> Thank you very much. That cleared things up for me and I am in the >> process of adding ferret to my project. However I am not clear on how >> to use the Drb server when you have multiple instances of the app. It >> is not the more common deployment scheme where you have multiple app >> servers or mongrels but share the same data. Our app will be shared >> but the database and the resources are private to each customer. So >> can we use Drb to listen on separate ports for serve different >> customers? > > yes you can, check this [1]. Each app will have its own config file > (including a port setting) for the ferret drb server. You should > definitely use the drb server when using ferret. > > Ben > > > [1] http://projects.jkraemer.net/acts_as_ferret/browser/trunk/plugin/ > acts_as_ferret/config/ferret_server.yml > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From bakki.kudva at gmail.com Thu Sep 13 16:56:15 2007 From: bakki.kudva at gmail.com (Bakki Kudva) Date: Thu, 13 Sep 2007 16:56:15 -0400 Subject: [Ferret-talk] refreshing indexes? In-Reply-To: <69A35CCA-2096-4C0B-A8BC-9FFF6ABFC2D7@benjaminkrause.com> References: <027CDC7A-7360-451F-9832-4D172F87C2C1@benjaminkrause.com> <69A35CCA-2096-4C0B-A8BC-9FFF6ABFC2D7@benjaminkrause.com> Message-ID: We WILL have different databases. So I will have to look into the aaf source and see if I can patch it or find a way to pass in a path for an index? Thank you again for your help. If I find a way to do this I'll post it here in case there are others who may have similar requirements. -bakki On 9/13/07, Benjamin Krause wrote: > on a 2nd thought.. if you want one app with different databases, > you should think about using different indexes for each customer. > but i don't think you can do this with aaf out of the box .. so you just > need one drb server and a way to select the right index to index to. > > Ben > > > On 2007-09-13, at 10:38 PM, Benjamin Krause wrote: > > >> > >> Thank you very much. That cleared things up for me and I am in the > >> process of adding ferret to my project. However I am not clear on how > >> to use the Drb server when you have multiple instances of the app. It > >> is not the more common deployment scheme where you have multiple app > >> servers or mongrels but share the same data. Our app will be shared > >> but the database and the resources are private to each customer. So > >> can we use Drb to listen on separate ports for serve different > >> customers? > > > > yes you can, check this [1]. Each app will have its own config file > > (including a port setting) for the ferret drb server. You should > > definitely use the drb server when using ferret. > > > > Ben > > > > > > [1] http://projects.jkraemer.net/acts_as_ferret/browser/trunk/plugin/ > > acts_as_ferret/config/ferret_server.yml > > _______________________________________________ > > Ferret-talk mailing list > > Ferret-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/ferret-talk > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From neeraj.jsr at gmail.com Fri Sep 14 10:47:02 2007 From: neeraj.jsr at gmail.com (Raj Singh) Date: Fri, 14 Sep 2007 16:47:02 +0200 Subject: [Ferret-talk] How to call optimize Message-ID: I was under the impression that Model.rebuild_index will automatically call optimize in the end. But that doesn't seem to be the case. After running Model.rebuild_index how do I optimize the index. I am using aaf most stable and also ferret most stable. def rebuild_index(*models) models << aaf_configuration[:class_name] unless models.include?(aaf_configuration[:class_name]) models = models.flatten.uniq.map(&:constantize) logger.debug "rebuild index: #{models.inspect}" index = Ferret::Index::Index.new(aaf_configuration[:ferret].dup.update(:auto_flush => false, :field_infos => ActsAsFerret::field_infos(models), :create => true)) index.batch_size = aaf_configuration[:reindex_batch_size] index.logger = logger index.index_models models end -- Posted via http://www.ruby-forum.com/. From hakita at gmail.com Fri Sep 14 11:43:20 2007 From: hakita at gmail.com (Hakita Hakita) Date: Fri, 14 Sep 2007 17:43:20 +0200 Subject: [Ferret-talk] bulk_index Message-ID: <7735296f16ec1f24eb06a7df532fdc21@ruby-forum.com> Hello Jens, Will bulk_index be available in ferret 0.11.5 ? -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Sun Sep 16 07:16:16 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Sun, 16 Sep 2007 13:16:16 +0200 Subject: [Ferret-talk] bulk_index In-Reply-To: <7735296f16ec1f24eb06a7df532fdc21@ruby-forum.com> References: <7735296f16ec1f24eb06a7df532fdc21@ruby-forum.com> Message-ID: <20070916111616.GC12727@thunder.jkraemer.net> On Fri, Sep 14, 2007 at 05:43:20PM +0200, Hakita Hakita wrote: > Hello Jens, > > Will bulk_index be available in ferret 0.11.5 ? > -- bulk_index is a feature of acts_as_ferret, and therefore independent from Ferret's release cycle. It will be available with the next release of acts_as_ferret. Cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From bincy at qburst.com Mon Sep 17 08:07:31 2007 From: bincy at qburst.com (Bincy P p) Date: Mon, 17 Sep 2007 14:07:31 +0200 Subject: [Ferret-talk] Ferret FileNotFound error Message-ID: <006294b7214da2262fe5bac80295c597@ruby-forum.com> Hi, I have a model that I've been indexing and searching with ferret some problem. We get the following error when accessing the model File Not Found Error occured at :117 in xpop_context Error occured in fs_store.c:329 - fs_open_input tried to open "/var/www/html/cuproject/index/development/album/_b.cfs" but it doesn't exist: But the error comes up only sometimes :( Any one pls help Thanks in advance Bincy -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Mon Sep 17 08:38:26 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Mon, 17 Sep 2007 14:38:26 +0200 Subject: [Ferret-talk] Ferret FileNotFound error In-Reply-To: <006294b7214da2262fe5bac80295c597@ruby-forum.com> References: <006294b7214da2262fe5bac80295c597@ruby-forum.com> Message-ID: > > File Not Found Error occured at :117 in xpop_context > Error occured in fs_store.c:329 - fs_open_input > tried to open "/var/www/html/cuproject/index/development/album/ > _b.cfs" > but it doesn't exist: > > But the error comes up only sometimes :( Hey .. to you use acts_as_ferret and the acts_as_ferret ferret-drb-server? if not, do so please :) http://projects.jkraemer.net/acts_as_ferret/wiki/DrbServer Ben From hakita at gmail.com Mon Sep 17 08:46:14 2007 From: hakita at gmail.com (Hakita Hakita) Date: Mon, 17 Sep 2007 14:46:14 +0200 Subject: [Ferret-talk] bulk_index In-Reply-To: <20070916111616.GC12727@thunder.jkraemer.net> References: <7735296f16ec1f24eb06a7df532fdc21@ruby-forum.com> <20070916111616.GC12727@thunder.jkraemer.net> Message-ID: <46cbe89b89cc3587617d1cbfb4e3f97e@ruby-forum.com> Ok, thank you Jens -- Posted via http://www.ruby-forum.com/. From bakki.kudva at gmail.com Mon Sep 17 15:21:02 2007 From: bakki.kudva at gmail.com (Bakki Kudva) Date: Mon, 17 Sep 2007 15:21:02 -0400 Subject: [Ferret-talk] Drb - shared app with multiple private databases - howto? Message-ID: Hello, I've been trying to get my head around ferret and the one thing that stumps me is how I might use Drb on a dedicated server with multiple clients using the SAME application but each with their own private database and other file assets (in an uploads directory within the app) Thank you for any suggestions, pointer etc. Also I'd be very interested in hearing from those who are using it in production apps right now and what your experience has been in terms of stability. -bakki From julioody at gmail.com Mon Sep 17 21:02:10 2007 From: julioody at gmail.com (Julio Cesar Ody) Date: Tue, 18 Sep 2007 11:02:10 +1000 Subject: [Ferret-talk] Ferret FileNotFound error In-Reply-To: References: <006294b7214da2262fe5bac80295c597@ruby-forum.com> Message-ID: Which is funny, because the only reason, as far as I understand it, for the DRb server to exist is to have a single process writing to the index. And sometimes when I'm in development mode, running a single instance of mongrel, I also get "File Not Found" errors with Ferret. Happens very very rarely, but it does. Of course this might be a bug in Ferret itself, and given that development is pretty much stuck, it's hard to tell (hey, I'm not C expert, and I wish I was). On 9/17/07, Benjamin Krause wrote: > > > > > File Not Found Error occured at :117 in xpop_context > > Error occured in fs_store.c:329 - fs_open_input > > tried to open "/var/www/html/cuproject/index/development/album/ > > _b.cfs" > > but it doesn't exist: > > > > But the error comes up only sometimes :( > > Hey .. > > to you use acts_as_ferret and the acts_as_ferret ferret-drb-server? > if not, do so please :) > > http://projects.jkraemer.net/acts_as_ferret/wiki/DrbServer > > Ben > > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From jrtowell at gmail.com Mon Sep 17 21:55:32 2007 From: jrtowell at gmail.com (Jonathan Towell) Date: Tue, 18 Sep 2007 03:55:32 +0200 Subject: [Ferret-talk] basic rdig setup Message-ID: I'm developing locally on Windows and I have a remote dev box that runs Linux. I'm trying to use RDig just to index using urls, no files. Both use acts_as_ferret for an administrative search that works fine. On the Windows machine, I get no errors, but get no results. On the Linux machine, I get: File Not Found Error occured at :93 in xraise Error occured in index.c:840 - sis_find_segments_file couldn't find segments file On both machines I have run the indexer with no errors using: rdig -c config/rdig_config.rb Both machines have an index dir at the rails root that has two files, segments and segments_0. Both files look like they have next to nothing in them. Both rdig_config.rb files look like: cfg.crawler.start_urls = [ 'http://domain.tpl/' ] cfg.crawler.include_hosts = [ 'domain.tpl/' ] cfg.index.path = './rdig_index' cfg.verbose = true cfg.content_extraction = OpenStruct.new( :hpricot => OpenStruct.new( :title_tag_selector => 'title', :content_tag_selector => 'body' ) Both enviroment.rb files have: require 'acts_as_ferret' require 'rdig' require 'rdig_config' Finally, both have rdig and hpricot gems installed. Any help would be great. -- Posted via http://www.ruby-forum.com/. From shigeruodani at yahoo.com Mon Sep 17 23:12:54 2007 From: shigeruodani at yahoo.com (Eggman Eggman) Date: Tue, 18 Sep 2007 05:12:54 +0200 Subject: [Ferret-talk] basic rdig setup In-Reply-To: References: Message-ID: I'm just getting started with rdig and rails as well. I'm working off Windows and when I do a crawl, I get 3 files - segments, segments_1 and _0.cfs. The cfs file appears to contain the actual crawl. After the crawl, you can do a query with rdig -c CONFIGFILE -q 'your query' if you haven't tried that already. I'm having trouble getting the crawl into a variable. There seems to be a problem with this statement: search_results = RDig.searcher.search(params[:query]) Perhaps you can PM me and we can work together. -- Posted via http://www.ruby-forum.com/. From shigeruodani at yahoo.com Mon Sep 17 23:22:14 2007 From: shigeruodani at yahoo.com (Eggman Eggman) Date: Tue, 18 Sep 2007 05:22:14 +0200 Subject: [Ferret-talk] basic rdig setup In-Reply-To: References: Message-ID: <1d0884cc3a0a90c602549318bd1ac617@ruby-forum.com> Actually there's no problem with that statement. I just overlooked the fact that a parameter needs to get passed to it. Eggman Eggman wrote: > I'm just getting started with rdig and rails as well. I'm working off > Windows and when I do a crawl, I get 3 files - segments, segments_1 and > _0.cfs. The cfs file appears to contain the actual crawl. > > After the crawl, you can do a query with rdig -c CONFIGFILE -q 'your > query' if you haven't tried that already. > > I'm having trouble getting the crawl into a variable. There seems to be > a problem with this statement: > search_results = RDig.searcher.search(params[:query]) > > Perhaps you can PM me and we can work together. -- Posted via http://www.ruby-forum.com/. From jrtowell at gmail.com Mon Sep 17 23:27:28 2007 From: jrtowell at gmail.com (Jonathan Towell) Date: Tue, 18 Sep 2007 05:27:28 +0200 Subject: [Ferret-talk] basic rdig setup In-Reply-To: References: Message-ID: <5f3920f8f28af95488feaaa735af9b4b@ruby-forum.com> > After the crawl, you can do a query with rdig -c CONFIGFILE -q 'your > query' if you haven't tried that already. Yeah, I've tried that and get no errors, but no results either. It appears that the crawl didn't actually index anything. Is "cfg.verbose = true" supposed to show me errors during the crawl? -- Posted via http://www.ruby-forum.com/. From adam at digirati.co.nz Mon Sep 17 23:31:25 2007 From: adam at digirati.co.nz (Adam Jones) Date: Tue, 18 Sep 2007 05:31:25 +0200 Subject: [Ferret-talk] acts_as_ferret with polymorphic associations In-Reply-To: <20061121085127.GE14323@cordoba.webit.de> References: <8e76359318638fb9f0d8ac5f5994aa6f@ruby-forum.com> <20061107073623.GA27583@cordoba.webit.de> <1012fbe7cb9fb07606c33a80dd630701@ruby-forum.com> <40216b7035e101d7cbb1b3b7bf82e9d8@ruby-forum.com> <20061107131530.GG14929@cordoba.webit.de> <070cd419b72e4e428decbad2de835cb7@ruby-forum.com> <3ed8a36595173a05d493e072a6d416a1@ruby-forum.com> <20061116154238.GP639@cordoba.webit.de> <0916c27162c0b712d588246eed5fcd9d@ruby-forum.com> <20061120110123.GF22508@cordoba.webit.de> <1577daf4cc669d362a9725626f395568@ruby-forum.com> <919b866d6f9fef08c079bbb7722eeb0e@ruby-forum.com> <20061121085127.GE14323@cordoba.webit.de> Message-ID: <2e4be276f8cbaef5705e24970bc079be@ruby-forum.com> Thanks guys, The above seems to work for me perfectly except when i add a new not it doesnt update the index. if i delete and rebuild the index it works fine. can some one stear me in the right direction please i am guesing i have to add some metod to the create function in my notes_controller to update the index ??? Thanks, Adam my code in contacts_controller: ------------------- class Contact < ActiveRecord::Base validates_presence_of :First_Name validates_presence_of :Last_Name has_many :notes, :as => :notable acts_as_ferret :store_class_name => true, :lazy => true, :remote => false, :fields => { :First_Name => { :store => :no, :boost => 2 }, :Last_Name => { :store => :no, :boost => 1 }, :description => { :store => :yes, :boost => 0.5 }, :agent_id => { :store => :no, :boost => 0 }, :note_note => { :store => :yes, :boost => 0 } } def note_note @index = Array.new for note in self.notes @index << note.note end return @index.join(" ") end -- Posted via http://www.ruby-forum.com/. From adam at digirati.co.nz Tue Sep 18 00:10:38 2007 From: adam at digirati.co.nz (Adam Jones) Date: Tue, 18 Sep 2007 06:10:38 +0200 Subject: [Ferret-talk] acts_as_ferret with polymorphic associations In-Reply-To: <2e4be276f8cbaef5705e24970bc079be@ruby-forum.com> References: <8e76359318638fb9f0d8ac5f5994aa6f@ruby-forum.com> <20061107073623.GA27583@cordoba.webit.de> <1012fbe7cb9fb07606c33a80dd630701@ruby-forum.com> <40216b7035e101d7cbb1b3b7bf82e9d8@ruby-forum.com> <20061107131530.GG14929@cordoba.webit.de> <070cd419b72e4e428decbad2de835cb7@ruby-forum.com> <3ed8a36595173a05d493e072a6d416a1@ruby-forum.com> <20061116154238.GP639@cordoba.webit.de> <0916c27162c0b712d588246eed5fcd9d@ruby-forum.com> <20061120110123.GF22508@cordoba.webit.de> <1577daf4cc669d362a9725626f395568@ruby-forum.com> <919b866d6f9fef08c079bbb7722eeb0e@ruby-forum.com> <20061121085127.GE14323@cordoba.webit.de> <2e4be276f8cbaef5705e24970bc079be@ruby-forum.com> Message-ID: <5da130ec52974aea9dd6eaa74c4286bf@ruby-forum.com> I think i answered my own question. in the create method for my note i added Contact.find(params[:id]).ferret_create() hooha Adam Jones wrote: > Thanks guys, > > The above seems to work for me perfectly except when i add a new not it > doesnt update the index. if i delete and rebuild the index it works > fine. can some one stear me in the right direction please i am guesing i > have to add some metod to the create function in my notes_controller to > update the index ??? > -- Posted via http://www.ruby-forum.com/. From valblade at gmail.com Wed Sep 19 09:39:14 2007 From: valblade at gmail.com (Matt Valblade) Date: Wed, 19 Sep 2007 15:39:14 +0200 Subject: [Ferret-talk] how to update index from a script In-Reply-To: <20061129131039.GH7968@cordoba.webit.de> References: <5542b84741b5b84f17e2a06392b0e458@ruby-forum.com> <20061129131039.GH7968@cordoba.webit.de> Message-ID: > You'd have to mimic the way how aaf uses the index in your script. the > to_doc method in instance_methods.rb should be a goot starting point. > > It would be way easier to use ActiveRecord in the script and run > it through script/runner - that way aaf will catch the updates. Little up :) Hi ! I have the same problem : I use a Ruby on rails (Hobo) application with a database filled by antother ruby script. I also use acts_as_ferret plugin to offer full text search capability to my rails application. This is the indexed model : ============================ class Job < ActiveRecord::Base acts_as_ferret :additional_fields => [:username, :hostname] hobo_model belongs_to :atuser belongs_to :athost #--- Used to index atUsers and atHosts ---# def username return atuser.name end def hostname return athost.name end etc... end #~ Job The problem is that the ferret index is not updated when I add/delete information in my database (because I use an external application (written in Ruby) to fill my database and I haven't acces save/destroy rails model functions (overloaded by the acts_as_ferret plugin...)) --------- I thought to 2 solutions : - WebService (WSDL) that give me acces to save/destroy function of my rails app - Update Ferret index "On the fly" when inserting new values in database. ---------- I prefer the second solution because the first risks to be too slow, and I need rapidity. Finaly, the question is : How can I update a Ferret index out of my rails application ? -- Posted via http://www.ruby-forum.com/. From valblade at gmail.com Wed Sep 19 09:42:52 2007 From: valblade at gmail.com (Matt Valblade) Date: Wed, 19 Sep 2007 15:42:52 +0200 Subject: [Ferret-talk] how to update index from a script In-Reply-To: References: <5542b84741b5b84f17e2a06392b0e458@ruby-forum.com> <20061129131039.GH7968@cordoba.webit.de> Message-ID: <99181a52dab3d7a021fe79bcee79c868@ruby-forum.com> Sorry, I forgot the most important : Tanks ! :-) ;) -- Posted via http://www.ruby-forum.com/. From valblade at gmail.com Wed Sep 19 09:55:05 2007 From: valblade at gmail.com (Matt Valblade) Date: Wed, 19 Sep 2007 15:55:05 +0200 Subject: [Ferret-talk] how to update index from a script In-Reply-To: <99181a52dab3d7a021fe79bcee79c868@ruby-forum.com> References: <5542b84741b5b84f17e2a06392b0e458@ruby-forum.com> <20061129131039.GH7968@cordoba.webit.de> <99181a52dab3d7a021fe79bcee79c868@ruby-forum.com> Message-ID: Matt Valblade wrote: > Sorry, I forgot the most important : > > Tanks ! :-) > > ;) s/Tanks/Thanks... -- Posted via http://www.ruby-forum.com/. From alex at stacktrace.com Wed Sep 19 21:41:03 2007 From: alex at stacktrace.com (Mitch Mon) Date: Thu, 20 Sep 2007 03:41:03 +0200 Subject: [Ferret-talk] Strange intermittent no search results problem In-Reply-To: <20070704074137.GI4305@cordoba.webit.de> References: <42d8808f0707021113kcfb1426w827437938d88e1e9@mail.gmail.com> <20070703072210.GB4305@cordoba.webit.de> <42d8808f0707030859q57e8f3d4i589e9d3e68b263b8@mail.gmail.com> <20070703162058.GG4305@cordoba.webit.de> <42d8808f0707031102n71c918a0g13335f75bcdca0ee@mail.gmail.com> <42d8808f0707031134qcba5eb3i861c0e16e9ce989@mail.gmail.com> <20070704074137.GI4305@cordoba.webit.de> Message-ID: <46794e165b260caef3e131ff0dd56821@ruby-forum.com> I am experiencing the same problem. Seems like some sort of caching race condition. Thanks for looking into it. My query looks like this: +listing_type:sub +(+lexo_geo_lon:[05760173 05795933] +lexo_geo_lat:[13751926 13774212]) +lexo_price:[000000000 000000650] No wildcards or even terms in this search It's very hard to reproduce if the query changes slightly, which makes me think it is a caching issue. -- Posted via http://www.ruby-forum.com/. From alex at stacktrace.com Wed Sep 19 21:46:37 2007 From: alex at stacktrace.com (Mitch Mon) Date: Thu, 20 Sep 2007 03:46:37 +0200 Subject: [Ferret-talk] Strange intermittent no search results problem In-Reply-To: <46794e165b260caef3e131ff0dd56821@ruby-forum.com> References: <42d8808f0707021113kcfb1426w827437938d88e1e9@mail.gmail.com> <20070703072210.GB4305@cordoba.webit.de> <42d8808f0707030859q57e8f3d4i589e9d3e68b263b8@mail.gmail.com> <20070703162058.GG4305@cordoba.webit.de> <42d8808f0707031102n71c918a0g13335f75bcdca0ee@mail.gmail.com> <42d8808f0707031134qcba5eb3i861c0e16e9ce989@mail.gmail.com> <20070704074137.GI4305@cordoba.webit.de> <46794e165b260caef3e131ff0dd56821@ruby-forum.com> Message-ID: <0a3e3b2c79cc1df3645da2f6f32433ec@ruby-forum.com> Side effect of multi_search is that :store_class_names must be set, which may require re-indexing. -- Posted via http://www.ruby-forum.com/. From jared at scribd.com Thu Sep 20 00:10:45 2007 From: jared at scribd.com (Jared Friedman) Date: Thu, 20 Sep 2007 06:10:45 +0200 Subject: [Ferret-talk] Interesting question on how ferret indexes work Message-ID: I have a question about how Ferret handles intersection queries. Say I do an intersection search on two fields, so I do 'ruby author:(jared)' (no quotes) Does Ferret do some kind of fancy index intersection to efficiently retrieve only items that meet both queries? Or does it just find all the ruby's, then prune away everything that doesn't match author:(jared)? Does it run both searches independently, then take the intersection? Help understanding this would be much appreciated. -- Posted via http://www.ruby-forum.com/. From mravibsp1 at gmail.com Thu Sep 20 09:29:18 2007 From: mravibsp1 at gmail.com (ravi kumar) Date: Thu, 20 Sep 2007 18:59:18 +0530 Subject: [Ferret-talk] ferret reindex_batch_size Message-ID: I am using the act_as_ferret in my app. I want to know that if i have set the reindex_batch_size => 500 or something . can u guide me how it works means the batch_size or can anyone provide me with a link to go through that.. Thanks Ravi -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20070920/b47e8a6d/attachment.html From bkelly01 at yahoo.com Thu Sep 20 14:01:48 2007 From: bkelly01 at yahoo.com (Brandon Kelly) Date: Thu, 20 Sep 2007 20:01:48 +0200 Subject: [Ferret-talk] Ferret DRB, UTF-8, Mongrel Message-ID: I have spent days trying to figure out how to get UTF-8 working with my site. Here's my environment: Linux version 2.6.16.29-xen_3.0.3.0 Ruby 1.8.4 (2005-12-24 [i386-linux] Rails 1.2.3 mongrel (1.0.1) mongrel_cluster (1.0.2, 0.2.1) ferret (0.11.4) acts_as_ferret stable plugin Ferret DRB server When I don't use an analyzer with my acts_as_ferret declaration, everything works fine. However, I can't expect users to enter "?lex Rodr?guez" when searching.. they're going to put "alex rodriguez" (or some variation of his name, which I handle using a fuzzy search) So then call an analyzer in my acts_as_ferret declaration: acts_as_ferret({ :fields => {:first_name => {:store => :no}, :last_name => {:store => :no}, :db_state => {:index => :untokenized_omit_norms, :term_vector => :no}}, :remote => true}, {:analyzer => UtfAnalyzer.new}) Here's the analyzer I'm using... pretty much taken from from here: http://ferret.davebalmain.com/api/classes/Ferret/Analysis/MappingFilter.html ----- class UtfAnalyzer < Ferret::Analysis::Analyzer include Ferret::Analysis CHARACTER_MAPPINGS = { ['?','?','?','?','?','?','?','?'] => 'a', '?' => 'ae', ['?','?'] => 'd', ['?','?','?','?','?'] => 'c', ['?','?','?','?','?','?','?','?','?',] => 'e', ['?'] => 'f', ['?','?','?','?'] => 'g', ['?','?'] => 'h', ['?','?','?','?','?','?','?','?'] => 'i', ['?','?','?','?'] => 'j', ['?','?'] => 'k', ['?','?','?','?','?'] => 'l', ['?','?','?','?','?','?'] => 'n', ['?','?','?','?','?','?','?','?','?','?'] => 'o', ['?'] => 'oek', ['?'] => 'q', ['?','?','?'] => 'r', ['?','?','?','?','?'] => 's', ['?','?','?','?'] => 't', ['?','?','?','?','?','?','?','?','?','?'] => 'u', ['?'] => 'w', ['?','?','?'] => 'y', ['?','?','?'] => 'z' } def token_stream(field, str) MappingFilter.new(StandardTokenizer.new(str), CHARACTER_MAPPINGS) end end I think Ferret is working fine... because when I run some tests, the mapping filter correctly pulls out the accented characters... exactly as it should. However, when something is persisted via the model (acts_as_ferret and DRB server), I get unexpected behavior... - using a model with ONE field declared in acts_as_ferret, and a string with accented characters -- I can search it as expected - with either accented or non-accented character, adn I get the results returned; however, I don't get any other results for the non-accented records. ONLY the accented records get returned when searching. - using a model with multiple characters defined (as in Player model above) -- nothing gets returned, neither accented or non-accented records, or any combination My ferret_server.log file shows characters that are very different from the accented characters I'm trying to search on... Search entered in form: ?lex Rodr?guez ferret_server.log: ?lex rodr??guez Not sure why this is occuring, but I've also redisplayed the submitted text on a web page and it displays correctly. This leads me to believe that Ruby/Rails is successfully getting the information, and that html page encoding is correct, along with environment variables, etc.. As I stated earlier, my Ferret test takes the string "Rodr?guez" and returns token["Rodriguez":0:10:1] demonstrating the UtfAnalyzer works fine outside of acts_as_ferret... So any help here would be much appreciated. Thanks, Brandon -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Thu Sep 20 16:28:38 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 20 Sep 2007 22:28:38 +0200 Subject: [Ferret-talk] how to update index from a script In-Reply-To: References: <5542b84741b5b84f17e2a06392b0e458@ruby-forum.com> <20061129131039.GH7968@cordoba.webit.de> Message-ID: <20070920202838.GM12727@thunder.jkraemer.net> Hi! On Wed, Sep 19, 2007 at 03:39:14PM +0200, Matt Valblade wrote: [..] > > The problem is that the ferret index is not updated when I add/delete > information in my database (because I use an external application > (written in Ruby) to fill my database and I haven't acces save/destroy > rails model functions (overloaded by the acts_as_ferret plugin...)) > > --------- > > I thought to 2 solutions : > > - WebService (WSDL) that give me acces to save/destroy function of my > rails app > - Update Ferret index "On the fly" when inserting new values in > database. > > ---------- > > I prefer the second solution because the first risks to be too slow, and > I need rapidity. > > Finaly, the question is : How can I update a Ferret index out of my > rails application ? The safest way would be to talk to Ferret's DRb server. If you don't do any updates to the index at all through your web app, you might also just use plain Ferret to update the index. Code for adding a record to the index via DRb might look like this: server = DRbObject.new(nil, 'druby://localhost:9010') server << record.class.name, record.to_ferret_doc Aaf adds it's own to_doc method to your AR model, but since you don't use AR in your external script, you'll have to implement your own - see instance_methods.rb for how aaf does this. Future versions of aaf might better support this scenario - there are plans to decouple it from ActiveRecord so it can be used with non-AR classes. But don't expect anything in this direction too soon ... cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Thu Sep 20 16:32:30 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 20 Sep 2007 22:32:30 +0200 Subject: [Ferret-talk] Ferret DRB, UTF-8, Mongrel In-Reply-To: References: Message-ID: <20070920203229.GN12727@thunder.jkraemer.net> Hi! This is really strange - are you sure the DRb server runs in a proper utf8 environment, just as your testcases do? Jens On Thu, Sep 20, 2007 at 08:01:48PM +0200, Brandon Kelly wrote: > I have spent days trying to figure out how to get UTF-8 working with my > site. > > Here's my environment: > > Linux version 2.6.16.29-xen_3.0.3.0 > Ruby 1.8.4 (2005-12-24 [i386-linux] > Rails 1.2.3 > mongrel (1.0.1) > mongrel_cluster (1.0.2, 0.2.1) > ferret (0.11.4) > acts_as_ferret stable plugin > Ferret DRB server > > > When I don't use an analyzer with my acts_as_ferret declaration, > everything works fine. However, I can't expect users to enter "?lex > Rodr?guez" when searching.. they're going to put "alex rodriguez" (or > some variation of his name, which I handle using a fuzzy search) > > So then call an analyzer in my acts_as_ferret declaration: > > > acts_as_ferret({ :fields => {:first_name => {:store => :no}, > :last_name => {:store => :no}, > :db_state => {:index => > :untokenized_omit_norms, :term_vector => :no}}, > :remote => true}, {:analyzer => UtfAnalyzer.new}) > > > Here's the analyzer I'm using... pretty much taken from from here: > http://ferret.davebalmain.com/api/classes/Ferret/Analysis/MappingFilter.html > > ----- > class UtfAnalyzer < Ferret::Analysis::Analyzer > include Ferret::Analysis > CHARACTER_MAPPINGS = { > ['?','?','?','?','?','?','?','?'] => 'a', > '?' => 'ae', > ['?','?'] => 'd', > ['?','?','?','?','?'] => 'c', > ['?','?','?','?','?','?','?','?','?',] => 'e', > ['?'] => 'f', > ['?','?','?','?'] => 'g', > ['?','?'] => 'h', > ['?','?','?','?','?','?','?','?'] => 'i', > ['?','?','?','?'] => 'j', > ['?','?'] => 'k', > ['?','?','?','?','?'] => 'l', > ['?','?','?','?','?','?'] => 'n', > ['?','?','?','?','?','?','?','?','?','?'] => 'o', > ['?'] => 'oek', > ['?'] => 'q', > ['?','?','?'] => 'r', > ['?','?','?','?','?'] => 's', > ['?','?','?','?'] => 't', > ['?','?','?','?','?','?','?','?','?','?'] => 'u', > ['?'] => 'w', > ['?','?','?'] => 'y', > ['?','?','?'] => 'z' > } > > def token_stream(field, str) > MappingFilter.new(StandardTokenizer.new(str), CHARACTER_MAPPINGS) > end > > end > > I think Ferret is working fine... because when I run some tests, the > mapping filter correctly pulls out the accented characters... exactly as > it should. > > However, when something is persisted via the model (acts_as_ferret and > DRB server), I get unexpected behavior... > > - using a model with ONE field declared in acts_as_ferret, and a string > with accented characters -- I can search it as expected - with either > accented or non-accented character, adn I get the results returned; > however, I don't get any other results for the non-accented records. > ONLY the accented records get returned when searching. > > - using a model with multiple characters defined (as in Player model > above) -- nothing gets returned, neither accented or non-accented > records, or any combination > > My ferret_server.log file shows characters that are very different from > the accented characters I'm trying to search on... > > Search entered in form: ?lex Rodr?guez > ferret_server.log: ?lex rodr??guez > > Not sure why this is occuring, but I've also redisplayed the submitted > text on a web page and it displays correctly. This leads me to believe > that Ruby/Rails is successfully getting the information, and that html > page encoding is correct, along with environment variables, etc.. As I > stated earlier, my Ferret test takes the string "Rodr?guez" and returns > token["Rodriguez":0:10:1] demonstrating the UtfAnalyzer works fine > outside of acts_as_ferret... > > So any help here would be much appreciated. > > Thanks, > > Brandon > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Thu Sep 20 16:35:33 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 20 Sep 2007 22:35:33 +0200 Subject: [Ferret-talk] acts_as_ferret with polymorphic associations In-Reply-To: <5da130ec52974aea9dd6eaa74c4286bf@ruby-forum.com> References: <070cd419b72e4e428decbad2de835cb7@ruby-forum.com> <3ed8a36595173a05d493e072a6d416a1@ruby-forum.com> <20061116154238.GP639@cordoba.webit.de> <0916c27162c0b712d588246eed5fcd9d@ruby-forum.com> <20061120110123.GF22508@cordoba.webit.de> <1577daf4cc669d362a9725626f395568@ruby-forum.com> <919b866d6f9fef08c079bbb7722eeb0e@ruby-forum.com> <20061121085127.GE14323@cordoba.webit.de> <2e4be276f8cbaef5705e24970bc079be@ruby-forum.com> <5da130ec52974aea9dd6eaa74c4286bf@ruby-forum.com> Message-ID: <20070920203533.GO12727@thunder.jkraemer.net> On Tue, Sep 18, 2007 at 06:10:38AM +0200, Adam Jones wrote: > I think i answered my own question. > in the create method for my note i added > Contact.find(params[:id]).ferret_create() right, however I'd stick that code in a NotesObserver... Jens > Adam Jones wrote: > > Thanks guys, > > > > The above seems to work for me perfectly except when i add a new not it > > doesnt update the index. if i delete and rebuild the index it works > > fine. can some one stear me in the right direction please i am guesing i > > have to add some metod to the create function in my notes_controller to > > update the index ??? > > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Thu Sep 20 16:42:08 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 20 Sep 2007 22:42:08 +0200 Subject: [Ferret-talk] Ferret FileNotFound error In-Reply-To: References: <006294b7214da2262fe5bac80295c597@ruby-forum.com> Message-ID: <20070920204208.GP12727@thunder.jkraemer.net> On Tue, Sep 18, 2007 at 11:02:10AM +1000, Julio Cesar Ody wrote: > Which is funny, because the only reason, as far as I understand it, > for the DRb server to exist is to have a single process writing to the > index. Right. > And sometimes when I'm in development mode, running a single > instance of mongrel, I also get "File Not Found" errors with Ferret. > Happens very very rarely, but it does. I didn't ever see this in dev mode. An explanation for this might be playing around on the console while your mongrel is running, since then you already have two processes writing to the index. Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Thu Sep 20 16:47:13 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 20 Sep 2007 22:47:13 +0200 Subject: [Ferret-talk] basic rdig setup In-Reply-To: <5f3920f8f28af95488feaaa735af9b4b@ruby-forum.com> References: <5f3920f8f28af95488feaaa735af9b4b@ruby-forum.com> Message-ID: <20070920204713.GQ12727@thunder.jkraemer.net> On Tue, Sep 18, 2007 at 05:27:28AM +0200, Jonathan Towell wrote: > > After the crawl, you can do a query with rdig -c CONFIGFILE -q 'your > > query' if you haven't tried that already. > > Yeah, I've tried that and get no errors, but no results either. It > appears that the crawl didn't actually index anything. > > Is "cfg.verbose = true" supposed to show me errors during the crawl? errors should be shown in any case, verbose adds stack traces to them, and will cause rdig to print out the uri of each document it indexed. If your index ends up with just the segment* files in it, RDig hasn't indexed anything. I'd suspect the cause to be your include hosts config entry, don't use '/' at the end of your hostname there. cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Thu Sep 20 16:52:05 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 20 Sep 2007 22:52:05 +0200 Subject: [Ferret-talk] Drb - shared app with multiple private databases - howto? In-Reply-To: References: Message-ID: <20070920205205.GR12727@thunder.jkraemer.net> On Mon, Sep 17, 2007 at 03:21:02PM -0400, Bakki Kudva wrote: > Hello, > > I've been trying to get my head around ferret and the one thing that > stumps me is how I might use Drb on a dedicated server with multiple > clients using the SAME application but each with their own private > database and other file assets (in an uploads directory within the > app) difficult thing, you'd have to severely hack the DRb server to achieve that (tell the server with each method call which db to use). And you should probably synchronize access inside the DRb server so the switching around of connections doesn't interfere with simultaneous indexing from multiple application processes. cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Thu Sep 20 17:19:00 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Thu, 20 Sep 2007 23:19:00 +0200 Subject: [Ferret-talk] Interesting question on how ferret indexes work In-Reply-To: References: Message-ID: <20070920211900.GS12727@thunder.jkraemer.net> On Thu, Sep 20, 2007 at 06:10:45AM +0200, Jared Friedman wrote: > I have a question about how Ferret handles intersection queries. Say I > do an intersection search on two fields, so I do 'ruby author:(jared)' > (no quotes) > > Does Ferret do some kind of fancy index intersection to efficiently > retrieve only items that meet both queries? Or does it just find all > the ruby's, then prune away everything that doesn't match > author:(jared)? Does it run both searches independently, then take the > intersection? that's really an interesting question, and I'm not really sure about the answer. >From looking at the ConjunctionScorer class of an old version of Ferret (0.3.2, which was still mostly Ruby), I'd say it first collects the document ids matching each single clause, and then steps through them collecting those that have been matched by all clauses. Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From bkelly01 at yahoo.com Thu Sep 20 20:31:04 2007 From: bkelly01 at yahoo.com (Brandon Kelly) Date: Fri, 21 Sep 2007 02:31:04 +0200 Subject: [Ferret-talk] Ferret DRB, UTF-8, Mongrel In-Reply-To: <20070920203229.GN12727@thunder.jkraemer.net> References: <20070920203229.GN12727@thunder.jkraemer.net> Message-ID: Thanks for the quick response Jens. Okay -- my problem apparently is that I've been deploying new code (which stops and starts the ferret server), then I would go in and delete the index. So the index gets recreated, but the DRB server "remembers" the previous index, or settings, or whatever. When I follow these steps, the index is created correctly, and the analyzer works fine... 1. deploy new code 2. script/ferret_stop 3. rm -rf index/production 4. script/ferret_start The key for me to remember is to stop the DRB server BEFORE deleting the index. I've created a simple capistrano recipe to handle this in the future. Thanks again. - Brandon >Jens Kraemer wrote: > Hi! > > This is really strange - are you sure the DRb server runs in a proper > utf8 environment, just as your testcases do? > > Jens > -- Posted via http://www.ruby-forum.com/. From jose.valim at gmail.com Thu Sep 20 21:44:24 2007 From: jose.valim at gmail.com (=?utf-8?Q?Jos=c3=a9_Valim?=) Date: Fri, 21 Sep 2007 03:44:24 +0200 Subject: [Ferret-talk] multi_score? Message-ID: Hi! I'm using acts_as_ferret in my rails project and i need to order the search results by a combination of the result given by ferret (score) and a ranking in the database (normalized also). this combination can be like that: 0.4*ferret_score+0.6*database_score this database_score is also indexed any idea? and someone know how to access the ferret data in the object returned by find_by_content in aaf? thanks in advance! Jos? Valim. -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Sep 21 04:04:37 2007 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 21 Sep 2007 10:04:37 +0200 Subject: [Ferret-talk] Ferret DRB, UTF-8, Mongrel In-Reply-To: References: <20070920203229.GN12727@thunder.jkraemer.net> Message-ID: <20070921080437.GA12116@cordoba.webit.de> On Fri, Sep 21, 2007 at 02:31:04AM +0200, Brandon Kelly wrote: > Thanks for the quick response Jens. > > Okay -- my problem apparently is that I've been deploying new code > (which stops and starts the ferret server), then I would go in and > delete the index. So the index gets recreated, but the DRB server > "remembers" the previous index, or settings, or whatever. > > When I follow these steps, the index is created correctly, and the > analyzer works fine... > > 1. deploy new code > 2. script/ferret_stop > 3. rm -rf index/production > 4. script/ferret_start > > The key for me to remember is to stop the DRB server BEFORE deleting the > index. > > I've created a simple capistrano recipe to handle this in the future. cool. I usually put the index directory into shared/ and symlink it into the current release during deploment. This saves you the index rebuild after deploying. Cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From stuart.hungerford at anu.edu.au Fri Sep 21 02:02:22 2007 From: stuart.hungerford at anu.edu.au (Stuart Hungerford) Date: Fri, 21 Sep 2007 16:02:22 +1000 Subject: [Ferret-talk] Using AAF with ActiveRecord outside of Rails?... Message-ID: <46F35E6E.4010307@anu.edu.au> Hi, My project has been happily using ActiveRecord outside of Rails for some time to populate various databases that are also used in Rails based websites. I'd like to bring AAF into this setup to index new records that are added to the databases as they're added. Has anyone had any success in installing and using AAF with ActiveRecord but outside the Rails plugin support? I may be going about this the wrong way too--if there's a better way of creating and updating the indexes outside of Rails I'd also welcome any advice on that. Thanks! Stu -- Stuart Hungerford ANUSF Data Intensive Projects From valblade at gmail.com Fri Sep 21 07:03:35 2007 From: valblade at gmail.com (Matt Valblade) Date: Fri, 21 Sep 2007 13:03:35 +0200 Subject: [Ferret-talk] how to update index from a script In-Reply-To: <20070920202838.GM12727@thunder.jkraemer.net> References: <5542b84741b5b84f17e2a06392b0e458@ruby-forum.com> <20061129131039.GH7968@cordoba.webit.de> <20070920202838.GM12727@thunder.jkraemer.net> Message-ID: <7267dcb09e12e4746d59b675a168c764@ruby-forum.com> Jens Kraemer wrote: > Hi! > > On Wed, Sep 19, 2007 at 03:39:14PM +0200, Matt Valblade wrote: > [..] >> - WebService (WSDL) that give me acces to save/destroy function of my >> rails application ? > The safest way would be to talk to Ferret's DRb server. If you don't do > any updates to the index at all through your web app, you might also > just use plain Ferret to update the index. > > Code for adding a record to the index via DRb might look like this: > > server = DRbObject.new(nil, 'druby://localhost:9010') > server << record.class.name, record.to_ferret_doc > > Aaf adds it's own to_doc method to your AR model, but since you don't > use AR in your external script, you'll have to implement your own - see > instance_methods.rb for how aaf does this. Hi, thanks for your answer ! I'm sorry, but don't understand why I should connect to this server. The only thing I want to do is to update my index while filling the database with my script : this is the aaf field list: [:username, :athost_id, :atuser_id, :date, :hostname, :url] I have the username, the date, the hostname and the url. I can get the athost_id and atuser_id using a SELECT. why can't I just connect to the aff generated index and update it using ferret itself : # Index already generated by acts_as_ferret (with testing info) index = Index::Index.new(:path => 'index') index.search_each("*") do |id, score| puts "Document #{id} found with a score of #{score}" end #==> puts nothing ! I use the default aff index dir : (From act_methods.rb) :index_dir => "#{ActsAsFerret::index_dir}/#{self.name.underscore}", I tryed to connect to index = Index::Index.new(:path => 'index/development/job') But it doesn't work anymore :( Thanks for your help. -- Posted via http://www.ruby-forum.com/. From bkelly01 at yahoo.com Fri Sep 21 10:10:37 2007 From: bkelly01 at yahoo.com (Brandon Kelly) Date: Fri, 21 Sep 2007 16:10:37 +0200 Subject: [Ferret-talk] Ferret DRB, UTF-8, Mongrel In-Reply-To: <20070921080437.GA12116@cordoba.webit.de> References: <20070920203229.GN12727@thunder.jkraemer.net> <20070921080437.GA12116@cordoba.webit.de> Message-ID: <7ec95aac86bf65941c1fa8006e3d52e5@ruby-forum.com> I do have the site setup this way. My deploy script stops and starts the DRB server without touching the index (which is what I want most of the time). My problem arose when I needed to delete the index. I'd deploy new code, DRB would restart with the old index in place... then I'd delete the old index (while DRB server was running)... and watch it rebuild. The rebuilt index had problems. Wasn't until I realized I need to delete the index only when DRB server isnt' running. (at least that works for me). Thanks again. > cool. I usually put the index directory into shared/ and symlink it into > the current release during deploment. This saves you the index rebuild > after deploying. -- Posted via http://www.ruby-forum.com/. From valblade at gmail.com Fri Sep 21 10:26:44 2007 From: valblade at gmail.com (Matt Valblade) Date: Fri, 21 Sep 2007 16:26:44 +0200 Subject: [Ferret-talk] how to update index from a script In-Reply-To: <7267dcb09e12e4746d59b675a168c764@ruby-forum.com> References: <5542b84741b5b84f17e2a06392b0e458@ruby-forum.com> <20061129131039.GH7968@cordoba.webit.de> <20070920202838.GM12727@thunder.jkraemer.net> <7267dcb09e12e4746d59b675a168c764@ruby-forum.com> Message-ID: <57d83f228776bc7861208fe3488c1967@ruby-forum.com> I have another Idea : My script which updates database have to be lunch periodically (using crontab) What do you think of using a Rake task which to the same job ? I will have a direct access to my Job ActiveRecord using my rails environment. It will be easier to add a new Job (No SQL, thanks to AR) and my ferret index will be updated by aaf! Am I wrong ? Thanks ! -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Sun Sep 23 04:19:06 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Sun, 23 Sep 2007 10:19:06 +0200 Subject: [Ferret-talk] how to update index from a script In-Reply-To: <57d83f228776bc7861208fe3488c1967@ruby-forum.com> References: <5542b84741b5b84f17e2a06392b0e458@ruby-forum.com> <20061129131039.GH7968@cordoba.webit.de> <20070920202838.GM12727@thunder.jkraemer.net> <7267dcb09e12e4746d59b675a168c764@ruby-forum.com> <57d83f228776bc7861208fe3488c1967@ruby-forum.com> Message-ID: <20070923081905.GX12727@thunder.jkraemer.net> On Fri, Sep 21, 2007 at 04:26:44PM +0200, Matt Valblade wrote: > I have another Idea : > > My script which updates database have to be lunch periodically (using > crontab) > > What do you think of using a Rake task which to the same job ? > > I will have a direct access to my Job ActiveRecord using my rails > environment. It will be easier to add a new Job (No SQL, thanks to AR) > and my ferret index will be updated by aaf! That's probably the best way to go. cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Sun Sep 23 04:20:57 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Sun, 23 Sep 2007 10:20:57 +0200 Subject: [Ferret-talk] Ferret DRB, UTF-8, Mongrel In-Reply-To: <7ec95aac86bf65941c1fa8006e3d52e5@ruby-forum.com> References: <20070920203229.GN12727@thunder.jkraemer.net> <20070921080437.GA12116@cordoba.webit.de> <7ec95aac86bf65941c1fa8006e3d52e5@ruby-forum.com> Message-ID: <20070923082057.GY12727@thunder.jkraemer.net> On Fri, Sep 21, 2007 at 04:10:37PM +0200, Brandon Kelly wrote: > I do have the site setup this way. > > My deploy script stops and starts the DRB server without touching the > index (which is what I want most of the time). > > My problem arose when I needed to delete the index. I'd deploy new > code, DRB would restart with the old index in place... then I'd delete > the old index (while DRB server was running)... and watch it rebuild. > The rebuilt index had problems. Wasn't until I realized I need to > delete the index only when DRB server isnt' running. (at least that > works for me). yes, deleting the index while the server is running isn't a good idea. You may also run Model.rebuild_index from a script after deployment to rebuild the index, or even create a rebuild_index deployment recipe via Capistrano. cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Sun Sep 23 04:27:13 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Sun, 23 Sep 2007 10:27:13 +0200 Subject: [Ferret-talk] Using AAF with ActiveRecord outside of Rails?... In-Reply-To: <46F35E6E.4010307@anu.edu.au> References: <46F35E6E.4010307@anu.edu.au> Message-ID: <20070923082713.GZ12727@thunder.jkraemer.net> On Fri, Sep 21, 2007 at 04:02:22PM +1000, Stuart Hungerford wrote: > Hi, > > My project has been happily using ActiveRecord outside of > Rails for some time to populate various databases that are > also used in Rails based websites. > > I'd like to bring AAF into this setup to index new records > that are added to the databases as they're added. > > Has anyone had any success in installing and using AAF > with ActiveRecord but outside the Rails plugin support? I'm not sure but imho this should be possible. From the top of my head, aaf depends on RAILS_ENV to be set to find the proper configuration settings in RAILS_ROOT/config/ferret_server.yml - so you should set up these variables accordingly. Otherwise, requiring the plugin's init.rb should be sufficient to get you going. cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Sun Sep 23 04:33:00 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Sun, 23 Sep 2007 10:33:00 +0200 Subject: [Ferret-talk] multi_score? In-Reply-To: References: Message-ID: <20070923083259.GA12727@thunder.jkraemer.net> On Fri, Sep 21, 2007 at 03:44:24AM +0200, Jos? Valim wrote: > Hi! > > I'm using acts_as_ferret in my rails project and i need to order the > search results by a combination of the result given by ferret (score) > and a ranking in the database (normalized also). this combination can be > like that: > > 0.4*ferret_score+0.6*database_score > > this database_score is also indexed > any idea? Probably the easiest way to achieve this is to use your ranking as a document specific boost (available in current trunk): acts_as_ferret :fields => { ... }, :boost => :normalized_ranking That way your ranking will directly affect the score Ferret calculates. Useful values for boosts in the index are 0..255, you'll have to experiment a bit to see what boosts work best for you. > and someone know how to access the ferret data in the object returned by > find_by_content in aaf? what do you mean by ferret data? You may access the score Ferret assigned to a specific result with the ferret_score method. cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jose.valim at gmail.com Sun Sep 23 06:40:26 2007 From: jose.valim at gmail.com (=?utf-8?Q?Jos=c3=a9_Valim?=) Date: Sun, 23 Sep 2007 12:40:26 +0200 Subject: [Ferret-talk] multi_score? In-Reply-To: <20070923083259.GA12727@thunder.jkraemer.net> References: <20070923083259.GA12727@thunder.jkraemer.net> Message-ID: <8313dd74ade59edf5c7b72090385d207@ruby-forum.com> > what do you mean by ferret data? My aaf is like that: acts_as_ferret :fields => { :title => {:boost => 3}, :url => {:boost => 1}, :content => {:boost => 2, :stored => true}, :keywords => {:boost => 0} }, :remote => true # use ferret_server specified in Only content is not an ActiveRecord attribute, it's a method. So, when i get the results of ferret and do: results.each |result| do result.content end It brings me nothing. I wish I could access the 'content' which is in ferret index. Thanks Jens for you reply. -- Posted via http://www.ruby-forum.com/. From jose.valim at gmail.com Mon Sep 24 07:06:34 2007 From: jose.valim at gmail.com (=?utf-8?Q?Jos=c3=a9_Valim?=) Date: Mon, 24 Sep 2007 13:06:34 +0200 Subject: [Ferret-talk] ferret fuzzy matches Message-ID: <2e4c57ffc0287b6b60efbdab1fe92d50@ruby-forum.com> Hi guys, Is there some way of getting ferret matches string, when i do a fuzzy search? The scenario is this: 1. The user search for 'show' 2. Nothing was found 3. So I do a fuzzy search, passing 'show~' 4. It gives me somes results, most of all was matched with 'showcase' 5. So I want to tell the user that: Nothing was found with 'show', didn't you mean 'showcase'? So, how do i get the ferret matches string, that was 'showcase' in that scenario? Thanks in advance, Jos? Valim -- Posted via http://www.ruby-forum.com/. From bakki.kudva at gmail.com Mon Sep 24 12:05:01 2007 From: bakki.kudva at gmail.com (Bakki Kudva) Date: Mon, 24 Sep 2007 12:05:01 -0400 Subject: [Ferret-talk] Drb - shared app with multiple private databases - howto? In-Reply-To: <20070920205205.GR12727@thunder.jkraemer.net> References: <20070920205205.GR12727@thunder.jkraemer.net> Message-ID: Thank you for your feedback. I am getting to the conclusion that for this and other reasons we should use VPS for each client. So we are thinking of using Xen. -bakki On 9/20/07, Jens Kraemer wrote: > On Mon, Sep 17, 2007 at 03:21:02PM -0400, Bakki Kudva wrote: > > Hello, > > > > I've been trying to get my head around ferret and the one thing that > > stumps me is how I might use Drb on a dedicated server with multiple > > clients using the SAME application but each with their own private > > database and other file assets (in an uploads directory within the > > app) > > difficult thing, you'd have to severely hack the DRb server to achieve > that (tell the server with each method call which db to use). > > And you should probably synchronize access inside the DRb server so the > switching around of connections doesn't interfere with simultaneous > indexing from multiple application processes. > > > cheers, > Jens > > -- > Jens Kr?mer > http://www.jkraemer.net/ - Blog > http://www.omdb.org/ - The new free film database > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > From jose.valim at gmail.com Mon Sep 24 16:24:40 2007 From: jose.valim at gmail.com (=?utf-8?Q?Jos=c3=a9_Valim?=) Date: Mon, 24 Sep 2007 22:24:40 +0200 Subject: [Ferret-talk] multi_score? In-Reply-To: <8313dd74ade59edf5c7b72090385d207@ruby-forum.com> References: <20070923083259.GA12727@thunder.jkraemer.net> <8313dd74ade59edf5c7b72090385d207@ruby-forum.com> Message-ID: <4bbf344c068814eefe3d1995a37c5816@ruby-forum.com> Oh, i've found why i can't get the fields. I should use the key :store => :yes And not :store => true. -- Posted via http://www.ruby-forum.com/. From jose.valim at gmail.com Tue Sep 25 15:06:13 2007 From: jose.valim at gmail.com (=?utf-8?Q?Jos=c3=a9_Valim?=) Date: Tue, 25 Sep 2007 21:06:13 +0200 Subject: [Ferret-talk] ferret fuzzy matches In-Reply-To: <2e4c57ffc0287b6b60efbdab1fe92d50@ruby-forum.com> References: <2e4c57ffc0287b6b60efbdab1fe92d50@ruby-forum.com> Message-ID: <89a4fbd4627665b17814804a2c40d879@ruby-forum.com> I've found out how to do that: You have a class 'Class' with acts_as_ferret, so do that: queryObj = Class.aff_index.ferret_index.process_query(@query_string) searchObj = Class.aaf_index.ferret_index.searcher query_results = queryObj.terms(searchObj).collect! {|item| item[:text] }.uniq query_results.is_a?(Array) => true -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Tue Sep 25 15:45:27 2007 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 25 Sep 2007 21:45:27 +0200 Subject: [Ferret-talk] ferret fuzzy matches In-Reply-To: <89a4fbd4627665b17814804a2c40d879@ruby-forum.com> References: <2e4c57ffc0287b6b60efbdab1fe92d50@ruby-forum.com> <89a4fbd4627665b17814804a2c40d879@ruby-forum.com> Message-ID: <20070925194527.GG12116@cordoba.webit.de> On Tue, Sep 25, 2007 at 09:06:13PM +0200, Jos? Valim wrote: > I've found out how to do that: > > You have a class 'Class' with acts_as_ferret, so do that: > > queryObj = Class.aff_index.ferret_index.process_query(@query_string) > searchObj = Class.aaf_index.ferret_index.searcher > > query_results = queryObj.terms(searchObj).collect! {|item| item[:text] > }.uniq right, however I doubt this kind of direct access to Ferret's searcher will work with aaf's DRb server. You should add a method like 'matching_terms(query_string)' to aaf's LocalIndex class to encapsulate above code, so you can call it across the DRb connection. Cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From vinay.khater at gmail.com Thu Sep 27 05:16:44 2007 From: vinay.khater at gmail.com (Vinay Khater) Date: Thu, 27 Sep 2007 11:16:44 +0200 Subject: [Ferret-talk] Problem getting "extract" from RDig Message-ID: <507ee400cea35b644322868b36f74460@ruby-forum.com> Hi All, I have to have a site wide search for my current application. By search I mean I have to search the static and the dynamic contents from the database. I have been searching on this for a while on the net and RDig seems to be a apt solution. While using it I have encountered a few problems. I know these might be very basic issues but I have not been able to figure out what is wrong with the code. I had the following lines in my /config/environment.rb 1. require 'rdig' 2. require 'rdig_config' I have the following code in my /config/rdig_config.rb 1. RDig.configuration do |cfg| 2. cfg.crawler.start_urls = [ 'http://localhost:3000/login/index' ] 3. cfg.index.path = "C:/rails/managedsupport/index/development/rdig-index" 4. cfg.verbose = true 5. cfg.content_extraction = OpenStruct.new( 6. :hpricot => OpenStruct.new( 7. :title_tag_selector => 'title', 8. :content_tag_selector => 'body' 9. ) 10. ) 11. 12. end I have created the index file using the code 1. rdig -c config/rdig_config.rb Now in my controller I have written a code for testing the functionality 1. search_results = RDig.searcher.search("some_string") 2. @results = search_results[:list] 3. @hitcount = search_results[:hitcount] My @result[:extract]is returning me the same initial view code that is common to the application that is my menus and sub menus.. I am not getting the extract on the basis of which I had searched. Any help in this regard would be highly appreciated.. Thanks in advance:) -- Posted via http://www.ruby-forum.com/. From vinay.khater at gmail.com Thu Sep 27 05:53:28 2007 From: vinay.khater at gmail.com (jazzy jazzy) Date: Thu, 27 Sep 2007 11:53:28 +0200 Subject: [Ferret-talk] can't search for OR (as in the state) In-Reply-To: <2188D5DF-DB87-4A95-AD1C-DA5A8F584FEE@gmx.net> References: <29b5ebfa3996859911e37ca055de4d1b@ruby-forum.com> <2188D5DF-DB87-4A95-AD1C-DA5A8F584FEE@gmx.net> Message-ID: Andreas Korth wrote: > Ferret::Analysis::FULL_ENGLISH_STOP_WORDS.include?('or') > => true > > Index the state field untokenized: > > field_infos.add_field(:state, :index => :untokenized, ...) Hi, I am a novice in rails. I too ran in to the same issue. I did follow your method but am a bit confused.. My model has acts_as_ferret :fields => { :description => {:store => :yes}, :state => {:index => :untokenized}}, #:store_class_name => :true, :remote => false and in my controller I search with this condition = "state:(#{params[:state]})" query = condition + "some_text" I do not understand where should I use the line code Ferret::Analysis::FULL_ENGLISH_STOP_WORDS.include?('or')=> true I am unable to get the desired result when 'OR' is passed as a state. Can you kindly be patient and explain me what am I missing? Any help would be highly appreciated. Cheers, Vinay -- Posted via http://www.ruby-forum.com/. From andy at shenie.info Thu Sep 27 08:41:53 2007 From: andy at shenie.info (Andy Shen) Date: Thu, 27 Sep 2007 22:41:53 +1000 Subject: [Ferret-talk] QueryParser.parse question Message-ID: <9de8cbf60709270541k10ec4c11vb83ac20803654c67@mail.gmail.com> Hi there, I am stomped as to why QueryParser's parse method behaves differently between query 'a' and 'b'. See http://pastie.caboo.se/private/4rlwrecyyow3yl6qtf4tq Could someone please help me understand why that is the case. p.s. I also found 'i' produce the same behavour as 'a' Cheers, Andy From cstrom at mdlogix.com Thu Sep 27 09:08:23 2007 From: cstrom at mdlogix.com (Chris Strom) Date: Thu, 27 Sep 2007 09:08:23 -0400 Subject: [Ferret-talk] QueryParser.parse question In-Reply-To: <9de8cbf60709270541k10ec4c11vb83ac20803654c67@mail.gmail.com> References: <9de8cbf60709270541k10ec4c11vb83ac20803654c67@mail.gmail.com> Message-ID: <20070927130823.GC7033@jaynestown.users.mdlogix.com> 'a' is a stop word by default (so is 'i'). If you instantiate the query parser with an Analyzer with no stop words, thusly: qp3 = Ferret::QueryParser.new( :analyzer => Ferret::Analysis::StandardAnalyzer.new([]) ) Then qp3.parse('a') (after the appropriate fields call) will return the desired results. -Chris On Thu, Sep 27, 2007 at 10:41:53PM +1000, Andy Shen wrote: > Hi there, > > I am stomped as to why QueryParser's parse method behaves differently > between query 'a' and 'b'. > See http://pastie.caboo.se/private/4rlwrecyyow3yl6qtf4tq > > Could someone please help me understand why that is the case. > > p.s. I also found 'i' produce the same behavour as 'a' > > Cheers, > Andy > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From bk at benjaminkrause.com Thu Sep 27 09:14:10 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Thu, 27 Sep 2007 15:14:10 +0200 (CEST) Subject: [Ferret-talk] QueryParser.parse question In-Reply-To: <9de8cbf60709270541k10ec4c11vb83ac20803654c67@mail.gmail.com> References: <9de8cbf60709270541k10ec4c11vb83ac20803654c67@mail.gmail.com> Message-ID: <50026.195.21.27.100.1190898850.squirrel@orkland.homeunix.org> > I am stomped as to why QueryParser's parse method behaves differently > between query 'a' and 'b'. > See http://pastie.caboo.se/private/4rlwrecyyow3yl6qtf4tq > > Could someone please help me understand why that is the case. > > p.s. I also found 'i' produce the same behavour as 'a' hey .. there is a concept called STOP_WORDS .. special words are simply ignored. The QueryParser will strip all STOP_WORDS. You should understand the way STOP_WORDS and QueryParsing works.. I would recommend: http://projects.jkraemer.net/acts_as_ferret/wiki/gotchas http://ferret.davebalmain.com/trac/wiki/FAQ%3AIndexing#HowdoIswitchofstop-wordremoval http://www.oreilly.com/catalog/9780596527853/index.html see also: ./script/console >> Ferret::Analysis::FULL_ENGLISH_STOP_WORDS Ben From andy at shenie.info Thu Sep 27 09:40:27 2007 From: andy at shenie.info (Andy Shen) Date: Thu, 27 Sep 2007 23:40:27 +1000 Subject: [Ferret-talk] QueryParser.parse question In-Reply-To: <20070927130823.GC7033@jaynestown.users.mdlogix.com> References: <9de8cbf60709270541k10ec4c11vb83ac20803654c67@mail.gmail.com> <20070927130823.GC7033@jaynestown.users.mdlogix.com> Message-ID: <9de8cbf60709270640v726a0dd4xc9fac7493e5a4d61@mail.gmail.com> On 9/27/07, Chris Strom wrote: > 'a' is a stop word by default (so is 'i'). If you instantiate the query > parser with an Analyzer with no stop words, thusly: > > qp3 = Ferret::QueryParser.new( > :analyzer => Ferret::Analysis::StandardAnalyzer.new([]) > ) > > Then qp3.parse('a') (after the appropriate fields call) will return the > desired results. > > -Chris > > On Thu, Sep 27, 2007 at 10:41:53PM +1000, Andy Shen wrote: > > Hi there, > > > > I am stomped as to why QueryParser's parse method behaves differently > > between query 'a' and 'b'. > > See http://pastie.caboo.se/private/4rlwrecyyow3yl6qtf4tq > > > > Could someone please help me understand why that is the case. > > > > p.s. I also found 'i' produce the same behavour as 'a' > > > > Cheers, > > Andy > > _______________________________________________ > > Ferret-talk mailing list > > Ferret-talk at rubyforge.org > > http://rubyforge.org/mailman/listinfo/ferret-talk > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > Thank you guys so much for pointing me to the right direction. I feel embarrassed that I have missed the Gotcha wiki page. Those reference links are very useful. Thanks again. Cheers, Andy From bk at benjaminkrause.com Thu Sep 27 09:46:32 2007 From: bk at benjaminkrause.com (Benjamin Krause) Date: Thu, 27 Sep 2007 15:46:32 +0200 (CEST) Subject: [Ferret-talk] QueryParser.parse question In-Reply-To: <9de8cbf60709270640v726a0dd4xc9fac7493e5a4d61@mail.gmail.com> References: <9de8cbf60709270541k10ec4c11vb83ac20803654c67@mail.gmail.com> <20070927130823.GC7033@jaynestown.users.mdlogix.com> <9de8cbf60709270640v726a0dd4xc9fac7493e5a4d61@mail.gmail.com> Message-ID: <52369.195.21.27.100.1190900792.squirrel@orkland.homeunix.org> > Thank you guys so much for pointing me to the right direction. I feel > embarrassed that I have missed the Gotcha wiki page. Those reference > links are very useful. Thanks again. You're welcome, but here're more useful links ;-) http://ferret.davebalmain.com/trac/wiki/DonationsFAQ (donate for ferret) http://tinyurl.com/2my2dp (jens kraemers amazon wishlist) Ben From kraemer at webit.de Thu Sep 27 14:30:35 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 27 Sep 2007 20:30:35 +0200 Subject: [Ferret-talk] QueryParser.parse question In-Reply-To: <52369.195.21.27.100.1190900792.squirrel@orkland.homeunix.org> References: <9de8cbf60709270541k10ec4c11vb83ac20803654c67@mail.gmail.com> <20070927130823.GC7033@jaynestown.users.mdlogix.com> <9de8cbf60709270640v726a0dd4xc9fac7493e5a4d61@mail.gmail.com> <52369.195.21.27.100.1190900792.squirrel@orkland.homeunix.org> Message-ID: <20070927183035.GA17667@cordoba.webit.de> On Thu, Sep 27, 2007 at 03:46:32PM +0200, Benjamin Krause wrote: > > > Thank you guys so much for pointing me to the right direction. I feel > > embarrassed that I have missed the Gotcha wiki page. Those reference > > links are very useful. Thanks again. > > You're welcome, but here're more useful links ;-) > > http://ferret.davebalmain.com/trac/wiki/DonationsFAQ (donate for ferret) > http://tinyurl.com/2my2dp (jens kraemers amazon wishlist) Nope, that's not my whishlist :-) If at all, have a look at this one: http://www.amazon.de/gp/registry/M7CN2KCVD1GT cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From eimorton at gmail.com Thu Sep 27 15:12:51 2007 From: eimorton at gmail.com (Erik Morton) Date: Thu, 27 Sep 2007 15:12:51 -0400 Subject: [Ferret-talk] Remote index blocks? Message-ID: <944D54BF-8203-4E4E-AA52-4D3803B2562D@gmail.com> Using the Drb allows me to synchronize writes to the index in a multi mongrel environment. I was under the impression that the remote index would not block if two mongrels were searching the index. Is that the case? This line in ferret_server.rb makes me think otherwise: # Calls are not queued atm, so this will block until the call returned. # def method_missing(name, *args) I see how the above would allow for synchronizing writes, but I don't see how it would allow for concurrent reads. I'm seeing some issues with production performance of queries, and I'd like to figure out if concurrent queries against the remote server will block or if they are run in different threads. I'm running on linux with the trunk version of AAF and Ruby 1.8.4. Erik From kraemer at webit.de Thu Sep 27 16:29:50 2007 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 27 Sep 2007 22:29:50 +0200 Subject: [Ferret-talk] Remote index blocks? In-Reply-To: <944D54BF-8203-4E4E-AA52-4D3803B2562D@gmail.com> References: <944D54BF-8203-4E4E-AA52-4D3803B2562D@gmail.com> Message-ID: <20070927202950.GB17667@cordoba.webit.de> On Thu, Sep 27, 2007 at 03:12:51PM -0400, Erik Morton wrote: > Using the Drb allows me to synchronize writes to the index in a multi > mongrel environment. I was under the impression that the remote index > would not block if two mongrels were searching the index. Is that the > case? This line in ferret_server.rb makes me think otherwise: > > # Calls are not queued atm, so this will block until the call > returned. Don't worry, it's only bad wording :-) What this means is only that indexing is not done in an asynchronous way. So your call to Model#save which triggers an index update won't return until the server has finished adding that record to the index. Other processes will get their own threads on the DRb side, synchronization is done in Ferret's Index class which allows concurrent searches. cheers, Jens -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold, Hagen Malessa From admin at fapbox.com Fri Sep 28 06:10:51 2007 From: admin at fapbox.com (Jim Neath) Date: Fri, 28 Sep 2007 12:10:51 +0200 Subject: [Ferret-talk] Prevent indexing depending on variable Message-ID: <9168c7de3212ddf5e55fccd275520324@ruby-forum.com> Hi guys, I'm using attachment_fu on my site to allow users to upload pictrues. Pictures can be set to only allow friends to view them. The problem I'm having is ferret is indexing the images along with their thumbnails which means I end up with a lot of broken images in the search. Also it's indexing images whose privacy is set to 1 (friends only). How could I go about only indexing the images with no parent_id (main image, not it's thumbnails) and privacy set to 0 (viewable by all)? -- Posted via http://www.ruby-forum.com/. From chris at nonfatal.com Fri Sep 28 06:38:15 2007 From: chris at nonfatal.com (Chris Williams) Date: Fri, 28 Sep 2007 12:38:15 +0200 Subject: [Ferret-talk] undefined method error Message-ID: <79a3ca39133ff687819e29d2b0bfe29c@ruby-forum.com> Hi, I'm running the trunk version with drb server. In my ferret_server.log I get a lot of messages like this: #method_missing(:add, ["User", {:first_name=>"blah", :comments=>nil, :company=>"blah", :phone_day=>""}]) Over and over again. But more importantly, since yesterday doing a search has caused an exception: A NoMethodError occurred in partners#user_search: undefined method `each' for :sales_region:Symbol [RAILS_ROOT]/vendor/plugins/acts_as_ferret/lib/class_methods.rb:246:in `retrieve_records' Any idea what these could have been caused by? Thanks, Chris. -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Fri Sep 28 07:47:50 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Fri, 28 Sep 2007 13:47:50 +0200 Subject: [Ferret-talk] undefined method error In-Reply-To: <79a3ca39133ff687819e29d2b0bfe29c@ruby-forum.com> References: <79a3ca39133ff687819e29d2b0bfe29c@ruby-forum.com> Message-ID: <20070928114750.GC29859@thunder.jkraemer.net> On Fri, Sep 28, 2007 at 12:38:15PM +0200, Chris Williams wrote: > Hi, > I'm running the trunk version with drb server. In my ferret_server.log I > get a lot of messages like this: > > #method_missing(:add, ["User", {:first_name=>"blah", :comments=>nil, > :company=>"blah", :phone_day=>""}]) > > Over and over again. with aaf trunk you can raise the log level in ferret_server.yml so this debug message doesn't appear any more: production: log_level: warn > But more importantly, since yesterday doing a search has caused an > exception: since yesterday? What did you change? > A NoMethodError occurred in partners#user_search: > > undefined method `each' for :sales_region:Symbol > [RAILS_ROOT]/vendor/plugins/acts_as_ferret/lib/class_methods.rb:246:in > `retrieve_records' > > Any idea what these could have been caused by? not yet - I can't seem to find the line the error refers too - what exact revision are you on? Or could you try that with the latest trunk (rev. 247). How do you use that symbol in your code? Cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Fri Sep 28 07:49:00 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Fri, 28 Sep 2007 13:49:00 +0200 Subject: [Ferret-talk] Prevent indexing depending on variable In-Reply-To: <9168c7de3212ddf5e55fccd275520324@ruby-forum.com> References: <9168c7de3212ddf5e55fccd275520324@ruby-forum.com> Message-ID: <20070928114900.GD29859@thunder.jkraemer.net> On Fri, Sep 28, 2007 at 12:10:51PM +0200, Jim Neath wrote: > Hi guys, I'm using attachment_fu on my site to allow users to upload > pictrues. Pictures can be set to only allow friends to view them. > > The problem I'm having is ferret is indexing the images along with their > thumbnails which means I end up with a lot of broken images in the > search. Also it's indexing images whose privacy is set to 1 (friends > only). > > How could I go about only indexing the images with no parent_id (main > image, not it's thumbnails) and privacy set to 0 (viewable by all)? override the ferret_enabled? instance method in your model class. Cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From admin at fapbox.com Fri Sep 28 07:54:20 2007 From: admin at fapbox.com (Jim Neath) Date: Fri, 28 Sep 2007 13:54:20 +0200 Subject: [Ferret-talk] Prevent indexing depending on variable In-Reply-To: <20070928114900.GD29859@thunder.jkraemer.net> References: <9168c7de3212ddf5e55fccd275520324@ruby-forum.com> <20070928114900.GD29859@thunder.jkraemer.net> Message-ID: <2d25b3cea247d6c94c8a45304037e2b0@ruby-forum.com> Thanks, Jens. How would I go about doing that? -- Posted via http://www.ruby-forum.com/. From john at johnleach.co.uk Fri Sep 28 12:51:16 2007 From: john at johnleach.co.uk (John Leach) Date: Fri, 28 Sep 2007 17:51:16 +0100 Subject: [Ferret-talk] Prevent indexing depending on variable In-Reply-To: <2d25b3cea247d6c94c8a45304037e2b0@ruby-forum.com> References: <9168c7de3212ddf5e55fccd275520324@ruby-forum.com> <20070928114900.GD29859@thunder.jkraemer.net> <2d25b3cea247d6c94c8a45304037e2b0@ruby-forum.com> Message-ID: <1190998276.14184.4.camel@dogen.thepride.> something like: class MyModel < ActiveRecord::Base def ferret_enabled? self.secret == false end end John. On Fri, 2007-09-28 at 13:54 +0200, Jim Neath wrote: > Thanks, Jens. > > How would I go about doing that? -- http://www.brightbox.co.uk - UK Rails Xen Hosting From eimorton at gmail.com Fri Sep 28 15:42:30 2007 From: eimorton at gmail.com (Erik Morton) Date: Fri, 28 Sep 2007 15:42:30 -0400 Subject: [Ferret-talk] Remote index blocks? In-Reply-To: <20070927202950.GB17667@cordoba.webit.de> References: <944D54BF-8203-4E4E-AA52-4D3803B2562D@gmail.com> <20070927202950.GB17667@cordoba.webit.de> Message-ID: <50D7FFF7-22E5-46A0-A824-B0941BB97806@gmail.com> Thanks Jens. I'm seeing very strange behavior. The ferret/drb server is running on the web server. I disable AAF in my environment.rb like so: MyModel.disable_ferret Then, every hour I run a script that grabs the records that have been updated and I call MyModel.bulk_index array_of_changed_objects When there are, say, 100 or so objects that have changed the server where DRB is will have approximately 50% of the CPU waiting on IO, and will drop from 120MB free of memory to 10MB free, though the ruby/ drb process doesn't seem to actually consume that memory -- or at least it's not reported to by top. While the batch update is happening it seems like my entire site is locked up. Requests usually hang until the indexing completes. Further, I can't even run script/ console from a different machine until the indexing completes. I'm doing the bulk index on two indexes. One is about 41K records, the other is 1 million records. In both cases there has been at most 100 or so objects that needed to be indexed in bulk. The fact that script/ console, when run from a different server, doesn't load until the index stops makes me think that either something is blocking in the ferret/drb server, or the optimization of the 3GB index after the bulk_index of 100 records is consuming all of the web server's resources. Any idea what is going on or how I can debug this issue? Thanks in advance. On Sep 27, 2007, at 4:29 PM, Jens Kraemer wrote: > On Thu, Sep 27, 2007 at 03:12:51PM -0400, Erik Morton wrote: >> Using the Drb allows me to synchronize writes to the index in a multi >> mongrel environment. I was under the impression that the remote index >> would not block if two mongrels were searching the index. Is that the >> case? This line in ferret_server.rb makes me think otherwise: >> >> # Calls are not queued atm, so this will block until the call >> returned. > > Don't worry, it's only bad wording :-) > > What this means is only that indexing is not done in an asynchronous > way. So your call to Model#save which triggers an index update won't > return until the server has finished adding that record to the index. > > Other processes will get their own threads on the DRb side, > synchronization is done in Ferret's Index class which allows > concurrent > searches. > > cheers, > Jens > > -- > Jens Kr?mer > webit! Gesellschaft f?r neue Medien mbH > Schnorrstra?e 76 | 01069 Dresden > Telefon +49 351 46766-0 | Telefax +49 351 46766-66 > kraemer at webit.de | www.webit.de > > Amtsgericht Dresden | HRB 15422 > GF Sven Haubold, Hagen Malessa > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From admin at fapbox.com Sat Sep 29 06:07:30 2007 From: admin at fapbox.com (Jim Neath) Date: Sat, 29 Sep 2007 12:07:30 +0200 Subject: [Ferret-talk] Prevent indexing depending on variable In-Reply-To: <1190998276.14184.4.camel@dogen.thepride.> References: <9168c7de3212ddf5e55fccd275520324@ruby-forum.com> <20070928114900.GD29859@thunder.jkraemer.net> <2d25b3cea247d6c94c8a45304037e2b0@ruby-forum.com> <1190998276.14184.4.camel@dogen.thepride.> Message-ID: Would something like this work: def ferret_enabled? self.privacy ? false : self.parent_id ? false : true end -- Posted via http://www.ruby-forum.com/. From admin at fapbox.com Sat Sep 29 06:20:08 2007 From: admin at fapbox.com (Jim Neath) Date: Sat, 29 Sep 2007 12:20:08 +0200 Subject: [Ferret-talk] Prevent indexing depending on variable In-Reply-To: References: <9168c7de3212ddf5e55fccd275520324@ruby-forum.com> <20070928114900.GD29859@thunder.jkraemer.net> <2d25b3cea247d6c94c8a45304037e2b0@ruby-forum.com> <1190998276.14184.4.camel@dogen.thepride.> Message-ID: <02ba5c478cd1f4dcc05cdef06b3d0c75@ruby-forum.com> Jim Neath wrote: > Would something like this work: > > def ferret_enabled? > self.privacy ? false : self.parent_id ? false : true > end Just tried it. Seems to work great with new images. How do I go about rebuilding the index of the current images? -- Posted via http://www.ruby-forum.com/. From jk at jkraemer.net Sat Sep 29 11:55:08 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Sat, 29 Sep 2007 17:55:08 +0200 Subject: [Ferret-talk] Prevent indexing depending on variable In-Reply-To: <02ba5c478cd1f4dcc05cdef06b3d0c75@ruby-forum.com> References: <9168c7de3212ddf5e55fccd275520324@ruby-forum.com> <20070928114900.GD29859@thunder.jkraemer.net> <2d25b3cea247d6c94c8a45304037e2b0@ruby-forum.com> <1190998276.14184.4.camel@dogen.thepride.> <02ba5c478cd1f4dcc05cdef06b3d0c75@ruby-forum.com> Message-ID: <20070929155508.GF29859@thunder.jkraemer.net> On Sat, Sep 29, 2007 at 12:20:08PM +0200, Jim Neath wrote: > Jim Neath wrote: > > Would something like this work: > > > > def ferret_enabled? > > self.privacy ? false : self.parent_id ? false : true > > end > > Just tried it. Seems to work great with new images. How do I go about > rebuilding the index of the current images? Model.rebuild_index -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From jk at jkraemer.net Sat Sep 29 12:12:18 2007 From: jk at jkraemer.net (Jens Kraemer) Date: Sat, 29 Sep 2007 18:12:18 +0200 Subject: [Ferret-talk] Remote index blocks? In-Reply-To: <50D7FFF7-22E5-46A0-A824-B0941BB97806@gmail.com> References: <944D54BF-8203-4E4E-AA52-4D3803B2562D@gmail.com> <20070927202950.GB17667@cordoba.webit.de> <50D7FFF7-22E5-46A0-A824-B0941BB97806@gmail.com> Message-ID: <20070929161218.GG29859@thunder.jkraemer.net> On Fri, Sep 28, 2007 at 03:42:30PM -0400, Erik Morton wrote: > Thanks Jens. > > I'm seeing very strange behavior. The ferret/drb server is running on > the web server. I disable AAF in my environment.rb like so: > > MyModel.disable_ferret > > Then, every hour I run a script that grabs the records that have been > updated and I call > > MyModel.bulk_index array_of_changed_objects > > When there are, say, 100 or so objects that have changed the server > where DRB is will have approximately 50% of the CPU waiting on IO, > and will drop from 120MB free of memory to 10MB free, though the ruby/ > drb process doesn't seem to actually consume that memory -- or at > least it's not reported to by top. Where do you get these numbers from? possibly it's just the os using unused ram for filesystem buffers? > While the batch update is happening it seems like my entire site is > locked up. Requests usually hang until the indexing completes. > Further, I can't even run script/ console from a different machine > until the indexing completes. I'm doing the bulk index on two > indexes. One is about 41K records, the other is 1 million records. In > both cases there has been at most 100 or so objects that needed to be > indexed in bulk. The fact that script/ console, when run from a > different server, doesn't load until the index stops makes me think > that either something is blocking in the ferret/drb server, or the > optimization of the 3GB index after the bulk_index of 100 records is > consuming all of the web server's resources. While the batch update is running, nobody else is able to update the index. So yes, every other request that wants to update the index will hang. However requests not using aaf at all, or searches, should do fine. If you feel like the DRb takes too much CPU, use renice or nice when you start it to lower it's priority. Another possibility that comes to mind is database locks, however I can't imagine where these should come from. With your index size, the optimizing might be the culprit - just comment out that portion and look how it goes without it (in ferret_extensions.rb). cheers, Jens -- Jens Kr?mer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database From eimorton at gmail.com Sat Sep 29 12:27:50 2007 From: eimorton at gmail.com (Erik Morton) Date: Sat, 29 Sep 2007 12:27:50 -0400 Subject: [Ferret-talk] Remote index blocks? In-Reply-To: <20070929161218.GG29859@thunder.jkraemer.net> References: <944D54BF-8203-4E4E-AA52-4D3803B2562D@gmail.com> <20070927202950.GB17667@cordoba.webit.de> <50D7FFF7-22E5-46A0-A824-B0941BB97806@gmail.com> <20070929161218.GG29859@thunder.jkraemer.net> Message-ID: <1F2A1124-D572-4A08-A7C5-D4917DECC343@gmail.com> We are going to hack AAF a bit to make the optimize optional. Should we flush every time we bulk index if we don't optimize? I'm running this application on EC2, so I think part of the problem is the poor IO performance on the VPS. Thanks. On Sep 29, 2007, at 12:12 PM, Jens Kraemer wrote: > On Fri, Sep 28, 2007 at 03:42:30PM -0400, Erik Morton wrote: >> Thanks Jens. >> >> I'm seeing very strange behavior. The ferret/drb server is running on >> the web server. I disable AAF in my environment.rb like so: >> >> MyModel.disable_ferret >> >> Then, every hour I run a script that grabs the records that have been >> updated and I call >> >> MyModel.bulk_index array_of_changed_objects >> >> When there are, say, 100 or so objects that have changed the server >> where DRB is will have approximately 50% of the CPU waiting on IO, >> and will drop from 120MB free of memory to 10MB free, though the >> ruby/ >> drb process doesn't seem to actually consume that memory -- or at >> least it's not reported to by top. > > Where do you get these numbers from? possibly it's just the os using > unused ram for filesystem buffers? > >> While the batch update is happening it seems like my entire site is >> locked up. Requests usually hang until the indexing completes. >> Further, I can't even run script/ console from a different machine >> until the indexing completes. I'm doing the bulk index on two >> indexes. One is about 41K records, the other is 1 million >> records. In >> both cases there has been at most 100 or so objects that needed >> to be >> indexed in bulk. The fact that script/ console, when run from a >> different server, doesn't load until the index stops makes me think >> that either something is blocking in the ferret/drb server, or the >> optimization of the 3GB index after the bulk_index of 100 records is >> consuming all of the web server's resources. > > While the batch update is running, nobody else is able to update the > index. So yes, every other request that wants to update the index will > hang. However requests not using aaf at all, or searches, should do > fine. > > If you feel like the DRb takes too much CPU, use renice or nice > when you > start it to lower it's priority. > > Another possibility that comes to mind is database locks, however I > can't imagine where these should come from. > > With your index size, the optimizing might be the culprit - just > comment > out that portion and look how it goes without it (in > ferret_extensions.rb). > > cheers, > Jens > > -- > Jens Kr?mer > http://www.jkraemer.net/ - Blog > http://www.omdb.org/ - The new free film database > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk From shigeruodani at yahoo.com Sun Sep 30 10:35:18 2007 From: shigeruodani at yahoo.com (Eggman Eggman) Date: Sun, 30 Sep 2007 16:35:18 +0200 Subject: [Ferret-talk] how to simply display the index Message-ID: <15fca432694d43719036315bbde06afd@ruby-forum.com> I've read the O'reilly short cut and I can't figure out how to display the index for testing purposes and view documents and fields. Also, I'm planning on creating an index of www html documents. Is there a preference between using the ferret file system and a mysql database? -- Posted via http://www.ruby-forum.com/. From scott at apewebsites.com.au Sun Sep 30 22:50:20 2007 From: scott at apewebsites.com.au (Scott Harvey) Date: Mon, 1 Oct 2007 04:50:20 +0200 Subject: [Ferret-talk] Custom Analyser .. where to put it ?? In-Reply-To: References: <41583cdffa9de1d213806123a4bf35ef@ruby-forum.com> <20070907145503.GA6338@cordoba.webit.de> <5444996b760cf4668155fffbc68dfc63@ruby-forum.com> Message-ID: <9804bcba74b1e76743ddf56cc5b4b80a@ruby-forum.com> > acts_as_ferret({ :fields => { > :nom => {}, > :description => {:boost => 0}, > :logiciel_nom => {}, > :logiciel_id => {}, > :difficulte_id => {}, > :systeme_nom => {}, > :fai_nom => {}, > :fai_id =>{}, > :lversion_id=>{}, > :site_nom => {}, > :siteutilise_id => {}, > :nom_for_sort => {:index => :untokenized}, > :note => {:index => :untokenized}, > :visions_count => {:index => :untokenized}, > :nb_vu => {:index => :untokenized}, > :date_sort => {:index => :untokenized} > }}, :analyzer => FrenchStemmingAnalyzer.new) The syntax for the acts_as_ferret options are a bit odd, the above code should be the following, notice the extra {} brackets around the :analyzer => FrenchStemmingAnalyzer.new acts_as_ferret({ :fields => { :nom => {}, :description => {:boost => 0}, :logiciel_nom => {}, :logiciel_id => {}, :difficulte_id => {}, :systeme_nom => {}, :fai_nom => {}, :fai_id =>{}, :lversion_id=>{}, :site_nom => {}, :siteutilise_id => {}, :nom_for_sort => {:index => :untokenized}, :note => {:index => :untokenized}, :visions_count => {:index => :untokenized}, :nb_vu => {:index => :untokenized}, :date_sort => {:index => :untokenized} }}, { :analyzer => FrenchStemmingAnalyzer.new }) -- Posted via http://www.ruby-forum.com/.