From nne at none.com Fri Dec 1 00:38:39 2006 From: nne at none.com (koloa) Date: Fri, 1 Dec 2006 06:38:39 +0100 Subject: [Ferret-talk] acts_as_ferret: can i specify a search on 1 field as sup In-Reply-To: <7580122dba1a4b76f2935244aa157bf7@ruby-forum.com> References: <7580122dba1a4b76f2935244aa157bf7@ruby-forum.com> Message-ID: hello, i would like to use find_by_contents, have it search the table for things that match my query, only if a column field is a certain value. for example i have an application for all the states in usa. If i am using the NJ search box, i would only like to limit my searches for where the state name is equal to new jersey. can i do this? from the above example it seems like i am looking for something like this? Model.find_by_contents(state:'nj'+"+statesearchstring") where state is the name of the column in my DB. thanks, -- Posted via http://www.ruby-forum.com/. From nne at none.com Fri Dec 1 00:46:44 2006 From: nne at none.com (koloa) Date: Fri, 1 Dec 2006 06:46:44 +0100 Subject: [Ferret-talk] acts_as_ferret: can i specify a search on 1 field as sup In-Reply-To: References: <7580122dba1a4b76f2935244aa157bf7@ruby-forum.com> Message-ID: maybe something like find_by_contents(searchstring +" "+ state:selectedstate) -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Dec 1 04:46:06 2006 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 1 Dec 2006 10:46:06 +0100 Subject: [Ferret-talk] acts_as_ferret: can i specify a search on 1 field as sup In-Reply-To: References: <7580122dba1a4b76f2935244aa157bf7@ruby-forum.com> Message-ID: <20061201094606.GS7968@cordoba.webit.de> On Fri, Dec 01, 2006 at 06:46:44AM +0100, koloa wrote: > maybe something like > > find_by_contents(searchstring +" "+ state:selectedstate) make that find_by_contents("#{searchstring} +state:#{selectedstate}") and it should work. if you store the state acronym in the state field, it might be a good idea to give :index => :untokenized as an option for this field in your call to acts_as_ferret: acts_as_ferret :fields => { :content => {}, :state => { :index => :untokenized }, ... any other fields you want to have indexed } this works great for i.e. having a select box to let the user select a state he wants to search in. Generally, not tokenizing a field is a good thing whenever you expect exact matches on a field, or want to sort your results using this field. You can have a look at the Ferret API docs (class FieldInfo) for more options and their explanations. cheers, Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From caverbruggen at gmail.com Fri Dec 1 04:46:23 2006 From: caverbruggen at gmail.com (=?utf-8?Q?Corn=c3=a9_Verbruggen?=) Date: Fri, 1 Dec 2006 10:46:23 +0100 Subject: [Ferret-talk] more_like_this not working? Message-ID: Hi all, using ferret 0.10.13 and aaf trunk, i cannot get more_like_this to work. I read other posts on this forum related to that, but that did not solve it. I have a model Question with a field question_text. I do: ----------------------------------------------- >> q = Question.find(:first) >> q.more_like_this NoMethodError: You have a nil object when you didn't expect it! You might have expected an instance of Array. The error occured while evaluating nil.each from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/more_like_this.rb:110:in `retrieve_terms' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/more_like_this.rb:65:in `more_like_this' from /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/more_like_this.rb:61:in `more_like_this' from (irb):8 >> q.more_like_this NoMethodError: You have a nil object when you didn't expect it! You might have expected an instance of Array. The error occured while evaluating nil.each from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/more_like_this.rb:110:in `retrieve_terms' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/more_like_this.rb:65:in `more_like_this' from /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/more_like_this.rb:61:in `more_like_this' from (irb):9 >> q.more_like_this(:fields => [:question_text]) NoMethodError: You have a nil object when you didn't expect it! You might have expected an instance of Array. The error occured while evaluating nil.each from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/more_like_this.rb:110:in `retrieve_terms' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/more_like_this.rb:65:in `more_like_this' from /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize' from ./script/../config/../config/../vendor/plugins/acts_as_ferret/lib/more_like_this.rb:61:in `more_like_this' from (irb):8 ----------------------------------------------- Am I doing something wrong or is this a bug? -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Dec 1 05:22:56 2006 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 1 Dec 2006 11:22:56 +0100 Subject: [Ferret-talk] usage and benefits of single-index with AAF In-Reply-To: <03293525-F6B0-4BA5-B268-A06AD2965C75@digitalpulp.com> References: <03293525-F6B0-4BA5-B268-A06AD2965C75@digitalpulp.com> Message-ID: <20061201102256.GT7968@cordoba.webit.de> On Thu, Nov 30, 2006 at 12:50:32PM -0500, John Bachir wrote: [..] > > [1] If I'm reading the code correctly, it seems that single-model > searches will behave the same as before, and AAF/Ferret will add the > extra column to the query for me. Is this correct? yep. > [2] How can I take advantage of the single-index when doing multi- > model searches? Through which model will I perform the query? Or do I > need to do this with raw Ferret queries and not through AAF? there's a secret option called :models to find_by_contents, which you can use to specify any additional models you want to search in. I just committed a fix adding that piece of documentation, and a fix to set the ferret_score attribute on results when using single_index. > [3] Are there other advantages or gotchas that I'm missing? generally the whole single_index thing seems to be less commonly used and therefore is less tested in the wild. Also with single_index each object in a result set is fetched with a single call to find, which clearly is not acceptable if a search returns a serious amount of results. Patches welcome ;-) I'd normally suggest using the multi-index search feature as an alternative to single_index (it's possible to search across multiple indexes at once with the multi_search method), but unfortunately this yields segfaults with recent Ferret versions (i think >= 0.10.10). cheers, Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From kraemer at webit.de Fri Dec 1 05:26:28 2006 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 1 Dec 2006 11:26:28 +0100 Subject: [Ferret-talk] more_like_this not working? In-Reply-To: References: Message-ID: <20061201102628.GU7968@cordoba.webit.de> On Fri, Dec 01, 2006 at 10:46:23AM +0100, Corn? Verbruggen wrote: > Hi all, > > using ferret 0.10.13 and aaf trunk, i cannot get more_like_this to work. > I read other posts on this forum related to that, but that did not solve > it. > > I have a model Question with a field question_text. I do: > > ----------------------------------------------- > >> q = Question.find(:first) > > >> q.more_like_this you'll have to give an array of field names to use for the similarity search: q.more_like_this :field_names => [ :question_text ] Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From neongrau at gmail.com Fri Dec 1 05:42:15 2006 From: neongrau at gmail.com (Ralf Vitasek) Date: Fri, 1 Dec 2006 11:42:15 +0100 Subject: [Ferret-talk] cannot use acts_as_ferret on legacy tables or tables with a Message-ID: <9ec3f6b4333cb916723d6186c4c9ee42@ruby-forum.com> when i used "find_by_contents" on a legacy table with a different primary_key than "id" i always got results=nil even when total_hits was > 0 as a quick fix i changed line 261 in "class_methods.rb" from: conditions = [ "#{self.table_name}.id in (?)", id_array ] to: conditions = [ "#{self.table_name}.#{self.primary_key} in (?)", id_array ] i tried to report that as a ticket on projects.jkraemer.net/acts_as_ferret/ but something is broken there :( -> internal error "Akismet rejected spam" -- Posted via http://www.ruby-forum.com/. From caverbruggen at gmail.com Fri Dec 1 05:52:16 2006 From: caverbruggen at gmail.com (=?utf-8?Q?Corn=c3=a9_Verbruggen?=) Date: Fri, 1 Dec 2006 11:52:16 +0100 Subject: [Ferret-talk] more_like_this not working? In-Reply-To: <20061201102628.GU7968@cordoba.webit.de> References: <20061201102628.GU7968@cordoba.webit.de> Message-ID: Jens Kraemer wrote: > On Fri, Dec 01, 2006 at 10:46:23AM +0100, Corn? Verbruggen wrote: >> >> >> q.more_like_this > > you'll have to give an array of field names to use for the similarity > search: > > q.more_like_this :field_names => [ :question_text ] > > Jens > Thanks! -- Posted via http://www.ruby-forum.com/. From koloa at hawaii.com Fri Dec 1 06:06:52 2006 From: koloa at hawaii.com (koloa) Date: Fri, 1 Dec 2006 12:06:52 +0100 Subject: [Ferret-talk] acts_as_ferret: can i specify a search on 1 field as sup In-Reply-To: <20061201094606.GS7968@cordoba.webit.de> References: <7580122dba1a4b76f2935244aa157bf7@ruby-forum.com> <20061201094606.GS7968@cordoba.webit.de> Message-ID: <1d143457d3162a0c056833aa65a9847c@ruby-forum.com> Hi Jens Thank you! -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Fri Dec 1 08:20:24 2006 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 1 Dec 2006 14:20:24 +0100 Subject: [Ferret-talk] cannot use acts_as_ferret on legacy tables or tables with a In-Reply-To: <9ec3f6b4333cb916723d6186c4c9ee42@ruby-forum.com> References: <9ec3f6b4333cb916723d6186c4c9ee42@ruby-forum.com> Message-ID: <20061201132024.GV7968@cordoba.webit.de> On Fri, Dec 01, 2006 at 11:42:15AM +0100, Ralf Vitasek wrote: > when i used "find_by_contents" on a legacy table with a different > primary_key than "id" i always got results=nil even when total_hits was > > 0 > > as a quick fix i changed line 261 in "class_methods.rb" > from: > conditions = [ "#{self.table_name}.id in (?)", id_array ] > > to: > conditions = [ "#{self.table_name}.#{self.primary_key} in (?)", > id_array ] this is already fixed in svn trunk. > i tried to report that as a ticket on > projects.jkraemer.net/acts_as_ferret/ but something is broken there :( > -> internal error "Akismet rejected spam" I know, something is broken with the akismet plugin. But without it, spam is even worse. still looking for a spam-proof wiki+issue-tracking solution... Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From henke at mac.se Fri Dec 1 08:52:31 2006 From: henke at mac.se (Henrik Zagerholm) Date: Fri, 1 Dec 2006 14:52:31 +0100 Subject: [Ferret-talk] Effective search when knowing ID Message-ID: Hi list, I'm new to ferret and I wonder what is the best way to search if I know one fields id and want to search on its content. I have experimented with FilteredQuery but I don't know if this is the best way. Eg. index = Index::Index.new(:path => '/tmp/all') @dbh = Database.instance.connect result = @dbh.exec("SELECT pk_fulltext_id, fulltext_text FROM tbl_fulltext") result.each do |row| #Inserting the PK id and text from DB index << {:pk_fulltext_id => row['pk_fulltext_id'], :content => row ['fulltext_text']} end content_query = Search::WildcardQuery.new(:content, "Text*") # I know the ID of the text to search in so I want to do it efficient id_filter = Search::QueryFilter.new(Search::TermQuery.new ('pk_fulltext_id', '1257667')) query = Search::FilteredQuery.new(content_query, id_filter) index.search_each(query) do |id, score| puts "Document #{id} found with a score of #{score}" highlights = index.highlight(content_query, id, :field => :content, :pre_tag => "\033[36m", :post_tag => "\033[m", :excerpt_length => 100) puts highlights end Please comment of this! Can I make my database PK id to be the ferret document ID?? Would that speed it up? Cheers, Henrik From henke at mac.se Fri Dec 1 10:24:44 2006 From: henke at mac.se (Henrik Zagerholm) Date: Fri, 1 Dec 2006 16:24:44 +0100 Subject: [Ferret-talk] Specify doc_id Message-ID: Hi list, I wonder if it is possible to specify my own doc_id? I want to have the same doc_id as my fulltexts tables primary keys. Cheers, Henrik From kraemer at webit.de Fri Dec 1 10:41:02 2006 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 1 Dec 2006 16:41:02 +0100 Subject: [Ferret-talk] Effective search when knowing ID In-Reply-To: References: Message-ID: <20061201154102.GX7968@cordoba.webit.de> On Fri, Dec 01, 2006 at 02:52:31PM +0100, Henrik Zagerholm wrote: > Hi list, > > I'm new to ferret and I wonder what is the best way to search if I > know one fields id and want to search on its content. > I have experimented with FilteredQuery but I don't know if this is > the best way. interesting question, but I'm afraid I can't give a definite answer. I'd guess that all you can do is try out what is faster - the query filter or an added boolean clause in your query. In general I wouldn't expect any speed difference in a query matching a single document issued with or without an additional clause matching the id field of that document. There will be some speed improvement if your query matches more documents, but mainly because you narrow the result set that Ferret has to return to down to one element, not because search is faster. Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From kraemer at webit.de Fri Dec 1 10:42:55 2006 From: kraemer at webit.de (Jens Kraemer) Date: Fri, 1 Dec 2006 16:42:55 +0100 Subject: [Ferret-talk] Specify doc_id In-Reply-To: References: Message-ID: <20061201154255.GY7968@cordoba.webit.de> On Fri, Dec 01, 2006 at 04:24:44PM +0100, Henrik Zagerholm wrote: > Hi list, > > I wonder if it is possible to specify my own doc_id? > I want to have the same doc_id as my fulltexts tables primary keys. no, that's not possible. the doc_id of documents may change over time (as deletions and index optimization take place) and is only guaranteed to be consistent if used with the IndexReader instance that first retrieved it. Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From pfitzpat72 at msn.com Fri Dec 1 12:28:45 2006 From: pfitzpat72 at msn.com (Pat Fitzpatrick) Date: Fri, 1 Dec 2006 18:28:45 +0100 Subject: [Ferret-talk] Ferret on Apache Installation Message-ID: <76e7dca6ad1422c1fba939046a67fa71@ruby-forum.com> Hi, had my local linux box setup and ferret was working like a champ via Webrick. However, when I moved to a different linux box using Apache web server, search no worky..giving the error listed below. The site itself comes up fine. I've tried deleting all the dirs under /index and restarted the webserver to see if re-indexing would help, but that made matters worse. Re-indexing didn't work ie didn't produce any of the normal subdirs under /index like it usually does. Is there some additional step that i need to do to Apache to get this working. Thanks Pat >>>>>>>>>>>> ERROR <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< uninitialized constant LockError RAILS_ROOT: /home/mysite/www/public/../config/.. Application Trace | Framework Trace | Full Trace /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:123:in `const_missing' /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:133:in `const_missing' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.13/lib/ferret/index.rb:674:in `ensure_reader_open' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.13/lib/ferret/index.rb:383:in `[]' /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.13/lib/ferret/index.rb:382:in `[]' #{RAILS_ROOT}/vendor/plugins/acts_as_ferret/lib/class_methods.rb:386:in `find_id_by_contents' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.13/lib/ferret/index.rb:371:in `search_each' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.13/lib/ferret/index.rb:370:in `search_each' /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.13/lib/ferret/index.rb:366:in `search_each' #{RAILS_ROOT}/vendor/plugins/acts_as_ferret/lib/class_methods.rb:384:in `find_id_by_contents' #{RAILS_ROOT}/vendor/plugins/acts_as_ferret/lib/class_methods.rb:248:in `find_by_contents' #{RAILS_ROOT}/app/models/event.rb:6:in `full_text_search' #{RAILS_ROOT}/app/controllers/page_controller.rb:140:in `search' This error occured while loading the following files: ferret/index/lock_error.rb -- Posted via http://www.ruby-forum.com/. From blah at blah.com Fri Dec 1 14:56:20 2006 From: blah at blah.com (Mark) Date: Fri, 1 Dec 2006 20:56:20 +0100 Subject: [Ferret-talk] Functional Testing AAF Plugin Message-ID: <2335ae5c9436e70abb01d07a7454b606@ruby-forum.com> Is there a way to reliably test the acts_as_ferret plugin using RoR functional tests? I load my fixtures and no matter what I do, the test won't return a result for what *should* be a query that returns results. Any help on this would be greatly appreciated. FIXTURE ******** article_001: id: 1 title: some valid article title body: This is a valid article entry with some images and stuff. FUNCTIONAL TEST ****************** def test_search get :search, :q => "some valid article title" assert_equal 1, assigns(:articles).size end MODEL ******* acts_as_ferret( { :fields => { 'title' => { :boost => 2 }, 'body' => { :boost => 1.5 }, 'ferret_index_article_comments' => { :boost => 1 }, 'ferret_index_article_image_captions' => { :boost => 1.5 } } }, { :analyzer => Ferret::Analysis::StandardAnalyzer.new([nil]) } ) -- Posted via http://www.ruby-forum.com/. From newsray at warmroom.com Fri Dec 1 17:43:43 2006 From: newsray at warmroom.com (Ray) Date: Fri, 1 Dec 2006 23:43:43 +0100 Subject: [Ferret-talk] Ferret on Apache Installation In-Reply-To: <76e7dca6ad1422c1fba939046a67fa71@ruby-forum.com> References: <76e7dca6ad1422c1fba939046a67fa71@ruby-forum.com> Message-ID: <230fc978247e5d73116f14047bb54fd2@ruby-forum.com> Pat Fitzpatrick wrote: > had my local linux box setup and ferret was working like a champ via > Webrick. > However, when I moved to a different linux box using Apache web server, > search no worky..giving the error listed below. The site itself comes up > fine. > I've tried deleting all the dirs under /index and restarted the > webserver to see if re-indexing would help, but that made matters worse. > Re-indexing didn't work ie didn't produce any of the normal subdirs > under /index like it usually does. It looks like you have a rails app. Does the user running rails have write permissions on the index directory? Make sure the index directory and subdirectories is owned by that user, or at least that that user has write permission (as well as read and execute). Ferrent is attempting to write a lock file in your index directory before it searches, and the write fails. The write fails in the c code under ensure_reader_open at lib/ferret/index.rb:673 (latest=@reader.latest?). The c code raises a LOCK_ERROR exception, but the exception is not caught because it is undefined in this scope (lib/ferret/index.rb:674). ActiveSupport attempts to load a non-existent file lock_error.rb to resolve this missing constant, but it fails You will get similar errors if you attempt to save a record. -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Sat Dec 2 05:43:07 2006 From: kraemer at webit.de (Jens Kraemer) Date: Sat, 2 Dec 2006 11:43:07 +0100 Subject: [Ferret-talk] Functional Testing AAF Plugin In-Reply-To: <2335ae5c9436e70abb01d07a7454b606@ruby-forum.com> References: <2335ae5c9436e70abb01d07a7454b606@ruby-forum.com> Message-ID: <20061202104307.GA6660@cordoba.webit.de> On Fri, Dec 01, 2006 at 08:56:20PM +0100, Mark wrote: > Is there a way to reliably test the acts_as_ferret plugin using RoR > functional tests? I load my fixtures and no matter what I do, the test > won't return a result for what *should* be a query that returns results. > Any help on this would be greatly appreciated. the problem is, that fixtures aren't loaded through active record, so no ferret indexing can takes place. You have to call Article.rebuild_index before running your tests (e.g. in setup), or create your test data through active record inside your test case. Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From blah at blah.com Sat Dec 2 12:11:07 2006 From: blah at blah.com (Mark) Date: Sat, 2 Dec 2006 18:11:07 +0100 Subject: [Ferret-talk] Functional Testing AAF Plugin In-Reply-To: <20061202104307.GA6660@cordoba.webit.de> References: <2335ae5c9436e70abb01d07a7454b606@ruby-forum.com> <20061202104307.GA6660@cordoba.webit.de> Message-ID: <9490e024943fdfc18bf1950c52eac0cf@ruby-forum.com> Jens Kraemer wrote: > On Fri, Dec 01, 2006 at 08:56:20PM +0100, Mark wrote: >> Is there a way to reliably test the acts_as_ferret plugin using RoR >> functional tests? I load my fixtures and no matter what I do, the test >> won't return a result for what *should* be a query that returns results. >> Any help on this would be greatly appreciated. > > the problem is, that fixtures aren't loaded through active record, so no > ferret indexing can takes place. You have to call > Article.rebuild_index before running your tests (e.g. in setup), or > create your test data through active record inside your test case. Right on Jens! That worked like a charm. I hadn't originally considered the fact that the fixtures aren't loaded through ActiveRecord. -- Posted via http://www.ruby-forum.com/. From blah at blah.com Sun Dec 3 12:39:27 2006 From: blah at blah.com (Mark) Date: Sun, 3 Dec 2006 18:39:27 +0100 Subject: [Ferret-talk] Functional Testing AAF Plugin In-Reply-To: <20061202104307.GA6660@cordoba.webit.de> References: <2335ae5c9436e70abb01d07a7454b606@ruby-forum.com> <20061202104307.GA6660@cordoba.webit.de> Message-ID: <7f903b011119061e3b817cf0e7eb729b@ruby-forum.com> Jens Kraemer wrote: > On Fri, Dec 01, 2006 at 08:56:20PM +0100, Mark wrote: >> Is there a way to reliably test the acts_as_ferret plugin using RoR >> functional tests? I load my fixtures and no matter what I do, the test >> won't return a result for what *should* be a query that returns results. >> Any help on this would be greatly appreciated. > > the problem is, that fixtures aren't loaded through active record, so no > ferret indexing can takes place. You have to call > Article.rebuild_index before running your tests (e.g. in setup), or > create your test data through active record inside your test case. Jens, is there a way to explicitly turn off ferret during testing except for the functional tests where I want to test it? -- Posted via http://www.ruby-forum.com/. From cswilliams at gmail.com Mon Dec 4 13:10:41 2006 From: cswilliams at gmail.com (Chris Williams) Date: Mon, 4 Dec 2006 19:10:41 +0100 Subject: [Ferret-talk] How to handle updates from other machine Message-ID: <8b86ce1c97807fc58a1dc5aa42a044da@ruby-forum.com> Hi, Right now I'm having ferret index about 1.5million documents. I plan on rebuilding the index on another machine and then copying the index over to the webserver. I read in one of the other posts that ferret has to reopen the index to see any new changes. Is there a way to script this? My frontend is a RoR app using AAF. Thanks, Chris -- Posted via http://www.ruby-forum.com/. From matt at mattschnitz.com Mon Dec 4 13:25:04 2006 From: matt at mattschnitz.com (Matt Schnitz) Date: Mon, 4 Dec 2006 10:25:04 -0800 Subject: [Ferret-talk] How to handle updates from other machine In-Reply-To: <8b86ce1c97807fc58a1dc5aa42a044da@ruby-forum.com> References: <8b86ce1c97807fc58a1dc5aa42a044da@ruby-forum.com> Message-ID: <497cc4a0612041025q47a0047bt64e2c13237c2984c@mail.gmail.com> A related question (apologies to Chris for jumping in on his question): Are there any issues we should be aware of when copying indexes between machines? Like, can we copy to and from any pair of machines? Or are there binary layout/word size/Endian issues? Schnitz On 12/4/06, Chris Williams wrote: > > Hi, > Right now I'm having ferret index about 1.5million documents. I plan on > rebuilding the index on another machine and then copying the index over > to the webserver. I read in one of the other posts that ferret has to > reopen the index to see any new changes. Is there a way to script this? > My frontend is a RoR app using AAF. > Thanks, > Chris > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20061204/374e169d/attachment-0001.html From kraemer at webit.de Tue Dec 5 06:20:39 2006 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 5 Dec 2006 12:20:39 +0100 Subject: [Ferret-talk] How to handle updates from other machine In-Reply-To: <8b86ce1c97807fc58a1dc5aa42a044da@ruby-forum.com> References: <8b86ce1c97807fc58a1dc5aa42a044da@ruby-forum.com> Message-ID: <20061205112039.GC7968@cordoba.webit.de> On Mon, Dec 04, 2006 at 07:10:41PM +0100, Chris Williams wrote: > Hi, > Right now I'm having ferret index about 1.5million documents. I plan on > rebuilding the index on another machine and then copying the index over > to the webserver. I read in one of the other posts that ferret has to > reopen the index to see any new changes. Is there a way to script this? > My frontend is a RoR app using AAF. you could build a 'secret' action in your app that triggers the reopening of the index. Or you check if your reader is reading from the current index before using it with IndexReader#is_latest? Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From kraemer at webit.de Tue Dec 5 07:40:27 2006 From: kraemer at webit.de (Jens Kraemer) Date: Tue, 5 Dec 2006 13:40:27 +0100 Subject: [Ferret-talk] Functional Testing AAF Plugin In-Reply-To: <7f903b011119061e3b817cf0e7eb729b@ruby-forum.com> References: <2335ae5c9436e70abb01d07a7454b606@ruby-forum.com> <20061202104307.GA6660@cordoba.webit.de> <7f903b011119061e3b817cf0e7eb729b@ruby-forum.com> Message-ID: <20061205124027.GF7968@cordoba.webit.de> On Sun, Dec 03, 2006 at 06:39:27PM +0100, Mark wrote: [..] > Jens, is there a way to explicitly turn off ferret during testing except > for the functional tests where I want to test it? No, none that I know of. But for sure it's possible to hack something together that disables the indexing under certain conditions. Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From nappin713 at yahoo.com Tue Dec 5 17:53:13 2006 From: nappin713 at yahoo.com (Raymond O'connor) Date: Tue, 5 Dec 2006 23:53:13 +0100 Subject: [Ferret-talk] Store Compressed Option Message-ID: <564a6c8dd2111f1920cfb5813a84ad16@ruby-forum.com> I noticed that store has a 'compressed' option, but was unsure what advantages/disadvantages are associated with it versus using the normal or 'yes' option. Does anyone know? Cheers! -- Posted via http://www.ruby-forum.com/. From bk at benjaminkrause.com Tue Dec 5 17:58:02 2006 From: bk at benjaminkrause.com (Benjamin Krause) Date: Tue, 5 Dec 2006 23:58:02 +0100 Subject: [Ferret-talk] Store Compressed Option In-Reply-To: <564a6c8dd2111f1920cfb5813a84ad16@ruby-forum.com> References: <564a6c8dd2111f1920cfb5813a84ad16@ruby-forum.com> Message-ID: On Dec 5, 2006, at 23:53, Raymond O'connor wrote: > I noticed that store has a 'compressed' option, but was unsure what > advantages/disadvantages are associated with it versus using the > normal > or 'yes' option. Does anyone know? Hey .. the compressed value is stored in the index, not the uncompressed (plain) .. keeps the index small, but you need cpu cycles to uncompress the data, if you access them or want to sort by them.. i guess it depends on the size of the data you want to store, typical queries, the size of your index and maybe the speed of your hard disk :) Ben From pfitzpatrick at clickhere.com Wed Dec 6 08:51:25 2006 From: pfitzpatrick at clickhere.com (Pat Fitzpatrick) Date: Wed, 6 Dec 2006 14:51:25 +0100 Subject: [Ferret-talk] Ferret on Apache Installation In-Reply-To: <230fc978247e5d73116f14047bb54fd2@ruby-forum.com> References: <76e7dca6ad1422c1fba939046a67fa71@ruby-forum.com> <230fc978247e5d73116f14047bb54fd2@ruby-forum.com> Message-ID: Ray wrote: > Pat Fitzpatrick wrote: > >> had my local linux box setup and ferret was working like a champ via >> Webrick. >> However, when I moved to a different linux box using Apache web server, >> search no worky..giving the error listed below. The site itself comes up >> fine. >> I've tried deleting all the dirs under /index and restarted the >> webserver to see if re-indexing would help, but that made matters worse. >> Re-indexing didn't work ie didn't produce any of the normal subdirs >> under /index like it usually does. > > > It looks like you have a rails app. Does the user running rails have > write permissions on the index directory? Make sure the index directory > and subdirectories is owned by that user, or at least that that user has > write permission (as well as read and execute). > > Ferrent is attempting to write a lock file in your index directory > before it searches, and the write fails. The write fails in the c code > under ensure_reader_open at lib/ferret/index.rb:673 > (latest=@reader.latest?). The c code raises a LOCK_ERROR exception, but > the exception is not caught because it is undefined in this scope > (lib/ferret/index.rb:674). ActiveSupport attempts to load a non-existent > file lock_error.rb to resolve this missing constant, but it fails > > You will get similar errors if you attempt to save a record. thx for the reply/interesting points Ray. Got it working by reinstalling mongrel and following documentation at: http://mongrel.rubyforge.org/docs/apache.html Also went through the deployment section in the 2nd edition Agile Web Development with Rails' by D Thomas. Well worth the buy and read. thx again. Probably helped by stepping away from the issue for a while and taking off the blinkers. -- Posted via http://www.ruby-forum.com/. From nappin713 at yahoo.com Wed Dec 6 15:22:07 2006 From: nappin713 at yahoo.com (Raymond O'connor) Date: Wed, 6 Dec 2006 21:22:07 +0100 Subject: [Ferret-talk] Stem Analyzer Message-ID: <6d381baeacc713e77c4db5629851b184@ruby-forum.com> Hi all, I am trying to implement a search that will use the Stem Analyzer. I added the Stem Anaylzer from the examples shown in another post http://ruby-forum.com/topic/80178#147014 module Ferret::Analysis class StemmingAnalyzer def token_stream(field, text) StemFilter.new(StandardTokenizer.new(text)) end end end The problem with the Stem analyzer is that when I search for a term such as 'engineering', it only matches whole words that fit the stem so the only results I get back are documents where 'engin' is a whole word (i.e. I don't get back documents with 'engineering', 'engineer', or 'engin*'). Am I using the Stem anaylzer correctly? Is there a better way to get the desired behavior? Any help would be much appreciated! Cheers! -- Posted via http://www.ruby-forum.com/. From matt at mattschnitz.com Wed Dec 6 16:04:45 2006 From: matt at mattschnitz.com (Matt Schnitz) Date: Wed, 6 Dec 2006 13:04:45 -0800 Subject: [Ferret-talk] Stem Analyzer In-Reply-To: <6d381baeacc713e77c4db5629851b184@ruby-forum.com> References: <6d381baeacc713e77c4db5629851b184@ruby-forum.com> Message-ID: <497cc4a0612061304l25979d72w9e07e2aef3f46a77@mail.gmail.com> You also need to stem-analyze the incoming query. I had this same problem. :^> Schnitz On 12/6/06, Raymond O'connor wrote: > > Hi all, > > I am trying to implement a search that will use the Stem Analyzer. I > added the Stem Anaylzer from the examples shown in another post > http://ruby-forum.com/topic/80178#147014 > > module Ferret::Analysis > class StemmingAnalyzer > def token_stream(field, text) > StemFilter.new(StandardTokenizer.new(text)) > end > end > end > > The problem with the Stem analyzer is that when I search for a term such > as 'engineering', it only matches whole words that fit the stem so the > only results I get back are documents where 'engin' is a whole word > (i.e. I don't get back documents with 'engineering', 'engineer', or > 'engin*'). Am I using the Stem anaylzer correctly? Is there a better > way to get the desired behavior? Any help would be much appreciated! > > Cheers! > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20061206/3b567aa4/attachment.html From nappin713 at yahoo.com Wed Dec 6 16:21:36 2006 From: nappin713 at yahoo.com (Raymond O'connor) Date: Wed, 6 Dec 2006 22:21:36 +0100 Subject: [Ferret-talk] Stem Analyzer In-Reply-To: <497cc4a0612061304l25979d72w9e07e2aef3f46a77@mail.gmail.com> References: <6d381baeacc713e77c4db5629851b184@ruby-forum.com> <497cc4a0612061304l25979d72w9e07e2aef3f46a77@mail.gmail.com> Message-ID: Matt Schnitz wrote: > You also need to stem-analyze the incoming query. > > I had this same problem. :^> > > > Schnitz Do you have an example of how to do this? I'm using AAF. Thanks, Ray -- Posted via http://www.ruby-forum.com/. From matt at mattschnitz.com Wed Dec 6 16:46:38 2006 From: matt at mattschnitz.com (Matt Schnitz) Date: Wed, 6 Dec 2006 13:46:38 -0800 Subject: [Ferret-talk] AAF - Stem Analyzer Message-ID: <497cc4a0612061346v1460cab4ld2de3b9331cef921@mail.gmail.com> I'm not on AAF. Can someone else help Raymond with an example? On 12/6/06, Raymond O'connor wrote: > > Matt Schnitz wrote: > > You also need to stem-analyze the incoming query. > > > > I had this same problem. :^> > > > > > > Schnitz > > Do you have an example of how to do this? I'm using AAF. > Thanks, > Ray > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20061206/3c3714a4/attachment.html From williammcguinty at yahoo.com Wed Dec 6 19:28:22 2006 From: williammcguinty at yahoo.com (William Mcguinty) Date: Thu, 7 Dec 2006 01:28:22 +0100 Subject: [Ferret-talk] Stem Analyzer In-Reply-To: References: <6d381baeacc713e77c4db5629851b184@ruby-forum.com> <497cc4a0612061304l25979d72w9e07e2aef3f46a77@mail.gmail.com> Message-ID: <714b0405ca731f28e2fe97668567b1b2@ruby-forum.com> Raymond O'connor wrote: > Matt Schnitz wrote: >> You also need to stem-analyze the incoming query. >> >> I had this same problem. :^> >> >> >> Schnitz > > Do you have an example of how to do this? I'm using AAF. > Thanks, > Ray I'm also using acts_as_ferret, and am stuck on exactly the same problem. :( Thanks in advance to anyone who can help on this. Will -- Posted via http://www.ruby-forum.com/. From andreas.korth at gmx.net Wed Dec 6 19:51:18 2006 From: andreas.korth at gmx.net (Andreas Korth) Date: Thu, 7 Dec 2006 01:51:18 +0100 Subject: [Ferret-talk] Stem Analyzer In-Reply-To: <714b0405ca731f28e2fe97668567b1b2@ruby-forum.com> References: <6d381baeacc713e77c4db5629851b184@ruby-forum.com> <497cc4a0612061304l25979d72w9e07e2aef3f46a77@mail.gmail.com> <714b0405ca731f28e2fe97668567b1b2@ruby-forum.com> Message-ID: On 07.12.2006, at 01:28, William Mcguinty wrote: > Raymond O'connor wrote: >> Matt Schnitz wrote: >>> You also need to stem-analyze the incoming query. >>> >>> I had this same problem. :^> >>> >>> >>> Schnitz >> >> Do you have an example of how to do this? I'm using AAF. >> Thanks, >> Ray > > I'm also using acts_as_ferret, and am stuck on exactly the same > problem. > :( How about passing your custom analyzer to acts_as_ferret as described in the docs? acts_as_ferret {:fields => ["id", "name", "body"]}, {:analyzer => MyFunkyStemAnalyzer} Cheers, Andy From blah at blah.com Wed Dec 6 22:07:06 2006 From: blah at blah.com (Mark) Date: Thu, 7 Dec 2006 04:07:06 +0100 Subject: [Ferret-talk] Search Multiple Models Message-ID: <55fedf5b9b311e6bef0635eaf64d46be@ruby-forum.com> Hello folks, I have four models, each with their own separate index. Models = Articles, Blogs, MusicTracks, and MediaFiles I have individual searches within each section of the site working just fine, but I want to have a gloabl search that searches across all of them at the same time and returns the results ordered by score. Here's how far I am now... def search query = params[:query] articles = Article.find_id_by_contents(query) blogs = Blog.find_id_by_contents(query) tracks = MusicTrack.find_id_by_contents(query) files = MediaFile.find_id_by_contents(query) matches = (articles + blogs + tracks + files).sort_by {|match| match[:score] } results = matches.collect {|match| :score => match[:score], :object => match[:model].find(match[:id])} end The block I'm passing isn't working and generates an exception at each =>. Can anyone give me some advice on what I might be doing wrong or if there is some better approach to doing this. It also seems that doing a .find on each id in the model is a little inefficient, but I'm not sure what else to try. Any help suggestions would be greatly appreciated. -- Posted via http://www.ruby-forum.com/. From nappin713 at yahoo.com Thu Dec 7 00:27:13 2006 From: nappin713 at yahoo.com (Raymond O'connor) Date: Thu, 7 Dec 2006 06:27:13 +0100 Subject: [Ferret-talk] Stem Analyzer In-Reply-To: References: <6d381baeacc713e77c4db5629851b184@ruby-forum.com> <497cc4a0612061304l25979d72w9e07e2aef3f46a77@mail.gmail.com> <714b0405ca731f28e2fe97668567b1b2@ruby-forum.com> Message-ID: <55edf2f5a6be44d8d9b61051295fe41c@ruby-forum.com> Well, you bring up another question I had. I was using a similar line as yours above to call the stem analyzer and I always would get a parse error. I even get a parse error when I paste your line in. I'm pretty new to ruby, and I'm sure its something obvious but I can't get rid of the parse error without removing the hash ticks such as acts_as_ferret :fields => ["id", "name", "body"], :analyzer => MyFunkyStemAnalyzer When I do that, AAF never seems to call my analyzer. So I ended up editing the AAF code and setting the analyzer option inside there and it worked except I get the whole problem I stated above. Thanks for your help, Ray -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Thu Dec 7 04:25:52 2006 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 7 Dec 2006 10:25:52 +0100 Subject: [Ferret-talk] Search Multiple Models In-Reply-To: <55fedf5b9b311e6bef0635eaf64d46be@ruby-forum.com> References: <55fedf5b9b311e6bef0635eaf64d46be@ruby-forum.com> Message-ID: <20061207092552.GB4076@cordoba.webit.de> On Thu, Dec 07, 2006 at 04:07:06AM +0100, Mark wrote: > Hello folks, > > I have four models, each with their own separate index. > > Models = Articles, Blogs, MusicTracks, and MediaFiles > > I have individual searches within each section of the site working just > fine, but I want to have a gloabl search that searches across all of > them at the same time and returns the results ordered by score. > > Here's how far I am now... > > def search > query = params[:query] > articles = Article.find_id_by_contents(query) > blogs = Blog.find_id_by_contents(query) > tracks = MusicTrack.find_id_by_contents(query) > files = MediaFile.find_id_by_contents(query) > matches = (articles + blogs + tracks + files).sort_by {|match| > match[:score] } > results = matches.collect {|match| :score => match[:score], :object > => match[:model].find(match[:id])} > end > > The block I'm passing isn't working and generates an exception at each > =>. Can anyone give me some advice on what I might be doing wrong or if > there is some better approach to doing this. It also seems that doing a > .find on each id in the model is a little inefficient, but I'm not sure > what else to try. you could retrieve the objects first with find_by_contents, and then sort by ferret_score (that's a method acts_as_ferret adds to your model, at least if you use the aaf trunk). But beware that the scores of instances of different models aren't comparable, so your sorting will be quite useless. In theory there's the multi_search method for exactly your scenario: results = Article.multi_search(query, [ Blog, MusicTrack, MediaFile ]) the big upside of this is that you get comparable scores and therefore a reasonable sorting by relevance. the downside is that it does a find on each id just as you do above. however the biggest problem with multi_search is that it tends to segfault with recent Ferret versions (I think 0.10.11 was the last version working). So if you want consistent sorting across models, and want to use the most recent Ferret, your best bet is to let all models share a single index with the :single_index option. cheers, Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From caleb at inforadical.net Thu Dec 7 04:38:56 2006 From: caleb at inforadical.net (Caleb Clausen) Date: Thu, 07 Dec 2006 01:38:56 -0800 Subject: [Ferret-talk] crash on repeated search Message-ID: <4577E130.8030003@inforadical.net> I have found another crash in ferret; this one just uses a regular search. It's similar to an issue reported by Matt Schnitz a while ago, but unlike his, mine does not go away if I turn off omit_norms. It does go away if I turn on the garbage collector more often, but I'm not sure that's a stable workaround under the circumstances. This one isn't a segfault, but more complicated versions have been. This bug manifests as a error in the garbage collector: $ ruby rw_ferret_test.rb .......rw_ferret_test.rb:20: [BUG] rb_gc_mark(): unknown data type 0x28(0x402ba3a4) non object ruby 1.8.2 (2005-04-11) [i386-linux] Aborted Here's the code: require 'rubygems' require 'ferret' include Ferret def phrase(num) "a_fairly_long_phrase%05d"%num end LOOPS=(ARGV[1]||40).to_i _fields = Index::FieldInfos.new _fields.add_field :phrase, :term_vector => :no, :store => :no, :index => :untokenized $phrases=I.new(:field_infos=>_fields) phrase(0).upto(phrase(LOOPS)){|phrase| $phrases << {:phrase=> phrase} } @phrases=Ferret::Search::Searcher.new $phrases.reader count=0 loop{ @phrases.search( Search::TermQuery.new(:phrase, "a_fairly_long_phrase%05d"%rand(LOOPS+1)) ) if ((count+=1)%1000).zero? #1309 print '.' STDOUT.flush # GC.start #enable this to workaround end } $phrases.close From andreas.korth at gmx.net Thu Dec 7 04:44:35 2006 From: andreas.korth at gmx.net (Andreas Korth) Date: Thu, 7 Dec 2006 10:44:35 +0100 Subject: [Ferret-talk] Stem Analyzer In-Reply-To: <55edf2f5a6be44d8d9b61051295fe41c@ruby-forum.com> References: <6d381baeacc713e77c4db5629851b184@ruby-forum.com> <497cc4a0612061304l25979d72w9e07e2aef3f46a77@mail.gmail.com> <714b0405ca731f28e2fe97668567b1b2@ruby-forum.com> <55edf2f5a6be44d8d9b61051295fe41c@ruby-forum.com> Message-ID: <3BBC8BAE-DB8D-4A36-BA63-0390987EC49E@gmx.net> On 07.12.2006, at 06:27, Raymond O'connor wrote: > Well, you bring up another question I had. I was using a similar line > as yours above to call the stem analyzer and I always would get a > parse > error. I even get a parse error when I paste your line in. I'm > pretty > new to ruby, and I'm sure its something obvious but I can't get rid of > the parse error without removing the hash ticks such as > acts_as_ferret :fields => ["id", "name", "body"], :analyzer => > MyFunkyStemAnalyzer > When I do that, AAF never seems to call my analyzer. So I ended up > editing the AAF code and setting the analyzer option inside there > and it > worked except I get the whole problem I stated above. This is the method signature as of the latest AAF rdoc: acts_as_ferret(options={}, ferret_options={}) It expects two hashes, defaulting to empty hashes if no arguments are supplied. In Ruby you can omit the parentheses for a method call and the curly braces for a hash argument if it is the last argument. A method with the signature my_method(options = {}) may be used in the following ways: my_method my_method({:key => "value"}) my_method(:key => "value") my_method {:key => "value"} my_method :key => "value" So the following call is supposed to work acts_as_ferret {:fields => ["id", "name", "body"]}, {:analyzer => MyFunkyStemAnalyzer} You could try parentheses acts_as_ferret({:fields => ["id", "name", "body"]}, {:analyzer => MyFunkyStemAnalyzer}) which should work in any case since it is the most explicit form. If you still get a parse error, you might want to post your actual code and the error message you get. HTH Andy From andreas.korth at gmx.net Thu Dec 7 05:00:55 2006 From: andreas.korth at gmx.net (Andreas Korth) Date: Thu, 7 Dec 2006 11:00:55 +0100 Subject: [Ferret-talk] crash on repeated search In-Reply-To: <4577E130.8030003@inforadical.net> References: <4577E130.8030003@inforadical.net> Message-ID: <207C9217-98BD-4AEF-B5CD-6539B0DBDB73@gmx.net> On 07.12.2006, at 10:38, Caleb Clausen wrote: > I have found another crash in ferret; this one just uses a regular > search. It's similar to an issue reported by Matt Schnitz a while ago, > but unlike his, mine does not go away if I turn off omit_norms. It > does > go away if I turn on the garbage collector more often, but I'm not > sure > that's a stable workaround under the circumstances. > > This one isn't a segfault, but more complicated versions have been. > This > bug manifests as a error in the garbage collector: > > $ ruby rw_ferret_test.rb > .......rw_ferret_test.rb:20: [BUG] rb_gc_mark(): unknown data type > 0x28(0x402ba3a4) non object > ruby 1.8.2 (2005-04-11) [i386-linux] > > Aborted I tried your code and I get lots of dots but no error ;) Maybe you should use a newer Ruby version. I'm using ruby 1.8.5 (2006-08-25) [powerpc-darwin8.8.0] Cheers, Andy From kraemer at webit.de Thu Dec 7 04:48:00 2006 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 7 Dec 2006 10:48:00 +0100 Subject: [Ferret-talk] Stem Analyzer In-Reply-To: <55edf2f5a6be44d8d9b61051295fe41c@ruby-forum.com> References: <6d381baeacc713e77c4db5629851b184@ruby-forum.com> <497cc4a0612061304l25979d72w9e07e2aef3f46a77@mail.gmail.com> <714b0405ca731f28e2fe97668567b1b2@ruby-forum.com> <55edf2f5a6be44d8d9b61051295fe41c@ruby-forum.com> Message-ID: <20061207094800.GE4076@cordoba.webit.de> On Thu, Dec 07, 2006 at 06:27:13AM +0100, Raymond O'connor wrote: > Well, you bring up another question I had. I was using a similar line > as yours above to call the stem analyzer and I always would get a parse > error. I even get a parse error when I paste your line in. I'm pretty > new to ruby, and I'm sure its something obvious but I can't get rid of > the parse error without removing the hash ticks such as > acts_as_ferret :fields => ["id", "name", "body"], :analyzer => > MyFunkyStemAnalyzer > When I do that, AAF never seems to call my analyzer. So I ended up > editing the AAF code and setting the analyzer option inside there and it > worked except I get the whole problem I stated above. correct syntax would be: acts_as_ferret( { :fields => [:id, :name, :body] }, :analyzer => MyFunkyStemAnalyzer.new) curly brackets are needed for the first options hash only, but may be placed around the second one (where the analyzer option belongs), too. the important thing to get over the parse error is to use () around the whole argument list. sorry, I think I forgot these subtle things when posing code in the past myself. Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From kraemer at webit.de Thu Dec 7 04:29:11 2006 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 7 Dec 2006 10:29:11 +0100 Subject: [Ferret-talk] AAF - Stem Analyzer In-Reply-To: <497cc4a0612061346v1460cab4ld2de3b9331cef921@mail.gmail.com> References: <497cc4a0612061346v1460cab4ld2de3b9331cef921@mail.gmail.com> Message-ID: <20061207092911.GC4076@cordoba.webit.de> On Wed, Dec 06, 2006 at 01:46:38PM -0800, Matt Schnitz wrote: > I'm not on AAF. Can someone else help Raymond with an example? > > On 12/6/06, Raymond O'connor wrote: > > > >Matt Schnitz wrote: > >> You also need to stem-analyze the incoming query. > >> > >> I had this same problem. :^> > >> > >> > >> Schnitz > > > >Do you have an example of how to do this? I'm using AAF. just specify your analyzer with the :analyzer option, i.e. acts_as_ferret :analyzer => GermanStemmingAnalyzer.new Here's what my GermanStemmingAnalyzer looks like: class GermanStemmingAnalyzer < Ferret::Analysis::Analyzer include Ferret::Analysis def initialize(stop_words = FULL_GERMAN_STOP_WORDS) @stop_words = stop_words end def token_stream(field, str) StemFilter.new(StopFilter.new(LowerCaseFilter.new(StandardTokenizer.new(str)), @stop_words), 'de') end end Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From nappin713 at yahoo.com Thu Dec 7 05:39:50 2006 From: nappin713 at yahoo.com (Raymond O'connor) Date: Thu, 7 Dec 2006 11:39:50 +0100 Subject: [Ferret-talk] Stem Analyzer In-Reply-To: <3BBC8BAE-DB8D-4A36-BA63-0390987EC49E@gmx.net> References: <6d381baeacc713e77c4db5629851b184@ruby-forum.com> <497cc4a0612061304l25979d72w9e07e2aef3f46a77@mail.gmail.com> <714b0405ca731f28e2fe97668567b1b2@ruby-forum.com> <55edf2f5a6be44d8d9b61051295fe41c@ruby-forum.com> <3BBC8BAE-DB8D-4A36-BA63-0390987EC49E@gmx.net> Message-ID: <71a63bceef5fba01d212cc9b6d0fb190@ruby-forum.com> Hi Jens, Thanks for the help! The parenthesis did the trick. Combined with the other post, I think I have all my issues ironed out... for now at least haha One other question: If I modify/add an analyzer does the index have to be rebuilt? I thought i noticed some searches not working with the stems analyzer until i rebuilt it, but then again its late for me and I maybe just seeing things ;-) -Ray -- Posted via http://www.ruby-forum.com/. From kraemer at webit.de Thu Dec 7 07:37:29 2006 From: kraemer at webit.de (Jens Kraemer) Date: Thu, 7 Dec 2006 13:37:29 +0100 Subject: [Ferret-talk] Stem Analyzer In-Reply-To: <71a63bceef5fba01d212cc9b6d0fb190@ruby-forum.com> References: <6d381baeacc713e77c4db5629851b184@ruby-forum.com> <497cc4a0612061304l25979d72w9e07e2aef3f46a77@mail.gmail.com> <714b0405ca731f28e2fe97668567b1b2@ruby-forum.com> <55edf2f5a6be44d8d9b61051295fe41c@ruby-forum.com> <3BBC8BAE-DB8D-4A36-BA63-0390987EC49E@gmx.net> <71a63bceef5fba01d212cc9b6d0fb190@ruby-forum.com> Message-ID: <20061207123729.GG4076@cordoba.webit.de> On Thu, Dec 07, 2006 at 11:39:50AM +0100, Raymond O'connor wrote: > Hi Jens, > Thanks for the help! The parenthesis did the trick. Combined with the > other post, I think I have all my issues ironed out... for now at least > haha > > > One other question: If I modify/add an analyzer does the index have to > be rebuilt? I thought i noticed some searches not working with the stems > analyzer until i rebuilt it, but then again its late for me and I maybe > just seeing things ;-) no, that's correct. Analysis is done when adding a document to the index. In addition to that, each query is run through the analyzer. So it's definitely needed to rebuild the index after changing the analyzer. Jens -- webit! Gesellschaft f?r neue Medien mbH www.webit.de Dipl.-Wirtschaftsingenieur Jens Kr?mer kraemer at webit.de Schnorrstra?e 76 Tel +49 351 46766 0 D-01069 Dresden Fax +49 351 46766 66 From pritchie at videotron.ca Thu Dec 7 08:56:00 2006 From: pritchie at videotron.ca (Patrick Ritchie) Date: Thu, 07 Dec 2006 08:56:00 -0500 Subject: [Ferret-talk] crash on repeated search In-Reply-To: <207C9217-98BD-4AEF-B5CD-6539B0DBDB73@gmx.net> References: <4577E130.8030003@inforadical.net> <207C9217-98BD-4AEF-B5CD-6539B0DBDB73@gmx.net> Message-ID: <45781D70.1020402@videotron.ca> Hi, > On 07.12.2006, at 10:38, Caleb Clausen wrote: > > >> I have found another crash in ferret; this one just uses a regular >> search. It's similar to an issue reported by Matt Schnitz a while ago, >> but unlike his, mine does not go away if I turn off omit_norms. It >> does >> go away if I turn on the garbage collector more often, but I'm not >> sure >> that's a stable workaround under the circumstances. >> >> This one isn't a segfault, but more complicated versions have been. >> This >> bug manifests as a error in the garbage collector: >> >> $ ruby rw_ferret_test.rb >> .......rw_ferret_test.rb:20: [BUG] rb_gc_mark(): unknown data type >> 0x28(0x402ba3a4) non object >> ruby 1.8.2 (2005-04-11) [i386-linux] >> >> Aborted >> > > I tried your code and I get lots of dots but no error ;) > > Maybe you should use a newer Ruby version. I'm using > > ruby 1.8.5 (2006-08-25) [powerpc-darwin8.8.0] I get the same crash on ruby 1.8.5 (2006-08-25) [i386-cygwin], perhaps it's specific to certain architectures? Cheers! Patrick -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20061207/95e397b8/attachment-0001.html From andreas.korth at gmx.net Thu Dec 7 09:39:10 2006 From: andreas.korth at gmx.net (Andreas Korth) Date: Thu, 7 Dec 2006 15:39:10 +0100 Subject: [Ferret-talk] crash on repeated search In-Reply-To: <45781D70.1020402@videotron.ca> References: <4577E130.8030003@inforadical.net> <207C9217-98BD-4AEF-B5CD-6539B0DBDB73@gmx.net> <45781D70.1020402@videotron.ca> Message-ID: On 07.12.2006, at 14:56, Patrick Ritchie wrote: >>> [BUG] rb_gc_mark(): unknown data type 0x28(0x402ba3a4) non object >>> ruby 1.8.2 (2005-04-11) [i386-linux] Aborted >> I tried your code and I get lots of dots but no error ;) Maybe you >> should use a newer Ruby version. I'm using ruby 1.8.5 (2006-08-25) >> [powerpc-darwin8.8.0] > I get the same crash on ruby 1.8.5 (2006-08-25) [i386-cygwin], > perhaps it's specific to certain architectures? Not sure if this is of any use, but I tried it on two other machines. Linux (Ubuntu): ................................test.rb:21: [BUG] rb_gc_mark(): unknown data type 0x0(0xb7d9e208) non object ruby 1.8.4 (2005-12-24) [i486-linux] Linux (Debian): ruby 1.8.5 (2006-08-25) [i486-linux] All test (including the first one on Mac OS X) were run with Ferret 0.10.13 Which version of Ferret are you using? The combination Ruby 1.8.5/ Ferret 0.10.13 seems to work best. -- Andy From bk at benjaminkrause.com Thu Dec 7 09:44:29 2006 From: bk at benjaminkrause.com (Benjamin Krause) Date: Thu, 7 Dec 2006 15:44:29 +0100 (CET) Subject: [Ferret-talk] crash on repeated search In-Reply-To: References: <4577E130.8030003@inforadical.net> <207C9217-98BD-4AEF-B5CD-6539B0DBDB73@gmx.net> <45781D70.1020402@videotron.ca> Message-ID: <43019.212.227.62.4.1165502669.squirrel@orkland.homeunix.org> > Which version of Ferret are you using? The combination Ruby 1.8.5/ > Ferret 0.10.13 seems to work best. 1.8.4 and 0.10.13 is fine, too :) for other test results take a look at http://camping.davebalmain.com/smoke_alarm/ Ben From andreas.korth at gmx.net Thu Dec 7 10:10:37 2006 From: andreas.korth at gmx.net (Andreas Korth) Date: Thu, 7 Dec 2006 16:10:37 +0100 Subject: [Ferret-talk] crash on repeated search In-Reply-To: <43019.212.227.62.4.1165502669.squirrel@orkland.homeunix.org> References: <4577E130.8030003@inforadical.net> <207C9217-98BD-4AEF-B5CD-6539B0DBDB73@gmx.net> <45781D70.1020402@videotron.ca> <43019.212.227.62.4.1165502669.squirrel@orkland.homeunix.org> Message-ID: Hi Ben! On 07.12.2006, at 15:44, Benjamin Krause wrote: >> Which version of Ferret are you using? The combination Ruby 1.8.5/ >> Ferret 0.10.13 seems to work best. > > 1.8.4 and 0.10.13 is fine, too :) Yes, I used that combo successfully on my staging system for quite some time. However, the test case that Caleb posted fails on that same machine. ................................test.rb:21: [BUG] rb_gc_mark(): unknown data type 0x0(0xb7d9e208) non object ruby 1.8.4 (2005-12-24) [i486-linux] Perhaps David should take a look at this. BTW, any news from him? Is he still missing in the outbacks down under? Cheers, Andreas From bk at benjaminkrause.com Thu Dec 7 10:20:00 2006 From: bk at benjaminkrause.com (Benjamin Krause) Date: Thu, 7 Dec 2006 16:20:00 +0100 (CET) Subject: [Ferret-talk] crash on repeated search In-Reply-To: References: <4577E130.8030003@inforadical.net> <207C9217-98BD-4AEF-B5CD-6539B0DBDB73@gmx.net> <45781D70.1020402@videotron.ca> <43019.212.227.62.4.1165502669.squirrel@orkland.homeunix.org> Message-ID: <43480.212.227.62.4.1165504800.squirrel@orkland.homeunix.org> > Perhaps David should take a look at this. BTW, any news from him? Is > he still missing in the outbacks down under? Hey .. Well, no news.. just some very short notices that he's still lost and without an internet connection.. If I hear anything i'll send it to this list. A bunch of things are left on Daves agenda and he wanted to implement a lot of great features this year, to finally reach a 1.0 status. But I guess we will need to wait a couple of more weeks until a new ferret release will be available. Ben From blah at blah.com Thu Dec 7 11:03:37 2006 From: blah at blah.com (Mark) Date: Thu, 7 Dec 2006 17:03:37 +0100 Subject: [Ferret-talk] Search Multiple Models In-Reply-To: <20061207092552.GB4076@cordoba.webit.de> References: <55fedf5b9b311e6bef0635eaf64d46be@ruby-forum.com> <20061207092552.GB4076@cordoba.webit.de> Message-ID: <3f56875e69fe9326122331c879dc5b99@ruby-forum.com> Jens, > In theory there's the multi_search method for exactly your scenario: > results = Article.multi_search(query, [ Blog, MusicTrack, MediaFile ]) > the big upside of this is that you get comparable scores and therefore a > reasonable sorting by relevance. the downside is that it does a find on > each id just as you do above. I'm tried the following, which semi-works: results =Blog.multi_search(@query, [Article, MediaFile, MusicTrack]) However, each on of those models has associations. Additionally, I need to pass :conditions statements (e.g., only find published articles). For instance, an article has comments, a MusicTrack has albums, genres, and artists. Right now, this works, but it is not pre-featching associations or using conditions. Is there a way to enable that like... results =Blog.multi_search(@query, [Article(:conditions => ['published = ?', yes], :include => [:article_comments], MediaFile, MusicTrack]) -- Posted via http://www.ruby-forum.com/. From cpjolicoeur at gmail.com Thu Dec 7 11:11:22 2006 From: cpjolicoeur at gmail.com (Craig Jolicoeur) Date: Thu, 7 Dec 2006 17:11:22 +0100 Subject: [Ferret-talk] Index location for multiple servers Message-ID: <098b876562c7c308bdea79bbd423c45c@ruby-forum.com> I have a rails app that is going to be deployed across several servers. In my understanding of acts_as_ferret, the index files are kept in the index folder in the root of the rails app. This won't work for multiple servers, of course, since all the servers will have their own directory tree. How are people using ferret for apps deployed on multiple servers? Could I run the index files through the database much like I do with session info to prevent this problem? Or is there a completely different work around that I am just not thinking of? -- Posted via http://www.ruby-forum.com/. From andreas.korth at gmx.net Thu Dec 7 11:20:51 2006 From: andreas.korth at gmx.net (Andreas Korth) Date: Thu, 7 Dec 2006 17:20:51 +0100 Subject: [Ferret-talk] Index location for multiple servers In-Reply-To: <098b876562c7c308bdea79bbd423c45c@ruby-forum.com> References: <098b876562c7c308bdea79bbd423c45c@ruby-forum.com> Message-ID: On 07.12.2006, at 17:11, Craig Jolicoeur wrote: > I have a rails app that is going to be deployed across several > servers. > > In my understanding of acts_as_ferret, the index files are kept in the > index folder in the root of the rails app. This won't work for > multiple > servers, of course, since all the servers will have their own > directory > tree. > > How are people using ferret for apps deployed on multiple servers? > Could I run the index files through the database much like I do with > session info to prevent this problem? Or is there a completely > different work around that I am just not thinking of? I haven't done this myself yet, but I think the best approach is to employ an index server. The Ferret index is kept on one machine which runs a DRb service. The other servers do not talk directly to Ferret but to the index server which performs the indexing/searching and returns the results as Ruby objects to the clients. I suppose this fit's best into Rails' share-nothing approach to scaling. -- Andy From bk at benjaminkrause.com Thu Dec 7 11:50:11 2006 From: bk at benjaminkrause.com (Benjamin Krause) Date: Thu, 7 Dec 2006 17:50:11 +0100 (CET) Subject: [Ferret-talk] Index location for multiple servers In-Reply-To: References: <098b876562c7c308bdea79bbd423c45c@ruby-forum.com> Message-ID: <44031.212.227.62.4.1165510211.squirrel@orkland.homeunix.org> Hey .. >> How are people using ferret for apps deployed on multiple servers? >> Could I run the index files through the database much like I do with >> session info to prevent this problem? Or is there a completely >> different work around that I am just not thinking of? > I haven't done this myself yet, but I think the best approach is to > employ an index server. The Ferret index is kept on one machine which > runs a DRb service. The other servers do not talk directly to Ferret > but to the index server which performs the indexing/searching and > returns the results as Ruby objects to the clients. actually that's the only way to do it.. you will run into severe problems if you want to modify the index from different severs on a shared filesystem.. at least with the current version of ferret. dave wanted to include support for a indexing server in ferret, having some sort of backgroundRb-alike mechanism, where you fire your indexing requests to a remote process/server/whatever .. Actually Jens and I will discuss this topic on Saturday, if we find a solution, we'll let you and the list know.. Ben From fox_qxnqrt at trashmail.net Thu Dec 7 12:19:41 2006 From: fox_qxnqrt at trashmail.net (damian) Date: Thu, 7 Dec 2006 18:19:41 +0100 Subject: [Ferret-talk] try Searchable ... Re: Index location for multiple servers In-Reply-To: <098b876562c7c308bdea79bbd423c45c@ruby-forum.com> References: <098b876562c7c308bdea79bbd423c45c@ruby-forum.com> Message-ID: <94ac5c3a36516e3514e36fef2e8ec062@ruby-forum.com> Craig Jolicoeur wrote: > I have a rails app that is going to be deployed across several servers. > > In my understanding of acts_as_ferret, the index files are kept in the > index folder in the root of the rails app. This won't work for multiple > servers, of course, since all the servers will have their own directory > tree. > > How are people using ferret for apps deployed on multiple servers? > Could I run the index files through the database much like I do with > session info to prevent this problem? Or is there a completely > different work around that I am just not thinking of? As others have already mentioned, you need to make sure the index writer is a singleton instance with all other servers as clients. It is a fun problem if you want to write it yourself and are familiar with DRb (quick to learn) and threading/locking issues. However if you don't want to spend the time to do that yourself I would highly suggest looking into the Searchable plugin which uses Ferret or Solr as search backends. It integrates nicely into your models using ActiveRecord hooks - much like aaf. The key feature in your case is that it also provides a DRb backend which you can turn on by a simple config switch, allowing you to have a central indexing server. more about this plugin at RubyForge: http://searchable.rubyforge.org/ It has been mentioned once before on this list but I think it deserves more exposure - especially as more people try to use Ferret for larger projects which require a multiple-server deployment. My company wrote our own DRb ferret implementation in house -- but that was before I saw the Searchable rails plugin ;) Good Luck, -damian -- Posted via http://www.ruby-forum.com/. From cpjolicoeur at gmail.com Thu Dec 7 12:29:47 2006 From: cpjolicoeur at gmail.com (Craig Jolicoeur) Date: Thu, 7 Dec 2006 18:29:47 +0100 Subject: [Ferret-talk] try Searchable ... Re: Index location for multiple servers In-Reply-To: <94ac5c3a36516e3514e36fef2e8ec062@ruby-forum.com> References: <098b876562c7c308bdea79bbd423c45c@ruby-forum.com> <94ac5c3a36516e3514e36fef2e8ec062@ruby-forum.com> Message-ID: damian wrote: > However if you don't want to spend the time to do that yourself I would > highly > suggest looking into the Searchable plugin which uses Ferret or Solr as > search backends. It integrates nicely into your models using > ActiveRecord > hooks - much like aaf. > > The key feature in your case is that it also provides a DRb backend > which you can turn on by a simple config switch, allowing you to > have a central indexing server. more about this plugin at RubyForge: > > http://searchable.rubyforge.org/ > > It has been mentioned once before on this list but I think it deserves > more exposure - especially as more people try to use Ferret for larger > projects which require a multiple-server deployment. My company wrote > our own DRb ferret implementation in house -- but that was before I saw > the Searchable rails plugin ;) > > Good Luck, > -damian Thanks Damian, I'm going to download and test out the searchable plugin. It seems as if it should be able to handle what I need, and if it does, you are right; it deserves more exposure. I had never heard of it before. -- Posted via http://www.ruby-forum.com/. From nappin713 at yahoo.com Thu Dec 7 13:19:56 2006 From: nappin713 at yahoo.com (Raymond O'connor) Date: Thu, 7 Dec 2006 19:19:56 +0100 Subject: [Ferret-talk] try Searchable ... Re: Index location for multiple servers In-Reply-To: References: <098b876562c7c308bdea79bbd423c45c@ruby-forum.com> <94ac5c3a36516e3514e36fef2e8ec062@ruby-forum.com> Message-ID: <7b4bb363b82e70eb17182bff0ad4575d@ruby-forum.com> I may be a special case since my index is never updated through my frontend app, but I was planning on keeping one "gold" index on a backend server and update this index through a script whenever there are updates to my documents (about once a week in my case). This server will update the index and then do basically a cp to the frontend webservers and copy over the old indexes on each of these machines. What do people think of this solution? The searchable drb solution looks interesting though and I may consider it if I run into issues doing it this way. -- Posted via http://www.ruby-forum.com/. From cswilliams at gmail.com Thu Dec 7 13:32:11 2006 From: cswilliams at gmail.com (Chris Williams) Date: Thu, 7 Dec 2006 19:32:11 +0100 Subject: [Ferret-talk] How to handle updates from other machine In-Reply-To: <20061205112039.GC7968@cordoba.webit.de> References: <8b86ce1c97807fc58a1dc5aa42a044da@ruby-forum.com> <20061205112039.GC7968@cordoba.webit.de> Message-ID: <2410dd39e2a20e11123f1baa0436e414@ruby-forum.com> I see. Doesn't AAF just reopen the index again on the next request though? I don't mind if updates aren't seen until the next request occurs... -- Posted via http://www.ruby-forum.com/. From andreas.korth at gmx.net Thu Dec 7 14:01:34 2006 From: andreas.korth at gmx.net (Andreas Korth) Date: Thu, 7 Dec 2006 20:01:34 +0100 Subject: [Ferret-talk] try Searchable ... Re: Index location for multiple servers In-Reply-To: <7b4bb363b82e70eb17182bff0ad4575d@ruby-forum.com> References: <098b876562c7c308bdea79bbd423c45c@ruby-forum.com> <94ac5c3a36516e3514e36fef2e8ec062@ruby-forum.com> <7b4bb363b82e70eb17182bff0ad4575d@ruby-forum.com> Message-ID: <13CF989F-C64E-43E4-84FA-381A3A497173@gmx.net> On 07.12.2006, at 19:19, Raymond O'connor wrote: > I may be a special case since my index is never updated through my > frontend app, but I was planning on keeping one "gold" index on a > backend server and update this index through a script whenever > there are > updates to my documents (about once a week in my case). This server > will update the index and then do basically a cp to the frontend > webservers and copy over the old indexes on each of these machines. > What do people think of this solution? The searchable drb solution > looks interesting though and I may consider it if I run into issues > doing it this way. If updates happen so rarely, this might be a feasible solution. At least, it involves less moving parts than an indexing server would and it's faster than querying a remote server. I'm not sure, however, if Ferret would like it if the index files are overwritten during a read operation. You could write a capistrano recipe which shuts down your app, updates the index files and restarts it afterwards. Depending on the size of your index, this involves downtimes of a few seconds which is certainly tolerable. -- Andy From blah at blah.com Thu Dec 7 14:34:13 2006 From: blah at blah.com (Mark) Date: Thu, 7 Dec 2006 20:34:13 +0100 Subject: [Ferret-talk] Search Multiple Models In-Reply-To: <3f56875e69fe9326122331c879dc5b99@ruby-forum.com> References: <55fedf5b9b311e6bef0635eaf64d46be@ruby-forum.com> <20061207092552.GB4076@cordoba.webit.de> <3f56875e69fe9326122331c879dc5b99@ruby-forum.com> Message-ID: <3542911c258948237a1c7b529bf92786@ruby-forum.com> Jens, One other thing. I can't seem to get the following working. options = {:limit => 10, :offset => 20} total_hits = Blog.id_multi_search(query, [], options) The result returned seems to ignore the offset for some reason and always returns the same resultset...am I doing something wrong? > Jens, > >> In theory there's the multi_search method for exactly your scenario: >> results = Article.multi_search(query, [ Blog, MusicTrack, MediaFile ]) >> the big upside of this is that you get comparable scores and therefore a >> reasonable sorting by relevance. the downside is that it does a find on >> each id just as you do above. > > I'm tried the following, which semi-works: > > results =Blog.multi_search(@query, [Article, MediaFile, MusicTrack]) > > However, each on of those models has associations. Additionally, I need > to pass :conditions statements (e.g., only find published articles). For > instance, an article has comments, a MusicTrack has albums, genres, and > artists. Right now, this works, but it is not pre-featching associations > or using conditions. Is there a way to enable that like... > > results =Blog.multi_search(@query, [Article(:conditions => ['published = > ?', yes], :include => [:article_comments], MediaFile, MusicTrack]) -- Posted via http://www.ruby-forum.com/. From blah at blah.com Thu Dec 7 14:50:21 2006 From: blah at blah.com (Mark) Date: Thu, 7 Dec 2006 20:50:21 +0100 Subject: [Ferret-talk] Search Multiple Models In-Reply-To: <3542911c258948237a1c7b529bf92786@ruby-forum.com> References: <55fedf5b9b311e6bef0635eaf64d46be@ruby-forum.com> <20061207092552.GB4076@cordoba.webit.de> <3f56875e69fe9326122331c879dc5b99@ruby-forum.com> <3542911c258948237a1c7b529bf92786@ruby-forum.com> Message-ID: <05186f25d42cad80e1da9831a0fca97a@ruby-forum.com> Jens, Answering my own question here after some more research...turns out this bug is still present. http://www.ruby-forum.com/topic/81987#148939 -- snip fix -- and I just found the culprit... Line 27 of multi_index.rb searcher.search_each(query, options={}, &block) should be searcher.search_each(query, options, &block) -- end snip -- I updated that and all worked fine again. -- Posted via http://www.ruby-forum.com/. From frank.blecha at gmail.com Thu Dec 7 15:12:08 2006 From: frank.blecha at gmail.com (Frank Blecha) Date: Thu, 7 Dec 2006 13:12:08 -0700 Subject: [Ferret-talk] Invalid character and win32 Message-ID: Hola, I've been having a bear of a time getting ferret (via aaf) working correctly on win32 (using ferret 0.10.9 win32). I am constantly getting either "invalid character" or "unexpected end" error messages from "normal" html in rhtml files. I am running ruby 1.8.5 (via 1-click installer) and get this error from both my functional tests and via mongrel. It seems very simliar to the thread at http://www.railsweenie.com/forums/1/topics/280?page=26 , but for all I know it's entirely unrelated. I have downgraded to 0.9.6and the aaf for that version, and don't experience these problems. However, that version seems to have problems with the queries I want to perform, so I'd like to run the newer versions (which when they don't get parse errors, work great). I assume this is more of a ferret issue I'm having instead of with aff, but if that's not the case, let me know. Being fairly new to both ferret and aaf, if this is the wrong list to submit this issue, just point me in the right direction. Any help is appreciated. - Frank -- Frank Blecha -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ferret-talk/attachments/20061207/b051bd48/attachment.html From jkahn_117 at yahoo.com Thu Dec 7 16:28:19 2006 From: jkahn_117 at yahoo.com (Joshua Kahn) Date: Thu, 7 Dec 2006 22:28:19 +0100 Subject: [Ferret-talk] Format dates prior to index Message-ID: <42e16c9a3aeedca6b2a6451786de8307@ruby-forum.com> Hi all- I would like to enable search in my application via Ferret and a number of my searchable classes contain dates (of type Time). When these classes are indexed, the dates are written per the standard to_s output (e.g. "Wed Dec 07 08:56:04 CDT 2006"), which is not suitable for date range queries. Is it possible to format the dates before they are indexed (e.g. "20061207") without writing additional methods that output the dates in this format? Thanks, Josh -- Posted via http://www.ruby-forum.com/. From andreas.korth at gmx.net Thu Dec 7 16:51:50 2006 From: andreas.korth at gmx.net (Andreas Korth) Date: Thu, 7 Dec 2006 22:51:50 +0100 Subject: [Ferret-talk] Format dates prior to index In-Reply-To: <42e16c9a3aeedca6b2a6451786de8307@ruby-forum.com> References: <42e16c9a3aeedca6b2a6451786de8307@ruby-forum.com> Message-ID: On 07.12.2006, at 22:28, Joshua Kahn wrote: > Hi all- > > I would like to enable search in my application via Ferret and a > number > of my searchable classes contain dates (of type Time). When these > classes are indexed, the dates are written per the standard to_s > output > (e.g. "Wed Dec 07 08:56:04 CDT 2006"), which is not suitable for date > range queries. Is it possible to format the dates before they are > indexed (e.g. "20061207") without writing additional methods that > output > the dates in this format? Is there something wrong with writing an additional method that does the conversion? If you don't need the extra precision of Time use Date. Its to_s method will give you "2006-12-07" right away. -- Andy From jkahn_117 at yahoo.com Thu Dec 7 16:54:59 2006 From: jkahn_117 at yahoo.com (Joshua Kahn) Date: Thu, 7 Dec 2006 22:54:59 +0100 Subject: [Ferret-talk] Format dates prior to index In-Reply-To: References: <42e16c9a3aeedca6b2a6451786de8307@ruby-forum.com> Message-ID: Nothing wrong, I was just hoping to avoid that. I will look into using Date. Thanks. -- Posted via http://www.ruby-forum.com/. From Neville.Burnell at bmsoft.com.au Thu Dec 7 17:52:53 2006 From: Neville.Burnell at bmsoft.com.au (Neville Burnell) Date: Fri, 8 Dec 2006 09:52:53 +1100 Subject: [Ferret-talk] Range Query Term parsing bug in 0.10.6 win32 ? Message-ID: <126EC586577FD611A28E00A0C9A03758B5BFDF@maui.bmsoft.com.au> Hi, I think I've found a Range Query Term parsing bug ... the following term should return names >= 'A', but instead generates a parsing error Term: name:[A> Message: Nil bounds for range. A range must include either lower bound or an upper bound However, the slightly larger term, name:[AA> works just fine. Any pointers please? Kind Regards Neville From blah at blah.com Thu Dec 7 20:32:04 2006 From: blah at blah.com (Mark) Date: Fri, 8 Dec 2006 02:32:04 +0100 Subject: [Ferret-talk] QueryParser Exception Handling Problem Message-ID: <0d9ca21646921fef5c353daf7d9a2735@ruby-forum.com> According to the following link: http://ferret.davebalmain.com/api/classes/Ferret/QueryParser.html :handle_parser_errors => true is enabled by default and seems to be in acts_as_ferret in class_methods.rb of the plugin. However, when I pass a special character as a query (e.g., !, -, <, >, etc) it throws an error and pukes. -- snip -- Error occured in src/q_parser.y:279 - yyerror couldn't parse query ``