From Isabelle.Julian at nrc-cnrc.gc.ca Tue Sep 23 12:32:41 2008 From: Isabelle.Julian at nrc-cnrc.gc.ca (Julian, Isabelle) Date: Tue, 23 Sep 2008 12:32:41 -0400 Subject: [Blacklight-development] Questions about Blacklight Message-ID: <509D651AE296014F98C513FA268B4D340630A8BD@nrccenexb1.nrc.ca> Hello, I have a few questions about Blacklight. 1. Blacklight can be displayed in what languages? Is French one of them? 2. What kind of ranking does it provide? 3. Can you give me specifics on proximity, range and fuzzy searching? 4. Is there a list of new features of the incoming release available somewhere? Thank you. Isabelle Julian M.A., MLIS Manager, e-Information Services / Gestionnaire, Services d'information ?lectronique | CISTI / ICIST isabelle.julian at nrc-cnrc.gc.ca tel/t?l: 613-993-1350| facsimile/t?l?copieur: 613-993-7619 National Research Council Canada | M-55, 1200 Montreal Road, Ottawa, Ontario K1A 0R6 Conseil national de recherches Canada | M-55, 1200 chemin Montr?al, Ottawa, Ontario K1A 0R6 Government of Canada | Gouvernement du Canada -------------- next part -------------- An HTML attachment was scrubbed... URL: From erikhatcher at mac.com Tue Sep 23 16:33:22 2008 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 23 Sep 2008 16:33:22 -0400 Subject: [Blacklight-development] Questions about Blacklight In-Reply-To: <509D651AE296014F98C513FA268B4D340630A8BD@nrccenexb1.nrc.ca> References: <509D651AE296014F98C513FA268B4D340630A8BD@nrccenexb1.nrc.ca> Message-ID: Isabelle, I'll reply with my (now ancient) experiences with Blacklight... On Sep 23, 2008, at 12:32 PM, Julian, Isabelle wrote: > 1. Blacklight can be displayed in what languages? Is French one > of them? > Yes, no problem, as it renders what is in the underlying data, UTF-8 and Unicode and all that willing. The original version of Blacklight and Flare (and Solr/Lucene even further down) worked very nicely with Chinese, Russian, etc. > 2. What kind of ranking does it provide? > Under the covers of Blacklight is Lucene, an open source search engine library. The relevancy is based on vector space TF/IDF factors, and quite malleable in tuning in a number of ways. Solr is a web service wrapper around Lucene, and provides a nice interface into a disjunction querying algorithm across a dynamically configurable sets of fields and boosts. Relevancy tuning is par for the course, and Lucene/Solr have proven quite capable in many cases to provide the knobs and dials to make users smile. > 3. Can you give me specifics on proximity, range and fuzzy > searching? > Lucene supports these query types. How the string a user types translates into these richer query types is the trick. But with Lucene's query parser, it is possible to get all of those types and more... anything you see here: and Proximity queries are created from "quoted phrases", and can be exact side-by-side matches or "sloppy phrases"~10 (with the number specifying a slop factor, regardless of order). Range queries - no problem there, just gotta recognize how terms got indexed and how range queries are expressed in query parser syntax, escaping rules and such. [inclusive TO ranges] and {exclusive TO ranges} work textually and also with dates (see the gory details in the links above). fuzzy~ terms use a Levenshtein distance algorithm, with some tolerance~0.8 support too. Fuzzy queries can be relatively expensive performance-wise, and need to be treated with care, but for moderate sized indexes it generally isn't too prohibitive. > 4. Is there a list of new features of the incoming release > available somewhere? > I'll let Bess & Co. fill in these blanks :) Erik From bess at virginia.edu Tue Sep 23 21:56:07 2008 From: bess at virginia.edu (Bess Sadler) Date: Tue, 23 Sep 2008 21:56:07 -0400 Subject: [Blacklight-development] Questions about Blacklight References: Message-ID: Weird... I sent this earlier, but it never showed up on the mailing list. Bess ************* Hi, Isabelle, Welcome to the blacklight list! 1. It sounds like you're asking about Blacklight's user interface. Currently, this only exists in English. However, it is an open source project, and I don't think it would be too much work to make it properly internationalized so that someone could create interface translations. If instead you're asking about whether it can handle French language *content*, yes, it does that already. 2. Blacklight uses the solr / lucene relevancy ranking. We customize our relevancy ranking algorithms according to what our users are asking for, and you would probably want to customize them locally for your own collection, too. 3. We use solr / lucene for all searching, and it does a very good job with complex queries. If you want more information about the kinds of queries that lucene can handle, I recommend the Lucene wiki: http://wiki.apache.org/lucene-java/FrontPage? action=show&redirect=FrontPageEN and the excellent book Lucene in Action. 4. Unfortunately we don't have a list of new features for the upcoming release. But in brief, we've re-worked the way relevancy ranking works, we've fleshed out the presentation quite a bit, we're re-working the graphic design, and we've upgraded the index to use solr 1.3. I hope this helps! Bess On Sep 23, 2008, at 12:32 PM, Julian, Isabelle wrote: > Hello, > > I have a few questions about Blacklight. > > 1. Blacklight can be displayed in what languages? Is French > one of them? > > 2. What kind of ranking does it provide? > > 3. Can you give me specifics on proximity, range and fuzzy > searching? > > 4. Is there a list of new features of the incoming release > available somewhere? > > > Thank you. > > > Isabelle Julian M.A., MLIS > > Manager, e-Information Services / Gestionnaire, Services > d'information ?lectronique | CISTI / ICIST isabelle.julian at nrc- > cnrc.gc.ca > > tel/t?l: 613-993-1350| facsimile/t?l?copieur: 613-993-7619 National > Research Council Canada | M-55, 1200 Montreal Road, Ottawa, Ontario > K1A 0R6 Conseil national de recherches Canada | M-55, 1200 chemin > Montr?al, Ottawa, Ontario K1A 0R6 Government of Canada | > Gouvernement du Canada > > > > From jamie at dang.com Tue Sep 23 22:46:15 2008 From: jamie at dang.com (Jamie Orchard-Hays) Date: Tue, 23 Sep 2008 22:46:15 -0400 Subject: [Blacklight-development] Questions about Blacklight In-Reply-To: References: Message-ID: The next version of Rails (which Blacklight uses) will make internationalization much easier: On Sep 23, 2008, at 9:56 PM, Bess Sadler wrote: > Weird... I sent this earlier, but it never showed up on the mailing > list. > > Bess > > ************* > > Hi, Isabelle, > > Welcome to the blacklight list! > > 1. It sounds like you're asking about Blacklight's user interface. > Currently, this only exists in English. However, it is an open source > project, and I don't think it would be too much work to make it > properly internationalized so that someone could create interface > translations. > > If instead you're asking about whether it can handle French language > *content*, yes, it does that already. > > 2. Blacklight uses the solr / lucene relevancy ranking. We customize > our relevancy ranking algorithms according to what our users are > asking for, and you would probably want to customize them locally for > your own collection, too. > > 3. We use solr / lucene for all searching, and it does a very good > job with complex queries. If you want more information about the > kinds of queries that lucene can handle, I recommend the Lucene wiki: > http://wiki.apache.org/lucene-java/FrontPage? > action=show&redirect=FrontPageEN and the excellent book Lucene in > Action. > > 4. Unfortunately we don't have a list of new features for the > upcoming release. But in brief, we've re-worked the way relevancy > ranking works, we've fleshed out the presentation quite a bit, we're > re-working the graphic design, and we've upgraded the index to use > solr 1.3. > > I hope this helps! > > Bess > > On Sep 23, 2008, at 12:32 PM, Julian, Isabelle wrote: > >> Hello, >> >> I have a few questions about Blacklight. >> >> 1. Blacklight can be displayed in what languages? Is French >> one of them? >> >> 2. What kind of ranking does it provide? >> >> 3. Can you give me specifics on proximity, range and fuzzy >> searching? >> >> 4. Is there a list of new features of the incoming release >> available somewhere? >> >> >> Thank you. >> >> >> Isabelle Julian M.A., MLIS >> >> Manager, e-Information Services / Gestionnaire, Services >> d'information ?lectronique | CISTI / ICIST isabelle.julian at nrc- >> cnrc.gc.ca >> >> tel/t?l: 613-993-1350| facsimile/t?l?copieur: 613-993-7619 National >> Research Council Canada | M-55, 1200 Montreal Road, Ottawa, Ontario >> K1A 0R6 Conseil national de recherches Canada | M-55, 1200 chemin >> Montr?al, Ottawa, Ontario K1A 0R6 Government of Canada | >> Gouvernement du Canada >> >> >> >> > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development >