[Blacklight-development] changing solrmarc indexing in demo app?

Jonathan Rochkind rochkind at jhu.edu
Tue Jun 23 13:50:16 EDT 2009 

Sweet, that's very helpful. I'll work on debugging it myself tomorrow 
too, with those hints about what's going on.

Jamie Orchard-Hays wrote:
> OK. I just downloaded 2.2, untarred and tried out rake solr:marc:index.
>
> If I'm in the rails dir, it fails due to errors (the path is wrong).
>
> If I'm in the blacklight plugin dir, it works, because the path is
> right.
>
> The way to fix it is to have it use the __FILE__ location to orient
> the path properly.
>
> Except, that IS what the rake task does. So somehow when rake loads up
> all the tasks, it's not referring to the actual __FILE__ the task is
> created in. This is not the behavior I expect from Ruby. My
> understanding is that it should use that particular file is the
> reference.
>
> I'm perplexed.
>
>
> On Jun 23, 2009, at 1:03 PM, Jonathan Rochkind wrote:
>
>   
>> I'd really like to help get things working so a new user can just
>> use the rake task included there out-of-the-box for a simple demo.
>>
>> Like a new user _can_ use "rake app:index:marc" right now, with the
>> newly checked out demo app, which is what the current instructions
>> say to use. But apparently that's not actually using SolrMarc, and
>> people think that task should be removed?
>>
>> In which case, the instructions which right now suggest that after
>> installing the demo app you should run "rake app:index:marc FILE=../
>> data/lc_records.utf8.mrc", should be changed to suggest you run rake
>> solr:marc:index instead, right?  I really think it's good to have
>> this available (and am happy to help make it so).
>>
>> But right now, if you try running sorl:marc:index, you get errors.
>> I'd really like to help fix things so this isn't so, so the
>> instructions can say "And kick off the rake task to index some test
>> data:  rake solr:marc:index MARC_RECORDS_PATH=../data/
>> test_data.utf8.mrc".
>>
>> That would be good, right?  Is it possible for me to help make it
>> so?  That's a much easier learning curve (for me and those that come
>> after me) to be able to start like this, then to say "Now, to index
>> some data, go figure out how to install and use SolrMarc."
>>
>> Jonathan
>>
>> Naomi Dushay wrote:
>>     
>>> Hi Jonathan,
>>>
>>> Oy, this is less obvious that we would have hoped.  More
>>> documentation
>>> is definitely indicated.
>>>
>>> The demo index is basically intended as
>>>
>>> 1.  a way for the plugin to have standalone tests, some of which need
>>> a SOLR index to run.
>>> 2.  an exemplar.
>>>
>>> It's the 2.  that's sub-optimal.
>>>
>>> We use solrmarc completely separately from the blacklight code to
>>> generate our index.  Then we set up rails/config/solr.yml  to point
>>> to
>>> the SOLR instance with the index we created.
>>>
>>> To do anything non-trivial with solrmarc probably implies you want to
>>> create a separate build process from solrmarc.  You can certainly
>>> base
>>> it on the demo app ... but that's a very simple index that doesn't
>>> exploit much of the MARC data richness.
>>>
>>> I have it in my queue to flesh out the demo index more, so it's a
>>> better exemplar for MARC data.  But I haven't gotten to it yet.  And
>>> your notes make it clear that we have to cope with BOTH intentions
>>> above ... perhaps separately, or at least clearly documented.
>>>
>>> My apologies for this being so ... messy.  Your experience, while
>>> frustrating for you, is very helpful to the project.
>>>
>>> And ... my recommendation is that you may want to use solrmarc
>>> independently, as UVa and Stanford do.  You might start with the
>>> simple SOLR index fields the demo app uses, but populated from your
>>> data.  However, you are likely to add a lot more fields to your
>>> index.
>>>
>>> You can check out http://searchworks-test.stanford.edu  to see what
>>> our current blacklight efforts look like.  You'll immediately notice
>>> how far we've strayed from the demo index.  And if you saw the
>>> complexity of our search boosting formulae ... oy!
>>>
>>> - Naomi
>>>
>>> On Jun 23, 2009, at 9:28 AM, Jonathan Rochkind wrote:
>>>
>>>
>>>       
>>>> Huh.  But the default locations in there are NOT the (current) demo
>>>> app locations.  So it sounds like they were intended to be, and I
>>>> can change them to be and submit a patch?
>>>>
>>>> I was going to ask Jamie if he wanted to have a chat in #blacklight
>>>> on the right way to get the rake task working (in a trivial case)
>>>> with the out-of-the-box demo app, but if he's not working on it
>>>> right now...
>>>>
>>>> I'll just do my best that seems to make sense to me, and submit a
>>>> patch?  Jamie, there's no particular reason for the defaults chosen
>>>> except that they are in fact supposed to match the demo app? I'll
>>>> make it so.
>>>>
>>>> Jamie Orchard-Hays wrote:
>>>>
>>>>         
>>>>> Sigh. You're right. I did that because I wanted some default in
>>>>> there
>>>>> and used the demo app locations. You all may need to revisit it.
>>>>> Unfortunately, I'm not actively on this project right now.
>>>>>
>>>>> Jamie
>>>>>
>>>>> On Jun 23, 2009, at 11:37 AM, Jonathan Rochkind wrote:
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> Heh, but the defaults for solr:marc:index are in general not
>>>>>> what's
>>>>>> in the demo app.
>>>>>>
>>>>>> rake solr:marc:index MARC_RECORDS_PATH=../data/lc_records.utf8.mrc
>>>>>> INFO [main] (MarcImporter.java:516) - Starting SolrMarc indexing.
>>>>>> Error: Supplied Solr home directory does not exist: ../../../../
>>>>>> jetty/solr
>>>>>>
>>>>>> I'm actually not sure what arg to give solr:marc:index to fix
>>>>>> where
>>>>>> it's looking for Solr home directory.  But I can poke around in
>>>>>> the
>>>>>> code and figure it out.
>>>>>>
>>>>>> It seems to me that the defaults for solr:marc:index should match
>>>>>> what you get in the demo app though, yes?  This can be changed by
>>>>>> changing the defaults, OR by changing the setup of the 'demo' app.
>>>>>> Which would you prefer? (I'd lean toward changing the defaults,
>>>>>> because once you've checked out the demo app and put it into your
>>>>>> local svn, you aren't going to get any future changes to it; but
>>>>>> there might be a reason the defaults are what they are currently).
>>>>>>
>>>>>> I am more than happy to make these changes, submit patches, and
>>>>>> write documentation. Just need guidance on what way you guys
>>>>>> prefer
>>>>>> it to go, so I don't spend time doing something you don't like and
>>>>>> don't accept.
>>>>>>
>>>>>> Jonathan
>>>>>>
>>>>>>
>>>>>>             
>>>>> _______________________________________________
>>>>> Blacklight-development mailing list
>>>>> Blacklight-development at rubyforge.org
>>>>> http://rubyforge.org/mailman/listinfo/blacklight-development
>>>>> Blacklightopac Blog http://blacklightopac.org/
>>>>>
>>>>>
>>>>>           
>>>> _______________________________________________
>>>> Blacklight-development mailing list
>>>> Blacklight-development at rubyforge.org
>>>> http://rubyforge.org/mailman/listinfo/blacklight-development
>>>> Blacklightopac Blog http://blacklightopac.org/
>>>>
>>>>         
>>> _______________________________________________
>>> Blacklight-development mailing list
>>> Blacklight-development at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/blacklight-development
>>> Blacklightopac Blog http://blacklightopac.org/
>>>
>>>       
>> _______________________________________________
>> Blacklight-development mailing list
>> Blacklight-development at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/blacklight-development
>> Blacklightopac Blog http://blacklightopac.org/
>>     
>
> _______________________________________________
> Blacklight-development mailing list
> Blacklight-development at rubyforge.org
> http://rubyforge.org/mailman/listinfo/blacklight-development
> Blacklightopac Blog http://blacklightopac.org/
>

More information about the Blacklight-development mailing list