<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>The java marc importer uses a different version of lucene than the .tar.gz of blacklight, and the index format is incompatible.</div><div><br></div><div>unpacking the solr.war, the lucene jar files are all named lucene-blah-2007-05-20</div><div><br></div><div>the lucene jars in the java importer are lucene version 2.3.1, which seem to date from 2008-02-22</div><div><br></div><div>I swapped out the old lucene jar files for the new ones and made a new solr.war. Putting my new solr.war into jetty, then firing up solr ... makes everything happy!</div><div><br></div><div>So, to summarize:</div><div><br></div><div>1. java importer: index.sh line breaks are not in (lin)ux format. (thanks Jonathan!)</div><div>2. java importer: sample file name in GettingStarted.txt has a typo (thanks Jonathan!)</div><div>3. java importer: sample data has a record with multiple 020 subfield a values. To get the data to index cleanly, you much change a line in blacklight.properties file:</div><div><br></div><div>from<span class="Apple-style-span" style="font-family: Monaco; font-size: 11px; "><span style="color: rgb(0, 0, 230); "></span></span></div><div><font class="Apple-style-span" color="#0000E6" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div><div><span class="Apple-style-span" style="font-family: Monaco; font-size: 11px; "><span style="color: rgb(0, 0, 230); ">field_list_25a</span> <span style="color: rgb(128, 128, 48); ">=</span> isbn<span style="color: rgb(128, 128, 48); ">_</span>display<span style="color: rgb(128, 128, 48); ">,</span> all<span style="color: rgb(128, 128, 48); ">,</span> 020a</span></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;">to</span></font></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><span class="Apple-style-span" style="font-family: Helvetica; font-size: 12px; "><div><span class="Apple-style-span" style="font-family: Monaco; font-size: 11px; "><span style="color: rgb(0, 0, 230); ">field_list_25a</span> <span style="color: rgb(128, 128, 48); ">=</span> isbn<span style="color: rgb(128, 128, 48); ">_</span>display<span style="color: rgb(128, 128, 48); ">,</span> first<span style="color: rgb(128, 128, 48); ">,</span> 020a</span></div><div><font class="Apple-style-span" face="Monaco" size="3"><span class="Apple-style-span" style="font-size: 11px;"><br></span></font></div></span></span></font></div><div>4. blacklight itself: solr.war in the jetty/webapps directory has an older version of lucene jars which is incompatible with the lucene jars in the marc importer. The jars need to be the same. I changed the jars in the solr.war by unpacking the war, substituting the new lucene jar files, then repacking the war and putting the new solr.war in jetty/webapps. It might also work to just put the lucene jars from the solr.war into the java importer lib, replacing the ones that are there.</div><div><br></div><div>Jonathan -- would you like to test these fixes on your system?</div><div><br></div><div>hope this helps!</div><div>- Naomi</div><div><br></div><div><br></div><div>On May 2, 2008, at 9:20 AM, Naomi Dushay wrote:</div><div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Hi Matt,</div><div><br></div><div>I got my blacklight code from the .tar.gz download file, not directly from svn. Do I need to pull some updates from the trunk?</div><div><br></div><div>- Naomi</div><div><br></div><br><div>On May 1, 2008, at 6:04 PM, Matt Mitchell wrote:<br class="Apple-interchange-newline"><blockquote type="cite">Hi Naomi,<br><br>Thanks for the detailed report! The _display fields should be multiValued and the current schema is here:<br><br><a href="http://blacklight.rubyforge.org/svn/trunk/rails/solr/conf/schema.xml">http://blacklight.rubyforge.org/svn/trunk/rails/solr/conf/schema.xml</a><br> <br>Are you using the trunk version or the last release?<br><br>Matt<br><br><div class="gmail_quote">On Thu, May 1, 2008 at 8:50 PM, Naomi Dushay <<a href="mailto:ndushay@stanford.edu">ndushay@stanford.edu</a>> wrote:<br> <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div style="">I tried the importer just now (just pulled it from svn, too.) and hit a few bumps also. I concur with the index.sh problems ... I just ended up executing the java command directly from the command line.<br> <br>I believe the sample data has a record with two 020 subfield a values. From the output of the importer on the sample file:<div><br></div><div>Adding record 8: u89<br>Error indexing<br>org.apache.solr.common.SolrException: ERROR: multiple values encountered for non multiValued field isbn_display: first='0877663637' second='0877663343 (pbk.)'<br> <span style="white-space: pre;"> </span>at org.apache.solr.update.DocumentBuilder.addSingleField(DocumentBuilder.java:67)<br><span style="white-space: pre;"> </span>at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:88)<br> <span style="white-space: pre;"> </span>at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:118)<br><span style="white-space: pre;"> </span>at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:101)<br> <span style="white-space: pre;"> </span>at SolrIndexer.addField(Unknown Source)<br><span style="white-space: pre;"> </span>at SolrIndexer.addFields(Unknown Source)<br><span style="white-space: pre;"> </span>at SolrIndexer.indexRecord(Unknown Source)<br> <span style="white-space: pre;"> </span>at MarcImporter.addToIndex(Unknown Source)<br><span style="white-space: pre;"> </span>at MarcImporter.importRecords(Unknown Source)<br><span style="white-space: pre;"> </span>at MarcImporter.main(Unknown Source)<br> Adding record 9: u144<br><br></div><div>The SOLR schema.xml file in blacklight/solr-home/conf directory says that all *_display fields are NOT multiValued.</div><div><br></div><div>To get the sample data to index without an error, it's just a matter of changing one line in the blacklight.properties file:</div> <div><br></div><div>from:</div><div><div style="margin: 0px;"><span style="color: rgb(0, 0, 230);">field_list_25a</span> <span style="color: rgb(128, 128, 48);">=</span> isbn<span style="color: rgb(128, 128, 48);">_</span>display<span style="color: rgb(128, 128, 48);">,</span> all<span style="color: rgb(128, 128, 48);">,</span> 020a</div> <div><font face="Monaco" size="3"><span style="font-size: 11px;"><br></span></font></div><div><font face="Monaco" size="3"><span style="font-size: 11px;">to</span></font></div><div><font face="Monaco" size="3"><span style="font-size: 11px;"><div style="margin: 0px;"> <span style="color: rgb(0, 0, 230);">field_list_25a</span> <span style="color: rgb(128, 128, 48);">=</span> isbn<span style="color: rgb(128, 128, 48);">_</span>display<span style="color: rgb(128, 128, 48);">,</span> first<span style="color: rgb(128, 128, 48);">,</span> 020a</div> <div style="margin: 0px;"><br></div><div style="margin: 0px;">HOWEVER, I am still getting a</div><div style="margin: 0px;"><br></div><div style="margin: 0px;"><div style="margin: 0px;">java.lang.RuntimeException: org.apache.lucene.index.CorruptIndexException: Unknown format version: -4</div> <div><font face="Helvetica" size="3"><span style="font-size: 12px;"><br></span></font></div><div><font face="Helvetica" size="3"><span style="font-size: 12px;">when trying to view the index with solr (<a href="http://yer.path:8983/solr/admin" target="_blank">http://yer.path:8983/solr/admin</a> ).</span></font></div> <div><font face="Helvetica" size="3"><span style="font-size: 12px;"><br></span></font></div><div><font face="Helvetica" size="3"><span style="font-size: 12px;">But I can look at the same index with luke without a problem.</span></font></div> <div><font face="Helvetica" size="3"><span style="font-size: 12px;"><br></span></font></div><div><br></div></div><div style="margin: 0px;">- Naomi Dushay</div><div style="margin: 0px;">Stanford University Libraries</div><div style="margin: 0px;"> <span style="font-family: Helvetica; font-size: 12px;"><a href="mailto:ndushay@stanford.edu" target="_blank">ndushay@stanford.edu</a></span></div></span></font></div></div><div><br><br><br></div></div><br>_______________________________________________<br> Blacklight-development mailing list<br> <a href="mailto:Blacklight-development@rubyforge.org">Blacklight-development@rubyforge.org</a><br> <a href="http://rubyforge.org/mailman/listinfo/blacklight-development" target="_blank">http://rubyforge.org/mailman/listinfo/blacklight-development</a><br> <br></blockquote></div><br> _______________________________________________<br>Blacklight-development mailing list<br><a href="mailto:Blacklight-development@rubyforge.org">Blacklight-development@rubyforge.org</a><br><a href="http://rubyforge.org/mailman/listinfo/blacklight-development">http://rubyforge.org/mailman/listinfo/blacklight-development</a><br></blockquote></div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Naomi Dushay</div><div><a href="mailto:ndushay@stanford.edu">ndushay@stanford.edu</a></div><div><br></div></div></span><br class="Apple-interchange-newline"> </div><br></div>_______________________________________________<br>Blacklight-development mailing list<br><a href="mailto:Blacklight-development@rubyforge.org">Blacklight-development@rubyforge.org</a><br>http://rubyforge.org/mailman/listinfo/blacklight-development<br></blockquote></div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Naomi Dushay</div><div><a href="mailto:ndushay@stanford.edu">ndushay@stanford.edu</a></div><div><br></div></div></span><br class="Apple-interchange-newline"> </div><br></body></html>