From rochkind at jhu.edu Thu May 1 10:44:33 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 01 May 2008 10:44:33 -0400 Subject: [Blacklight-development] new blacklight In-Reply-To: <47FE8BD4.3050209@virginia.edu> References: <47FE8BD4.3050209@virginia.edu> Message-ID: <4819D751.9010908@jhu.edu> Okay, I'm _finally_ getting to this. Can you give me a hint as to WHERE to find the java-based indexer and more importantly the Getting Started document in the svn checkout? :) But I'll root around and maybe find it my own before you answer me, but a tip would be nice. Once I figure it out, maybe I'll add it as a step to the "BLACKLIGHT_UVA_LIB" lib document? (And incidentally, why not make this the README, instead of leaving the README stock unhelpful rails, and putting the real readme in a mysterious location? :) ). I forget if I have commit rights to your svn or not. Jonathan Robert Haschart wrote: > Jonathan, > > Regarding using the new java-based indexer. I have updated the code > stored in SVN at Rubyforge and added a Getting started document to walk > you through getting started. I also included a set of sample records, > that you can use to see how the process works. > > > If you do a svn update you should see the new source code, the sample > records, and the documentation about the program. > > -Bob > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From rochkind at jhu.edu Thu May 1 11:11:15 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 01 May 2008 11:11:15 -0400 Subject: [Blacklight-development] new blacklight In-Reply-To: References: <47FE8BD4.3050209@virginia.edu> <4819D751.9010908@jhu.edu> Message-ID: <4819DD93.90005@jhu.edu> [Can someone change this list to reply to list instead of replying to author?] Thanks Bess! No wonder I couldn't find it in: http://blacklight.rubyforge.org/svn/trunk Which is what I was checking out before! It's not under trunk? Okeydokey then. (I AM checking out the right thing for blacklight itself, right?). Going through the existing README instructions, I have also discovered that you need to (at least) copy the good solr.yml file over BEFORE you can run the migration--probably something in your environment looks at solr.yml. So I'll re-order that. (And see what else I need to do to actually get it running, not quite there yet. :) ). I'll also add a note about the importer. If I follow the directions in the current BLACKLIGHT_UVA_LIB file, I'll get a working blacklight installation with no data in it, I guess? I'll find out soon enough! My rubyforge id is jrochkind. I know how to deal with rubyforge svn. Be happy to fix up the documentation and such. Obviously won't touch any code without permission. Jonathan Bess Sadler wrote: > Hi, Jonathan. > > The importer is at > http://blacklight.rubyforge.org/svn/blacklight_importer/ > > The getting started document is > > http://blacklight.rubyforge.org/svn/blacklight_importer/GettingStarted.txt > > > If you'd like to re-write the readme I would be very happy to give you > commit rights to let you do so. What's your rubyforge id? You'll need > to check the code out with svn+ssh in order to be able to commit. > > Please let us know how testing the indexer works. > > Cheers! > Bess > > On May 1, 2008, at 10:44 AM, Jonathan Rochkind wrote: >> Okay, I'm _finally_ getting to this. Can you give me a hint as to WHERE >> to find the java-based indexer and more importantly the Getting Started >> document in the svn checkout? :) >> >> But I'll root around and maybe find it my own before you answer me, but >> a tip would be nice. >> >> Once I figure it out, maybe I'll add it as a step to the >> "BLACKLIGHT_UVA_LIB" lib document? (And incidentally, why not make this >> the README, instead of leaving the README stock unhelpful rails, and >> putting the real readme in a mysterious location? :) ). I forget if I >> have commit rights to your svn or not. >> >> Jonathan >> >> Robert Haschart wrote: >>> Jonathan, >>> >>> Regarding using the new java-based indexer. I have updated the code >>> stored in SVN at Rubyforge and added a Getting started document to walk >>> you through getting started. I also included a set of sample records, >>> that you can use to see how the process works. >>> >>> >>> If you do a svn update you should see the new source code, the >>> sample >>> records, and the documentation about the program. >>> >>> -Bob >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >> >> -- >> Jonathan Rochkind >> Digital Services Software Engineer >> The Sheridan Libraries >> Johns Hopkins University >> 410.516.8886 >> rochkind (at) jhu.edu >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development > > Elizabeth (Bess) Sadler > Research and Development Librarian > Digital Scholarship Services > Box 400129 > Alderman Library > University of Virginia > Charlottesville, VA 22904 > > bess at virginia.edu > (434) 243-2305 > -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From jronallo at gmail.com Thu May 1 11:15:54 2008 From: jronallo at gmail.com (Jason Ronallo) Date: Thu, 1 May 2008 11:15:54 -0400 Subject: [Blacklight-development] new blacklight In-Reply-To: <4819D751.9010908@jhu.edu> References: <47FE8BD4.3050209@virginia.edu> <4819D751.9010908@jhu.edu> Message-ID: <763570460805010815r64a6ae56w5a9b1c53edd11a27@mail.gmail.com> Jonathan, If you haven't, you need to checkout more than just trunk. See the directory blacklight_importer at the root of the svn repo: http://blacklight.rubyforge.org/svn/ Just a tip that it looks like they're developing on Windows so you'll need to change importSample.properties (or whichever import*properties you use) to find your solr-home (/trunk/rails/solr/ IIRC). I just used this so let me know if you have other questions. I remember having to monkeypatch the code in some places to get trunk blacklight up and running. I hope to have a chance in the next couple weeks to report those. Jason On Thu, May 1, 2008 at 10:44 AM, Jonathan Rochkind wrote: > Okay, I'm _finally_ getting to this. Can you give me a hint as to WHERE > to find the java-based indexer and more importantly the Getting Started > document in the svn checkout? :) > > But I'll root around and maybe find it my own before you answer me, but > a tip would be nice. > > Once I figure it out, maybe I'll add it as a step to the > "BLACKLIGHT_UVA_LIB" lib document? (And incidentally, why not make this the > README, instead of leaving the README stock unhelpful rails, and putting the > real readme in a mysterious location? :) ). I forget if I have commit > rights to your svn or not. > > Jonathan > > > > Robert Haschart wrote: > > > Jonathan, > > > > Regarding using the new java-based indexer. I have updated the code > stored in SVN at Rubyforge and added a Getting started document to walk you > through getting started. I also included a set of sample records, that you > can use to see how the process works. > > > > > > If you do a svn update you should see the new source code, the sample > records, and the documentation about the program. > > > > -Bob > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > > > -- > Jonathan Rochkind > Digital Services Software Engineer > The Sheridan Libraries > Johns Hopkins University > 410.516.8886 > rochkind (at) jhu.edu > > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From rochkind at jhu.edu Thu May 1 11:16:23 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 01 May 2008 11:16:23 -0400 Subject: [Blacklight-development] more blacklight getting started problems Message-ID: <4819DEC7.4040008@jhu.edu> Okay, this is weird. I have setup my database.yml file properly to point to my mysql instance, where i have created databases accessible to the user 'blacklight'. That is the user I specify in my database.yml. (I am experienced with Rails, I know how it works). Yet, when I run the rake db:migrate, I get: Access denied for user 'root'@'localhost' (using password: YES) I don't know why it's trying to connect with that user in the first place, since that's NOT the user I've specified in my database.yml. Hmm, maybe something in your environment that does a db connection with some different configured password or connection or something? Very weird. Let's check out the rake trace... Gives me no real hints. Included below in case it gives someone else hints. I don't get it. [jrochki1 at testbox rails]$ rake db:migrate --trace (in /home/jrochki1/blacklight/rails) ** Invoke db:migrate (first_time) ** Invoke environment (first_time) ** Execute environment ** Execute db:migrate rake aborted! Access denied for user 'root'@'localhost' (using password: YES) /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in `real_connect' /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in `connect' /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:165:in `initialize' /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in `new' /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in `mysql_connection' /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in `send' /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in `connection=' /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:259:in `retrieve_connection' /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:78:in `connection' /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/migration.rb:294:in `migrate' /usr/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/tasks/databases.rake:85 /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `call' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `execute' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `each' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `execute' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:508:in `invoke_with_call_chain' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in `synchronize' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in `invoke_with_call_chain' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:494:in `invoke' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1931:in `invoke_task' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `each' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in `standard_exception_handling' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1903:in `top_level' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1881:in `run' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in `standard_exception_handling' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1878:in `run' /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/bin/rake:31 /usr/bin/rake:16:in `load' /usr/bin/rake:16 -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From goodieboy at gmail.com Thu May 1 11:58:49 2008 From: goodieboy at gmail.com (Matt Mitchell) Date: Thu, 1 May 2008 11:58:49 -0400 Subject: [Blacklight-development] more blacklight getting started problems In-Reply-To: <4819DEC7.4040008@jhu.edu> References: <4819DEC7.4040008@jhu.edu> Message-ID: Jonathan, I can't quite say for sure what happening but, in your database.yml file, is it possible you're using "username" instead of "user"? Matt On Thu, May 1, 2008 at 11:16 AM, Jonathan Rochkind wrote: > Okay, this is weird. > > I have setup my database.yml file properly to point to my mysql instance, > where i have created databases accessible to the user 'blacklight'. That is > the user I specify in my database.yml. (I am experienced with Rails, I know > how it works). > > Yet, when I run the rake db:migrate, I get: > Access denied for user 'root'@'localhost' (using password: YES) > > > I don't know why it's trying to connect with that user in the first place, > since that's NOT the user I've specified in my database.yml. Hmm, maybe > something in your environment that does a db connection with some different > configured password or connection or something? Very weird. Let's check out > the rake trace... Gives me no real hints. Included below in case it gives > someone else hints. I don't get it. > > [jrochki1 at testbox rails]$ rake db:migrate --trace > (in /home/jrochki1/blacklight/rails) > ** Invoke db:migrate (first_time) > ** Invoke environment (first_time) > ** Execute environment > ** Execute db:migrate > rake aborted! > Access denied for user 'root'@'localhost' (using password: YES) > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in > `real_connect' > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in > `connect' > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:165:in > `initialize' > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in > `new' > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in > `mysql_connection' > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in > `send' > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in > `connection=' > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:259:in > `retrieve_connection' > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:78:in > `connection' > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/migration.rb:294:in > `migrate' > /usr/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/tasks/databases.rake:85 > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `call' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `execute' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `each' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `execute' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:508:in > `invoke_with_call_chain' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in `synchronize' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in > `invoke_with_call_chain' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:494:in `invoke' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1931:in `invoke_task' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `each' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in > `standard_exception_handling' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1903:in `top_level' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1881:in `run' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in > `standard_exception_handling' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1878:in `run' > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/bin/rake:31 > /usr/bin/rake:16:in `load' > /usr/bin/rake:16 > > > -- > Jonathan Rochkind > Digital Services Software Engineer > The Sheridan Libraries > Johns Hopkins University > 410.516.8886 rochkind (at) jhu.edu > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -------------- next part -------------- An HTML attachment was scrubbed... URL: From goodieboy at gmail.com Thu May 1 12:07:19 2008 From: goodieboy at gmail.com (Matt Mitchell) Date: Thu, 1 May 2008 12:07:19 -0400 Subject: [Blacklight-development] README now primary "getting-started" Message-ID: I've moved all of the instructions from BLACKLIGHT_UVA_LIB to README. The README is also now in RDoc syntax. The BLACKLIGHT_UVA_LIB file is only a visual cue; "what project am I working on here?" Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From rochkind at jhu.edu Thu May 1 12:53:55 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 01 May 2008 12:53:55 -0400 Subject: [Blacklight-development] more blacklight getting started problems In-Reply-To: <4819F523.5070006@jhu.edu> References: <4819DEC7.4040008@jhu.edu> <4819F523.5070006@jhu.edu> Message-ID: <4819F5A3.2060304@jhu.edu> Jonathan Rochkind wrote: > Sadly, no, it's "user". I guess I'll have to debug this the hard way. > Stay tuned. > > Matt Mitchell wrote: >> Jonathan, >> >> I can't quite say for sure what happening but, in your database.yml >> file, is >> it possible you're using "username" instead of "user"? >> >> Matt >> >> On Thu, May 1, 2008 at 11:16 AM, Jonathan Rochkind >> wrote: >> >> >>> Okay, this is weird. >>> >>> I have setup my database.yml file properly to point to my mysql >>> instance, >>> where i have created databases accessible to the user 'blacklight'. >>> That is >>> the user I specify in my database.yml. (I am experienced with Rails, >>> I know >>> how it works). >>> >>> Yet, when I run the rake db:migrate, I get: >>> Access denied for user 'root'@'localhost' (using password: YES) >>> >>> >>> I don't know why it's trying to connect with that user in the first >>> place, >>> since that's NOT the user I've specified in my database.yml. Hmm, maybe >>> something in your environment that does a db connection with some >>> different >>> configured password or connection or something? Very weird. Let's >>> check out >>> the rake trace... Gives me no real hints. Included below in case it >>> gives >>> someone else hints. I don't get it. >>> >>> [jrochki1 at testbox rails]$ rake db:migrate --trace >>> (in /home/jrochki1/blacklight/rails) >>> ** Invoke db:migrate (first_time) >>> ** Invoke environment (first_time) >>> ** Execute environment >>> ** Execute db:migrate >>> rake aborted! >>> Access denied for user 'root'@'localhost' (using password: YES) >>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in >>> >>> `real_connect' >>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in >>> >>> `connect' >>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:165:in >>> >>> `initialize' >>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in >>> >>> `new' >>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in >>> >>> `mysql_connection' >>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in >>> >>> `send' >>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in >>> >>> `connection=' >>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:259:in >>> >>> `retrieve_connection' >>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:78:in >>> >>> `connection' >>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/migration.rb:294:in >>> >>> `migrate' >>> /usr/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/tasks/databases.rake:85 >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `call' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `execute' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `each' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `execute' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:508:in >>> `invoke_with_call_chain' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in `synchronize' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in >>> `invoke_with_call_chain' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:494:in `invoke' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1931:in >>> `invoke_task' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `each' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in >>> `standard_exception_handling' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1903:in `top_level' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1881:in `run' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in >>> `standard_exception_handling' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1878:in `run' >>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/bin/rake:31 >>> /usr/bin/rake:16:in `load' >>> /usr/bin/rake:16 >>> >>> >>> -- >>> Jonathan Rochkind >>> Digital Services Software Engineer >>> The Sheridan Libraries >>> Johns Hopkins University >>> 410.516.8886 rochkind (at) jhu.edu >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >>> >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> > -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From rochkind at jhu.edu Thu May 1 13:05:59 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 01 May 2008 13:05:59 -0400 Subject: [Blacklight-development] more blacklight getting started problems In-Reply-To: <4819F5A3.2060304@jhu.edu> References: <4819DEC7.4040008@jhu.edu> <4819F523.5070006@jhu.edu> <4819F5A3.2060304@jhu.edu> Message-ID: <4819F877.9010208@jhu.edu> Hmm, it looks like it's the REVERSE of what Matt said. "username" is the _correct_ key, NOT "user". I was using "user" because that's what was in the same file. Weird. "user" works for you guys rather than "username". My Rails installation wants "username", NOT "user". Very odd. Jonathan Jonathan Rochkind wrote: > > > Jonathan Rochkind wrote: >> Sadly, no, it's "user". I guess I'll have to debug this the hard >> way. Stay tuned. >> >> Matt Mitchell wrote: >>> Jonathan, >>> >>> I can't quite say for sure what happening but, in your database.yml >>> file, is >>> it possible you're using "username" instead of "user"? >>> >>> Matt >>> >>> On Thu, May 1, 2008 at 11:16 AM, Jonathan Rochkind >>> wrote: >>> >>> >>>> Okay, this is weird. >>>> >>>> I have setup my database.yml file properly to point to my mysql >>>> instance, >>>> where i have created databases accessible to the user 'blacklight'. >>>> That is >>>> the user I specify in my database.yml. (I am experienced with >>>> Rails, I know >>>> how it works). >>>> >>>> Yet, when I run the rake db:migrate, I get: >>>> Access denied for user 'root'@'localhost' (using password: YES) >>>> >>>> >>>> I don't know why it's trying to connect with that user in the first >>>> place, >>>> since that's NOT the user I've specified in my database.yml. Hmm, >>>> maybe >>>> something in your environment that does a db connection with some >>>> different >>>> configured password or connection or something? Very weird. Let's >>>> check out >>>> the rake trace... Gives me no real hints. Included below in case it >>>> gives >>>> someone else hints. I don't get it. >>>> >>>> [jrochki1 at testbox rails]$ rake db:migrate --trace >>>> (in /home/jrochki1/blacklight/rails) >>>> ** Invoke db:migrate (first_time) >>>> ** Invoke environment (first_time) >>>> ** Execute environment >>>> ** Execute db:migrate >>>> rake aborted! >>>> Access denied for user 'root'@'localhost' (using password: YES) >>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in >>>> >>>> `real_connect' >>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in >>>> >>>> `connect' >>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:165:in >>>> >>>> `initialize' >>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in >>>> >>>> `new' >>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in >>>> >>>> `mysql_connection' >>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in >>>> >>>> `send' >>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in >>>> >>>> `connection=' >>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:259:in >>>> >>>> `retrieve_connection' >>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:78:in >>>> >>>> `connection' >>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/migration.rb:294:in >>>> >>>> `migrate' >>>> /usr/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/tasks/databases.rake:85 >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `call' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `execute' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `each' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `execute' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:508:in >>>> `invoke_with_call_chain' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in >>>> `synchronize' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in >>>> `invoke_with_call_chain' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:494:in `invoke' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1931:in >>>> `invoke_task' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `each' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in >>>> `standard_exception_handling' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1903:in `top_level' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1881:in `run' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in >>>> `standard_exception_handling' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1878:in `run' >>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/bin/rake:31 >>>> /usr/bin/rake:16:in `load' >>>> /usr/bin/rake:16 >>>> >>>> >>>> -- >>>> Jonathan Rochkind >>>> Digital Services Software Engineer >>>> The Sheridan Libraries >>>> Johns Hopkins University >>>> 410.516.8886 rochkind (at) jhu.edu >>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> >>>> >>> >>> >>> ------------------------------------------------------------------------ >>> >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >> > -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From goodieboy at gmail.com Thu May 1 13:14:12 2008 From: goodieboy at gmail.com (Matt Mitchell) Date: Thu, 1 May 2008 13:14:12 -0400 Subject: [Blacklight-development] more blacklight getting started problems In-Reply-To: <4819F877.9010208@jhu.edu> References: <4819DEC7.4040008@jhu.edu> <4819F523.5070006@jhu.edu> <4819F5A3.2060304@jhu.edu> <4819F877.9010208@jhu.edu> Message-ID: You're using 2.0.2 right? That is strange. Matt On Thu, May 1, 2008 at 1:05 PM, Jonathan Rochkind wrote: > Hmm, it looks like it's the REVERSE of what Matt said. > > "username" is the _correct_ key, NOT "user". I was using "user" because > that's what was in the same file. Weird. "user" works for you guys rather > than "username". My Rails installation wants "username", NOT "user". Very > odd. > > Jonathan > > > Jonathan Rochkind wrote: > > > > > > > Jonathan Rochkind wrote: > > > > > Sadly, no, it's "user". I guess I'll have to debug this the hard way. > > > Stay tuned. > > > > > > Matt Mitchell wrote: > > > > > > > Jonathan, > > > > > > > > I can't quite say for sure what happening but, in your database.yml > > > > file, is > > > > it possible you're using "username" instead of "user"? > > > > > > > > Matt > > > > > > > > On Thu, May 1, 2008 at 11:16 AM, Jonathan Rochkind > > > > wrote: > > > > > > > > > > > > > > > > > Okay, this is weird. > > > > > > > > > > I have setup my database.yml file properly to point to my mysql > > > > > instance, > > > > > where i have created databases accessible to the user > > > > > 'blacklight'. That is > > > > > the user I specify in my database.yml. (I am experienced with > > > > > Rails, I know > > > > > how it works). > > > > > > > > > > Yet, when I run the rake db:migrate, I get: > > > > > Access denied for user 'root'@'localhost' (using password: YES) > > > > > > > > > > > > > > > I don't know why it's trying to connect with that user in the > > > > > first place, > > > > > since that's NOT the user I've specified in my database.yml. Hmm, > > > > > maybe > > > > > something in your environment that does a db connection with some > > > > > different > > > > > configured password or connection or something? Very weird. Let's > > > > > check out > > > > > the rake trace... Gives me no real hints. Included below in case > > > > > it gives > > > > > someone else hints. I don't get it. > > > > > > > > > > [jrochki1 at testbox rails]$ rake db:migrate --trace > > > > > (in /home/jrochki1/blacklight/rails) > > > > > ** Invoke db:migrate (first_time) > > > > > ** Invoke environment (first_time) > > > > > ** Execute environment > > > > > ** Execute db:migrate > > > > > rake aborted! > > > > > Access denied for user 'root'@'localhost' (using password: YES) > > > > > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in > > > > > > > > > > `real_connect' > > > > > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in > > > > > > > > > > `connect' > > > > > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:165:in > > > > > > > > > > `initialize' > > > > > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in > > > > > > > > > > `new' > > > > > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in > > > > > > > > > > `mysql_connection' > > > > > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in > > > > > > > > > > `send' > > > > > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in > > > > > > > > > > `connection=' > > > > > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:259:in > > > > > > > > > > `retrieve_connection' > > > > > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:78:in > > > > > > > > > > `connection' > > > > > /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/migration.rb:294:in > > > > > > > > > > `migrate' > > > > > > > > > > /usr/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/tasks/databases.rake:85 > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `call' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in > > > > > `execute' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `each' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in > > > > > `execute' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:508:in > > > > > `invoke_with_call_chain' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in > > > > > `synchronize' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in > > > > > `invoke_with_call_chain' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:494:in `invoke' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1931:in > > > > > `invoke_task' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in > > > > > `top_level' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `each' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in > > > > > `top_level' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in > > > > > `standard_exception_handling' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1903:in > > > > > `top_level' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1881:in `run' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in > > > > > `standard_exception_handling' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1878:in `run' > > > > > /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/bin/rake:31 > > > > > /usr/bin/rake:16:in `load' > > > > > /usr/bin/rake:16 > > > > > > > > > > > > > > > -- > > > > > Jonathan Rochkind > > > > > Digital Services Software Engineer > > > > > The Sheridan Libraries > > > > > Johns Hopkins University > > > > > 410.516.8886 rochkind (at) jhu.edu > > > > > > > > > > _______________________________________________ > > > > > Blacklight-development mailing list > > > > > Blacklight-development at rubyforge.org > > > > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------ > > > > > > > > > > > > _______________________________________________ > > > > Blacklight-development mailing list > > > > Blacklight-development at rubyforge.org > > > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > > > > > > > > > > > > > > -- > Jonathan Rochkind > Digital Services Software Engineer > The Sheridan Libraries > Johns Hopkins University > 410.516.8886 rochkind (at) jhu.edu > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rochkind at jhu.edu Thu May 1 13:14:37 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 01 May 2008 13:14:37 -0400 Subject: [Blacklight-development] more blacklight getting started problems In-Reply-To: <4819F774.5090602@jhu.edu> References: <4819DEC7.4040008@jhu.edu> <4819F523.5070006@jhu.edu> <4819F5A3.2060304@jhu.edu> <4819F774.5090602@jhu.edu> Message-ID: <4819FA7D.4060307@jhu.edu> By the way, your documentation is excellent, thanks Matt! Jonathan Jonathan Rochkind wrote: > Yeah, when I create a brand new rails skeleton, I get the same error. > WEIRD. Okay, fun debugging of what's going on with my Rails > environment today. > > Jonathan Rochkind wrote: >> >> >> Jonathan Rochkind wrote: >>> Sadly, no, it's "user". I guess I'll have to debug this the hard >>> way. Stay tuned. >>> >>> Matt Mitchell wrote: >>>> Jonathan, >>>> >>>> I can't quite say for sure what happening but, in your database.yml >>>> file, is >>>> it possible you're using "username" instead of "user"? >>>> >>>> Matt >>>> >>>> On Thu, May 1, 2008 at 11:16 AM, Jonathan Rochkind >>>> wrote: >>>> >>>> >>>>> Okay, this is weird. >>>>> >>>>> I have setup my database.yml file properly to point to my mysql >>>>> instance, >>>>> where i have created databases accessible to the user >>>>> 'blacklight'. That is >>>>> the user I specify in my database.yml. (I am experienced with >>>>> Rails, I know >>>>> how it works). >>>>> >>>>> Yet, when I run the rake db:migrate, I get: >>>>> Access denied for user 'root'@'localhost' (using password: YES) >>>>> >>>>> >>>>> I don't know why it's trying to connect with that user in the >>>>> first place, >>>>> since that's NOT the user I've specified in my database.yml. Hmm, >>>>> maybe >>>>> something in your environment that does a db connection with some >>>>> different >>>>> configured password or connection or something? Very weird. Let's >>>>> check out >>>>> the rake trace... Gives me no real hints. Included below in case >>>>> it gives >>>>> someone else hints. I don't get it. >>>>> >>>>> [jrochki1 at testbox rails]$ rake db:migrate --trace >>>>> (in /home/jrochki1/blacklight/rails) >>>>> ** Invoke db:migrate (first_time) >>>>> ** Invoke environment (first_time) >>>>> ** Execute environment >>>>> ** Execute db:migrate >>>>> rake aborted! >>>>> Access denied for user 'root'@'localhost' (using password: YES) >>>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in >>>>> >>>>> `real_connect' >>>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:471:in >>>>> >>>>> `connect' >>>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:165:in >>>>> >>>>> `initialize' >>>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in >>>>> >>>>> `new' >>>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:88:in >>>>> >>>>> `mysql_connection' >>>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in >>>>> >>>>> `send' >>>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:291:in >>>>> >>>>> `connection=' >>>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:259:in >>>>> >>>>> `retrieve_connection' >>>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:78:in >>>>> >>>>> `connection' >>>>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/migration.rb:294:in >>>>> >>>>> `migrate' >>>>> /usr/lib/ruby/gems/1.8/gems/rails-2.0.2/lib/tasks/databases.rake:85 >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `call' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `execute' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `each' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `execute' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:508:in >>>>> `invoke_with_call_chain' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in >>>>> `synchronize' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in >>>>> `invoke_with_call_chain' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:494:in `invoke' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1931:in >>>>> `invoke_task' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in >>>>> `top_level' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `each' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in >>>>> `top_level' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in >>>>> `standard_exception_handling' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1903:in >>>>> `top_level' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1881:in `run' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in >>>>> `standard_exception_handling' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1878:in `run' >>>>> /usr/lib/ruby/gems/1.8/gems/rake-0.8.1/bin/rake:31 >>>>> /usr/bin/rake:16:in `load' >>>>> /usr/bin/rake:16 >>>>> >>>>> >>>>> -- >>>>> Jonathan Rochkind >>>>> Digital Services Software Engineer >>>>> The Sheridan Libraries >>>>> Johns Hopkins University >>>>> 410.516.8886 rochkind (at) jhu.edu >>>>> >>>>> _______________________________________________ >>>>> Blacklight-development mailing list >>>>> Blacklight-development at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>> >>>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> >>> >> > -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From rochkind at jhu.edu Thu May 1 14:32:50 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 01 May 2008 14:32:50 -0400 Subject: [Blacklight-development] more little importer tricks Message-ID: <481A0CD2.8000902@jhu.edu> 1. On unix, you've got to re-save index_file.sh with unix line endings (or at least the first line), so it will find the interpreter properly instead of saying: -bash: ./index_file.sh: /bin/sh^M: bad interpreter: No such file or directory 2. The sample file name given in GettingStarted.txt is not quite right. NOT: ./index_file.sh sample_records/selected_recs.mrc importSamples.properties BUT INSTEAD: ./index_file.sh sample_records/selectedRecs.mrc importSamples.properties With that, it SAYS it indexed the files---but they don't seem to actually show up in my blacklight installation. I pointed solr.path in importSamples.properties at the solr that's embedded with blacklight, that's right, right? Hmm, how do I confirm that these 8 records actually ARE in blacklight? Blacklight used to show ALL records as it's default screen, but I don't think it does anymore--nor is it showing ANY facets on the home page, like it used to. Does this mean that I for some reason haven't in fact succesfully imported the records? Thanks all. Jonathan -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From jronallo at gmail.com Thu May 1 14:40:32 2008 From: jronallo at gmail.com (Jason Ronallo) Date: Thu, 1 May 2008 14:40:32 -0400 Subject: [Blacklight-development] more little importer tricks In-Reply-To: <481A0CD2.8000902@jhu.edu> References: <481A0CD2.8000902@jhu.edu> Message-ID: <763570460805011140v477e899bo401de34141000c80@mail.gmail.com> Jonathan, Have you started up solr? >From within the trunk/rails directory you can run: rake solr_int:start to start solr + jetty Then you can go to http://localhost:8983/solr/admin (I think that's the correct port) do the query: [* TO *] and see what your results are. At least that's what I'd try. Jason On Thu, May 1, 2008 at 2:32 PM, Jonathan Rochkind wrote: > 1. On unix, you've got to re-save index_file.sh with unix line endings (or > at least the first line), so it will find the interpreter properly instead > of saying: > > -bash: ./index_file.sh: /bin/sh^M: bad interpreter: No such file or > directory > > 2. The sample file name given in GettingStarted.txt is not quite right. > NOT: > ./index_file.sh sample_records/selected_recs.mrc importSamples.properties > BUT INSTEAD: > ./index_file.sh sample_records/selectedRecs.mrc importSamples.properties > > With that, it SAYS it indexed the files---but they don't seem to actually > show up in my blacklight installation. I pointed solr.path in > importSamples.properties at the solr that's embedded with blacklight, that's > right, right? Hmm, how do I confirm that these 8 records actually ARE in > blacklight? Blacklight used to show ALL records as it's default screen, but > I don't think it does anymore--nor is it showing ANY facets on the home > page, like it used to. Does this mean that I for some reason haven't in fact > succesfully imported the records? > > Thanks all. > > Jonathan > > -- > Jonathan Rochkind > Digital Services Software Engineer > The Sheridan Libraries > Johns Hopkins University > 410.516.8886 rochkind (at) jhu.edu > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From goodieboy at gmail.com Thu May 1 14:58:54 2008 From: goodieboy at gmail.com (Matt Mitchell) Date: Thu, 1 May 2008 14:58:54 -0400 Subject: [Blacklight-development] more little importer tricks In-Reply-To: <763570460805011140v477e899bo401de34141000c80@mail.gmail.com> References: <481A0CD2.8000902@jhu.edu> <763570460805011140v477e899bo401de34141000c80@mail.gmail.com> Message-ID: Yeah I'd do what Jason suggested. I wonder if the script didn't execute a commit. You should still be able to see some kind of indicator in the solr admin: http://localhost:8983/solr/admin/stats.jsp Matt On Thu, May 1, 2008 at 2:40 PM, Jason Ronallo wrote: > Jonathan, > Have you started up solr? > >From within the trunk/rails directory you can run: rake solr_int:start > to start solr + jetty > Then you can go to http://localhost:8983/solr/admin (I think that's > the correct port) > do the query: [* TO *] > and see what your results are. > > At least that's what I'd try. > > Jason > > On Thu, May 1, 2008 at 2:32 PM, Jonathan Rochkind > wrote: > > 1. On unix, you've got to re-save index_file.sh with unix line endings > (or > > at least the first line), so it will find the interpreter properly > instead > > of saying: > > > > -bash: ./index_file.sh: /bin/sh^M: bad interpreter: No such file or > > directory > > > > 2. The sample file name given in GettingStarted.txt is not quite right. > > NOT: > > ./index_file.sh sample_records/selected_recs.mrc > importSamples.properties > > BUT INSTEAD: > > ./index_file.sh sample_records/selectedRecs.mrc > importSamples.properties > > > > With that, it SAYS it indexed the files---but they don't seem to > actually > > show up in my blacklight installation. I pointed solr.path in > > importSamples.properties at the solr that's embedded with blacklight, > that's > > right, right? Hmm, how do I confirm that these 8 records actually ARE in > > blacklight? Blacklight used to show ALL records as it's default screen, > but > > I don't think it does anymore--nor is it showing ANY facets on the home > > page, like it used to. Does this mean that I for some reason haven't in > fact > > succesfully imported the records? > > > > Thanks all. > > > > Jonathan > > > > -- > > Jonathan Rochkind > > Digital Services Software Engineer > > The Sheridan Libraries > > Johns Hopkins University > > 410.516.8886 rochkind (at) jhu.edu > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rochkind at jhu.edu Thu May 1 15:06:46 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 01 May 2008 15:06:46 -0400 Subject: [Blacklight-development] more little importer tricks In-Reply-To: References: <481A0CD2.8000902@jhu.edu> <763570460805011140v477e899bo401de34141000c80@mail.gmail.com> Message-ID: <481A14C6.60506@jhu.edu> Yeah, my stats says numDocs = 0. Guess the importer failed entirely. And indeed rails/solr/data is empty. Okay, that settles it, importer failed. I'll try it with a different batch of data. Maybe I still have Bess's larger sample set lying around that used to come with blacklight. I want to get it working with a known good data set, before I try with my own marc that will require some different marc field mappings to make any sense, and may also be completely broken in other ways, who knows. If you have a chance (no hurry really), would appreciate you checking the sample script to make sure it does a commit and works for _anyone_. But I guess Jaron has used it succesfully? Jonathan Matt Mitchell wrote: > Yeah I'd do what Jason suggested. I wonder if the script didn't execute a > commit. You should still be able to see some kind of indicator in the solr > admin: > > http://localhost:8983/solr/admin/stats.jsp > > Matt > > On Thu, May 1, 2008 at 2:40 PM, Jason Ronallo wrote: > > >> Jonathan, >> Have you started up solr? >> >From within the trunk/rails directory you can run: rake solr_int:start >> to start solr + jetty >> Then you can go to http://localhost:8983/solr/admin (I think that's >> the correct port) >> do the query: [* TO *] >> and see what your results are. >> >> At least that's what I'd try. >> >> Jason >> >> On Thu, May 1, 2008 at 2:32 PM, Jonathan Rochkind >> wrote: >> >>> 1. On unix, you've got to re-save index_file.sh with unix line endings >>> >> (or >> >>> at least the first line), so it will find the interpreter properly >>> >> instead >> >>> of saying: >>> >>> -bash: ./index_file.sh: /bin/sh^M: bad interpreter: No such file or >>> directory >>> >>> 2. The sample file name given in GettingStarted.txt is not quite right. >>> NOT: >>> ./index_file.sh sample_records/selected_recs.mrc >>> >> importSamples.properties >> >>> BUT INSTEAD: >>> ./index_file.sh sample_records/selectedRecs.mrc >>> >> importSamples.properties >> >>> With that, it SAYS it indexed the files---but they don't seem to >>> >> actually >> >>> show up in my blacklight installation. I pointed solr.path in >>> importSamples.properties at the solr that's embedded with blacklight, >>> >> that's >> >>> right, right? Hmm, how do I confirm that these 8 records actually ARE in >>> blacklight? Blacklight used to show ALL records as it's default screen, >>> >> but >> >>> I don't think it does anymore--nor is it showing ANY facets on the home >>> page, like it used to. Does this mean that I for some reason haven't in >>> >> fact >> >>> succesfully imported the records? >>> >>> Thanks all. >>> >>> Jonathan >>> >>> -- >>> Jonathan Rochkind >>> Digital Services Software Engineer >>> The Sheridan Libraries >>> Johns Hopkins University >>> 410.516.8886 rochkind (at) jhu.edu >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >>> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> > > > ------------------------------------------------------------------------ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From jronallo at gmail.com Thu May 1 15:24:02 2008 From: jronallo at gmail.com (Jason Ronallo) Date: Thu, 1 May 2008 15:24:02 -0400 Subject: [Blacklight-development] more little importer tricks In-Reply-To: <481A14C6.60506@jhu.edu> References: <481A0CD2.8000902@jhu.edu> <763570460805011140v477e899bo401de34141000c80@mail.gmail.com> <481A14C6.60506@jhu.edu> Message-ID: <763570460805011224s40bd0f74g5194de1209e968ec@mail.gmail.com> Jonathan, Yes, I was able to index records successfully (well, mostly, but that's another thread). I used the OSU bib records up at archive.org as one of the sets I indexed. (Search "collection:ol_data marc" methinks) But that's a lot of records for a test. If you look in /blacklight_importer/blacklight.properties you'll see what MARC fields are mapped to which solr fields. Any MARC21 records ought to work for a test of the indexer. I did not use the shell script. I invoked the importer directly with something like: java -Xmx1024M -jar dist/MarcImporter.jar importSamples.properties I didn't have 1024M of RAM so I changed that number. Otherwise as long as I had importSamples.properties configured correctly it worked fine. Jason > If you have a chance (no hurry really), would appreciate you checking the > sample script to make sure it does a commit and works for _anyone_. But I > guess Jaron has used it succesfully? > > Jonathan > > Matt Mitchell wrote: > > > > > > > > > Yeah I'd do what Jason suggested. I wonder if the script didn't execute a > > commit. You should still be able to see some kind of indicator in the solr > > admin: > > > > http://localhost:8983/solr/admin/stats.jsp > > > > Matt > > > > On Thu, May 1, 2008 at 2:40 PM, Jason Ronallo wrote: > > > > > > > > > Jonathan, > > > Have you started up solr? > > > >From within the trunk/rails directory you can run: rake solr_int:start > > > to start solr + jetty > > > Then you can go to http://localhost:8983/solr/admin (I think that's > > > the correct port) > > > do the query: [* TO *] > > > and see what your results are. > > > > > > At least that's what I'd try. > > > > > > Jason > > > > > > On Thu, May 1, 2008 at 2:32 PM, Jonathan Rochkind > > > wrote: > > > > > > > > > > 1. On unix, you've got to re-save index_file.sh with unix line endings > > > > > > > > > > > (or > > > > > > > > > > at least the first line), so it will find the interpreter properly > > > > > > > > > > > instead > > > > > > > > > > of saying: > > > > > > > > -bash: ./index_file.sh: /bin/sh^M: bad interpreter: No such file or > > > > directory > > > > > > > > 2. The sample file name given in GettingStarted.txt is not quite > right. > > > > NOT: > > > > ./index_file.sh sample_records/selected_recs.mrc > > > > > > > > > > > importSamples.properties > > > > > > > > > > BUT INSTEAD: > > > > ./index_file.sh sample_records/selectedRecs.mrc > > > > > > > > > > > importSamples.properties > > > > > > > > > > With that, it SAYS it indexed the files---but they don't seem to > > > > > > > > > > > actually > > > > > > > > > > show up in my blacklight installation. I pointed solr.path in > > > > importSamples.properties at the solr that's embedded with blacklight, > > > > > > > > > > > that's > > > > > > > > > > right, right? Hmm, how do I confirm that these 8 records actually ARE > in > > > > blacklight? Blacklight used to show ALL records as it's default > screen, > > > > > > > > > > > but > > > > > > > > > > I don't think it does anymore--nor is it showing ANY facets on the > home > > > > page, like it used to. Does this mean that I for some reason haven't > in > > > > > > > > > > > fact > > > > > > > > > > succesfully imported the records? > > > > > > > > Thanks all. > > > > > > > > Jonathan > > > > > > > > -- > > > > Jonathan Rochkind > > > > Digital Services Software Engineer > > > > The Sheridan Libraries > > > > Johns Hopkins University > > > > 410.516.8886 rochkind (at) jhu.edu > > > > > > > > _______________________________________________ > > > > Blacklight-development mailing list > > > > Blacklight-development at rubyforge.org > > > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > > > > > > > > > > > > _______________________________________________ > > > Blacklight-development mailing list > > > Blacklight-development at rubyforge.org > > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > > > > > > > > > > ------------------------------------------------------------------------ > > > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > > > -- > > Jonathan Rochkind > Digital Services Software Engineer > The Sheridan Libraries > Johns Hopkins University > 410.516.8886 rochkind (at) jhu.edu > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From rochkind at jhu.edu Thu May 1 16:31:04 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 01 May 2008 16:31:04 -0400 Subject: [Blacklight-development] more indexing woes Message-ID: <481A2888.5070301@jhu.edu> Okay, now I tried to index the file of ~1000 MARC records that Bess gave me once. When they indexer ran, it produced a LOT of errors of the form: Error indexing org.marc4j.MarcException: unable to parse record length at org.marc4j.MarcStreamReader.parseLeader(MarcStreamReader.java:317) at org.marc4j.MarcStreamReader.next(MarcStreamReader.java:138) at MarcTranslatedReader.next(Unknown Source) at MarcImporter.importRecords(Unknown Source) at MarcImporter.main(Unknown Source) Caused by: java.lang.NumberFormatException: For input string: "or, R" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) at java.lang.Integer.parseInt(Integer.java:447) at java.lang.Integer.parseInt(Integer.java:497) I can't tell if it got this error for _every_ record or not, the output isn't sufficient. When it finished, it did claim that it "Indexed 1090 at a rate of about 1302.0per sec". (Yes, it sure is fast). But I don't think I believe it. Looking at my SOLR stats... Still, numDocs : 0 When I do the sample set that came with blacklight_importer, it doesn't report any errors, but STILL doesn't succesfully import anything. So, I'm stymied. Well, that was my open source day for the week. Hear me again next week on Thursday when I have another blacklight day and bang my head against this again. But I'm not quite sure what to do next, I'm stymied. Jonathan -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From ndushay at stanford.edu Thu May 1 20:50:57 2008 From: ndushay at stanford.edu (Naomi Dushay) Date: Thu, 1 May 2008 17:50:57 -0700 Subject: [Blacklight-development] more little importer tricks In-Reply-To: <481A14C6.60506@jhu.edu> References: <481A0CD2.8000902@jhu.edu> <763570460805011140v477e899bo401de34141000c80@mail.gmail.com> <481A14C6.60506@jhu.edu> Message-ID: I tried the importer just now (just pulled it from svn, too.) and hit a few bumps also. I concur with the index.sh problems ... I just ended up executing the java command directly from the command line. I believe the sample data has a record with two 020 subfield a values. From the output of the importer on the sample file: Adding record 8: u89 Error indexing org.apache.solr.common.SolrException: ERROR: multiple values encountered for non multiValued field isbn_display: first='0877663637' second='0877663343 (pbk.)' at org .apache .solr.update.DocumentBuilder.addSingleField(DocumentBuilder.java:67) at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:88) at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java: 118) at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java: 101) at SolrIndexer.addField(Unknown Source) at SolrIndexer.addFields(Unknown Source) at SolrIndexer.indexRecord(Unknown Source) at MarcImporter.addToIndex(Unknown Source) at MarcImporter.importRecords(Unknown Source) at MarcImporter.main(Unknown Source) Adding record 9: u144 The SOLR schema.xml file in blacklight/solr-home/conf directory says that all *_display fields are NOT multiValued. To get the sample data to index without an error, it's just a matter of changing one line in the blacklight.properties file: from: field_list_25a = isbn_display, all, 020a to field_list_25a = isbn_display, first, 020a HOWEVER, I am still getting a java.lang.RuntimeException: org.apache.lucene.index.CorruptIndexException: Unknown format version: -4 when trying to view the index with solr (http://yer.path:8983/solr/admin ). But I can look at the same index with luke without a problem. - Naomi Dushay Stanford University Libraries ndushay at stanford.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From goodieboy at gmail.com Thu May 1 21:04:51 2008 From: goodieboy at gmail.com (Matt Mitchell) Date: Thu, 1 May 2008 21:04:51 -0400 Subject: [Blacklight-development] more little importer tricks In-Reply-To: References: <481A0CD2.8000902@jhu.edu> <763570460805011140v477e899bo401de34141000c80@mail.gmail.com> <481A14C6.60506@jhu.edu> Message-ID: Hi Naomi, Thanks for the detailed report! The _display fields should be multiValued and the current schema is here: http://blacklight.rubyforge.org/svn/trunk/rails/solr/conf/schema.xml Are you using the trunk version or the last release? Matt On Thu, May 1, 2008 at 8:50 PM, Naomi Dushay wrote: > I tried the importer just now (just pulled it from svn, too.) and hit a > few bumps also. I concur with the index.sh problems ... I just ended up > executing the java command directly from the command line. > > I believe the sample data has a record with two 020 subfield a values. > From the output of the importer on the sample file: > Adding record 8: u89 > Error indexing > org.apache.solr.common.SolrException: ERROR: multiple values encountered > for non multiValued field isbn_display: first='0877663637' > second='0877663343 (pbk.)' > at > org.apache.solr.update.DocumentBuilder.addSingleField(DocumentBuilder.java:67) > at > org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:88) > at > org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:118) > at > org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:101) > at SolrIndexer.addField(Unknown Source) > at SolrIndexer.addFields(Unknown Source) > at SolrIndexer.indexRecord(Unknown Source) > at MarcImporter.addToIndex(Unknown Source) > at MarcImporter.importRecords(Unknown Source) > at MarcImporter.main(Unknown Source) > Adding record 9: u144 > > The SOLR schema.xml file in blacklight/solr-home/conf directory says > that all *_display fields are NOT multiValued. > > To get the sample data to index without an error, it's just a matter of > changing one line in the blacklight.properties file: > > from: > field_list_25a = isbn_display, all, 020a > > to > field_list_25a = isbn_display, first, 020a > > HOWEVER, I am still getting a > > java.lang.RuntimeException: org.apache.lucene.index.CorruptIndexException: > Unknown format version: -4 > > when trying to view the index with solr (http://yer.path:8983/solr/admin). > > But I can look at the same index with luke without a problem. > > > - Naomi Dushay > Stanford University Libraries > ndushay at stanford.edu > > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ndushay at stanford.edu Fri May 2 12:20:51 2008 From: ndushay at stanford.edu (Naomi Dushay) Date: Fri, 2 May 2008 09:20:51 -0700 Subject: [Blacklight-development] more little importer tricks In-Reply-To: References: <481A0CD2.8000902@jhu.edu> <763570460805011140v477e899bo401de34141000c80@mail.gmail.com> <481A14C6.60506@jhu.edu> Message-ID: <85610CE3-985E-40C3-8913-2D6075F0660B@stanford.edu> Hi Matt, I got my blacklight code from the .tar.gz download file, not directly from svn. Do I need to pull some updates from the trunk? - Naomi On May 1, 2008, at 6:04 PM, Matt Mitchell wrote: > Hi Naomi, > > Thanks for the detailed report! The _display fields should be > multiValued and the current schema is here: > > http://blacklight.rubyforge.org/svn/trunk/rails/solr/conf/schema.xml > > Are you using the trunk version or the last release? > > Matt > > On Thu, May 1, 2008 at 8:50 PM, Naomi Dushay > wrote: > I tried the importer just now (just pulled it from svn, too.) and > hit a few bumps also. I concur with the index.sh problems ... I > just ended up executing the java command directly from the command > line. > > I believe the sample data has a record with two 020 subfield a > values. From the output of the importer on the sample file: > > Adding record 8: u89 > Error indexing > org.apache.solr.common.SolrException: ERROR: multiple values > encountered for non multiValued field isbn_display: > first='0877663637' second='0877663343 (pbk.)' > at > org > .apache > .solr.update.DocumentBuilder.addSingleField(DocumentBuilder.java:67) > at > org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java: > 88) > at > org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java: > 118) > at > org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java: > 101) > at SolrIndexer.addField(Unknown Source) > at SolrIndexer.addFields(Unknown Source) > at SolrIndexer.indexRecord(Unknown Source) > at MarcImporter.addToIndex(Unknown Source) > at MarcImporter.importRecords(Unknown Source) > at MarcImporter.main(Unknown Source) > Adding record 9: u144 > > The SOLR schema.xml file in blacklight/solr-home/conf directory > says that all *_display fields are NOT multiValued. > > To get the sample data to index without an error, it's just a matter > of changing one line in the blacklight.properties file: > > from: > field_list_25a = isbn_display, all, 020a > > to > field_list_25a = isbn_display, first, 020a > > HOWEVER, I am still getting a > > java.lang.RuntimeException: > org.apache.lucene.index.CorruptIndexException: Unknown format > version: -4 > > when trying to view the index with solr (http://yer.path:8983/solr/admin > ). > > But I can look at the same index with luke without a problem. > > > - Naomi Dushay > Stanford University Libraries > ndushay at stanford.edu > > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development Naomi Dushay ndushay at stanford.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From ndushay at stanford.edu Fri May 2 14:11:16 2008 From: ndushay at stanford.edu (Naomi Dushay) Date: Fri, 2 May 2008 11:11:16 -0700 Subject: [Blacklight-development] more little importer tricks - a fix? In-Reply-To: <85610CE3-985E-40C3-8913-2D6075F0660B@stanford.edu> References: <481A0CD2.8000902@jhu.edu> <763570460805011140v477e899bo401de34141000c80@mail.gmail.com> <481A14C6.60506@jhu.edu> <85610CE3-985E-40C3-8913-2D6075F0660B@stanford.edu> Message-ID: <3958F3E7-2701-418B-92D2-9FF740831AE2@stanford.edu> The java marc importer uses a different version of lucene than the .tar.gz of blacklight, and the index format is incompatible. unpacking the solr.war, the lucene jar files are all named lucene- blah-2007-05-20 the lucene jars in the java importer are lucene version 2.3.1, which seem to date from 2008-02-22 I swapped out the old lucene jar files for the new ones and made a new solr.war. Putting my new solr.war into jetty, then firing up solr ... makes everything happy! So, to summarize: 1. java importer: index.sh line breaks are not in (lin)ux format. (thanks Jonathan!) 2. java importer: sample file name in GettingStarted.txt has a typo (thanks Jonathan!) 3. java importer: sample data has a record with multiple 020 subfield a values. To get the data to index cleanly, you much change a line in blacklight.properties file: from field_list_25a = isbn_display, all, 020a to field_list_25a = isbn_display, first, 020a 4. blacklight itself: solr.war in the jetty/webapps directory has an older version of lucene jars which is incompatible with the lucene jars in the marc importer. The jars need to be the same. I changed the jars in the solr.war by unpacking the war, substituting the new lucene jar files, then repacking the war and putting the new solr.war in jetty/webapps. It might also work to just put the lucene jars from the solr.war into the java importer lib, replacing the ones that are there. Jonathan -- would you like to test these fixes on your system? hope this helps! - Naomi On May 2, 2008, at 9:20 AM, Naomi Dushay wrote: > Hi Matt, > > I got my blacklight code from the .tar.gz download file, not > directly from svn. Do I need to pull some updates from the trunk? > > - Naomi > > > On May 1, 2008, at 6:04 PM, Matt Mitchell wrote: >> Hi Naomi, >> >> Thanks for the detailed report! The _display fields should be >> multiValued and the current schema is here: >> >> http://blacklight.rubyforge.org/svn/trunk/rails/solr/conf/schema.xml >> >> Are you using the trunk version or the last release? >> >> Matt >> >> On Thu, May 1, 2008 at 8:50 PM, Naomi Dushay >> wrote: >> I tried the importer just now (just pulled it from svn, too.) and >> hit a few bumps also. I concur with the index.sh problems ... I >> just ended up executing the java command directly from the command >> line. >> >> I believe the sample data has a record with two 020 subfield a >> values. From the output of the importer on the sample file: >> >> Adding record 8: u89 >> Error indexing >> org.apache.solr.common.SolrException: ERROR: multiple values >> encountered for non multiValued field isbn_display: >> first='0877663637' second='0877663343 (pbk.)' >> at >> org >> .apache >> .solr.update.DocumentBuilder.addSingleField(DocumentBuilder.java:67) >> at >> org >> .apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:88) >> at >> org >> .apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java: >> 118) >> at >> org >> .apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java: >> 101) >> at SolrIndexer.addField(Unknown Source) >> at SolrIndexer.addFields(Unknown Source) >> at SolrIndexer.indexRecord(Unknown Source) >> at MarcImporter.addToIndex(Unknown Source) >> at MarcImporter.importRecords(Unknown Source) >> at MarcImporter.main(Unknown Source) >> Adding record 9: u144 >> >> The SOLR schema.xml file in blacklight/solr-home/conf directory >> says that all *_display fields are NOT multiValued. >> >> To get the sample data to index without an error, it's just a >> matter of changing one line in the blacklight.properties file: >> >> from: >> field_list_25a = isbn_display, all, 020a >> >> to >> field_list_25a = isbn_display, first, 020a >> >> HOWEVER, I am still getting a >> >> java.lang.RuntimeException: >> org.apache.lucene.index.CorruptIndexException: Unknown format >> version: -4 >> >> when trying to view the index with solr (http://yer.path:8983/solr/admin >> ). >> >> But I can look at the same index with luke without a problem. >> >> >> - Naomi Dushay >> Stanford University Libraries >> ndushay at stanford.edu >> >> >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development > > Naomi Dushay > ndushay at stanford.edu > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development Naomi Dushay ndushay at stanford.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From ndushay at stanford.edu Fri May 2 21:17:07 2008 From: ndushay at stanford.edu (Naomi Dushay) Date: Fri, 2 May 2008 18:17:07 -0700 Subject: [Blacklight-development] blacklight install from trunk References: Message-ID: I just pulled the latest and greatest code from the blacklight trunk. The README in the rails directory has lovely instructions. A few minor problems: 1. the config/database.example has "user" where it should have "username". Without this change, the "rake db:migrate" command doesn't work. 2. in the README, when it says "setup your database config file" it probably should say "set up config/database.yml to match your installation. The "development" portion is the one that will be used by default." 3. in the README, you need to be in the rails directory to start the web server with the command given. Woot woot! BTW, the new code from the trunk DID play nicely with the lucene index created by the java importer. - Naomi Begin forwarded message: > Hi Naomi, > > The latest UI/rails code is in trunk. The latest indexing code is in > blacklight_indexer. So you must be using the 0.2.0 release? > > Thanks, > Matt Naomi Dushay ndushay at stanford.edu From rh9ec at virginia.edu Mon May 5 12:14:08 2008 From: rh9ec at virginia.edu (Robert Haschart) Date: Mon, 05 May 2008 12:14:08 -0400 Subject: [Blacklight-development] Blacklight-development Digest, Vol 4, Issue 4 In-Reply-To: References: Message-ID: <481F3250.1040907@virginia.edu> Jonathon, Regarding the error you are seeing in indexing the 1000 record sample: Marc4j (as distributed) is not a very forgiving MARC record reader. If it encounters a structural flaw in a record when it is read it in, it throws an exception. If you attempt to have it continue from that point, it will start reading from the point the error occurred, assuming that a new record begins there. At which point it will fail horrendously. Over the past few days I have been looking at how it reads records, and have modified the marc4j library slightly so that if it encounters an error in reading one record, it can skip over that record and continue reading at the start of the next record. I will be checking in the modified library in the next day or so. Additionally, Bess has requested that I make myself available Thursday to help you work through any problems you are having with the blacklight importer. -Robert Haschart From rh9ec at virginia.edu Mon May 5 14:43:28 2008 From: rh9ec at virginia.edu (Robert Haschart) Date: Mon, 05 May 2008 14:43:28 -0400 Subject: [Blacklight-development] blacklight MARC importer In-Reply-To: <763570460804230542m5364eb84m64ac37826610bb38@mail.gmail.com> References: <763570460804210724n1978169fo270786d41906cc5@mail.gmail.com> <698BE1DD-385D-4EEE-89D2-5DF484ACE213@virginia.edu> <480E4C57.7040501@virginia.edu> <763570460804230542m5364eb84m64ac37826610bb38@mail.gmail.com> Message-ID: <481F5550.6060202@virginia.edu> Jason, I've taken a look at the Oregon State records, and the records are something of a mess. I wrote earlier today on the blacklight dev list that marc4j (as distributed) has a problem where if a malformed record is encountered, it stops reading at the point of the error, and then if you try to continue, it starts reading at the point of the error, expecting a new record to begin there. That must be why MarcImporter was reporting more records. Although at that point it is so confused, anything it reports is just nonsense. In the next day or so I plan to check in a modified marc4j that handles malformed data more robustly. I need to decide how much robustness to try for. Also Jonathon Rochkind made a good suggestion that it should somehow report about the malformed records rather than quietly skipping over them. Simply getting a reliable count of the records in the Oregon State data has proven to be a challenge. Since marc4j was having trouble reading them, my first thought was to run yaz-marcdump and grep for the '001' fields, which produced a count of 1272508, quite a bit less than the 1311996 records you were reporting. Trying the same for the '245' field produced a count of 1312463, quite a few more than you reported. Eventually I patched the marc4j library and ran it on the data and there are indeed 1311996 records, of which 76 have bad Marc record leaders rendering them unreadable and 39534 of them have no 001 field making them invalid records which even if they successfully ran through the indexer, each of them would replace the preceding one. -Robert Haschart From eos8d at virginia.edu Tue May 6 13:16:23 2008 From: eos8d at virginia.edu (Bess Sadler) Date: Tue, 6 May 2008 13:16:23 -0400 Subject: [Blacklight-development] how to get blacklight svn commit messages Message-ID: <8EF71CA4-D599-43B6-8C63-3158CB6A5CD2@virginia.edu> I've set up a mailing list called blacklight-commits that will receive all of the commit messages from the blacklight subversion server. I put these on a separate list so that people can easily choose whether or not to receive them. If you want to receive them, subscribe here: http://rubyforge.org/mailman/listinfo/blacklight-commits Let me know if you have any trouble! Bess Elizabeth (Bess) Sadler Research and Development Librarian Digital Scholarship Services Box 400129 Alderman Library University of Virginia Charlottesville, VA 22904 bess at virginia.edu (434) 243-2305 From ndushay at stanford.edu Wed May 7 19:46:51 2008 From: ndushay at stanford.edu (Naomi Dushay) Date: Wed, 7 May 2008 16:46:51 -0700 Subject: [Blacklight-development] more little importer tricks - another small fix In-Reply-To: References: <481A0CD2.8000902@jhu.edu> <763570460805011140v477e899bo401de34141000c80@mail.gmail.com> <481A14C6.60506@jhu.edu> <85610CE3-985E-40C3-8913-2D6075F0660B@stanford.edu> <3958F3E7-2701-418B-92D2-9FF740831AE2@stanford.edu> Message-ID: I forgot to mention that you need to add this line to your import.properties file: solr.data.dir = /yerBlacklightTopDir/rails/jetty/data (obviously with the right path for your installation) - this is because the latest version of blacklight doesn't expect the index to be in the default of solr-home/data, but instead in the rails/ jetty/data directory. - Naomi -------------- next part -------------- An HTML attachment was scrubbed... URL: From rochkind at jhu.edu Thu May 8 12:14:00 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 08 May 2008 12:14:00 -0400 Subject: [Blacklight-development] more little importer tricks - another small fix In-Reply-To: References: <481A0CD2.8000902@jhu.edu> <763570460805011140v477e899bo401de34141000c80@mail.gmail.com> <481A14C6.60506@jhu.edu> <85610CE3-985E-40C3-8913-2D6075F0660B@stanford.edu> <3958F3E7-2701-418B-92D2-9FF740831AE2@stanford.edu> Message-ID: <482326C8.5070506@jhu.edu> Thanks for this important note too Naomi! I never would have figured that out myself I don't think. Once I get a succesful import, I'll commit changes to the various READMEs so that they include all the necessary instructions to actually get it to work. If UVa could send me a "known-good" .mrc file including sufficient records to test out the UVa Blacklight setup (including musical records for the music interface), that would be convenient. That is, a file that UVa has gotten to succesfully import. Otherwise, I'm still using the file of around 1000 records that Bess at one point sent me, which I'd think should be "known good", but you never know. If I'm lucky, I'll get a succesful import at least from sample known-good data, if not from my own data, today. Jonathan Naomi Dushay wrote: > I forgot to mention that you need to add this line to your > import.properties file: > > solr.data.dir = /yerBlacklightTopDir/rails/jetty/data > > (obviously with the right path for your installation) > > - this is because the latest version of blacklight doesn't expect the > index to be in the default of solr-home/data, but instead in the > rails/jetty/data directory. > > - Naomi > ------------------------------------------------------------------------ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From rochkind at jhu.edu Thu May 8 12:26:24 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 08 May 2008 12:26:24 -0400 Subject: [Blacklight-development] indexing woes Message-ID: <482329B0.50704@jhu.edu> Okay, I'm having trouble even getting the 11 sample records supplied with the blacklight importer to index correctly! The error that appears in console is given below. Note that the indexer still reports "Indexed 11" at the end. But when I look at my SOLR stats, there are still 0 docs. I have tried both with and without Naomi's suggested change to importSamples.properties. As an enhancement request, it would be nice if the indexer told you how many records it ACTUALLY indexed. If an error prevented the indexing of some or all records, it should tell you that, not lie to you and say it indexed 11. :) I'm about to go to lunch, but maybe after lunch Rob would be available for a phone call or something? Jonathan ******************************** Adding record 1: u2 java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at SolrIndexer.handleCustom(Unknown Source) at SolrIndexer.indexRecord(Unknown Source) at MarcImporter.addToIndex(Unknown Source) at MarcImporter.importRecords(Unknown Source) at MarcImporter.main(Unknown Source) Caused by: java.lang.IncompatibleClassChangeError at BlacklightIndexer.getCallNumberCleaned(Unknown Source) ... 9 more java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at SolrIndexer.handleCustom(Unknown Source) at SolrIndexer.indexRecord(Unknown Source) at MarcImporter.addToIndex(Unknown Source) at MarcImporter.importRecords(Unknown Source) at MarcImporter.main(Unknown Source) Caused by: java.lang.IncompatibleClassChangeError at BlacklightIndexer.getCallNumberPrefix(Unknown Source) ... 9 more *********************************************************** -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From rochkind at jhu.edu Thu May 8 16:00:40 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 08 May 2008 16:00:40 -0400 Subject: [Blacklight-development] more indexing progress Message-ID: <48235BE8.6060008@jhu.edu> Okay, thanks to Bob's help I succesfully indexed the 11 sample records. And they show up in blacklight and everything. (I _think_ Naomi's addition to the properties file was NOT needed with most recent svn ups for both importer and blacklight.) When I try to index a sample package from Bess a long time ago, it errors out all over the place on "unable to parse record length" (but still lies and says it indexes 1k records. Anyway.) But maybe that was a bad mrc file. When I try to index a more recent sample file Bess gave---geez, that sample file is way too big Bess, it causes my testy test server to crash. (The test server has a habit of crashing when you do too much I/O too rapidly. Oops.) So, okay, I guess I'll go right to indexing my own sample mrc from my catalog... When I try on my own sample of 10k records, it gets to record 2771, and then it just hangs forever. Could be a problem with my server though. Hmm. Jonathan -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From eos8d at virginia.edu Fri May 9 11:49:22 2008 From: eos8d at virginia.edu (Bess Sadler) Date: Fri, 9 May 2008 11:49:22 -0400 Subject: [Blacklight-development] where to get catalog records to index Message-ID: <47478037-68EA-4339-8B09-A2049F8012BB@virginia.edu> Since the legal status of our UVA catalog records is somewhat questionable, we need to come up with a test set that we can distribute for indexing. I propose we use the Library of Congress set that the incomparable Casey Bisson purchased and provided to the community for precisely this purpose. They are downloadable here: http://www.archive.org/details/marc_records_scriblio_net I suggest we start running indexing tests with these. And there's even one (part29.dat) that's only 19M... perfect for getting started with. Bess Elizabeth (Bess) Sadler Research and Development Librarian Digital Scholarship Services Box 400129 Alderman Library University of Virginia Charlottesville, VA 22904 bess at virginia.edu (434) 243-2305 From rochkind at jhu.edu Fri May 9 12:07:21 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Fri, 09 May 2008 12:07:21 -0400 Subject: [Blacklight-development] where to get catalog records to index In-Reply-To: <47478037-68EA-4339-8B09-A2049F8012BB@virginia.edu> References: <47478037-68EA-4339-8B09-A2049F8012BB@virginia.edu> Message-ID: <482476B9.6030206@jhu.edu> That's a great plan. For an actual getting started set, I'd suggest that someone try to make a ~10k record set, 19M is still way too big for just getting started and testing and tuning your indexing, takes too long to re-index. Ideally that 10k sample set would have musical records with data in MARC that could be taken advantage of by the Blacklight music library interface, so that could be tested/tuned too, since some of us are interested in that specifically. And actually, there's another big problem with that as a sample set. It doesn't include any item-level data. Holdings data. We need a sample set with that too. I suppose we could add "fake" holdings data in the same data fields that UVa uses _to_ an LC sample set, as a sample. But we need it, to put that stuff through it's paces too. So now I've made this more complicated then you hoped. :) But I do think we need all those things to have a useful sample set. Jonathan Bess Sadler wrote: > Since the legal status of our UVA catalog records is somewhat > questionable, we need to come up with a test set that we can > distribute for indexing. I propose we use the Library of Congress set > that the incomparable Casey Bisson purchased and provided to the > community for precisely this purpose. > > They are downloadable here: > > http://www.archive.org/details/marc_records_scriblio_net > > I suggest we start running indexing tests with these. And there's even > one (part29.dat) that's only 19M... perfect for getting started with. > > Bess > > > Elizabeth (Bess) Sadler > Research and Development Librarian > Digital Scholarship Services > Box 400129 > Alderman Library > University of Virginia > Charlottesville, VA 22904 > > bess at virginia.edu > (434) 243-2305 > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu From ndushay at stanford.edu Fri May 16 14:19:11 2008 From: ndushay at stanford.edu (Naomi Dushay) Date: Fri, 16 May 2008 11:19:11 -0700 Subject: [Blacklight-development] solr dir within and without rails dir? In-Reply-To: <47478037-68EA-4339-8B09-A2049F8012BB@virginia.edu> References: <47478037-68EA-4339-8B09-A2049F8012BB@virginia.edu> Message-ID: I was just examining the blacklight code, and it appears the code I pulled down from SVN has a solr directory both inside the rails directory and also at the same level as the rails directory. Is this intentional? - Naomi (who suddenly is thinking of the Beatles song "Across the Universe" ... within you and without you ... From eos8d at virginia.edu Fri May 16 15:11:21 2008 From: eos8d at virginia.edu (Elizabeth Sadler) Date: Fri, 16 May 2008 15:11:21 -0400 Subject: [Blacklight-development] solr dir within and without rails dir? In-Reply-To: References: <47478037-68EA-4339-8B09-A2049F8012BB@virginia.edu> Message-ID: <85E0D637-0960-4488-B952-FB474BBC7B7B@virginia.edu> Hi, Naomi. Well, you're right, it probably shouldn't come that way by default. The thing is, we haven't really settled on a good "out of the box" way to run blacklight. For testing purposes, you can run a copy of solr locally, and that's what that solr directory inside the rails directory is doing there. I think its also handy for running unit tests. However, you get greatly increased performance if you can run solr on a separate server from the rails side of things, and to make that easy we've split out a solr directory that could easily be checked out to that other server. So for me, if I'm just playing around trying to see if something runs and is passing tests, I fire it up and use the default (internal) solr. If I'm for reals setting up a new installation, I check out rails on one machine, I check out the external solr on a different machine, configure them and make them talk to each other. Is it too confusing to have both of them in the checkout, though? Would it help if we put something in the readme? I'm in the process of writing up a pretty detailed description of how we set up a production instance of blacklight... maybe that will also help? Bess On 16-May-08, at 2:19 PM, Naomi Dushay wrote: > I was just examining the blacklight code, and it appears the code I > pulled down from SVN has a solr directory both inside the rails > directory and also at the same level as the rails directory. Is this > intentional? > > - Naomi > (who suddenly is thinking of the Beatles song "Across the > Universe" ... within you and without you ... > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development Elizabeth (Bess) Sadler Research and Development Librarian Digital Scholarship Services (DSS) Box 400129 Alderman Library University of Virginia Charlottesville, VA 22904 bess at virginia.edu (434) 243-2305 From ndushay at stanford.edu Fri May 16 17:08:07 2008 From: ndushay at stanford.edu (Naomi Dushay) Date: Fri, 16 May 2008 14:08:07 -0700 Subject: [Blacklight-development] solr dir within and without rails dir? In-Reply-To: <85E0D637-0960-4488-B952-FB474BBC7B7B@virginia.edu> References: <47478037-68EA-4339-8B09-A2049F8012BB@virginia.edu> <85E0D637-0960-4488-B952-FB474BBC7B7B@virginia.edu> Message-ID: <3520446B-6B8E-4824-A0E5-EF1C28ABE321@stanford.edu> Bess, This is helpful information. I think explanations in the readme is probably sufficient. At this "get the prototype running, check out features, use local data" stage, it was more a point of curiosity as to why there were two solr directories. Like so many others, we are evaluating blacklight and other next generation disocvery environments, so understanding how to do things such as configure code for your institution's practices, for production vs. testing and so on is really helpful. - Naomi who got the incorrect title of the Beatles song: "Within You Without You" is the correct title On May 16, 2008, at 12:11 PM, Elizabeth Sadler wrote: > Hi, Naomi. > > Well, you're right, it probably shouldn't come that way by default. > The thing is, we haven't really settled on a good "out of the box" > way to run blacklight. For testing purposes, you can run a copy of > solr locally, and that's what that solr directory inside the rails > directory is doing there. I think its also handy for running unit > tests. However, you get greatly increased performance if you can run > solr on a separate server from the rails side of things, and to make > that easy we've split out a solr directory that could easily be > checked out to that other server. > > So for me, if I'm just playing around trying to see if something > runs and is passing tests, I fire it up and use the default > (internal) solr. If I'm for reals setting up a new installation, I > check out rails on one machine, I check out the external solr on a > different machine, configure them and make them talk to each other. > > Is it too confusing to have both of them in the checkout, though? > Would it help if we put something in the readme? I'm in the process > of writing up a pretty detailed description of how we set up a > production instance of blacklight... maybe that will also help? > > Bess > > On 16-May-08, at 2:19 PM, Naomi Dushay wrote: > >> I was just examining the blacklight code, and it appears the code I >> pulled down from SVN has a solr directory both inside the rails >> directory and also at the same level as the rails directory. Is this >> intentional? >> >> - Naomi >> (who suddenly is thinking of the Beatles song "Across the >> Universe" ... within you and without you ... >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development > > Elizabeth (Bess) Sadler > Research and Development Librarian > Digital Scholarship Services (DSS) > Box 400129 > Alderman Library > University of Virginia > Charlottesville, VA 22904 > > bess at virginia.edu > (434) 243-2305 > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development Naomi Dushay ndushay at stanford.edu From goodieboy at gmail.com Fri May 16 18:44:29 2008 From: goodieboy at gmail.com (Matt Mitchell) Date: Fri, 16 May 2008 18:44:29 -0400 Subject: [Blacklight-development] solr dir within and without rails dir? In-Reply-To: <3520446B-6B8E-4824-A0E5-EF1C28ABE321@stanford.edu> References: <47478037-68EA-4339-8B09-A2049F8012BB@virginia.edu> <85E0D637-0960-4488-B952-FB474BBC7B7B@virginia.edu> <3520446B-6B8E-4824-A0E5-EF1C28ABE321@stanford.edu> Message-ID: Just wanted to jump in and explain why I made 2 solr "directories". There were a few times when our solr indexing schema/config was out of sync with our front-end solr. I had made some changes to the front-end solr files, but because our indexer was not using the same/up-to-date solr we ran into some problems. So my idea was for the indexer to use the same solr as the front-end. And by that, I mean SVN externals. So, trunk/rails/solr is an SVN externals link to trunk/solr. But we still have to SVN externals the blacklight_indexer stuff. If you look at the following directory, you won't see "solr" in the listing. It gets pulled in when you do a checkout/update: http://blacklight.rubyforge.org/svn/trunk/rails/ Does that help? Bob, if you want... you can sync up your indexer's solr by doing: export SVN_EDITOR= cd blacklight_importer svn pe svn:externals . When your text editor opens, add this to the first line: solr http://blacklight.rubyforge.org/svn/trunk/solr Do a commit and you'll be good to go! Then you can start using the blacklight_importer/solr as your solr home. Any time I make a change and commit, you'll download the changes when you run an update. I've already added svn:ignore to the solr/data dir. Matt On Fri, May 16, 2008 at 5:08 PM, Naomi Dushay wrote: > Bess, > > This is helpful information. I think explanations in the readme is > probably sufficient. At this "get the prototype running, check out > features, use local data" stage, it was more a point of curiosity as to why > there were two solr directories. > > Like so many others, we are evaluating blacklight and other next generation > disocvery environments, so understanding how to do things such as configure > code for your institution's practices, for production vs. testing and so on > is really helpful. > > - Naomi > who got the incorrect title of the Beatles song: "Within You Without You" > is the correct title > > > On May 16, 2008, at 12:11 PM, Elizabeth Sadler wrote: > > Hi, Naomi. >> >> Well, you're right, it probably shouldn't come that way by default. The >> thing is, we haven't really settled on a good "out of the box" way to run >> blacklight. For testing purposes, you can run a copy of solr locally, and >> that's what that solr directory inside the rails directory is doing there. I >> think its also handy for running unit tests. However, you get greatly >> increased performance if you can run solr on a separate server from the >> rails side of things, and to make that easy we've split out a solr directory >> that could easily be checked out to that other server. >> >> So for me, if I'm just playing around trying to see if something runs and >> is passing tests, I fire it up and use the default (internal) solr. If I'm >> for reals setting up a new installation, I check out rails on one machine, I >> check out the external solr on a different machine, configure them and make >> them talk to each other. >> >> Is it too confusing to have both of them in the checkout, though? Would it >> help if we put something in the readme? I'm in the process of writing up a >> pretty detailed description of how we set up a production instance of >> blacklight... maybe that will also help? >> >> Bess >> >> On 16-May-08, at 2:19 PM, Naomi Dushay wrote: >> >> I was just examining the blacklight code, and it appears the code I >>> pulled down from SVN has a solr directory both inside the rails >>> directory and also at the same level as the rails directory. Is this >>> intentional? >>> >>> - Naomi >>> (who suddenly is thinking of the Beatles song "Across the >>> Universe" ... within you and without you ... >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >> >> Elizabeth (Bess) Sadler >> Research and Development Librarian >> Digital Scholarship Services (DSS) >> Box 400129 >> Alderman Library >> University of Virginia >> Charlottesville, VA 22904 >> >> bess at virginia.edu >> (434) 243-2305 >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> > > Naomi Dushay > ndushay at stanford.edu > > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eos8d at virginia.edu Tue May 20 10:22:36 2008 From: eos8d at virginia.edu (Elizabeth Sadler) Date: Tue, 20 May 2008 10:22:36 -0400 Subject: [Blacklight-development] Question on BlackLight OPAC In-Reply-To: <8E9135F9EBC359449F03366E52F1AFAE02EF01@JDLSERVER01.email.jackson.lib.mi.us> References: <8E9135F9EBC359449F03366E52F1AFAE02EF01@JDLSERVER01.email.jackson.lib.mi.us> Message-ID: Hi, Daniel. You should be able to install blacklight on a windows machine by following the standard installation instructions at http://blacklight.rubyforge.org/svn/trunk/rails/README , you'll just need to adjust them for installing ruby on rails etc on windows. There are lots of instructions online for how to install ruby on windows, so give it a try, adjusting the installation as needed. You might also want to join the mailing list (which I'm copying here) to ask questions if you get stuck. We do have other people working with windows, so I know it's possible. Please let us know how it goes! Bess On 20-May-08, at 9:38 AM, Kuras, Daniel wrote: > Is there a set of instructions for installing this on a Windows > server. > > Thanks > > Daniel Kuras > Systems Operator > Jackson District Library > Jackson, MI > voice = 517-788-4099 > FAX = 517-788-6024 > Elizabeth (Bess) Sadler Research and Development Librarian Digital Scholarship Services (DSS) Box 400129 Alderman Library University of Virginia Charlottesville, VA 22904 bess at virginia.edu (434) 243-2305 From ndushay at stanford.edu Tue May 20 17:47:30 2008 From: ndushay at stanford.edu (Naomi Dushay) Date: Tue, 20 May 2008 14:47:30 -0700 Subject: [Blacklight-development] holdings in batch for locations, call numbers Message-ID: <3E9FC8E4-E805-47A0-ABE0-285DA425C0BC@stanford.edu> Has anyone implemented batch loading of information from holdings records? I believe I've heard of some sites putting specific holdings fields into particular MARC bib fields. Any other approaches in use or underway or thought about? Reason: We've had interest here in having facets for both library locations (there are about 20 physical library locations on campus) and for call numbers (presumably first letter major divisions and second letter subdivisions - we use LC). I suspect we will also want to browse by full call number within and across physical locations. Clearly, to do the above, we must batch load our locations and call numbers. This information, as I understand it, resides in our item/ holdings records, not our bib records. So we need a way to batch load holdings information -- at least for location and call number. We would want to continue to provide availability, due date and possibly reserve information with a realtime query. Naomi Dushay ndushay at stanford.edu From rochkind at jhu.edu Tue May 20 17:59:37 2008 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Tue, 20 May 2008 17:59:37 -0400 Subject: [Blacklight-development] holdings in batch for locations, call numbers In-Reply-To: <3E9FC8E4-E805-47A0-ABE0-285DA425C0BC@stanford.edu> References: <3E9FC8E4-E805-47A0-ABE0-285DA425C0BC@stanford.edu> Message-ID: <483349C9.90003@jhu.edu> The UVa blacklight is putting holdings in the MARC bib, and indexing those, and it does allow you to limit by location, yes. I am planning on doing the same. Serials holdings pose a special challenge though, that I don't thin UVa is dealing with yet. It's not an insurmountable challenge, but it's tricky, largely because I don't think you can generally put serials holdings in a marc bib, at least not without loss of meaning. I don't think anyone ("anyone" basically being UVa; nobody else is any more than playing with Blacklight yet) is actually indexing based on both bib and MARC Holding records. Right now the indexer only looks at MARC Bib records, but in UVa's case they do include (non-serial) location and call number information. Jonathan Naomi Dushay wrote: > Has anyone implemented batch loading of information from holdings > records? I believe I've heard of some sites putting specific holdings > fields into particular MARC bib fields. Any other approaches in use > or underway or thought about? > > Reason: > > We've had interest here in having facets for both library locations > (there are about 20 physical library locations on campus) and for call > numbers (presumably first letter major divisions and second letter > subdivisions - we use LC). I suspect we will also want to browse by > full call number within and across physical locations. > > Clearly, to do the above, we must batch load our locations and call > numbers. This information, as I understand it, resides in our > item/holdings records, not our bib records. > > So we need a way to batch load holdings information -- at least for > location and call number. We would want to continue to provide > availability, due date and possibly reserve information with a > realtime query. > > > Naomi Dushay > ndushay at stanford.edu > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development -- Jonathan Rochkind Digital Services Software Engineer The Sheridan Libraries Johns Hopkins University 410.516.8886 rochkind (at) jhu.edu