From chapman.lists at gmail.com Tue Mar 3 17:22:48 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Tue, 03 Mar 2009 17:22:48 -0500 Subject: [Blacklight-development] blacklight with mod_rails (Passenger) Message-ID: <49ADADB8.1050500@gmail.com> Hello all blacklight newb here. I am trying to install the demo , and run it using mod_rails (Passenger) + apache. I am able to install as per the installation document, my problem is that when I attempt to access the url (RubyBasURI /discovery) I get the following error message and trace. Any help is greatly appreciated. Error message: undefined method `cache_template_loading=' for ActionView::Base:Class Exception class: NoMethodError Here is the trace from Passenger 0 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 530 in `send' 1 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 530 in `initialize_framework_settings' 2 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 529 in `each' 3 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 529 in `initialize_framework_settings' 4 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 526 in `each' 5 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 526 in `initialize_framework_settings' 6 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 154 in `process' 7 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 112 in `send' 8 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 112 in `run' 9 ./config/environment.rb 15 10 /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb 31 in `gem_original_require' 11 /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb 31 in `require' 12 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb 142 in `spawn_application!' 13 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 179 in `report_app_init_status' 14 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb 136 in `spawn_application!' 15 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 163 in `safe_fork' 16 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 161 in `fork' 17 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 161 in `safe_fork' 18 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb 132 in `spawn_application!' 19 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 163 in `safe_fork' 20 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 161 in `fork' 21 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 161 in `safe_fork' 22 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb 131 in `spawn_application!' 23 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb 234 in `spawn_rails_application' 24 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb 217 in `synchronize' 25 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb 217 in `spawn_rails_application' 26 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb 126 in `spawn_application' 27 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb 251 in `handle_spawn_application' 28 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb 317 in `__send__' 29 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb 317 in `main_loop' 30 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb 168 in `start_synchronously' 31 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/bin/passenger-spawn-server 46 32 /usr/bin/passenger-spawn-server 19 in `load' 33 /usr/bin/passenger-spawn-server 19 From goodieboy at gmail.com Tue Mar 3 20:43:44 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 3 Mar 2009 20:43:44 -0500 Subject: [Blacklight-development] blacklight with mod_rails (Passenger) In-Reply-To: <49ADADB8.1050500@gmail.com> References: <49ADADB8.1050500@gmail.com> Message-ID: Hi Vernon, What version of Rails are you using? What happens when you comment that line out? Matt On Tue, Mar 3, 2009 at 5:22 PM, Vernon Chapman wrote: > Hello all blacklight newb here. > > I am trying to install the demo , and run it using mod_rails (Passenger) + > apache. I am able to install as per the installation document, my problem is > that when I attempt to access the url (RubyBasURI /discovery) I get the > following error message and trace. Any help is greatly appreciated. > > > Error message: > > undefined method `cache_template_loading=' for ActionView::Base:Class > > Exception class: > NoMethodError > > > Here is the trace from Passenger > > 0 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 530 > in `send' > 1 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 530 > in `initialize_framework_settings' > 2 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 529 > in `each' > 3 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 529 > in `initialize_framework_settings' > 4 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 526 > in `each' > 5 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 526 > in `initialize_framework_settings' > 6 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 154 > in `process' > 7 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 112 > in `send' > 8 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb 112 > in `run' > 9 ./config/environment.rb 15 > 10 /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb 31 > in `gem_original_require' > 11 /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb 31 > in `require' > 12 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 142 in `spawn_application!' > 13 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 179 > in `report_app_init_status' > 14 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 136 in `spawn_application!' > 15 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 163 > in `safe_fork' > 16 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 161 > in `fork' > 17 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 161 > in `safe_fork' > 18 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 132 in `spawn_application!' > 19 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 163 > in `safe_fork' > 20 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 161 > in `fork' > 21 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb 161 > in `safe_fork' > 22 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 131 in `spawn_application!' > 23 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 234 in `spawn_rails_application' > 24 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 217 in `synchronize' > 25 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 217 in `spawn_rails_application' > 26 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 126 in `spawn_application' > 27 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 251 in `handle_spawn_application' > 28 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb > 317 in `__send__' > 29 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb > 317 in `main_loop' > 30 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb > 168 in `start_synchronously' > 31 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/bin/passenger-spawn-server > 46 > 32 /usr/bin/passenger-spawn-server 19 in `load' > 33 /usr/bin/passenger-spawn-server 19 > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chapman.lists at gmail.com Tue Mar 3 21:52:52 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Tue, 03 Mar 2009 21:52:52 -0500 Subject: [Blacklight-development] blacklight with mod_rails (Passenger) In-Reply-To: References: <49ADADB8.1050500@gmail.com> Message-ID: <49ADED04.6010604@gmail.com> Hi Matt, I am using rails version 2.2.2 When I comment out that line I get an error saying No route matches "/discovery" with {:method=>:get} My VirtualHost config reads ServerName fatboy.local ServerAdmin webmaster at localhost DocumentRoot /opt/www ........... RailsBaseURI /discovery So in order to access the demo app I want the url to be http://localhost/discovery/ Not sure what to do now. Thanks for your help Matt Mitchell wrote: > Hi Vernon, > > What version of Rails are you using? What happens when you comment > that line out? > > Matt > > On Tue, Mar 3, 2009 at 5:22 PM, Vernon Chapman > > wrote: > > Hello all blacklight newb here. > > I am trying to install the demo , and run it using mod_rails > (Passenger) + apache. I am able to install as per the installation > document, my problem is that when I attempt to access the url > (RubyBasURI /discovery) I get the following error message and > trace. Any help is greatly appreciated. > > > Error message: > > undefined method `cache_template_loading=' for > ActionView::Base:Class > > Exception class: > NoMethodError > > > Here is the trace from Passenger > > 0 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 530 in `send' > 1 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 530 in `initialize_framework_settings' > 2 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 529 in `each' > 3 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 529 in `initialize_framework_settings' > 4 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 526 in `each' > 5 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 526 in `initialize_framework_settings' > 6 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 154 in `process' > 7 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 112 in `send' > 8 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 112 in `run' > 9 ./config/environment.rb 15 > 10 /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb > 31 in `gem_original_require' > 11 /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb > 31 in `require' > 12 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 142 in `spawn_application!' > 13 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 179 in `report_app_init_status' > 14 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 136 in `spawn_application!' > 15 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 163 in `safe_fork' > 16 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 161 in `fork' > 17 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 161 in `safe_fork' > 18 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 132 in `spawn_application!' > 19 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 163 in `safe_fork' > 20 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 161 in `fork' > 21 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 161 in `safe_fork' > 22 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 131 in `spawn_application!' > 23 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 234 in `spawn_rails_application' > 24 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 217 in `synchronize' > 25 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 217 in `spawn_rails_application' > 26 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 126 in `spawn_application' > 27 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 251 in `handle_spawn_application' > 28 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb > 317 in `__send__' > 29 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb > 317 in `main_loop' > 30 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb > 168 in `start_synchronously' > 31 > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/bin/passenger-spawn-server > 46 > 32 /usr/bin/passenger-spawn-server 19 in `load' > 33 /usr/bin/passenger-spawn-server 19 > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > ------------------------------------------------------------------------ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From goodieboy at gmail.com Tue Mar 3 22:16:56 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 3 Mar 2009 22:16:56 -0500 Subject: [Blacklight-development] blacklight with mod_rails (Passenger) In-Reply-To: <49ADED04.6010604@gmail.com> References: <49ADADB8.1050500@gmail.com> <49ADED04.6010604@gmail.com> Message-ID: Hi Vernon, ... just found this: http://github.com/rails/rails/commit/a87462afcb6c642e59bfcd2e11e3351e2b718c38 Try putting this in your environment.rb file, inside the config initialize block: config.action_controller.relative_url_root = 'discover' Here's a helpful little note about the line you commented out: http://forums.mor.ph/forums/8/topics/191 - looks like we need to comment that out from the source! Matt On Tue, Mar 3, 2009 at 9:52 PM, Vernon Chapman wrote: > Hi Matt, > > I am using rails version 2.2.2 > > When I comment out that line I get an error saying > No route matches "/discovery" with {:method=>:get} > > My VirtualHost config reads > > > ServerName fatboy.local > ServerAdmin webmaster at localhost > DocumentRoot /opt/www > > ........... > > RailsBaseURI /discovery > > > > So in order to access the demo app I want the > url to be http://localhost/discovery/ > > > Not sure what to do now. > > Thanks for your help > > > > > Matt Mitchell wrote: > >> Hi Vernon, >> >> What version of Rails are you using? What happens when you comment that >> line out? >> >> Matt >> >> On Tue, Mar 3, 2009 at 5:22 PM, Vernon Chapman > chapman.lists at gmail.com>> wrote: >> >> Hello all blacklight newb here. >> >> I am trying to install the demo , and run it using mod_rails >> (Passenger) + apache. I am able to install as per the installation >> document, my problem is that when I attempt to access the url >> (RubyBasURI /discovery) I get the following error message and >> trace. Any help is greatly appreciated. >> >> >> Error message: >> >> undefined method `cache_template_loading=' for >> ActionView::Base:Class >> >> Exception class: >> NoMethodError >> >> >> Here is the trace from Passenger >> >> 0 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 530 in `send' >> 1 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 530 in `initialize_framework_settings' >> 2 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 529 in `each' >> 3 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 529 in `initialize_framework_settings' >> 4 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 526 in `each' >> 5 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 526 in `initialize_framework_settings' >> 6 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 154 in `process' >> 7 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 112 in `send' >> 8 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 112 in `run' >> 9 ./config/environment.rb 15 10 >> /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb 31 >> in `gem_original_require' >> 11 /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb >> 31 in `require' >> 12 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb >> 142 in `spawn_application!' >> 13 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 179 in `report_app_init_status' >> 14 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb >> 136 in `spawn_application!' >> 15 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 163 in `safe_fork' >> 16 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 161 in `fork' >> 17 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 161 in `safe_fork' >> 18 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb >> 132 in `spawn_application!' >> 19 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 163 in `safe_fork' >> 20 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 161 in `fork' >> 21 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 161 in `safe_fork' >> 22 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb >> 131 in `spawn_application!' >> 23 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb >> 234 in `spawn_rails_application' >> 24 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb >> 217 in `synchronize' >> 25 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb >> 217 in `spawn_rails_application' >> 26 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb >> 126 in `spawn_application' >> 27 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb >> 251 in `handle_spawn_application' >> 28 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb >> 317 in `__send__' >> 29 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb >> 317 in `main_loop' >> 30 >> >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb >> 168 in `start_synchronously' >> 31 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/bin/passenger-spawn-server >> 46 32 /usr/bin/passenger-spawn-server 19 in >> `load' >> 33 /usr/bin/passenger-spawn-server 19 >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eos8d at virginia.edu Tue Mar 3 22:35:56 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Tue, 3 Mar 2009 22:35:56 -0500 Subject: [Blacklight-development] blacklight with mod_rails (Passenger) References: Message-ID: <58B18DB7-7D58-4212-8504-79E1888546B3@virginia.edu> Vernon, is your rails app actually installed in /opt/www? I think your DocumentRoot needs to point at the public directory of your rails app. E.g.: DocumentRoot /usr/local/projects/blacklight/current/public Where does your rails app live on the filesystem? If this doesn't work, please write back with: - where on the filesystem your rails app is deployed - the full text of your VirtualHost config Thanks! Bess On 3-Mar-09, at 9:52 PM, Vernon Chapman wrote: > Hi Matt, > > I am using rails version 2.2.2 > > When I comment out that line I get an error saying > No route matches "/discovery" with {:method=>:get} > > My VirtualHost config reads > > > ServerName fatboy.local > ServerAdmin webmaster at localhost > DocumentRoot /opt/www > > ........... > > RailsBaseURI /discovery > > > > So in order to access the demo app I want the > url to be http://localhost/discovery/ > > > Not sure what to do now. > > Thanks for your help > > > > > Matt Mitchell wrote: >> Hi Vernon, >> >> What version of Rails are you using? What happens when you comment >> that line out? >> >> Matt >> >> On Tue, Mar 3, 2009 at 5:22 PM, Vernon Chapman >> > wrote: >> >> Hello all blacklight newb here. >> >> I am trying to install the demo , and run it using mod_rails >> (Passenger) + apache. I am able to install as per the installation >> document, my problem is that when I attempt to access the url >> (RubyBasURI /discovery) I get the following error message and >> trace. Any help is greatly appreciated. >> >> >> Error message: >> >> undefined method `cache_template_loading=' for >> ActionView::Base:Class >> >> Exception class: >> NoMethodError >> >> >> Here is the trace from Passenger >> >> 0 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 530 in `send' >> 1 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 530 in `initialize_framework_settings' >> 2 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 529 in `each' >> 3 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 529 in `initialize_framework_settings' >> 4 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 526 in `each' >> 5 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 526 in `initialize_framework_settings' >> 6 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 154 in `process' >> 7 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 112 in `send' >> 8 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >> 112 in `run' >> 9 ./config/environment.rb 15 >> 10 /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb >> 31 in `gem_original_require' >> 11 /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb >> 31 in `require' >> 12 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/ >> application_spawner.rb >> 142 in `spawn_application!' >> 13 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 179 in `report_app_init_status' >> 14 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/ >> application_spawner.rb >> 136 in `spawn_application!' >> 15 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 163 in `safe_fork' >> 16 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 161 in `fork' >> 17 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 161 in `safe_fork' >> 18 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/ >> application_spawner.rb >> 132 in `spawn_application!' >> 19 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 163 in `safe_fork' >> 20 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 161 in `fork' >> 21 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >> 161 in `safe_fork' >> 22 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/ >> application_spawner.rb >> 131 in `spawn_application!' >> 23 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >> spawn_manager.rb >> 234 in `spawn_rails_application' >> 24 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >> spawn_manager.rb >> 217 in `synchronize' >> 25 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >> spawn_manager.rb >> 217 in `spawn_rails_application' >> 26 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >> spawn_manager.rb >> 126 in `spawn_application' >> 27 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >> spawn_manager.rb >> 251 in `handle_spawn_application' >> 28 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >> abstract_server.rb >> 317 in `__send__' >> 29 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >> abstract_server.rb >> 317 in `main_loop' >> 30 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >> abstract_server.rb >> 168 in `start_synchronously' >> 31 >> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/bin/passenger-spawn- >> server >> 46 >> 32 /usr/bin/passenger-spawn-server 19 in `load' >> 33 /usr/bin/passenger-spawn-server 19 >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development From chapman.lists at gmail.com Tue Mar 3 23:10:43 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Tue, 03 Mar 2009 23:10:43 -0500 Subject: [Blacklight-development] blacklight with mod_rails (Passenger) In-Reply-To: References: <49ADADB8.1050500@gmail.com> <49ADED04.6010604@gmail.com> Message-ID: <49ADFF43.7080302@gmail.com> Matt, Thanks that did the trick. I am able to see the index view of the demo app. Now the problem is that if I click on the link in the results that points at the catalog I am getting a 404 not sure what that is about but at least I have it running. Thanks Matt Mitchell wrote: > Hi Vernon, > > ... just found this: > http://github.com/rails/rails/commit/a87462afcb6c642e59bfcd2e11e3351e2b718c38 > > Try putting this in your environment.rb file, inside the config > initialize block: > > config.action_controller.relative_url_root = 'discover' > > Here's a helpful little note about the line you commented out: > http://forums.mor.ph/forums/8/topics/191 - looks like we need to > comment that out from the source! > > Matt > > On Tue, Mar 3, 2009 at 9:52 PM, Vernon Chapman > > wrote: > > Hi Matt, > > I am using rails version 2.2.2 > > When I comment out that line I get an error saying > No route matches "/discovery" with {:method=>:get} > > My VirtualHost config reads > > > ServerName fatboy.local > ServerAdmin webmaster at localhost > DocumentRoot /opt/www > > ........... > > RailsBaseURI /discovery > > > > So in order to access the demo app I want the > url to be http://localhost/discovery/ > > > Not sure what to do now. > > Thanks for your help > > > > > Matt Mitchell wrote: > > Hi Vernon, > > What version of Rails are you using? What happens when you > comment that line out? > > Matt > > On Tue, Mar 3, 2009 at 5:22 PM, Vernon Chapman > > >> wrote: > > Hello all blacklight newb here. > > I am trying to install the demo , and run it using mod_rails > (Passenger) + apache. I am able to install as per the > installation > document, my problem is that when I attempt to access the url > (RubyBasURI /discovery) I get the following error message and > trace. Any help is greatly appreciated. > > > Error message: > > undefined method `cache_template_loading=' for > ActionView::Base:Class > > Exception class: > NoMethodError > > > Here is the trace from Passenger > > 0 > /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 530 in `send' > 1 > /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 530 in `initialize_framework_settings' > 2 > /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 529 in `each' > 3 > /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 529 in `initialize_framework_settings' > 4 > /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 526 in `each' > 5 > /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 526 in `initialize_framework_settings' > 6 > /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 154 in `process' > 7 > /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 112 in `send' > 8 > /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb > 112 in `run' > 9 ./config/environment.rb 15 10 > /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb > 31 in `gem_original_require' > 11 > /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb > 31 in `require' > 12 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 142 in `spawn_application!' > 13 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 179 in `report_app_init_status' > 14 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 136 in `spawn_application!' > 15 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 163 in `safe_fork' > 16 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 161 in `fork' > 17 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 161 in `safe_fork' > 18 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 132 in `spawn_application!' > 19 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 163 in `safe_fork' > 20 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 161 in `fork' > 21 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb > 161 in `safe_fork' > 22 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/application_spawner.rb > 131 in `spawn_application!' > 23 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 234 in `spawn_rails_application' > 24 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 217 in `synchronize' > 25 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 217 in `spawn_rails_application' > 26 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 126 in `spawn_application' > 27 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/spawn_manager.rb > 251 in `handle_spawn_application' > 28 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb > 317 in `__send__' > 29 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb > 317 in `main_loop' > 30 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/abstract_server.rb > 168 in `start_synchronously' > 31 > > /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/bin/passenger-spawn-server > 46 32 /usr/bin/passenger-spawn-server > 19 in `load' > 33 /usr/bin/passenger-spawn-server 19 > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > > > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > ------------------------------------------------------------------------ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From chapman.lists at gmail.com Tue Mar 3 23:22:03 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Tue, 03 Mar 2009 23:22:03 -0500 Subject: [Blacklight-development] blacklight with mod_rails (Passenger) In-Reply-To: <58B18DB7-7D58-4212-8504-79E1888546B3@virginia.edu> References: <58B18DB7-7D58-4212-8504-79E1888546B3@virginia.edu> Message-ID: <49AE01EB.2060907@gmail.com> Hello Bess, Thanks for your reply, and by the way awesome presentation and breakout at code4lib. Well not really I created a symlink in the /opt/www directory using ln -s /opt/bl-demo/public /opt/www/discovery In any event, Matt's last reply did the trick. I am able to access the demo app at http://localhost/discovery/ . I do have some other questions, so here they go 1 - Now that I can access the demo page, when I click the title which points to the catalog i.e http://localhost/discovery/catalog/2008348511?f[geographic_subject_facet][]=Politics+and+government&f[subject_era_facet][]=20th+century&page=4 I am getting 404 errors 2 - My data at this point in not marc based it is a modified dublin core metadata schema. In order to customize blacklight where should I start overriding blacklight's templates and or solr schema. 3 - Is it better to completely overide the default solr schema or change my schema to use the blacklight nameing conventions i.e *_facet Thanks Vern Bess Sadler wrote: > Vernon, is your rails app actually installed in /opt/www? > I think your DocumentRoot needs to point at the public directory of > your rails app. E.g.: > > DocumentRoot /usr/local/projects/blacklight/current/public > > Where does your rails app live on the filesystem? > > If this doesn't work, please write back with: > > - where on the filesystem your rails app is deployed > - the full text of your VirtualHost config > > Thanks! > > Bess > > On 3-Mar-09, at 9:52 PM, Vernon Chapman wrote: > >> Hi Matt, >> >> I am using rails version 2.2.2 >> >> When I comment out that line I get an error saying >> No route matches "/discovery" with {:method=>:get} >> >> My VirtualHost config reads >> >> >> ServerName fatboy.local >> ServerAdmin webmaster at localhost >> DocumentRoot /opt/www >> >> ........... >> >> RailsBaseURI /discovery >> >> >> >> So in order to access the demo app I want the >> url to be http://localhost/discovery/ >> >> >> Not sure what to do now. >> >> Thanks for your help >> >> >> >> >> Matt Mitchell wrote: >>> Hi Vernon, >>> >>> What version of Rails are you using? What happens when you comment >>> that line out? >>> >>> Matt >>> >>> On Tue, Mar 3, 2009 at 5:22 PM, Vernon Chapman >>> > wrote: >>> >>> Hello all blacklight newb here. >>> >>> I am trying to install the demo , and run it using mod_rails >>> (Passenger) + apache. I am able to install as per the installation >>> document, my problem is that when I attempt to access the url >>> (RubyBasURI /discovery) I get the following error message and >>> trace. Any help is greatly appreciated. >>> >>> >>> Error message: >>> >>> undefined method `cache_template_loading=' for >>> ActionView::Base:Class >>> >>> Exception class: >>> NoMethodError >>> >>> >>> Here is the trace from Passenger >>> >>> 0 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >>> 530 in `send' >>> 1 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >>> 530 in `initialize_framework_settings' >>> 2 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >>> 529 in `each' >>> 3 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >>> 529 in `initialize_framework_settings' >>> 4 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >>> 526 in `each' >>> 5 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >>> 526 in `initialize_framework_settings' >>> 6 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >>> 154 in `process' >>> 7 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >>> 112 in `send' >>> 8 /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/initializer.rb >>> 112 in `run' >>> 9 ./config/environment.rb 15 >>> 10 /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb >>> 31 in `gem_original_require' >>> 11 /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb >>> 31 in `require' >>> 12 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/ >>> application_spawner.rb >>> 142 in `spawn_application!' >>> 13 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >>> 179 in `report_app_init_status' >>> 14 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/ >>> application_spawner.rb >>> 136 in `spawn_application!' >>> 15 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >>> 163 in `safe_fork' >>> 16 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >>> 161 in `fork' >>> 17 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >>> 161 in `safe_fork' >>> 18 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/ >>> application_spawner.rb >>> 132 in `spawn_application!' >>> 19 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >>> 163 in `safe_fork' >>> 20 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >>> 161 in `fork' >>> 21 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/utils.rb >>> 161 in `safe_fork' >>> 22 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/railz/ >>> application_spawner.rb >>> 131 in `spawn_application!' >>> 23 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >>> spawn_manager.rb >>> 234 in `spawn_rails_application' >>> 24 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >>> spawn_manager.rb >>> 217 in `synchronize' >>> 25 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >>> spawn_manager.rb >>> 217 in `spawn_rails_application' >>> 26 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >>> spawn_manager.rb >>> 126 in `spawn_application' >>> 27 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >>> spawn_manager.rb >>> 251 in `handle_spawn_application' >>> 28 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >>> abstract_server.rb >>> 317 in `__send__' >>> 29 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >>> abstract_server.rb >>> 317 in `main_loop' >>> 30 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/lib/passenger/ >>> abstract_server.rb >>> 168 in `start_synchronously' >>> 31 >>> /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/bin/passenger-spawn- >>> server >>> 46 >>> 32 /usr/bin/passenger-spawn-server 19 in `load' >>> 33 /usr/bin/passenger-spawn-server 19 >>> >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >>> >>> ------------------------------------------------------------------------ >>> >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From bill at dueber.com Wed Mar 4 09:24:02 2009 From: bill at dueber.com (Bill Dueber) Date: Wed, 4 Mar 2009 09:24:02 -0500 Subject: [Blacklight-development] Blacklight under jruby? Message-ID: Is anyone running (or at least experimenting) under jruby? The only obvious hole is the native code in the bcrypt library (which I assume can potentially be addressed with an existing java bcrypt library) -- are the other obstacles? -Bill- -- Bill Dueber Library Systems Programmer University of Michigan Library -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossfsinger at gmail.com Wed Mar 4 10:32:35 2009 From: rossfsinger at gmail.com (Ross Singer) Date: Wed, 4 Mar 2009 10:32:35 -0500 Subject: [Blacklight-development] Blacklight under jruby? In-Reply-To: References: Message-ID: <23b83f160903040732x7cf9689bsfd3dfd8a61563864@mail.gmail.com> Bill, I am generally interested in seeing Blacklight run under JRuby, but that's because I'm an unabashed JRuby cheerleader. That being said, I haven't actually tried it in JRuby, yet. Beside the bcrypt thing, there's this: http://devblog.imedo.de/2008/9/18/using-rails-engines-with-jruby-and-glassfish Also, I'm not sure how (or even if) Blacklight deals internally with XML or JSON, but those would probably need to be swapped out to more appropriate gems, as well. Is your interest in JRuby mainly for deployment? Or do you have a Java integration interest? -Ross. On Wed, Mar 4, 2009 at 9:24 AM, Bill Dueber wrote: > Is anyone running (or at least experimenting) under jruby? The only obvious > hole is the native code in the bcrypt library (which I assume can > potentially be addressed with an existing java bcrypt library) -- are the > other obstacles? > > ?-Bill- > > > -- > Bill Dueber > Library Systems Programmer > University of Michigan Library > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > From goodieboy at gmail.com Wed Mar 4 10:41:04 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Wed, 4 Mar 2009 10:41:04 -0500 Subject: [Blacklight-development] Blacklight under jruby? In-Reply-To: References: Message-ID: Hi Bill, bcrypt could be swapped out with a pure Ruby library I'm sure. If it's something you're intrested in, we'd love to hear your suggestions. Especially if that's the only thing stopping BL from easily being deployed thru jRuby. You might want to get on the irc jruby room and ask about native extension support. I seem to remember there being a solution of some kind? Maybe it was just this: http://blog.headius.com/2007/09/java-native-access-jruby-true-posix.html So, to answer your question... I don't *think* there is anything else right now using native extensions. BL has been deployed through jRuby/Tomcat in the early days, and it seemed to work fine. On a related note: currently, UVa is using a jRuby servlet to do availability lookups: http://blacklight.rubyforge.org/svn/trunk/availability_service/ -- It was kind of an experiment, driven by jRuby deployment attempts early on. Our previous code used the YAZ/Z39.50 zoom gem, which doesn't work under jRuby. My opinion though, is that UVa might want to go back to using the simple zoom implementation; we're sticking with Passenger for deployment. I think an availability service is going to be hard to generalize, so making it plugable in some way would be more usful than creating a huge one-size-fits all thingy. Matt On Wed, Mar 4, 2009 at 9:24 AM, Bill Dueber wrote: > Is anyone running (or at least experimenting) under jruby? The only obvious > hole is the native code in the bcrypt library (which I assume can > potentially be addressed with an existing java bcrypt library) -- are the > other obstacles? > > -Bill- > > > -- > Bill Dueber > Library Systems Programmer > University of Michigan Library > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bill at dueber.com Wed Mar 4 10:54:25 2009 From: bill at dueber.com (Bill Dueber) Date: Wed, 4 Mar 2009 10:54:25 -0500 Subject: [Blacklight-development] Blacklight under jruby? In-Reply-To: <23b83f160903040732x7cf9689bsfd3dfd8a61563864@mail.gmail.com> References: <23b83f160903040732x7cf9689bsfd3dfd8a61563864@mail.gmail.com> Message-ID: Mostly for deployment -- our core services group is very wary (not unreasonably, in my mind) of fastCGI, and mod_passenger might be the straw that breaks our already-funky apache build's back. I'm actually not a java-head at all -- just trying to learn enough to implement a couple custom solr types right now -- but if using jruby will buy me non-stooooopid XML support, too, I'm all in. I'm not against an MRI implementation, but it's a much tougher sell around here. And, of course, it's pretty clear that the jruby guys are thrashing the MRI folks in terms of speed of improvements, so that's a wagon I'd like to tie my ...er...I forget the metaphor, but I want on that bus. Thanks for the rails engine link -- I'm sure I would have gone crazy with that error (if I ever get that far). On Wed, Mar 4, 2009 at 10:32 AM, Ross Singer wrote: > Bill, > > I am generally interested in seeing Blacklight run under JRuby, but > that's because I'm an unabashed JRuby cheerleader. That being said, I > haven't actually tried it in JRuby, yet. > > Beside the bcrypt thing, there's this: > > > http://devblog.imedo.de/2008/9/18/using-rails-engines-with-jruby-and-glassfish > > Also, I'm not sure how (or even if) Blacklight deals internally with > XML or JSON, but those would probably need to be swapped out to more > appropriate gems, as well. > > Is your interest in JRuby mainly for deployment? Or do you have a > Java integration interest? > > -Ross. > > On Wed, Mar 4, 2009 at 9:24 AM, Bill Dueber wrote: > > Is anyone running (or at least experimenting) under jruby? The only > obvious > > hole is the native code in the bcrypt library (which I assume can > > potentially be addressed with an existing java bcrypt library) -- are the > > other obstacles? > > > > -Bill- > > > > > > -- > > Bill Dueber > > Library Systems Programmer > > University of Michigan Library > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -- Bill Dueber Library Systems Programmer University of Michigan Library -------------- next part -------------- An HTML attachment was scrubbed... URL: From rochkind at jhu.edu Wed Mar 4 11:27:30 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Wed, 04 Mar 2009 11:27:30 -0500 Subject: [Blacklight-development] Blacklight under jruby? In-Reply-To: References: <23b83f160903040732x7cf9689bsfd3dfd8a61563864@mail.gmail.com> Message-ID: <49AEABF2.3090902@jhu.edu> MRI in mod_rails/passenger should, I expect, be a much easier sell than dealing with mongrel. It's not too much different than mod_php for deployment, really. Jonathan Bill Dueber wrote: > Mostly for deployment -- our core services group is very wary (not unreasonably, in my mind) of fastCGI, and mod_passenger might be the straw that breaks our already-funky apache build's back. > > I'm actually not a java-head at all -- just trying to learn enough to implement a couple custom solr types right now -- but if using jruby will buy me non-stooooopid XML support, too, I'm all in. > > I'm not against an MRI implementation, but it's a much tougher sell around here. And, of course, it's pretty clear that the jruby guys are thrashing the MRI folks in terms of speed of improvements, so that's a wagon I'd like to tie my ...er...I forget the metaphor, but I want on that bus. > > Thanks for the rails engine link -- I'm sure I would have gone crazy with that error (if I ever get that far). > > On Wed, Mar 4, 2009 at 10:32 AM, Ross Singer > wrote: > Bill, > > I am generally interested in seeing Blacklight run under JRuby, but > that's because I'm an unabashed JRuby cheerleader. That being said, I > haven't actually tried it in JRuby, yet. > > Beside the bcrypt thing, there's this: > > http://devblog.imedo.de/2008/9/18/using-rails-engines-with-jruby-and-glassfish > > Also, I'm not sure how (or even if) Blacklight deals internally with > XML or JSON, but those would probably need to be swapped out to more > appropriate gems, as well. > > Is your interest in JRuby mainly for deployment? Or do you have a > Java integration interest? > > -Ross. > > On Wed, Mar 4, 2009 at 9:24 AM, Bill Dueber > wrote: > >> Is anyone running (or at least experimenting) under jruby? The only obvious >> hole is the native code in the bcrypt library (which I assume can >> potentially be addressed with an existing java bcrypt library) -- are the >> other obstacles? >> >> -Bill- >> >> >> -- >> Bill Dueber >> Library Systems Programmer >> University of Michigan Library >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > -- > Bill Dueber > Library Systems Programmer > University of Michigan Library > > From chapman.lists at gmail.com Wed Mar 4 17:06:31 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Wed, 04 Mar 2009 17:06:31 -0500 Subject: [Blacklight-development] Customizing Blacklight Message-ID: <49AEFB67.7080008@gmail.com> Is there some documentation on how to customize blacklight? I have a lot or Dublin Core(ish) metadata in a solr index and would like to use blacklight as a front end to it. Thanks in advance Vern From eos8d at virginia.edu Wed Mar 4 19:40:23 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Wed, 4 Mar 2009 19:40:23 -0500 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: <49AEFB67.7080008@gmail.com> References: <49AEFB67.7080008@gmail.com> Message-ID: Hi, Vern. Unfortunately we don't have a lot of documentation about customizing Blacklight written yet, but we are working on that. Are you mainly interested in customizing the index, or the interface, or both? If you're fairly happy with the default index fields, you could just map your dublin core fields to them, and then customize the interface to fit your look and feel. We have dedicated some folks to writing documentation (and maybe even creating a screencast) about installation and customization, and if you give us more details about what you're trying to do we might be able to shape the documentation to better fit your needs. Thanks! Bess On 4-Mar-09, at 5:06 PM, Vernon Chapman wrote: > Is there some documentation on how to customize blacklight? > I have a lot or Dublin Core(ish) metadata in a solr index and would > like > to use blacklight as a front end to it. > > Thanks in advance > > Vern > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development From jkeck at stanford.edu Wed Mar 4 20:10:32 2009 From: jkeck at stanford.edu (Jessie Keck) Date: Wed, 04 Mar 2009 17:10:32 -0800 Subject: [Blacklight-development] Plugin Splash Page almost ready Message-ID: <49AF2688.8000101@stanford.edu> Hi all, I'm writing to make sure that some code that I'm about to commit won't interfere w/ anybody else's current work. I'm working on a splash page for the plugin, so that the user isn't taken to the catalog controller w/ search results as the root. I will be adding the following files (note that the 'home' directory under views will be created) : /controllers/home_controller.rb /views/home/index.html.erb /views/home/_home_text.html.erb I will be making modifications to the following files: routes.rb (map.root now points to the HomeController) application.css (adding some padding to the home page text div) Unless I hear any yelps from anybody by tomorrow morning (PST) I'll try and get this committed. -Jessie Keck Stanford University Libraries From chapman.lists at gmail.com Wed Mar 4 20:28:19 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Wed, 04 Mar 2009 20:28:19 -0500 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: References: <49AEFB67.7080008@gmail.com> Message-ID: <49AF2AB3.7020103@gmail.com> HI Bess, I decided to just try and wade through the code, and I think I have a pretty good idea of what I need to do. I am actually interested in customizing both the index and the interface. Customizing the index would basically consist of me changing the schema on my external solr server at least AFAIKT. Next I think I would need to extend the BlackLight module and add somtehting like: Blacklight.solr.param_mappers[:news] = :dismax Blacklight.solr.param_mappers[:events] = :dismax Blacklight.solr.param_mappers[:content] = :dismax where :news || :events || :content is the name of the handler that I have created in my solrconfig.xml and :dismax is the type of handler my handler is. That is as far as I have gotten am I on the right track? Thanks Vern Bess Sadler wrote: > Hi, Vern. > > Unfortunately we don't have a lot of documentation about customizing > Blacklight written yet, but we are working on that. Are you mainly > interested in customizing the index, or the interface, or both? > > If you're fairly happy with the default index fields, you could just > map your dublin core fields to them, and then customize the interface > to fit your look and feel. > > We have dedicated some folks to writing documentation (and maybe even > creating a screencast) about installation and customization, and if > you give us more details about what you're trying to do we might be > able to shape the documentation to better fit your needs. > > Thanks! > Bess > > On 4-Mar-09, at 5:06 PM, Vernon Chapman wrote: > >> Is there some documentation on how to customize blacklight? >> I have a lot or Dublin Core(ish) metadata in a solr index and would like >> to use blacklight as a front end to it. >> >> Thanks in advance >> >> Vern >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From jamie at dang.com Wed Mar 4 20:40:58 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Wed, 4 Mar 2009 20:40:58 -0500 Subject: [Blacklight-development] Blacklight under jruby? In-Reply-To: References: <23b83f160903040732x7cf9689bsfd3dfd8a61563864@mail.gmail.com> Message-ID: <01339CE5-2F8D-4902-9BF8-BF90B1FB2C4E@dang.com> I can't recommend Phusion Passenger enough. I've got it running on several production servers (CentOS 5) along with Phusion's Enterprise Ruby. I also use it for development with the Passenger Preferences Pane (Mac OS X). Pretty easy to setup and then it just hums along. Jamie On Mar 4, 2009, at 10:54 AM, Bill Dueber wrote: > Mostly for deployment -- our core services group is very wary (not > unreasonably, in my mind) of fastCGI, and mod_passenger might be the > straw that breaks our already-funky apache build's back. > > I'm actually not a java-head at all -- just trying to learn enough > to implement a couple custom solr types right now -- but if using > jruby will buy me non-stooooopid XML support, too, I'm all in. > > I'm not against an MRI implementation, but it's a much tougher sell > around here. And, of course, it's pretty clear that the jruby guys > are thrashing the MRI folks in terms of speed of improvements, so > that's a wagon I'd like to tie my ...er...I forget the metaphor, but > I want on that bus. > > Thanks for the rails engine link -- I'm sure I would have gone crazy > with that error (if I ever get that far). > > On Wed, Mar 4, 2009 at 10:32 AM, Ross Singer > wrote: > Bill, > > I am generally interested in seeing Blacklight run under JRuby, but > that's because I'm an unabashed JRuby cheerleader. That being said, I > haven't actually tried it in JRuby, yet. > > Beside the bcrypt thing, there's this: > > http://devblog.imedo.de/2008/9/18/using-rails-engines-with-jruby-and-glassfish > > Also, I'm not sure how (or even if) Blacklight deals internally with > XML or JSON, but those would probably need to be swapped out to more > appropriate gems, as well. > > Is your interest in JRuby mainly for deployment? Or do you have a > Java integration interest? > > -Ross. > > On Wed, Mar 4, 2009 at 9:24 AM, Bill Dueber wrote: > > Is anyone running (or at least experimenting) under jruby? The > only obvious > > hole is the native code in the bcrypt library (which I assume can > > potentially be addressed with an existing java bcrypt library) -- > are the > > other obstacles? > > > > -Bill- > > > > > > -- > > Bill Dueber > > Library Systems Programmer > > University of Michigan Library > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > -- > Bill Dueber > Library Systems Programmer > University of Michigan Library > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development -------------- next part -------------- An HTML attachment was scrubbed... URL: From jkeck at stanford.edu Wed Mar 4 20:50:11 2009 From: jkeck at stanford.edu (Jessie Keck) Date: Wed, 04 Mar 2009 17:50:11 -0800 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: <49AEFB67.7080008@gmail.com> References: <49AEFB67.7080008@gmail.com> Message-ID: <49AF2FD3.1020102@stanford.edu> Hi Vern, Just chiming in about overriding the interface. If you find a view that you would like to override then you just need to copy that file from the plugin and paste it in your top level application (the one generated with =>> rails {your_app_name}) making sure to keep the same directory structure as the plugin. The engines plugin will look at your top level app first to find a file then to the blacklight plugin. An example of this is if you wanted the search box to be available on all pages not just search results. This is an easy way to do that: 1) Copy file views/catalog/index.html.erb from plugin to the top level app 2) Remove render tag from the index view in the top level app 3) Copy file views/catalog/_search_form.html erb to the top level app to the views/shared directory (note that I did not maintain the directory structure because the search form partial is now shared amongst other controllers) 4) Copy file views/layouts/application.html.erb to the top level app 5) Add the render tag to application view to now be render :partial => 'shared/search_form' (Note: putting this is the application view is probably not the best idea, but this is just an example. I'm actually rendering the search form partial from a header partial, not the application view) That's about it. You can go nuts with overriding all kinds of files. Do remember that certain things may require a restart to take affect (most notably changes to helpers) Hope this helps, Let us know if any of this is unclear or you run into any snags. -Jessie Keck Stanford University Libraries Vernon Chapman wrote: > Is there some documentation on how to customize blacklight? > I have a lot or Dublin Core(ish) metadata in a solr index and would like > to use blacklight as a front end to it. > > Thanks in advance > > Vern > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development From goodieboy at gmail.com Wed Mar 4 21:07:48 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Wed, 4 Mar 2009 21:07:48 -0500 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: <49AF2AB3.7020103@gmail.com> References: <49AEFB67.7080008@gmail.com> <49AF2AB3.7020103@gmail.com> Message-ID: Hi Vernon, You're right on with that param_mappers thing. Good work. The schema defines the fields/params in the request handlers. This is something that has dramatically simplified the controllers/configs from the previous version of BL. It also offers the advantages of having the fields defined in one place, and any other client that uses your solr instance can expect the same response as BL does. If we were to put all of the mapping in the UI layer, other clients would have to mimic the params, which would be cruel. The client can always override what's been defined in the solrconfig.xml file. Matt On Wed, Mar 4, 2009 at 8:28 PM, Vernon Chapman wrote: > HI Bess, > > I decided to just try and wade through the code, and I think I have a > pretty good idea of what I need to do. > > I am actually interested in customizing both the index and the interface. > Customizing the index would basically consist of me changing > the schema on my external solr server at least AFAIKT. > > Next I think I would need to extend the BlackLight module and add > somtehting like: > > Blacklight.solr.param_mappers[:news] = :dismax > Blacklight.solr.param_mappers[:events] = :dismax > Blacklight.solr.param_mappers[:content] = :dismax > > where :news || :events || :content is the name of the handler that I have > created > in my solrconfig.xml and :dismax is the type of handler my handler is. > > That is as far as I have gotten am I on the right track? > > Thanks > > Vern > > > Bess Sadler wrote: > >> Hi, Vern. >> >> Unfortunately we don't have a lot of documentation about customizing >> Blacklight written yet, but we are working on that. Are you mainly >> interested in customizing the index, or the interface, or both? >> >> If you're fairly happy with the default index fields, you could just map >> your dublin core fields to them, and then customize the interface to fit >> your look and feel. >> >> We have dedicated some folks to writing documentation (and maybe even >> creating a screencast) about installation and customization, and if you give >> us more details about what you're trying to do we might be able to shape the >> documentation to better fit your needs. >> >> Thanks! >> Bess >> >> On 4-Mar-09, at 5:06 PM, Vernon Chapman wrote: >> >> Is there some documentation on how to customize blacklight? >>> I have a lot or Dublin Core(ish) metadata in a solr index and would like >>> to use blacklight as a front end to it. >>> >>> Thanks in advance >>> >>> Vern >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chapman.lists at gmail.com Wed Mar 4 21:15:04 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Wed, 04 Mar 2009 21:15:04 -0500 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: <49AF2FD3.1020102@stanford.edu> References: <49AEFB67.7080008@gmail.com> <49AF2FD3.1020102@stanford.edu> Message-ID: <49AF35A8.5080207@gmail.com> Jessie, Thanks so much for the detail explanation, I had just read the documentation on the engine plugin, but your explanation definitely made it crystal clear in my head. I will give that a shot and report back. Thanks Vern Jessie Keck wrote: > Hi Vern, > Just chiming in about overriding the interface. > > If you find a view that you would like to override then you just need > to copy that file from the plugin and paste it in your top level > application (the one generated with =>> rails {your_app_name}) making > sure to keep the same directory structure as the plugin. > > The engines plugin will look at your top level app first to find a > file then to the blacklight plugin. An example of this is if you > wanted the search box to be available on all pages not just search > results. This is an easy way to do that: > 1) Copy file views/catalog/index.html.erb from plugin to the top level > app > 2) Remove render tag from the index view in the top level app > 3) Copy file views/catalog/_search_form.html erb to the top level app > to the views/shared directory (note that I did not maintain the > directory structure because the search form partial is now shared > amongst other controllers) > 4) Copy file views/layouts/application.html.erb to the top level app > 5) Add the render tag to application view to now be render :partial => > 'shared/search_form' > (Note: putting this is the application view is probably not the best > idea, but this is just an example. I'm actually rendering the search > form partial from a header partial, not the application view) > > That's about it. You can go nuts with overriding all kinds of files. > Do remember that certain things may require a restart to take affect > (most notably changes to helpers) > > Hope this helps, > Let us know if any of this is unclear or you run into any snags. > > -Jessie Keck > Stanford University Libraries > > Vernon Chapman wrote: >> Is there some documentation on how to customize blacklight? >> I have a lot or Dublin Core(ish) metadata in a solr index and would like >> to use blacklight as a front end to it. >> >> Thanks in advance >> >> Vern >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From chapman.lists at gmail.com Wed Mar 4 21:53:50 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Wed, 04 Mar 2009 21:53:50 -0500 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: References: <49AEFB67.7080008@gmail.com> <49AF2AB3.7020103@gmail.com> Message-ID: <49AF3EBE.6050106@gmail.com> Matt, Thanks. I just did what I probably should have done before asking the question. I'm glad I actually figured that part out by myself, there is something about when the light goes on that you just can't beat. That little tidbit along with the detailed explanation by Jessie gives me a little insight hopefully I can get something cracking to show my boss by next week. Thanks to everyone who has helped the newb with the dumb questions. I really am interested in learning more about BL & Rails, if I can get something working by middle of next week my boss might just pay for railsconf in vegas! Wish me luck! Thanks Again Vern Matt Mitchell wrote: > Hi Vernon, > > You're right on with that param_mappers thing. Good work. The schema > defines the fields/params in the request handlers. This is something > that has dramatically simplified the controllers/configs from the > previous version of BL. It also offers the advantages of having the > fields defined in one place, and any other client that uses your solr > instance can expect the same response as BL does. If we were to put > all of the mapping in the UI layer, other clients would have to mimic > the params, which would be cruel. The client can always override > what's been defined in the solrconfig.xml file. > > Matt > > On Wed, Mar 4, 2009 at 8:28 PM, Vernon Chapman > > wrote: > > HI Bess, > > I decided to just try and wade through the code, and I think I > have a pretty good idea of what I need to do. > > I am actually interested in customizing both the index and the > interface. > Customizing the index would basically consist of me changing > the schema on my external solr server at least AFAIKT. > > Next I think I would need to extend the BlackLight module and add > somtehting like: > > Blacklight.solr.param_mappers[:news] = :dismax > Blacklight.solr.param_mappers[:events] = :dismax > Blacklight.solr.param_mappers[:content] = :dismax > > where :news || :events || :content is the name of the handler that > I have created > in my solrconfig.xml and :dismax is the type of handler my handler is. > > That is as far as I have gotten am I on the right track? > > Thanks > > Vern > > > Bess Sadler wrote: > > Hi, Vern. > > Unfortunately we don't have a lot of documentation about > customizing Blacklight written yet, but we are working on > that. Are you mainly interested in customizing the index, or > the interface, or both? > > If you're fairly happy with the default index fields, you > could just map your dublin core fields to them, and then > customize the interface to fit your look and feel. > > We have dedicated some folks to writing documentation (and > maybe even creating a screencast) about installation and > customization, and if you give us more details about what > you're trying to do we might be able to shape the > documentation to better fit your needs. > > Thanks! > Bess > > On 4-Mar-09, at 5:06 PM, Vernon Chapman wrote: > > Is there some documentation on how to customize blacklight? > I have a lot or Dublin Core(ish) metadata in a solr index > and would like > to use blacklight as a front end to it. > > Thanks in advance > > Vern > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > ------------------------------------------------------------------------ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From rochkind at jhu.edu Thu Mar 5 11:24:25 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 05 Mar 2009 11:24:25 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <49AF2688.8000101@stanford.edu> References: <49AF2688.8000101@stanford.edu> Message-ID: <49AFFCB9.3000908@jhu.edu> Huh, can you give more background on why this is a good idea? I'm not seeing the purpose, I think I like it the way it is. Jessie Keck wrote: > Hi all, > I'm writing to make sure that some code that I'm about to commit won't > interfere w/ anybody else's current work. > > I'm working on a splash page for the plugin, so that the user isn't > taken to the catalog controller w/ search results as the root. > > I will be adding the following files (note that the 'home' directory > under views will be created) : > /controllers/home_controller.rb > /views/home/index.html.erb > /views/home/_home_text.html.erb > > I will be making modifications to the following files: > routes.rb (map.root now points to the HomeController) > application.css (adding some padding to the home page text div) > > Unless I hear any yelps from anybody by tomorrow morning (PST) I'll try > and get this committed. > > -Jessie Keck > Stanford University Libraries > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From jvine at stanford.edu Thu Mar 5 12:55:40 2009 From: jvine at stanford.edu (Jennifer Vine) Date: Thu, 5 Mar 2009 09:55:40 -0800 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <49AFFCB9.3000908@jhu.edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> Message-ID: <002701c99dbb$990fbb40$cb2f31c0$@edu> Jonathan, I think this was discussed the week before you joined the Friday meeting. I can't speak to the technical side (Bess/Naomi?), but we wanted to create a flexible landing page, with the search box and facets as page elements, but also the ability to highlight specific collections, add communications to the users, and so on. I haven't done extensive user testing, but my observation so far is that users have been somewhat disconcerted by landing directly in a default search and take a few seconds to get their bearings. JV Jennifer Vine | jvine at stanford.edu UI Designer < Digital Library Systems & Services < SULAIR -----Original Message----- From: blacklight-development-bounces at rubyforge.org [mailto:blacklight-development-bounces at rubyforge.org] On Behalf Of Jonathan Rochkind Sent: Thursday, March 05, 2009 8:24 AM To: blacklight-development at rubyforge.org Subject: Re: [Blacklight-development] Plugin Splash Page almost ready Huh, can you give more background on why this is a good idea? I'm not seeing the purpose, I think I like it the way it is. Jessie Keck wrote: > Hi all, > I'm writing to make sure that some code that I'm about to commit won't > interfere w/ anybody else's current work. > > I'm working on a splash page for the plugin, so that the user isn't > taken to the catalog controller w/ search results as the root. > > I will be adding the following files (note that the 'home' directory > under views will be created) : > /controllers/home_controller.rb > /views/home/index.html.erb > /views/home/_home_text.html.erb > > I will be making modifications to the following files: > routes.rb (map.root now points to the HomeController) > application.css (adding some padding to the home page text div) > > Unless I hear any yelps from anybody by tomorrow morning (PST) I'll try > and get this committed. > > -Jessie Keck > Stanford University Libraries > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org http://rubyforge.org/mailman/listinfo/blacklight-development From rochkind at jhu.edu Thu Mar 5 13:27:42 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 05 Mar 2009 13:27:42 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <002701c99dbb$990fbb40$cb2f31c0$@edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> Message-ID: <49B0199E.3080009@jhu.edu> Cool, thanks for the background. We may or may not want an easy config switch to restore the previous behavior. Jennifer Vine wrote: > Jonathan, I think this was discussed the week before you joined the Friday > meeting. I can't speak to the technical side (Bess/Naomi?), but we wanted to > create a flexible landing page, with the search box and facets as page > elements, but also the ability to highlight specific collections, add > communications to the users, and so on. I haven't done extensive user > testing, but my observation so far is that users have been somewhat > disconcerted by landing directly in a default search and take a few seconds > to get their bearings. > > JV > > > Jennifer Vine | jvine at stanford.edu > UI Designer < Digital Library Systems & Services < SULAIR > > > > -----Original Message----- > From: blacklight-development-bounces at rubyforge.org > [mailto:blacklight-development-bounces at rubyforge.org] On Behalf Of Jonathan > Rochkind > Sent: Thursday, March 05, 2009 8:24 AM > To: blacklight-development at rubyforge.org > Subject: Re: [Blacklight-development] Plugin Splash Page almost ready > > Huh, can you give more background on why this is a good idea? I'm not > seeing the purpose, I think I like it the way it is. > > Jessie Keck wrote: > >> Hi all, >> I'm writing to make sure that some code that I'm about to commit won't >> interfere w/ anybody else's current work. >> >> I'm working on a splash page for the plugin, so that the user isn't >> taken to the catalog controller w/ search results as the root. >> >> I will be adding the following files (note that the 'home' directory >> under views will be created) : >> /controllers/home_controller.rb >> /views/home/index.html.erb >> /views/home/_home_text.html.erb >> >> I will be making modifications to the following files: >> routes.rb (map.root now points to the HomeController) >> application.css (adding some padding to the home page text div) >> >> Unless I hear any yelps from anybody by tomorrow morning (PST) I'll try >> and get this committed. >> >> -Jessie Keck >> Stanford University Libraries >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From goodieboy at gmail.com Thu Mar 5 13:34:56 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Thu, 5 Mar 2009 13:34:56 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <002701c99dbb$990fbb40$cb2f31c0$@edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> Message-ID: To be open and honest, this is where github and git in general could help out so much. Because we each have our own branches, all separate. We'd only have to *imply* that one is the authoritative source. Jennifer could implement her home page and if Johnathan liked it, he'd pull and merge it into his repo. If not, no problem. Instead of one central repo, they're all spread out and managed independently by each orgranization. We might want to at least read up on git and github, to see how things could be easier? There's a reason so many projects, with lots of developers are starting to use it (them). Matt On Thu, Mar 5, 2009 at 12:55 PM, Jennifer Vine wrote: > Jonathan, I think this was discussed the week before you joined the Friday > meeting. I can't speak to the technical side (Bess/Naomi?), but we wanted > to > create a flexible landing page, with the search box and facets as page > elements, but also the ability to highlight specific collections, add > communications to the users, and so on. I haven't done extensive user > testing, but my observation so far is that users have been somewhat > disconcerted by landing directly in a default search and take a few seconds > to get their bearings. > > JV > > > Jennifer Vine | jvine at stanford.edu > UI Designer < Digital Library Systems & Services < SULAIR > > > > -----Original Message----- > From: blacklight-development-bounces at rubyforge.org > [mailto:blacklight-development-bounces at rubyforge.org] On Behalf Of > Jonathan > Rochkind > Sent: Thursday, March 05, 2009 8:24 AM > To: blacklight-development at rubyforge.org > Subject: Re: [Blacklight-development] Plugin Splash Page almost ready > > Huh, can you give more background on why this is a good idea? I'm not > seeing the purpose, I think I like it the way it is. > > Jessie Keck wrote: > > Hi all, > > I'm writing to make sure that some code that I'm about to commit won't > > interfere w/ anybody else's current work. > > > > I'm working on a splash page for the plugin, so that the user isn't > > taken to the catalog controller w/ search results as the root. > > > > I will be adding the following files (note that the 'home' directory > > under views will be created) : > > /controllers/home_controller.rb > > /views/home/index.html.erb > > /views/home/_home_text.html.erb > > > > I will be making modifications to the following files: > > routes.rb (map.root now points to the HomeController) > > application.css (adding some padding to the home page text div) > > > > Unless I hear any yelps from anybody by tomorrow morning (PST) I'll try > > and get this committed. > > > > -Jessie Keck > > Stanford University Libraries > > > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rochkind at jhu.edu Thu Mar 5 14:48:47 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 05 Mar 2009 14:48:47 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> Message-ID: <49B02C9F.5090703@jhu.edu> Hmm, that solution sounds to me like nothing but somewhat improved support for merging forks. I think avoiding forks in the first place is probably better, where possible. Matt Mitchell wrote: > To be open and honest, this is where github and git in general could help out so much. Because we each have our own branches, all separate. We'd only have to *imply* that one is the authoritative source. Jennifer could implement her home page and if Johnathan liked it, he'd pull and merge it into his repo. If not, no problem. Instead of one central repo, they're all spread out and managed independently by each orgranization. We might want to at least read up on git and github, to see how things could be easier? There's a reason so many projects, with lots of developers are starting to use it (them). > > Matt > > On Thu, Mar 5, 2009 at 12:55 PM, Jennifer Vine > wrote: > Jonathan, I think this was discussed the week before you joined the Friday > meeting. I can't speak to the technical side (Bess/Naomi?), but we wanted to > create a flexible landing page, with the search box and facets as page > elements, but also the ability to highlight specific collections, add > communications to the users, and so on. I haven't done extensive user > testing, but my observation so far is that users have been somewhat > disconcerted by landing directly in a default search and take a few seconds > to get their bearings. > > JV > > > Jennifer Vine | jvine at stanford.edu > UI Designer < Digital Library Systems & Services < SULAIR > > > > -----Original Message----- > From: blacklight-development-bounces at rubyforge.org > [mailto:blacklight-development-bounces at rubyforge.org] On Behalf Of Jonathan > Rochkind > Sent: Thursday, March 05, 2009 8:24 AM > To: blacklight-development at rubyforge.org > Subject: Re: [Blacklight-development] Plugin Splash Page almost ready > > Huh, can you give more background on why this is a good idea? I'm not > seeing the purpose, I think I like it the way it is. > > Jessie Keck wrote: > >> Hi all, >> I'm writing to make sure that some code that I'm about to commit won't >> interfere w/ anybody else's current work. >> >> I'm working on a splash page for the plugin, so that the user isn't >> taken to the catalog controller w/ search results as the root. >> >> I will be adding the following files (note that the 'home' directory >> under views will be created) : >> /controllers/home_controller.rb >> /views/home/index.html.erb >> /views/home/_home_text.html.erb >> >> I will be making modifications to the following files: >> routes.rb (map.root now points to the HomeController) >> application.css (adding some padding to the home page text div) >> >> Unless I hear any yelps from anybody by tomorrow morning (PST) I'll try >> and get this committed. >> >> -Jessie Keck >> Stanford University Libraries >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > > From erikhatcher at mac.com Thu Mar 5 15:02:44 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Thu, 05 Mar 2009 15:02:44 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> Message-ID: <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> I'm not really sure why this is something Blacklight itself needs to concern itself with. It's a plugin, and the encapsulating application has full control over the routing configuration. A home page seems within scope for the application, not the Blacklight engine. In fact, couldn't the Blacklight plugin effectively not do any routing at all, and require the enclosing application to map them in however it likes? Erik On Mar 5, 2009, at 1:34 PM, Matt Mitchell wrote: > To be open and honest, this is where github and git in general could > help out so much. Because we each have our own branches, all > separate. We'd only have to *imply* that one is the authoritative > source. Jennifer could implement her home page and if Johnathan > liked it, he'd pull and merge it into his repo. If not, no problem. > Instead of one central repo, they're all spread out and managed > independently by each orgranization. We might want to at least read > up on git and github, to see how things could be easier? There's a > reason so many projects, with lots of developers are starting to use > it (them). > > Matt > > On Thu, Mar 5, 2009 at 12:55 PM, Jennifer Vine > wrote: > Jonathan, I think this was discussed the week before you joined the > Friday > meeting. I can't speak to the technical side (Bess/Naomi?), but we > wanted to > create a flexible landing page, with the search box and facets as page > elements, but also the ability to highlight specific collections, add > communications to the users, and so on. I haven't done extensive user > testing, but my observation so far is that users have been somewhat > disconcerted by landing directly in a default search and take a few > seconds > to get their bearings. > > JV > > > Jennifer Vine | jvine at stanford.edu > UI Designer < Digital Library Systems & Services < SULAIR > > > > -----Original Message----- > From: blacklight-development-bounces at rubyforge.org > [mailto:blacklight-development-bounces at rubyforge.org] On Behalf Of > Jonathan > Rochkind > Sent: Thursday, March 05, 2009 8:24 AM > To: blacklight-development at rubyforge.org > Subject: Re: [Blacklight-development] Plugin Splash Page almost ready > > Huh, can you give more background on why this is a good idea? I'm not > seeing the purpose, I think I like it the way it is. > > Jessie Keck wrote: > > Hi all, > > I'm writing to make sure that some code that I'm about to commit > won't > > interfere w/ anybody else's current work. > > > > I'm working on a splash page for the plugin, so that the user isn't > > taken to the catalog controller w/ search results as the root. > > > > I will be adding the following files (note that the 'home' directory > > under views will be created) : > > /controllers/home_controller.rb > > /views/home/index.html.erb > > /views/home/_home_text.html.erb > > > > I will be making modifications to the following files: > > routes.rb (map.root now points to the HomeController) > > application.css (adding some padding to the home page text div) > > > > Unless I hear any yelps from anybody by tomorrow morning (PST) > I'll try > > and get this committed. > > > > -Jessie Keck > > Stanford University Libraries > > > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development From rochkind at jhu.edu Thu Mar 5 15:40:03 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 05 Mar 2009 15:40:03 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> Message-ID: <49B038A3.6060403@jhu.edu> That's an excellent point. I don't actually agree that the plugin should avoid doing any routing at all, I think it makes sense for it to do routing. But that routing can be easily over-ridden by the local application, can it not? I think it's appropriate for an individual application to do that in the individual application, "over-riding" the plug-ins routing. Isn't that the point of making this an Engines plug-in? PS: In general, personally, I would prefer discussing issues like this over email to doing so in weekly phone conversations. But that's just my personal preference. Among other things, it involves the wider Blacklight community like Erik, which I suppose could be a plus or a minus. :) Jonathan Erik Hatcher wrote: > I'm not really sure why this is something Blacklight itself needs to > concern itself with. It's a plugin, and the encapsulating application > has full control over the routing configuration. A home page seems > within scope for the application, not the Blacklight engine. > > In fact, couldn't the Blacklight plugin effectively not do any routing > at all, and require the enclosing application to map them in however > it likes? > > Erik > > > > On Mar 5, 2009, at 1:34 PM, Matt Mitchell wrote: > > >> To be open and honest, this is where github and git in general could >> help out so much. Because we each have our own branches, all >> separate. We'd only have to *imply* that one is the authoritative >> source. Jennifer could implement her home page and if Johnathan >> liked it, he'd pull and merge it into his repo. If not, no problem. >> Instead of one central repo, they're all spread out and managed >> independently by each orgranization. We might want to at least read >> up on git and github, to see how things could be easier? There's a >> reason so many projects, with lots of developers are starting to use >> it (them). >> >> Matt >> >> On Thu, Mar 5, 2009 at 12:55 PM, Jennifer Vine >> wrote: >> Jonathan, I think this was discussed the week before you joined the >> Friday >> meeting. I can't speak to the technical side (Bess/Naomi?), but we >> wanted to >> create a flexible landing page, with the search box and facets as page >> elements, but also the ability to highlight specific collections, add >> communications to the users, and so on. I haven't done extensive user >> testing, but my observation so far is that users have been somewhat >> disconcerted by landing directly in a default search and take a few >> seconds >> to get their bearings. >> >> JV >> >> >> Jennifer Vine | jvine at stanford.edu >> UI Designer < Digital Library Systems & Services < SULAIR >> >> >> >> -----Original Message----- >> From: blacklight-development-bounces at rubyforge.org >> [mailto:blacklight-development-bounces at rubyforge.org] On Behalf Of >> Jonathan >> Rochkind >> Sent: Thursday, March 05, 2009 8:24 AM >> To: blacklight-development at rubyforge.org >> Subject: Re: [Blacklight-development] Plugin Splash Page almost ready >> >> Huh, can you give more background on why this is a good idea? I'm not >> seeing the purpose, I think I like it the way it is. >> >> Jessie Keck wrote: >> >>> Hi all, >>> I'm writing to make sure that some code that I'm about to commit >>> >> won't >> >>> interfere w/ anybody else's current work. >>> >>> I'm working on a splash page for the plugin, so that the user isn't >>> taken to the catalog controller w/ search results as the root. >>> >>> I will be adding the following files (note that the 'home' directory >>> under views will be created) : >>> /controllers/home_controller.rb >>> /views/home/index.html.erb >>> /views/home/_home_text.html.erb >>> >>> I will be making modifications to the following files: >>> routes.rb (map.root now points to the HomeController) >>> application.css (adding some padding to the home page text div) >>> >>> Unless I hear any yelps from anybody by tomorrow morning (PST) >>> >> I'll try >> >>> and get this committed. >>> >>> -Jessie Keck >>> Stanford University Libraries >>> >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >>> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From erikhatcher at mac.com Thu Mar 5 15:43:33 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Thu, 05 Mar 2009 15:43:33 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> Message-ID: <7784D031-5751-4648-9000-7A69A8EF0813@mac.com> On Mar 5, 2009, at 3:02 PM, Erik Hatcher wrote: > In fact, couldn't the Blacklight plugin effectively not do any > routing at all, and require the enclosing application to map them in > however it likes? For example, in Flare, there is a BrowseController like this: http://svn.apache.org/repos/asf/lucene/solr/trunk/client/ruby/flare/vendor/plugins/flare/app/controllers/browse_controller.rb class BrowseController < ApplicationController flare end Granted, it's part of the plugin, but it could really be made part of the application instead such that the plugin doesn't kick in until it's wired into an _application_ controller and could be parameterized there on a per-controller basis, say like a MusicController and SemesterAtSeaController, etc specifying filter query parameters or qf, etc. Just rambling out loud, Erik From erikhatcher at mac.com Thu Mar 5 16:20:12 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Thu, 05 Mar 2009 16:20:12 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <49B038A3.6060403@jhu.edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <49B038A3.6060403@jhu.edu> Message-ID: <98DA6E6C-ABBD-4ACA-9141-C2299CC36F05@mac.com> On Mar 5, 2009, at 3:40 PM, Jonathan Rochkind wrote: > I don't actually agree that the plugin should avoid doing any > routing at all, I think it makes sense for it to do routing. But > that routing can be easily over-ridden by the local application, can > it not? Why override routing when all it takes is for a Blacklight using application to have it's own controller that simply says "blacklight :qf => 'title^2 body'" or something like that? [I say this being a bit out of the Rails loop these days though] Blacklight is NOT the application, but a plugin. The application can wire in the routing very simply. And of course Blacklight could include generators to easily do something like: script/generate blacklight_controller [params] to add in a controller. And of course the default install of Blacklight would/could be a full- on app skeleton that already had the controller in place to kick start the whole thing. > I think it's appropriate for an individual application to do that in > the individual application, "over-riding" the plug-ins routing. > Isn't that the point of making this an Engines plug-in? Overriding is fine for some things, but in this case it seems like Blacklight plugin is doing too much for application and somewhat getting in the way. Erik From goodieboy at gmail.com Thu Mar 5 16:21:53 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Thu, 5 Mar 2009 16:21:53 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <49B02C9F.5090703@jhu.edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <49B02C9F.5090703@jhu.edu> Message-ID: Jonathan, Well, I guess it really just comes to how you look at it. Forks for creating entirely new projects is one thing, but forking for managing code that will continuously be playing together is another. It's actually a lot more than somewhat improved support for merging. But don't take my word for it! Matt On Thu, Mar 5, 2009 at 2:48 PM, Jonathan Rochkind wrote: > Hmm, that solution sounds to me like nothing but somewhat improved support > for merging forks. I think avoiding forks in the first place is probably > better, where possible. > > Matt Mitchell wrote: > >> To be open and honest, this is where github and git in general could help >> out so much. Because we each have our own branches, all separate. We'd only >> have to *imply* that one is the authoritative source. Jennifer could >> implement her home page and if Johnathan liked it, he'd pull and merge it >> into his repo. If not, no problem. Instead of one central repo, they're all >> spread out and managed independently by each orgranization. We might want to >> at least read up on git and github, to see how things could be easier? >> There's a reason so many projects, with lots of developers are starting to >> use it (them). >> >> Matt >> >> On Thu, Mar 5, 2009 at 12:55 PM, Jennifer Vine > > wrote: >> Jonathan, I think this was discussed the week before you joined the Friday >> meeting. I can't speak to the technical side (Bess/Naomi?), but we wanted >> to >> create a flexible landing page, with the search box and facets as page >> elements, but also the ability to highlight specific collections, add >> communications to the users, and so on. I haven't done extensive user >> testing, but my observation so far is that users have been somewhat >> disconcerted by landing directly in a default search and take a few >> seconds >> to get their bearings. >> >> JV >> >> >> Jennifer Vine | jvine at stanford.edu >> UI Designer < Digital Library Systems & Services < SULAIR >> >> >> >> -----Original Message----- >> From: blacklight-development-bounces at rubyforge.org> blacklight-development-bounces at rubyforge.org> >> [mailto:blacklight-development-bounces at rubyforge.org> blacklight-development-bounces at rubyforge.org>] On Behalf Of Jonathan >> Rochkind >> Sent: Thursday, March 05, 2009 8:24 AM >> To: blacklight-development at rubyforge.org> blacklight-development at rubyforge.org> >> Subject: Re: [Blacklight-development] Plugin Splash Page almost ready >> >> Huh, can you give more background on why this is a good idea? I'm not >> seeing the purpose, I think I like it the way it is. >> >> Jessie Keck wrote: >> >> >>> Hi all, >>> I'm writing to make sure that some code that I'm about to commit won't >>> interfere w/ anybody else's current work. >>> >>> I'm working on a splash page for the plugin, so that the user isn't >>> taken to the catalog controller w/ search results as the root. >>> >>> I will be adding the following files (note that the 'home' directory >>> under views will be created) : >>> /controllers/home_controller.rb >>> /views/home/index.html.erb >>> /views/home/_home_text.html.erb >>> >>> I will be making modifications to the following files: >>> routes.rb (map.root now points to the HomeController) >>> application.css (adding some padding to the home page text div) >>> >>> Unless I hear any yelps from anybody by tomorrow morning (PST) I'll try >>> and get this committed. >>> >>> -Jessie Keck >>> Stanford University Libraries >>> >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org>> Blacklight-development at rubyforge.org> >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >>> >>> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> >> >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -------------- next part -------------- An HTML attachment was scrubbed... URL: From goodieboy at gmail.com Thu Mar 5 16:23:39 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Thu, 5 Mar 2009 16:23:39 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> Message-ID: On Thu, Mar 5, 2009 at 3:02 PM, Erik Hatcher wrote: > I'm not really sure why this is something Blacklight itself needs to > concern itself with. It's a plugin, and the encapsulating application has > full control over the routing configuration. A home page seems within scope > for the application, not the Blacklight engine. > > In fact, couldn't the Blacklight plugin effectively not do any routing at > all, and require the enclosing application to map them in however it likes? Absolutely. That'd be a nice clean way to do things. In fact, that makes me wonder how you'd go about *removing* a route set by the plugin? Hmm. Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From rochkind at jhu.edu Thu Mar 5 16:36:10 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 05 Mar 2009 16:36:10 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <49B02C9F.5090703@jhu.edu> Message-ID: <49B045CA.3080708@jhu.edu> But if you take your argument to it's logical conclusion, why do we need Engines at all? We can just have everyone using git, and making whatever local changes they need directly to the source, and then merging in changes with git. No matter how well git does merging, that still sounds like a last resort to me, for cases where cleanly designed abstraction (like using Engines) can't handle it. Jonathan Matt Mitchell wrote: > Jonathan, > > Well, I guess it really just comes to how you look at it. Forks for creating entirely new projects is one thing, but forking for managing code that will continuously be playing together is another. It's actually a lot more than somewhat improved support for merging. But don't take my word for it! > > Matt > > On Thu, Mar 5, 2009 at 2:48 PM, Jonathan Rochkind > wrote: > Hmm, that solution sounds to me like nothing but somewhat improved support for merging forks. I think avoiding forks in the first place is probably better, where possible. > > Matt Mitchell wrote: > To be open and honest, this is where github and git in general could help out so much. Because we each have our own branches, all separate. We'd only have to *imply* that one is the authoritative source. Jennifer could implement her home page and if Johnathan liked it, he'd pull and merge it into his repo. If not, no problem. Instead of one central repo, they're all spread out and managed independently by each orgranization. We might want to at least read up on git and github, to see how things could be easier? There's a reason so many projects, with lots of developers are starting to use it (them). > > Matt > > On Thu, Mar 5, 2009 at 12:55 PM, Jennifer Vine >> wrote: > Jonathan, I think this was discussed the week before you joined the Friday > meeting. I can't speak to the technical side (Bess/Naomi?), but we wanted to > create a flexible landing page, with the search box and facets as page > elements, but also the ability to highlight specific collections, add > communications to the users, and so on. I haven't done extensive user > testing, but my observation so far is that users have been somewhat > disconcerted by landing directly in a default search and take a few seconds > to get their bearings. > > JV > > > Jennifer Vine | jvine at stanford.edu> > > UI Designer < Digital Library Systems & Services < SULAIR > > > > -----Original Message----- > From: blacklight-development-bounces at rubyforge.org> > [mailto:blacklight-development-bounces at rubyforge.org>] On Behalf Of Jonathan > Rochkind > Sent: Thursday, March 05, 2009 8:24 AM > To: blacklight-development at rubyforge.org> > Subject: Re: [Blacklight-development] Plugin Splash Page almost ready > > Huh, can you give more background on why this is a good idea? I'm not > seeing the purpose, I think I like it the way it is. > > Jessie Keck wrote: > > Hi all, > I'm writing to make sure that some code that I'm about to commit won't > interfere w/ anybody else's current work. > > I'm working on a splash page for the plugin, so that the user isn't > taken to the catalog controller w/ search results as the root. > > I will be adding the following files (note that the 'home' directory > under views will be created) : > /controllers/home_controller.rb > /views/home/index.html.erb > /views/home/_home_text.html.erb > > I will be making modifications to the following files: > routes.rb (map.root now points to the HomeController) > application.css (adding some padding to the home page text div) > > Unless I hear any yelps from anybody by tomorrow morning (PST) I'll try > and get this committed. > > -Jessie Keck > Stanford University Libraries > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org> > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org> > > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org> > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > > From rochkind at jhu.edu Thu Mar 5 16:40:51 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 05 Mar 2009 16:40:51 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <7784D031-5751-4648-9000-7A69A8EF0813@mac.com> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <7784D031-5751-4648-9000-7A69A8EF0813@mac.com> Message-ID: <49B046E3.3000007@jhu.edu> I get it, but I think one of our priorities here is having Blacklight work as _easily_ as possible out of the box for people with standard needs, with as little setup as possible. While also supporting advanced needs through rational local 'over-riding' at the right granularity. So requiring certain code in the application itself for base level functionality kind of goes against that. Now, you _could_ provide a generator to create that stub code in application, to make install easier. But what do you gain from that? When the choice is between a generator or a live link to code, I always choose a live link to code. Because then when the live code changes, you don't need to re-generate, among other reasons. So I like the plugin having it's own routes, as long as those routes can be easily over-ridden at the application level when desired. Perusing the Engines documentation, I'm having trouble finding how routes work in Engines to confirm that they are easily over-rideable in the Application. If they are, routes in the plug-in are definitely the way to go to me -- but when you don't want the standard routes, over-riding them in your individual application (still without needing to touch common Blacklight plugin code to implement your local changes) is also definitely the way to go. Jonathan Erik Hatcher wrote: > On Mar 5, 2009, at 3:02 PM, Erik Hatcher wrote: > >> In fact, couldn't the Blacklight plugin effectively not do any >> routing at all, and require the enclosing application to map them in >> however it likes? >> > > For example, in Flare, there is a BrowseController like this: > > http://svn.apache.org/repos/asf/lucene/solr/trunk/client/ruby/flare/vendor/plugins/flare/app/controllers/browse_controller.rb > > class BrowseController < ApplicationController > flare > end > > Granted, it's part of the plugin, but it could really be made part of > the application instead such that the plugin doesn't kick in until > it's wired into an _application_ controller and could be parameterized > there on a per-controller basis, say like a MusicController and > SemesterAtSeaController, etc specifying filter query parameters or qf, > etc. > > Just rambling out loud, > Erik > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From rochkind at jhu.edu Thu Mar 5 16:51:02 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 05 Mar 2009 16:51:02 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> Message-ID: <49B04946.4030306@jhu.edu> Matt Mitchell wrote: > Absolutely. That'd be a nice clean way to do things. In fact, that makes me wonder how you'd go about *removing* a route set by the plugin? Hmm. > Trying to figure this out myself, I can't even find documentation that says Engines supports routines in a plug-in in the first place! Or is this a standard rails plugin thing? Or did you have to hack it in yourself? There is pretty easy 'reflection' on the routes, so they could always be removed that way, but that's kind of a pain. This is definitely something we want to figure out though, the best/easiest way to override Blacklight plugin routes in the application. If neccesary we could provide some helper code for it (and perhaps share it with Engines to see if they're interested). Jonathan > Matt > > From goodieboy at gmail.com Thu Mar 5 18:35:18 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Thu, 5 Mar 2009 18:35:18 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <49B045CA.3080708@jhu.edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <49B02C9F.5090703@jhu.edu> <49B045CA.3080708@jhu.edu> Message-ID: Jonathan, That would be one *extreme* logical conclusion yes. There is a balance between the two that takes thought. I'm simply suggesting an alternative, and would hope that people would at least look into it before concluding that it doesn't make sense. It's certainly not a last resort; lots of people are developing this way today. Matt On Thu, Mar 5, 2009 at 4:36 PM, Jonathan Rochkind wrote: > But if you take your argument to it's logical conclusion, why do we need > Engines at all? We can just have everyone using git, and making whatever > local changes they need directly to the source, and then merging in changes > with git. > > No matter how well git does merging, that still sounds like a last resort > to me, for cases where cleanly designed abstraction (like using Engines) > can't handle it. > > Jonathan > > Matt Mitchell wrote: > >> Jonathan, >> >> Well, I guess it really just comes to how you look at it. Forks for >> creating entirely new projects is one thing, but forking for managing code >> that will continuously be playing together is another. It's actually a lot >> more than somewhat improved support for merging. But don't take my word for >> it! >> >> Matt >> >> On Thu, Mar 5, 2009 at 2:48 PM, Jonathan Rochkind > > wrote: >> Hmm, that solution sounds to me like nothing but somewhat improved support >> for merging forks. I think avoiding forks in the first place is probably >> better, where possible. >> >> Matt Mitchell wrote: >> To be open and honest, this is where github and git in general could help >> out so much. Because we each have our own branches, all separate. We'd only >> have to *imply* that one is the authoritative source. Jennifer could >> implement her home page and if Johnathan liked it, he'd pull and merge it >> into his repo. If not, no problem. Instead of one central repo, they're all >> spread out and managed independently by each orgranization. We might want to >> at least read up on git and github, to see how things could be easier? >> There's a reason so many projects, with lots of developers are starting to >> use it (them). >> >> Matt >> >> On Thu, Mar 5, 2009 at 12:55 PM, Jennifer Vine > > jvine at stanford.edu>>> wrote: >> Jonathan, I think this was discussed the week before you joined the Friday >> meeting. I can't speak to the technical side (Bess/Naomi?), but we wanted >> to >> create a flexible landing page, with the search box and facets as page >> elements, but also the ability to highlight specific collections, add >> communications to the users, and so on. I haven't done extensive user >> testing, but my observation so far is that users have been somewhat >> disconcerted by landing directly in a default search and take a few >> seconds >> to get their bearings. >> >> JV >> >> >> Jennifer Vine | jvine at stanford.edu> jvine at stanford.edu> >> >> UI Designer < Digital Library Systems & Services < SULAIR >> >> >> >> -----Original Message----- >> From: blacklight-development-bounces at rubyforge.org> blacklight-development-bounces at rubyforge.org>> blacklight-development-bounces at rubyforge.org> blacklight-development-bounces at rubyforge.org>> >> [mailto:blacklight-development-bounces at rubyforge.org> blacklight-development-bounces at rubyforge.org>> blacklight-development-bounces at rubyforge.org> blacklight-development-bounces at rubyforge.org>>] On Behalf Of Jonathan >> Rochkind >> Sent: Thursday, March 05, 2009 8:24 AM >> To: blacklight-development at rubyforge.org> blacklight-development at rubyforge.org>> blacklight-development at rubyforge.org> blacklight-development at rubyforge.org>> >> Subject: Re: [Blacklight-development] Plugin Splash Page almost ready >> >> Huh, can you give more background on why this is a good idea? I'm not >> seeing the purpose, I think I like it the way it is. >> >> Jessie Keck wrote: >> >> Hi all, >> I'm writing to make sure that some code that I'm about to commit won't >> interfere w/ anybody else's current work. >> >> I'm working on a splash page for the plugin, so that the user isn't >> taken to the catalog controller w/ search results as the root. >> >> I will be adding the following files (note that the 'home' directory >> under views will be created) : >> /controllers/home_controller.rb >> /views/home/index.html.erb >> /views/home/_home_text.html.erb >> >> I will be making modifications to the following files: >> routes.rb (map.root now points to the HomeController) >> application.css (adding some padding to the home page text div) >> >> Unless I hear any yelps from anybody by tomorrow morning (PST) I'll try >> and get this committed. >> >> -Jessie Keck >> Stanford University Libraries >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org>> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org>> >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org>> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org>> >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org>> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org>> >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> >> >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erikhatcher at mac.com Thu Mar 5 22:50:20 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Thu, 05 Mar 2009 22:50:20 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <49B046E3.3000007@jhu.edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <7784D031-5751-4648-9000-7A69A8EF0813@mac.com> <49B046E3.3000007@jhu.edu> Message-ID: <3D351A3B-F480-4B88-B55A-284D7A567D88@mac.com> On Mar 5, 2009, at 4:40 PM, Jonathan Rochkind wrote: > So requiring certain code in the application itself for base level > functionality kind of goes against that. I disagree. The app is custom, Blacklight plugin is general. We're talking one controller with one line of code in it in the most degenerate (and perhaps most likely) scenario. And this could be prebuilt in any starter download. Think about the Solr case... it comes with a default example app. Most folks just start with that, fire it up, test it out, then start customizing the schema, etc. [not sure if that is a good enough example, but the handiest one I have :)] > Now, you _could_ provide a generator to create that stub code in > application, to make install easier. But what do you gain from that? Being able to make custom controllers that leverage the Blacklight plugin infrastructure... custom controllers for various subsets of the data (like a music interface that filtered to music materials), that injected their own data into the views (from a relational database, or from other sources), etc. Lots of good reasons to let the app *control* things, rather than the plugin. > When the choice is between a generator or a live link to code, I > always choose a live link to code. Because then when the live code > changes, you don't need to re-generate, among other reasons. There are right reasons to use a generator, it's not always a negative. > So I like the plugin having it's own routes I don't, at least not routes that inject into the main app without them being specifically pulled in. > , as long as those routes can be easily over-ridden at the > application level when desired. Of course they can, but looks like removing a route is trickier, or at least uglier. > Perusing the Engines documentation, I'm having trouble finding how > routes work in Engines to confirm that they are easily over-rideable > in the Application. If they are, routes in the plug-in are > definitely the way to go to me -- but when you don't want the > standard routes, over-riding them in your individual application > (still without needing to touch common Blacklight plugin code to > implement your local changes) is also definitely the way to go. I have a hard time believing there are "standard" routes though, at least ones that aren't going to get in the way of someone's idea of how to use Blacklight. I'd suggest, Jonathan, that you build a real app using Blacklight while discussing this so you have experience to base your feedback on rather than just sending tons of reasons why most everything is a bad idea :) Working code trumps endless discussion any day. Erik From erikhatcher at mac.com Thu Mar 5 22:53:05 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Thu, 05 Mar 2009 22:53:05 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <49B02C9F.5090703@jhu.edu> <49B045CA.3080708@jhu.edu> Message-ID: <29AC5B84-7A3A-4F12-A69D-D8E2C86E0560@mac.com> It seems orthogonal to Blacklight to mandate any kind of particular version control system in how folks use it. That'd surely be a non- starter, or at least painful, situation in most environments. Erik On Mar 5, 2009, at 6:35 PM, Matt Mitchell wrote: > Jonathan, > > That would be one *extreme* logical conclusion yes. There is a > balance between the two that takes thought. I'm simply suggesting an > alternative, and would hope that people would at least look into it > before concluding that it doesn't make sense. It's certainly not a > last resort; lots of people are developing this way today. > > Matt > > On Thu, Mar 5, 2009 at 4:36 PM, Jonathan Rochkind > wrote: > But if you take your argument to it's logical conclusion, why do we > need Engines at all? We can just have everyone using git, and making > whatever local changes they need directly to the source, and then > merging in changes with git. > > No matter how well git does merging, that still sounds like a last > resort to me, for cases where cleanly designed abstraction (like > using Engines) can't handle it. > > Jonathan > > Matt Mitchell wrote: > Jonathan, > > Well, I guess it really just comes to how you look at it. Forks for > creating entirely new projects is one thing, but forking for > managing code that will continuously be playing together is another. > It's actually a lot more than somewhat improved support for merging. > But don't take my word for it! > > Matt > > On Thu, Mar 5, 2009 at 2:48 PM, Jonathan Rochkind >> wrote: > Hmm, that solution sounds to me like nothing but somewhat improved > support for merging forks. I think avoiding forks in the first place > is probably better, where possible. > > Matt Mitchell wrote: > To be open and honest, this is where github and git in general could > help out so much. Because we each have our own branches, all > separate. We'd only have to *imply* that one is the authoritative > source. Jennifer could implement her home page and if Johnathan > liked it, he'd pull and merge it into his repo. If not, no problem. > Instead of one central repo, they're all spread out and managed > independently by each orgranization. We might want to at least read > up on git and github, to see how things could be easier? There's a > reason so many projects, with lots of developers are starting to use > it (them). > > Matt > > On Thu, Mar 5, 2009 at 12:55 PM, Jennifer Vine >>> wrote: > Jonathan, I think this was discussed the week before you joined the > Friday > meeting. I can't speak to the technical side (Bess/Naomi?), but we > wanted to > create a flexible landing page, with the search box and facets as page > elements, but also the ability to highlight specific collections, add > communications to the users, and so on. I haven't done extensive user > testing, but my observation so far is that users have been somewhat > disconcerted by landing directly in a default search and take a few > seconds > to get their bearings. > > JV > > > Jennifer Vine | jvine at stanford.edu > > > > UI Designer < Digital Library Systems & Services < SULAIR > > > > -----Original Message----- > From: blacklight-development-bounces at rubyforge.org > >> > [mailto:blacklight-development-bounces at rubyforge.org > >>] On Behalf Of Jonathan > Rochkind > Sent: Thursday, March 05, 2009 8:24 AM > To: blacklight-development at rubyforge.org > >> > Subject: Re: [Blacklight-development] Plugin Splash Page almost ready > > Huh, can you give more background on why this is a good idea? I'm not > seeing the purpose, I think I like it the way it is. > > Jessie Keck wrote: > > Hi all, > I'm writing to make sure that some code that I'm about to commit won't > interfere w/ anybody else's current work. > > I'm working on a splash page for the plugin, so that the user isn't > taken to the catalog controller w/ search results as the root. > > I will be adding the following files (note that the 'home' directory > under views will be created) : > /controllers/home_controller.rb > /views/home/index.html.erb > /views/home/_home_text.html.erb > > I will be making modifications to the following files: > routes.rb (map.root now points to the HomeController) > application.css (adding some padding to the home page text div) > > Unless I hear any yelps from anybody by tomorrow morning (PST) I'll > try > and get this committed. > > -Jessie Keck > Stanford University Libraries > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > >> > > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > >> > > > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > >> > > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development From bill at dueber.com Thu Mar 5 23:20:14 2009 From: bill at dueber.com (Bill Dueber) Date: Thu, 5 Mar 2009 23:20:14 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <49B046E3.3000007@jhu.edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <7784D031-5751-4648-9000-7A69A8EF0813@mac.com> <49B046E3.3000007@jhu.edu> Message-ID: On Thu, Mar 5, 2009 at 4:40 PM, Jonathan Rochkind wrote: > I get it, but I think one of our priorities here is having Blacklight work > as _easily_ as possible out of the box for people with standard needs, with > as little setup as possible. While also supporting advanced needs through > rational local 'over-riding' at the right granularity. > I think this goal, at least, could be met by a combination of blacklight-is-only-a-plugin architecture along with a shipped out-of-the-box application with sane defaults. We are, then, talking about four components, yes? 1. solr 2. solrmarc 2.5 Sample blacklight configuration and customizations for solrmarc 3. blacklight, the rails plugin 4. a rails application that uses blacklight #4 would contain default routing and such, with the expectation (but not requirement) that much of it be revised or replaced. There will inevitably be a lot of configuration overlap between these components -- can someone who actually knows what they're doing map it out in a rough way? It'd be nice if configuration didn't need to be repeated (e.g., can't blacklight.properties and schema.xml come from a single source file??) -Bill- -------------- next part -------------- An HTML attachment was scrubbed... URL: From rochkind at jhu.edu Thu Mar 5 23:38:25 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 5 Mar 2009 23:38:25 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <29AC5B84-7A3A-4F12-A69D-D8E2C86E0560@mac.com> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <49B02C9F.5090703@jhu.edu> <49B045CA.3080708@jhu.edu> , <29AC5B84-7A3A-4F12-A69D-D8E2C86E0560@mac.com> Message-ID: <90FF863A96E1EC42B8B240D04C88FB1D719FDFD1@JHEMTEXVS2.win.ad.jhu.edu> I agree that it's important that Blacklight be installable and customizable by people regardless of what version control system they use, or if they use any at all. You should be able to install a particular tagged release of blacklight and customize it for expected use cases in supported ways without being required to use a version control system at all -- which may make sense for people who are not changing Blacklight core code and committing back to Blacklight, but only customizing in their own local 'application' using the Blacklight engine. ________________________________________ From: blacklight-development-bounces at rubyforge.org [blacklight-development-bounces at rubyforge.org] On Behalf Of Erik Hatcher [erikhatcher at mac.com] Sent: Thursday, March 05, 2009 10:53 PM To: blacklight-development at rubyforge.org Subject: Re: [Blacklight-development] Plugin Splash Page almost ready It seems orthogonal to Blacklight to mandate any kind of particular version control system in how folks use it. That'd surely be a non- starter, or at least painful, situation in most environments. Erik On Mar 5, 2009, at 6:35 PM, Matt Mitchell wrote: > Jonathan, > > That would be one *extreme* logical conclusion yes. There is a > balance between the two that takes thought. I'm simply suggesting an > alternative, and would hope that people would at least look into it > before concluding that it doesn't make sense. It's certainly not a > last resort; lots of people are developing this way today. > > Matt > > On Thu, Mar 5, 2009 at 4:36 PM, Jonathan Rochkind > wrote: > But if you take your argument to it's logical conclusion, why do we > need Engines at all? We can just have everyone using git, and making > whatever local changes they need directly to the source, and then > merging in changes with git. > > No matter how well git does merging, that still sounds like a last > resort to me, for cases where cleanly designed abstraction (like > using Engines) can't handle it. > > Jonathan > > Matt Mitchell wrote: > Jonathan, > > Well, I guess it really just comes to how you look at it. Forks for > creating entirely new projects is one thing, but forking for > managing code that will continuously be playing together is another. > It's actually a lot more than somewhat improved support for merging. > But don't take my word for it! > > Matt > > On Thu, Mar 5, 2009 at 2:48 PM, Jonathan Rochkind >> wrote: > Hmm, that solution sounds to me like nothing but somewhat improved > support for merging forks. I think avoiding forks in the first place > is probably better, where possible. > > Matt Mitchell wrote: > To be open and honest, this is where github and git in general could > help out so much. Because we each have our own branches, all > separate. We'd only have to *imply* that one is the authoritative > source. Jennifer could implement her home page and if Johnathan > liked it, he'd pull and merge it into his repo. If not, no problem. > Instead of one central repo, they're all spread out and managed > independently by each orgranization. We might want to at least read > up on git and github, to see how things could be easier? There's a > reason so many projects, with lots of developers are starting to use > it (them). > > Matt > > On Thu, Mar 5, 2009 at 12:55 PM, Jennifer Vine >>> wrote: > Jonathan, I think this was discussed the week before you joined the > Friday > meeting. I can't speak to the technical side (Bess/Naomi?), but we > wanted to > create a flexible landing page, with the search box and facets as page > elements, but also the ability to highlight specific collections, add > communications to the users, and so on. I haven't done extensive user > testing, but my observation so far is that users have been somewhat > disconcerted by landing directly in a default search and take a few > seconds > to get their bearings. > > JV > > > Jennifer Vine | jvine at stanford.edu > > > > UI Designer < Digital Library Systems & Services < SULAIR > > > > -----Original Message----- > From: blacklight-development-bounces at rubyforge.org > >> > [mailto:blacklight-development-bounces at rubyforge.org > >>] On Behalf Of Jonathan > Rochkind > Sent: Thursday, March 05, 2009 8:24 AM > To: blacklight-development at rubyforge.org > >> > Subject: Re: [Blacklight-development] Plugin Splash Page almost ready > > Huh, can you give more background on why this is a good idea? I'm not > seeing the purpose, I think I like it the way it is. > > Jessie Keck wrote: > > Hi all, > I'm writing to make sure that some code that I'm about to commit won't > interfere w/ anybody else's current work. > > I'm working on a splash page for the plugin, so that the user isn't > taken to the catalog controller w/ search results as the root. > > I will be adding the following files (note that the 'home' directory > under views will be created) : > /controllers/home_controller.rb > /views/home/index.html.erb > /views/home/_home_text.html.erb > > I will be making modifications to the following files: > routes.rb (map.root now points to the HomeController) > application.css (adding some padding to the home page text div) > > Unless I hear any yelps from anybody by tomorrow morning (PST) I'll > try > and get this committed. > > -Jessie Keck > Stanford University Libraries > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > >> > > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > >> > > > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > >> > > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org http://rubyforge.org/mailman/listinfo/blacklight-development From rochkind at jhu.edu Thu Mar 5 23:50:53 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 5 Mar 2009 23:50:53 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <7784D031-5751-4648-9000-7A69A8EF0813@mac.com> <49B046E3.3000007@jhu.edu>, Message-ID: <90FF863A96E1EC42B8B240D04C88FB1D719FDFD2@JHEMTEXVS2.win.ad.jhu.edu> To me, the problem with this approach, which is basically just one species of 'generator' approach, is that the stuff you get from Blacklight to set up your basic application is fixed at the moment you install, unless you make manual changes. That's the problem with all 'generator' approaches. But Blacklight might change it's mind about what the default setup should be; there may be refactorings which require such changes in order to keep working, or to take advantage of new features. In any generator style approach like this, you need to merge those changes in yourself. If the default stuff in Bill's #4 is instead _contained_ in the Plugin, you get it's changed automatically when you update the Plugin. You might need to fine-tune your local 'overrides' depending on how significant the changes are, but many changes can be made backwards compatible, the amount of human attention you need to pay to merging localizations with new versions is significantly reduced. It's like the difference between giving someone a CSS file, and telling them to customize it to their local needs; vs. having a standard CSS file, which is then over-ridden by a local CSS file (the "Cascading" in CSS). The latter option means when a new version of the standard distro CSS file comes out, and is installed into place, you don't need to worry about merging changes into your locations. Or for another analogy, when upgrading Umlaut to a new version of Rails, the biggest headache was figuring out that some problems were caused by the "boot.rb" and "environment.rb" files in Umlaut, which needed to be manually merged with changes required by the new versions of Rails. Most everything else just worked, since the Rails code lives in the gem external to the application itself and was almost entirely backwards compatible at the API level. Of course, some things need to be in the application, there's no getting around that for a simple boot.rb in Rails (but apparently not so simple that it could remain forwards-compatible!). But imagine if Rails 'generated' all it's base class code into your application instead of keeping it in the gem, upgrades would be a nightmare. For any kind of non-trivial standard logic or configuration, I'll always prefer a dynamic/linked style solution, like having the stuff live in the plug-in, then a generator solution that has the stuff live in the local application but be downloaded or generated there at the moment of installation. Jonathan ________________________________________ From: blacklight-development-bounces at rubyforge.org [blacklight-development-bounces at rubyforge.org] On Behalf Of Bill Dueber [bill at dueber.com] Sent: Thursday, March 05, 2009 11:20 PM To: blacklight-development at rubyforge.org Subject: Re: [Blacklight-development] Plugin Splash Page almost ready On Thu, Mar 5, 2009 at 4:40 PM, Jonathan Rochkind > wrote: I get it, but I think one of our priorities here is having Blacklight work as _easily_ as possible out of the box for people with standard needs, with as little setup as possible. While also supporting advanced needs through rational local 'over-riding' at the right granularity. I think this goal, at least, could be met by a combination of blacklight-is-only-a-plugin architecture along with a shipped out-of-the-box application with sane defaults. We are, then, talking about four components, yes? 1. solr 2. solrmarc 2.5 Sample blacklight configuration and customizations for solrmarc 3. blacklight, the rails plugin 4. a rails application that uses blacklight #4 would contain default routing and such, with the expectation (but not requirement) that much of it be revised or replaced. There will inevitably be a lot of configuration overlap between these components -- can someone who actually knows what they're doing map it out in a rough way? It'd be nice if configuration didn't need to be repeated (e.g., can't blacklight.properties and schema.xml come from a single source file??) -Bill- From rochkind at jhu.edu Fri Mar 6 11:24:34 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Fri, 06 Mar 2009 11:24:34 -0500 Subject: [Blacklight-development] Plugin Splash Page almost ready In-Reply-To: <98DA6E6C-ABBD-4ACA-9141-C2299CC36F05@mac.com> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <49B038A3.6060403@jhu.edu> <98DA6E6C-ABBD-4ACA-9141-C2299CC36F05@mac.com> Message-ID: <49B14E42.8090904@jhu.edu> I think a nice middle ground might be if the app can import Blacklight default roots with a single command in routes.rb. No need to generate multiple controllers (unless you actually want to over-ride something), just one line in routes.rb. But the app can choose not to include that line if it really wants to do all the routes itself. Even if each controller simply needs to call one line to get Blacklight functionality for the controller, this generator approach would still mean that if a new version of Blacklight introduced _new_ controllers, new generation/merging would have to be done. Among other issues. So I really still like having routes in the plugin, but rather than have them automatically added to the app by the plugin, perhaps it would meet Erik's concerns if it required an explicit one-line import type statement in your app's routes.rb. Jonathan Erik Hatcher wrote: > On Mar 5, 2009, at 3:40 PM, Jonathan Rochkind wrote: > >> I don't actually agree that the plugin should avoid doing any >> routing at all, I think it makes sense for it to do routing. But >> that routing can be easily over-ridden by the local application, can >> it not? >> > > Why override routing when all it takes is for a Blacklight using > application to have it's own controller that simply says > "blacklight :qf => 'title^2 body'" or something like that? [I say > this being a bit out of the Rails loop these days though] > > Blacklight is NOT the application, but a plugin. The application can > wire in the routing very simply. And of course Blacklight could > include generators to easily do something like: > > script/generate blacklight_controller [params] > > to add in a controller. > > And of course the default install of Blacklight would/could be a full- > on app skeleton that already had the controller in place to kick start > the whole thing. > > >> I think it's appropriate for an individual application to do that in >> the individual application, "over-riding" the plug-ins routing. >> Isn't that the point of making this an Engines plug-in? >> > > Overriding is fine for some things, but in this case it seems like > Blacklight plugin is doing too much for application and somewhat > getting in the way. > > Erik > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From ndushay at stanford.edu Fri Mar 6 14:23:49 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Fri, 6 Mar 2009 11:23:49 -0800 Subject: [Blacklight-development] landing/home page for blacklight In-Reply-To: <49B14E42.8090904@jhu.edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <49B038A3.6060403@jhu.edu> <98DA6E6C-ABBD-4ACA-9141-C2299CC36F05@mac.com> <49B14E42.8090904@jhu.edu> Message-ID: <439C6CD6-08F4-4384-B38F-13A9EF883B8D@stanford.edu> Jessie's original post of a few days ago was about a landing/home page for blacklight. He will be posting a message later today about the motivations for this change. As a preview: think about your OPAC or about www.google.com or about other discovery applications: do they start out with search results, or do they start out with a landing page? What should you see when you click "start over"? What you decide to put *on* the landing page is your choice -- it can be a search query box, or facets, or search results, or whatever. Note that the code has NOT been committed yet. We are having the discussion. We should probably have kicked off the discussion before Jessie started coding ... but so far, the only harm done is Jessie's time and effort). We are all committed to the notion of open source, and we all want the process to be as inclusive as possible. It is the nature of the beast that there will be side conversations, and that ideas will be discussed in them. Some of those side conversations will be in person, on the phone, in IM, or whatever. I think I can speak for us all when I say that we *all* have the goal of promoting broad discussion here. When we slip up, let us know ... nicely. I will add this little anecdote. I was fortunate in 1. attending code4lib and 2. spending some time with Matt Zumwalt. Matt was talking about situations where he's got a lot of complex information to communicate. He talked to an expert about this, and the expert's advice was: "pick up the phone." Real time conversation has its merits. - Naomi From ndushay at stanford.edu Fri Mar 6 14:31:56 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Fri, 6 Mar 2009 11:31:56 -0800 Subject: [Blacklight-development] managing our general code base vs. application specific Message-ID: <58A2008A-5C86-4559-BC72-88C2764BA436@stanford.edu> I think we are now having at least three different discussions under a single topic, so I am splitting them out. How should we be managing our general code base vs. application specific? Matt proposes using Git. I agree with Erik that requiring a specific source control methodology could be a barrier. It adds one more technology to the blacklight stack. Frankly, I think one deterrent to blacklight adoption is RoR ... because I doubt there is a lot of RoR expertise avail in the library techies world. We're already asking sysadmins for Ruby, Rails, various fun with specific gems. And adding Git into the mix is unlikely to make us *more* popular. Also, approaching the "community code" as Matt suggests seems to imply that we all would make a git repository avail to all, or put our site specific code in a general repos. That raises a different question: we probably will need a way to keep a searchable list of local mods that could be of use to others. I suppose we could use JIRA ... For the record, I decided against it for our local development, for the reasons above. Back to Git: I have also heard that the documentation is not great. Also, does it integrate well with our tools? I use Eclipse and Textmate, personally. I'm willing to try git; I've heard great things about it. But are the tradeoffs worth it? Naomi Dushay ndushay at stanford.edu From ndushay at stanford.edu Fri Mar 6 14:40:20 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Fri, 6 Mar 2009 11:40:20 -0800 Subject: [Blacklight-development] routing - how to deal with it : generic code? site code? Message-ID: <77B4B12A-489D-45DC-9CCA-A9985652D2D0@stanford.edu> This is the third thread kicked off by Jessie's note. I confess i got a little lost. Are we saying that adding the additional controller for the home/ landing page affects the routing irrevocably? Isn't it simple to just do a redirect or change the view if you DON'T want the home/landing page? But harder to add routing in if you DO want it? I think a discovery application is non-trivial; I liked Erik's parallel to solr. Solr comes with a demo; blacklight provides one. Solr comes with most of the code most sites will need, and configuration files ... and documentation. Blacklight is working on those things. (Naturally, documentation lags a bit.) Doing anything significant with solr requires rolling up your sleeves. I don't have a problem with blacklight requiring the same ... especially if a great deal can be done without much coding. Solrmarc and some naming conventions make that possible with blacklight, no? Naomi Dushay ndushay at stanford.edu From rochkind at jhu.edu Fri Mar 6 15:15:19 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Fri, 06 Mar 2009 15:15:19 -0500 Subject: [Blacklight-development] routing - how to deal with it : generic code? site code? In-Reply-To: <77B4B12A-489D-45DC-9CCA-A9985652D2D0@stanford.edu> References: <77B4B12A-489D-45DC-9CCA-A9985652D2D0@stanford.edu> Message-ID: <49B18457.1000105@jhu.edu> It is very easy to add any routing in your app if you do want it but it's not in the plugin. It seems that it's more difficult to change existing routing from the plugin. Jonathan Naomi Dushay wrote: > This is the third thread kicked off by Jessie's note. I confess i got > a little lost. > > Are we saying that adding the additional controller for the home/ > landing page affects the routing irrevocably? > > Isn't it simple to just do a redirect or change the view if you DON'T > want the home/landing page? But harder to add routing in if you DO > want it? > > I think a discovery application is non-trivial; I liked Erik's > parallel to solr. Solr comes with a demo; blacklight provides one. > Solr comes with most of the code most sites will need, and > configuration files ... and documentation. Blacklight is working on > those things. (Naturally, documentation lags a bit.) Doing anything > significant with solr requires rolling up your sleeves. I don't have > a problem with blacklight requiring the same ... especially if a great > deal can be done without much coding. Solrmarc and some naming > conventions make that possible with blacklight, no? > > Naomi Dushay > ndushay at stanford.edu > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From jkeck at stanford.edu Fri Mar 6 19:11:53 2009 From: jkeck at stanford.edu (Jessie Keck) Date: Fri, 06 Mar 2009 16:11:53 -0800 Subject: [Blacklight-development] landing/home page for blacklight In-Reply-To: <439C6CD6-08F4-4384-B38F-13A9EF883B8D@stanford.edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <49B038A3.6060403@jhu.edu> <98DA6E6C-ABBD-4ACA-9141-C2299CC36F05@mac.com> <49B14E42.8090904@jhu.edu> <439C6CD6-08F4-4384-B38F-13A9EF883B8D@stanford.edu> Message-ID: <49B1BBC9.5080707@stanford.edu> Hi all, Here is my breakdown of the thoughts behind a splash/landing page and why it should be in the plugin: First off, the notion that the plugin itself will not have any UI patterns is pretty far off. The plugin as is right now has 27 views. In order to get the application developer up and running as quickly as possible, we need to have a base set of views ready to be overwritten. IMHO a landing page of a results set if just plain confusing. I would be interested to see if anybody who will use blacklight will want this as their landing page. I have yet to find a production catalog with that behavior. At that point the vast majority of the developers who are installing blacklight will then need to develop their homepage functionality directly (plus w/ the difficulty in overriding the blacklight default map.root ). Everybody has some sort of landing page. The UVA modified instance of blacklight has landing page w/ Recent items, vuFind ships with home page text. JHU, Stanford, UVA, we all have a landing page for our catalog and blacklight should ship no different. With a default landing page developer will only need to override 1 view (_home_text.html.erb) to get started with their first modification. An instant sense of gratification and accomplishment. This is one of the big reasons why vuFind got so popular. It's functional and usable right out of the box. Look at almost all of the vuFind instances up and they all look relatively the same. Also, I'm sure that the first modification to almost all the vuFind apps is the change to that front page text (the text was instructions on how to change that text, which is what I am proposing to do). While I understand that this does bring up a larger conversation of how much application level code is in an engine, I think it goes without saying that users will expect some application to be provided by the engine. Otherwise all of their development will be on their shoulders and that's not very desirable in any perspective. Stanford (as well as those looking into blacklight) is adopting blacklight expecting an application that is both flexible and extensible (Rails and Engines is providing the extensibility). I as a developer would want to do the least amount of application development to get basic and generalizable functionality out of an application (a home page is functionally generalizable in any application, an OPAC not being an exception here). A basic application framework that has a good architecture out of the box will be very desirable to future adopters. In any case, if a developer for some reason would like their landing page to be a search result of a nill query, then they can override the HomeController and delete 3 lines of code. This will then direct any requests to the HomeController to the CatalogController, which is the current out of the box behavior. Although it requires a redirect it's a lot easier than creating a new controller, 2 new views, and trying to figure out how to get the override blacklights map.root to your new controller. On a final note, to all of those on the list interested in helping drive the architecture of blacklight, I would highly recommend checking out the Roadmap ( http://wiki.blacklightopac.org/doku.php?id=roadmap ) and our JIRA ( http://blacklightopac.org:8080/jira ). While this feature was not discussed on this list, it was in both of these places. I just don't want to give any impression that we were trying to not include those in the blacklight community on this decision (we mistakenly only had in 2 of the 3 forums of discussion) . Thoughts? Stones to be thrown? -Jessie Keck Stanford University Naomi Dushay wrote: > Jessie's original post of a few days ago was about a landing/home page > for blacklight. He will be posting a message later today about the > motivations for this change. > > As a preview: think about your OPAC or about www.google.com or about > other discovery applications: do they start out with search results, > or do they start out with a landing page? What should you see when > you click "start over"? What you decide to put *on* the landing page > is your choice -- it can be a search query box, or facets, or search > results, or whatever. > > Note that the code has NOT been committed yet. We are having the > discussion. We should probably have kicked off the discussion before > Jessie started coding ... but so far, the only harm done is Jessie's > time and effort). > > We are all committed to the notion of open source, and we all want the > process to be as inclusive as possible. It is the nature of the beast > that there will be side conversations, and that ideas will be > discussed in them. Some of those side conversations will be in > person, on the phone, in IM, or whatever. I think I can speak for us > all when I say that we *all* have the goal of promoting broad > discussion here. When we slip up, let us know ... nicely. > > I will add this little anecdote. I was fortunate in 1. attending > code4lib and 2. spending some time with Matt Zumwalt. Matt was > talking about situations where he's got a lot of complex information > to communicate. He talked to an expert about this, and the expert's > advice was: "pick up the phone." Real time conversation has its > merits. > > - Naomi > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development From eos8d at virginia.edu Fri Mar 6 20:43:07 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Fri, 6 Mar 2009 20:43:07 -0500 Subject: [Blacklight-development] landing/home page for blacklight In-Reply-To: <49B1BBC9.5080707@stanford.edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <49B038A3.6060403@jhu.edu> <98DA6E6C-ABBD-4ACA-9141-C2299CC36F05@mac.com> <49B14E42.8090904@jhu.edu> <439C6CD6-08F4-4384-B38F-13A9EF883B8D@stanford.edu> <49B1BBC9.5080707@stanford.edu> Message-ID: Well said, Jessie. I think this would be a great feature and I'd like to see you commit the code. Bess On 6-Mar-09, at 7:11 PM, Jessie Keck wrote: > Hi all, > Here is my breakdown of the thoughts behind a splash/landing page and > why it should be in the plugin: > > First off, the notion that the plugin itself will not have any UI > patterns is pretty far off. The plugin as is right now has 27 views. > In order to get the application developer up and running as quickly as > possible, we need to have a base set of views ready to be overwritten. > > IMHO a landing page of a results set if just plain confusing. I would > be interested to see if anybody who will use blacklight will want this > as their landing page. I have yet to find a production catalog with > that behavior. At that point the vast majority of the developers who > are installing blacklight will then need to develop their homepage > functionality directly (plus w/ the difficulty in overriding the > blacklight default map.root ). Everybody has some sort of landing > page. The UVA modified instance of blacklight has landing page w/ > Recent items, vuFind ships with home page text. JHU, Stanford, UVA, > we > all have a landing page for our catalog and blacklight should ship no > different. > > With a default landing page developer will only need to override 1 > view > (_home_text.html.erb) to get started with their first modification. > An > instant sense of gratification and accomplishment. > > This is one of the big reasons why vuFind got so popular. It's > functional and usable right out of the box. Look at almost all of the > vuFind instances up and they all look relatively the same. Also, I'm > sure that the first modification to almost all the vuFind apps is the > change to that front page text (the text was instructions on how to > change that text, which is what I am proposing to do). > > While I understand that this does bring up a larger conversation of > how > much application level code is in an engine, I think it goes without > saying that users will expect some application to be provided by the > engine. Otherwise all of their development will be on their shoulders > and that's not very desirable in any perspective. > > Stanford (as well as those looking into blacklight) is adopting > blacklight expecting an application that is both flexible and > extensible > (Rails and Engines is providing the extensibility). I as a developer > would want to do the least amount of application development to get > basic and generalizable functionality out of an application (a home > page > is functionally generalizable in any application, an OPAC not being an > exception here). A basic application framework that has a good > architecture out of the box will be very desirable to future adopters. > > In any case, if a developer for some reason would like their landing > page to be a search result of a nill query, then they can override the > HomeController and delete 3 lines of code. This will then direct any > requests to the HomeController to the CatalogController, which is the > current out of the box behavior. Although it requires a redirect > it's a > lot easier than creating a new controller, 2 new views, and trying to > figure out how to get the override blacklights map.root to your new > controller. > > On a final note, to all of those on the list interested in helping > drive > the architecture of blacklight, I would highly recommend checking out > the Roadmap ( http://wiki.blacklightopac.org/doku.php?id=roadmap ) and > our JIRA ( http://blacklightopac.org:8080/jira ). While this feature > was not discussed on this list, it was in both of these places. I > just > don't want to give any impression that we were trying to not include > those in the blacklight community on this decision (we mistakenly only > had in 2 of the 3 forums of discussion) . > > Thoughts? Stones to be thrown? > -Jessie Keck > Stanford University > > Naomi Dushay wrote: >> Jessie's original post of a few days ago was about a landing/home >> page >> for blacklight. He will be posting a message later today about the >> motivations for this change. >> >> As a preview: think about your OPAC or about www.google.com or about >> other discovery applications: do they start out with search results, >> or do they start out with a landing page? What should you see when >> you click "start over"? What you decide to put *on* the landing >> page >> is your choice -- it can be a search query box, or facets, or search >> results, or whatever. >> >> Note that the code has NOT been committed yet. We are having the >> discussion. We should probably have kicked off the discussion before >> Jessie started coding ... but so far, the only harm done is Jessie's >> time and effort). >> >> We are all committed to the notion of open source, and we all want >> the >> process to be as inclusive as possible. It is the nature of the >> beast >> that there will be side conversations, and that ideas will be >> discussed in them. Some of those side conversations will be in >> person, on the phone, in IM, or whatever. I think I can speak for us >> all when I say that we *all* have the goal of promoting broad >> discussion here. When we slip up, let us know ... nicely. >> >> I will add this little anecdote. I was fortunate in 1. attending >> code4lib and 2. spending some time with Matt Zumwalt. Matt was >> talking about situations where he's got a lot of complex information >> to communicate. He talked to an expert about this, and the expert's >> advice was: "pick up the phone." Real time conversation has its >> merits. >> >> - Naomi >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development From goodieboy at gmail.com Fri Mar 6 22:52:29 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Fri, 6 Mar 2009 22:52:29 -0500 Subject: [Blacklight-development] landing/home page for blacklight In-Reply-To: <49B1BBC9.5080707@stanford.edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <49B038A3.6060403@jhu.edu> <98DA6E6C-ABBD-4ACA-9141-C2299CC36F05@mac.com> <49B14E42.8090904@jhu.edu> <439C6CD6-08F4-4384-B38F-13A9EF883B8D@stanford.edu> <49B1BBC9.5080707@stanford.edu> Message-ID: <056DAFE8-6EC9-4129-8984-ABD21A81C76C@gmail.com> +1 great points! Thanks for taking the time to lay that all out for us Jessie. Matt Sent from my iPhone On Mar 6, 2009, at 7:11 PM, Jessie Keck wrote: > Hi all, > Here is my breakdown of the thoughts behind a splash/landing page > and why it should be in the plugin: > > First off, the notion that the plugin itself will not have any UI > patterns is pretty far off. The plugin as is right now has 27 > views. In order to get the application developer up and running as > quickly as possible, we need to have a base set of views ready to be > overwritten. > > IMHO a landing page of a results set if just plain confusing. I > would be interested to see if anybody who will use blacklight will > want this as their landing page. I have yet to find a production > catalog with that behavior. At that point the vast majority of the > developers who are installing blacklight will then need to develop > their homepage functionality directly (plus w/ the difficulty in > overriding the blacklight default map.root ). Everybody has some > sort of landing page. The UVA modified instance of blacklight has > landing page w/ Recent items, vuFind ships with home page text. > JHU, Stanford, UVA, we all have a landing page for our catalog and > blacklight should ship no different. > > With a default landing page developer will only need to override 1 > view (_home_text.html.erb) to get started with their first > modification. An instant sense of gratification and accomplishment. > This is one of the big reasons why vuFind got so popular. It's > functional and usable right out of the box. Look at almost all of > the vuFind instances up and they all look relatively the same. > Also, I'm sure that the first modification to almost all the vuFind > apps is the change to that front page text (the text was > instructions on how to change that text, which is what I am > proposing to do). > > While I understand that this does bring up a larger conversation of > how much application level code is in an engine, I think it goes > without saying that users will expect some application to be > provided by the engine. Otherwise all of their development will be > on their shoulders and that's not very desirable in any perspective. > > Stanford (as well as those looking into blacklight) is adopting > blacklight expecting an application that is both flexible and > extensible (Rails and Engines is providing the extensibility). I as > a developer would want to do the least amount of application > development to get basic and generalizable functionality out of an > application (a home page is functionally generalizable in any > application, an OPAC not being an exception here). A basic > application framework that has a good architecture out of the box > will be very desirable to future adopters. > > In any case, if a developer for some reason would like their landing > page to be a search result of a nill query, then they can override > the HomeController and delete 3 lines of code. This will then > direct any requests to the HomeController to the CatalogController, > which is the current out of the box behavior. Although it requires > a redirect it's a lot easier than creating a new controller, 2 new > views, and trying to figure out how to get the override blacklights > map.root to your new controller. > > On a final note, to all of those on the list interested in helping > drive the architecture of blacklight, I would highly recommend > checking out the Roadmap ( http://wiki.blacklightopac.org/doku.php?id=roadmap > ) and our JIRA ( http://blacklightopac.org:8080/jira ). While this > feature was not discussed on this list, it was in both of these > places. I just don't want to give any impression that we were > trying to not include those in the blacklight community on this > decision (we mistakenly only had in 2 of the 3 forums of discussion) . > > Thoughts? Stones to be thrown? > -Jessie Keck > Stanford University > > Naomi Dushay wrote: >> Jessie's original post of a few days ago was about a landing/home >> page for blacklight. He will be posting a message later today >> about the motivations for this change. >> >> As a preview: think about your OPAC or about www.google.com or >> about other discovery applications: do they start out with search >> results, or do they start out with a landing page? What should you >> see when you click "start over"? What you decide to put *on* the >> landing page is your choice -- it can be a search query box, or >> facets, or search results, or whatever. >> >> Note that the code has NOT been committed yet. We are having the >> discussion. We should probably have kicked off the discussion >> before Jessie started coding ... but so far, the only harm done is >> Jessie's time and effort). >> >> We are all committed to the notion of open source, and we all want >> the process to be as inclusive as possible. It is the nature of >> the beast that there will be side conversations, and that ideas >> will be discussed in them. Some of those side conversations will >> be in person, on the phone, in IM, or whatever. I think I can >> speak for us all when I say that we *all* have the goal of >> promoting broad discussion here. When we slip up, let us know ... >> nicely. >> >> I will add this little anecdote. I was fortunate in 1. attending >> code4lib and 2. spending some time with Matt Zumwalt. Matt was >> talking about situations where he's got a lot of complex >> information to communicate. He talked to an expert about this, and >> the expert's advice was: "pick up the phone." Real time >> conversation has its merits. >> >> - Naomi >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development From erikhatcher at mac.com Sat Mar 7 06:34:11 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Sat, 07 Mar 2009 06:34:11 -0500 Subject: [Blacklight-development] landing/home page for blacklight In-Reply-To: <49B1BBC9.5080707@stanford.edu> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <49B038A3.6060403@jhu.edu> <98DA6E6C-ABBD-4ACA-9141-C2299CC36F05@mac.com> <49B14E42.8090904@jhu.edu> <439C6CD6-08F4-4384-B38F-13A9EF883B8D@stanford.edu> <49B1BBC9.5080707@stanford.edu> Message-ID: <41C4F35B-B2A2-4109-9E6E-98D36A444812@mac.com> On Mar 6, 2009, at 7:11 PM, Jessie Keck wrote: > First off, the notion that the plugin itself will not have any UI > patterns is pretty far off. The plugin as is right now has 27 > views. In order to get the application developer up and running as > quickly as possible, we need to have a base set of views ready to be > overwritten. I don't think anyone suggested otherwise. My only suggestion was that the *application* control routing rather than the plugin. > IMHO a landing page of a results set if just plain confusing. But, a landing page that shows facets is highly desirable, I'd think. And that requires a request to Solr. > (the text was instructions on how to change that text, which is > what I am proposing to do). +1 for sure. Putting in CHANGE THIS kinda stuff makes a lot of sense. > While I understand that this does bring up a larger conversation of > how much application level code is in an engine, I think it goes > without saying that users will expect some application to be > provided by the engine. Otherwise all of their development will be > on their shoulders and that's not very desirable in any perspective. They'll expect a starter application, for sure. But whether the engine provides all that without any thing required in the surrounding Rails application is the question. I expect a ton of other stuff in most surrounding applications besides just what Blacklight offers, and keeping Blacklight from interfering with general routes is all I'm putting forther here. > On a final note, to all of those on the list interested in helping > drive the architecture of blacklight, I would highly recommend > checking out the Roadmap ( http://wiki.blacklightopac.org/doku.php?id=roadmap > ) and our JIRA ( http://blacklightopac.org:8080/jira ). While this > feature was not discussed on this list, it was in both of these > places. I just don't want to give any impression that we were > trying to not include those in the blacklight community on this > decision (we mistakenly only had in 2 of the 3 forums of discussion) . +1 Good discussion, Jesse. As always, the real goodness is in working systems - so keep pressing on developing what you need for Stanford, and along with UVa and perhaps JHU, things will generalize out nicely. It just takes working through the needs of each site, and I'm confident that the expertise is here to generalize this stuff cleanly. Erik From ndushay at stanford.edu Sat Mar 7 12:11:23 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Sat, 7 Mar 2009 09:11:23 -0800 Subject: [Blacklight-development] landing/home page for blacklight In-Reply-To: <41C4F35B-B2A2-4109-9E6E-98D36A444812@mac.com> References: <49AF2688.8000101@stanford.edu> <49AFFCB9.3000908@jhu.edu> <002701c99dbb$990fbb40$cb2f31c0$@edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <49B038A3.6060403@jhu.edu> <98DA6E6C-ABBD-4ACA-9141-C2299CC36F05@mac.com> <49B14E42.8090904@jhu.edu> <439C6CD6-08F4-4384-B38F-13A9EF883B8D@stanford.edu> <49B1BBC9.5080707@stanford.edu> <41C4F35B-B2A2-4109-9E6E-98D36A444812@mac.com> Message-ID: On Mar 7, 2009, at 3:34 AM, Erik Hatcher wrote: > > On Mar 6, 2009, at 7:11 PM, Jessie Keck wrote: >> First off, the notion that the plugin itself will not have any UI >> patterns is pretty far off. The plugin as is right now has 27 >> views. In order to get the application developer up and running as >> quickly as possible, we need to have a base set of views ready to >> be overwritten. > > I don't think anyone suggested otherwise. My only suggestion was > that the *application* control routing rather than the plugin. >> While I understand that this does bring up a larger conversation >> of how much application level code is in an engine, I think it goes >> without saying that users will expect some application to be >> provided by the engine. Otherwise all of their development will be >> on their shoulders and that's not very desirable in any perspective. > > They'll expect a starter application, for sure. But whether the > engine provides all that without any thing required in the > surrounding Rails application is the question. I expect a ton of > other stuff in most surrounding applications besides just what > Blacklight offers, and keeping Blacklight from interfering with > general routes is all I'm putting forther here. > Can someone explain how we would do routing in the rails app, vs. doing in in the engine/plugin? Or is this a RTFM moment? I think my RoR noob-ness may be getting in my way here. >> IMHO a landing page of a results set if just plain confusing. > > But, a landing page that shows facets is highly desirable, I'd > think. And that requires a request to Solr. exactly! That what we want, and what Jessie is going for. > As always, the real goodness is in working systems - so keep > pressing on developing what you need for Stanford, and along with > UVa and perhaps JHU, things will generalize out nicely. It just > takes working through the needs of each site, and I'm confident that > the expertise is here to generalize this stuff cleanly. +1 - Naomi From goodieboy at gmail.com Sat Mar 7 13:19:18 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Sat, 7 Mar 2009 13:19:18 -0500 Subject: [Blacklight-development] landing/home page for blacklight In-Reply-To: References: <49AF2688.8000101@stanford.edu> <12BA36ED-DAE2-4B27-979C-E07FF3486904@mac.com> <49B038A3.6060403@jhu.edu> <98DA6E6C-ABBD-4ACA-9141-C2299CC36F05@mac.com> <49B14E42.8090904@jhu.edu> <439C6CD6-08F4-4384-B38F-13A9EF883B8D@stanford.edu> <49B1BBC9.5080707@stanford.edu> <41C4F35B-B2A2-4109-9E6E-98D36A444812@mac.com> Message-ID: OK here is the deal with the routes... The current BL architecture is using the Engine plugin; a third-party plugin for Rails. This plugin allows other plugins to function as min-apps. The Engine plugin makes it possible for your plugin (BL) to interact with your app (Johns Hopkins, Stanford, UVa etc.). The routing mechanism in the Engine plugin works by calling a special method in your routes file, which is created by the Engine plugin. It's not absolutely required, just a convenience for auto-routing. When this method is used, it's simply grabbing all of the routes from the mini-app plugin (BL) and executed them within the context of the main app (Stanford etc.). One of the questions seems to be, how do we then remove routes that won't get used by the main app? At this point, I don't know. I'd have to look at the ActionController::Routing API. But I can't imagine it'd be impossible to do. Another way to do this would be to create a Blacklight::Routes class that contains the route definitions, and a mechinism for specifying which routes get sent to the main app: # inside the stanford/config/routes.rb block -> blroutes = Blacklight::Routes.new blroutes.apply(map, :only=>[:catalog]) where map is the map variable in the routes file. By the way... the new version of Rails (2.3) does a lot of what the Engine plugin does, minus code magical mixing, routes and migrations. Might be good to keep that in mind. Matt On Sat, Mar 7, 2009 at 12:11 PM, Naomi Dushay wrote: > > On Mar 7, 2009, at 3:34 AM, Erik Hatcher wrote: > > >> On Mar 6, 2009, at 7:11 PM, Jessie Keck wrote: >> >>> First off, the notion that the plugin itself will not have any UI >>> patterns is pretty far off. The plugin as is right now has 27 views. In >>> order to get the application developer up and running as quickly as >>> possible, we need to have a base set of views ready to be overwritten. >>> >> >> I don't think anyone suggested otherwise. My only suggestion was that the >> *application* control routing rather than the plugin. >> > > While I understand that this does bring up a larger conversation of how >>> much application level code is in an engine, I think it goes without saying >>> that users will expect some application to be provided by the engine. >>> Otherwise all of their development will be on their shoulders and that's >>> not very desirable in any perspective. >>> >> >> They'll expect a starter application, for sure. But whether the engine >> provides all that without any thing required in the surrounding Rails >> application is the question. I expect a ton of other stuff in most >> surrounding applications besides just what Blacklight offers, and keeping >> Blacklight from interfering with general routes is all I'm putting forther >> here. >> >> > Can someone explain how we would do routing in the rails app, vs. doing in > in the engine/plugin? Or is this a RTFM moment? I think my RoR noob-ness > may be getting in my way here. > > IMHO a landing page of a results set if just plain confusing. >>> >> >> But, a landing page that shows facets is highly desirable, I'd think. And >> that requires a request to Solr. >> > > exactly! That what we want, and what Jessie is going for. > > As always, the real goodness is in working systems - so keep pressing on >> developing what you need for Stanford, and along with UVa and perhaps JHU, >> things will generalize out nicely. It just takes working through the needs >> of each site, and I'm confident that the expertise is here to generalize >> this stuff cleanly. >> > > +1 > > - Naomi > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chapman.lists at gmail.com Mon Mar 9 14:37:18 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Mon, 09 Mar 2009 14:37:18 -0400 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: <49AF2FD3.1020102@stanford.edu> References: <49AEFB67.7080008@gmail.com> <49AF2FD3.1020102@stanford.edu> Message-ID: <49B561DE.6020109@gmail.com> I am trying to follow Jessie's directions (below) to customize the demo app for my own purposes. I copied the view (_document_list.html.erb) for catalog index into the controlling app's view/catalog directory. My problem seems is that when the link_to_document helper is called. The line reads <%= link_to_document document, :label=>:title_t, :extra_params=>{:counter => counter + 1 + @response.start.to_i } %> .My title field is named well title and not title_t so I changed the controling app's _document_list.html.erb to <%= link_to_document document, :label=>:title, :extra_params=>{:counter => counter + 1 + @response.start.to_i } %> Now I get the following when I try to access the index page of the catalog Showing app/views/catalog/_document_list.html.erb where line #21 raised: catalog_url failed to generate from {:commit=>nil, :action=>"show", :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, diff: {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} Extracted source (around line #21): 18: <% # main title container for doc partial view -%> 19: 20:
21:
<%= counter + 1 + @response.start.to_i %>. <%= link_to_document document, :label=>:title, :extra_params=>{:counter => counter + 1 + @response.start.to_i } %>
22:
23: 24: What am I screwing up this time? (LOL) Thnks in advance folks. Jessie Keck wrote: > Hi Vern, > Just chiming in about overriding the interface. > > If you find a view that you would like to override then you just need > to copy that file from the plugin and paste it in your top level > application (the one generated with =>> rails {your_app_name}) making > sure to keep the same directory structure as the plugin. > > The engines plugin will look at your top level app first to find a > file then to the blacklight plugin. An example of this is if you > wanted the search box to be available on all pages not just search > results. This is an easy way to do that: > 1) Copy file views/catalog/index.html.erb from plugin to the top level > app > 2) Remove render tag from the index view in the top level app > 3) Copy file views/catalog/_search_form.html erb to the top level app > to the views/shared directory (note that I did not maintain the > directory structure because the search form partial is now shared > amongst other controllers) > 4) Copy file views/layouts/application.html.erb to the top level app > 5) Add the render tag to application view to now be render :partial => > 'shared/search_form' > (Note: putting this is the application view is probably not the best > idea, but this is just an example. I'm actually rendering the search > form partial from a header partial, not the application view) > > That's about it. You can go nuts with overriding all kinds of files. > Do remember that certain things may require a restart to take affect > (most notably changes to helpers) > > Hope this helps, > Let us know if any of this is unclear or you run into any snags. > > -Jessie Keck > Stanford University Libraries > > Vernon Chapman wrote: >> Is there some documentation on how to customize blacklight? >> I have a lot or Dublin Core(ish) metadata in a solr index and would like >> to use blacklight as a front end to it. >> >> Thanks in advance >> >> Vern >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > From mpc3c at virginia.edu Mon Mar 9 15:53:56 2009 From: mpc3c at virginia.edu (Molly Pickral Cadieux) Date: Mon, 9 Mar 2009 14:53:56 -0500 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: <49B561DE.6020109@gmail.com> References: <49AEFB67.7080008@gmail.com> <49AF2FD3.1020102@stanford.edu> <49B561DE.6020109@gmail.com> Message-ID: <34e39bfd0903091253g1e44354y7260d129299985e4@mail.gmail.com> Folks, I am working on fixing this now. Molly On Mon, Mar 9, 2009 at 1:37 PM, Vernon Chapman wrote: > I am trying to follow Jessie's directions (below) ?to customize the demo > app for my own purposes. > I copied the view (_document_list.html.erb) for catalog index into the > controlling app's view/catalog directory. > > My problem seems is that when the link_to_document helper is called. The > line reads > <%= link_to_document document, :label=>:title_t, > :extra_params=>{:counter => counter + 1 + @response.start.to_i } %> > > .My title field is named well title and not title_t so I changed the > controling app's ?_document_list.html.erb to > <%= link_to_document document, :label=>:title, :extra_params=>{:counter > => counter + 1 + @response.start.to_i } %> > > Now I get the following when I try to access the index page of the catalog > > Showing app/views/catalog/_document_list.html.erb where line #21 raised: > > catalog_url failed to generate from {:commit=>nil, :action=>"show", > :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", > :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, diff: > {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} > > Extracted source (around line #21): > > 18: ? ? ? ? <% # main title container for doc partial view -%> > 19: > 20: ? ? ? ?
> 21: ? ? ? ? ?
<%= counter + 1 + @response.start.to_i %>. <%= > link_to_document document, :label=>:title, :extra_params=>{:counter => > counter + 1 + @response.start.to_i } %>
> 22: ? ? ? ?
> 23: > 24: ? ? ? > > What am I screwing up this time? (LOL) > Thnks in advance folks. > - Show quoted text - > > > Jessie Keck wrote: >> >> Hi Vern, >> Just chiming in about overriding the interface. >> >> If you find a view that you would like to override then you just need to >> copy that file from the plugin and paste it in your top level application >> (the one generated with =>> rails {your_app_name}) making sure to keep the >> same directory structure as the plugin. >> >> The engines plugin will look at your top level app first to find a file >> then to the blacklight plugin. An example of this is if you wanted the >> search box to be available on all pages not just search results. ?This is an >> easy way to do that: >> 1) Copy file views/catalog/index.html.erb from plugin to the top level app >> 2) Remove render tag from the index view in the top level app >> 3) Copy file views/catalog/_search_form.html erb to the top level app to >> the views/shared directory (note that I did not maintain the directory >> structure because the search form partial is now shared amongst other >> controllers) >> 4) Copy file views/layouts/application.html.erb to the top level app >> 5) Add the render tag to application view to now be render :partial => >> 'shared/search_form' >> (Note: putting this is the application view is probably not the best idea, >> but this is just an example. ?I'm actually rendering the search form partial >> from a header partial, not the application view) >> >> That's about it. ?You can go nuts with overriding all kinds of files. ?Do >> remember that certain things may require a restart to take affect (most >> notably changes to helpers) >> >> Hope this helps, >> Let us know if any of this is unclear or you run into any snags. >> >> -Jessie Keck >> Stanford University Libraries >> >> Vernon Chapman wrote: >>> >>> Is there some documentation on how to customize blacklight? >>> I have a lot or Dublin Core(ish) metadata in a solr index and would like >>> to use blacklight as a front end to it. >>> >>> Thanks in advance >>> >>> Vern >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From elarson at library.wisc.edu Mon Mar 9 15:35:53 2009 From: elarson at library.wisc.edu (Eric Larson) Date: Mon, 09 Mar 2009 14:35:53 -0500 Subject: [Blacklight-development] SolrMarc 2.0 - Getting Started? Message-ID: Hi all, We have begun playing with Blacklight at UW-Madison. While installation and importing data via the ruby rake task works fine, we'd like to try pushing 10% of our collection into our test install (1M+ records). To handle this larger import, I've been trying (unsuccessfully) to get the solrmarc-2.0 branch to work. The documentation in the GettingStarted.txt is not up to date. Below is a list of the steps I've been taking. If anyone can offer advice, or help me understand how solrmarc-2.0 wants to be setup, I'd greatly appreciate it. Right now, I'm planning to dive farther into vufind's import script to see how they have solrmarc working (I've successfully imported this marc set into a test vufind install). As referenced here, a SM guide for Blacklight would be *very appreciated* :) http://rubyforge.org/pipermail/blacklight-development/2009-January/000246.html Thanks! - Eric = = = = Test install (OSX) - - - - 1) Download blacklight-trunk /Users/elarson/Sites/blacklight * No problems here. * Test Solr import via rake works fine. 2) Download solrmarc-2.0 * In blacklight root dir - svn co svn checkout http://solrmarc.googlecode.com/svn/branches/solrmarc-2.0 solrmarc * Open and attempt to use /solrmarc/docs/GettingStarted.txt - ant -version => 1.7.0 - java -version => 1.5.0 - sudo ant : Init site => "uwmadison" : Jar file name => "uwmadison" : Prefix => "uwmadison" : Final jar file => "uwmadison_SolrMarc" : Heap => -Xmx265m (default) : Solr version => 1.3 (default) - BUILD SUCCESSFUL : Note: uwmadison_config_properties and uwmadison_index.properties are not created? - Edit importSamples.properties : This file doesn't exist... : Realize I'm on my own now... - Assuming I want to edit the VanillaBlacklightDemo properites files : demo_config.properties : demo_index.properties : Blacklight solr schema.xml should sync with demo_index.properties? : Copy these files to /solrmarc/uwmadison dir created during build : Rename to uwmadison_config.properties and uwmadison_index.properties - Edit: uwmadison_config.properties : solr.path => /Users/elarson/Sites/blacklight/jetty/solr - Leaving: uwmadison_index.properties : This *should* work with Vanilla blacklight install? - Assuming all should be ready to run the example import_file.sh script... : cd /solrmarc/dist : Using importfile script : # ./indexfile ../uwmadison/uwmadison_config.properties ../../../ vufind/import/catalog.mrc (yeah, we're testing vufind too) # Problem 1: Unable to find uwmadison_index.properties - Let's try the full path : /Users/elarson/Sites/blacklight/solrmarc/uwmadison/ uwmadison_index.properties # Problem 2: "Error: Problem instantiating SolrCore" - Let's start Solr :) - cd /blacklight/jetty; java -jar start.jar - Solr Starts - Retrying index script # Problem 3: "Error: Problem instantiating SolrCore" - Trace below ERROR [main] (SolrCoreLoader.java:149) - Error: Problem instantiating SolrCore java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun .reflect .NativeConstructorAccessorImpl .newInstance(NativeConstructorAccessorImpl.java:39) at sun .reflect .DelegatingConstructorAccessorImpl .newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:494) at org.solrmarc.solr.SolrCoreLoader.loadCore(SolrCoreLoader.java:121) at org.solrmarc.marc.MarcImporter.getSolrCoreProxy(MarcImporter.java: 446) at org.solrmarc.marc.MarcImporter.(MarcImporter.java:73) at org.solrmarc.marc.MarcImporter.main(MarcImporter.java:462) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun .reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: 39) at sun .reflect .DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 25) at java.lang.reflect.Method.invoke(Method.java:585) at com.simontuffs.onejar.Boot.run(Boot.java:306) at com.simontuffs.onejar.Boot.main(Boot.java:159) Caused by: org.apache.solr.common.SolrException: Error loading class 'solr.FastLRUCache' at org .apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java: 273) at org.apache.solr.search.CacheConfig.getConfig(CacheConfig.java:90) at org.apache.solr.search.CacheConfig.getConfig(CacheConfig.java:73) at org.apache.solr.core.SolrConfig.(SolrConfig.java:128) at org.apache.solr.core.SolrConfig.(SolrConfig.java:101) ... 14 more Caused by: java.lang.ClassNotFoundException: solr.FastLRUCache at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:316) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:579) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:374) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:242) at org .apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java: 257) ... 18 more - I'm not a Java person, so debugging the rest is _painful_ - A step by step Blacklight / SolrMarc guide would be fantastic!- I'm willing to help write this documentation, if someone can help we get this working locally. - - - - Again, thanks for the help. From elarson at library.wisc.edu Mon Mar 9 16:49:16 2009 From: elarson at library.wisc.edu (Eric Larson) Date: Mon, 09 Mar 2009 15:49:16 -0500 Subject: [Blacklight-development] SolrMarc 2.0 - Getting Started? In-Reply-To: <28533_1236631177_ZZg0p5jJY9vvz.00_EDF0AB56-7784-48C7-9D3B-818E145BDAEE@library.wisc.edu> References: <28533_1236631177_ZZg0p5jJY9vvz.00_EDF0AB56-7784-48C7-9D3B-818E145BDAEE@library.wisc.edu> Message-ID: <23D8BC3B-54BB-4FDC-9F3B-DD86A2D471C9@library.wisc.edu> YIKES! Please disregard the previous message. - - - - I managed to download blacklight without getting the importer external?... that might come in handy! I'll let everyone know if I run into any problems with the _full_ version of trunk. Uffdah, - Eric On Mar 9, 2009, at 2:35 PM, Eric Larson wrote: > Hi all, > > We have begun playing with Blacklight at UW-Madison. While > installation and importing data via the ruby rake task works fine, > we'd like to try pushing 10% of our collection into our test install > (1M+ records). > > To handle this larger import, I've been trying (unsuccessfully) to > get the solrmarc-2.0 branch to work. The documentation in the > GettingStarted.txt is not up to date. > > Below is a list of the steps I've been taking. If anyone can offer > advice, or help me understand how solrmarc-2.0 wants to be setup, > I'd greatly appreciate it. > > Right now, I'm planning to dive farther into vufind's import script > to see how they have solrmarc working (I've successfully imported > this marc set into a test vufind install). > > As referenced here, a SM guide for Blacklight would be *very > appreciated* :) > http://rubyforge.org/pipermail/blacklight-development/2009-January/000246.html > > Thanks! > - Eric > > = = = = > > Test install (OSX) > - - - - > 1) Download blacklight-trunk /Users/elarson/Sites/blacklight > * No problems here. > * Test Solr import via rake works fine. > > 2) Download solrmarc-2.0 > * In blacklight root dir > - svn co svn checkout http://solrmarc.googlecode.com/svn/branches/solrmarc-2.0 > solrmarc > > * Open and attempt to use /solrmarc/docs/GettingStarted.txt > - ant -version => 1.7.0 > - java -version => 1.5.0 > - sudo ant > : Init site => "uwmadison" > : Jar file name => "uwmadison" > : Prefix => "uwmadison" > : Final jar file => "uwmadison_SolrMarc" > : Heap => -Xmx265m (default) > : Solr version => 1.3 (default) > - BUILD SUCCESSFUL > : Note: uwmadison_config_properties and uwmadison_index.properties > are not created? > > - Edit importSamples.properties > : This file doesn't exist... > : Realize I'm on my own now... > > - Assuming I want to edit the VanillaBlacklightDemo properites files > : demo_config.properties > : demo_index.properties > : Blacklight solr schema.xml should sync with demo_index.properties? > : Copy these files to /solrmarc/uwmadison dir created during build > : Rename to uwmadison_config.properties and uwmadison_index.properties > > - Edit: uwmadison_config.properties > : solr.path => /Users/elarson/Sites/blacklight/jetty/solr > > - Leaving: uwmadison_index.properties > : This *should* work with Vanilla blacklight install? > > - Assuming all should be ready to run the example import_file.sh > script... > : cd /solrmarc/dist > : Using importfile script > : # ./indexfile ../uwmadison/uwmadison_config.properties ../../../ > vufind/import/catalog.mrc (yeah, we're testing vufind too) > > # Problem 1: Unable to find uwmadison_index.properties > - Let's try the full path > : /Users/elarson/Sites/blacklight/solrmarc/uwmadison/ > uwmadison_index.properties > > # Problem 2: "Error: Problem instantiating SolrCore" > - Let's start Solr :) > - cd /blacklight/jetty; java -jar start.jar > - Solr Starts > - Retrying index script > > # Problem 3: "Error: Problem instantiating SolrCore" > - Trace below > ERROR [main] (SolrCoreLoader.java:149) - Error: Problem > instantiating SolrCore > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun > .reflect > .NativeConstructorAccessorImpl > .newInstance(NativeConstructorAccessorImpl.java:39) > at > sun > .reflect > .DelegatingConstructorAccessorImpl > .newInstance(DelegatingConstructorAccessorImpl.java:27) > at java.lang.reflect.Constructor.newInstance(Constructor.java:494) > at org.solrmarc.solr.SolrCoreLoader.loadCore(SolrCoreLoader.java:121) > at > org.solrmarc.marc.MarcImporter.getSolrCoreProxy(MarcImporter.java:446) > at org.solrmarc.marc.MarcImporter.(MarcImporter.java:73) > at org.solrmarc.marc.MarcImporter.main(MarcImporter.java:462) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun > .reflect > .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun > .reflect > .DelegatingMethodAccessorImpl > .invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at com.simontuffs.onejar.Boot.run(Boot.java:306) > at com.simontuffs.onejar.Boot.main(Boot.java:159) > Caused by: org.apache.solr.common.SolrException: Error loading class > 'solr.FastLRUCache' > at > org > .apache > .solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:273) > at org.apache.solr.search.CacheConfig.getConfig(CacheConfig.java:90) > at org.apache.solr.search.CacheConfig.getConfig(CacheConfig.java:73) > at org.apache.solr.core.SolrConfig.(SolrConfig.java:128) > at org.apache.solr.core.SolrConfig.(SolrConfig.java:101) > ... 14 more > Caused by: java.lang.ClassNotFoundException: solr.FastLRUCache > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > at java.lang.ClassLoader.loadClass(ClassLoader.java:316) > at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:579) > at java.lang.ClassLoader.loadClass(ClassLoader.java:251) > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:374) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:242) > at > org > .apache > .solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:257) > ... 18 more > > - I'm not a Java person, so debugging the rest is _painful_ > - A step by step Blacklight / SolrMarc guide would be fantastic!- > I'm willing to help write this documentation, if someone can help we > get this working locally. > > - - - - > > Again, thanks for the help. > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From mpc3c at virginia.edu Tue Mar 10 09:45:11 2009 From: mpc3c at virginia.edu (Molly Pickral Cadieux) Date: Tue, 10 Mar 2009 09:45:11 -0400 Subject: [Blacklight-development] routing problem Message-ID: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> Hi everyone, I recently committed some changes to the repository that offer the following features: 1. A drop-down list that allows users to specify how may results per page to display (the previous default was 10 per page) 2. "Previous" and "Next" links that allow a user to page through a search result, as opposed to going back to the previous page to select the previous or next item. 3. Several rspec tests for the catalog controller and various models. This all works perfectly on my setup, but when I went to deploy the changes on demo.blacklightopac.org, I ran into a routing issue. Vernon also reported this error yesterday. The error is as follows: ------------ catalog_url failed to generate from {:commit=>nil, :action=>"show", :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, diff: {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} Extracted source (around line #21): 18: <% # main title container for doc partial view -%> 19: 20:
21:
<%= counter + 1 + @response.start.to_i %>. <%= link_to_document document, :label=>:title, :extra_params=>{:counter => counter + 1 + @response.start.to_i } %>
22:
23: 24: ----------- Bess had reported a similar problem back in February (I've included the message below). Unfortunately, I have been unable to reproduce the reported problem on my Mac running the same versions of Ruby, Rails, and Passenger. I would like to try Bess's recommended solution if I can get a setup where I can reproduce the problem. In the mean time, I'd love it if Vernon or others who may have seen the problem could try modifying this file: rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb and change this line: link_to label, catalog_path(doc[:id], params.merge(opts[:extra_params]).merge(:action=>nil, :commit=>nil)) to be: link_to label, catalog_path(doc[:id], params.merge(opts[:extra_params]).merge(:action=>"show", :commit=>nil)) and see if that fixes it. A restart of passenger will be necessary. If I can't get this worked out soon, I will revert the changes in the repository and work through this again here. In any case, I wanted to let you all know that what is currently in the repository may cause routing issues under certain environments. I'll keep you all posted on progress. Molly ---------- >From eos8d at virginia.edu Sun Feb 1 17:25:35 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Sun, 1 Feb 2009 17:25:35 -0500 Subject: [Blacklight-development] demo app bug fix Message-ID: Hey, folks. I've spent the last day or so playing pretty intensively with the demo app. I fixed a couple of minor bugs in the installation instructions (here: http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) and I've deployed the demo app several times, on several different systems. Strangely, although I could get it running right away with mongrel or webrick, I found that I could not get it running with mod_passenger, which is our preferred method of running rails apps in production. I think I tracked down the bug today, and you can see the demo app running with mod_passenger as a virtual host at http://demo.blacklightopac.org I was getting two errors. One, encountered whenever the index screen was loaded, said: ActionView::TemplateError (catalog_url failed to generate from {:action=>"index", :controller=>"catalog", :id=>"u1"}, expected: {:action=>"show", :controller=>"catalog"}, diff: {:action=>"show", :id=>"u1"}) on line #11 of vendor/plugins/blacklight/ app/views/catalog/_index_partials/_default.html.erb: 8: 9:
10:
11: <%= link_to document.get(:id), catalog_path(document[:id], params) %> 12:
13:
14: The other was generated whenever the show screen was loaded, and it said: ActionView::TemplateError (catalog_index_url failed to generate from {:action=>"show", :controller=>"catalog", :id=>nil}, expected: {:action=>"index", :controller=>"catalog"}, diff: {:action=>"index", :id=>nil}) on line #8 of vendor/plugins/blacklight/ app/views/catalog/show.html.erb: 5: 6: 7: <% @sidebar = capture do %> 8:

<%= link_to '<< SEARCH', catalog_index_path(params.merge(:id=>nil)) %>

9: <% end %> Turns out these were basically the same error. In both cases, we're using the link_to function to create a link to another part of the application, in this case from index to show, and from show to index. Let's talk about the link from index to show as an example. <%= link_to document.get(:id), catalog_path(document[:id], params) %> That says create a link, and the text of the link is the id number of the current document (say, u999), and the target of the link is catalog_path(u999) which will evaluate to /catalog/u999. Incidentally, the rails console was really valuable in helping me understand how these worked, and you can play with it like this: paz:rails eos8d$ ./script/console Loading development environment (Rails 2.2.2) >> app.catalog_path("u999") => "/catalog/u999" Anyway, I found that if I just changed the link_to statement to look like this instead, removing the params object: <%= link_to document.get(:id), catalog_path(document[:id]) %> the error went away. However, this wasn't an acceptable solution, because whenever a user performs a search or selects a facet, that value goes in the params hash, and you want to keep it around when you go from screen to screen, so that the user isn't always having to re- enter their current search. Let's say you're on the index screen, and you've entered a search and selected a facet. At that point your params hash looks like this: {"commit"=>"search", "action"=>"index", "q"=>"poetry", "f"=>{"language_facet"=>["English"]}, "controller"=>"catalog"} When you want to look at a detailed view for a specific item, you're linking to the show action of your same catalog controller, and passing it a specific document id. But you also want to hang onto the fact that you've searched for poetry and limited to language_facet:English, so you also pass the params object in. At which point your link_to really looks like this: <%= link_to document.get(:id), catalog_path (:id =>document[:id], :commit=>"search", :action=>"index", :q=>"poetry" ... etc ... ) %> Do you see the problem? It took me awhile to notice it. I've just called the show action, and then I also pass in a parameter of action=index. It's not obvious, because catalog_path sort of obscures the fact that you're calling the show action. Interestingly, webrick and mongrel don't seem to care. I guess they just assume that even though you've passed two values for action, you must really mean action=show because that's what's defined for catalog_path. The fix is easy, once you know what the problem is: <%= link_to document.get(:id), catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> Now you're passing in a params object that looks like this: {"commit"=>"search", "action"=>"show", "id"=>"2008349843", "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", "controller"=>"catalog"} So, all your user defined parameters, plus the id number of the specific item you want to show, plus action=show, which is what catalog_path expects. Problem solved. Sorry this is so long, but I learned a lot in fixing this bug and I thought it might be helpful for someone else too. But if it just looks like gibberish, ignore it. Cheers, Bess >From goodieboy at gmail.com Sun Feb 1 17:59:36 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Sun, 1 Feb 2009 17:59:36 -0500 Subject: [Blacklight-development] demo app bug fix In-Reply-To: References: Message-ID: Seriously, that's a kickin' write up Bess. Great bug catch. And the docs look so nice too! For the link_to fix here: <%= link_to document.get(:id), catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> You should be able to do this too: <%= link_to document.get(:id), catalog_path(document[:id], params.merge(:action=>nil)) %> I think this kind of thing will become a url-param-linking pattern in BL and might do well in a view helper. I think there's one in the UVa trunk, called link_to_document or something. We could then have one called link_to_documents or link_back_to_search? The upshot of it being in a helper is that if someone overrides a view partial, they don't have to re-write the param merging stuff. Food for thought... Matt From mpc3c at virginia.edu Tue Mar 10 10:01:07 2009 From: mpc3c at virginia.edu (Molly Pickral Cadieux) Date: Tue, 10 Mar 2009 10:01:07 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> Message-ID: <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> Hi again, I went ahead and tried the fix on demo.blacklightopac.org, and it works great! So, Vernon, if you could grab the new version of catalog_helper.rb from subversion, I'm almost certain that will fix the bug you reported yesterday. Molly On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux wrote: > Hi everyone, > > I recently committed some changes to the repository that offer the > following features: > > 1. ?A drop-down list that allows users to specify how may results per > page to display (the previous default was 10 per page) > 2. ?"Previous" and "Next" links that allow a user to page through a > search result, as opposed to going back to the previous page to select > the previous or next item. > 3. ?Several rspec tests for the catalog controller and various models. > > This all works perfectly on my setup, but when I went to deploy the > changes on demo.blacklightopac.org, I ran into a routing issue. > Vernon also reported this error yesterday. ?The error is as follows: > > ------------ > > catalog_url failed to generate from {:commit=>nil, :action=>"show", > :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", > :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, diff: > {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} > > Extracted source (around line #21): > > 18: ? ? ? ? <% # main title container for doc partial view -%> > 19: > 20: ? ? ? ?
> 21: ? ? ? ? ?
<%= counter + 1 + @response.start.to_i %>. <%= > link_to_document document, :label=>:title, :extra_params=>{:counter => > counter + 1 + @response.start.to_i } %>
> 22: ? ? ? ?
> 23: > 24: ? ? ? > > ----------- > > Bess had reported a similar problem back in February (I've included > the message below). ?Unfortunately, I have been unable to reproduce > the reported problem on my Mac running the same versions of Ruby, > Rails, and Passenger. ?I would like to try Bess's recommended solution > if I can get a setup where I can reproduce the problem. ?In the mean > time, I'd love it if Vernon or others who may have seen the problem > could try modifying this file: > > rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb > > and change this line: > > link_to label, catalog_path(doc[:id], > params.merge(opts[:extra_params]).merge(:action=>nil, :commit=>nil)) > > to be: > > link_to label, catalog_path(doc[:id], > params.merge(opts[:extra_params]).merge(:action=>"show", > :commit=>nil)) > > and see if that fixes it. ?A restart of passenger will be necessary. > > If I can't get this worked out soon, I will revert the changes in the > repository and work through this again here. > > In any case, I wanted to let you all know that what is currently in > the repository may cause routing issues under certain environments. > I'll keep you all posted on progress. > > Molly > > > ---------- > > From eos8d at virginia.edu ?Sun Feb ?1 17:25:35 2009 > From: eos8d at virginia.edu (Bess Sadler) > Date: Sun, 1 Feb 2009 17:25:35 -0500 > Subject: [Blacklight-development] demo app bug fix > Message-ID: > > Hey, folks. > > I've spent the last day or so playing pretty intensively with the demo > app. I fixed a couple of minor bugs in the installation instructions > (here: http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) > ?and I've deployed the demo app several times, on several different > systems. Strangely, although I could get it running right away with > mongrel or webrick, I found that I could not get it running with > mod_passenger, which is our preferred method of running rails apps in > production. I think I tracked down the bug today, and you can see the > demo app running with mod_passenger as a virtual host at > http://demo.blacklightopac.org > > I was getting two errors. One, encountered whenever the index screen > was loaded, said: > > ActionView::TemplateError (catalog_url failed to generate from > {:action=>"index", :controller=>"catalog", :id=>"u1"}, expected: > {:action=>"show", :controller=>"catalog"}, diff: > {:action=>"show", :id=>"u1"}) on line #11 of vendor/plugins/blacklight/ > app/views/catalog/_index_partials/_default.html.erb: > 8: > 9: ? ?
> 10: ? ? ?
> 11: ? ? ? ? <%= link_to document.get(:id), catalog_path(document[:id], > params) %> > 12: ? ? ?
> 13: ? ?
> 14: ? > > > The other was generated whenever the show screen was loaded, and it > said: > > ActionView::TemplateError (catalog_index_url failed to generate from > {:action=>"show", :controller=>"catalog", :id=>nil}, expected: > {:action=>"index", :controller=>"catalog"}, diff: > {:action=>"index", :id=>nil}) on line #8 of vendor/plugins/blacklight/ > app/views/catalog/show.html.erb: > 5: > 6: > 7: <% @sidebar = capture do %> > 8: ?

<%= link_to '<< SEARCH', > catalog_index_path(params.merge(:id=>nil)) %>

> 9: <% end %> > > Turns out these were basically the same error. In both cases, we're > using the link_to function to create a link to another part of the > application, in this case from index to show, and from show to index. > Let's talk about the link from index to show as an example. > > <%= link_to document.get(:id), catalog_path(document[:id], params) %> > > That says create a link, and the text of the link is the id number of > the current document (say, u999), and the target of the link is > catalog_path(u999) which will evaluate to /catalog/u999. Incidentally, > the rails console was really valuable in helping me understand how > these worked, and you can play with it like this: > > paz:rails eos8d$ ./script/console > Loading development environment (Rails 2.2.2) > ?>> app.catalog_path("u999") > => "/catalog/u999" > > Anyway, I found that if I just changed the link_to statement to look > like this instead, removing the params object: > > ? ? ? ?<%= link_to document.get(:id), catalog_path(document[:id]) %> > > the error went away. However, this wasn't an acceptable solution, > because whenever a user performs a search or selects a facet, that > value goes in the params hash, and you want to keep it around when you > go from screen to screen, so that the user isn't always having to re- > enter their current search. Let's say you're on the index screen, and > you've entered a search and selected a facet. At that point your > params hash looks like this: > > ? ? ? ?{"commit"=>"search", "action"=>"index", "q"=>"poetry", > "f"=>{"language_facet"=>["English"]}, "controller"=>"catalog"} > > When you want to look at a detailed view for a specific item, you're > linking to the show action of your same catalog controller, and > passing it a specific document id. But you also want to hang onto the > fact that you've searched for poetry and limited to > language_facet:English, so you also pass the params object in. At > which point your link_to really looks like this: > > ? ? ? ?<%= link_to document.get(:id), > catalog_path > (:id > =>document[:id], :commit=>"search", :action=>"index", :q=>"poetry" ... > etc ... ) %> > > Do you see the problem? It took me awhile to notice it. I've just > called the show action, and then I also pass in a parameter of > action=index. It's not obvious, because catalog_path sort of obscures > the fact that you're calling the show action. Interestingly, webrick > and mongrel don't seem to care. I guess they just assume that even > though you've passed two values for action, you must really mean > action=show because that's what's defined for catalog_path. The fix is > easy, once you know what the problem is: > > ? ? ? ? <%= link_to document.get(:id), > catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> > > Now you're passing in a params object that looks like this: > > ? ? ? ?{"commit"=>"search", "action"=>"show", "id"=>"2008349843", > "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", > "controller"=>"catalog"} > > So, all your user defined parameters, plus the id number of the > specific item you want to show, plus action=show, which is what > catalog_path expects. Problem solved. > > Sorry this is so long, but I learned a lot in fixing this bug and I > thought it might be helpful for someone else too. But if it just looks > like gibberish, ignore it. > > Cheers, > Bess > > > > > > > > > > > From goodieboy at gmail.com ?Sun Feb ?1 17:59:36 2009 > From: goodieboy at gmail.com (Matt Mitchell) > Date: Sun, 1 Feb 2009 17:59:36 -0500 > Subject: [Blacklight-development] demo app bug fix > In-Reply-To: > References: > Message-ID: > > Seriously, that's a kickin' write up Bess. Great bug catch. And the docs > look so nice too! > > For the link_to fix here: > > <%= link_to document.get(:id), > catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> > > You should be able to do this too: > > <%= link_to document.get(:id), catalog_path(document[:id], > params.merge(:action=>nil)) %> > > I think this kind of thing will become a url-param-linking pattern in BL and > might do well in a view helper. I think there's one in the UVa trunk, called > link_to_document or something. We could then have one called > link_to_documents or link_back_to_search? The upshot of it being in a helper > is that if someone overrides a view partial, they don't have to re-write the > param merging stuff. Food for thought... > > Matt > From rh9ec at virginia.edu Tue Mar 10 12:35:59 2009 From: rh9ec at virginia.edu (Robert Haschart) Date: Tue, 10 Mar 2009 12:35:59 -0400 Subject: [Blacklight-development] SolrMarc 2.0 - Getting Started? In-Reply-To: References: Message-ID: <49B696EF.1090101@virginia.edu> Eric, I am in the process of updating the Getting Started document, as well as polishing up the rough corners of the installation process. You were on the right track below, and actually got very close, until you ran into something I had set up incorrectly. More comments are interspersed below. -Bob Haschart Eric Larson wrote: > Hi all, > > We have begun playing with Blacklight at UW-Madison. While > installation and importing data via the ruby rake task works fine, > we'd like to try pushing 10% of our collection into our test install > (1M+ records). > > To handle this larger import, I've been trying (unsuccessfully) to > get the solrmarc-2.0 branch to work. The documentation in the > GettingStarted.txt is not up to date. > > Below is a list of the steps I've been taking. If anyone can offer > advice, or help me understand how solrmarc-2.0 wants to be setup, I'd > greatly appreciate it. > > Right now, I'm planning to dive farther into vufind's import script > to see how they have solrmarc working (I've successfully imported > this marc set into a test vufind install). > > As referenced here, a SM guide for Blacklight would be *very > appreciated* :) > http://rubyforge.org/pipermail/blacklight-development/2009-January/000246.html > > > Thanks! > - Eric > > = = = = > > Test install (OSX) > - - - - > 1) Download blacklight-trunk /Users/elarson/Sites/blacklight > * No problems here. > * Test Solr import via rake works fine. > > 2) Download solrmarc-2.0 > * In blacklight root dir > - svn co svn checkout > http://solrmarc.googlecode.com/svn/branches/solrmarc-2.0 solrmarc > > * Open and attempt to use /solrmarc/docs/GettingStarted.txt > - ant -version => 1.7.0 > - java -version => 1.5.0 > - sudo ant > : Init site => "uwmadison" > : Jar file name => "uwmadison" > : Prefix => "uwmadison" > : Final jar file => "uwmadison_SolrMarc" > : Heap => -Xmx265m (default) > : Solr version => 1.3 (default) This might be the first step where you went astray. I chose to make the default solr version 1.3 since that is the latest released version. However the version being used in the blacklight demo is a prerelease nightly-build version of solr 1.4. One of the configuration options for solrmarc's solr version is "war", which if selected requires you to enter the full path of the solr.war file that you will be using to deploy solr in jetty or tomcat or tanother webapp container of your choice. The build process will then extract the jar files it needs directly from that war file. Thus guaranteeing that the version of solr used by solrmarc and the version of solr used in the webapp container for searching the solr index, will be in sync. It is becoming increasingly clear to me that this should be the default option, and possibly the only option. > - BUILD SUCCESSFUL > : Note: uwmadison_config_properties and uwmadison_index.properties > are not created? > It is also becoming clear to me that the build process should walk you through the creation of the xxx_config.properties file and at least create a placeholder xxx_index.properties file for you. > - Edit importSamples.properties > : This file doesn't exist... > : Realize I'm on my own now... > > - Assuming I want to edit the VanillaBlacklightDemo properites files > : demo_config.properties > : demo_index.properties > : Blacklight solr schema.xml should sync with demo_index.properties? > : Copy these files to /solrmarc/uwmadison dir created during build > : Rename to uwmadison_config.properties and uwmadison_index.properties > > - Edit: uwmadison_config.properties > : solr.path => /Users/elarson/Sites/blacklight/jetty/solr > > - Leaving: uwmadison_index.properties > : This *should* work with Vanilla blacklight install? > Correct the vanilla blacklight demo example directory was designed to produce an indexer that will create exactly the type of index that the blacklight demo expects. > - Assuming all should be ready to run the example import_file.sh > script... > : cd /solrmarc/dist > : Using importfile script > : # ./indexfile ../uwmadison/uwmadison_config.properties ../../../ > vufind/import/catalog.mrc (yeah, we're testing vufind too) You should be able to run it using the following command: ./indexfile ../../../ vufind/import/catalog.mrc it should store the uwmadison_config.properties file in the jar file it produces, and it should find and use that file when you run the script indexfile. > > # Problem 1: Unable to find uwmadison_index.properties > - Let's try the full path > : /Users/elarson/Sites/blacklight/solrmarc/uwmadison/ > uwmadison_index.properties > You shouldn't need a full path here. As with the uwmadison_config.properties file the uwmadison_index.properties shold be stored in the jar file produced by the build process, and it should be able to find that index.properties file based solely on its name. Here's a perfect example of why having the build process walk you though the creation of the xxx_config.properties file would be very helpful. > # Problem 2: "Error: Problem instantiating SolrCore" > - Let's start Solr :) > - cd /blacklight/jetty; java -jar start.jar > - Solr Starts > - Retrying index script You shouldn't need to have a solr server running, however if you are able to get the solr server running, running solr marc should be very easy. > # Problem 3: "Error: Problem instantiating SolrCore" > - Trace below > ERROR [main] (SolrCoreLoader.java:149) - Error: Problem instantiating > SolrCore > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at sun .reflect .NativeConstructorAccessorImpl > .newInstance(NativeConstructorAccessorImpl.java:39) > at sun .reflect .DelegatingConstructorAccessorImpl > .newInstance(DelegatingConstructorAccessorImpl.java:27) > at java.lang.reflect.Constructor.newInstance(Constructor.java:494) > at org.solrmarc.solr.SolrCoreLoader.loadCore(SolrCoreLoader.java:121) > at > org.solrmarc.marc.MarcImporter.getSolrCoreProxy(MarcImporter.java: 446) > at org.solrmarc.marc.MarcImporter.(MarcImporter.java:73) > at org.solrmarc.marc.MarcImporter.main(MarcImporter.java:462) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun > .reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: > 39) > at sun .reflect > .DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: > 25) > at java.lang.reflect.Method.invoke(Method.java:585) > at com.simontuffs.onejar.Boot.run(Boot.java:306) > at com.simontuffs.onejar.Boot.main(Boot.java:159) > Caused by: org.apache.solr.common.SolrException: Error loading class > 'solr.FastLRUCache' > at org > .apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java: > 273) > at org.apache.solr.search.CacheConfig.getConfig(CacheConfig.java:90) > at org.apache.solr.search.CacheConfig.getConfig(CacheConfig.java:73) > at org.apache.solr.core.SolrConfig.(SolrConfig.java:128) > at org.apache.solr.core.SolrConfig.(SolrConfig.java:101) > ... 14 more > Caused by: java.lang.ClassNotFoundException: solr.FastLRUCache > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > at java.lang.ClassLoader.loadClass(ClassLoader.java:316) > at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:579) > at java.lang.ClassLoader.loadClass(ClassLoader.java:251) > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:374) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:242) > at org > .apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java: > 257) > ... 18 more I'm pretty sure that solr.FastLRUCache class that it is trying to load, was a new class as of solr 1.4, so the program is looking in the solr 1.3 libraries it has, (as specified above for the sol version default) The shortest path to working should be: 1) edit uwmadison_config.properties to remove the full path for solr.indexer.properties solr.indexer.properties = uwmadison_index.properties 2) find where the property for solr.version is stored and delete it. (It should be in uwmadison/build.properties or uwmadison/build_override.properties) 3) re-run ant and when asked for the solr.version enter war and then enter the full path to the solr.war file in answer to the next prompt. (It should be /blacklight/jetty/webapps/solr.war in your setup) 4) start indexing dist/indexfile ../../ vufind/import/catalog.mrc Let me know if this helps you -Bob Haschart > - I'm not a Java person, so debugging the rest is _painful_ > - A step by step Blacklight / SolrMarc guide would be fantastic!- I'm > willing to help write this documentation, if someone can help we get > this working locally. > > - - - - > > Again, thanks for the help. > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From ndushay at stanford.edu Tue Mar 10 13:12:06 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Tue, 10 Mar 2009 10:12:06 -0700 Subject: [Blacklight-development] solr query for single document: standard or dismax? Message-ID: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> In refactoring the solr request code for blacklight, I had an insight: Would it be better to use the standard request handler for requesting a single document (the "details" handler)? I thought of three possible ways to request a single document 1. fq=id:(666) in a dismax handler. This is how it is done now. Because dismax doesn't take fielded searches, a filter query must be used. Downside: do we want single document filter queries in the filter cache? 2. dismax handler with boosting set up something like this: qf=id^2 all_search^000.1 The question is: can we set it up so the query strings will only match on the id field? Also, why do we want dismax for this? See option 3. 3. standard handler: This may be the simplest, as merely passing in a query such as: q=id:666 will only match the exact document. Easy-peasy. I'm leaning towards the standard request handler approach. What do folks think? Also, what do you think of renaming the handler to "getDocument" or "document" rather than "details"? In solr parlance, it is set up to retrieve a single document ... - Naomi From jkeck at stanford.edu Tue Mar 10 13:32:36 2009 From: jkeck at stanford.edu (Jessie Keck) Date: Tue, 10 Mar 2009 10:32:36 -0700 Subject: [Blacklight-development] current_user problem Message-ID: <49B6A434.80008@stanford.edu> Hi all, I just tried a fresh install of blacklight (twice) and keep getting an Undefined Local Variable current_user error. I think what may have been the problem is the way I installed all of the require plugins, or possible a migration issue. I pretty much copied and pasted the commands right out of the readme to ensure that I didn't have any typos. When I removed the current_user stuff from the bookmark_controller partial, I started to get the same error but for the application_name variable, which I know works correctly. What could cause these problems? (Maybe this should be documented as a way to tell how you screwed up the install?) Any help would be greatly appreciated. I'm trying to merge my splash page code in with the latest and greatest from SVN to make sure everything works smoothly before I commit it. Thanks, -Jessie Keck From erikhatcher at mac.com Tue Mar 10 14:32:40 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 10 Mar 2009 14:32:40 -0400 Subject: [Blacklight-development] solr query for single document: standard or dismax? In-Reply-To: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> References: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> Message-ID: <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> On Mar 10, 2009, at 1:12 PM, Naomi Dushay wrote: > In refactoring the solr request code for blacklight, I had an insight: > > Would it be better to use the standard request handler for > requesting a single document (the "details" handler)? Again, let's be sure not to confuse "query parser" with "request handler" (but in some cases it'll matter if you've locked in a defType query parser to a particular request handler). > I thought of three possible ways to request a single document > > 1. fq=id:(666) in a dismax handler. > > This is how it is done now. Because dismax doesn't take fielded > searches, a filter query must be used. Downside: do we want single > document filter queries in the filter cache? No! Besides, you'd also have to ensure q/q.alt for dismax matched the document desired (such as q=&q.alt=*:*, of course). > 2. dismax handler with boosting set up something like this: > qf=id^2 all_search^000.1 > > The question is: can we set it up so the query strings will only > match on the id field? Also, why do we want dismax for this? See > option 3. qf=id&q= should do the trick, but you don't want dismax for this either. > 3. standard handler: > > This may be the simplest, as merely passing in a query such as: > > q=id:666 > > will only match the exact document. Easy-peasy. Indeed. But also keep in mind that q is parsed using Solr's standard query parser in this example. And that requires escaping of any special characters that might be in the id. > I'm leaning towards the standard request handler approach. What do > folks think? A better way is to use: /solr/select?q={!raw f=id}666 The magic {!...} syntax is an underexploited feature of Solr (and of course under documented too). This translates to a Lucene TermQuery. The benefit is no parsing is done, no escaping is necessary (other than URL encoding for an HTTP request). This is also a technique that should be used for fq's that come from facet clicks. You probably right now are using field:"facet value", which works most of the time, but is subject to query parsing and escaping rules when it's an unnecessary process that can sometimes be problematic. > Also, what do you think of renaming the handler to "getDocument" or > "document" rather than "details"? In solr parlance, it is set up > to retrieve a single document ... /document (not camel-casing) is what I'd recommend. And you can do clever stuff like this (pasted from LucidFind's solrconfig.xml): json all {!field f=id v=$id} 1 * We should have used {!raw} instead of {!field} though. The {!field} qparser (what Solr calls a query parser in API-speak) will do the same thing as raw for non-"text" field types, create a TermQuery for the value as-is. But for text field types it analyzes the value and creates either a PhraseQuery or TermQuery. In all of our uses {! field} is used with non-text fields The clever thing here is that the request is very clean to Solr: /solr/document?id=666 Note the indirection from id into a q parameter, making the request cleaner (but the config a bit more obtuse). Erik From chapman.lists at gmail.com Tue Mar 10 14:51:39 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Tue, 10 Mar 2009 14:51:39 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> Message-ID: <49B6B6BB.5020309@gmail.com> Molly, Thanks for your work on this, but unfortunately that didn't fix the issue. I did an svn update on my checkout of blacklight and restarted apache and I still get the same error. Vern Molly Pickral Cadieux wrote: > Hi again, > > I went ahead and tried the fix on demo.blacklightopac.org, and it > works great! So, Vernon, if you could grab the new version of > catalog_helper.rb from subversion, I'm almost certain that will fix > the bug you reported yesterday. > > Molly > > On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux > wrote: > >> Hi everyone, >> >> I recently committed some changes to the repository that offer the >> following features: >> >> 1. A drop-down list that allows users to specify how may results per >> page to display (the previous default was 10 per page) >> 2. "Previous" and "Next" links that allow a user to page through a >> search result, as opposed to going back to the previous page to select >> the previous or next item. >> 3. Several rspec tests for the catalog controller and various models. >> >> This all works perfectly on my setup, but when I went to deploy the >> changes on demo.blacklightopac.org, I ran into a routing issue. >> Vernon also reported this error yesterday. The error is as follows: >> >> ------------ >> >> catalog_url failed to generate from {:commit=>nil, :action=>"show", >> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", >> :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, diff: >> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} >> >> Extracted source (around line #21): >> >> 18: <% # main title container for doc partial view -%> >> 19: >> 20:
>> 21:
<%= counter + 1 + @response.start.to_i %>. <%= >> link_to_document document, :label=>:title, :extra_params=>{:counter => >> counter + 1 + @response.start.to_i } %>
>> 22:
>> 23: >> 24: >> >> ----------- >> >> Bess had reported a similar problem back in February (I've included >> the message below). Unfortunately, I have been unable to reproduce >> the reported problem on my Mac running the same versions of Ruby, >> Rails, and Passenger. I would like to try Bess's recommended solution >> if I can get a setup where I can reproduce the problem. In the mean >> time, I'd love it if Vernon or others who may have seen the problem >> could try modifying this file: >> >> rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb >> >> and change this line: >> >> link_to label, catalog_path(doc[:id], >> params.merge(opts[:extra_params]).merge(:action=>nil, :commit=>nil)) >> >> to be: >> >> link_to label, catalog_path(doc[:id], >> params.merge(opts[:extra_params]).merge(:action=>"show", >> :commit=>nil)) >> >> and see if that fixes it. A restart of passenger will be necessary. >> >> If I can't get this worked out soon, I will revert the changes in the >> repository and work through this again here. >> >> In any case, I wanted to let you all know that what is currently in >> the repository may cause routing issues under certain environments. >> I'll keep you all posted on progress. >> >> Molly >> >> >> ---------- >> >> From eos8d at virginia.edu Sun Feb 1 17:25:35 2009 >> From: eos8d at virginia.edu (Bess Sadler) >> Date: Sun, 1 Feb 2009 17:25:35 -0500 >> Subject: [Blacklight-development] demo app bug fix >> Message-ID: >> >> Hey, folks. >> >> I've spent the last day or so playing pretty intensively with the demo >> app. I fixed a couple of minor bugs in the installation instructions >> (here: http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) >> and I've deployed the demo app several times, on several different >> systems. Strangely, although I could get it running right away with >> mongrel or webrick, I found that I could not get it running with >> mod_passenger, which is our preferred method of running rails apps in >> production. I think I tracked down the bug today, and you can see the >> demo app running with mod_passenger as a virtual host at >> http://demo.blacklightopac.org >> >> I was getting two errors. One, encountered whenever the index screen >> was loaded, said: >> >> ActionView::TemplateError (catalog_url failed to generate from >> {:action=>"index", :controller=>"catalog", :id=>"u1"}, expected: >> {:action=>"show", :controller=>"catalog"}, diff: >> {:action=>"show", :id=>"u1"}) on line #11 of vendor/plugins/blacklight/ >> app/views/catalog/_index_partials/_default.html.erb: >> 8: >> 9:
>> 10:
>> 11: <%= link_to document.get(:id), catalog_path(document[:id], >> params) %> >> 12:
>> 13:
>> 14: >> >> >> The other was generated whenever the show screen was loaded, and it >> said: >> >> ActionView::TemplateError (catalog_index_url failed to generate from >> {:action=>"show", :controller=>"catalog", :id=>nil}, expected: >> {:action=>"index", :controller=>"catalog"}, diff: >> {:action=>"index", :id=>nil}) on line #8 of vendor/plugins/blacklight/ >> app/views/catalog/show.html.erb: >> 5: >> 6: >> 7: <% @sidebar = capture do %> >> 8:

<%= link_to '<< SEARCH', >> catalog_index_path(params.merge(:id=>nil)) %>

>> 9: <% end %> >> >> Turns out these were basically the same error. In both cases, we're >> using the link_to function to create a link to another part of the >> application, in this case from index to show, and from show to index. >> Let's talk about the link from index to show as an example. >> >> <%= link_to document.get(:id), catalog_path(document[:id], params) %> >> >> That says create a link, and the text of the link is the id number of >> the current document (say, u999), and the target of the link is >> catalog_path(u999) which will evaluate to /catalog/u999. Incidentally, >> the rails console was really valuable in helping me understand how >> these worked, and you can play with it like this: >> >> paz:rails eos8d$ ./script/console >> Loading development environment (Rails 2.2.2) >> >> app.catalog_path("u999") >> => "/catalog/u999" >> >> Anyway, I found that if I just changed the link_to statement to look >> like this instead, removing the params object: >> >> <%= link_to document.get(:id), catalog_path(document[:id]) %> >> >> the error went away. However, this wasn't an acceptable solution, >> because whenever a user performs a search or selects a facet, that >> value goes in the params hash, and you want to keep it around when you >> go from screen to screen, so that the user isn't always having to re- >> enter their current search. Let's say you're on the index screen, and >> you've entered a search and selected a facet. At that point your >> params hash looks like this: >> >> {"commit"=>"search", "action"=>"index", "q"=>"poetry", >> "f"=>{"language_facet"=>["English"]}, "controller"=>"catalog"} >> >> When you want to look at a detailed view for a specific item, you're >> linking to the show action of your same catalog controller, and >> passing it a specific document id. But you also want to hang onto the >> fact that you've searched for poetry and limited to >> language_facet:English, so you also pass the params object in. At >> which point your link_to really looks like this: >> >> <%= link_to document.get(:id), >> catalog_path >> (:id >> =>document[:id], :commit=>"search", :action=>"index", :q=>"poetry" ... >> etc ... ) %> >> >> Do you see the problem? It took me awhile to notice it. I've just >> called the show action, and then I also pass in a parameter of >> action=index. It's not obvious, because catalog_path sort of obscures >> the fact that you're calling the show action. Interestingly, webrick >> and mongrel don't seem to care. I guess they just assume that even >> though you've passed two values for action, you must really mean >> action=show because that's what's defined for catalog_path. The fix is >> easy, once you know what the problem is: >> >> <%= link_to document.get(:id), >> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >> >> Now you're passing in a params object that looks like this: >> >> {"commit"=>"search", "action"=>"show", "id"=>"2008349843", >> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", >> "controller"=>"catalog"} >> >> So, all your user defined parameters, plus the id number of the >> specific item you want to show, plus action=show, which is what >> catalog_path expects. Problem solved. >> >> Sorry this is so long, but I learned a lot in fixing this bug and I >> thought it might be helpful for someone else too. But if it just looks >> like gibberish, ignore it. >> >> Cheers, >> Bess >> >> >> >> >> >> >> >> >> >> >> From goodieboy at gmail.com Sun Feb 1 17:59:36 2009 >> From: goodieboy at gmail.com (Matt Mitchell) >> Date: Sun, 1 Feb 2009 17:59:36 -0500 >> Subject: [Blacklight-development] demo app bug fix >> In-Reply-To: >> References: >> Message-ID: >> >> Seriously, that's a kickin' write up Bess. Great bug catch. And the docs >> look so nice too! >> >> For the link_to fix here: >> >> <%= link_to document.get(:id), >> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >> >> You should be able to do this too: >> >> <%= link_to document.get(:id), catalog_path(document[:id], >> params.merge(:action=>nil)) %> >> >> I think this kind of thing will become a url-param-linking pattern in BL and >> might do well in a view helper. I think there's one in the UVa trunk, called >> link_to_document or something. We could then have one called >> link_to_documents or link_back_to_search? The upshot of it being in a helper >> is that if someone overrides a view partial, they don't have to re-write the >> param merging stuff. Food for thought... >> >> Matt >> >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > From jkeck at stanford.edu Tue Mar 10 15:18:54 2009 From: jkeck at stanford.edu (Jessie Keck) Date: Tue, 10 Mar 2009 12:18:54 -0700 Subject: [Blacklight-development] current_user problem In-Reply-To: <49B6A434.80008@stanford.edu> References: <49B6A434.80008@stanford.edu> Message-ID: <49B6BD1E.6050903@stanford.edu> Hi all, Just figured it out. This is something that should definitely add to the documentation. The issue was that I was installing the plugin to a rails generated application (not installing the demo application) When using the rails my_rails_app command there are some files that are generated for you. 2 important ones here being the ApplicationController and the ApplicationHelper. Since these 2 empty files are generated, they're overriding the blacklight plugins ApplicationController and ApplicationHelper. Deleting the generated apps files controllers/application.rb and helpers/application_helper.rb fix this problem. Getting this into the documentation will be very helpful to users. Let me know if you have any questions. Thanks, -Jessie Jessie Keck wrote: > Hi all, > I just tried a fresh install of blacklight (twice) and keep getting an > Undefined Local Variable current_user error. > > I think what may have been the problem is the way I installed all of > the require plugins, or possible a migration issue. I pretty much > copied and pasted the commands right out of the readme to ensure that > I didn't have any typos. > > When I removed the current_user stuff from the bookmark_controller > partial, I started to get the same error but for the application_name > variable, which I know works correctly. > > What could cause these problems? (Maybe this should be documented as > a way to tell how you screwed up the install?) > > Any help would be greatly appreciated. I'm trying to merge my splash > page code in with the latest and greatest from SVN to make sure > everything works smoothly before I commit it. > > Thanks, > -Jessie Keck > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From erikhatcher at mac.com Tue Mar 10 15:46:11 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 10 Mar 2009 15:46:11 -0400 Subject: [Blacklight-development] current_user problem In-Reply-To: <49B6BD1E.6050903@stanford.edu> References: <49B6A434.80008@stanford.edu> <49B6BD1E.6050903@stanford.edu> Message-ID: Sounds like this is one example of where Blacklight is doing too much... it should be made to work as a plugin to an existing (starter) Rails application without this conflict. Erik On Mar 10, 2009, at 3:18 PM, Jessie Keck wrote: > Hi all, > Just figured it out. This is something that should definitely add > to the documentation. > > The issue was that I was installing the plugin to a rails generated > application (not installing the demo application) > > When using the rails my_rails_app command there are some files that > are generated for you. 2 important ones here being the > ApplicationController and the ApplicationHelper. Since these 2 > empty files are generated, they're overriding the blacklight plugins > ApplicationController and ApplicationHelper. > > Deleting the generated apps files controllers/application.rb and > helpers/application_helper.rb fix this problem. > > Getting this into the documentation will be very helpful to users. > > Let me know if you have any questions. > Thanks, > -Jessie > > Jessie Keck wrote: >> Hi all, >> I just tried a fresh install of blacklight (twice) and keep getting >> an Undefined Local Variable current_user error. >> >> I think what may have been the problem is the way I installed all >> of the require plugins, or possible a migration issue. I pretty >> much copied and pasted the commands right out of the readme to >> ensure that I didn't have any typos. >> >> When I removed the current_user stuff from the bookmark_controller >> partial, I started to get the same error but for the >> application_name variable, which I know works correctly. >> >> What could cause these problems? (Maybe this should be documented >> as a way to tell how you screwed up the install?) >> >> Any help would be greatly appreciated. I'm trying to merge my >> splash page code in with the latest and greatest from SVN to make >> sure everything works smoothly before I commit it. >> >> Thanks, >> -Jessie Keck >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From ndushay at stanford.edu Tue Mar 10 17:59:55 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Tue, 10 Mar 2009 14:59:55 -0700 Subject: [Blacklight-development] solr query for single document: standard or dismax? In-Reply-To: <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> References: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> Message-ID: <9BB431CA-C6C0-4587-BB1B-3A5C73629DF4@stanford.edu> Erik, I adore this idea. If I go with: > > > ruby > all > {!raw f=id v=$id} > 1 > * > > Do I need to / should I set defType to something? or will it know from syntax that it's for the RawQParserPlugin? Does the q syntax as specified above work with defType=standard? The mwmitchell-rsolr gem used by blacklight appears only to speak "dismax" or "standard". If we're using your suggestions, do we need the gem to speak other defTypes? For example, we may need to have a request handler using the LuceneQParserPlugin so it will understand Boolean syntax. How do we use the rsolr gem to do this? Or, to gain this flexibility, would it be prudent for blacklight to use the solr-ruby gem instead of the mwmitchell-solr gem? I'm already refactoring the ruby code for solr queries into a solr_helper class (abstracting to "get_single_document" and "get_search_results_and_facets" sort of concepts), so if the ruby code is a little more complex within blacklight, it's nicely isolated. (Actually, it doesn't look like solr-ruby 0.0.5 speaks more than dismax or standard either ...) - Naomi On Mar 10, 2009, at 11:32 AM, Erik Hatcher wrote: > > On Mar 10, 2009, at 1:12 PM, Naomi Dushay wrote: >> In refactoring the solr request code for blacklight, I had an >> insight: >> >> Would it be better to use the standard request handler for >> requesting a single document (the "details" handler)? > > Again, let's be sure not to confuse "query parser" with "request > handler" (but in some cases it'll matter if you've locked in a > defType query parser to a particular request handler). > >> I thought of three possible ways to request a single document >> >> 1. fq=id:(666) in a dismax handler. >> >> This is how it is done now. Because dismax doesn't take fielded >> searches, a filter query must be used. Downside: do we want >> single document filter queries in the filter cache? > > No! Besides, you'd also have to ensure q/q.alt for dismax matched > the document desired (such as q=&q.alt=*:*, of course). > >> 2. dismax handler with boosting set up something like this: >> qf=id^2 all_search^000.1 >> >> The question is: can we set it up so the query strings will only >> match on the id field? Also, why do we want dismax for this? See >> option 3. > > qf=id&q= should do the trick, but you don't want dismax for this > either. > >> 3. standard handler: >> >> This may be the simplest, as merely passing in a query such as: >> >> q=id:666 >> >> will only match the exact document. Easy-peasy. > > Indeed. But also keep in mind that q is parsed using Solr's > standard query parser in this example. And that requires escaping > of any special characters that might be in the id. > >> I'm leaning towards the standard request handler approach. What do >> folks think? > > A better way is to use: > > /solr/select?q={!raw f=id}666 > > The magic {!...} syntax is an underexploited feature of Solr (and of > course under documented too). This translates to a Lucene > TermQuery. The benefit is no parsing is done, no escaping is > necessary (other than URL encoding for an HTTP request). > > This is also a technique that should be used for fq's that come from > facet clicks. You probably right now are using field:"facet value", > which works most of the time, but is subject to query parsing and > escaping rules when it's an unnecessary process that can sometimes > be problematic. > >> Also, what do you think of renaming the handler to "getDocument" or >> "document" rather than "details"? In solr parlance, it is set up >> to retrieve a single document ... > > > /document (not camel-casing) is what I'd recommend. And you can do > clever stuff like this (pasted from LucidFind's solrconfig.xml): > > > > json > all > {!field f=id v=$id} > 1 > * > > > > We should have used {!raw} instead of {!field} though. The {!field} > qparser (what Solr calls a query parser in API-speak) will do the > same thing as raw for non-"text" field types, create a TermQuery for > the value as-is. But for text field types it analyzes the value and > creates either a PhraseQuery or TermQuery. In all of our uses {! > field} is used with non-text fields > > The clever thing here is that the request is very clean to Solr: > > /solr/document?id=666 > > Note the indirection from id into a q parameter, making the request > cleaner (but the config a bit more obtuse). > > Erik > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From rochkind at jhu.edu Tue Mar 10 18:47:24 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Tue, 10 Mar 2009 18:47:24 -0400 Subject: [Blacklight-development] current_user problem In-Reply-To: References: <49B6A434.80008@stanford.edu> <49B6BD1E.6050903@stanford.edu>, Message-ID: <90FF863A96E1EC42B8B240D04C88FB1D719FDFDD@JHEMTEXVS2.win.ad.jhu.edu> This is odd, I didn't think Engines worked that way, I would have thought the blank ApplicationController and ApplicationHelper files should have just been 'mixed in' on top of the Blacklight ones, having no effect since they were empty. Maybe that doesn't apply to the special ApplicationController and Helper like it does to other controllers/helpers? I still philosophically disagree with Erik on where the bounds of 'doing too much are', but certainly Blacklight plugin should work with a standard Rails app. In this case, the intended effect can be had by renaming those classes to BlacklightController and BlacklightHelper either modules or classes, and making sure all BL plugin controllers/helpers include or inherit from the Blacklight base controller/helper. Not sure whether inheritance of module include makes more sense -- if one is going to make it easier for the local application controllers/helpers to use this functionality too when desired, that's important. Jonathan ________________________________________ From: blacklight-development-bounces at rubyforge.org [blacklight-development-bounces at rubyforge.org] On Behalf Of Erik Hatcher [erikhatcher at mac.com] Sent: Tuesday, March 10, 2009 3:46 PM To: blacklight-development at rubyforge.org Subject: Re: [Blacklight-development] current_user problem Sounds like this is one example of where Blacklight is doing too much... it should be made to work as a plugin to an existing (starter) Rails application without this conflict. Erik On Mar 10, 2009, at 3:18 PM, Jessie Keck wrote: > Hi all, > Just figured it out. This is something that should definitely add > to the documentation. > > The issue was that I was installing the plugin to a rails generated > application (not installing the demo application) > > When using the rails my_rails_app command there are some files that > are generated for you. 2 important ones here being the > ApplicationController and the ApplicationHelper. Since these 2 > empty files are generated, they're overriding the blacklight plugins > ApplicationController and ApplicationHelper. > > Deleting the generated apps files controllers/application.rb and > helpers/application_helper.rb fix this problem. > > Getting this into the documentation will be very helpful to users. > > Let me know if you have any questions. > Thanks, > -Jessie > > Jessie Keck wrote: >> Hi all, >> I just tried a fresh install of blacklight (twice) and keep getting >> an Undefined Local Variable current_user error. >> >> I think what may have been the problem is the way I installed all >> of the require plugins, or possible a migration issue. I pretty >> much copied and pasted the commands right out of the readme to >> ensure that I didn't have any typos. >> >> When I removed the current_user stuff from the bookmark_controller >> partial, I started to get the same error but for the >> application_name variable, which I know works correctly. >> >> What could cause these problems? (Maybe this should be documented >> as a way to tell how you screwed up the install?) >> >> Any help would be greatly appreciated. I'm trying to merge my >> splash page code in with the latest and greatest from SVN to make >> sure everything works smoothly before I commit it. >> >> Thanks, >> -Jessie Keck >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org http://rubyforge.org/mailman/listinfo/blacklight-development Blacklightopac Blog http://blacklightopac.org/ From jkeck at stanford.edu Tue Mar 10 19:00:58 2009 From: jkeck at stanford.edu (Jessie Keck) Date: Tue, 10 Mar 2009 16:00:58 -0700 Subject: [Blacklight-development] Home Page comitted Message-ID: <49B6F12A.901@stanford.edu> Hello all, Just letting you know that I have just committed the home page files and they're good to go. Just FYI, this is what I've done. Added a HomeController (very light, has separate solr_params, just an index method which has a redirect to the CatalogController if any queries are passed to it [because of the facet links]) Added views/home directory Added views/home/index.html.erb (for the Index method of the HomeController. Has render call to the search form, home_text partial, and sidebar capture block which renders facets) Added views/home/_home_text.html.erb (partial which just contains the actual text) Modified routes.rb to reflect the new root route to be the HomeController Added tests for all above Added files Please note that I believe that some of the tests I have committed may be a little weak. We're still learning rSpec and if somebody wants to improve them go for it and let us know. I think that would help all of us learn rSpec better. Please let us know if you run into any problems with this new feature. Thanks, -Jessie Keck Stanford University From goodieboy at gmail.com Tue Mar 10 19:25:39 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 10 Mar 2009 19:25:39 -0400 Subject: [Blacklight-development] current_user problem In-Reply-To: <90FF863A96E1EC42B8B240D04C88FB1D719FDFDD@JHEMTEXVS2.win.ad.jhu.edu> References: <49B6A434.80008@stanford.edu> <49B6BD1E.6050903@stanford.edu> <90FF863A96E1EC42B8B240D04C88FB1D719FDFDD@JHEMTEXVS2.win.ad.jhu.edu> Message-ID: Yeah this is something I've dealt with in my engines apps by either removing the application controller/helper or putting the controller/helper code into a module. The plugin then "includes" the appropriate module just like your app would, if it chose to override those files. Matt On Tue, Mar 10, 2009 at 6:47 PM, Jonathan Rochkind wrote: > This is odd, I didn't think Engines worked that way, I would have thought > the blank ApplicationController and ApplicationHelper files should have just > been 'mixed in' on top of the Blacklight ones, having no effect since they > were empty. Maybe that doesn't apply to the special ApplicationController > and Helper like it does to other controllers/helpers? > > I still philosophically disagree with Erik on where the bounds of 'doing > too much are', but certainly Blacklight plugin should work with a standard > Rails app. > > In this case, the intended effect can be had by renaming those classes to > BlacklightController and BlacklightHelper either modules or classes, and > making sure all BL plugin controllers/helpers include or inherit from the > Blacklight base controller/helper. Not sure whether inheritance of module > include makes more sense -- if one is going to make it easier for the local > application controllers/helpers to use this functionality too when desired, > that's important. > > Jonathan > ________________________________________ > From: blacklight-development-bounces at rubyforge.org [ > blacklight-development-bounces at rubyforge.org] On Behalf Of Erik Hatcher [ > erikhatcher at mac.com] > Sent: Tuesday, March 10, 2009 3:46 PM > To: blacklight-development at rubyforge.org > Subject: Re: [Blacklight-development] current_user problem > > Sounds like this is one example of where Blacklight is doing too > much... it should be made to work as a plugin to an existing (starter) > Rails application without this conflict. > > Erik > > On Mar 10, 2009, at 3:18 PM, Jessie Keck wrote: > > > Hi all, > > Just figured it out. This is something that should definitely add > > to the documentation. > > > > The issue was that I was installing the plugin to a rails generated > > application (not installing the demo application) > > > > When using the rails my_rails_app command there are some files that > > are generated for you. 2 important ones here being the > > ApplicationController and the ApplicationHelper. Since these 2 > > empty files are generated, they're overriding the blacklight plugins > > ApplicationController and ApplicationHelper. > > > > Deleting the generated apps files controllers/application.rb and > > helpers/application_helper.rb fix this problem. > > > > Getting this into the documentation will be very helpful to users. > > > > Let me know if you have any questions. > > Thanks, > > -Jessie > > > > Jessie Keck wrote: > >> Hi all, > >> I just tried a fresh install of blacklight (twice) and keep getting > >> an Undefined Local Variable current_user error. > >> > >> I think what may have been the problem is the way I installed all > >> of the require plugins, or possible a migration issue. I pretty > >> much copied and pasted the commands right out of the readme to > >> ensure that I didn't have any typos. > >> > >> When I removed the current_user stuff from the bookmark_controller > >> partial, I started to get the same error but for the > >> application_name variable, which I know works correctly. > >> > >> What could cause these problems? (Maybe this should be documented > >> as a way to tell how you screwed up the install?) > >> > >> Any help would be greatly appreciated. I'm trying to merge my > >> splash page code in with the latest and greatest from SVN to make > >> sure everything works smoothly before I commit it. > >> > >> Thanks, > >> -Jessie Keck > >> _______________________________________________ > >> Blacklight-development mailing list > >> Blacklight-development at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/blacklight-development > >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From goodieboy at gmail.com Tue Mar 10 19:31:33 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 10 Mar 2009 19:31:33 -0400 Subject: [Blacklight-development] solr query for single document: standard or dismax? In-Reply-To: <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> References: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> Message-ID: This is sweet! I had no idea solr could do that fun stuff with params. This method would get my vote. Matt > The clever thing here is that the request is very clean to Solr: > > /solr/document?id=666 > > Note the indirection from id into a q parameter, making the request cleaner > (but the config a bit more obtuse). > > Erik > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From goodieboy at gmail.com Tue Mar 10 19:38:58 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 10 Mar 2009 19:38:58 -0400 Subject: [Blacklight-development] solr query for single document: standard or dismax? In-Reply-To: <9BB431CA-C6C0-4587-BB1B-3A5C73629DF4@stanford.edu> References: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> <9BB431CA-C6C0-4587-BB1B-3A5C73629DF4@stanford.edu> Message-ID: > > > (Actually, it doesn't look like solr-ruby 0.0.5 speaks more than dismax or > standard either ...) > > - Naomi > > > Naomi, you're kind of right. Currently the rsolr gem makes it hard to specify a request handler mapping. That has changed though; I have a branch of rsolr that has been simplified and makes it possible to do all kinds of requests to solr, with any arbitrary params: response = solr.send_request('/documents', :q=>"id:#{id}") The code is here: http://github.com/mwmitchell/rsolr/tree/no-response-wrap This new branch also removes all of the extra input param/response mapping. So no more :filters, :facets for input params, and no more response.documents, response.current_page for the response etc.. All of that convenience stuff will be in a separate library and will make the core of rsolr much leaner. Blacklight will then need to change a few things to make it work, but nothing too major. Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From elarson at library.wisc.edu Tue Mar 10 19:41:41 2009 From: elarson at library.wisc.edu (Eric Larson) Date: Tue, 10 Mar 2009 18:41:41 -0500 Subject: [Blacklight-development] SolrMarc 2.0 - Getting Started? In-Reply-To: <49B696EF.1090101@virginia.edu> References: <49B696EF.1090101@virginia.edu> Message-ID: <10367407-1D8D-41F5-8FA5-3550ED825841@library.wisc.edu> Hi Bob, I was eventually able to get solrmarc-2.0 to push my marc records into solr-1.3.0, which I configured to work with the Blacklight Demo schema and solrconfig (excluding the FastLRUCache caching). After the index was built, I pushed this working solr and index into the Blacklight bl- demo. It was a lengthy work around, but it did the trick. I'd love to get going with solr-1.4, so I'll try using the war fix mentioned below soon. When you have an updated GettingStarted doc, I'm happy to assist in testing its accuracy. Thanks for the help! - Eric On Mar 10, 2009, at 11:35 AM, Robert Haschart wrote: > Eric, > > I am in the process of updating the Getting Started document, as > well as polishing up the rough corners of the installation > process. You were on the right track below, and actually got very > close, until you ran into something I had set up incorrectly. > > More comments are interspersed below. > > > -Bob Haschart > > > Eric Larson wrote: > >> Hi all, >> >> We have begun playing with Blacklight at UW-Madison. While >> installation and importing data via the ruby rake task works fine, >> we'd like to try pushing 10% of our collection into our test >> install (1M+ records). >> >> To handle this larger import, I've been trying (unsuccessfully) to >> get the solrmarc-2.0 branch to work. The documentation in the >> GettingStarted.txt is not up to date. >> >> Below is a list of the steps I've been taking. If anyone can >> offer advice, or help me understand how solrmarc-2.0 wants to be >> setup, I'd greatly appreciate it. >> >> Right now, I'm planning to dive farther into vufind's import script >> to see how they have solrmarc working (I've successfully imported >> this marc set into a test vufind install). >> >> As referenced here, a SM guide for Blacklight would be *very >> appreciated* :) >> http://rubyforge.org/pipermail/blacklight-development/2009-January/000246.html >> >> Thanks! >> - Eric >> >> = = = = >> >> Test install (OSX) >> - - - - >> 1) Download blacklight-trunk /Users/elarson/Sites/blacklight >> * No problems here. >> * Test Solr import via rake works fine. >> >> 2) Download solrmarc-2.0 >> * In blacklight root dir >> - svn co svn checkout http://solrmarc.googlecode.com/svn/branches/solrmarc-2.0 >> solrmarc >> >> * Open and attempt to use /solrmarc/docs/GettingStarted.txt >> - ant -version => 1.7.0 >> - java -version => 1.5.0 >> - sudo ant >> : Init site => "uwmadison" >> : Jar file name => "uwmadison" >> : Prefix => "uwmadison" >> : Final jar file => "uwmadison_SolrMarc" >> : Heap => -Xmx265m (default) >> : Solr version => 1.3 (default) > > This might be the first step where you went astray. I chose to make > the default solr version 1.3 since that is the latest released > version. However the version being used in the blacklight demo is a > prerelease nightly-build version of solr 1.4. One of the > configuration options for solrmarc's solr version is "war", which if > selected requires you to enter the full path of the solr.war file > that you will be using to deploy solr in jetty or tomcat or tanother > webapp container of your choice. The build process will then > extract the jar files it needs directly from that war file. Thus > guaranteeing that the version of solr used by solrmarc and the > version of solr used in the webapp container for searching the solr > index, will be in sync. > > It is becoming increasingly clear to me that this should be the > default option, and possibly the only option. >> - BUILD SUCCESSFUL >> : Note: uwmadison_config_properties and uwmadison_index.properties >> are not created? >> > It is also becoming clear to me that the build process should walk > you through the creation of the xxx_config.properties file and at > least create a placeholder xxx_index.properties file for you. > >> - Edit importSamples.properties >> : This file doesn't exist... >> : Realize I'm on my own now... >> >> - Assuming I want to edit the VanillaBlacklightDemo properites files >> : demo_config.properties >> : demo_index.properties >> : Blacklight solr schema.xml should sync with demo_index.properties? >> : Copy these files to /solrmarc/uwmadison dir created during build >> : Rename to uwmadison_config.properties and >> uwmadison_index.properties >> >> - Edit: uwmadison_config.properties >> : solr.path => /Users/elarson/Sites/blacklight/jetty/solr >> >> - Leaving: uwmadison_index.properties >> : This *should* work with Vanilla blacklight install? >> > Correct the vanilla blacklight demo example directory was designed > to produce an indexer that will create exactly the type of index > that the blacklight demo expects. > >> - Assuming all should be ready to run the example import_file.sh >> script... >> : cd /solrmarc/dist >> : Using importfile script >> : # ./indexfile ../uwmadison/uwmadison_config.properties ../../../ >> vufind/import/catalog.mrc (yeah, we're testing vufind too) > > You should be able to run it using the following command: > > ./indexfile ../../../ vufind/import/catalog.mrc > > it should store the uwmadison_config.properties file in the jar file > it produces, and it should find and use that file when you run the > script indexfile. > >> >> # Problem 1: Unable to find uwmadison_index.properties >> - Let's try the full path >> : /Users/elarson/Sites/blacklight/solrmarc/uwmadison/ >> uwmadison_index.properties >> > You shouldn't need a full path here. As with the > uwmadison_config.properties file the uwmadison_index.properties > shold be stored in the jar file produced by the build process, and > it should be able to find that index.properties file based solely on > its name. Here's a perfect example of why having the build > process walk you though the creation of the xxx_config.properties > file would be very helpful. > >> # Problem 2: "Error: Problem instantiating SolrCore" >> - Let's start Solr :) >> - cd /blacklight/jetty; java -jar start.jar >> - Solr Starts >> - Retrying index script > > You shouldn't need to have a solr server running, however if you are > able to get the solr server running, running solr marc should be > very easy. > >> # Problem 3: "Error: Problem instantiating SolrCore" >> - Trace below >> ERROR [main] (SolrCoreLoader.java:149) - Error: Problem >> instantiating SolrCore >> java.lang.reflect.InvocationTargetException >> at >> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> at >> sun >> .reflect >> .NativeConstructorAccessorImpl >> .newInstance(NativeConstructorAccessorImpl.java:39) >> at >> sun >> .reflect >> .DelegatingConstructorAccessorImpl >> .newInstance(DelegatingConstructorAccessorImpl.java:27) >> at java.lang.reflect.Constructor.newInstance(Constructor.java:494) >> at org.solrmarc.solr.SolrCoreLoader.loadCore(SolrCoreLoader.java: >> 121) >> at >> org.solrmarc.marc.MarcImporter.getSolrCoreProxy(MarcImporter.java: >> 446) >> at org.solrmarc.marc.MarcImporter.(MarcImporter.java:73) >> at org.solrmarc.marc.MarcImporter.main(MarcImporter.java:462) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun >> .reflect >> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: 39) >> at >> sun >> .reflect >> .DelegatingMethodAccessorImpl >> .invoke(DelegatingMethodAccessorImpl.java: 25) >> at java.lang.reflect.Method.invoke(Method.java:585) >> at com.simontuffs.onejar.Boot.run(Boot.java:306) >> at com.simontuffs.onejar.Boot.main(Boot.java:159) >> Caused by: org.apache.solr.common.SolrException: Error loading >> class 'solr.FastLRUCache' >> at >> org >> .apache >> .solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java: 273) >> at org.apache.solr.search.CacheConfig.getConfig(CacheConfig.java: >> 90) >> at org.apache.solr.search.CacheConfig.getConfig(CacheConfig.java: >> 73) >> at org.apache.solr.core.SolrConfig.(SolrConfig.java:128) >> at org.apache.solr.core.SolrConfig.(SolrConfig.java:101) >> ... 14 more >> Caused by: java.lang.ClassNotFoundException: solr.FastLRUCache >> at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:188) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:316) >> at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java: >> 579) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:251) >> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:374) >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:242) >> at >> org >> .apache >> .solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java: 257) >> ... 18 more > > I'm pretty sure that solr.FastLRUCache class that it is trying to > load, was a new class as of solr 1.4, so the program is looking in > the solr 1.3 libraries it has, (as specified above for the sol > version default) > The shortest path to working should be: > > 1) edit uwmadison_config.properties to remove the full path > for solr.indexer.properties > > solr.indexer.properties = uwmadison_index.properties > > 2) find where the property for solr.version is stored and delete > it. (It should be in uwmadison/build.properties or > uwmadison/build_override.properties) > > 3) re-run ant and when asked for the solr.version enter war > and then enter the full path to the solr.war file in answer to the > next prompt. > (It should be /blacklight/jetty/webapps/solr.war in > your setup) > > 4) start indexing > > dist/indexfile ../../ vufind/import/catalog.mrc > > > Let me know if this helps you > > -Bob Haschart > >> - I'm not a Java person, so debugging the rest is _painful_ >> - A step by step Blacklight / SolrMarc guide would be fantastic!- >> I'm willing to help write this documentation, if someone can help >> we get this working locally. >> >> - - - - >> >> Again, thanks for the help. >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > From goodieboy at gmail.com Tue Mar 10 20:09:23 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 10 Mar 2009 20:09:23 -0400 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: <49B561DE.6020109@gmail.com> References: <49AEFB67.7080008@gmail.com> <49AF2FD3.1020102@stanford.edu> <49B561DE.6020109@gmail.com> Message-ID: Vernon, Not sure if this has been solved or not? Are you saying the only thing you changed when this error occurred is the value of :label ? If so, that's very puzzling. The only thing that really stands out to me about that error message is that the document.id is "agrecord.2008-11-27.1267669533". I'm sure that the default id param pattern for a show route is something that does not have a period. In fact, an id with a period in it could be confusing when it comes to specifying content-types like /documents/1234.json for example. I'd recommend not using periods in your document ids if at all possible. Matt On Mon, Mar 9, 2009 at 2:37 PM, Vernon Chapman wrote: > I am trying to follow Jessie's directions (below) to customize the demo > app for my own purposes. > I copied the view (_document_list.html.erb) for catalog index into the > controlling app's view/catalog directory. > > My problem seems is that when the link_to_document helper is called. The > line reads > <%= link_to_document document, :label=>:title_t, > :extra_params=>{:counter => counter + 1 + @response.start.to_i } %> > > .My title field is named well title and not title_t so I changed the > controling app's _document_list.html.erb to > <%= link_to_document document, :label=>:title, :extra_params=>{:counter > => counter + 1 + @response.start.to_i } %> > > Now I get the following when I try to access the index page of the catalog > > Showing app/views/catalog/_document_list.html.erb where line #21 raised: > > catalog_url failed to generate from {:commit=>nil, :action=>"show", > :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", > :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, diff: > {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} > > Extracted source (around line #21): > > 18: <% # main title container for doc partial view -%> > 19: > 20:
> 21:
<%= counter + 1 + @response.start.to_i %>. <%= > link_to_document document, :label=>:title, :extra_params=>{:counter => > counter + 1 + @response.start.to_i } %>
> 22:
> 23: > 24: > > What am I screwing up this time? (LOL) > Thnks in advance folks. > > > > Jessie Keck wrote: > >> Hi Vern, >> Just chiming in about overriding the interface. >> >> If you find a view that you would like to override then you just need to >> copy that file from the plugin and paste it in your top level application >> (the one generated with =>> rails {your_app_name}) making sure to keep the >> same directory structure as the plugin. >> >> The engines plugin will look at your top level app first to find a file >> then to the blacklight plugin. An example of this is if you wanted the >> search box to be available on all pages not just search results. This is an >> easy way to do that: >> 1) Copy file views/catalog/index.html.erb from plugin to the top level app >> 2) Remove render tag from the index view in the top level app >> 3) Copy file views/catalog/_search_form.html erb to the top level app to >> the views/shared directory (note that I did not maintain the directory >> structure because the search form partial is now shared amongst other >> controllers) >> 4) Copy file views/layouts/application.html.erb to the top level app >> 5) Add the render tag to application view to now be render :partial => >> 'shared/search_form' >> (Note: putting this is the application view is probably not the best idea, >> but this is just an example. I'm actually rendering the search form partial >> from a header partial, not the application view) >> >> That's about it. You can go nuts with overriding all kinds of files. Do >> remember that certain things may require a restart to take affect (most >> notably changes to helpers) >> >> Hope this helps, >> Let us know if any of this is unclear or you run into any snags. >> >> -Jessie Keck >> Stanford University Libraries >> >> Vernon Chapman wrote: >> >>> Is there some documentation on how to customize blacklight? >>> I have a lot or Dublin Core(ish) metadata in a solr index and would like >>> to use blacklight as a front end to it. >>> >>> Thanks in advance >>> >>> Vern >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chapman.lists at gmail.com Tue Mar 10 20:48:37 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Tue, 10 Mar 2009 20:48:37 -0400 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: References: <49AEFB67.7080008@gmail.com> <49AF2FD3.1020102@stanford.edu> <49B561DE.6020109@gmail.com> Message-ID: <49B70A65.3050503@gmail.com> Matt, Never thought of that, however that has never been a problem with solr at least. I'll try and remove the "." in the id and see what happens. thanks Vern Matt Mitchell wrote: > Vernon, > > Not sure if this has been solved or not? Are you saying the only thing > you changed when this error occurred is the value of :label ? If so, > that's very puzzling. > > The only thing that really stands out to me about that error message > is that the document.id is > "agrecord.2008-11-27.1267669533". I'm sure that the default id param > pattern for a show route is something that does not have a period. In > fact, an id with a period in it could be confusing when it comes to > specifying content-types like /documents/1234.json for example. I'd > recommend not using periods in your document ids if at all possible. > > Matt > > On Mon, Mar 9, 2009 at 2:37 PM, Vernon Chapman > > wrote: > > I am trying to follow Jessie's directions (below) to customize > the demo > app for my own purposes. > I copied the view (_document_list.html.erb) for catalog index into the > controlling app's view/catalog directory. > > My problem seems is that when the link_to_document helper is > called. The > line reads > <%= link_to_document document, :label=>:title_t, > :extra_params=>{:counter => counter + 1 + @response.start.to_i } %> > > .My title field is named well title and not title_t so I changed the > controling app's _document_list.html.erb to > <%= link_to_document document, :label=>:title, > :extra_params=>{:counter > => counter + 1 + @response.start.to_i } %> > > Now I get the following when I try to access the index page of the > catalog > > Showing app/views/catalog/_document_list.html.erb where line #21 > raised: > > catalog_url failed to generate from {:commit=>nil, :action=>"show", > :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", > :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, > diff: > {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} > > Extracted source (around line #21): > > 18: <% # main title container for doc partial view -%> > 19: > 20:
> 21:
<%= counter + 1 + @response.start.to_i %>. <%= > link_to_document document, :label=>:title, :extra_params=>{:counter => > counter + 1 + @response.start.to_i } %>
> 22:
> 23: > 24: > > What am I screwing up this time? (LOL) > Thnks in advance folks. > > > > Jessie Keck wrote: > > Hi Vern, > Just chiming in about overriding the interface. > > If you find a view that you would like to override then you > just need to copy that file from the plugin and paste it in > your top level application (the one generated with =>> rails > {your_app_name}) making sure to keep the same directory > structure as the plugin. > > The engines plugin will look at your top level app first to > find a file then to the blacklight plugin. An example of this > is if you wanted the search box to be available on all pages > not just search results. This is an easy way to do that: > 1) Copy file views/catalog/index.html.erb from plugin to the > top level app > 2) Remove render tag from the index view in the top level app > 3) Copy file views/catalog/_search_form.html erb to the top > level app to the views/shared directory (note that I did not > maintain the directory structure because the search form > partial is now shared amongst other controllers) > 4) Copy file views/layouts/application.html.erb to the top > level app > 5) Add the render tag to application view to now be render > :partial => 'shared/search_form' > (Note: putting this is the application view is probably not > the best idea, but this is just an example. I'm actually > rendering the search form partial from a header partial, not > the application view) > > That's about it. You can go nuts with overriding all kinds of > files. Do remember that certain things may require a restart > to take affect (most notably changes to helpers) > > Hope this helps, > Let us know if any of this is unclear or you run into any snags. > > -Jessie Keck > Stanford University Libraries > > Vernon Chapman wrote: > > Is there some documentation on how to customize blacklight? > I have a lot or Dublin Core(ish) metadata in a solr index > and would like > to use blacklight as a front end to it. > > Thanks in advance > > Vern > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From erikhatcher at mac.com Tue Mar 10 21:02:28 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 10 Mar 2009 21:02:28 -0400 Subject: [Blacklight-development] current_user problem In-Reply-To: <90FF863A96E1EC42B8B240D04C88FB1D719FDFDD@JHEMTEXVS2.win.ad.jhu.edu> References: <49B6A434.80008@stanford.edu> <49B6BD1E.6050903@stanford.edu> <90FF863A96E1EC42B8B240D04C88FB1D719FDFDD@JHEMTEXVS2.win.ad.jhu.edu> Message-ID: <2CEE3943-079A-455F-9B51-5BBBDBF50BF4@mac.com> On Mar 10, 2009, at 6:47 PM, Jonathan Rochkind wrote: > I still philosophically disagree with Erik on where the bounds of > 'doing too much are', but certainly Blacklight plugin should work > with a standard Rails app. It's not a philosophical discussion. That was over when we chose Ruby :) It's a technical issue with plugging Blacklight into an existing Rails app, apparently. If the plugin interferes with a straightforward Rails app, Blacklight has done too much. That's all. And we don't need to have a discussion about it other than to address the issue as it crops up in Real Working Code :) Erik From erikhatcher at mac.com Tue Mar 10 21:08:03 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 10 Mar 2009 21:08:03 -0400 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: <49B70A65.3050503@gmail.com> References: <49AEFB67.7080008@gmail.com> <49AF2FD3.1020102@stanford.edu> <49B561DE.6020109@gmail.com> <49B70A65.3050503@gmail.com> Message-ID: Solr has no issues with any characters in any field... but there are query parser and URL encoding issues to contend with. Erik On Mar 10, 2009, at 8:48 PM, Vernon Chapman wrote: > Matt, > > Never thought of that, however that has never been a problem with > solr at least. I'll try > and remove the "." in the id and see what happens. > > thanks > > Vern > > Matt Mitchell wrote: >> Vernon, >> >> Not sure if this has been solved or not? Are you saying the only >> thing you changed when this error occurred is the value of :label ? >> If so, that's very puzzling. >> >> The only thing that really stands out to me about that error >> message is that the document.id is "agrecord. >> 2008-11-27.1267669533". I'm sure that the default id param pattern >> for a show route is something that does not have a period. In fact, >> an id with a period in it could be confusing when it comes to >> specifying content-types like /documents/1234.json for example. I'd >> recommend not using periods in your document ids if at all possible. >> >> Matt >> >> On Mon, Mar 9, 2009 at 2:37 PM, Vernon Chapman > > wrote: >> >> I am trying to follow Jessie's directions (below) to customize >> the demo >> app for my own purposes. >> I copied the view (_document_list.html.erb) for catalog index >> into the >> controlling app's view/catalog directory. >> >> My problem seems is that when the link_to_document helper is >> called. The >> line reads >> <%= link_to_document document, :label=>:title_t, >> :extra_params=>{:counter => counter + 1 + @response.start.to_i } >> %> >> >> .My title field is named well title and not title_t so I changed >> the >> controling app's _document_list.html.erb to >> <%= link_to_document document, :label=>:title, >> :extra_params=>{:counter >> => counter + 1 + @response.start.to_i } %> >> >> Now I get the following when I try to access the index page of the >> catalog >> >> Showing app/views/catalog/_document_list.html.erb where line #21 >> raised: >> >> catalog_url failed to generate from >> {:commit=>nil, :action=>"show", >> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", >> :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, >> diff: >> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} >> >> Extracted source (around line #21): >> >> 18: <% # main title container for doc partial view -%> >> 19: >> 20:
>> 21:
<%= counter + 1 + @response.start.to_i %>. <%= >> link_to_document >> document, :label=>:title, :extra_params=>{:counter => >> counter + 1 + @response.start.to_i } %>
>> 22:
>> 23: >> 24: >> >> What am I screwing up this time? (LOL) >> Thnks in advance folks. >> >> >> >> Jessie Keck wrote: >> >> Hi Vern, >> Just chiming in about overriding the interface. >> >> If you find a view that you would like to override then you >> just need to copy that file from the plugin and paste it in >> your top level application (the one generated with =>> rails >> {your_app_name}) making sure to keep the same directory >> structure as the plugin. >> >> The engines plugin will look at your top level app first to >> find a file then to the blacklight plugin. An example of this >> is if you wanted the search box to be available on all pages >> not just search results. This is an easy way to do that: >> 1) Copy file views/catalog/index.html.erb from plugin to the >> top level app >> 2) Remove render tag from the index view in the top level app >> 3) Copy file views/catalog/_search_form.html erb to the top >> level app to the views/shared directory (note that I did not >> maintain the directory structure because the search form >> partial is now shared amongst other controllers) >> 4) Copy file views/layouts/application.html.erb to the top >> level app >> 5) Add the render tag to application view to now be render >> :partial => 'shared/search_form' >> (Note: putting this is the application view is probably not >> the best idea, but this is just an example. I'm actually >> rendering the search form partial from a header partial, not >> the application view) >> >> That's about it. You can go nuts with overriding all kinds of >> files. Do remember that certain things may require a restart >> to take affect (most notably changes to helpers) >> >> Hope this helps, >> Let us know if any of this is unclear or you run into any >> snags. >> >> -Jessie Keck >> Stanford University Libraries >> >> Vernon Chapman wrote: >> >> Is there some documentation on how to customize >> blacklight? >> I have a lot or Dublin Core(ish) metadata in a solr index >> and would like >> to use blacklight as a front end to it. >> >> Thanks in advance >> >> Vern >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From erikhatcher at mac.com Tue Mar 10 21:29:14 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 10 Mar 2009 21:29:14 -0400 Subject: [Blacklight-development] solr query for single document: standard or dismax? In-Reply-To: <9BB431CA-C6C0-4587-BB1B-3A5C73629DF4@stanford.edu> References: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> <9BB431CA-C6C0-4587-BB1B-3A5C73629DF4@stanford.edu> Message-ID: <419F32C3-02C8-4F20-B957-D6E2AB5DFF19@mac.com> On Mar 10, 2009, at 5:59 PM, Naomi Dushay wrote: > If I go with: > >> >> >> ruby >> all >> {!raw f=id v=$id} >> 1 >> * >> >> > > > Do I need to / should I set defType to something? defType is the default query parser type for the q parameter, but is overridden with {!...} syntax - so defType wouldn't matter at all in this example. > The mwmitchell-rsolr gem used by blacklight appears only to speak > "dismax" or "standard". And that's an issue... communication to Solr needs to be made very transparent and configurable in all this query parsing mojo. You folks will have to decide exactly how to arrange it to be as generic and configurable as possible, from being entirely UI controllable, to being totally fixed to only allowing a q parameter to be set from the UI and nothing else, and any number of combinations of some UI configurability and some fixed pieces (like controlling qf from a drop- down, for example). And defType is just one of the many parameters that need to fall in that spectrum. And it of course different controllers could have different presets/defaults/invariants and so on. > If we're using your suggestions, do we need the gem to speak other > defTypes? Absolutely. > For example, we may need to have a request handler using the > LuceneQParserPlugin so it will understand Boolean syntax. Of course (and we covered this in a previous thread, no?) > How do we use the rsolr gem to do this? Over to Matt on this one :) > Or, to gain this flexibility, would it be prudent for blacklight > to use the solr-ruby gem instead of the mwmitchell-solr gem? I'm working with Matt to get his ideas/implementation into a solr-ruby refactoring, so in the not too distant future we won't have this discrepancy... we'll all happily be using solr-ruby 1.0 :) Note that within Blacklight code there really should only be one up to a few places that a request is sent to Solr. As long as the entire spectrum of parameters can be controlled in those spots in some way, all is fine with any ruby->solr API. > I'm already refactoring the ruby code for solr queries into a > solr_helper class (abstracting to "get_single_document" and > "get_search_results_and_facets" sort of concepts), so if the ruby > code is a little more complex within blacklight, it's nicely isolated. +1 > (Actually, it doesn't look like solr-ruby 0.0.5 speaks more than > dismax or standard either ...) Not true. There's built-in Standard and Dismax requests, yes. But there is also a general purpose Select request that can be used: $ irb irb(main):002:0> solr = Solr::Connection.new => #, @url=#> irb(main):007:0> req = Solr::Request::Select.new('dismax') => # irb(main):008:0> solr.send(req) Note that solr-ruby's Solr::Request::Select doesn't handle requests with leading slashes appropriately (it uses /solr/select? qt=, and leading slashes Solr dislikes on qt). But in the case of a request handler with a leading slash, making a Select subclass (and a corresponding response class) make it pretty easy to use any request handler you like, including a response class to hang helpers onto. Erik From erikhatcher at mac.com Tue Mar 10 21:32:22 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 10 Mar 2009 21:32:22 -0400 Subject: [Blacklight-development] solr query for single document: standard or dismax? In-Reply-To: References: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> <9BB431CA-C6C0-4587-BB1B-3A5C73629DF4@stanford.edu> Message-ID: <3B3225A9-709A-4FBD-8CA4-D3CDD0383021@mac.com> Probably of no use here, but solr-ruby 0.0.5 is ancient. 0.0.6 has been out since July '08. And I recently pushed an 0.0.7 version. Pleasantly, the CHANGES list is in YAML :) Probably should make that JSON instead, just to be philosophical about it. Erik On Mar 10, 2009, at 7:38 PM, Matt Mitchell wrote: > > > > (Actually, it doesn't look like solr-ruby 0.0.5 speaks more than > dismax or standard either ...) > > - Naomi > > > > Naomi, you're kind of right. Currently the rsolr gem makes it hard > to specify a request handler mapping. That has changed though; I > have a branch of rsolr that has been simplified and makes it > possible to do all kinds of requests to solr, with any arbitrary > params: > > response = solr.send_request('/documents', :q=>"id:#{id}") > > The code is here: http://github.com/mwmitchell/rsolr/tree/no-response-wrap > > This new branch also removes all of the extra input param/response > mapping. So no more :filters, :facets for input params, and no more > response.documents, response.current_page for the response etc.. All > of that convenience stuff will be in a separate library and will > make the core of rsolr much leaner. Blacklight will then need to > change a few things to make it work, but nothing too major. > > Matt > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From goodieboy at gmail.com Tue Mar 10 22:07:04 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 10 Mar 2009 20:07:04 -0600 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: References: <49AEFB67.7080008@gmail.com> <49AF2FD3.1020102@stanford.edu> <49B561DE.6020109@gmail.com> <49B70A65.3050503@gmail.com> Message-ID: <62BFE7F4-662C-419A-BFCC-C1BE0B836D76@gmail.com> Oh, just to be clear on what I meant... I was strictly talking about rails. Also, the regex for any route param Is customizable. Sent from my iPhone On Mar 10, 2009, at 7:08 PM, Erik Hatcher wrote: > Solr has no issues with any characters in any field... but there are > query parser and URL encoding issues to contend with. > > Erik > > On Mar 10, 2009, at 8:48 PM, Vernon Chapman wrote: > >> Matt, >> >> Never thought of that, however that has never been a problem with >> solr at least. I'll try >> and remove the "." in the id and see what happens. >> >> thanks >> >> Vern >> >> Matt Mitchell wrote: >>> Vernon, >>> >>> Not sure if this has been solved or not? Are you saying the only >>> thing you changed when this error occurred is the value >>> of :label ? If so, that's very puzzling. >>> >>> The only thing that really stands out to me about that error >>> message is that the document.id is "agrecord. >>> 2008-11-27.1267669533". I'm sure that the default id param pattern >>> for a show route is something that does not have a period. In >>> fact, an id with a period in it could be confusing when it comes >>> to specifying content-types like /documents/1234.json for example. >>> I'd recommend not using periods in your document ids if at all >>> possible. >>> >>> Matt >>> >>> On Mon, Mar 9, 2009 at 2:37 PM, Vernon Chapman >> > wrote: >>> >>> I am trying to follow Jessie's directions (below) to customize >>> the demo >>> app for my own purposes. >>> I copied the view (_document_list.html.erb) for catalog index >>> into the >>> controlling app's view/catalog directory. >>> >>> My problem seems is that when the link_to_document helper is >>> called. The >>> line reads >>> <%= link_to_document document, :label=>:title_t, >>> :extra_params=>{:counter => counter + 1 + @response.start.to_i } >>> %> >>> >>> .My title field is named well title and not title_t so I changed >>> the >>> controling app's _document_list.html.erb to >>> <%= link_to_document document, :label=>:title, >>> :extra_params=>{:counter >>> => counter + 1 + @response.start.to_i } %> >>> >>> Now I get the following when I try to access the index page of the >>> catalog >>> >>> Showing app/views/catalog/_document_list.html.erb where line #21 >>> raised: >>> >>> catalog_url failed to generate from >>> {:commit=>nil, :action=>"show", >>> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", >>> :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, >>> diff: >>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} >>> >>> Extracted source (around line #21): >>> >>> 18: <% # main title container for doc partial view -%> >>> 19: >>> 20:
>>> 21:
<%= counter + 1 + @response.start.to_i %>. <%= >>> link_to_document >>> document, :label=>:title, :extra_params=>{:counter => >>> counter + 1 + @response.start.to_i } %>
>>> 22:
>>> 23: >>> 24: >>> >>> What am I screwing up this time? (LOL) >>> Thnks in advance folks. >>> >>> >>> >>> Jessie Keck wrote: >>> >>> Hi Vern, >>> Just chiming in about overriding the interface. >>> >>> If you find a view that you would like to override then you >>> just need to copy that file from the plugin and paste it in >>> your top level application (the one generated with =>> rails >>> {your_app_name}) making sure to keep the same directory >>> structure as the plugin. >>> >>> The engines plugin will look at your top level app first to >>> find a file then to the blacklight plugin. An example of this >>> is if you wanted the search box to be available on all pages >>> not just search results. This is an easy way to do that: >>> 1) Copy file views/catalog/index.html.erb from plugin to the >>> top level app >>> 2) Remove render tag from the index view in the top level app >>> 3) Copy file views/catalog/_search_form.html erb to the top >>> level app to the views/shared directory (note that I did not >>> maintain the directory structure because the search form >>> partial is now shared amongst other controllers) >>> 4) Copy file views/layouts/application.html.erb to the top >>> level app >>> 5) Add the render tag to application view to now be render >>> :partial => 'shared/search_form' >>> (Note: putting this is the application view is probably not >>> the best idea, but this is just an example. I'm actually >>> rendering the search form partial from a header partial, not >>> the application view) >>> >>> That's about it. You can go nuts with overriding all kinds of >>> files. Do remember that certain things may require a restart >>> to take affect (most notably changes to helpers) >>> >>> Hope this helps, >>> Let us know if any of this is unclear or you run into any >>> snags. >>> >>> -Jessie Keck >>> Stanford University Libraries >>> >>> Vernon Chapman wrote: >>> >>> Is there some documentation on how to customize >>> blacklight? >>> I have a lot or Dublin Core(ish) metadata in a solr index >>> and would like >>> to use blacklight as a front end to it. >>> >>> Thanks in advance >>> >>> Vern >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >>> >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> >>> --- >>> --- >>> ------------------------------------------------------------------ >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From mpc3c at virginia.edu Wed Mar 11 10:35:37 2009 From: mpc3c at virginia.edu (Molly Pickral Cadieux) Date: Wed, 11 Mar 2009 10:35:37 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <49B6B6BB.5020309@gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> Message-ID: <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> Vern, Just to make sure -- did you restart passenger or mongrel or whatever it is you are using? Changes to helpers seem to need an extra kick. Molly On Tue, Mar 10, 2009 at 2:51 PM, Vernon Chapman wrote: > Molly, > > Thanks for your work on this, but unfortunately that didn't fix the > issue. I did an svn update on my checkout of blacklight ?and restarted > apache and I still get the same error. > > Vern > > Molly Pickral Cadieux wrote: >> >> Hi again, >> >> I went ahead and tried the fix on demo.blacklightopac.org, and it >> works great! ?So, Vernon, if you could grab the new version of >> catalog_helper.rb from subversion, I'm almost certain that will fix >> the bug you reported yesterday. >> >> Molly >> >> On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux >> wrote: >> >>> >>> Hi everyone, >>> >>> I recently committed some changes to the repository that offer the >>> following features: >>> >>> 1. ?A drop-down list that allows users to specify how may results per >>> page to display (the previous default was 10 per page) >>> 2. ?"Previous" and "Next" links that allow a user to page through a >>> search result, as opposed to going back to the previous page to select >>> the previous or next item. >>> 3. ?Several rspec tests for the catalog controller and various models. >>> >>> This all works perfectly on my setup, but when I went to deploy the >>> changes on demo.blacklightopac.org, I ran into a routing issue. >>> Vernon also reported this error yesterday. ?The error is as follows: >>> >>> ------------ >>> >>> catalog_url failed to generate from {:commit=>nil, :action=>"show", >>> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", >>> :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, diff: >>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} >>> >>> Extracted source (around line #21): >>> >>> 18: ? ? ? ? <% # main title container for doc partial view -%> >>> 19: >>> 20: ? ? ? ?
>>> 21: ? ? ? ? ?
<%= counter + 1 + @response.start.to_i %>. <%= >>> link_to_document document, :label=>:title, :extra_params=>{:counter => >>> counter + 1 + @response.start.to_i } %>
>>> 22: ? ? ? ?
>>> 23: >>> 24: ? ? ? >>> >>> ----------- >>> >>> Bess had reported a similar problem back in February (I've included >>> the message below). ?Unfortunately, I have been unable to reproduce >>> the reported problem on my Mac running the same versions of Ruby, >>> Rails, and Passenger. ?I would like to try Bess's recommended solution >>> if I can get a setup where I can reproduce the problem. ?In the mean >>> time, I'd love it if Vernon or others who may have seen the problem >>> could try modifying this file: >>> >>> rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb >>> >>> and change this line: >>> >>> link_to label, catalog_path(doc[:id], >>> params.merge(opts[:extra_params]).merge(:action=>nil, :commit=>nil)) >>> >>> to be: >>> >>> link_to label, catalog_path(doc[:id], >>> params.merge(opts[:extra_params]).merge(:action=>"show", >>> :commit=>nil)) >>> >>> and see if that fixes it. ?A restart of passenger will be necessary. >>> >>> If I can't get this worked out soon, I will revert the changes in the >>> repository and work through this again here. >>> >>> In any case, I wanted to let you all know that what is currently in >>> the repository may cause routing issues under certain environments. >>> I'll keep you all posted on progress. >>> >>> Molly >>> >>> >>> ---------- >>> >>> From eos8d at virginia.edu ?Sun Feb ?1 17:25:35 2009 >>> From: eos8d at virginia.edu (Bess Sadler) >>> Date: Sun, 1 Feb 2009 17:25:35 -0500 >>> Subject: [Blacklight-development] demo app bug fix >>> Message-ID: >>> >>> Hey, folks. >>> >>> I've spent the last day or so playing pretty intensively with the demo >>> app. I fixed a couple of minor bugs in the installation instructions >>> (here: >>> http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) >>> ?and I've deployed the demo app several times, on several different >>> systems. Strangely, although I could get it running right away with >>> mongrel or webrick, I found that I could not get it running with >>> mod_passenger, which is our preferred method of running rails apps in >>> production. I think I tracked down the bug today, and you can see the >>> demo app running with mod_passenger as a virtual host at >>> http://demo.blacklightopac.org >>> >>> I was getting two errors. One, encountered whenever the index screen >>> was loaded, said: >>> >>> ActionView::TemplateError (catalog_url failed to generate from >>> {:action=>"index", :controller=>"catalog", :id=>"u1"}, expected: >>> {:action=>"show", :controller=>"catalog"}, diff: >>> {:action=>"show", :id=>"u1"}) on line #11 of vendor/plugins/blacklight/ >>> app/views/catalog/_index_partials/_default.html.erb: >>> 8: >>> 9: ? ?
>>> 10: ? ? ?
>>> 11: ? ? ? ? <%= link_to document.get(:id), catalog_path(document[:id], >>> params) %> >>> 12: ? ? ?
>>> 13: ? ?
>>> 14: ? >>> >>> >>> The other was generated whenever the show screen was loaded, and it >>> said: >>> >>> ActionView::TemplateError (catalog_index_url failed to generate from >>> {:action=>"show", :controller=>"catalog", :id=>nil}, expected: >>> {:action=>"index", :controller=>"catalog"}, diff: >>> {:action=>"index", :id=>nil}) on line #8 of vendor/plugins/blacklight/ >>> app/views/catalog/show.html.erb: >>> 5: >>> 6: >>> 7: <% @sidebar = capture do %> >>> 8: ?

<%= link_to '<< SEARCH', >>> catalog_index_path(params.merge(:id=>nil)) %>

>>> 9: <% end %> >>> >>> Turns out these were basically the same error. In both cases, we're >>> using the link_to function to create a link to another part of the >>> application, in this case from index to show, and from show to index. >>> Let's talk about the link from index to show as an example. >>> >>> <%= link_to document.get(:id), catalog_path(document[:id], params) %> >>> >>> That says create a link, and the text of the link is the id number of >>> the current document (say, u999), and the target of the link is >>> catalog_path(u999) which will evaluate to /catalog/u999. Incidentally, >>> the rails console was really valuable in helping me understand how >>> these worked, and you can play with it like this: >>> >>> paz:rails eos8d$ ./script/console >>> Loading development environment (Rails 2.2.2) >>> ?>> app.catalog_path("u999") >>> => "/catalog/u999" >>> >>> Anyway, I found that if I just changed the link_to statement to look >>> like this instead, removing the params object: >>> >>> ? ? ? <%= link_to document.get(:id), catalog_path(document[:id]) %> >>> >>> the error went away. However, this wasn't an acceptable solution, >>> because whenever a user performs a search or selects a facet, that >>> value goes in the params hash, and you want to keep it around when you >>> go from screen to screen, so that the user isn't always having to re- >>> enter their current search. Let's say you're on the index screen, and >>> you've entered a search and selected a facet. At that point your >>> params hash looks like this: >>> >>> ? ? ? {"commit"=>"search", "action"=>"index", "q"=>"poetry", >>> "f"=>{"language_facet"=>["English"]}, "controller"=>"catalog"} >>> >>> When you want to look at a detailed view for a specific item, you're >>> linking to the show action of your same catalog controller, and >>> passing it a specific document id. But you also want to hang onto the >>> fact that you've searched for poetry and limited to >>> language_facet:English, so you also pass the params object in. At >>> which point your link_to really looks like this: >>> >>> ? ? ? <%= link_to document.get(:id), >>> catalog_path >>> (:id >>> =>document[:id], :commit=>"search", :action=>"index", :q=>"poetry" ... >>> etc ... ) %> >>> >>> Do you see the problem? It took me awhile to notice it. I've just >>> called the show action, and then I also pass in a parameter of >>> action=index. It's not obvious, because catalog_path sort of obscures >>> the fact that you're calling the show action. Interestingly, webrick >>> and mongrel don't seem to care. I guess they just assume that even >>> though you've passed two values for action, you must really mean >>> action=show because that's what's defined for catalog_path. The fix is >>> easy, once you know what the problem is: >>> >>> ? ? ? ?<%= link_to document.get(:id), >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >>> >>> Now you're passing in a params object that looks like this: >>> >>> ? ? ? {"commit"=>"search", "action"=>"show", "id"=>"2008349843", >>> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", >>> "controller"=>"catalog"} >>> >>> So, all your user defined parameters, plus the id number of the >>> specific item you want to show, plus action=show, which is what >>> catalog_path expects. Problem solved. >>> >>> Sorry this is so long, but I learned a lot in fixing this bug and I >>> thought it might be helpful for someone else too. But if it just looks >>> like gibberish, ignore it. >>> >>> Cheers, >>> Bess >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> From goodieboy at gmail.com ?Sun Feb ?1 17:59:36 2009 >>> From: goodieboy at gmail.com (Matt Mitchell) >>> Date: Sun, 1 Feb 2009 17:59:36 -0500 >>> Subject: [Blacklight-development] demo app bug fix >>> In-Reply-To: >>> References: >>> Message-ID: >>> >>> Seriously, that's a kickin' write up Bess. Great bug catch. And the docs >>> look so nice too! >>> >>> For the link_to fix here: >>> >>> <%= link_to document.get(:id), >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >>> >>> You should be able to do this too: >>> >>> <%= link_to document.get(:id), catalog_path(document[:id], >>> params.merge(:action=>nil)) %> >>> >>> I think this kind of thing will become a url-param-linking pattern in BL >>> and >>> might do well in a view helper. I think there's one in the UVa trunk, >>> called >>> link_to_document or something. We could then have one called >>> link_to_documents or link_back_to_search? The upshot of it being in a >>> helper >>> is that if someone overrides a view partial, they don't have to re-write >>> the >>> param merging stuff. Food for thought... >>> >>> Matt >>> >>> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From jamie at dang.com Wed Mar 11 10:52:57 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Wed, 11 Mar 2009 10:52:57 -0400 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: <62BFE7F4-662C-419A-BFCC-C1BE0B836D76@gmail.com> References: <49AEFB67.7080008@gmail.com> <49AF2FD3.1020102@stanford.edu> <49B561DE.6020109@gmail.com> <49B70A65.3050503@gmail.com> <62BFE7F4-662C-419A-BFCC-C1BE0B836D76@gmail.com> Message-ID: <91B4F6DC-3EC1-4AC1-A6EF-F2BFAA0B682B@dang.com> In a standard rails route, a period is a delimiter for the extension, which then gets used to deliver the mapped mime-type. This way you can do things like /foo.xml, /foo.json, /foo.js in order to use the same URL pattern, but deliver different types/formats. If you have a period in your URL and it's not part of a query string, it would cause problems. I had this issue with doc ids with a rails client to XTF that I wrote. I dealt with it my transforming the periods in the doc ids to something else (and then back again when sent to XTF). Jamie On Mar 10, 2009, at 10:07 PM, Matt Mitchell wrote: > Oh, just to be clear on what I meant... I was strictly talking about > rails. Also, the regex for any route param > Is customizable. > > Sent from my iPhone > > On Mar 10, 2009, at 7:08 PM, Erik Hatcher wrote: > >> Solr has no issues with any characters in any field... but there >> are query parser and URL encoding issues to contend with. >> >> Erik >> >> On Mar 10, 2009, at 8:48 PM, Vernon Chapman wrote: >> >>> Matt, >>> >>> Never thought of that, however that has never been a problem with >>> solr at least. I'll try >>> and remove the "." in the id and see what happens. >>> >>> thanks >>> >>> Vern >>> >>> Matt Mitchell wrote: >>>> Vernon, >>>> >>>> Not sure if this has been solved or not? Are you saying the only >>>> thing you changed when this error occurred is the value >>>> of :label ? If so, that's very puzzling. >>>> >>>> The only thing that really stands out to me about that error >>>> message is that the document.id is "agrecord. >>>> 2008-11-27.1267669533". I'm sure that the default id param >>>> pattern for a show route is something that does not have a >>>> period. In fact, an id with a period in it could be confusing >>>> when it comes to specifying content-types like /documents/ >>>> 1234.json for example. I'd recommend not using periods in your >>>> document ids if at all possible. >>>> >>>> Matt >>>> >>>> On Mon, Mar 9, 2009 at 2:37 PM, Vernon Chapman >>> > wrote: >>>> >>>> I am trying to follow Jessie's directions (below) to customize >>>> the demo >>>> app for my own purposes. >>>> I copied the view (_document_list.html.erb) for catalog index >>>> into the >>>> controlling app's view/catalog directory. >>>> >>>> My problem seems is that when the link_to_document helper is >>>> called. The >>>> line reads >>>> <%= link_to_document document, :label=>:title_t, >>>> :extra_params=>{:counter => counter + 1 + @response.start.to_i } >>>> %> >>>> >>>> .My title field is named well title and not title_t so I changed >>>> the >>>> controling app's _document_list.html.erb to >>>> <%= link_to_document document, :label=>:title, >>>> :extra_params=>{:counter >>>> => counter + 1 + @response.start.to_i } %> >>>> >>>> Now I get the following when I try to access the index page of the >>>> catalog >>>> >>>> Showing app/views/catalog/_document_list.html.erb where line #21 >>>> raised: >>>> >>>> catalog_url failed to generate from >>>> {:commit=>nil, :action=>"show", >>>> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", >>>> :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, >>>> diff: >>>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} >>>> >>>> Extracted source (around line #21): >>>> >>>> 18: <% # main title container for doc partial view -%> >>>> 19: >>>> 20:
>>>> 21:
<%= counter + 1 + @response.start.to_i %>. <%= >>>> link_to_document >>>> document, :label=>:title, :extra_params=>{:counter => >>>> counter + 1 + @response.start.to_i } %>
>>>> 22:
>>>> 23: >>>> 24: >>>> >>>> What am I screwing up this time? (LOL) >>>> Thnks in advance folks. >>>> >>>> >>>> >>>> Jessie Keck wrote: >>>> >>>> Hi Vern, >>>> Just chiming in about overriding the interface. >>>> >>>> If you find a view that you would like to override then you >>>> just need to copy that file from the plugin and paste it in >>>> your top level application (the one generated with =>> rails >>>> {your_app_name}) making sure to keep the same directory >>>> structure as the plugin. >>>> >>>> The engines plugin will look at your top level app first to >>>> find a file then to the blacklight plugin. An example of this >>>> is if you wanted the search box to be available on all pages >>>> not just search results. This is an easy way to do that: >>>> 1) Copy file views/catalog/index.html.erb from plugin to the >>>> top level app >>>> 2) Remove render tag from the index view in the top level app >>>> 3) Copy file views/catalog/_search_form.html erb to the top >>>> level app to the views/shared directory (note that I did not >>>> maintain the directory structure because the search form >>>> partial is now shared amongst other controllers) >>>> 4) Copy file views/layouts/application.html.erb to the top >>>> level app >>>> 5) Add the render tag to application view to now be render >>>> :partial => 'shared/search_form' >>>> (Note: putting this is the application view is probably not >>>> the best idea, but this is just an example. I'm actually >>>> rendering the search form partial from a header partial, not >>>> the application view) >>>> >>>> That's about it. You can go nuts with overriding all kinds of >>>> files. Do remember that certain things may require a restart >>>> to take affect (most notably changes to helpers) >>>> >>>> Hope this helps, >>>> Let us know if any of this is unclear or you run into any >>>> snags. >>>> >>>> -Jessie Keck >>>> Stanford University Libraries >>>> >>>> Vernon Chapman wrote: >>>> >>>> Is there some documentation on how to customize >>>> blacklight? >>>> I have a lot or Dublin Core(ish) metadata in a solr index >>>> and would like >>>> to use blacklight as a front end to it. >>>> >>>> Thanks in advance >>>> >>>> Vern >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> >>>> >>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From mccahill at duke.edu Wed Mar 11 10:47:51 2009 From: mccahill at duke.edu (Mark P. McCahill) Date: Wed, 11 Mar 2009 10:47:51 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> Message-ID: I had exactly the same problem and the fix worked for me. I made sure that the fix was applied by editing the file in question and then restarting mongrel. On Mar 11, 2009, at 10:35 AM, Molly Pickral Cadieux wrote: > Vern, > > Just to make sure -- did you restart passenger or mongrel or whatever > it is you are using? Changes to helpers seem to need an extra kick. > > Molly > > On Tue, Mar 10, 2009 at 2:51 PM, Vernon Chapman > wrote: >> Molly, >> >> Thanks for your work on this, but unfortunately that didn't fix the >> issue. I did an svn update on my checkout of blacklight and >> restarted >> apache and I still get the same error. >> >> Vern >> >> Molly Pickral Cadieux wrote: >>> >>> Hi again, >>> >>> I went ahead and tried the fix on demo.blacklightopac.org, and it >>> works great! So, Vernon, if you could grab the new version of >>> catalog_helper.rb from subversion, I'm almost certain that will fix >>> the bug you reported yesterday. >>> >>> Molly >>> >>> On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux >>> wrote: >>> >>>> >>>> Hi everyone, >>>> >>>> I recently committed some changes to the repository that offer the >>>> following features: >>>> >>>> 1. A drop-down list that allows users to specify how may results >>>> per >>>> page to display (the previous default was 10 per page) >>>> 2. "Previous" and "Next" links that allow a user to page through a >>>> search result, as opposed to going back to the previous page to >>>> select >>>> the previous or next item. >>>> 3. Several rspec tests for the catalog controller and various >>>> models. >>>> >>>> This all works perfectly on my setup, but when I went to deploy the >>>> changes on demo.blacklightopac.org, I ran into a routing issue. >>>> Vernon also reported this error yesterday. The error is as >>>> follows: >>>> >>>> ------------ >>>> >>>> catalog_url failed to generate from {:commit=>nil, :action=>"show", >>>> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", >>>> :counter=>1}, expected: >>>> {:action=>"show", :controller=>"catalog"}, diff: >>>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} >>>> >>>> Extracted source (around line #21): >>>> >>>> 18: <% # main title container for doc partial view -%> >>>> 19: >>>> 20:
>>>> 21:
<%= counter + 1 + @response.start.to_i %>. <%= >>>> link_to_document >>>> document, :label=>:title, :extra_params=>{:counter => >>>> counter + 1 + @response.start.to_i } %>
>>>> 22:
>>>> 23: >>>> 24: >>>> >>>> ----------- >>>> >>>> Bess had reported a similar problem back in February (I've included >>>> the message below). Unfortunately, I have been unable to reproduce >>>> the reported problem on my Mac running the same versions of Ruby, >>>> Rails, and Passenger. I would like to try Bess's recommended >>>> solution >>>> if I can get a setup where I can reproduce the problem. In the >>>> mean >>>> time, I'd love it if Vernon or others who may have seen the problem >>>> could try modifying this file: >>>> >>>> rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb >>>> >>>> and change this line: >>>> >>>> link_to label, catalog_path(doc[:id], >>>> params >>>> .merge(opts[:extra_params]).merge(:action=>nil, :commit=>nil)) >>>> >>>> to be: >>>> >>>> link_to label, catalog_path(doc[:id], >>>> params.merge(opts[:extra_params]).merge(:action=>"show", >>>> :commit=>nil)) >>>> >>>> and see if that fixes it. A restart of passenger will be >>>> necessary. >>>> >>>> If I can't get this worked out soon, I will revert the changes in >>>> the >>>> repository and work through this again here. >>>> >>>> In any case, I wanted to let you all know that what is currently in >>>> the repository may cause routing issues under certain environments. >>>> I'll keep you all posted on progress. >>>> >>>> Molly >>>> >>>> >>>> ---------- >>>> >>>> From eos8d at virginia.edu Sun Feb 1 17:25:35 2009 >>>> From: eos8d at virginia.edu (Bess Sadler) >>>> Date: Sun, 1 Feb 2009 17:25:35 -0500 >>>> Subject: [Blacklight-development] demo app bug fix >>>> Message-ID: >>>> >>>> Hey, folks. >>>> >>>> I've spent the last day or so playing pretty intensively with the >>>> demo >>>> app. I fixed a couple of minor bugs in the installation >>>> instructions >>>> (here: >>>> http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) >>>> and I've deployed the demo app several times, on several different >>>> systems. Strangely, although I could get it running right away with >>>> mongrel or webrick, I found that I could not get it running with >>>> mod_passenger, which is our preferred method of running rails >>>> apps in >>>> production. I think I tracked down the bug today, and you can see >>>> the >>>> demo app running with mod_passenger as a virtual host at >>>> http://demo.blacklightopac.org >>>> >>>> I was getting two errors. One, encountered whenever the index >>>> screen >>>> was loaded, said: >>>> >>>> ActionView::TemplateError (catalog_url failed to generate from >>>> {:action=>"index", :controller=>"catalog", :id=>"u1"}, expected: >>>> {:action=>"show", :controller=>"catalog"}, diff: >>>> {:action=>"show", :id=>"u1"}) on line #11 of vendor/plugins/ >>>> blacklight/ >>>> app/views/catalog/_index_partials/_default.html.erb: >>>> 8: >>>> 9:
>>>> 10:
>>>> 11: <%= link_to document.get(:id), >>>> catalog_path(document[:id], >>>> params) %> >>>> 12:
>>>> 13:
>>>> 14: >>>> >>>> >>>> The other was generated whenever the show screen was loaded, and it >>>> said: >>>> >>>> ActionView::TemplateError (catalog_index_url failed to generate >>>> from >>>> {:action=>"show", :controller=>"catalog", :id=>nil}, expected: >>>> {:action=>"index", :controller=>"catalog"}, diff: >>>> {:action=>"index", :id=>nil}) on line #8 of vendor/plugins/ >>>> blacklight/ >>>> app/views/catalog/show.html.erb: >>>> 5: >>>> 6: >>>> 7: <% @sidebar = capture do %> >>>> 8:

<%= link_to '<< SEARCH', >>>> catalog_index_path(params.merge(:id=>nil)) %>

>>>> 9: <% end %> >>>> >>>> Turns out these were basically the same error. In both cases, we're >>>> using the link_to function to create a link to another part of the >>>> application, in this case from index to show, and from show to >>>> index. >>>> Let's talk about the link from index to show as an example. >>>> >>>> <%= link_to document.get(:id), catalog_path(document[:id], >>>> params) %> >>>> >>>> That says create a link, and the text of the link is the id >>>> number of >>>> the current document (say, u999), and the target of the link is >>>> catalog_path(u999) which will evaluate to /catalog/u999. >>>> Incidentally, >>>> the rails console was really valuable in helping me understand how >>>> these worked, and you can play with it like this: >>>> >>>> paz:rails eos8d$ ./script/console >>>> Loading development environment (Rails 2.2.2) >>>> >> app.catalog_path("u999") >>>> => "/catalog/u999" >>>> >>>> Anyway, I found that if I just changed the link_to statement to >>>> look >>>> like this instead, removing the params object: >>>> >>>> <%= link_to document.get(:id), catalog_path(document[:id]) %> >>>> >>>> the error went away. However, this wasn't an acceptable solution, >>>> because whenever a user performs a search or selects a facet, that >>>> value goes in the params hash, and you want to keep it around >>>> when you >>>> go from screen to screen, so that the user isn't always having to >>>> re- >>>> enter their current search. Let's say you're on the index screen, >>>> and >>>> you've entered a search and selected a facet. At that point your >>>> params hash looks like this: >>>> >>>> {"commit"=>"search", "action"=>"index", "q"=>"poetry", >>>> "f"=>{"language_facet"=>["English"]}, "controller"=>"catalog"} >>>> >>>> When you want to look at a detailed view for a specific item, >>>> you're >>>> linking to the show action of your same catalog controller, and >>>> passing it a specific document id. But you also want to hang onto >>>> the >>>> fact that you've searched for poetry and limited to >>>> language_facet:English, so you also pass the params object in. At >>>> which point your link_to really looks like this: >>>> >>>> <%= link_to document.get(:id), >>>> catalog_path >>>> (:id >>>> = >>>> > >>>> document >>>> [:id], :commit=>"search", :action=>"index", :q=>"poetry" ... >>>> etc ... ) %> >>>> >>>> Do you see the problem? It took me awhile to notice it. I've just >>>> called the show action, and then I also pass in a parameter of >>>> action=index. It's not obvious, because catalog_path sort of >>>> obscures >>>> the fact that you're calling the show action. Interestingly, >>>> webrick >>>> and mongrel don't seem to care. I guess they just assume that even >>>> though you've passed two values for action, you must really mean >>>> action=show because that's what's defined for catalog_path. The >>>> fix is >>>> easy, once you know what the problem is: >>>> >>>> <%= link_to document.get(:id), >>>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >>>> >>>> Now you're passing in a params object that looks like this: >>>> >>>> {"commit"=>"search", "action"=>"show", "id"=>"2008349843", >>>> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", >>>> "controller"=>"catalog"} >>>> >>>> So, all your user defined parameters, plus the id number of the >>>> specific item you want to show, plus action=show, which is what >>>> catalog_path expects. Problem solved. >>>> >>>> Sorry this is so long, but I learned a lot in fixing this bug and I >>>> thought it might be helpful for someone else too. But if it just >>>> looks >>>> like gibberish, ignore it. >>>> >>>> Cheers, >>>> Bess >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> From goodieboy at gmail.com Sun Feb 1 17:59:36 2009 >>>> From: goodieboy at gmail.com (Matt Mitchell) >>>> Date: Sun, 1 Feb 2009 17:59:36 -0500 >>>> Subject: [Blacklight-development] demo app bug fix >>>> In-Reply-To: >>>> References: >>>> Message-ID: >>> > >>>> >>>> Seriously, that's a kickin' write up Bess. Great bug catch. And >>>> the docs >>>> look so nice too! >>>> >>>> For the link_to fix here: >>>> >>>> <%= link_to document.get(:id), >>>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >>>> >>>> You should be able to do this too: >>>> >>>> <%= link_to document.get(:id), catalog_path(document[:id], >>>> params.merge(:action=>nil)) %> >>>> >>>> I think this kind of thing will become a url-param-linking >>>> pattern in BL >>>> and >>>> might do well in a view helper. I think there's one in the UVa >>>> trunk, >>>> called >>>> link_to_document or something. We could then have one called >>>> link_to_documents or link_back_to_search? The upshot of it being >>>> in a >>>> helper >>>> is that if someone overrides a view partial, they don't have to >>>> re-write >>>> the >>>> param merging stuff. Food for thought... >>>> >>>> Matt >>>> >>>> >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From jamie at dang.com Wed Mar 11 11:18:05 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Wed, 11 Mar 2009 11:18:05 -0400 Subject: [Blacklight-development] git info In-Reply-To: <58A2008A-5C86-4559-BC72-88C2764BA436@stanford.edu> References: <58A2008A-5C86-4559-BC72-88C2764BA436@stanford.edu> Message-ID: <0C38CE09-0A73-4093-B7D4-968924E1B6F6@dang.com> There's an awesome Textmate bundle that I use daily: http://blog.macromates.com/2008/git-bundle/ As for documentation, every time I have been stuck, I google for git and the command name and I'm taken straight to what I need. Documentation hasn't been a problem. I usually end up at http://www.kernel.org/pub/software/scm/git/docs/ for example, if I google for "git status" I'll end up at http://www.kernel.org/pub/software/scm/git/docs/git-status.html There are some great resources: http://github.com/guides/home http://git-scm.com/ I don't use eclipse, so don't have info on that. Here's a list of git front ends and tools: http://git.or.cz/gitwiki/InterfacesFrontendsAndTools Jamie On Mar 6, 2009, at 2:31 PM, Naomi Dushay wrote: > Back to Git: I have also heard that the documentation is not > great. Also, does it integrate well with our tools? I use Eclipse > and Textmate, personally. > > I'm willing to try git; I've heard great things about it. But are > the tradeoffs worth it? From jamie at dang.com Wed Mar 11 12:29:20 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Wed, 11 Mar 2009 12:29:20 -0400 Subject: [Blacklight-development] rsolr version? Message-ID: <4BC89DC5-AC13-4E1E-9416-BB03CA88B33D@dang.com> Is this still true? required_rsolr_version = '0.6.9' I installed mwmitchell-rsolr and its version is 0.7.1 Jamie From chapman.lists at gmail.com Wed Mar 11 12:40:22 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Wed, 11 Mar 2009 12:40:22 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> Message-ID: <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> I am not sure what I am doing wrong so here is what I did 1 - checkout blacklight from svn using svn co http://....../trunk blacklight 2 - followed the direction to setup internal solr/jetty server and index records 3 - test the setup, everything works when I got to http://localhost/ i see the new home page with text etc. Here is where I am getting KILLED I need to change the solr url , handlers and the controller for my application so I 1 - modified solr.yml from http://127.0.0.1:8983/solr to http://127.0.0.1:8983/solr/content (my solr is multicore) 2 - copied the home_controller & catalog_controller to my app/controllers 3 - modified the catalog_controllers by replacing the :search symbol with the name of the handler on my solr server in this case :select 4 - copied the vendors/plugins/blacklight/lib/blacklight.rb to my app/lib/blacklight.rb 5 - added Blacklight.solr.param_mappers[:select] = :dismax to app/lib/blacklight.rb 6 - restart passtenger ( touch rails/tmp/restart.txt) 7 - restart apache2 (just to make sure passenger restarted) The fist time I access http://localhost/ I see the home page with the facets from my external solr server listed on the left, if I click on one of the facets, I get the following : You have a nil object when you didn't expect it! The error occurred while evaluating nil.commit /opt/appz/blacklight/rails/app/controllers/home_controller.rb:15:in `solr_params' /opt/appz/blacklight/rails/app/controllers/home_controller.rb:7:in `index' the line with .commit is the Blacklight.commit line in solr_params, so for some reason Blacklight is nil. Any ideas On Wed, Mar 11, 2009 at 10:35 AM, Molly Pickral Cadieux wrote: > Vern, > > Just to make sure -- did you restart passenger or mongrel or whatever > it is you are using? Changes to helpers seem to need an extra kick. > > Molly > > On Tue, Mar 10, 2009 at 2:51 PM, Vernon Chapman > wrote: > > Molly, > > > > Thanks for your work on this, but unfortunately that didn't fix the > > issue. I did an svn update on my checkout of blacklight and restarted > > apache and I still get the same error. > > > > Vern > > > > Molly Pickral Cadieux wrote: > >> > >> Hi again, > >> > >> I went ahead and tried the fix on demo.blacklightopac.org, and it > >> works great! So, Vernon, if you could grab the new version of > >> catalog_helper.rb from subversion, I'm almost certain that will fix > >> the bug you reported yesterday. > >> > >> Molly > >> > >> On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux > >> wrote: > >> > >>> > >>> Hi everyone, > >>> > >>> I recently committed some changes to the repository that offer the > >>> following features: > >>> > >>> 1. A drop-down list that allows users to specify how may results per > >>> page to display (the previous default was 10 per page) > >>> 2. "Previous" and "Next" links that allow a user to page through a > >>> search result, as opposed to going back to the previous page to select > >>> the previous or next item. > >>> 3. Several rspec tests for the catalog controller and various models. > >>> > >>> This all works perfectly on my setup, but when I went to deploy the > >>> changes on demo.blacklightopac.org, I ran into a routing issue. > >>> Vernon also reported this error yesterday. The error is as follows: > >>> > >>> ------------ > >>> > >>> catalog_url failed to generate from {:commit=>nil, :action=>"show", > >>> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", > >>> :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, > diff: > >>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} > >>> > >>> Extracted source (around line #21): > >>> > >>> 18: <% # main title container for doc partial view -%> > >>> 19: > >>> 20:
> >>> 21:
<%= counter + 1 + @response.start.to_i %>. <%= > >>> link_to_document document, :label=>:title, :extra_params=>{:counter => > >>> counter + 1 + @response.start.to_i } %>
> >>> 22:
> >>> 23: > >>> 24: > >>> > >>> ----------- > >>> > >>> Bess had reported a similar problem back in February (I've included > >>> the message below). Unfortunately, I have been unable to reproduce > >>> the reported problem on my Mac running the same versions of Ruby, > >>> Rails, and Passenger. I would like to try Bess's recommended solution > >>> if I can get a setup where I can reproduce the problem. In the mean > >>> time, I'd love it if Vernon or others who may have seen the problem > >>> could try modifying this file: > >>> > >>> rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb > >>> > >>> and change this line: > >>> > >>> link_to label, catalog_path(doc[:id], > >>> params.merge(opts[:extra_params]).merge(:action=>nil, :commit=>nil)) > >>> > >>> to be: > >>> > >>> link_to label, catalog_path(doc[:id], > >>> params.merge(opts[:extra_params]).merge(:action=>"show", > >>> :commit=>nil)) > >>> > >>> and see if that fixes it. A restart of passenger will be necessary. > >>> > >>> If I can't get this worked out soon, I will revert the changes in the > >>> repository and work through this again here. > >>> > >>> In any case, I wanted to let you all know that what is currently in > >>> the repository may cause routing issues under certain environments. > >>> I'll keep you all posted on progress. > >>> > >>> Molly > >>> > >>> > >>> ---------- > >>> > >>> From eos8d at virginia.edu Sun Feb 1 17:25:35 2009 > >>> From: eos8d at virginia.edu (Bess Sadler) > >>> Date: Sun, 1 Feb 2009 17:25:35 -0500 > >>> Subject: [Blacklight-development] demo app bug fix > >>> Message-ID: > >>> > >>> Hey, folks. > >>> > >>> I've spent the last day or so playing pretty intensively with the demo > >>> app. I fixed a couple of minor bugs in the installation instructions > >>> (here: > >>> http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) > >>> and I've deployed the demo app several times, on several different > >>> systems. Strangely, although I could get it running right away with > >>> mongrel or webrick, I found that I could not get it running with > >>> mod_passenger, which is our preferred method of running rails apps in > >>> production. I think I tracked down the bug today, and you can see the > >>> demo app running with mod_passenger as a virtual host at > >>> http://demo.blacklightopac.org > >>> > >>> I was getting two errors. One, encountered whenever the index screen > >>> was loaded, said: > >>> > >>> ActionView::TemplateError (catalog_url failed to generate from > >>> {:action=>"index", :controller=>"catalog", :id=>"u1"}, expected: > >>> {:action=>"show", :controller=>"catalog"}, diff: > >>> {:action=>"show", :id=>"u1"}) on line #11 of vendor/plugins/blacklight/ > >>> app/views/catalog/_index_partials/_default.html.erb: > >>> 8: > >>> 9:
> >>> 10:
> >>> 11: <%= link_to document.get(:id), catalog_path(document[:id], > >>> params) %> > >>> 12:
> >>> 13:
> >>> 14: > >>> > >>> > >>> The other was generated whenever the show screen was loaded, and it > >>> said: > >>> > >>> ActionView::TemplateError (catalog_index_url failed to generate from > >>> {:action=>"show", :controller=>"catalog", :id=>nil}, expected: > >>> {:action=>"index", :controller=>"catalog"}, diff: > >>> {:action=>"index", :id=>nil}) on line #8 of vendor/plugins/blacklight/ > >>> app/views/catalog/show.html.erb: > >>> 5: > >>> 6: > >>> 7: <% @sidebar = capture do %> > >>> 8:

<%= link_to '<< SEARCH', > >>> catalog_index_path(params.merge(:id=>nil)) %>

> >>> 9: <% end %> > >>> > >>> Turns out these were basically the same error. In both cases, we're > >>> using the link_to function to create a link to another part of the > >>> application, in this case from index to show, and from show to index. > >>> Let's talk about the link from index to show as an example. > >>> > >>> <%= link_to document.get(:id), catalog_path(document[:id], params) %> > >>> > >>> That says create a link, and the text of the link is the id number of > >>> the current document (say, u999), and the target of the link is > >>> catalog_path(u999) which will evaluate to /catalog/u999. Incidentally, > >>> the rails console was really valuable in helping me understand how > >>> these worked, and you can play with it like this: > >>> > >>> paz:rails eos8d$ ./script/console > >>> Loading development environment (Rails 2.2.2) > >>> >> app.catalog_path("u999") > >>> => "/catalog/u999" > >>> > >>> Anyway, I found that if I just changed the link_to statement to look > >>> like this instead, removing the params object: > >>> > >>> <%= link_to document.get(:id), catalog_path(document[:id]) %> > >>> > >>> the error went away. However, this wasn't an acceptable solution, > >>> because whenever a user performs a search or selects a facet, that > >>> value goes in the params hash, and you want to keep it around when you > >>> go from screen to screen, so that the user isn't always having to re- > >>> enter their current search. Let's say you're on the index screen, and > >>> you've entered a search and selected a facet. At that point your > >>> params hash looks like this: > >>> > >>> {"commit"=>"search", "action"=>"index", "q"=>"poetry", > >>> "f"=>{"language_facet"=>["English"]}, "controller"=>"catalog"} > >>> > >>> When you want to look at a detailed view for a specific item, you're > >>> linking to the show action of your same catalog controller, and > >>> passing it a specific document id. But you also want to hang onto the > >>> fact that you've searched for poetry and limited to > >>> language_facet:English, so you also pass the params object in. At > >>> which point your link_to really looks like this: > >>> > >>> <%= link_to document.get(:id), > >>> catalog_path > >>> (:id > >>> =>document[:id], :commit=>"search", :action=>"index", :q=>"poetry" ... > >>> etc ... ) %> > >>> > >>> Do you see the problem? It took me awhile to notice it. I've just > >>> called the show action, and then I also pass in a parameter of > >>> action=index. It's not obvious, because catalog_path sort of obscures > >>> the fact that you're calling the show action. Interestingly, webrick > >>> and mongrel don't seem to care. I guess they just assume that even > >>> though you've passed two values for action, you must really mean > >>> action=show because that's what's defined for catalog_path. The fix is > >>> easy, once you know what the problem is: > >>> > >>> <%= link_to document.get(:id), > >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> > >>> > >>> Now you're passing in a params object that looks like this: > >>> > >>> {"commit"=>"search", "action"=>"show", "id"=>"2008349843", > >>> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", > >>> "controller"=>"catalog"} > >>> > >>> So, all your user defined parameters, plus the id number of the > >>> specific item you want to show, plus action=show, which is what > >>> catalog_path expects. Problem solved. > >>> > >>> Sorry this is so long, but I learned a lot in fixing this bug and I > >>> thought it might be helpful for someone else too. But if it just looks > >>> like gibberish, ignore it. > >>> > >>> Cheers, > >>> Bess > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> From goodieboy at gmail.com Sun Feb 1 17:59:36 2009 > >>> From: goodieboy at gmail.com (Matt Mitchell) > >>> Date: Sun, 1 Feb 2009 17:59:36 -0500 > >>> Subject: [Blacklight-development] demo app bug fix > >>> In-Reply-To: > >>> References: > >>> Message-ID: < > c66dcac40902011459y279c7187yd46effb14248da9a at mail.gmail.com> > >>> > >>> Seriously, that's a kickin' write up Bess. Great bug catch. And the > docs > >>> look so nice too! > >>> > >>> For the link_to fix here: > >>> > >>> <%= link_to document.get(:id), > >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> > >>> > >>> You should be able to do this too: > >>> > >>> <%= link_to document.get(:id), catalog_path(document[:id], > >>> params.merge(:action=>nil)) %> > >>> > >>> I think this kind of thing will become a url-param-linking pattern in > BL > >>> and > >>> might do well in a view helper. I think there's one in the UVa trunk, > >>> called > >>> link_to_document or something. We could then have one called > >>> link_to_documents or link_back_to_search? The upshot of it being in a > >>> helper > >>> is that if someone overrides a view partial, they don't have to > re-write > >>> the > >>> param merging stuff. Food for thought... > >>> > >>> Matt > >>> > >>> > >> > >> _______________________________________________ > >> Blacklight-development mailing list > >> Blacklight-development at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/blacklight-development > >> Blacklightopac Blog http://blacklightopac.org/ > >> > >> > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chapman.lists at gmail.com Wed Mar 11 12:44:31 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Wed, 11 Mar 2009 12:44:31 -0400 Subject: [Blacklight-development] Customizing Blacklight In-Reply-To: References: <49AEFB67.7080008@gmail.com> <49AF2FD3.1020102@stanford.edu> <49B561DE.6020109@gmail.com> Message-ID: <7c20cede0903110944v48784aax75468064dea23e2d@mail.gmail.com> Matt, This issue has not been resolved because there are other issues that are keeping me from even getting to that point. I am ok with replacing he "." in my ids with say "@" or the like and just putting them back when searching by id i.e. a show action. Thanks Vern On Tue, Mar 10, 2009 at 8:09 PM, Matt Mitchell wrote: > Vernon, > > Not sure if this has been solved or not? Are you saying the only thing you > changed when this error occurred is the value of :label ? If so, that's very > puzzling. > > The only thing that really stands out to me about that error message is > that the document.id is "agrecord.2008-11-27.1267669533". I'm sure that > the default id param pattern for a show route is something that does not > have a period. In fact, an id with a period in it could be confusing when it > comes to specifying content-types like /documents/1234.json for example. I'd > recommend not using periods in your document ids if at all possible. > > Matt > > On Mon, Mar 9, 2009 at 2:37 PM, Vernon Chapman wrote: > >> I am trying to follow Jessie's directions (below) to customize the demo >> app for my own purposes. >> I copied the view (_document_list.html.erb) for catalog index into the >> controlling app's view/catalog directory. >> >> My problem seems is that when the link_to_document helper is called. The >> line reads >> <%= link_to_document document, :label=>:title_t, >> :extra_params=>{:counter => counter + 1 + @response.start.to_i } %> >> >> .My title field is named well title and not title_t so I changed the >> controling app's _document_list.html.erb to >> <%= link_to_document document, :label=>:title, :extra_params=>{:counter >> => counter + 1 + @response.start.to_i } %> >> >> Now I get the following when I try to access the index page of the catalog >> >> Showing app/views/catalog/_document_list.html.erb where line #21 raised: >> >> catalog_url failed to generate from {:commit=>nil, :action=>"show", >> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", >> :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, diff: >> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} >> >> Extracted source (around line #21): >> >> 18: <% # main title container for doc partial view -%> >> 19: >> 20:
>> 21:
<%= counter + 1 + @response.start.to_i %>. <%= >> link_to_document document, :label=>:title, :extra_params=>{:counter => >> counter + 1 + @response.start.to_i } %>
>> 22:
>> 23: >> 24: >> >> What am I screwing up this time? (LOL) >> Thnks in advance folks. >> >> >> >> Jessie Keck wrote: >> >>> Hi Vern, >>> Just chiming in about overriding the interface. >>> >>> If you find a view that you would like to override then you just need to >>> copy that file from the plugin and paste it in your top level application >>> (the one generated with =>> rails {your_app_name}) making sure to keep the >>> same directory structure as the plugin. >>> >>> The engines plugin will look at your top level app first to find a file >>> then to the blacklight plugin. An example of this is if you wanted the >>> search box to be available on all pages not just search results. This is an >>> easy way to do that: >>> 1) Copy file views/catalog/index.html.erb from plugin to the top level >>> app >>> 2) Remove render tag from the index view in the top level app >>> 3) Copy file views/catalog/_search_form.html erb to the top level app to >>> the views/shared directory (note that I did not maintain the directory >>> structure because the search form partial is now shared amongst other >>> controllers) >>> 4) Copy file views/layouts/application.html.erb to the top level app >>> 5) Add the render tag to application view to now be render :partial => >>> 'shared/search_form' >>> (Note: putting this is the application view is probably not the best >>> idea, but this is just an example. I'm actually rendering the search form >>> partial from a header partial, not the application view) >>> >>> That's about it. You can go nuts with overriding all kinds of files. Do >>> remember that certain things may require a restart to take affect (most >>> notably changes to helpers) >>> >>> Hope this helps, >>> Let us know if any of this is unclear or you run into any snags. >>> >>> -Jessie Keck >>> Stanford University Libraries >>> >>> Vernon Chapman wrote: >>> >>>> Is there some documentation on how to customize blacklight? >>>> I have a lot or Dublin Core(ish) metadata in a solr index and would like >>>> to use blacklight as a front end to it. >>>> >>>> Thanks in advance >>>> >>>> Vern >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >>> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ndushay at stanford.edu Wed Mar 11 12:48:12 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Wed, 11 Mar 2009 09:48:12 -0700 Subject: [Blacklight-development] solr query for single document: standard or dismax? In-Reply-To: <419F32C3-02C8-4F20-B957-D6E2AB5DFF19@mac.com> References: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> <9BB431CA-C6C0-4587-BB1B-3A5C73629DF4@stanford.edu> <419F32C3-02C8-4F20-B957-D6E2AB5DFF19@mac.com> Message-ID: <42F023C7-A62F-48B7-B484-5E751A437CE9@stanford.edu> >> Or, to gain this flexibility, would it be prudent for blacklight >> to use the solr-ruby gem instead of the mwmitchell-solr gem? > > I'm working with Matt to get his ideas/implementation into a solr- > ruby refactoring, so in the not too distant future we won't have > this discrepancy... we'll all happily be using solr-ruby 1.0 :) +1 I might hold off on big changes to the Gems and/or Gem override functionality until this is ready; I'm too much of a RoR noob. But my refactoring will make it easier. One step at a time. - Naomi From ndushay at stanford.edu Wed Mar 11 12:55:22 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Wed, 11 Mar 2009 09:55:22 -0700 Subject: [Blacklight-development] routing problem In-Reply-To: <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> Message-ID: <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> Vernon, Just for giggles, try this: > 4. copied the vendors/plugins/blacklight/lib/blacklight.rb to my > app/lib/blacklight.rb don't do this step > 5 - added Blacklight.solr.param_mappers[:select] = :dismax to app/ > lib/blacklight.rb do this in place. I had trouble with my solr_helper refactoring when I needed to add and read add'l info in solr.yml; I couldn't get it to work when I did it in my local rails app -- I had to do it in vendor/plugins/ blacklight/lib If my "fix" works, then we need to discuss if this can be done differently and/or change the documentation accordingly. - Naomi On Mar 11, 2009, at 9:40 AM, Vernon Chapman wrote: > I am not sure what I am doing wrong so here is what I did > > 1 - checkout blacklight from svn using svn co http://....../trunk > blacklight > 2 - followed the direction to setup internal solr/jetty server and > index records > 3 - test the setup, everything works when I got to http://localhost/ > i see the new home page with text etc. > > Here is where I am getting KILLED > > I need to change the solr url , handlers and the controller for my > application so I > > 1 - modified solr.yml from http://127.0.0.1:8983/solr to http://127.0.0.1:8983/solr/content > (my solr is multicore) > 2 - copied the home_controller & catalog_controller to my app/ > controllers > 3 - modified the catalog_controllers by replacing the :search symbol > with the name of the handler > on my solr server in this case :select > 4 - copied the vendors/plugins/blacklight/lib/blacklight.rb to my > app/lib/blacklight.rb > 5 - added Blacklight.solr.param_mappers[:select] = :dismax to app/ > lib/blacklight.rb > 6 - restart passtenger ( touch rails/tmp/restart.txt) > 7 - restart apache2 (just to make sure passenger restarted) > > The fist time I access http://localhost/ I see the home page with > the facets from my external > solr server listed on the left, if I click on one of the facets, I > get the following : > > You have a nil object when you didn't expect it! > The error occurred while evaluating nil.commit > > > /opt/appz/blacklight/rails/app/controllers/home_controller.rb:15:in > `solr_params' > > > /opt/appz/blacklight/rails/app/controllers/home_controller.rb:7:in > `index' > > > the line with .commit is the Blacklight.commit line in solr_params, > so for some reason Blacklight > is nil. > > Any ideas > On Wed, Mar 11, 2009 at 10:35 AM, Molly Pickral Cadieux > wrote: > Vern, > > Just to make sure -- did you restart passenger or mongrel or whatever > it is you are using? Changes to helpers seem to need an extra kick. > > Molly > > On Tue, Mar 10, 2009 at 2:51 PM, Vernon Chapman > wrote: > > Molly, > > > > Thanks for your work on this, but unfortunately that didn't fix the > > issue. I did an svn update on my checkout of blacklight and > restarted > > apache and I still get the same error. > > > > Vern > > > > Molly Pickral Cadieux wrote: > >> > >> Hi again, > >> > >> I went ahead and tried the fix on demo.blacklightopac.org, and it > >> works great! So, Vernon, if you could grab the new version of > >> catalog_helper.rb from subversion, I'm almost certain that will fix > >> the bug you reported yesterday. > >> > >> Molly > >> > >> On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux > >> wrote: > >> > >>> > >>> Hi everyone, > >>> > >>> I recently committed some changes to the repository that offer the > >>> following features: > >>> > >>> 1. A drop-down list that allows users to specify how may > results per > >>> page to display (the previous default was 10 per page) > >>> 2. "Previous" and "Next" links that allow a user to page > through a > >>> search result, as opposed to going back to the previous page to > select > >>> the previous or next item. > >>> 3. Several rspec tests for the catalog controller and various > models. > >>> > >>> This all works perfectly on my setup, but when I went to deploy > the > >>> changes on demo.blacklightopac.org, I ran into a routing issue. > >>> Vernon also reported this error yesterday. The error is as > follows: > >>> > >>> ------------ > >>> > >>> catalog_url failed to generate from > {:commit=>nil, :action=>"show", > >>> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", > >>> :counter=>1}, expected: > {:action=>"show", :controller=>"catalog"}, diff: > >>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} > >>> > >>> Extracted source (around line #21): > >>> > >>> 18: <% # main title container for doc partial view -%> > >>> 19: > >>> 20:
> >>> 21:
<%= counter + 1 + @response.start.to_i %>. <%= > >>> link_to_document > document, :label=>:title, :extra_params=>{:counter => > >>> counter + 1 + @response.start.to_i } %>
> >>> 22:
> >>> 23: > >>> 24: > >>> > >>> ----------- > >>> > >>> Bess had reported a similar problem back in February (I've > included > >>> the message below). Unfortunately, I have been unable to > reproduce > >>> the reported problem on my Mac running the same versions of Ruby, > >>> Rails, and Passenger. I would like to try Bess's recommended > solution > >>> if I can get a setup where I can reproduce the problem. In the > mean > >>> time, I'd love it if Vernon or others who may have seen the > problem > >>> could try modifying this file: > >>> > >>> rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb > >>> > >>> and change this line: > >>> > >>> link_to label, catalog_path(doc[:id], > >>> > params.merge(opts[:extra_params]).merge(:action=>nil, :commit=>nil)) > >>> > >>> to be: > >>> > >>> link_to label, catalog_path(doc[:id], > >>> params.merge(opts[:extra_params]).merge(:action=>"show", > >>> :commit=>nil)) > >>> > >>> and see if that fixes it. A restart of passenger will be > necessary. > >>> > >>> If I can't get this worked out soon, I will revert the changes > in the > >>> repository and work through this again here. > >>> > >>> In any case, I wanted to let you all know that what is currently > in > >>> the repository may cause routing issues under certain > environments. > >>> I'll keep you all posted on progress. > >>> > >>> Molly > >>> > >>> > >>> ---------- > >>> > >>> From eos8d at virginia.edu Sun Feb 1 17:25:35 2009 > >>> From: eos8d at virginia.edu (Bess Sadler) > >>> Date: Sun, 1 Feb 2009 17:25:35 -0500 > >>> Subject: [Blacklight-development] demo app bug fix > >>> Message-ID: > >>> > >>> Hey, folks. > >>> > >>> I've spent the last day or so playing pretty intensively with > the demo > >>> app. I fixed a couple of minor bugs in the installation > instructions > >>> (here: > >>> http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) > >>> and I've deployed the demo app several times, on several > different > >>> systems. Strangely, although I could get it running right away > with > >>> mongrel or webrick, I found that I could not get it running with > >>> mod_passenger, which is our preferred method of running rails > apps in > >>> production. I think I tracked down the bug today, and you can > see the > >>> demo app running with mod_passenger as a virtual host at > >>> http://demo.blacklightopac.org > >>> > >>> I was getting two errors. One, encountered whenever the index > screen > >>> was loaded, said: > >>> > >>> ActionView::TemplateError (catalog_url failed to generate from > >>> {:action=>"index", :controller=>"catalog", :id=>"u1"}, expected: > >>> {:action=>"show", :controller=>"catalog"}, diff: > >>> {:action=>"show", :id=>"u1"}) on line #11 of vendor/plugins/ > blacklight/ > >>> app/views/catalog/_index_partials/_default.html.erb: > >>> 8: > >>> 9:
> >>> 10:
> >>> 11: <%= link_to document.get(:id), > catalog_path(document[:id], > >>> params) %> > >>> 12:
> >>> 13:
> >>> 14: > >>> > >>> > >>> The other was generated whenever the show screen was loaded, and > it > >>> said: > >>> > >>> ActionView::TemplateError (catalog_index_url failed to generate > from > >>> {:action=>"show", :controller=>"catalog", :id=>nil}, expected: > >>> {:action=>"index", :controller=>"catalog"}, diff: > >>> {:action=>"index", :id=>nil}) on line #8 of vendor/plugins/ > blacklight/ > >>> app/views/catalog/show.html.erb: > >>> 5: > >>> 6: > >>> 7: <% @sidebar = capture do %> > >>> 8:

<%= link_to '<< SEARCH', > >>> catalog_index_path(params.merge(:id=>nil)) %>

> >>> 9: <% end %> > >>> > >>> Turns out these were basically the same error. In both cases, > we're > >>> using the link_to function to create a link to another part of the > >>> application, in this case from index to show, and from show to > index. > >>> Let's talk about the link from index to show as an example. > >>> > >>> <%= link_to document.get(:id), catalog_path(document[:id], > params) %> > >>> > >>> That says create a link, and the text of the link is the id > number of > >>> the current document (say, u999), and the target of the link is > >>> catalog_path(u999) which will evaluate to /catalog/u999. > Incidentally, > >>> the rails console was really valuable in helping me understand how > >>> these worked, and you can play with it like this: > >>> > >>> paz:rails eos8d$ ./script/console > >>> Loading development environment (Rails 2.2.2) > >>> >> app.catalog_path("u999") > >>> => "/catalog/u999" > >>> > >>> Anyway, I found that if I just changed the link_to statement to > look > >>> like this instead, removing the params object: > >>> > >>> <%= link_to document.get(:id), catalog_path(document[:id]) > %> > >>> > >>> the error went away. However, this wasn't an acceptable solution, > >>> because whenever a user performs a search or selects a facet, that > >>> value goes in the params hash, and you want to keep it around > when you > >>> go from screen to screen, so that the user isn't always having > to re- > >>> enter their current search. Let's say you're on the index > screen, and > >>> you've entered a search and selected a facet. At that point your > >>> params hash looks like this: > >>> > >>> {"commit"=>"search", "action"=>"index", "q"=>"poetry", > >>> "f"=>{"language_facet"=>["English"]}, "controller"=>"catalog"} > >>> > >>> When you want to look at a detailed view for a specific item, > you're > >>> linking to the show action of your same catalog controller, and > >>> passing it a specific document id. But you also want to hang > onto the > >>> fact that you've searched for poetry and limited to > >>> language_facet:English, so you also pass the params object in. At > >>> which point your link_to really looks like this: > >>> > >>> <%= link_to document.get(:id), > >>> catalog_path > >>> (:id > >>> > =>document[:id], :commit=>"search", :action=>"index", :q=>"poetry" ... > >>> etc ... ) %> > >>> > >>> Do you see the problem? It took me awhile to notice it. I've just > >>> called the show action, and then I also pass in a parameter of > >>> action=index. It's not obvious, because catalog_path sort of > obscures > >>> the fact that you're calling the show action. Interestingly, > webrick > >>> and mongrel don't seem to care. I guess they just assume that even > >>> though you've passed two values for action, you must really mean > >>> action=show because that's what's defined for catalog_path. The > fix is > >>> easy, once you know what the problem is: > >>> > >>> <%= link_to document.get(:id), > >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> > >>> > >>> Now you're passing in a params object that looks like this: > >>> > >>> {"commit"=>"search", "action"=>"show", "id"=>"2008349843", > >>> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", > >>> "controller"=>"catalog"} > >>> > >>> So, all your user defined parameters, plus the id number of the > >>> specific item you want to show, plus action=show, which is what > >>> catalog_path expects. Problem solved. > >>> > >>> Sorry this is so long, but I learned a lot in fixing this bug > and I > >>> thought it might be helpful for someone else too. But if it just > looks > >>> like gibberish, ignore it. > >>> > >>> Cheers, > >>> Bess > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> From goodieboy at gmail.com Sun Feb 1 17:59:36 2009 > >>> From: goodieboy at gmail.com (Matt Mitchell) > >>> Date: Sun, 1 Feb 2009 17:59:36 -0500 > >>> Subject: [Blacklight-development] demo app bug fix > >>> In-Reply-To: > >>> References: > >>> Message-ID: > > >>> > >>> Seriously, that's a kickin' write up Bess. Great bug catch. And > the docs > >>> look so nice too! > >>> > >>> For the link_to fix here: > >>> > >>> <%= link_to document.get(:id), > >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> > >>> > >>> You should be able to do this too: > >>> > >>> <%= link_to document.get(:id), catalog_path(document[:id], > >>> params.merge(:action=>nil)) %> > >>> > >>> I think this kind of thing will become a url-param-linking > pattern in BL > >>> and > >>> might do well in a view helper. I think there's one in the UVa > trunk, > >>> called > >>> link_to_document or something. We could then have one called > >>> link_to_documents or link_back_to_search? The upshot of it being > in a > >>> helper > >>> is that if someone overrides a view partial, they don't have to > re-write > >>> the > >>> param merging stuff. Food for thought... > >>> > >>> Matt > >>> > >>> > >> > >> _______________________________________________ > >> Blacklight-development mailing list > >> Blacklight-development at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/blacklight-development > >> Blacklightopac Blog http://blacklightopac.org/ > >> > >> > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jamie at dang.com Wed Mar 11 13:12:21 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Wed, 11 Mar 2009 13:12:21 -0400 Subject: [Blacklight-development] rsolr version? In-Reply-To: <4BC89DC5-AC13-4E1E-9416-BB03CA88B33D@dang.com> References: <4BC89DC5-AC13-4E1E-9416-BB03CA88B33D@dang.com> Message-ID: <5659122B-5387-49F8-892A-2548DB75450A@dang.com> OK, I answered my own question: YES. 0.7.1 throws an error: rails: rake db:create:all --trace (in /Users/jamieorc/dev/blacklight/trunk/rails) ** Invoke db:create:all (first_time) ** Invoke environment (first_time) ** Execute environment rake aborted! undefined method `param_mappers' for # Hey Matt, the required version in blacklight.rb ought to be handled in init.rb's require statement. I had both 0.7.1 and 0.6.9 installed, but got the "Blacklight requires RSolr #{required_rsolr_version}" error until I removed the 0.7.1 gem. I'd make the changes myself, but don't have commit rights yet. Jamie On Mar 11, 2009, at 12:29 PM, Jamie Orchard-Hays wrote: > Is this still true? > > required_rsolr_version = '0.6.9' > > I installed mwmitchell-rsolr and its version is 0.7.1 > > Jamie > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From goodieboy at gmail.com Wed Mar 11 13:33:47 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Wed, 11 Mar 2009 13:33:47 -0400 Subject: [Blacklight-development] rsolr version? In-Reply-To: <5659122B-5387-49F8-892A-2548DB75450A@dang.com> References: <4BC89DC5-AC13-4E1E-9416-BB03CA88B33D@dang.com> <5659122B-5387-49F8-892A-2548DB75450A@dang.com> Message-ID: Yep, this is correct. The current version of BL will not work with the latest version of RSolr. The reason is that the RSolr is moving towards a very lean and flexible core. All of the param mapping and response object wrapping is gone. That stuff will be moved into a separate gem and can be used optionally. The future of RSolr (master) (and to some extent solr-ruby) is here: http://github.com/mwmitchell/rsolr/tree/no-response-wrap On Friday, I'm going to see if I can merge the no-response-wrap branch into the master branch, extract the mapping stuff into a seperate lib, and then update Blacklight to be current with it all. At that point BL gains a ton of flexibility, and when solr-ruby becomes 1.0, BL's migration to solr-ruby will be very straight forward. Anyone have any objections to me doing this? Matt On Wed, Mar 11, 2009 at 1:12 PM, Jamie Orchard-Hays wrote: > OK, I answered my own question: YES. 0.7.1 throws an error: > > rails: rake db:create:all --trace > (in /Users/jamieorc/dev/blacklight/trunk/rails) > ** Invoke db:create:all (first_time) > ** Invoke environment (first_time) > ** Execute environment > rake aborted! > undefined method `param_mappers' for # > > Hey Matt, the required version in blacklight.rb ought to be handled in > init.rb's require statement. I had both 0.7.1 and 0.6.9 installed, but got > the "Blacklight requires RSolr #{required_rsolr_version}" error until I > removed the 0.7.1 gem. I'd make the changes myself, but don't have commit > rights yet. > > Jamie > > > On Mar 11, 2009, at 12:29 PM, Jamie Orchard-Hays wrote: > > Is this still true? >> >> required_rsolr_version = '0.6.9' >> >> I installed mwmitchell-rsolr and its version is 0.7.1 >> >> Jamie >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From goodieboy at gmail.com Wed Mar 11 13:37:00 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Wed, 11 Mar 2009 13:37:00 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> Message-ID: My gut is telling me that this was never a BL problem, but an :id param format problem. If the format of the :id param does not match the route rules, the route helper will not work. I'd seriously consider re-formatting the id param to use something that doesn't use a . Matt On Wed, Mar 11, 2009 at 12:55 PM, Naomi Dushay wrote: > Vernon, > Just for giggles, try this: > > 4. copied the vendors/plugins/blacklight/lib/blacklight.rb to my > app/lib/blacklight.rb > > > don't do this step > > 5 - added Blacklight.solr.param_mappers[:select] = :dismax to > app/lib/blacklight.rb > > > do this in place. > > I had trouble with my solr_helper refactoring when I needed to add and read > add'l info in solr.yml; I couldn't get it to work when I did it in my local > rails app -- I had to do it in vendor/plugins/blacklight/lib > > > If my "fix" works, then we need to discuss if this can be done differently > and/or change the documentation accordingly. > > - Naomi > > > On Mar 11, 2009, at 9:40 AM, Vernon Chapman wrote: > > I am not sure what I am doing wrong so here is what I did > > 1 - checkout blacklight from svn using svn co http://....../trunk > blacklight > 2 - followed the direction to setup internal solr/jetty server and index > records > 3 - test the setup, everything works when I got to http://localhost/ i see > the new home page with text etc. > > Here is where I am getting KILLED > > I need to change the solr url , handlers and the controller for my > application so I > > 1 - modified solr.yml from http://127.0.0.1:8983/solr to > http://127.0.0.1:8983/solr/content (my solr is multicore) > 2 - copied the home_controller & catalog_controller to my app/controllers > 3 - modified the catalog_controllers by replacing the :search symbol with > the name of the handler > on my solr server in this case :select > 4 - copied the vendors/plugins/blacklight/lib/blacklight.rb to my > app/lib/blacklight.rb > 5 - added Blacklight.solr.param_mappers[:select] = :dismax to > app/lib/blacklight.rb > 6 - restart passtenger ( touch rails/tmp/restart.txt) > 7 - restart apache2 (just to make sure passenger restarted) > > The fist time I access http://localhost/ I see the home page with the > facets from my external > solr server listed on the left, if I click on one of the facets, I get the > following : > > You have a nil object when you didn't expect it! > The error occurred while evaluating nil.commit > > > /opt/appz/blacklight/rails/app/controllers/home_controller.rb:15:in `solr_params' > > > /opt/appz/blacklight/rails/app/controllers/home_controller.rb:7:in `index' > > > the line with .commit is the Blacklight.commit line in solr_params, so for > some reason Blacklight > is nil. > > Any ideas > On Wed, Mar 11, 2009 at 10:35 AM, Molly Pickral Cadieux < > mpc3c at virginia.edu> wrote: > >> Vern, >> >> Just to make sure -- did you restart passenger or mongrel or whatever >> it is you are using? Changes to helpers seem to need an extra kick. >> >> Molly >> >> On Tue, Mar 10, 2009 at 2:51 PM, Vernon Chapman >> wrote: >> > Molly, >> > >> > Thanks for your work on this, but unfortunately that didn't fix the >> > issue. I did an svn update on my checkout of blacklight and restarted >> > apache and I still get the same error. >> > >> > Vern >> > >> > Molly Pickral Cadieux wrote: >> >> >> >> Hi again, >> >> >> >> I went ahead and tried the fix on demo.blacklightopac.org, and it >> >> works great! So, Vernon, if you could grab the new version of >> >> catalog_helper.rb from subversion, I'm almost certain that will fix >> >> the bug you reported yesterday. >> >> >> >> Molly >> >> >> >> On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux >> >> wrote: >> >> >> >>> >> >>> Hi everyone, >> >>> >> >>> I recently committed some changes to the repository that offer the >> >>> following features: >> >>> >> >>> 1. A drop-down list that allows users to specify how may results per >> >>> page to display (the previous default was 10 per page) >> >>> 2. "Previous" and "Next" links that allow a user to page through a >> >>> search result, as opposed to going back to the previous page to select >> >>> the previous or next item. >> >>> 3. Several rspec tests for the catalog controller and various models. >> >>> >> >>> This all works perfectly on my setup, but when I went to deploy the >> >>> changes on demo.blacklightopac.org, I ran into a routing issue. >> >>> Vernon also reported this error yesterday. The error is as follows: >> >>> >> >>> ------------ >> >>> >> >>> catalog_url failed to generate from {:commit=>nil, :action=>"show", >> >>> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", >> >>> :counter=>1}, expected: {:action=>"show", :controller=>"catalog"}, >> diff: >> >>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} >> >>> >> >>> Extracted source (around line #21): >> >>> >> >>> 18: <% # main title container for doc partial view -%> >> >>> 19: >> >>> 20:
>> >>> 21:
<%= counter + 1 + @response.start.to_i %>. <%= >> >>> link_to_document document, :label=>:title, :extra_params=>{:counter => >> >>> counter + 1 + @response.start.to_i } %>
>> >>> 22:
>> >>> 23: >> >>> 24: >> >>> >> >>> ----------- >> >>> >> >>> Bess had reported a similar problem back in February (I've included >> >>> the message below). Unfortunately, I have been unable to reproduce >> >>> the reported problem on my Mac running the same versions of Ruby, >> >>> Rails, and Passenger. I would like to try Bess's recommended solution >> >>> if I can get a setup where I can reproduce the problem. In the mean >> >>> time, I'd love it if Vernon or others who may have seen the problem >> >>> could try modifying this file: >> >>> >> >>> rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb >> >>> >> >>> and change this line: >> >>> >> >>> link_to label, catalog_path(doc[:id], >> >>> params.merge(opts[:extra_params]).merge(:action=>nil, :commit=>nil)) >> >>> >> >>> to be: >> >>> >> >>> link_to label, catalog_path(doc[:id], >> >>> params.merge(opts[:extra_params]).merge(:action=>"show", >> >>> :commit=>nil)) >> >>> >> >>> and see if that fixes it. A restart of passenger will be necessary. >> >>> >> >>> If I can't get this worked out soon, I will revert the changes in the >> >>> repository and work through this again here. >> >>> >> >>> In any case, I wanted to let you all know that what is currently in >> >>> the repository may cause routing issues under certain environments. >> >>> I'll keep you all posted on progress. >> >>> >> >>> Molly >> >>> >> >>> >> >>> ---------- >> >>> >> >>> From eos8d at virginia.edu Sun Feb 1 17:25:35 2009 >> >>> From: eos8d at virginia.edu (Bess Sadler) >> >>> Date: Sun, 1 Feb 2009 17:25:35 -0500 >> >>> Subject: [Blacklight-development] demo app bug fix >> >>> Message-ID: >> >>> >> >>> Hey, folks. >> >>> >> >>> I've spent the last day or so playing pretty intensively with the demo >> >>> app. I fixed a couple of minor bugs in the installation instructions >> >>> (here: >> >>> http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) >> >>> and I've deployed the demo app several times, on several different >> >>> systems. Strangely, although I could get it running right away with >> >>> mongrel or webrick, I found that I could not get it running with >> >>> mod_passenger, which is our preferred method of running rails apps in >> >>> production. I think I tracked down the bug today, and you can see the >> >>> demo app running with mod_passenger as a virtual host at >> >>> http://demo.blacklightopac.org >> >>> >> >>> I was getting two errors. One, encountered whenever the index screen >> >>> was loaded, said: >> >>> >> >>> ActionView::TemplateError (catalog_url failed to generate from >> >>> {:action=>"index", :controller=>"catalog", :id=>"u1"}, expected: >> >>> {:action=>"show", :controller=>"catalog"}, diff: >> >>> {:action=>"show", :id=>"u1"}) on line #11 of >> vendor/plugins/blacklight/ >> >>> app/views/catalog/_index_partials/_default.html.erb: >> >>> 8: >> >>> 9:
>> >>> 10:
>> >>> 11: <%= link_to document.get(:id), catalog_path(document[:id], >> >>> params) %> >> >>> 12:
>> >>> 13:
>> >>> 14: >> >>> >> >>> >> >>> The other was generated whenever the show screen was loaded, and it >> >>> said: >> >>> >> >>> ActionView::TemplateError (catalog_index_url failed to generate from >> >>> {:action=>"show", :controller=>"catalog", :id=>nil}, expected: >> >>> {:action=>"index", :controller=>"catalog"}, diff: >> >>> {:action=>"index", :id=>nil}) on line #8 of vendor/plugins/blacklight/ >> >>> app/views/catalog/show.html.erb: >> >>> 5: >> >>> 6: >> >>> 7: <% @sidebar = capture do %> >> >>> 8:

<%= link_to '<< SEARCH', >> >>> catalog_index_path(params.merge(:id=>nil)) %>

>> >>> 9: <% end %> >> >>> >> >>> Turns out these were basically the same error. In both cases, we're >> >>> using the link_to function to create a link to another part of the >> >>> application, in this case from index to show, and from show to index. >> >>> Let's talk about the link from index to show as an example. >> >>> >> >>> <%= link_to document.get(:id), catalog_path(document[:id], params) %> >> >>> >> >>> That says create a link, and the text of the link is the id number of >> >>> the current document (say, u999), and the target of the link is >> >>> catalog_path(u999) which will evaluate to /catalog/u999. Incidentally, >> >>> the rails console was really valuable in helping me understand how >> >>> these worked, and you can play with it like this: >> >>> >> >>> paz:rails eos8d$ ./script/console >> >>> Loading development environment (Rails 2.2.2) >> >>> >> app.catalog_path("u999") >> >>> => "/catalog/u999" >> >>> >> >>> Anyway, I found that if I just changed the link_to statement to look >> >>> like this instead, removing the params object: >> >>> >> >>> <%= link_to document.get(:id), catalog_path(document[:id]) %> >> >>> >> >>> the error went away. However, this wasn't an acceptable solution, >> >>> because whenever a user performs a search or selects a facet, that >> >>> value goes in the params hash, and you want to keep it around when you >> >>> go from screen to screen, so that the user isn't always having to re- >> >>> enter their current search. Let's say you're on the index screen, and >> >>> you've entered a search and selected a facet. At that point your >> >>> params hash looks like this: >> >>> >> >>> {"commit"=>"search", "action"=>"index", "q"=>"poetry", >> >>> "f"=>{"language_facet"=>["English"]}, "controller"=>"catalog"} >> >>> >> >>> When you want to look at a detailed view for a specific item, you're >> >>> linking to the show action of your same catalog controller, and >> >>> passing it a specific document id. But you also want to hang onto the >> >>> fact that you've searched for poetry and limited to >> >>> language_facet:English, so you also pass the params object in. At >> >>> which point your link_to really looks like this: >> >>> >> >>> <%= link_to document.get(:id), >> >>> catalog_path >> >>> (:id >> >>> =>document[:id], :commit=>"search", :action=>"index", :q=>"poetry" ... >> >>> etc ... ) %> >> >>> >> >>> Do you see the problem? It took me awhile to notice it. I've just >> >>> called the show action, and then I also pass in a parameter of >> >>> action=index. It's not obvious, because catalog_path sort of obscures >> >>> the fact that you're calling the show action. Interestingly, webrick >> >>> and mongrel don't seem to care. I guess they just assume that even >> >>> though you've passed two values for action, you must really mean >> >>> action=show because that's what's defined for catalog_path. The fix is >> >>> easy, once you know what the problem is: >> >>> >> >>> <%= link_to document.get(:id), >> >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >> >>> >> >>> Now you're passing in a params object that looks like this: >> >>> >> >>> {"commit"=>"search", "action"=>"show", "id"=>"2008349843", >> >>> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", >> >>> "controller"=>"catalog"} >> >>> >> >>> So, all your user defined parameters, plus the id number of the >> >>> specific item you want to show, plus action=show, which is what >> >>> catalog_path expects. Problem solved. >> >>> >> >>> Sorry this is so long, but I learned a lot in fixing this bug and I >> >>> thought it might be helpful for someone else too. But if it just looks >> >>> like gibberish, ignore it. >> >>> >> >>> Cheers, >> >>> Bess >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> From goodieboy at gmail.com Sun Feb 1 17:59:36 2009 >> >>> From: goodieboy at gmail.com (Matt Mitchell) >> >>> Date: Sun, 1 Feb 2009 17:59:36 -0500 >> >>> Subject: [Blacklight-development] demo app bug fix >> >>> In-Reply-To: >> >>> References: >> >>> Message-ID: < >> c66dcac40902011459y279c7187yd46effb14248da9a at mail.gmail.com> >> >>> >> >>> Seriously, that's a kickin' write up Bess. Great bug catch. And the >> docs >> >>> look so nice too! >> >>> >> >>> For the link_to fix here: >> >>> >> >>> <%= link_to document.get(:id), >> >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >> >>> >> >>> You should be able to do this too: >> >>> >> >>> <%= link_to document.get(:id), catalog_path(document[:id], >> >>> params.merge(:action=>nil)) %> >> >>> >> >>> I think this kind of thing will become a url-param-linking pattern in >> BL >> >>> and >> >>> might do well in a view helper. I think there's one in the UVa trunk, >> >>> called >> >>> link_to_document or something. We could then have one called >> >>> link_to_documents or link_back_to_search? The upshot of it being in a >> >>> helper >> >>> is that if someone overrides a view partial, they don't have to >> re-write >> >>> the >> >>> param merging stuff. Food for thought... >> >>> >> >>> Matt >> >>> >> >>> >> >> >> >> _______________________________________________ >> >> Blacklight-development mailing list >> >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> >> >> > >> > _______________________________________________ >> > Blacklight-development mailing list >> > Blacklight-development at rubyforge.org >> > http://rubyforge.org/mailman/listinfo/blacklight-development >> > Blacklightopac Blog http://blacklightopac.org/ >> > >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ndushay at stanford.edu Wed Mar 11 13:49:18 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Wed, 11 Mar 2009 10:49:18 -0700 Subject: [Blacklight-development] routing problem In-Reply-To: <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> Message-ID: <1E26C47D-8167-4583-A140-7754530A0FD4@stanford.edu> Vernon: > /opt/appz/blacklight/rails/app/controllers/home_controller.rb:15:in > `solr_params' One more thought: what happens if you comment out in app/controllers/home_controller.rb Blacklight.solr.commit The error *seems* to be saying that Blacklight.solr is a nil object. Yet, in order to give you the facets on the home page, it is NOT a nil object; I believe the following is working: @response = Blacklight.solr.search(solr_params) This would be another piece of information to help figure out what is going on. FYI, Blacklight.solr.commit is only about ensuring that your search results are getting the latest data from solr (that any pending updates to the solr index are, essentially, flushed to the index). If you're like us, where you don't have live updates to your solr index, then it's not necessary. But of course we need to get the code working for you. - Naomi On Mar 11, 2009, at 9:40 AM, Vernon Chapman wrote: > I am not sure what I am doing wrong so here is what I did > > 1 - checkout blacklight from svn using svn co http://....../trunk > blacklight > 2 - followed the direction to setup internal solr/jetty server and > index records > 3 - test the setup, everything works when I got to http://localhost/ > i see the new home page with text etc. > > Here is where I am getting KILLED > > I need to change the solr url , handlers and the controller for my > application so I > > 1 - modified solr.yml from http://127.0.0.1:8983/solr to http://127.0.0.1:8983/solr/content > (my solr is multicore) > 2 - copied the home_controller & catalog_controller to my app/ > controllers > 3 - modified the catalog_controllers by replacing the :search symbol > with the name of the handler > on my solr server in this case :select > 4 - copied the vendors/plugins/blacklight/lib/blacklight.rb to my > app/lib/blacklight.rb > 5 - added Blacklight.solr.param_mappers[:select] = :dismax to app/ > lib/blacklight.rb > 6 - restart passtenger ( touch rails/tmp/restart.txt) > 7 - restart apache2 (just to make sure passenger restarted) > > The fist time I access http://localhost/ I see the home page with > the facets from my external > solr server listed on the left, if I click on one of the facets, I > get the following : > > You have a nil object when you didn't expect it! > The error occurred while evaluating nil.commit > > > /opt/appz/blacklight/rails/app/controllers/home_controller.rb:15:in > `solr_params' > > > /opt/appz/blacklight/rails/app/controllers/home_controller.rb:7:in > `index' > > > the line with .commit is the Blacklight.commit line in solr_params, > so for some reason Blacklight > is nil. > > Any ideas > On Wed, Mar 11, 2009 at 10:35 AM, Molly Pickral Cadieux > wrote: > Vern, > > Just to make sure -- did you restart passenger or mongrel or whatever > it is you are using? Changes to helpers seem to need an extra kick. > > Molly > > On Tue, Mar 10, 2009 at 2:51 PM, Vernon Chapman > wrote: > > Molly, > > > > Thanks for your work on this, but unfortunately that didn't fix the > > issue. I did an svn update on my checkout of blacklight and > restarted > > apache and I still get the same error. > > > > Vern > > > > Molly Pickral Cadieux wrote: > >> > >> Hi again, > >> > >> I went ahead and tried the fix on demo.blacklightopac.org, and it > >> works great! So, Vernon, if you could grab the new version of > >> catalog_helper.rb from subversion, I'm almost certain that will fix > >> the bug you reported yesterday. > >> > >> Molly > >> > >> On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux > >> wrote: > >> > >>> > >>> Hi everyone, > >>> > >>> I recently committed some changes to the repository that offer the > >>> following features: > >>> > >>> 1. A drop-down list that allows users to specify how may > results per > >>> page to display (the previous default was 10 per page) > >>> 2. "Previous" and "Next" links that allow a user to page > through a > >>> search result, as opposed to going back to the previous page to > select > >>> the previous or next item. > >>> 3. Several rspec tests for the catalog controller and various > models. > >>> > >>> This all works perfectly on my setup, but when I went to deploy > the > >>> changes on demo.blacklightopac.org, I ran into a routing issue. > >>> Vernon also reported this error yesterday. The error is as > follows: > >>> > >>> ------------ > >>> > >>> catalog_url failed to generate from > {:commit=>nil, :action=>"show", > >>> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", > >>> :counter=>1}, expected: > {:action=>"show", :controller=>"catalog"}, diff: > >>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", :counter=>1} > >>> > >>> Extracted source (around line #21): > >>> > >>> 18: <% # main title container for doc partial view -%> > >>> 19: > >>> 20:
> >>> 21:
<%= counter + 1 + @response.start.to_i %>. <%= > >>> link_to_document > document, :label=>:title, :extra_params=>{:counter => > >>> counter + 1 + @response.start.to_i } %>
> >>> 22:
> >>> 23: > >>> 24: > >>> > >>> ----------- > >>> > >>> Bess had reported a similar problem back in February (I've > included > >>> the message below). Unfortunately, I have been unable to > reproduce > >>> the reported problem on my Mac running the same versions of Ruby, > >>> Rails, and Passenger. I would like to try Bess's recommended > solution > >>> if I can get a setup where I can reproduce the problem. In the > mean > >>> time, I'd love it if Vernon or others who may have seen the > problem > >>> could try modifying this file: > >>> > >>> rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb > >>> > >>> and change this line: > >>> > >>> link_to label, catalog_path(doc[:id], > >>> > params.merge(opts[:extra_params]).merge(:action=>nil, :commit=>nil)) > >>> > >>> to be: > >>> > >>> link_to label, catalog_path(doc[:id], > >>> params.merge(opts[:extra_params]).merge(:action=>"show", > >>> :commit=>nil)) > >>> > >>> and see if that fixes it. A restart of passenger will be > necessary. > >>> > >>> If I can't get this worked out soon, I will revert the changes > in the > >>> repository and work through this again here. > >>> > >>> In any case, I wanted to let you all know that what is currently > in > >>> the repository may cause routing issues under certain > environments. > >>> I'll keep you all posted on progress. > >>> > >>> Molly > >>> > >>> > >>> ---------- > >>> > >>> From eos8d at virginia.edu Sun Feb 1 17:25:35 2009 > >>> From: eos8d at virginia.edu (Bess Sadler) > >>> Date: Sun, 1 Feb 2009 17:25:35 -0500 > >>> Subject: [Blacklight-development] demo app bug fix > >>> Message-ID: > >>> > >>> Hey, folks. > >>> > >>> I've spent the last day or so playing pretty intensively with > the demo > >>> app. I fixed a couple of minor bugs in the installation > instructions > >>> (here: > >>> http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) > >>> and I've deployed the demo app several times, on several > different > >>> systems. Strangely, although I could get it running right away > with > >>> mongrel or webrick, I found that I could not get it running with > >>> mod_passenger, which is our preferred method of running rails > apps in > >>> production. I think I tracked down the bug today, and you can > see the > >>> demo app running with mod_passenger as a virtual host at > >>> http://demo.blacklightopac.org > >>> > >>> I was getting two errors. One, encountered whenever the index > screen > >>> was loaded, said: > >>> > >>> ActionView::TemplateError (catalog_url failed to generate from > >>> {:action=>"index", :controller=>"catalog", :id=>"u1"}, expected: > >>> {:action=>"show", :controller=>"catalog"}, diff: > >>> {:action=>"show", :id=>"u1"}) on line #11 of vendor/plugins/ > blacklight/ > >>> app/views/catalog/_index_partials/_default.html.erb: > >>> 8: > >>> 9:
> >>> 10:
> >>> 11: <%= link_to document.get(:id), > catalog_path(document[:id], > >>> params) %> > >>> 12:
> >>> 13:
> >>> 14: > >>> > >>> > >>> The other was generated whenever the show screen was loaded, and > it > >>> said: > >>> > >>> ActionView::TemplateError (catalog_index_url failed to generate > from > >>> {:action=>"show", :controller=>"catalog", :id=>nil}, expected: > >>> {:action=>"index", :controller=>"catalog"}, diff: > >>> {:action=>"index", :id=>nil}) on line #8 of vendor/plugins/ > blacklight/ > >>> app/views/catalog/show.html.erb: > >>> 5: > >>> 6: > >>> 7: <% @sidebar = capture do %> > >>> 8:

<%= link_to '<< SEARCH', > >>> catalog_index_path(params.merge(:id=>nil)) %>

> >>> 9: <% end %> > >>> > >>> Turns out these were basically the same error. In both cases, > we're > >>> using the link_to function to create a link to another part of the > >>> application, in this case from index to show, and from show to > index. > >>> Let's talk about the link from index to show as an example. > >>> > >>> <%= link_to document.get(:id), catalog_path(document[:id], > params) %> > >>> > >>> That says create a link, and the text of the link is the id > number of > >>> the current document (say, u999), and the target of the link is > >>> catalog_path(u999) which will evaluate to /catalog/u999. > Incidentally, > >>> the rails console was really valuable in helping me understand how > >>> these worked, and you can play with it like this: > >>> > >>> paz:rails eos8d$ ./script/console > >>> Loading development environment (Rails 2.2.2) > >>> >> app.catalog_path("u999") > >>> => "/catalog/u999" > >>> > >>> Anyway, I found that if I just changed the link_to statement to > look > >>> like this instead, removing the params object: > >>> > >>> <%= link_to document.get(:id), catalog_path(document[:id]) > %> > >>> > >>> the error went away. However, this wasn't an acceptable solution, > >>> because whenever a user performs a search or selects a facet, that > >>> value goes in the params hash, and you want to keep it around > when you > >>> go from screen to screen, so that the user isn't always having > to re- > >>> enter their current search. Let's say you're on the index > screen, and > >>> you've entered a search and selected a facet. At that point your > >>> params hash looks like this: > >>> > >>> {"commit"=>"search", "action"=>"index", "q"=>"poetry", > >>> "f"=>{"language_facet"=>["English"]}, "controller"=>"catalog"} > >>> > >>> When you want to look at a detailed view for a specific item, > you're > >>> linking to the show action of your same catalog controller, and > >>> passing it a specific document id. But you also want to hang > onto the > >>> fact that you've searched for poetry and limited to > >>> language_facet:English, so you also pass the params object in. At > >>> which point your link_to really looks like this: > >>> > >>> <%= link_to document.get(:id), > >>> catalog_path > >>> (:id > >>> > =>document[:id], :commit=>"search", :action=>"index", :q=>"poetry" ... > >>> etc ... ) %> > >>> > >>> Do you see the problem? It took me awhile to notice it. I've just > >>> called the show action, and then I also pass in a parameter of > >>> action=index. It's not obvious, because catalog_path sort of > obscures > >>> the fact that you're calling the show action. Interestingly, > webrick > >>> and mongrel don't seem to care. I guess they just assume that even > >>> though you've passed two values for action, you must really mean > >>> action=show because that's what's defined for catalog_path. The > fix is > >>> easy, once you know what the problem is: > >>> > >>> <%= link_to document.get(:id), > >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> > >>> > >>> Now you're passing in a params object that looks like this: > >>> > >>> {"commit"=>"search", "action"=>"show", "id"=>"2008349843", > >>> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", > >>> "controller"=>"catalog"} > >>> > >>> So, all your user defined parameters, plus the id number of the > >>> specific item you want to show, plus action=show, which is what > >>> catalog_path expects. Problem solved. > >>> > >>> Sorry this is so long, but I learned a lot in fixing this bug > and I > >>> thought it might be helpful for someone else too. But if it just > looks > >>> like gibberish, ignore it. > >>> > >>> Cheers, > >>> Bess > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> From goodieboy at gmail.com Sun Feb 1 17:59:36 2009 > >>> From: goodieboy at gmail.com (Matt Mitchell) > >>> Date: Sun, 1 Feb 2009 17:59:36 -0500 > >>> Subject: [Blacklight-development] demo app bug fix > >>> In-Reply-To: > >>> References: > >>> Message-ID: > > >>> > >>> Seriously, that's a kickin' write up Bess. Great bug catch. And > the docs > >>> look so nice too! > >>> > >>> For the link_to fix here: > >>> > >>> <%= link_to document.get(:id), > >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> > >>> > >>> You should be able to do this too: > >>> > >>> <%= link_to document.get(:id), catalog_path(document[:id], > >>> params.merge(:action=>nil)) %> > >>> > >>> I think this kind of thing will become a url-param-linking > pattern in BL > >>> and > >>> might do well in a view helper. I think there's one in the UVa > trunk, > >>> called > >>> link_to_document or something. We could then have one called > >>> link_to_documents or link_back_to_search? The upshot of it being > in a > >>> helper > >>> is that if someone overrides a view partial, they don't have to > re-write > >>> the > >>> param merging stuff. Food for thought... > >>> > >>> Matt > >>> > >>> > >> > >> _______________________________________________ > >> Blacklight-development mailing list > >> Blacklight-development at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/blacklight-development > >> Blacklightopac Blog http://blacklightopac.org/ > >> > >> > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mpc3c at virginia.edu Wed Mar 11 14:01:36 2009 From: mpc3c at virginia.edu (Molly Pickral Cadieux) Date: Wed, 11 Mar 2009 14:01:36 -0400 Subject: [Blacklight-development] rss feeds in plugin Message-ID: <34e39bfd0903111101w1fb81fvc358e09415029339@mail.gmail.com> Hi all, This is in reference to Jira ticket CODEBASE-20. I have added RSS output for Blacklight searches, though it is currently a "stealth" feature, since it's not obvious from the BL interface. Also, I kept the RSS output very lean, but it should be easy to expand upon it. Basically, you can add a .rss extension to "catalog" to render as RSS. For example: http://demo.blacklightopac.org/catalog.rss http://demo.blacklightopac.org/catalog.rss?f[subject_era_facet][]=20th+century http://demo.blacklightopac.org/catalog.rss?f[subject_era_facet][]=20th+century&per_page=20 Molly From goodieboy at gmail.com Wed Mar 11 14:34:16 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Wed, 11 Mar 2009 14:34:16 -0400 Subject: [Blacklight-development] solr query for single document: standard or dismax? In-Reply-To: <42F023C7-A62F-48B7-B484-5E751A437CE9@stanford.edu> References: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> <9BB431CA-C6C0-4587-BB1B-3A5C73629DF4@stanford.edu> <419F32C3-02C8-4F20-B957-D6E2AB5DFF19@mac.com> <42F023C7-A62F-48B7-B484-5E751A437CE9@stanford.edu> Message-ID: Naomi, I'm going to (asap) update RSolr soon, and will integrate it all into BL as well (if everyone here is OK with that). As soon as that's done, making changes, or even hacking will be easier and less likely to break when solr-ruby 1.0 is released. But yeah, I'd recommend not hacking too much for the time being. Matt On Wed, Mar 11, 2009 at 12:48 PM, Naomi Dushay wrote: > Or, to gain this flexibility, would it be prudent for blacklight to use >>> the solr-ruby gem instead of the mwmitchell-solr gem? >>> >> >> I'm working with Matt to get his ideas/implementation into a solr-ruby >> refactoring, so in the not too distant future we won't have this >> discrepancy... we'll all happily be using solr-ruby 1.0 :) >> > > +1 > > I might hold off on big changes to the Gems and/or Gem override > functionality until this is ready; I'm too much of a RoR noob. > > But my refactoring will make it easier. One step at a time. > > - Naomi > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chapman.lists at gmail.com Wed Mar 11 14:34:23 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Wed, 11 Mar 2009 14:34:23 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> Message-ID: <49B8042F.8020108@gmail.com> Naomi, Thanks for the idea. That actually did the trick. I am now able to access the new Welcome page. Thanks, but doesn't that make that volatile "fix" as the next time I run svn update that change could be destroyed if that file has been updated in any way? Now I am back to the error when rendering the documents. I'll try to figure some of the suggestions already given and report back. Thanks Naomi Dushay wrote: > Vernon, > > Just for giggles, try this: > >> 4. copied the vendors/plugins/blacklight/lib/blacklight.rb to my >> app/lib/blacklight.rb > > don't do this step > >> 5 - added Blacklight.solr.param_mappers[:select] = :dismax to >> app/lib/blacklight.rb > > do this in place. > > I had trouble with my solr_helper refactoring when I needed to add and > read add'l info in solr.yml; I couldn't get it to work when I did it > in my local rails app -- I had to do it in > vendor/plugins/blacklight/lib > > > If my "fix" works, then we need to discuss if this can be done > differently and/or change the documentation accordingly. > > - Naomi > > > On Mar 11, 2009, at 9:40 AM, Vernon Chapman wrote: > >> I am not sure what I am doing wrong so here is what I did >> >> 1 - checkout blacklight from svn using svn co http://....../trunk >> blacklight >> 2 - followed the direction to setup internal solr/jetty server and >> index records >> 3 - test the setup, everything works when I got to http://localhost/ >> i see the new home page with text etc. >> >> Here is where I am getting KILLED >> >> I need to change the solr url , handlers and the controller for my >> application so I >> >> 1 - modified solr.yml from http://127.0.0.1:8983/solr to >> http://127.0.0.1:8983/solr/content (my solr is multicore) >> 2 - copied the home_controller & catalog_controller to my app/controllers >> 3 - modified the catalog_controllers by replacing the :search symbol >> with the name of the handler >> on my solr server in this case :select >> 4 - copied the vendors/plugins/blacklight/lib/blacklight.rb to my >> app/lib/blacklight.rb >> 5 - added Blacklight.solr.param_mappers[:select] = :dismax to >> app/lib/blacklight.rb >> 6 - restart passtenger ( touch rails/tmp/restart.txt) >> 7 - restart apache2 (just to make sure passenger restarted) >> >> The fist time I access http://localhost/ I see the home page with the >> facets from my external >> solr server listed on the left, if I click on one of the facets, I >> get the following : >> >> You have a nil object when you didn't expect it! >> The error occurred while evaluating nil.commit >> >> >> |/opt/appz/blacklight/rails/app/controllers/home_controller.rb:15:in `solr_params' >> >> >> /opt/appz/blacklight/rails/app/controllers/home_controller.rb:7:in `index'| >> >> >> >> the line with .commit is the Blacklight.commit line in solr_params, >> so for some reason Blacklight >> is nil. >> >> Any ideas >> On Wed, Mar 11, 2009 at 10:35 AM, Molly Pickral Cadieux >> > wrote: >> >> Vern, >> >> Just to make sure -- did you restart passenger or mongrel or whatever >> it is you are using? Changes to helpers seem to need an extra kick. >> >> Molly >> >> On Tue, Mar 10, 2009 at 2:51 PM, Vernon Chapman >> > wrote: >> > Molly, >> > >> > Thanks for your work on this, but unfortunately that didn't fix the >> > issue. I did an svn update on my checkout of blacklight and >> restarted >> > apache and I still get the same error. >> > >> > Vern >> > >> > Molly Pickral Cadieux wrote: >> >> >> >> Hi again, >> >> >> >> I went ahead and tried the fix on demo.blacklightopac.org >> , and it >> >> works great! So, Vernon, if you could grab the new version of >> >> catalog_helper.rb from subversion, I'm almost certain that >> will fix >> >> the bug you reported yesterday. >> >> >> >> Molly >> >> >> >> On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux >> >> > wrote: >> >> >> >>> >> >>> Hi everyone, >> >>> >> >>> I recently committed some changes to the repository that >> offer the >> >>> following features: >> >>> >> >>> 1. A drop-down list that allows users to specify how may >> results per >> >>> page to display (the previous default was 10 per page) >> >>> 2. "Previous" and "Next" links that allow a user to page >> through a >> >>> search result, as opposed to going back to the previous page >> to select >> >>> the previous or next item. >> >>> 3. Several rspec tests for the catalog controller and >> various models. >> >>> >> >>> This all works perfectly on my setup, but when I went to >> deploy the >> >>> changes on demo.blacklightopac.org >> , I ran into a routing issue. >> >>> Vernon also reported this error yesterday. The error is as >> follows: >> >>> >> >>> ------------ >> >>> >> >>> catalog_url failed to generate from {:commit=>nil, >> :action=>"show", >> >>> :id=>"agrecord.2008-11-27.1267669533", :controller=>"catalog", >> >>> :counter=>1}, expected: {:action=>"show", >> :controller=>"catalog"}, diff: >> >>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", >> :counter=>1} >> >>> >> >>> Extracted source (around line #21): >> >>> >> >>> 18: <% # main title container for doc partial view -%> >> >>> 19: >> >>> 20:
>> >>> 21:
<%= counter + 1 + @response.start.to_i %>. <%= >> >>> link_to_document document, :label=>:title, >> :extra_params=>{:counter => >> >>> counter + 1 + @response.start.to_i } %>
>> >>> 22:
>> >>> 23: >> >>> 24: >> >>> >> >>> ----------- >> >>> >> >>> Bess had reported a similar problem back in February (I've >> included >> >>> the message below). Unfortunately, I have been unable to >> reproduce >> >>> the reported problem on my Mac running the same versions of Ruby, >> >>> Rails, and Passenger. I would like to try Bess's recommended >> solution >> >>> if I can get a setup where I can reproduce the problem. In >> the mean >> >>> time, I'd love it if Vernon or others who may have seen the >> problem >> >>> could try modifying this file: >> >>> >> >>> rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb >> >>> >> >>> and change this line: >> >>> >> >>> link_to label, catalog_path(doc[:id], >> >>> params.merge(opts[:extra_params]).merge(:action=>nil, >> :commit=>nil)) >> >>> >> >>> to be: >> >>> >> >>> link_to label, catalog_path(doc[:id], >> >>> params.merge(opts[:extra_params]).merge(:action=>"show", >> >>> :commit=>nil)) >> >>> >> >>> and see if that fixes it. A restart of passenger will be >> necessary. >> >>> >> >>> If I can't get this worked out soon, I will revert the >> changes in the >> >>> repository and work through this again here. >> >>> >> >>> In any case, I wanted to let you all know that what is >> currently in >> >>> the repository may cause routing issues under certain >> environments. >> >>> I'll keep you all posted on progress. >> >>> >> >>> Molly >> >>> >> >>> >> >>> ---------- >> >>> >> >>> From eos8d at virginia.edu Sun Feb 1 >> 17:25:35 2009 >> >>> From: eos8d at virginia.edu (Bess Sadler) >> >>> Date: Sun, 1 Feb 2009 17:25:35 -0500 >> >>> Subject: [Blacklight-development] demo app bug fix >> >>> Message-ID: >> > > >> >>> >> >>> Hey, folks. >> >>> >> >>> I've spent the last day or so playing pretty intensively with >> the demo >> >>> app. I fixed a couple of minor bugs in the installation >> instructions >> >>> (here: >> >>> >> http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) >> >>> and I've deployed the demo app several times, on several >> different >> >>> systems. Strangely, although I could get it running right >> away with >> >>> mongrel or webrick, I found that I could not get it running with >> >>> mod_passenger, which is our preferred method of running rails >> apps in >> >>> production. I think I tracked down the bug today, and you can >> see the >> >>> demo app running with mod_passenger as a virtual host at >> >>> http://demo.blacklightopac.org >> >>> >> >>> I was getting two errors. One, encountered whenever the index >> screen >> >>> was loaded, said: >> >>> >> >>> ActionView::TemplateError (catalog_url failed to generate from >> >>> {:action=>"index", :controller=>"catalog", :id=>"u1"}, expected: >> >>> {:action=>"show", :controller=>"catalog"}, diff: >> >>> {:action=>"show", :id=>"u1"}) on line #11 of >> vendor/plugins/blacklight/ >> >>> app/views/catalog/_index_partials/_default.html.erb: >> >>> 8: >> >>> 9:
>> >>> 10:
>> >>> 11: <%= link_to document.get(:id), >> catalog_path(document[:id], >> >>> params) %> >> >>> 12:
>> >>> 13:
>> >>> 14: >> >>> >> >>> >> >>> The other was generated whenever the show screen was loaded, >> and it >> >>> said: >> >>> >> >>> ActionView::TemplateError (catalog_index_url failed to >> generate from >> >>> {:action=>"show", :controller=>"catalog", :id=>nil}, expected: >> >>> {:action=>"index", :controller=>"catalog"}, diff: >> >>> {:action=>"index", :id=>nil}) on line #8 of >> vendor/plugins/blacklight/ >> >>> app/views/catalog/show.html.erb: >> >>> 5: >> >>> 6: >> >>> 7: <% @sidebar = capture do %> >> >>> 8:

<%= link_to '<< SEARCH', >> >>> catalog_index_path(params.merge(:id=>nil)) %>

>> >>> 9: <% end %> >> >>> >> >>> Turns out these were basically the same error. In both cases, >> we're >> >>> using the link_to function to create a link to another part >> of the >> >>> application, in this case from index to show, and from show >> to index. >> >>> Let's talk about the link from index to show as an example. >> >>> >> >>> <%= link_to document.get(:id), catalog_path(document[:id], >> params) %> >> >>> >> >>> That says create a link, and the text of the link is the id >> number of >> >>> the current document (say, u999), and the target of the link is >> >>> catalog_path(u999) which will evaluate to /catalog/u999. >> Incidentally, >> >>> the rails console was really valuable in helping me >> understand how >> >>> these worked, and you can play with it like this: >> >>> >> >>> paz:rails eos8d$ ./script/console >> >>> Loading development environment (Rails 2.2.2) >> >>> >> app.catalog_path("u999") >> >>> => "/catalog/u999" >> >>> >> >>> Anyway, I found that if I just changed the link_to statement >> to look >> >>> like this instead, removing the params object: >> >>> >> >>> <%= link_to document.get(:id), >> catalog_path(document[:id]) %> >> >>> >> >>> the error went away. However, this wasn't an acceptable solution, >> >>> because whenever a user performs a search or selects a facet, >> that >> >>> value goes in the params hash, and you want to keep it around >> when you >> >>> go from screen to screen, so that the user isn't always >> having to re- >> >>> enter their current search. Let's say you're on the index >> screen, and >> >>> you've entered a search and selected a facet. At that point your >> >>> params hash looks like this: >> >>> >> >>> {"commit"=>"search", "action"=>"index", "q"=>"poetry", >> >>> "f"=>{"language_facet"=>["English"]}, "controller"=>"catalog"} >> >>> >> >>> When you want to look at a detailed view for a specific item, >> you're >> >>> linking to the show action of your same catalog controller, and >> >>> passing it a specific document id. But you also want to hang >> onto the >> >>> fact that you've searched for poetry and limited to >> >>> language_facet:English, so you also pass the params object in. At >> >>> which point your link_to really looks like this: >> >>> >> >>> <%= link_to document.get(:id), >> >>> catalog_path >> >>> (:id >> >>> =>document[:id], :commit=>"search", :action=>"index", >> :q=>"poetry" ... >> >>> etc ... ) %> >> >>> >> >>> Do you see the problem? It took me awhile to notice it. I've just >> >>> called the show action, and then I also pass in a parameter of >> >>> action=index. It's not obvious, because catalog_path sort of >> obscures >> >>> the fact that you're calling the show action. Interestingly, >> webrick >> >>> and mongrel don't seem to care. I guess they just assume that >> even >> >>> though you've passed two values for action, you must really mean >> >>> action=show because that's what's defined for catalog_path. >> The fix is >> >>> easy, once you know what the problem is: >> >>> >> >>> <%= link_to document.get(:id), >> >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >> >>> >> >>> Now you're passing in a params object that looks like this: >> >>> >> >>> {"commit"=>"search", "action"=>"show", "id"=>"2008349843", >> >>> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", >> >>> "controller"=>"catalog"} >> >>> >> >>> So, all your user defined parameters, plus the id number of the >> >>> specific item you want to show, plus action=show, which is what >> >>> catalog_path expects. Problem solved. >> >>> >> >>> Sorry this is so long, but I learned a lot in fixing this bug >> and I >> >>> thought it might be helpful for someone else too. But if it >> just looks >> >>> like gibberish, ignore it. >> >>> >> >>> Cheers, >> >>> Bess >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> From goodieboy at gmail.com Sun Feb 1 >> 17:59:36 2009 >> >>> From: goodieboy at gmail.com (Matt Mitchell) >> >>> Date: Sun, 1 Feb 2009 17:59:36 -0500 >> >>> Subject: [Blacklight-development] demo app bug fix >> >>> In-Reply-To: >> > > >> >>> References: >> > > >> >>> Message-ID: >> > > >> >>> >> >>> Seriously, that's a kickin' write up Bess. Great bug catch. >> And the docs >> >>> look so nice too! >> >>> >> >>> For the link_to fix here: >> >>> >> >>> <%= link_to document.get(:id), >> >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >> >>> >> >>> You should be able to do this too: >> >>> >> >>> <%= link_to document.get(:id), catalog_path(document[:id], >> >>> params.merge(:action=>nil)) %> >> >>> >> >>> I think this kind of thing will become a url-param-linking >> pattern in BL >> >>> and >> >>> might do well in a view helper. I think there's one in the >> UVa trunk, >> >>> called >> >>> link_to_document or something. We could then have one called >> >>> link_to_documents or link_back_to_search? The upshot of it >> being in a >> >>> helper >> >>> is that if someone overrides a view partial, they don't have >> to re-write >> >>> the >> >>> param merging stuff. Food for thought... >> >>> >> >>> Matt >> >>> >> >>> >> >> >> >> _______________________________________________ >> >> Blacklight-development mailing list >> >> Blacklight-development at rubyforge.org >> >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> >> >> > >> > _______________________________________________ >> > Blacklight-development mailing list >> > Blacklight-development at rubyforge.org >> >> > http://rubyforge.org/mailman/listinfo/blacklight-development >> > Blacklightopac Blog http://blacklightopac.org/ >> > >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > ------------------------------------------------------------------------ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From chapman.lists at gmail.com Wed Mar 11 15:20:06 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Wed, 11 Mar 2009 15:20:06 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> Message-ID: <49B80EE6.9050401@gmail.com> Matt, That gut of yours seems to be right on point. There seems to be an issue with the id having "." in the id I am going to go back and recreate the ids. Thanks Matt Mitchell wrote: > My gut is telling me that this was never a BL problem, but an :id > param format problem. If the format of the :id param does not match > the route rules, the route helper will not work. I'd seriously > consider re-formatting the id param to use something that doesn't use a . > > Matt > > On Wed, Mar 11, 2009 at 12:55 PM, Naomi Dushay > wrote: > > Vernon, > > Just for giggles, try this: > >> 4. copied the vendors/plugins/blacklight/lib/blacklight.rb to >> my app/lib/blacklight.rb > > don't do this step > >> 5 - added Blacklight.solr.param_mappers[:select] = :dismax to >> app/lib/blacklight.rb > > do this in place. > > I had trouble with my solr_helper refactoring when I needed to add > and read add'l info in solr.yml; I couldn't get it to work when I > did it in my local rails app -- I had to do it in > vendor/plugins/blacklight/lib > > > If my "fix" works, then we need to discuss if this can be done > differently and/or change the documentation accordingly. > > - Naomi > > > On Mar 11, 2009, at 9:40 AM, Vernon Chapman wrote: > >> I am not sure what I am doing wrong so here is what I did >> >> 1 - checkout blacklight from svn using svn co http://....../trunk >> blacklight >> 2 - followed the direction to setup internal solr/jetty server >> and index records >> 3 - test the setup, everything works when I got to >> http://localhost/ i see the new home page with text etc. >> >> Here is where I am getting KILLED >> >> I need to change the solr url , handlers and the controller for >> my application so I >> >> 1 - modified solr.yml from http://127.0.0.1:8983/solr to >> http://127.0.0.1:8983/solr/content (my solr is multicore) >> 2 - copied the home_controller & catalog_controller to my >> app/controllers >> 3 - modified the catalog_controllers by replacing the :search >> symbol with the name of the handler >> on my solr server in this case :select >> 4 - copied the vendors/plugins/blacklight/lib/blacklight.rb to >> my app/lib/blacklight.rb >> 5 - added Blacklight.solr.param_mappers[:select] = :dismax to >> app/lib/blacklight.rb >> 6 - restart passtenger ( touch rails/tmp/restart.txt) >> 7 - restart apache2 (just to make sure passenger restarted) >> >> The fist time I access http://localhost/ I see the home page with >> the facets from my external >> solr server listed on the left, if I click on one of the facets, >> I get the following : >> >> You have a nil object when you didn't expect it! >> The error occurred while evaluating nil.commit >> >> >> |/opt/appz/blacklight/rails/app/controllers/home_controller.rb:15:in `solr_params' >> >> >> /opt/appz/blacklight/rails/app/controllers/home_controller.rb:7:in `index'| >> >> >> >> the line with .commit is the Blacklight.commit line in >> solr_params, so for some reason Blacklight >> is nil. >> >> Any ideas >> On Wed, Mar 11, 2009 at 10:35 AM, Molly Pickral Cadieux >> > wrote: >> >> Vern, >> >> Just to make sure -- did you restart passenger or mongrel or >> whatever >> it is you are using? Changes to helpers seem to need an >> extra kick. >> >> Molly >> >> On Tue, Mar 10, 2009 at 2:51 PM, Vernon Chapman >> > wrote: >> > Molly, >> > >> > Thanks for your work on this, but unfortunately that didn't >> fix the >> > issue. I did an svn update on my checkout of blacklight >> and restarted >> > apache and I still get the same error. >> > >> > Vern >> > >> > Molly Pickral Cadieux wrote: >> >> >> >> Hi again, >> >> >> >> I went ahead and tried the fix on demo.blacklightopac.org >> , and it >> >> works great! So, Vernon, if you could grab the new version of >> >> catalog_helper.rb from subversion, I'm almost certain that >> will fix >> >> the bug you reported yesterday. >> >> >> >> Molly >> >> >> >> On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux >> >> > wrote: >> >> >> >>> >> >>> Hi everyone, >> >>> >> >>> I recently committed some changes to the repository that >> offer the >> >>> following features: >> >>> >> >>> 1. A drop-down list that allows users to specify how may >> results per >> >>> page to display (the previous default was 10 per page) >> >>> 2. "Previous" and "Next" links that allow a user to page >> through a >> >>> search result, as opposed to going back to the previous >> page to select >> >>> the previous or next item. >> >>> 3. Several rspec tests for the catalog controller and >> various models. >> >>> >> >>> This all works perfectly on my setup, but when I went to >> deploy the >> >>> changes on demo.blacklightopac.org >> , I ran into a routing issue. >> >>> Vernon also reported this error yesterday. The error is >> as follows: >> >>> >> >>> ------------ >> >>> >> >>> catalog_url failed to generate from {:commit=>nil, >> :action=>"show", >> >>> :id=>"agrecord.2008-11-27.1267669533", >> :controller=>"catalog", >> >>> :counter=>1}, expected: {:action=>"show", >> :controller=>"catalog"}, diff: >> >>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", >> :counter=>1} >> >>> >> >>> Extracted source (around line #21): >> >>> >> >>> 18: <% # main title container for doc partial >> view -%> >> >>> 19: >> >>> 20:
>> >>> 21:
<%= counter + 1 + @response.start.to_i >> %>. <%= >> >>> link_to_document document, :label=>:title, >> :extra_params=>{:counter => >> >>> counter + 1 + @response.start.to_i } %>
>> >>> 22:
>> >>> 23: >> >>> 24: >> >>> >> >>> ----------- >> >>> >> >>> Bess had reported a similar problem back in February >> (I've included >> >>> the message below). Unfortunately, I have been unable to >> reproduce >> >>> the reported problem on my Mac running the same versions >> of Ruby, >> >>> Rails, and Passenger. I would like to try Bess's >> recommended solution >> >>> if I can get a setup where I can reproduce the problem. >> In the mean >> >>> time, I'd love it if Vernon or others who may have seen >> the problem >> >>> could try modifying this file: >> >>> >> >>> rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb >> >>> >> >>> and change this line: >> >>> >> >>> link_to label, catalog_path(doc[:id], >> >>> params.merge(opts[:extra_params]).merge(:action=>nil, >> :commit=>nil)) >> >>> >> >>> to be: >> >>> >> >>> link_to label, catalog_path(doc[:id], >> >>> params.merge(opts[:extra_params]).merge(:action=>"show", >> >>> :commit=>nil)) >> >>> >> >>> and see if that fixes it. A restart of passenger will be >> necessary. >> >>> >> >>> If I can't get this worked out soon, I will revert the >> changes in the >> >>> repository and work through this again here. >> >>> >> >>> In any case, I wanted to let you all know that what is >> currently in >> >>> the repository may cause routing issues under certain >> environments. >> >>> I'll keep you all posted on progress. >> >>> >> >>> Molly >> >>> >> >>> >> >>> ---------- >> >>> >> >>> From eos8d at virginia.edu Sun Feb >> 1 17:25:35 2009 >> >>> From: eos8d at virginia.edu (Bess >> Sadler) >> >>> Date: Sun, 1 Feb 2009 17:25:35 -0500 >> >>> Subject: [Blacklight-development] demo app bug fix >> >>> Message-ID: >> > > >> >>> >> >>> Hey, folks. >> >>> >> >>> I've spent the last day or so playing pretty intensively >> with the demo >> >>> app. I fixed a couple of minor bugs in the installation >> instructions >> >>> (here: >> >>> >> http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) >> >>> and I've deployed the demo app several times, on several >> different >> >>> systems. Strangely, although I could get it running right >> away with >> >>> mongrel or webrick, I found that I could not get it >> running with >> >>> mod_passenger, which is our preferred method of running >> rails apps in >> >>> production. I think I tracked down the bug today, and you >> can see the >> >>> demo app running with mod_passenger as a virtual host at >> >>> http://demo.blacklightopac.org >> >>> >> >>> I was getting two errors. One, encountered whenever the >> index screen >> >>> was loaded, said: >> >>> >> >>> ActionView::TemplateError (catalog_url failed to generate >> from >> >>> {:action=>"index", :controller=>"catalog", :id=>"u1"}, >> expected: >> >>> {:action=>"show", :controller=>"catalog"}, diff: >> >>> {:action=>"show", :id=>"u1"}) on line #11 of >> vendor/plugins/blacklight/ >> >>> app/views/catalog/_index_partials/_default.html.erb: >> >>> 8: >> >>> 9:
>> >>> 10:
>> >>> 11: <%= link_to document.get(:id), >> catalog_path(document[:id], >> >>> params) %> >> >>> 12:
>> >>> 13:
>> >>> 14: >> >>> >> >>> >> >>> The other was generated whenever the show screen was >> loaded, and it >> >>> said: >> >>> >> >>> ActionView::TemplateError (catalog_index_url failed to >> generate from >> >>> {:action=>"show", :controller=>"catalog", :id=>nil}, >> expected: >> >>> {:action=>"index", :controller=>"catalog"}, diff: >> >>> {:action=>"index", :id=>nil}) on line #8 of >> vendor/plugins/blacklight/ >> >>> app/views/catalog/show.html.erb: >> >>> 5: >> >>> 6: >> >>> 7: <% @sidebar = capture do %> >> >>> 8:

<%= link_to '<< SEARCH', >> >>> catalog_index_path(params.merge(:id=>nil)) %>

>> >>> 9: <% end %> >> >>> >> >>> Turns out these were basically the same error. In both >> cases, we're >> >>> using the link_to function to create a link to another >> part of the >> >>> application, in this case from index to show, and from >> show to index. >> >>> Let's talk about the link from index to show as an example. >> >>> >> >>> <%= link_to document.get(:id), >> catalog_path(document[:id], params) %> >> >>> >> >>> That says create a link, and the text of the link is the >> id number of >> >>> the current document (say, u999), and the target of the >> link is >> >>> catalog_path(u999) which will evaluate to /catalog/u999. >> Incidentally, >> >>> the rails console was really valuable in helping me >> understand how >> >>> these worked, and you can play with it like this: >> >>> >> >>> paz:rails eos8d$ ./script/console >> >>> Loading development environment (Rails 2.2.2) >> >>> >> app.catalog_path("u999") >> >>> => "/catalog/u999" >> >>> >> >>> Anyway, I found that if I just changed the link_to >> statement to look >> >>> like this instead, removing the params object: >> >>> >> >>> <%= link_to document.get(:id), >> catalog_path(document[:id]) %> >> >>> >> >>> the error went away. However, this wasn't an acceptable >> solution, >> >>> because whenever a user performs a search or selects a >> facet, that >> >>> value goes in the params hash, and you want to keep it >> around when you >> >>> go from screen to screen, so that the user isn't always >> having to re- >> >>> enter their current search. Let's say you're on the index >> screen, and >> >>> you've entered a search and selected a facet. At that >> point your >> >>> params hash looks like this: >> >>> >> >>> {"commit"=>"search", "action"=>"index", "q"=>"poetry", >> >>> "f"=>{"language_facet"=>["English"]}, >> "controller"=>"catalog"} >> >>> >> >>> When you want to look at a detailed view for a specific >> item, you're >> >>> linking to the show action of your same catalog >> controller, and >> >>> passing it a specific document id. But you also want to >> hang onto the >> >>> fact that you've searched for poetry and limited to >> >>> language_facet:English, so you also pass the params >> object in. At >> >>> which point your link_to really looks like this: >> >>> >> >>> <%= link_to document.get(:id), >> >>> catalog_path >> >>> (:id >> >>> =>document[:id], :commit=>"search", :action=>"index", >> :q=>"poetry" ... >> >>> etc ... ) %> >> >>> >> >>> Do you see the problem? It took me awhile to notice it. >> I've just >> >>> called the show action, and then I also pass in a >> parameter of >> >>> action=index. It's not obvious, because catalog_path sort >> of obscures >> >>> the fact that you're calling the show action. >> Interestingly, webrick >> >>> and mongrel don't seem to care. I guess they just assume >> that even >> >>> though you've passed two values for action, you must >> really mean >> >>> action=show because that's what's defined for >> catalog_path. The fix is >> >>> easy, once you know what the problem is: >> >>> >> >>> <%= link_to document.get(:id), >> >>> >> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >> >>> >> >>> Now you're passing in a params object that looks like this: >> >>> >> >>> {"commit"=>"search", "action"=>"show", >> "id"=>"2008349843", >> >>> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", >> >>> "controller"=>"catalog"} >> >>> >> >>> So, all your user defined parameters, plus the id number >> of the >> >>> specific item you want to show, plus action=show, which >> is what >> >>> catalog_path expects. Problem solved. >> >>> >> >>> Sorry this is so long, but I learned a lot in fixing this >> bug and I >> >>> thought it might be helpful for someone else too. But if >> it just looks >> >>> like gibberish, ignore it. >> >>> >> >>> Cheers, >> >>> Bess >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >From goodieboy at gmail.com Sun Feb >> 1 17:59:36 2009 >> >>> From: goodieboy at gmail.com (Matt >> Mitchell) >> >>> Date: Sun, 1 Feb 2009 17:59:36 -0500 >> >>> Subject: [Blacklight-development] demo app bug fix >> >>> In-Reply-To: >> > > >> >>> References: >> > > >> >>> Message-ID: >> > > >> >>> >> >>> Seriously, that's a kickin' write up Bess. Great bug >> catch. And the docs >> >>> look so nice too! >> >>> >> >>> For the link_to fix here: >> >>> >> >>> <%= link_to document.get(:id), >> >>> >> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >> >>> >> >>> You should be able to do this too: >> >>> >> >>> <%= link_to document.get(:id), catalog_path(document[:id], >> >>> params.merge(:action=>nil)) %> >> >>> >> >>> I think this kind of thing will become a >> url-param-linking pattern in BL >> >>> and >> >>> might do well in a view helper. I think there's one in >> the UVa trunk, >> >>> called >> >>> link_to_document or something. We could then have one called >> >>> link_to_documents or link_back_to_search? The upshot of >> it being in a >> >>> helper >> >>> is that if someone overrides a view partial, they don't >> have to re-write >> >>> the >> >>> param merging stuff. Food for thought... >> >>> >> >>> Matt >> >>> >> >>> >> >> >> >> _______________________________________________ >> >> Blacklight-development mailing list >> >> Blacklight-development at rubyforge.org >> >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> >> >> > >> > _______________________________________________ >> > Blacklight-development mailing list >> > Blacklight-development at rubyforge.org >> >> > http://rubyforge.org/mailman/listinfo/blacklight-development >> > Blacklightopac Blog http://blacklightopac.org/ >> > >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From erikhatcher at mac.com Wed Mar 11 15:38:18 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Wed, 11 Mar 2009 15:38:18 -0400 Subject: [Blacklight-development] solr query for single document: standard or dismax? In-Reply-To: References: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> <9BB431CA-C6C0-4587-BB1B-3A5C73629DF4@stanford.edu> <419F32C3-02C8-4F20-B957-D6E2AB5DFF19@mac.com> <42F023C7-A62F-48B7-B484-5E751A437CE9@stanford.edu> Message-ID: On Mar 11, 2009, at 2:34 PM, Matt Mitchell wrote: > I'm going to (asap) update RSolr soon, and will integrate it all > into BL as well (if everyone here is OK with that). As soon as > that's done, making changes, or even hacking will be easier and less > likely to break when solr-ruby 1.0 is released. But yeah, I'd > recommend not hacking too much for the time being. +1 having a real app using the API as you and I envision it is important. the touchpoints should be minimal in Blacklight, eh? shouldn't really be any reason to hack at that layer anyway, as it'll be very simple parameter pass through, allowing Blacklight to cleanly parameterize things in Solr-speak (or wrap it however makes sense for Blacklight-speak). Erik From jamie at dang.com Wed Mar 11 15:43:04 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Wed, 11 Mar 2009 15:43:04 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <49B80EE6.9050401@gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> <49B80EE6.9050401@gmail.com> Message-ID: Vernon, if your IDs have periods in them, then the default route pattern-matching will view the first one as a delimiter for the format/ mime-type of the response. /foo/2/bar.xml /foo/2/bar.json So if you have /foo/2132.456/bar, the route will read the extension as ".456/bar" and you're in trouble. If the ids are under your control and periods don't matter, then just eliminate them. If you want periods in the ids, then you'll have to figure out some routing mojo or do some on-the-fly translation of the ids while they're used in rails and then translate them back when accessing solr. Jamie On Mar 11, 2009, at 3:20 PM, Vernon Chapman wrote: > Matt, > > That gut of yours seems to be right on point. > There seems to be an issue with the id having "." in the id > > I am going to go back and recreate the ids. > > Thanks > > Matt Mitchell wrote: >> My gut is telling me that this was never a BL problem, but an :id >> param format problem. If the format of the :id param does not match >> the route rules, the route helper will not work. I'd seriously >> consider re-formatting the id param to use something that doesn't >> use a . >> >> Matt >> >> On Wed, Mar 11, 2009 at 12:55 PM, Naomi Dushay >> > wrote: >> >> Vernon, >> >> Just for giggles, try this: >> >>> 4. copied the vendors/plugins/blacklight/lib/blacklight.rb to >>> my app/lib/blacklight.rb >> >> don't do this step >> >>> 5 - added Blacklight.solr.param_mappers[:select] = :dismax >>> to app/lib/blacklight.rb >> >> do this in place. >> >> I had trouble with my solr_helper refactoring when I needed to add >> and read add'l info in solr.yml; I couldn't get it to work when I >> did it in my local rails app -- I had to do it in vendor/ >> plugins/blacklight/lib >> >> >> If my "fix" works, then we need to discuss if this can be done >> differently and/or change the documentation accordingly. >> >> - Naomi >> >> >> On Mar 11, 2009, at 9:40 AM, Vernon Chapman wrote: >> >>> I am not sure what I am doing wrong so here is what I did >>> >>> 1 - checkout blacklight from svn using svn co http://....../trunk >>> blacklight >>> 2 - followed the direction to setup internal solr/jetty server >>> and index records >>> 3 - test the setup, everything works when I got to >>> http://localhost/ i see the new home page with text etc. >>> >>> Here is where I am getting KILLED >>> >>> I need to change the solr url , handlers and the controller for >>> my application so I >>> >>> 1 - modified solr.yml from http://127.0.0.1:8983/solr to >>> http://127.0.0.1:8983/solr/content (my solr is multicore) >>> 2 - copied the home_controller & catalog_controller to my >>> app/controllers >>> 3 - modified the catalog_controllers by replacing the :search >>> symbol with the name of the handler >>> on my solr server in this case :select >>> 4 - copied the vendors/plugins/blacklight/lib/blacklight.rb to >>> my app/lib/blacklight.rb >>> 5 - added Blacklight.solr.param_mappers[:select] = :dismax >>> to app/lib/blacklight.rb >>> 6 - restart passtenger ( touch rails/tmp/restart.txt) >>> 7 - restart apache2 (just to make sure passenger restarted) >>> >>> The fist time I access http://localhost/ I see the home page with >>> the facets from my external >>> solr server listed on the left, if I click on one of the facets, >>> I get the following : >>> >>> You have a nil object when you didn't expect it! >>> The error occurred while evaluating nil.commit >>> >>> >>> |/opt/appz/blacklight/rails/app/controllers/home_controller.rb: >>> 15:in `solr_params' >>> >>> >>> /opt/appz/blacklight/rails/app/controllers/home_controller.rb: >>> 7:in `index'| >>> >>> >>> the line with .commit is the Blacklight.commit line in >>> solr_params, so for some reason Blacklight >>> is nil. >>> >>> Any ideas >>> On Wed, Mar 11, 2009 at 10:35 AM, Molly Pickral Cadieux >>> > wrote: >>> >>> Vern, >>> >>> Just to make sure -- did you restart passenger or mongrel or >>> whatever >>> it is you are using? Changes to helpers seem to need an >>> extra kick. >>> >>> Molly >>> >>> On Tue, Mar 10, 2009 at 2:51 PM, Vernon Chapman >>> > >>> wrote: >>> > Molly, >>> > >>> > Thanks for your work on this, but unfortunately that didn't >>> fix the >>> > issue. I did an svn update on my checkout of blacklight >>> and restarted >>> > apache and I still get the same error. >>> > >>> > Vern >>> > >>> > Molly Pickral Cadieux wrote: >>> >> >>> >> Hi again, >>> >> >>> >> I went ahead and tried the fix on demo.blacklightopac.org >>> , and it >>> >> works great! So, Vernon, if you could grab the new >>> version of >>> >> catalog_helper.rb from subversion, I'm almost certain that >>> will fix >>> >> the bug you reported yesterday. >>> >> >>> >> Molly >>> >> >>> >> On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux >>> >> > wrote: >>> >> >>> >>> >>> >>> Hi everyone, >>> >>> >>> >>> I recently committed some changes to the repository that >>> offer the >>> >>> following features: >>> >>> >>> >>> 1. A drop-down list that allows users to specify how may >>> results per >>> >>> page to display (the previous default was 10 per page) >>> >>> 2. "Previous" and "Next" links that allow a user to page >>> through a >>> >>> search result, as opposed to going back to the previous >>> page to select >>> >>> the previous or next item. >>> >>> 3. Several rspec tests for the catalog controller and >>> various models. >>> >>> >>> >>> This all works perfectly on my setup, but when I went to >>> deploy the >>> >>> changes on demo.blacklightopac.org >>> , I ran into a routing issue. >>> >>> Vernon also reported this error yesterday. The error is >>> as follows: >>> >>> >>> >>> ------------ >>> >>> >>> >>> catalog_url failed to generate from {:commit=>nil, >>> :action=>"show", >>> >>> :id=>"agrecord.2008-11-27.1267669533", >>> :controller=>"catalog", >>> >>> :counter=>1}, expected: {:action=>"show", >>> :controller=>"catalog"}, diff: >>> >>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", >>> :counter=>1} >>> >>> >>> >>> Extracted source (around line #21): >>> >>> >>> >>> 18: <% # main title container for doc partial >>> view -%> >>> >>> 19: >>> >>> 20:
>>> >>> 21:
<%= counter + 1 + @response.start.to_i >>> %>. <%= >>> >>> link_to_document document, :label=>:title, >>> :extra_params=>{:counter => >>> >>> counter + 1 + @response.start.to_i } %>
>>> >>> 22:
>>> >>> 23: >>> >>> 24: >>> >>> >>> >>> ----------- >>> >>> >>> >>> Bess had reported a similar problem back in February >>> (I've included >>> >>> the message below). Unfortunately, I have been unable to >>> reproduce >>> >>> the reported problem on my Mac running the same versions >>> of Ruby, >>> >>> Rails, and Passenger. I would like to try Bess's >>> recommended solution >>> >>> if I can get a setup where I can reproduce the problem. >>> In the mean >>> >>> time, I'd love it if Vernon or others who may have seen >>> the problem >>> >>> could try modifying this file: >>> >>> >>> >>> rails/vendor/plugins/blacklight/app/helpers/ >>> catalog_helper.rb >>> >>> >>> >>> and change this line: >>> >>> >>> >>> link_to label, catalog_path(doc[:id], >>> >>> params.merge(opts[:extra_params]).merge(:action=>nil, >>> :commit=>nil)) >>> >>> >>> >>> to be: >>> >>> >>> >>> link_to label, catalog_path(doc[:id], >>> >>> params.merge(opts[:extra_params]).merge(:action=>"show", >>> >>> :commit=>nil)) >>> >>> >>> >>> and see if that fixes it. A restart of passenger will be >>> necessary. >>> >>> >>> >>> If I can't get this worked out soon, I will revert the >>> changes in the >>> >>> repository and work through this again here. >>> >>> >>> >>> In any case, I wanted to let you all know that what is >>> currently in >>> >>> the repository may cause routing issues under certain >>> environments. >>> >>> I'll keep you all posted on progress. >>> >>> >>> >>> Molly >>> >>> >>> >>> >>> >>> ---------- >>> >>> >>> >>> From eos8d at virginia.edu Sun Feb >>> 1 17:25:35 2009 >>> >>> From: eos8d at virginia.edu (Bess >>> Sadler) >>> >>> Date: Sun, 1 Feb 2009 17:25:35 -0500 >>> >>> Subject: [Blacklight-development] demo app bug fix >>> >>> Message-ID: >>> >> > >>> >>> >>> >>> Hey, folks. >>> >>> >>> >>> I've spent the last day or so playing pretty intensively >>> with the demo >>> >>> app. I fixed a couple of minor bugs in the installation >>> instructions >>> >>> (here: >>> >>> >>> http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) >>> >>> and I've deployed the demo app several times, on several >>> different >>> >>> systems. Strangely, although I could get it running right >>> away with >>> >>> mongrel or webrick, I found that I could not get it >>> running with >>> >>> mod_passenger, which is our preferred method of running >>> rails apps in >>> >>> production. I think I tracked down the bug today, and you >>> can see the >>> >>> demo app running with mod_passenger as a virtual host at >>> >>> http://demo.blacklightopac.org >>> >>> >>> >>> I was getting two errors. One, encountered whenever the >>> index screen >>> >>> was loaded, said: >>> >>> >>> >>> ActionView::TemplateError (catalog_url failed to generate >>> from >>> >>> {:action=>"index", :controller=>"catalog", :id=>"u1"}, >>> expected: >>> >>> {:action=>"show", :controller=>"catalog"}, diff: >>> >>> {:action=>"show", :id=>"u1"}) on line #11 of >>> vendor/plugins/blacklight/ >>> >>> app/views/catalog/_index_partials/_default.html.erb: >>> >>> 8: >>> >>> 9:
>>> >>> 10:
>>> >>> 11: <%= link_to document.get(:id), >>> catalog_path(document[:id], >>> >>> params) %> >>> >>> 12:
>>> >>> 13:
>>> >>> 14: >>> >>> >>> >>> >>> >>> The other was generated whenever the show screen was >>> loaded, and it >>> >>> said: >>> >>> >>> >>> ActionView::TemplateError (catalog_index_url failed to >>> generate from >>> >>> {:action=>"show", :controller=>"catalog", :id=>nil}, >>> expected: >>> >>> {:action=>"index", :controller=>"catalog"}, diff: >>> >>> {:action=>"index", :id=>nil}) on line #8 of >>> vendor/plugins/blacklight/ >>> >>> app/views/catalog/show.html.erb: >>> >>> 5: >>> >>> 6: >>> >>> 7: <% @sidebar = capture do %> >>> >>> 8:

<%= link_to '<< SEARCH', >>> >>> catalog_index_path(params.merge(:id=>nil)) %>

>>> >>> 9: <% end %> >>> >>> >>> >>> Turns out these were basically the same error. In both >>> cases, we're >>> >>> using the link_to function to create a link to another >>> part of the >>> >>> application, in this case from index to show, and from >>> show to index. >>> >>> Let's talk about the link from index to show as an >>> example. >>> >>> >>> >>> <%= link_to document.get(:id), >>> catalog_path(document[:id], params) %> >>> >>> >>> >>> That says create a link, and the text of the link is the >>> id number of >>> >>> the current document (say, u999), and the target of the >>> link is >>> >>> catalog_path(u999) which will evaluate to /catalog/u999. >>> Incidentally, >>> >>> the rails console was really valuable in helping me >>> understand how >>> >>> these worked, and you can play with it like this: >>> >>> >>> >>> paz:rails eos8d$ ./script/console >>> >>> Loading development environment (Rails 2.2.2) >>> >>> >> app.catalog_path("u999") >>> >>> => "/catalog/u999" >>> >>> >>> >>> Anyway, I found that if I just changed the link_to >>> statement to look >>> >>> like this instead, removing the params object: >>> >>> >>> >>> <%= link_to document.get(:id), >>> catalog_path(document[:id]) %> >>> >>> >>> >>> the error went away. However, this wasn't an acceptable >>> solution, >>> >>> because whenever a user performs a search or selects a >>> facet, that >>> >>> value goes in the params hash, and you want to keep it >>> around when you >>> >>> go from screen to screen, so that the user isn't always >>> having to re- >>> >>> enter their current search. Let's say you're on the index >>> screen, and >>> >>> you've entered a search and selected a facet. At that >>> point your >>> >>> params hash looks like this: >>> >>> >>> >>> {"commit"=>"search", "action"=>"index", >>> "q"=>"poetry", >>> >>> "f"=>{"language_facet"=>["English"]}, >>> "controller"=>"catalog"} >>> >>> >>> >>> When you want to look at a detailed view for a specific >>> item, you're >>> >>> linking to the show action of your same catalog >>> controller, and >>> >>> passing it a specific document id. But you also want to >>> hang onto the >>> >>> fact that you've searched for poetry and limited to >>> >>> language_facet:English, so you also pass the params >>> object in. At >>> >>> which point your link_to really looks like this: >>> >>> >>> >>> <%= link_to document.get(:id), >>> >>> catalog_path >>> >>> (:id >>> >>> =>document[:id], :commit=>"search", :action=>"index", >>> :q=>"poetry" ... >>> >>> etc ... ) %> >>> >>> >>> >>> Do you see the problem? It took me awhile to notice it. >>> I've just >>> >>> called the show action, and then I also pass in a >>> parameter of >>> >>> action=index. It's not obvious, because catalog_path sort >>> of obscures >>> >>> the fact that you're calling the show action. >>> Interestingly, webrick >>> >>> and mongrel don't seem to care. I guess they just assume >>> that even >>> >>> though you've passed two values for action, you must >>> really mean >>> >>> action=show because that's what's defined for >>> catalog_path. The fix is >>> >>> easy, once you know what the problem is: >>> >>> >>> >>> <%= link_to document.get(:id), >>> >>> >>> >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >>> >>> >>> >>> Now you're passing in a params object that looks like >>> this: >>> >>> >>> >>> {"commit"=>"search", "action"=>"show", >>> "id"=>"2008349843", >>> >>> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", >>> >>> "controller"=>"catalog"} >>> >>> >>> >>> So, all your user defined parameters, plus the id number >>> of the >>> >>> specific item you want to show, plus action=show, which >>> is what >>> >>> catalog_path expects. Problem solved. >>> >>> >>> >>> Sorry this is so long, but I learned a lot in fixing this >>> bug and I >>> >>> thought it might be helpful for someone else too. But if >>> it just looks >>> >>> like gibberish, ignore it. >>> >>> >>> >>> Cheers, >>> >>> Bess >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >From goodieboy at gmail.com Sun Feb >>> 1 17:59:36 2009 >>> >>> From: goodieboy at gmail.com (Matt >>> Mitchell) >>> >>> Date: Sun, 1 Feb 2009 17:59:36 -0500 >>> >>> Subject: [Blacklight-development] demo app bug fix >>> >>> In-Reply-To: >>> >> > >>> >>> References: >>> >> > >>> >>> Message-ID: >>> >> >> >> >>> >>> >>> >>> Seriously, that's a kickin' write up Bess. Great bug >>> catch. And the docs >>> >>> look so nice too! >>> >>> >>> >>> For the link_to fix here: >>> >>> >>> >>> <%= link_to document.get(:id), >>> >>> >>> >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >>> >>> >>> >>> You should be able to do this too: >>> >>> >>> >>> <%= link_to document.get(:id), >>> catalog_path(document[:id], >>> >>> params.merge(:action=>nil)) %> >>> >>> >>> >>> I think this kind of thing will become a >>> url-param-linking pattern in BL >>> >>> and >>> >>> might do well in a view helper. I think there's one in >>> the UVa trunk, >>> >>> called >>> >>> link_to_document or something. We could then have one >>> called >>> >>> link_to_documents or link_back_to_search? The upshot of >>> it being in a >>> >>> helper >>> >>> is that if someone overrides a view partial, they don't >>> have to re-write >>> >>> the >>> >>> param merging stuff. Food for thought... >>> >>> >>> >>> Matt >>> >>> >>> >>> >>> >> >>> >> _______________________________________________ >>> >> Blacklight-development mailing list >>> >> Blacklight-development at rubyforge.org >>> >>> >> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >> Blacklightopac Blog http://blacklightopac.org/ >>> >> >>> >> >>> > >>> > _______________________________________________ >>> > Blacklight-development mailing list >>> > Blacklight-development at rubyforge.org >>> >>> > http://rubyforge.org/mailman/listinfo/blacklight- >>> development >>> > Blacklightopac Blog http://blacklightopac.org/ >>> > >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From erikhatcher at mac.com Wed Mar 11 15:44:41 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Wed, 11 Mar 2009 15:44:41 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <49B80EE6.9050401@gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> <49B80EE6.9050401@gmail.com> Message-ID: This is disturbing... Rails (and Blacklight) should not impose this kind of constraint on what gets put into Solr. What are the solutions to this to allow id's to be whatever someone desires? It's common-place to have id's like "http://lucene.apache.org/java/2_4_0/queryparsersyntax.html ". Which is, for example, the actual id we use for the first hit to this query: Erik On Mar 11, 2009, at 3:20 PM, Vernon Chapman wrote: > Matt, > > That gut of yours seems to be right on point. > There seems to be an issue with the id having "." in the id > > I am going to go back and recreate the ids. > > Thanks > > Matt Mitchell wrote: >> My gut is telling me that this was never a BL problem, but an :id >> param format problem. If the format of the :id param does not match >> the route rules, the route helper will not work. I'd seriously >> consider re-formatting the id param to use something that doesn't >> use a . >> >> Matt >> >> On Wed, Mar 11, 2009 at 12:55 PM, Naomi Dushay >> > wrote: >> >> Vernon, >> >> Just for giggles, try this: >> >>> 4. copied the vendors/plugins/blacklight/lib/blacklight.rb to >>> my app/lib/blacklight.rb >> >> don't do this step >> >>> 5 - added Blacklight.solr.param_mappers[:select] = :dismax >>> to app/lib/blacklight.rb >> >> do this in place. >> >> I had trouble with my solr_helper refactoring when I needed to add >> and read add'l info in solr.yml; I couldn't get it to work when I >> did it in my local rails app -- I had to do it in vendor/ >> plugins/blacklight/lib >> >> >> If my "fix" works, then we need to discuss if this can be done >> differently and/or change the documentation accordingly. >> >> - Naomi >> >> >> On Mar 11, 2009, at 9:40 AM, Vernon Chapman wrote: >> >>> I am not sure what I am doing wrong so here is what I did >>> >>> 1 - checkout blacklight from svn using svn co http://....../trunk >>> blacklight >>> 2 - followed the direction to setup internal solr/jetty server >>> and index records >>> 3 - test the setup, everything works when I got to >>> http://localhost/ i see the new home page with text etc. >>> >>> Here is where I am getting KILLED >>> >>> I need to change the solr url , handlers and the controller for >>> my application so I >>> >>> 1 - modified solr.yml from http://127.0.0.1:8983/solr to >>> http://127.0.0.1:8983/solr/content (my solr is multicore) >>> 2 - copied the home_controller & catalog_controller to my >>> app/controllers >>> 3 - modified the catalog_controllers by replacing the :search >>> symbol with the name of the handler >>> on my solr server in this case :select >>> 4 - copied the vendors/plugins/blacklight/lib/blacklight.rb to >>> my app/lib/blacklight.rb >>> 5 - added Blacklight.solr.param_mappers[:select] = :dismax >>> to app/lib/blacklight.rb >>> 6 - restart passtenger ( touch rails/tmp/restart.txt) >>> 7 - restart apache2 (just to make sure passenger restarted) >>> >>> The fist time I access http://localhost/ I see the home page with >>> the facets from my external >>> solr server listed on the left, if I click on one of the facets, >>> I get the following : >>> >>> You have a nil object when you didn't expect it! >>> The error occurred while evaluating nil.commit >>> >>> >>> |/opt/appz/blacklight/rails/app/controllers/home_controller.rb: >>> 15:in `solr_params' >>> >>> >>> /opt/appz/blacklight/rails/app/controllers/home_controller.rb: >>> 7:in `index'| >>> >>> >>> the line with .commit is the Blacklight.commit line in >>> solr_params, so for some reason Blacklight >>> is nil. >>> >>> Any ideas >>> On Wed, Mar 11, 2009 at 10:35 AM, Molly Pickral Cadieux >>> > wrote: >>> >>> Vern, >>> >>> Just to make sure -- did you restart passenger or mongrel or >>> whatever >>> it is you are using? Changes to helpers seem to need an >>> extra kick. >>> >>> Molly >>> >>> On Tue, Mar 10, 2009 at 2:51 PM, Vernon Chapman >>> > >>> wrote: >>> > Molly, >>> > >>> > Thanks for your work on this, but unfortunately that didn't >>> fix the >>> > issue. I did an svn update on my checkout of blacklight >>> and restarted >>> > apache and I still get the same error. >>> > >>> > Vern >>> > >>> > Molly Pickral Cadieux wrote: >>> >> >>> >> Hi again, >>> >> >>> >> I went ahead and tried the fix on demo.blacklightopac.org >>> , and it >>> >> works great! So, Vernon, if you could grab the new >>> version of >>> >> catalog_helper.rb from subversion, I'm almost certain that >>> will fix >>> >> the bug you reported yesterday. >>> >> >>> >> Molly >>> >> >>> >> On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux >>> >> > wrote: >>> >> >>> >>> >>> >>> Hi everyone, >>> >>> >>> >>> I recently committed some changes to the repository that >>> offer the >>> >>> following features: >>> >>> >>> >>> 1. A drop-down list that allows users to specify how may >>> results per >>> >>> page to display (the previous default was 10 per page) >>> >>> 2. "Previous" and "Next" links that allow a user to page >>> through a >>> >>> search result, as opposed to going back to the previous >>> page to select >>> >>> the previous or next item. >>> >>> 3. Several rspec tests for the catalog controller and >>> various models. >>> >>> >>> >>> This all works perfectly on my setup, but when I went to >>> deploy the >>> >>> changes on demo.blacklightopac.org >>> , I ran into a routing issue. >>> >>> Vernon also reported this error yesterday. The error is >>> as follows: >>> >>> >>> >>> ------------ >>> >>> >>> >>> catalog_url failed to generate from {:commit=>nil, >>> :action=>"show", >>> >>> :id=>"agrecord.2008-11-27.1267669533", >>> :controller=>"catalog", >>> >>> :counter=>1}, expected: {:action=>"show", >>> :controller=>"catalog"}, diff: >>> >>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", >>> :counter=>1} >>> >>> >>> >>> Extracted source (around line #21): >>> >>> >>> >>> 18: <% # main title container for doc partial >>> view -%> >>> >>> 19: >>> >>> 20:
>>> >>> 21:
<%= counter + 1 + @response.start.to_i >>> %>. <%= >>> >>> link_to_document document, :label=>:title, >>> :extra_params=>{:counter => >>> >>> counter + 1 + @response.start.to_i } %>
>>> >>> 22:
>>> >>> 23: >>> >>> 24: >>> >>> >>> >>> ----------- >>> >>> >>> >>> Bess had reported a similar problem back in February >>> (I've included >>> >>> the message below). Unfortunately, I have been unable to >>> reproduce >>> >>> the reported problem on my Mac running the same versions >>> of Ruby, >>> >>> Rails, and Passenger. I would like to try Bess's >>> recommended solution >>> >>> if I can get a setup where I can reproduce the problem. >>> In the mean >>> >>> time, I'd love it if Vernon or others who may have seen >>> the problem >>> >>> could try modifying this file: >>> >>> >>> >>> rails/vendor/plugins/blacklight/app/helpers/ >>> catalog_helper.rb >>> >>> >>> >>> and change this line: >>> >>> >>> >>> link_to label, catalog_path(doc[:id], >>> >>> params.merge(opts[:extra_params]).merge(:action=>nil, >>> :commit=>nil)) >>> >>> >>> >>> to be: >>> >>> >>> >>> link_to label, catalog_path(doc[:id], >>> >>> params.merge(opts[:extra_params]).merge(:action=>"show", >>> >>> :commit=>nil)) >>> >>> >>> >>> and see if that fixes it. A restart of passenger will be >>> necessary. >>> >>> >>> >>> If I can't get this worked out soon, I will revert the >>> changes in the >>> >>> repository and work through this again here. >>> >>> >>> >>> In any case, I wanted to let you all know that what is >>> currently in >>> >>> the repository may cause routing issues under certain >>> environments. >>> >>> I'll keep you all posted on progress. >>> >>> >>> >>> Molly >>> >>> >>> >>> >>> >>> ---------- >>> >>> >>> >>> From eos8d at virginia.edu Sun Feb >>> 1 17:25:35 2009 >>> >>> From: eos8d at virginia.edu (Bess >>> Sadler) >>> >>> Date: Sun, 1 Feb 2009 17:25:35 -0500 >>> >>> Subject: [Blacklight-development] demo app bug fix >>> >>> Message-ID: >>> >> > >>> >>> >>> >>> Hey, folks. >>> >>> >>> >>> I've spent the last day or so playing pretty intensively >>> with the demo >>> >>> app. I fixed a couple of minor bugs in the installation >>> instructions >>> >>> (here: >>> >>> >>> http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) >>> >>> and I've deployed the demo app several times, on several >>> different >>> >>> systems. Strangely, although I could get it running right >>> away with >>> >>> mongrel or webrick, I found that I could not get it >>> running with >>> >>> mod_passenger, which is our preferred method of running >>> rails apps in >>> >>> production. I think I tracked down the bug today, and you >>> can see the >>> >>> demo app running with mod_passenger as a virtual host at >>> >>> http://demo.blacklightopac.org >>> >>> >>> >>> I was getting two errors. One, encountered whenever the >>> index screen >>> >>> was loaded, said: >>> >>> >>> >>> ActionView::TemplateError (catalog_url failed to generate >>> from >>> >>> {:action=>"index", :controller=>"catalog", :id=>"u1"}, >>> expected: >>> >>> {:action=>"show", :controller=>"catalog"}, diff: >>> >>> {:action=>"show", :id=>"u1"}) on line #11 of >>> vendor/plugins/blacklight/ >>> >>> app/views/catalog/_index_partials/_default.html.erb: >>> >>> 8: >>> >>> 9:
>>> >>> 10:
>>> >>> 11: <%= link_to document.get(:id), >>> catalog_path(document[:id], >>> >>> params) %> >>> >>> 12:
>>> >>> 13:
>>> >>> 14: >>> >>> >>> >>> >>> >>> The other was generated whenever the show screen was >>> loaded, and it >>> >>> said: >>> >>> >>> >>> ActionView::TemplateError (catalog_index_url failed to >>> generate from >>> >>> {:action=>"show", :controller=>"catalog", :id=>nil}, >>> expected: >>> >>> {:action=>"index", :controller=>"catalog"}, diff: >>> >>> {:action=>"index", :id=>nil}) on line #8 of >>> vendor/plugins/blacklight/ >>> >>> app/views/catalog/show.html.erb: >>> >>> 5: >>> >>> 6: >>> >>> 7: <% @sidebar = capture do %> >>> >>> 8:

<%= link_to '<< SEARCH', >>> >>> catalog_index_path(params.merge(:id=>nil)) %>

>>> >>> 9: <% end %> >>> >>> >>> >>> Turns out these were basically the same error. In both >>> cases, we're >>> >>> using the link_to function to create a link to another >>> part of the >>> >>> application, in this case from index to show, and from >>> show to index. >>> >>> Let's talk about the link from index to show as an >>> example. >>> >>> >>> >>> <%= link_to document.get(:id), >>> catalog_path(document[:id], params) %> >>> >>> >>> >>> That says create a link, and the text of the link is the >>> id number of >>> >>> the current document (say, u999), and the target of the >>> link is >>> >>> catalog_path(u999) which will evaluate to /catalog/u999. >>> Incidentally, >>> >>> the rails console was really valuable in helping me >>> understand how >>> >>> these worked, and you can play with it like this: >>> >>> >>> >>> paz:rails eos8d$ ./script/console >>> >>> Loading development environment (Rails 2.2.2) >>> >>> >> app.catalog_path("u999") >>> >>> => "/catalog/u999" >>> >>> >>> >>> Anyway, I found that if I just changed the link_to >>> statement to look >>> >>> like this instead, removing the params object: >>> >>> >>> >>> <%= link_to document.get(:id), >>> catalog_path(document[:id]) %> >>> >>> >>> >>> the error went away. However, this wasn't an acceptable >>> solution, >>> >>> because whenever a user performs a search or selects a >>> facet, that >>> >>> value goes in the params hash, and you want to keep it >>> around when you >>> >>> go from screen to screen, so that the user isn't always >>> having to re- >>> >>> enter their current search. Let's say you're on the index >>> screen, and >>> >>> you've entered a search and selected a facet. At that >>> point your >>> >>> params hash looks like this: >>> >>> >>> >>> {"commit"=>"search", "action"=>"index", >>> "q"=>"poetry", >>> >>> "f"=>{"language_facet"=>["English"]}, >>> "controller"=>"catalog"} >>> >>> >>> >>> When you want to look at a detailed view for a specific >>> item, you're >>> >>> linking to the show action of your same catalog >>> controller, and >>> >>> passing it a specific document id. But you also want to >>> hang onto the >>> >>> fact that you've searched for poetry and limited to >>> >>> language_facet:English, so you also pass the params >>> object in. At >>> >>> which point your link_to really looks like this: >>> >>> >>> >>> <%= link_to document.get(:id), >>> >>> catalog_path >>> >>> (:id >>> >>> =>document[:id], :commit=>"search", :action=>"index", >>> :q=>"poetry" ... >>> >>> etc ... ) %> >>> >>> >>> >>> Do you see the problem? It took me awhile to notice it. >>> I've just >>> >>> called the show action, and then I also pass in a >>> parameter of >>> >>> action=index. It's not obvious, because catalog_path sort >>> of obscures >>> >>> the fact that you're calling the show action. >>> Interestingly, webrick >>> >>> and mongrel don't seem to care. I guess they just assume >>> that even >>> >>> though you've passed two values for action, you must >>> really mean >>> >>> action=show because that's what's defined for >>> catalog_path. The fix is >>> >>> easy, once you know what the problem is: >>> >>> >>> >>> <%= link_to document.get(:id), >>> >>> >>> >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >>> >>> >>> >>> Now you're passing in a params object that looks like >>> this: >>> >>> >>> >>> {"commit"=>"search", "action"=>"show", >>> "id"=>"2008349843", >>> >>> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", >>> >>> "controller"=>"catalog"} >>> >>> >>> >>> So, all your user defined parameters, plus the id number >>> of the >>> >>> specific item you want to show, plus action=show, which >>> is what >>> >>> catalog_path expects. Problem solved. >>> >>> >>> >>> Sorry this is so long, but I learned a lot in fixing this >>> bug and I >>> >>> thought it might be helpful for someone else too. But if >>> it just looks >>> >>> like gibberish, ignore it. >>> >>> >>> >>> Cheers, >>> >>> Bess >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >From goodieboy at gmail.com Sun Feb >>> 1 17:59:36 2009 >>> >>> From: goodieboy at gmail.com (Matt >>> Mitchell) >>> >>> Date: Sun, 1 Feb 2009 17:59:36 -0500 >>> >>> Subject: [Blacklight-development] demo app bug fix >>> >>> In-Reply-To: >>> >> > >>> >>> References: >>> >> > >>> >>> Message-ID: >>> >> >> >> >>> >>> >>> >>> Seriously, that's a kickin' write up Bess. Great bug >>> catch. And the docs >>> >>> look so nice too! >>> >>> >>> >>> For the link_to fix here: >>> >>> >>> >>> <%= link_to document.get(:id), >>> >>> >>> >>> catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >>> >>> >>> >>> You should be able to do this too: >>> >>> >>> >>> <%= link_to document.get(:id), >>> catalog_path(document[:id], >>> >>> params.merge(:action=>nil)) %> >>> >>> >>> >>> I think this kind of thing will become a >>> url-param-linking pattern in BL >>> >>> and >>> >>> might do well in a view helper. I think there's one in >>> the UVa trunk, >>> >>> called >>> >>> link_to_document or something. We could then have one >>> called >>> >>> link_to_documents or link_back_to_search? The upshot of >>> it being in a >>> >>> helper >>> >>> is that if someone overrides a view partial, they don't >>> have to re-write >>> >>> the >>> >>> param merging stuff. Food for thought... >>> >>> >>> >>> Matt >>> >>> >>> >>> >>> >> >>> >> _______________________________________________ >>> >> Blacklight-development mailing list >>> >> Blacklight-development at rubyforge.org >>> >>> >> http://rubyforge.org/mailman/listinfo/blacklight-development >>> >> Blacklightopac Blog http://blacklightopac.org/ >>> >> >>> >> >>> > >>> > _______________________________________________ >>> > Blacklight-development mailing list >>> > Blacklight-development at rubyforge.org >>> >>> > http://rubyforge.org/mailman/listinfo/blacklight- >>> development >>> > Blacklightopac Blog http://blacklightopac.org/ >>> > >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From ndushay at stanford.edu Wed Mar 11 16:20:17 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Wed, 11 Mar 2009 13:20:17 -0700 Subject: [Blacklight-development] solr query for single document: standard or dismax? In-Reply-To: References: <8376BBCB-6A39-4F12-ACEE-ABB7F5746E94@stanford.edu> <2757E103-7F4C-4DCE-9A48-6FE82838D68E@mac.com> <9BB431CA-C6C0-4587-BB1B-3A5C73629DF4@stanford.edu> <419F32C3-02C8-4F20-B957-D6E2AB5DFF19@mac.com> <42F023C7-A62F-48B7-B484-5E751A437CE9@stanford.edu> Message-ID: <25FA0049-2A80-449C-A146-58D618CAFEDC@stanford.edu> On Mar 11, 2009, at 11:34 AM, Matt Mitchell wrote: > Naomi, > > I'm going to (asap) update RSolr soon, and will integrate it all > into BL as well (if everyone here is OK with that). As soon as > that's done, making changes, or even hacking will be easier and less > likely to break when solr-ruby 1.0 is released. But yeah, I'd > recommend not hacking too much for the time being. +1 I am trying to get my solr_helper code checked in in the next few days or so. I'm slowed up with n00b-ness, but making good progress. Please check in w me before you start to make mods to the controllers .... - Naomi From rossfsinger at gmail.com Wed Mar 11 21:49:47 2009 From: rossfsinger at gmail.com (Ross Singer) Date: Wed, 11 Mar 2009 20:49:47 -0500 Subject: [Blacklight-development] routing problem In-Reply-To: References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> <49B80EE6.9050401@gmail.com> Message-ID: <23b83f160903111849n73fe8717n5f6cc0161ee9b8dd@mail.gmail.com> Well, this could probably be fixed in routes.rb somehow. I haven't messed with Rails routes in forever, but it seems like this could be hacked around if it's a huge problem (which evidently, it's not). This is, though, one of those things that would be useful for the docs -- how to construct a good, composite ID that works well with BL and Solr. For example, for the Jangle hack, I was using IDs that were ":" delimited (foo:1234 where foo represents the source) -- but that is awkward for Solr (well, specifically, for Lucene), since you have to escape the colon when requesting by ID. So what are good id delimiter characters? -Ross. On Wed, Mar 11, 2009 at 2:44 PM, Erik Hatcher wrote: > This is disturbing... Rails (and Blacklight) should not impose this kind of > constraint on what gets put into Solr. > > What are the solutions to this to allow id's to be whatever someone desires? > > It's common-place to have id's like > "http://lucene.apache.org/java/2_4_0/queryparsersyntax.html". ?Which is, for > example, the actual id we use for the first hit to this query: > ? > > ? ? ? ?Erik > > On Mar 11, 2009, at 3:20 PM, Vernon Chapman wrote: > >> Matt, >> >> That gut of yours seems to be right on point. >> There seems to be an issue with the id having "." in the id >> >> I am going to go back and recreate the ids. >> >> Thanks >> >> Matt Mitchell wrote: >>> >>> My gut is telling me that this was never a BL problem, but an :id param >>> format problem. If the format of the :id param does not match the route >>> rules, the route helper will not work. I'd seriously consider re-formatting >>> the id param to use something that doesn't use a . >>> >>> Matt >>> >>> On Wed, Mar 11, 2009 at 12:55 PM, Naomi Dushay >> > wrote: >>> >>> ? Vernon, >>> >>> ? Just for giggles, try this: >>> >>>> ? 4. ?copied the vendors/plugins/blacklight/lib/blacklight.rb ?to >>>> ? my app/lib/blacklight.rb >>> >>> ? don't do this step >>> >>>> ? 5 - added Blacklight.solr.param_mappers[:select] = :dismax ?to >>>> app/lib/blacklight.rb >>> >>> ? do this in place. >>> >>> ? I had trouble with my solr_helper refactoring when I needed to add >>> ? and read add'l info in solr.yml; ?I couldn't get it to work when I >>> ? did it in my local rails app -- I had to do it in >>> vendor/plugins/blacklight/lib >>> >>> >>> ? If my "fix" works, then we need to discuss if this can be done >>> ? differently and/or change the documentation accordingly. >>> >>> ? - Naomi >>> >>> >>> ? On Mar 11, 2009, at 9:40 AM, Vernon Chapman wrote: >>> >>>> ? I am not sure what I am doing wrong so here is what I did >>>> >>>> ? 1 - checkout blacklight from svn using svn co http://....../trunk >>>> ? blacklight >>>> ? 2 - followed the direction to setup internal solr/jetty server >>>> ? and index records >>>> ? 3 - test the setup, everything works when I got to >>>> ? http://localhost/ i see the new home page with text etc. >>>> >>>> ? Here is where I am getting KILLED >>>> >>>> ? I need to change the solr url , handlers and the controller for >>>> ? my application so I >>>> >>>> ? 1 - modified solr.yml from http://127.0.0.1:8983/solr to >>>> ? http://127.0.0.1:8983/solr/content (my solr is multicore) >>>> ? 2 - copied the home_controller & catalog_controller to my >>>> ? app/controllers >>>> ? 3 - modified the catalog_controllers by replacing the :search >>>> ? symbol with the name of the handler >>>> ? ? ? ?on my solr server in this case :select >>>> ? 4 - copied the vendors/plugins/blacklight/lib/blacklight.rb ?to >>>> ? my app/lib/blacklight.rb >>>> ? 5 - added Blacklight.solr.param_mappers[:select] = :dismax ?to >>>> app/lib/blacklight.rb >>>> ? 6 - restart passtenger ( touch rails/tmp/restart.txt) >>>> ? 7 - restart apache2 (just to make sure passenger restarted) >>>> >>>> ? The fist time I access http://localhost/ I see the home page with >>>> ? the facets from my external >>>> ? solr server listed on the left, if I click on one of the facets, >>>> ? I get the following : >>>> >>>> ? You have a nil object when you didn't expect it! >>>> ? The error occurred while evaluating nil.commit >>>> >>>> >>>> ? |/opt/appz/blacklight/rails/app/controllers/home_controller.rb:15:in >>>> `solr_params' >>>> >>>> >>>> ? /opt/appz/blacklight/rails/app/controllers/home_controller.rb:7:in >>>> `index'| >>>> >>>> >>>> ? ? ? ? ? ? the line with .commit is the Blacklight.commit line in >>>> ? solr_params, so for some reason Blacklight >>>> ? is nil. >>>> >>>> ? Any ideas >>>> ? On Wed, Mar 11, 2009 at 10:35 AM, Molly Pickral Cadieux >>>> ? > wrote: >>>> >>>> ? ? ? Vern, >>>> >>>> ? ? ? Just to make sure -- did you restart passenger or mongrel or >>>> ? ? ? whatever >>>> ? ? ? it is you are using? ?Changes to helpers seem to need an >>>> ? ? ? extra kick. >>>> >>>> ? ? ? Molly >>>> >>>> ? ? ? On Tue, Mar 10, 2009 at 2:51 PM, Vernon Chapman >>>> ? ? ? > wrote: >>>> ? ? ? > Molly, >>>> ? ? ? > >>>> ? ? ? > Thanks for your work on this, but unfortunately that didn't >>>> ? ? ? fix the >>>> ? ? ? > issue. I did an svn update on my checkout of blacklight >>>> ? ? ? ?and restarted >>>> ? ? ? > apache and I still get the same error. >>>> ? ? ? > >>>> ? ? ? > Vern >>>> ? ? ? > >>>> ? ? ? > Molly Pickral Cadieux wrote: >>>> ? ? ? >> >>>> ? ? ? >> Hi again, >>>> ? ? ? >> >>>> ? ? ? >> I went ahead and tried the fix on demo.blacklightopac.org >>>> ? ? ? , and it >>>> ? ? ? >> works great! ?So, Vernon, if you could grab the new version of >>>> ? ? ? >> catalog_helper.rb from subversion, I'm almost certain that >>>> ? ? ? will fix >>>> ? ? ? >> the bug you reported yesterday. >>>> ? ? ? >> >>>> ? ? ? >> Molly >>>> ? ? ? >> >>>> ? ? ? >> On Tue, Mar 10, 2009 at 9:45 AM, Molly Pickral Cadieux >>>> ? ? ? >> > wrote: >>>> ? ? ? >> >>>> ? ? ? >>> >>>> ? ? ? >>> Hi everyone, >>>> ? ? ? >>> >>>> ? ? ? >>> I recently committed some changes to the repository that >>>> ? ? ? offer the >>>> ? ? ? >>> following features: >>>> ? ? ? >>> >>>> ? ? ? >>> 1. ?A drop-down list that allows users to specify how may >>>> ? ? ? results per >>>> ? ? ? >>> page to display (the previous default was 10 per page) >>>> ? ? ? >>> 2. ?"Previous" and "Next" links that allow a user to page >>>> ? ? ? through a >>>> ? ? ? >>> search result, as opposed to going back to the previous >>>> ? ? ? page to select >>>> ? ? ? >>> the previous or next item. >>>> ? ? ? >>> 3. ?Several rspec tests for the catalog controller and >>>> ? ? ? various models. >>>> ? ? ? >>> >>>> ? ? ? >>> This all works perfectly on my setup, but when I went to >>>> ? ? ? deploy the >>>> ? ? ? >>> changes on demo.blacklightopac.org >>>> ? ? ? , I ran into a routing issue. >>>> ? ? ? >>> Vernon also reported this error yesterday. ?The error is >>>> ? ? ? as follows: >>>> ? ? ? >>> >>>> ? ? ? >>> ------------ >>>> ? ? ? >>> >>>> ? ? ? >>> catalog_url failed to generate from {:commit=>nil, >>>> ? ? ? :action=>"show", >>>> ? ? ? >>> :id=>"agrecord.2008-11-27.1267669533", >>>> ? ? ? :controller=>"catalog", >>>> ? ? ? >>> :counter=>1}, expected: {:action=>"show", >>>> ? ? ? :controller=>"catalog"}, diff: >>>> ? ? ? >>> {:commit=>nil, :id=>"agrecord.2008-11-27.1267669533", >>>> ? ? ? :counter=>1} >>>> ? ? ? >>> >>>> ? ? ? >>> Extracted source (around line #21): >>>> ? ? ? >>> >>>> ? ? ? >>> 18: ? ? ? ? <% # main title container for doc partial >>>> ? ? ? view -%> >>>> ? ? ? >>> 19: >>>> ? ? ? >>> 20: ? ? ? ?
>>>> ? ? ? >>> 21: ? ? ? ? ?
<%= counter + 1 + @response.start.to_i >>>> ? ? ? %>. <%= >>>> ? ? ? >>> link_to_document document, :label=>:title, >>>> ? ? ? :extra_params=>{:counter => >>>> ? ? ? >>> counter + 1 + @response.start.to_i } %>
>>>> ? ? ? >>> 22: ? ? ? ?
>>>> ? ? ? >>> 23: >>>> ? ? ? >>> 24: ? ? ? >>>> ? ? ? >>> >>>> ? ? ? >>> ----------- >>>> ? ? ? >>> >>>> ? ? ? >>> Bess had reported a similar problem back in February >>>> ? ? ? (I've included >>>> ? ? ? >>> the message below). ?Unfortunately, I have been unable to >>>> ? ? ? reproduce >>>> ? ? ? >>> the reported problem on my Mac running the same versions >>>> ? ? ? of Ruby, >>>> ? ? ? >>> Rails, and Passenger. ?I would like to try Bess's >>>> ? ? ? recommended solution >>>> ? ? ? >>> if I can get a setup where I can reproduce the problem. >>>> ? ? ? ?In the mean >>>> ? ? ? >>> time, I'd love it if Vernon or others who may have seen >>>> ? ? ? the problem >>>> ? ? ? >>> could try modifying this file: >>>> ? ? ? >>> >>>> ? ? ? >>> rails/vendor/plugins/blacklight/app/helpers/catalog_helper.rb >>>> ? ? ? >>> >>>> ? ? ? >>> and change this line: >>>> ? ? ? >>> >>>> ? ? ? >>> link_to label, catalog_path(doc[:id], >>>> ? ? ? >>> params.merge(opts[:extra_params]).merge(:action=>nil, >>>> ? ? ? :commit=>nil)) >>>> ? ? ? >>> >>>> ? ? ? >>> to be: >>>> ? ? ? >>> >>>> ? ? ? >>> link_to label, catalog_path(doc[:id], >>>> ? ? ? >>> params.merge(opts[:extra_params]).merge(:action=>"show", >>>> ? ? ? >>> :commit=>nil)) >>>> ? ? ? >>> >>>> ? ? ? >>> and see if that fixes it. ?A restart of passenger will be >>>> ? ? ? necessary. >>>> ? ? ? >>> >>>> ? ? ? >>> If I can't get this worked out soon, I will revert the >>>> ? ? ? changes in the >>>> ? ? ? >>> repository and work through this again here. >>>> ? ? ? >>> >>>> ? ? ? >>> In any case, I wanted to let you all know that what is >>>> ? ? ? currently in >>>> ? ? ? >>> the repository may cause routing issues under certain >>>> ? ? ? environments. >>>> ? ? ? >>> I'll keep you all posted on progress. >>>> ? ? ? >>> >>>> ? ? ? >>> Molly >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >>> ---------- >>>> ? ? ? >>> >>>> ? ? ? >>> From eos8d at virginia.edu ?Sun Feb >>>> ? ? ? ?1 17:25:35 2009 >>>> ? ? ? >>> From: eos8d at virginia.edu (Bess >>>> ? ? ? Sadler) >>>> ? ? ? >>> Date: Sun, 1 Feb 2009 17:25:35 -0500 >>>> ? ? ? >>> Subject: [Blacklight-development] demo app bug fix >>>> ? ? ? >>> Message-ID: >>>> ? ? ? >>> ? ? ? > >>>> ? ? ? >>> >>>> ? ? ? >>> Hey, folks. >>>> ? ? ? >>> >>>> ? ? ? >>> I've spent the last day or so playing pretty intensively >>>> ? ? ? with the demo >>>> ? ? ? >>> app. I fixed a couple of minor bugs in the installation >>>> ? ? ? instructions >>>> ? ? ? >>> (here: >>>> ? ? ? >>> >>>> >>>> http://blacklight.rubyforge.org/wiki/wiki.pl?Installing_The_Demo_App) >>>> ? ? ? >>> ?and I've deployed the demo app several times, on several >>>> ? ? ? different >>>> ? ? ? >>> systems. Strangely, although I could get it running right >>>> ? ? ? away with >>>> ? ? ? >>> mongrel or webrick, I found that I could not get it >>>> ? ? ? running with >>>> ? ? ? >>> mod_passenger, which is our preferred method of running >>>> ? ? ? rails apps in >>>> ? ? ? >>> production. I think I tracked down the bug today, and you >>>> ? ? ? can see the >>>> ? ? ? >>> demo app running with mod_passenger as a virtual host at >>>> ? ? ? >>> http://demo.blacklightopac.org >>>> ? ? ? >>> >>>> ? ? ? >>> I was getting two errors. One, encountered whenever the >>>> ? ? ? index screen >>>> ? ? ? >>> was loaded, said: >>>> ? ? ? >>> >>>> ? ? ? >>> ActionView::TemplateError (catalog_url failed to generate >>>> ? ? ? from >>>> ? ? ? >>> {:action=>"index", :controller=>"catalog", :id=>"u1"}, >>>> ? ? ? expected: >>>> ? ? ? >>> {:action=>"show", :controller=>"catalog"}, diff: >>>> ? ? ? >>> {:action=>"show", :id=>"u1"}) on line #11 of >>>> ? ? ? vendor/plugins/blacklight/ >>>> ? ? ? >>> app/views/catalog/_index_partials/_default.html.erb: >>>> ? ? ? >>> 8: >>>> ? ? ? >>> 9: ? ?
>>>> ? ? ? >>> 10: ? ? ?
>>>> ? ? ? >>> 11: ? ? ? ? <%= link_to document.get(:id), >>>> ? ? ? catalog_path(document[:id], >>>> ? ? ? >>> params) %> >>>> ? ? ? >>> 12: ? ? ?
>>>> ? ? ? >>> 13: ? ?
>>>> ? ? ? >>> 14: ? >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >>> The other was generated whenever the show screen was >>>> ? ? ? loaded, and it >>>> ? ? ? >>> said: >>>> ? ? ? >>> >>>> ? ? ? >>> ActionView::TemplateError (catalog_index_url failed to >>>> ? ? ? generate from >>>> ? ? ? >>> {:action=>"show", :controller=>"catalog", :id=>nil}, >>>> ? ? ? expected: >>>> ? ? ? >>> {:action=>"index", :controller=>"catalog"}, diff: >>>> ? ? ? >>> {:action=>"index", :id=>nil}) on line #8 of >>>> ? ? ? vendor/plugins/blacklight/ >>>> ? ? ? >>> app/views/catalog/show.html.erb: >>>> ? ? ? >>> 5: >>>> ? ? ? >>> 6: >>>> ? ? ? >>> 7: <% @sidebar = capture do %> >>>> ? ? ? >>> 8: ?

<%= link_to '<< SEARCH', >>>> ? ? ? >>> catalog_index_path(params.merge(:id=>nil)) %>

>>>> ? ? ? >>> 9: <% end %> >>>> ? ? ? >>> >>>> ? ? ? >>> Turns out these were basically the same error. In both >>>> ? ? ? cases, we're >>>> ? ? ? >>> using the link_to function to create a link to another >>>> ? ? ? part of the >>>> ? ? ? >>> application, in this case from index to show, and from >>>> ? ? ? show to index. >>>> ? ? ? >>> Let's talk about the link from index to show as an example. >>>> ? ? ? >>> >>>> ? ? ? >>> <%= link_to document.get(:id), >>>> ? ? ? catalog_path(document[:id], params) %> >>>> ? ? ? >>> >>>> ? ? ? >>> That says create a link, and the text of the link is the >>>> ? ? ? id number of >>>> ? ? ? >>> the current document (say, u999), and the target of the >>>> ? ? ? link is >>>> ? ? ? >>> catalog_path(u999) which will evaluate to /catalog/u999. >>>> ? ? ? Incidentally, >>>> ? ? ? >>> the rails console was really valuable in helping me >>>> ? ? ? understand how >>>> ? ? ? >>> these worked, and you can play with it like this: >>>> ? ? ? >>> >>>> ? ? ? >>> paz:rails eos8d$ ./script/console >>>> ? ? ? >>> Loading development environment (Rails 2.2.2) >>>> ? ? ? >>> ?>> app.catalog_path("u999") >>>> ? ? ? >>> => "/catalog/u999" >>>> ? ? ? >>> >>>> ? ? ? >>> Anyway, I found that if I just changed the link_to >>>> ? ? ? statement to look >>>> ? ? ? >>> like this instead, removing the params object: >>>> ? ? ? >>> >>>> ? ? ? >>> ? ? ? <%= link_to document.get(:id), >>>> ? ? ? catalog_path(document[:id]) %> >>>> ? ? ? >>> >>>> ? ? ? >>> the error went away. However, this wasn't an acceptable >>>> ? ? ? solution, >>>> ? ? ? >>> because whenever a user performs a search or selects a >>>> ? ? ? facet, that >>>> ? ? ? >>> value goes in the params hash, and you want to keep it >>>> ? ? ? around when you >>>> ? ? ? >>> go from screen to screen, so that the user isn't always >>>> ? ? ? having to re- >>>> ? ? ? >>> enter their current search. Let's say you're on the index >>>> ? ? ? screen, and >>>> ? ? ? >>> you've entered a search and selected a facet. At that >>>> ? ? ? point your >>>> ? ? ? >>> params hash looks like this: >>>> ? ? ? >>> >>>> ? ? ? >>> ? ? ? {"commit"=>"search", "action"=>"index", "q"=>"poetry", >>>> ? ? ? >>> "f"=>{"language_facet"=>["English"]}, >>>> ? ? ? "controller"=>"catalog"} >>>> ? ? ? >>> >>>> ? ? ? >>> When you want to look at a detailed view for a specific >>>> ? ? ? item, you're >>>> ? ? ? >>> linking to the show action of your same catalog >>>> ? ? ? controller, and >>>> ? ? ? >>> passing it a specific document id. But you also want to >>>> ? ? ? hang onto the >>>> ? ? ? >>> fact that you've searched for poetry and limited to >>>> ? ? ? >>> language_facet:English, so you also pass the params >>>> ? ? ? object in. At >>>> ? ? ? >>> which point your link_to really looks like this: >>>> ? ? ? >>> >>>> ? ? ? >>> ? ? ? <%= link_to document.get(:id), >>>> ? ? ? >>> catalog_path >>>> ? ? ? >>> (:id >>>> ? ? ? >>> =>document[:id], :commit=>"search", :action=>"index", >>>> ? ? ? :q=>"poetry" ... >>>> ? ? ? >>> etc ... ) %> >>>> ? ? ? >>> >>>> ? ? ? >>> Do you see the problem? It took me awhile to notice it. >>>> ? ? ? I've just >>>> ? ? ? >>> called the show action, and then I also pass in a >>>> ? ? ? parameter of >>>> ? ? ? >>> action=index. It's not obvious, because catalog_path sort >>>> ? ? ? of obscures >>>> ? ? ? >>> the fact that you're calling the show action. >>>> ? ? ? Interestingly, webrick >>>> ? ? ? >>> and mongrel don't seem to care. I guess they just assume >>>> ? ? ? that even >>>> ? ? ? >>> though you've passed two values for action, you must >>>> ? ? ? really mean >>>> ? ? ? >>> action=show because that's what's defined for >>>> ? ? ? catalog_path. The fix is >>>> ? ? ? >>> easy, once you know what the problem is: >>>> ? ? ? >>> >>>> ? ? ? >>> ? ? ? ?<%= link_to document.get(:id), >>>> ? ? ? >>> >>>> ? ? ? catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >>>> ? ? ? >>> >>>> ? ? ? >>> Now you're passing in a params object that looks like this: >>>> ? ? ? >>> >>>> ? ? ? >>> ? ? ? {"commit"=>"search", "action"=>"show", >>>> ? ? ? "id"=>"2008349843", >>>> ? ? ? >>> "f"=>{"language_facet"=>["English"]}, "q"=>"poetry", >>>> ? ? ? >>> "controller"=>"catalog"} >>>> ? ? ? >>> >>>> ? ? ? >>> So, all your user defined parameters, plus the id number >>>> ? ? ? of the >>>> ? ? ? >>> specific item you want to show, plus action=show, which >>>> ? ? ? is what >>>> ? ? ? >>> catalog_path expects. Problem solved. >>>> ? ? ? >>> >>>> ? ? ? >>> Sorry this is so long, but I learned a lot in fixing this >>>> ? ? ? bug and I >>>> ? ? ? >>> thought it might be helpful for someone else too. But if >>>> ? ? ? it just looks >>>> ? ? ? >>> like gibberish, ignore it. >>>> ? ? ? >>> >>>> ? ? ? >>> Cheers, >>>> ? ? ? >>> Bess >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >>> >From goodieboy at gmail.com ?Sun Feb >>>> ? ? ? ?1 17:59:36 2009 >>>> ? ? ? >>> From: goodieboy at gmail.com (Matt >>>> ? ? ? Mitchell) >>>> ? ? ? >>> Date: Sun, 1 Feb 2009 17:59:36 -0500 >>>> ? ? ? >>> Subject: [Blacklight-development] demo app bug fix >>>> ? ? ? >>> In-Reply-To: >>>> ? ? ? >>> ? ? ? > >>>> ? ? ? >>> References: >>>> ? ? ? >>> ? ? ? > >>>> ? ? ? >>> Message-ID: >>>> ? ? ? >>> >>>> > >>>> ? ? ? >>> >>>> ? ? ? >>> Seriously, that's a kickin' write up Bess. Great bug >>>> ? ? ? catch. And the docs >>>> ? ? ? >>> look so nice too! >>>> ? ? ? >>> >>>> ? ? ? >>> For the link_to fix here: >>>> ? ? ? >>> >>>> ? ? ? >>> <%= link_to document.get(:id), >>>> ? ? ? >>> >>>> ? ? ? catalog_path(params.merge(:id=>document[:id],:action=>"show"))%> >>>> ? ? ? >>> >>>> ? ? ? >>> You should be able to do this too: >>>> ? ? ? >>> >>>> ? ? ? >>> <%= link_to document.get(:id), catalog_path(document[:id], >>>> ? ? ? >>> params.merge(:action=>nil)) %> >>>> ? ? ? >>> >>>> ? ? ? >>> I think this kind of thing will become a >>>> ? ? ? url-param-linking pattern in BL >>>> ? ? ? >>> and >>>> ? ? ? >>> might do well in a view helper. I think there's one in >>>> ? ? ? the UVa trunk, >>>> ? ? ? >>> called >>>> ? ? ? >>> link_to_document or something. We could then have one called >>>> ? ? ? >>> link_to_documents or link_back_to_search? The upshot of >>>> ? ? ? it being in a >>>> ? ? ? >>> helper >>>> ? ? ? >>> is that if someone overrides a view partial, they don't >>>> ? ? ? have to re-write >>>> ? ? ? >>> the >>>> ? ? ? >>> param merging stuff. Food for thought... >>>> ? ? ? >>> >>>> ? ? ? >>> Matt >>>> ? ? ? >>> >>>> ? ? ? >>> >>>> ? ? ? >> >>>> ? ? ? >> _______________________________________________ >>>> ? ? ? >> Blacklight-development mailing list >>>> ? ? ? >> Blacklight-development at rubyforge.org >>>> ? ? ? >>>> ? ? ? >> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> ? ? ? >> Blacklightopac Blog http://blacklightopac.org/ >>>> ? ? ? >> >>>> ? ? ? >> >>>> ? ? ? > >>>> ? ? ? > _______________________________________________ >>>> ? ? ? > Blacklight-development mailing list >>>> ? ? ? > Blacklight-development at rubyforge.org >>>> ? ? ? >>>> ? ? ? > http://rubyforge.org/mailman/listinfo/blacklight-development >>>> ? ? ? > Blacklightopac Blog http://blacklightopac.org/ >>>> ? ? ? > >>>> ? ? ? _______________________________________________ >>>> ? ? ? Blacklight-development mailing list >>>> ? ? ? Blacklight-development at rubyforge.org >>>> ? ? ? >>>> ? ? ? http://rubyforge.org/mailman/listinfo/blacklight-development >>>> ? ? ? Blacklightopac Blog http://blacklightopac.org/ >>>> >>>> >>>> ? _______________________________________________ >>>> ? Blacklight-development mailing list >>>> ? Blacklight-development at rubyforge.org >>>> ? >>>> ? http://rubyforge.org/mailman/listinfo/blacklight-development >>>> ? Blacklightopac Blog http://blacklightopac.org/ >>> >>> >>> ? _______________________________________________ >>> ? Blacklight-development mailing list >>> ? Blacklight-development at rubyforge.org >>> ? >>> ? http://rubyforge.org/mailman/listinfo/blacklight-development >>> ? Blacklightopac Blog http://blacklightopac.org/ >>> >>> >>> ------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From erikhatcher at mac.com Wed Mar 11 22:14:03 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Wed, 11 Mar 2009 22:14:03 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <23b83f160903111849n73fe8717n5f6cc0161ee9b8dd@mail.gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <34e39bfd0903100701i77d8bd36rf12e7849a2d18ffe@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> <49B80EE6.9050401@gmail.com> <23b83f160903111849n73fe8717n5f6cc0161ee9b8dd@mail.gmail.com> Message-ID: <4C13B387-3E52-42BC-86AF-F623AB0321B4@mac.com> On Mar 11, 2009, at 9:49 PM, Ross Singer wrote: > For example, for the Jangle hack, I was using IDs that were ":" > delimited (foo:1234 where foo represents the source) -- but that is > awkward for Solr (well, specifically, for Lucene), since you have to > escape the colon when requesting by ID. That's not true that you _have_ to escape though. That's only if you try to request by id using the standard query parser. But if you do like I mentioned in a previous mail, there are no issues (other than URL encoding, which APIs handle automatically anyway): /solr/select?q={!raw f=id}whatever:id.format-you&want (again, that q parameter needs to be URL encoded, but that's all) > So what are good id delimiter characters? That's a domain question... and the answer really is to use whatever characters make the most sense for your domain. Erik From rossfsinger at gmail.com Wed Mar 11 23:01:59 2009 From: rossfsinger at gmail.com (Ross Singer) Date: Wed, 11 Mar 2009 23:01:59 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <4C13B387-3E52-42BC-86AF-F623AB0321B4@mac.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> <49B80EE6.9050401@gmail.com> <23b83f160903111849n73fe8717n5f6cc0161ee9b8dd@mail.gmail.com> <4C13B387-3E52-42BC-86AF-F623AB0321B4@mac.com> Message-ID: <23b83f160903112001t11c41cf2kf8ab2fd3b240110a@mail.gmail.com> But doesn't it seem counterproductive to use a delimiter that doesn't "just work" (whether this be in Solr or Rails) if there are other characters that carry no baggage whatsoever? Yes, it will be somewhat domain specific, but focusing on the two domains we have some prior control over (the aforementioned Solr and Rails), what is the set good delimiter chars that don't require any consideration from either Rails or Solr? "-"? "_"? I would think it helps the developer to know "what's safe" before they start minting identifiers specific to their domain, right? -Ross. On Wed, Mar 11, 2009 at 10:14 PM, Erik Hatcher wrote: > > On Mar 11, 2009, at 9:49 PM, Ross Singer wrote: >> >> For example, for the Jangle hack, I was using IDs that were ":" >> delimited (foo:1234 where foo represents the source) -- but that is >> awkward for Solr (well, specifically, for Lucene), since you have to >> escape the colon when requesting by ID. > > That's not true that you _have_ to escape though. ?That's only if you try to > request by id using the standard query parser. ?But if you do like I > mentioned in a previous mail, there are no issues (other than URL encoding, > which APIs handle automatically anyway): > > > ? /solr/select?q={!raw f=id}whatever:id.format-you&want > > ? ?(again, that q parameter needs to be URL encoded, but that's all) > >> So what are good id delimiter characters? > > That's a domain question... and the answer really is to use whatever > characters make the most sense for your domain. > > ? ? ? ?Erik > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From erikhatcher at mac.com Thu Mar 12 00:11:11 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Thu, 12 Mar 2009 00:11:11 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <23b83f160903112001t11c41cf2kf8ab2fd3b240110a@mail.gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> <49B80EE6.9050401@gmail.com> <23b83f160903111849n73fe8717n5f6cc0161ee9b8dd@mail.gmail.com> <4C13B387-3E52-42BC-86AF-F623AB0321B4@mac.com> <23b83f160903112001t11c41cf2kf8ab2fd3b240110a@mail.gmail.com> Message-ID: <6ECC72EA-518C-456C-83A6-5E1EB1E995C0@mac.com> In solr-ruby speak, just call Solr::Util.query_parser_escape(id) - irb(main):002:0> Solr::Util.query_parser_escape("your:damn.identifier") => "your\\:damn\\.identifier" On Mar 11, 2009, at 11:01 PM, Ross Singer wrote: > But doesn't it seem counterproductive to use a delimiter that doesn't > "just work" (whether this be in Solr or Rails) if there are other > characters that carry no baggage whatsoever? Well, whether it works or not depends on how you parse it. But ok, I'll play along... > Yes, it will be somewhat domain specific, but focusing on the two > domains we have some prior control over (the aforementioned Solr and > Rails), what is the set good delimiter chars that don't require any > consideration from either Rails or Solr? "-"? "_"? dash or underscore both are reasonably safe with Solr/Lucene standard query parser, but you'd have to be careful about a leading dash as that'd be interpreted as a negative term. Underscore would be fine in all cases, I think. But again, I advise against trusting the query parser on arbitrary terms to "just work". Too many cases where it wouldn't just work. Use the raw "parser" for exact term queries and you'll save yourself headaches later. > I would think it helps the developer to know "what's safe" before they > start minting identifiers specific to their domain, right? Well, all characters should be safe to use, if used properly. I'd say that the issue here is to fix routing to actions that require an id as part of the path rather than "fixing" your indexing to use awkward id's requiring some translation just to make Rails happy. Rails may not be the only client hitting your Solr, and letting your id's be natural is a better strategy to avoid clients from having to deal with any sort of id mangling. You might need that id to tie back to your ILS, for example. Granted, in that case if you're mangling the id's I'd recommend that you store the actual id to tie back to your ILS in another field. And if you do go down the id mangling route... you can use a pattern tokenizer to regex it on the Solr-side if you like. Make id a "text" field that outputs a single term but tokenized with a regex translation in the middle. Then you could copyField to another field that keeps it as-is. But I still suggest letting your id's be natural to your domain. Erik From erikhatcher at mac.com Thu Mar 12 00:14:10 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Thu, 12 Mar 2009 00:14:10 -0400 Subject: [Blacklight-development] routing problem In-Reply-To: <23b83f160903112001t11c41cf2kf8ab2fd3b240110a@mail.gmail.com> References: <34e39bfd0903100645ybb66a83qad78f4a4bce3268c@mail.gmail.com> <49B6B6BB.5020309@gmail.com> <34e39bfd0903110735l2a502faeke679b6adbe4e686@mail.gmail.com> <7c20cede0903110940k858dcd2w57208ee0e56fb9dd@mail.gmail.com> <3C682828-8278-4EA3-9E00-1562C20C4797@stanford.edu> <49B80EE6.9050401@gmail.com> <23b83f160903111849n73fe8717n5f6cc0161ee9b8dd@mail.gmail.com> <4C13B387-3E52-42BC-86AF-F623AB0321B4@mac.com> <23b83f160903112001t11c41cf2kf8ab2fd3b240110a@mail.gmail.com> Message-ID: <078A3BA4-7A5E-4E88-9865-77AF17CD943C@mac.com> Further on this... even if you did use currently safe characters for Lucene's QueryParser, those characters could later be co-opted for some new fangled query syntax. Solr::Util.query_parser_escape simply does this to cover it's bases: def self.query_parser_escape(string) # backslash prefix everything that isn't a word character string.gsub(/(\W)/,'\\\\\1') end Escaping more than necessary is fine. Erik On Mar 11, 2009, at 11:01 PM, Ross Singer wrote: > But doesn't it seem counterproductive to use a delimiter that doesn't > "just work" (whether this be in Solr or Rails) if there are other > characters that carry no baggage whatsoever? > > Yes, it will be somewhat domain specific, but focusing on the two > domains we have some prior control over (the aforementioned Solr and > Rails), what is the set good delimiter chars that don't require any > consideration from either Rails or Solr? "-"? "_"? > > I would think it helps the developer to know "what's safe" before they > start minting identifiers specific to their domain, right? > > -Ross. From mpc3c at virginia.edu Thu Mar 12 10:35:15 2009 From: mpc3c at virginia.edu (Molly Pickral Cadieux) Date: Thu, 12 Mar 2009 10:35:15 -0400 Subject: [Blacklight-development] moved location of RSpec tests Message-ID: <34e39bfd0903120735q5e8c3bf0ja429e60ddebe93ab@mail.gmail.com> All, I just committed some changes to the repository. Specifically, I moved the RSpec tests into the plugin so that they can be run independently from the container application. Let me know if you have any trouble running these tests. Molly From jamie at dang.com Thu Mar 12 14:43:17 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Thu, 12 Mar 2009 14:43:17 -0400 Subject: [Blacklight-development] moved location of RSpec tests In-Reply-To: <34e39bfd0903120735q5e8c3bf0ja429e60ddebe93ab@mail.gmail.com> References: <34e39bfd0903120735q5e8c3bf0ja429e60ddebe93ab@mail.gmail.com> Message-ID: Molly, how are you running them from there? Thanks, Jamie On Mar 12, 2009, at 10:35 AM, Molly Pickral Cadieux wrote: > All, > > I just committed some changes to the repository. Specifically, I > moved the RSpec tests into the plugin so that they can be run > independently from the container application. Let me know if you have > any trouble running these tests. > > Molly > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From mpc3c at virginia.edu Thu Mar 12 14:49:43 2009 From: mpc3c at virginia.edu (Molly Pickral Cadieux) Date: Thu, 12 Mar 2009 14:49:43 -0400 Subject: [Blacklight-development] moved location of RSpec tests In-Reply-To: References: <34e39bfd0903120735q5e8c3bf0ja429e60ddebe93ab@mail.gmail.com> Message-ID: <34e39bfd0903121149k16ae6b51o5dca0b90d537d695@mail.gmail.com> Hey Jamie, I've run them by using the TextMate bundle or by cd'ing to that directory and doing: ruby spec/controllers/catalog_controller_spec.rb Molly On Thu, Mar 12, 2009 at 2:43 PM, Jamie Orchard-Hays wrote: > Molly, how are you running them from there? > > Thanks, > > Jamie > On Mar 12, 2009, at 10:35 AM, Molly Pickral Cadieux wrote: > >> All, >> >> I just committed some changes to the repository. ?Specifically, I >> moved the RSpec tests into the plugin so that they can be run >> independently from the container application. ?Let me know if you have >> any trouble running these tests. >> >> Molly >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From mpc3c at virginia.edu Thu Mar 12 14:58:13 2009 From: mpc3c at virginia.edu (Molly Pickral Cadieux) Date: Thu, 12 Mar 2009 14:58:13 -0400 Subject: [Blacklight-development] moved location of RSpec tests In-Reply-To: <34e39bfd0903121149k16ae6b51o5dca0b90d537d695@mail.gmail.com> References: <34e39bfd0903120735q5e8c3bf0ja429e60ddebe93ab@mail.gmail.com> <34e39bfd0903121149k16ae6b51o5dca0b90d537d695@mail.gmail.com> Message-ID: <34e39bfd0903121158x41ce953p6547824e2f8a63f5@mail.gmail.com> Oh, yeah, and by "that directory" I mean vendor/plugins/blacklight. Molly On Thu, Mar 12, 2009 at 2:49 PM, Molly Pickral Cadieux wrote: > Hey Jamie, > > I've run them by using the TextMate bundle or by cd'ing to that > directory and doing: > > ruby spec/controllers/catalog_controller_spec.rb > > Molly > > On Thu, Mar 12, 2009 at 2:43 PM, Jamie Orchard-Hays wrote: >> Molly, how are you running them from there? >> >> Thanks, >> >> Jamie >> On Mar 12, 2009, at 10:35 AM, Molly Pickral Cadieux wrote: >> >>> All, >>> >>> I just committed some changes to the repository. ?Specifically, I >>> moved the RSpec tests into the plugin so that they can be run >>> independently from the container application. ?Let me know if you have >>> any trouble running these tests. >>> >>> Molly >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > From jamie at dang.com Thu Mar 12 15:19:57 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Thu, 12 Mar 2009 15:19:57 -0400 Subject: [Blacklight-development] moved location of RSpec tests In-Reply-To: <34e39bfd0903121149k16ae6b51o5dca0b90d537d695@mail.gmail.com> References: <34e39bfd0903120735q5e8c3bf0ja429e60ddebe93ab@mail.gmail.com> <34e39bfd0903121149k16ae6b51o5dca0b90d537d695@mail.gmail.com> Message-ID: <8A980D97-BFE0-4A52-9BA1-9C86047FFD42@dang.com> I'm working on setting up CruiseControl for continuous integration on blacklightopac.org. Once I get far enough along, I'll add the rspec rake commands to the demo app enclosing the BL plugin and then we'll be able to fire off the tests from the rails root dir. Currently, there are a bunch of errors thrown by the tests in the demo app, so I've got to get those under control first. Jamie On Mar 12, 2009, at 2:49 PM, Molly Pickral Cadieux wrote: > Hey Jamie, > > I've run them by using the TextMate bundle or by cd'ing to that > directory and doing: > > ruby spec/controllers/catalog_controller_spec.rb > > Molly > > On Thu, Mar 12, 2009 at 2:43 PM, Jamie Orchard-Hays > wrote: >> Molly, how are you running them from there? >> >> Thanks, >> >> Jamie >> On Mar 12, 2009, at 10:35 AM, Molly Pickral Cadieux wrote: >> >>> All, >>> >>> I just committed some changes to the repository. Specifically, I >>> moved the RSpec tests into the plugin so that they can be run >>> independently from the container application. Let me know if you >>> have >>> any trouble running these tests. >>> >>> Molly >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From ndushay at stanford.edu Thu Mar 12 17:24:48 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Thu, 12 Mar 2009 14:24:48 -0700 Subject: [Blacklight-development] what is facet action is catalog_controller? Message-ID: Can someone explain what the facet action is used for in the catalog_controller? Thanks, - Naomi From chapman.lists at gmail.com Thu Mar 12 18:54:16 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Thu, 12 Mar 2009 18:54:16 -0400 Subject: [Blacklight-development] what is facet action is catalog_controller? In-Reply-To: References: Message-ID: <49B99298.7030805@gmail.com> Thank you Naomi I thought I was the only one who didn't know Vern Naomi Dushay wrote: > Can someone explain what the facet action is used for in the > catalog_controller? > > Thanks, > > - Naomi > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From rochkind at jhu.edu Fri Mar 13 10:34:14 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Fri, 13 Mar 2009 10:34:14 -0400 Subject: [Blacklight-development] [NGC4LIB] Whose elephant is it, anyway? (the OLE project) In-Reply-To: References: Message-ID: <49BA6EE6.4080505@jhu.edu> The work Mark has done in Australia to implement a 'browse' list based on solr with vufind seems potentially useful to blacklight. Especially since he says he did most of it in a custom Solr handler. Jonathan Mark Triggs wrote: > [Apologies if this comes through twice. I sent this about 14 hours ago and > haven't seen it arrive yet, so...] > > Hi all, > > I've been watching this discussion with some interest because I'm the > guy who implemented the browse functionality in the NLA's catalogue. I > just thought I'd jump in and confirm/deny a few things here. > > Our current title and uniform title browses were among the first browses > we attempted to implement, so they're currently a bit of a legacy > feature. We implemented these using a combination of Solr range queries > and sorting and they mostly sort of work, but perhaps not quite as > smoothly as the other browses (as evidenced by the 'internal server > error' that Owen managed to produce ;o). I'm on holidays at the moment, > but this will probably be revisited when I'm back at work. > > Our other browses (names, subjects, callnumbers and series) make use of > a combination of SQLite databases and Lucene indexes. Each browse > consists of an SQLite database with a single table of two columns: a > sort key and the text of the browse heading. When we receive a request > to browse from a certain point we can get back the pageful of headings > to display by using a simple SQL SELECT statement. For each heading > listed we determine the number of titles matched and any > cross-references by performing Lucene term queries (fast) on indexes of > our bib data and authority data respectively. All of this is handled by > a Solr browse handler I've written, so all our VuFind code needs to know > is to hit the browse handler and style the XML it gets back. > > Regarding scalability, our largest browse is the callnumber browse, > which consists of about 3 million entries (for 4 million bib records). > I've tested this SQLite approach up to 20 million entries and it > continued to perform well, so I'm not terribly worried for now. Finding > the point to browse from is effectively just searching a big sorted text > file, so I would expect O(log N) growth here anyway. Plus, our largest > SQLite database still fits entirely in memory, so that's nice too. > > In terms of indexing performance, the SQLite databases take about 5-10 > minutes to build in total, and they're built from scratch every time. > For each type of browse we pull all the browse headings from our bib and > authority data, remove any duplicates then load them all into an SQLite > database. My browse handler notices when these databases have been > updated and automatically reopens them, so the update is transparent. > Currently we just do these updates once per night, as this is how often > we update our main bib indexes and it makes sense to keep the updates > synchronised, but I don't see any problem with doing this more often if > it made sense. > > I'm happy to answer any questions about our implementation either on or > off list. > > Cheers, > > Mark > > > "Stephens, Owen" writes: > > >> Bernhard, >> >> Just to understand what you are looking for in terms of Browse. The >> NLA implementation of VuFind has what I would regard as a Browse >> function - you can Browse the following: >> >> Names at http://catalogue.nla.gov.au/Browse/Names?browse=names&from= >> Subjects at >> http://catalogue.nla.gov.au/Browse/Subjects?browse=subjects&from= >> Callnumbers at >> http://catalogue.nla.gov.au/Browse/Subjects?browse=subjects&from= >> Series at >> http://catalogue.nla.gov.au/Browse/Series?browse=series&from= >> >> All these options are available in the user interface at >> http://catalogue.nla.gov.au/Browse/Home ('Browse' is an option in the >> horizontal menu under the main 'catalogue' banner) >> >> This page also offers Title and Uniform Title browsing, but these seem >> not to work in the same way at the moment (I've sent feedback about >> this) >> >> Is this browsing as you mean it? If not, what would you require >> additionally? >> >> (also you question the scalability - what scale are you thinking of? >> I'd guess that NLA is reasonably large - but I can't easily find a >> figure for the number of bib records - but obviously it may not be as >> large as other national libraries or consortium collections) >> From mpc3c at virginia.edu Fri Mar 13 11:11:35 2009 From: mpc3c at virginia.edu (Molly Pickral Cadieux) Date: Fri, 13 Mar 2009 11:11:35 -0400 Subject: [Blacklight-development] adding more specs Message-ID: <34e39bfd0903130811o7bdbdc88vf87b26c3ccce57d8@mail.gmail.com> All, I updated the RSpec test for the catalog_controller and added a skeletal test for the catalog/show view to include tests for the previous/next links. I'm still learning RSpec, so if you have any feedback on these tests, please share. Thanks, Molly From eos8d at virginia.edu Fri Mar 13 12:29:06 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Fri, 13 Mar 2009 12:29:06 -0400 Subject: [Blacklight-development] browse lists in Blacklight In-Reply-To: <49BA6EE6.4080505@jhu.edu> References: <49BA6EE6.4080505@jhu.edu> Message-ID: <066E1600-5855-47D4-A236-C97F115D0184@virginia.edu> Hey, Jonathan. I agree, and actually, I already wrote to him privately and he just wrote back: ************ Hi Bess, I've bundled up the code for my Solr browse handler and plonked it here: ftp://ftp.nla.gov.au/pub/vufind/nla-browse-handler.tgz I've tested against the latest stable version of Solr and it worked for me so hopefully it'll work for you too :o). There's a README included in that tarball that explains how to compile everything, build your indexes and configure Solr to use it, but please let me know if you have any problems--I'm happy to help where ever I can. I've released the code under the Apache 2.0 licence, so hopefully that will let you do whatever you want to do with it. Cheers, Mark ************** Naomi has already been doing some work with getting call-number sort working with solr. I haven't looked at Mark's tarball yet, but anyone who has some free time and want to poke at it, please do. I think it would be great to add this kind of feature into BL. Bess On 13-Mar-09, at 10:34 AM, Jonathan Rochkind wrote: > The work Mark has done in Australia to implement a 'browse' list based > on solr with vufind seems potentially useful to blacklight. Especially > since he says he did most of it in a custom Solr handler. > > Jonathan > > Mark Triggs wrote: >> [Apologies if this comes through twice. I sent this about 14 hours >> ago and >> haven't seen it arrive yet, so...] >> >> Hi all, >> >> I've been watching this discussion with some interest because I'm the >> guy who implemented the browse functionality in the NLA's >> catalogue. I >> just thought I'd jump in and confirm/deny a few things here. >> >> Our current title and uniform title browses were among the first >> browses >> we attempted to implement, so they're currently a bit of a legacy >> feature. We implemented these using a combination of Solr range >> queries >> and sorting and they mostly sort of work, but perhaps not quite as >> smoothly as the other browses (as evidenced by the 'internal server >> error' that Owen managed to produce ;o). I'm on holidays at the >> moment, >> but this will probably be revisited when I'm back at work. >> >> Our other browses (names, subjects, callnumbers and series) make >> use of >> a combination of SQLite databases and Lucene indexes. Each browse >> consists of an SQLite database with a single table of two columns: a >> sort key and the text of the browse heading. When we receive a >> request >> to browse from a certain point we can get back the pageful of >> headings >> to display by using a simple SQL SELECT statement. For each heading >> listed we determine the number of titles matched and any >> cross-references by performing Lucene term queries (fast) on >> indexes of >> our bib data and authority data respectively. All of this is >> handled by >> a Solr browse handler I've written, so all our VuFind code needs to >> know >> is to hit the browse handler and style the XML it gets back. >> >> Regarding scalability, our largest browse is the callnumber browse, >> which consists of about 3 million entries (for 4 million bib >> records). >> I've tested this SQLite approach up to 20 million entries and it >> continued to perform well, so I'm not terribly worried for now. >> Finding >> the point to browse from is effectively just searching a big sorted >> text >> file, so I would expect O(log N) growth here anyway. Plus, our >> largest >> SQLite database still fits entirely in memory, so that's nice too. >> >> In terms of indexing performance, the SQLite databases take about >> 5-10 >> minutes to build in total, and they're built from scratch every time. >> For each type of browse we pull all the browse headings from our >> bib and >> authority data, remove any duplicates then load them all into an >> SQLite >> database. My browse handler notices when these databases have been >> updated and automatically reopens them, so the update is transparent. >> Currently we just do these updates once per night, as this is how >> often >> we update our main bib indexes and it makes sense to keep the updates >> synchronised, but I don't see any problem with doing this more >> often if >> it made sense. >> >> I'm happy to answer any questions about our implementation either >> on or >> off list. >> >> Cheers, >> >> Mark >> >> >> "Stephens, Owen" writes: >> >> >>> Bernhard, >>> >>> Just to understand what you are looking for in terms of Browse. The >>> NLA implementation of VuFind has what I would regard as a Browse >>> function - you can Browse the following: >>> >>> Names at http://catalogue.nla.gov.au/Browse/Names?browse=names&from= >>> Subjects at >>> http://catalogue.nla.gov.au/Browse/Subjects?browse=subjects&from= >>> Callnumbers at >>> http://catalogue.nla.gov.au/Browse/Subjects?browse=subjects&from= >>> Series at >>> http://catalogue.nla.gov.au/Browse/Series?browse=series&from= >>> >>> All these options are available in the user interface at >>> http://catalogue.nla.gov.au/Browse/Home ('Browse' is an option in >>> the >>> horizontal menu under the main 'catalogue' banner) >>> >>> This page also offers Title and Uniform Title browsing, but these >>> seem >>> not to work in the same way at the moment (I've sent feedback about >>> this) >>> >>> Is this browsing as you mean it? If not, what would you require >>> additionally? >>> >>> (also you question the scalability - what scale are you thinking of? >>> I'd guess that NLA is reasonably large - but I can't easily find a >>> figure for the number of bib records - but obviously it may not be >>> as >>> large as other national libraries or consortium collections) >>> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rochkind at jhu.edu Fri Mar 13 12:31:23 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Fri, 13 Mar 2009 12:31:23 -0400 Subject: [Blacklight-development] browse lists in Blacklight In-Reply-To: <066E1600-5855-47D4-A236-C97F115D0184@virginia.edu> References: <49BA6EE6.4080505@jhu.edu> <066E1600-5855-47D4-A236-C97F115D0184@virginia.edu> Message-ID: <49BA8A5B.60401@jhu.edu> Yeah, it sounds like he was using an approach roughly similar to one I had considered. Building a seperate index of your indexes to keep track of what range to start at for a given browser query. The potential problem here is keeping that seperate index-of-the-index up to date as you continually update your 'main' index. I get the feeling that one of our goals is allowing an index that can be incrementally updated at any time, not just in bulk. Will be interesting to see if his design manages that. Jonathan Bess Sadler wrote: > Hey, Jonathan. I agree, and actually, I already wrote to him privately and he just wrote back: > > ************ > > Hi Bess, > > I've bundled up the code for my Solr browse handler and plonked it here: > > ftp://ftp.nla.gov.au/pub/vufind/nla-browse-handler.tgz > > I've tested against the latest stable version of Solr and it worked for > me so hopefully it'll work for you too :o). There's a README included > in that tarball that explains how to compile everything, build your > indexes and configure Solr to use it, but please let me know if you have > any problems--I'm happy to help where ever I can. > > I've released the code under the Apache 2.0 licence, so hopefully that > will let you do whatever you want to do with it. > > Cheers, > > Mark > > ************** > > Naomi has already been doing some work with getting call-number sort working with solr. I haven't looked at Mark's tarball yet, but anyone who has some free time and want to poke at it, please do. I think it would be great to add this kind of feature into BL. > > Bess > > On 13-Mar-09, at 10:34 AM, Jonathan Rochkind wrote: > > The work Mark has done in Australia to implement a 'browse' list based > on solr with vufind seems potentially useful to blacklight. Especially > since he says he did most of it in a custom Solr handler. > > Jonathan > > Mark Triggs wrote: > [Apologies if this comes through twice. I sent this about 14 hours ago and > haven't seen it arrive yet, so...] > > Hi all, > > I've been watching this discussion with some interest because I'm the > guy who implemented the browse functionality in the NLA's catalogue. I > just thought I'd jump in and confirm/deny a few things here. > > Our current title and uniform title browses were among the first browses > we attempted to implement, so they're currently a bit of a legacy > feature. We implemented these using a combination of Solr range queries > and sorting and they mostly sort of work, but perhaps not quite as > smoothly as the other browses (as evidenced by the 'internal server > error' that Owen managed to produce ;o). I'm on holidays at the moment, > but this will probably be revisited when I'm back at work. > > Our other browses (names, subjects, callnumbers and series) make use of > a combination of SQLite databases and Lucene indexes. Each browse > consists of an SQLite database with a single table of two columns: a > sort key and the text of the browse heading. When we receive a request > to browse from a certain point we can get back the pageful of headings > to display by using a simple SQL SELECT statement. For each heading > listed we determine the number of titles matched and any > cross-references by performing Lucene term queries (fast) on indexes of > our bib data and authority data respectively. All of this is handled by > a Solr browse handler I've written, so all our VuFind code needs to know > is to hit the browse handler and style the XML it gets back. > > Regarding scalability, our largest browse is the callnumber browse, > which consists of about 3 million entries (for 4 million bib records). > I've tested this SQLite approach up to 20 million entries and it > continued to perform well, so I'm not terribly worried for now. Finding > the point to browse from is effectively just searching a big sorted text > file, so I would expect O(log N) growth here anyway. Plus, our largest > SQLite database still fits entirely in memory, so that's nice too. > > In terms of indexing performance, the SQLite databases take about 5-10 > minutes to build in total, and they're built from scratch every time. > For each type of browse we pull all the browse headings from our bib and > authority data, remove any duplicates then load them all into an SQLite > database. My browse handler notices when these databases have been > updated and automatically reopens them, so the update is transparent. > Currently we just do these updates once per night, as this is how often > we update our main bib indexes and it makes sense to keep the updates > synchronised, but I don't see any problem with doing this more often if > it made sense. > > I'm happy to answer any questions about our implementation either on or > off list. > > Cheers, > > Mark > > > "Stephens, Owen" > writes: > > > Bernhard, > > Just to understand what you are looking for in terms of Browse. The > NLA implementation of VuFind has what I would regard as a Browse > function - you can Browse the following: > > Names at http://catalogue.nla.gov.au/Browse/Names?browse=names&from= > Subjects at > http://catalogue.nla.gov.au/Browse/Subjects?browse=subjects&from= > Callnumbers at > http://catalogue.nla.gov.au/Browse/Subjects?browse=subjects&from= > Series at > http://catalogue.nla.gov.au/Browse/Series?browse=series&from= > > All these options are available in the user interface at > http://catalogue.nla.gov.au/Browse/Home ('Browse' is an option in the > horizontal menu under the main 'catalogue' banner) > > This page also offers Title and Uniform Title browsing, but these seem > not to work in the same way at the moment (I've sent feedback about > this) > > Is this browsing as you mean it? If not, what would you require > additionally? > > (also you question the scalability - what scale are you thinking of? > I'd guess that NLA is reasonably large - but I can't easily find a > figure for the number of bib records - but obviously it may not be as > large as other national libraries or consortium collections) > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > > From jamie at dang.com Fri Mar 13 13:40:03 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Fri, 13 Mar 2009 13:40:03 -0400 Subject: [Blacklight-development] irc Message-ID: <4B0584F1-61A7-4311-B11B-216FB5F64C7E@dang.com> irc.freenode.net#blacklight From mtriggs at nla.gov.au Fri Mar 13 17:04:21 2009 From: mtriggs at nla.gov.au (Mark Triggs) Date: Sat, 14 Mar 2009 08:04:21 +1100 Subject: [Blacklight-development] browse lists in Blacklight In-Reply-To: <49BA8A5B.60401@jhu.edu> (Jonathan Rochkind's message of "Sat, 14 Mar 2009 03:31:23 +1100") References: <49BA6EE6.4080505@jhu.edu> <066E1600-5855-47D4-A236-C97F115D0184@virginia.edu> <49BA8A5B.60401@jhu.edu> Message-ID: <87ljr9xhgq.fsf@nla.gov.au> Hi all, At the National Library of Australia we incrementally update our Solr indexes once per day and rebuild our browse indexes immediately after doing that. Because it only takes around 5-10 minutes to create the Lucene indexes and SQLite databases required, this isn't a big deal for us. Theoretically I guess the SQLite databases could be updated in place with the right SQL INSERTs, but rebuilding from scratch saved me having to think about messy things like duplicates and deletes so I went with the simpler option of rebuilding every time :o) Cheers, Mark Jonathan Rochkind writes: > Yeah, it sounds like he was using an approach roughly similar to one I > had considered. Building a seperate index of your indexes to keep track > of what range to start at for a given browser query. The potential > problem here is keeping that seperate index-of-the-index up to date as > you continually update your 'main' index. I get the feeling that one of > our goals is allowing an index that can be incrementally updated at any > time, not just in bulk. Will be interesting to see if his design > manages that. -- Mark Triggs From rochkind at jhu.edu Fri Mar 13 17:48:16 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Fri, 13 Mar 2009 17:48:16 -0400 Subject: [Blacklight-development] browse lists in Blacklight In-Reply-To: <87ljr9xhgq.fsf@nla.gov.au> References: <49BA6EE6.4080505@jhu.edu> <066E1600-5855-47D4-A236-C97F115D0184@virginia.edu> <49BA8A5B.60401@jhu.edu> <87ljr9xhgq.fsf@nla.gov.au> Message-ID: <49BAD4A0.6000401@jhu.edu> Interesting. Glad to see you on the list, Mark. I was kind of hoping to be able to update the SOLR index an individual record at a time, on demand. I think others had some ideas of implementing a realtime update whenever the bib/item status changed in our catalog, or polling for updates every 5 or 10 minutes, etc. I guess making that work with the browse indexes would require thinking about the messy process of updating the SQLite databases in place, which could get pretty tricky. The other approach I was thinking about when trying to approach this problem was, instead of having extra indexes-of-indexes to determine where to start for a given browse request (that's the point of them, right?), was actually doing a binary-search algorithm on the indexes themselves for each browse. They want to see a browse starting at "Twain, Mark". How do we know what item to start them at in the Solr index? Well, find out the total number of items in the index, check the one in the middle, is it higher or lower? Go cut the next half in the middle, etc. I'm not sure if that would really be performance workable or not, since it would require a bunch of requests to find the correct 'start point'. I don't have it in me this late on a Friday to the math. :) Jonathan Mark Triggs wrote: > Hi all, > > At the National Library of Australia we incrementally update our Solr > indexes once per day and rebuild our browse indexes immediately after > doing that. Because it only takes around 5-10 minutes to create the > Lucene indexes and SQLite databases required, this isn't a big deal for > us. > > Theoretically I guess the SQLite databases could be updated in place > with the right SQL INSERTs, but rebuilding from scratch saved me having > to think about messy things like duplicates and deletes so I went with > the simpler option of rebuilding every time :o) > > Cheers, > > Mark > > > Jonathan Rochkind writes: > > >> Yeah, it sounds like he was using an approach roughly similar to one I >> had considered. Building a seperate index of your indexes to keep track >> of what range to start at for a given browser query. The potential >> problem here is keeping that seperate index-of-the-index up to date as >> you continually update your 'main' index. I get the feeling that one of >> our goals is allowing an index that can be incrementally updated at any >> time, not just in bulk. Will be interesting to see if his design >> manages that. >> > > From mtriggs at nla.gov.au Fri Mar 13 18:21:45 2009 From: mtriggs at nla.gov.au (Mark Triggs) Date: Sat, 14 Mar 2009 09:21:45 +1100 Subject: [Blacklight-development] browse lists in Blacklight In-Reply-To: <49BAD4A0.6000401@jhu.edu> (Jonathan Rochkind's message of "Sat, 14 Mar 2009 08:48:16 +1100") References: <49BA6EE6.4080505@jhu.edu> <066E1600-5855-47D4-A236-C97F115D0184@virginia.edu> <49BA8A5B.60401@jhu.edu> <87ljr9xhgq.fsf@nla.gov.au> <49BAD4A0.6000401@jhu.edu> Message-ID: <87bps5vzba.fsf@nla.gov.au> Hi Jonathan, The point of the SQLite indexes in my implementation are really to answer questions like "The user started browsing from Twain, Mark and they're on page 5. What are the headings they're going to see on this page?". When my Solr handler gets a request to start browsing from Twain, Mark, it: 1. Queries the appropriate SQLite database for a name browse and executes some SQL like: SELECT heading from headings WHERE sortkey >= 'twain, mark' ORDER BY sortkey LIMIT 20 (this isn't *strictly* accurate because I actually do some fiddling with offsets and rowids to support browsing backwards and forwards through pages, but you get the idea) 2. Gets back a list of headings from SQLite: ['Twain, Mark', 'Twain, Mark, 1835-1910', 'Twait, Scott', ...] 3. For each heading, it performs a TermQuery against the bib index to determine how many records contain each heading (filling out the "Titles" column in the browse). 4. For each heading, it performs a TermQuery against a Lucene index of our authority data to determine whether any "See instead" or "See also" links need to be displayed. 5. Bundles all of that into an XML representation and ships it back to the caller. So, the SQLite database is really what determines which headings appear in which browse list, and (importantly) the way they are sorted. I did think a bit about how this could be done purely using Lucene, but a few experiments I did weren't terribly promising. If you take the approach of using a TermEnum to build the browse list then you don't have any control (that I could see) over the sort ordering of the headings you get back. If you try to use a regular IndexSearcher with sorting then performance seemed to suffer and it's not obvious how to do backwards and forwards pagination. Regarding the issues with more regular updates, the hard part seems to be dealing with headings that need to be removed. I wonder if an approach of dealing with additions at index time and deletions at query time might work: * For each record indexed, extract a list of all its headings. Check that each heading appears in the SQLite table and INSERT it if it doesn't. * At runtime, when the list of headings for a page is being calculated (step 1 above), we want to exclude any heading where: - that heading is a preferred form that doesn't appear in the bib data any more; or - that heading is a non-preferred form whose preferred form doesn't appear in the bib data any more So, you would start out doing a SELECT .. LIMIT 20, then if you found that two headings in your list needed to be removed, repeat with SELECT .. LIMIT 22, and keep doing that until you either get a pageful of headings or run out of headings completely. Yuck! That's still pretty hairy, and I haven't tested it so it probably doesn't work either. I think I'd better sit down :o) Cheers, Mark Jonathan Rochkind writes: > Interesting. Glad to see you on the list, Mark. > > I was kind of hoping to be able to update the SOLR index an individual > record at a time, on demand. I think others had some ideas of > implementing a realtime update whenever the bib/item status changed in > our catalog, or polling for updates every 5 or 10 minutes, etc. > > I guess making that work with the browse indexes would require thinking > about the messy process of updating the SQLite databases in place, which > could get pretty tricky. > > The other approach I was thinking about when trying to approach this > problem was, instead of having extra indexes-of-indexes to determine > where to start for a given browse request (that's the point of them, > right?), was actually doing a binary-search algorithm on the indexes > themselves for each browse. They want to see a browse starting at > "Twain, Mark". How do we know what item to start them at in the Solr > index? Well, find out the total number of items in the index, check the > one in the middle, is it higher or lower? Go cut the next half in the > middle, etc. I'm not sure if that would really be performance workable > or not, since it would require a bunch of requests to find the correct > 'start point'. I don't have it in me this late on a Friday to the math. :) -- Mark Triggs From rochkind at jhu.edu Fri Mar 13 18:36:15 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Fri, 13 Mar 2009 18:36:15 -0400 Subject: [Blacklight-development] browse lists in Blacklight In-Reply-To: <87bps5vzba.fsf@nla.gov.au> References: <49BA6EE6.4080505@jhu.edu> <066E1600-5855-47D4-A236-C97F115D0184@virginia.edu> <49BA8A5B.60401@jhu.edu> <87ljr9xhgq.fsf@nla.gov.au> <49BAD4A0.6000401@jhu.edu> <87bps5vzba.fsf@nla.gov.au> Message-ID: <49BADFDF.7070300@jhu.edu> Yeah, makes sense. I had been thinking of trying to use Solr a bit more, as I think Solr does have a way to display an entire list of unique values from an index. But what it doesn't have is a way to know what i-value to start on if you want to start at "Twain, Mark" instead of the beginning. So basically the same thing you're talking about, yeah. Jonathan Mark Triggs wrote: > Hi Jonathan, > > The point of the SQLite indexes in my implementation are really to > answer questions like "The user started browsing from Twain, Mark and > they're on page 5. What are the headings they're going to see on this > page?". When my Solr handler gets a request to start browsing from > Twain, Mark, it: > > 1. Queries the appropriate SQLite database for a name browse and > executes some SQL like: > > SELECT heading from headings > WHERE sortkey >= 'twain, mark' > ORDER BY sortkey > LIMIT 20 > > (this isn't *strictly* accurate because I actually do some fiddling > with offsets and rowids to support browsing backwards and forwards > through pages, but you get the idea) > > 2. Gets back a list of headings from SQLite: > > ['Twain, Mark', 'Twain, Mark, 1835-1910', 'Twait, Scott', ...] > > 3. For each heading, it performs a TermQuery against the bib index to > determine how many records contain each heading (filling out the > "Titles" column in the browse). > > 4. For each heading, it performs a TermQuery against a Lucene index of > our authority data to determine whether any "See instead" or "See > also" links need to be displayed. > > 5. Bundles all of that into an XML representation and ships it back to > the caller. > > > So, the SQLite database is really what determines which headings appear > in which browse list, and (importantly) the way they are sorted. I did > think a bit about how this could be done purely using Lucene, but a few > experiments I did weren't terribly promising. If you take the approach > of using a TermEnum to build the browse list then you don't have any > control (that I could see) over the sort ordering of the headings you > get back. If you try to use a regular IndexSearcher with sorting then > performance seemed to suffer and it's not obvious how to do backwards > and forwards pagination. > > Regarding the issues with more regular updates, the hard part seems to > be dealing with headings that need to be removed. I wonder if an > approach of dealing with additions at index time and deletions at query > time might work: > > * For each record indexed, extract a list of all its headings. Check > that each heading appears in the SQLite table and INSERT it if it > doesn't. > > * At runtime, when the list of headings for a page is being > calculated (step 1 above), we want to exclude any heading where: > > - that heading is a preferred form that doesn't appear in the bib > data any more; or > > - that heading is a non-preferred form whose preferred form > doesn't appear in the bib data any more > > So, you would start out doing a SELECT .. LIMIT 20, then if you > found that two headings in your list needed to be removed, repeat > with SELECT .. LIMIT 22, and keep doing that until you either get a > pageful of headings or run out of headings completely. > > > Yuck! That's still pretty hairy, and I haven't tested it so it probably > doesn't work either. I think I'd better sit down :o) > > Cheers, > > Mark > > > Jonathan Rochkind writes: > > >> Interesting. Glad to see you on the list, Mark. >> >> I was kind of hoping to be able to update the SOLR index an individual >> record at a time, on demand. I think others had some ideas of >> implementing a realtime update whenever the bib/item status changed in >> our catalog, or polling for updates every 5 or 10 minutes, etc. >> >> I guess making that work with the browse indexes would require thinking >> about the messy process of updating the SQLite databases in place, which >> could get pretty tricky. >> >> The other approach I was thinking about when trying to approach this >> problem was, instead of having extra indexes-of-indexes to determine >> where to start for a given browse request (that's the point of them, >> right?), was actually doing a binary-search algorithm on the indexes >> themselves for each browse. They want to see a browse starting at >> "Twain, Mark". How do we know what item to start them at in the Solr >> index? Well, find out the total number of items in the index, check the >> one in the middle, is it higher or lower? Go cut the next half in the >> middle, etc. I'm not sure if that would really be performance workable >> or not, since it would require a bunch of requests to find the correct >> 'start point'. I don't have it in me this late on a Friday to the math. :) >> > > From ndushay at stanford.edu Fri Mar 13 22:10:14 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Fri, 13 Mar 2009 19:10:14 -0700 Subject: [Blacklight-development] browse lists in Blacklight In-Reply-To: <49BADFDF.7070300@jhu.edu> References: <49BA6EE6.4080505@jhu.edu> <066E1600-5855-47D4-A236-C97F115D0184@virginia.edu> <49BA8A5B.60401@jhu.edu> <87ljr9xhgq.fsf@nla.gov.au> <49BAD4A0.6000401@jhu.edu> <87bps5vzba.fsf@nla.gov.au> <49BADFDF.7070300@jhu.edu> Message-ID: <4C88220F-62A4-4401-B56E-D6FEA06F2333@stanford.edu> AFAIK, Lucene has a way for you to step through the unique values in lexical order, but it does NOT have a way to step through values in lexical order AND easily figure out which document(s) contain those values.. Were that true, range queries such as: callnumber:[AAA1234.5 TO *] rows:10 would be so fast that you wouldn't need the database. http://lucene.apache.org/java/2_4_1/fileformats.html If I totally missed the point of the discussion ... whoops! - Naomi On Mar 13, 2009, at 3:36 PM, Jonathan Rochkind wrote: > Yeah, makes sense. > I had been thinking of trying to use Solr a bit more, as I think > Solr does have a way to display an entire list of unique values from > an index. But what it doesn't have is a way to know what i-value to > start on if you want to start at "Twain, Mark" instead of the > beginning. So basically the same thing you're talking about, yeah. > > Jonathan > > Mark Triggs wrote: >> Hi Jonathan, >> >> The point of the SQLite indexes in my implementation are really to >> answer questions like "The user started browsing from Twain, Mark and >> they're on page 5. What are the headings they're going to see on >> this >> page?". When my Solr handler gets a request to start browsing from >> Twain, Mark, it: >> >> 1. Queries the appropriate SQLite database for a name browse and >> executes some SQL like: >> >> SELECT heading from headings >> WHERE sortkey >= 'twain, mark' >> ORDER BY sortkey >> LIMIT 20 >> >> (this isn't *strictly* accurate because I actually do some >> fiddling >> with offsets and rowids to support browsing backwards and >> forwards >> through pages, but you get the idea) >> >> 2. Gets back a list of headings from SQLite: >> >> ['Twain, Mark', 'Twain, Mark, 1835-1910', 'Twait, Scott', ...] >> >> 3. For each heading, it performs a TermQuery against the bib index >> to >> determine how many records contain each heading (filling out the >> "Titles" column in the browse). >> >> 4. For each heading, it performs a TermQuery against a Lucene >> index of >> our authority data to determine whether any "See instead" or "See >> also" links need to be displayed. >> >> 5. Bundles all of that into an XML representation and ships it >> back to >> the caller. >> >> >> So, the SQLite database is really what determines which headings >> appear >> in which browse list, and (importantly) the way they are sorted. I >> did >> think a bit about how this could be done purely using Lucene, but a >> few >> experiments I did weren't terribly promising. If you take the >> approach >> of using a TermEnum to build the browse list then you don't have any >> control (that I could see) over the sort ordering of the headings you >> get back. If you try to use a regular IndexSearcher with sorting >> then >> performance seemed to suffer and it's not obvious how to do backwards >> and forwards pagination. >> >> Regarding the issues with more regular updates, the hard part seems >> to >> be dealing with headings that need to be removed. I wonder if an >> approach of dealing with additions at index time and deletions at >> query >> time might work: >> >> * For each record indexed, extract a list of all its headings. >> Check >> that each heading appears in the SQLite table and INSERT it if it >> doesn't. >> >> * At runtime, when the list of headings for a page is being >> calculated (step 1 above), we want to exclude any heading where: >> >> - that heading is a preferred form that doesn't appear in the >> bib >> data any more; or >> >> - that heading is a non-preferred form whose preferred form >> doesn't appear in the bib data any more >> >> So, you would start out doing a SELECT .. LIMIT 20, then if you >> found that two headings in your list needed to be removed, repeat >> with SELECT .. LIMIT 22, and keep doing that until you either >> get a >> pageful of headings or run out of headings completely. >> >> >> Yuck! That's still pretty hairy, and I haven't tested it so it >> probably >> doesn't work either. I think I'd better sit down :o) >> >> Cheers, >> >> Mark >> >> >> Jonathan Rochkind writes: >> >> >>> Interesting. Glad to see you on the list, Mark. >>> >>> I was kind of hoping to be able to update the SOLR index an >>> individual >>> record at a time, on demand. I think others had some ideas of >>> implementing a realtime update whenever the bib/item status >>> changed in >>> our catalog, or polling for updates every 5 or 10 minutes, etc. >>> >>> I guess making that work with the browse indexes would require >>> thinking >>> about the messy process of updating the SQLite databases in place, >>> which >>> could get pretty tricky. >>> >>> The other approach I was thinking about when trying to approach this >>> problem was, instead of having extra indexes-of-indexes to determine >>> where to start for a given browse request (that's the point of them, >>> right?), was actually doing a binary-search algorithm on the indexes >>> themselves for each browse. They want to see a browse starting at >>> "Twain, Mark". How do we know what item to start them at in the Solr >>> index? Well, find out the total number of items in the index, >>> check the >>> one in the middle, is it higher or lower? Go cut the next half in >>> the >>> middle, etc. I'm not sure if that would really be performance >>> workable >>> or not, since it would require a bunch of requests to find the >>> correct >>> 'start point'. I don't have it in me this late on a Friday to the >>> math. :) >>> >> >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From erikhatcher at mac.com Sat Mar 14 05:44:24 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Sat, 14 Mar 2009 05:44:24 -0400 Subject: [Blacklight-development] browse lists in Blacklight In-Reply-To: <4C88220F-62A4-4401-B56E-D6FEA06F2333@stanford.edu> References: <49BA6EE6.4080505@jhu.edu> <066E1600-5855-47D4-A236-C97F115D0184@virginia.edu> <49BA8A5B.60401@jhu.edu> <87ljr9xhgq.fsf@nla.gov.au> <49BAD4A0.6000401@jhu.edu> <87bps5vzba.fsf@nla.gov.au> <49BADFDF.7070300@jhu.edu> <4C88220F-62A4-4401-B56E-D6FEA06F2333@stanford.edu> Message-ID: <0AFC18EB-0D76-4BD6-BE02-FC83AD7817C8@mac.com> On Mar 13, 2009, at 10:10 PM, Naomi Dushay wrote: > Lucene has a way for you to step through the unique values in > lexical order, but it does NOT have a way to step through values in > lexical order AND easily figure out which document(s) contain those > values.. Ummmm.... > Were that true, range queries such as: > > callnumber:[AAA1234.5 TO *] rows:10 I'm not sure what that rows:10 is in the query, but maybe you mean &start=0&rows=10 in Solr-speak. But Lucene absolutely does allow lexicographic range queries, such as text:[aardvark TO zebra] as a standard query parser expression. And in Solr a * can be used at either (or both) ends of a range query to open end it. If I recall correctly, the difficulty was actually getting a lexicographical orderable value of your call numbers. Right? > would be so fast that you wouldn't need the database. And yes, it would! Erik From erikhatcher at mac.com Sat Mar 14 07:13:02 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Sat, 14 Mar 2009 07:13:02 -0400 Subject: [Blacklight-development] browse lists in Blacklight In-Reply-To: <49BADFDF.7070300@jhu.edu> References: <49BA6EE6.4080505@jhu.edu> <066E1600-5855-47D4-A236-C97F115D0184@virginia.edu> <49BA8A5B.60401@jhu.edu> <87ljr9xhgq.fsf@nla.gov.au> <49BAD4A0.6000401@jhu.edu> <87bps5vzba.fsf@nla.gov.au> <49BADFDF.7070300@jhu.edu> Message-ID: On Mar 13, 2009, at 6:36 PM, Jonathan Rochkind wrote: > I had been thinking of trying to use Solr a bit more, as I think > Solr does have a way to display an entire list of unique values from > an index. Well, faceting is a prime example of exactly that, unique values from an indexed field :) But there are other ways to get at deeper info - there is the Luke request handler, term component, and term vector component. Remember #4 from my code4lib09 preso: http://code4lib.org/files/code4lib09-SolrRisingSun.pdf If Solr doesn't provide access to the Lucene data you want, it's easy to add a component or request handler that can. > But what it doesn't have is a way to know what i-value to start on > if you want to start at "Twain, Mark" instead of the beginning. Can you elaborate on this and what is missing exactly? I have to admit that even after Naomi patiently describing this to me I guess I still don't grok what's missing for Solr to be able to provide this capability. This is a case where if we can demonstrate from some fixed data what we want that we can't get out of Solr, we can take it to the solr-user community and ask about how to get there from here. Erik From ndushay at stanford.edu Sat Mar 14 13:22:29 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Sat, 14 Mar 2009 10:22:29 -0700 Subject: [Blacklight-development] browse lists in Blacklight In-Reply-To: <0AFC18EB-0D76-4BD6-BE02-FC83AD7817C8@mac.com> References: <49BA6EE6.4080505@jhu.edu> <066E1600-5855-47D4-A236-C97F115D0184@virginia.edu> <49BA8A5B.60401@jhu.edu> <87ljr9xhgq.fsf@nla.gov.au> <49BAD4A0.6000401@jhu.edu> <87bps5vzba.fsf@nla.gov.au> <49BADFDF.7070300@jhu.edu> <4C88220F-62A4-4401-B56E-D6FEA06F2333@stanford.edu> <0AFC18EB-0D76-4BD6-BE02-FC83AD7817C8@mac.com> Message-ID: <3B578C94-FCDB-4B04-9309-20EC2DF9003A@stanford.edu> On Mar 14, 2009, at 2:44 AM, Erik Hatcher wrote: > > On Mar 13, 2009, at 10:10 PM, Naomi Dushay wrote: >> Lucene has a way for you to step through the unique values in >> lexical order, but it does NOT have a way to step through values in >> lexical order AND easily figure out which document(s) contain those >> values.. > > Ummmm.... > >> Were that true, range queries such as: >> >> callnumber:[AAA1234.5 TO *] rows:10 > > I'm not sure what that rows:10 is in the query, but maybe you mean > &start=0&rows=10 in Solr-speak. Yes, that's what I meant. I typed incorrect syntax. > But Lucene absolutely does allow lexicographic range queries, such > as text:[aardvark TO zebra] as a standard query parser expression. > And in Solr a * can be used at either (or both) ends of a range > query to open end it. > > If I recall correctly, the difficulty was actually getting a > lexicographical orderable value of your call numbers. Right? I already wrote code to get from LC or Dewey call numbers to a lexicographical orderable value of the same. What I need: A way to get the *documents* for the next 10 call numbers and the previous 10 call numbers. I need not just the call number itself, but title, author, location in order to do a "nearby on shelf" presentation for our users. It's easy-peasy to get the call numbers themselves, as you point out. The other issue revolves around the lack of a specific upper (or lower) bounds in the query. We must have [M123.4 TO *] and [* TO M123.4] because we don't know what call number will produce at least 10 rows. This is a veeeeeery slow query -- I posted to the solr-user list on this issue, and the experts said "yes, this is a very slow query." That's why the NLA folks started using a database for their lookups into a *sorted* list of values. Here's the thread: http://www.mail-archive.com/solr-user at lucene.apache.org/msg16159.html "But looking at what Hoss wrote you might actually want the first 10 results sorted by field myField, that meet the restriction that callNumer>X (a filter). If so, this is a long standing problem that isn't solved yet (slowness of range queries with many terms), but we do know methods of fixing it, most involving indexing prefixes." Glen Newton gave work around. http://www.mail-archive.com/solr-user at lucene.apache.org/msg16193.html which I haven't had a chance to implement. - Naomi From eos8d at virginia.edu Mon Mar 16 09:22:06 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Mon, 16 Mar 2009 09:22:06 -0400 Subject: [Blacklight-development] Please help Naomi get solr-helper committed Message-ID: <739DC6F6-DA40-4F74-8E18-0AC7F6EBA646@virginia.edu> Hi, Everyone. Naomi is trying right now to commit her solr-helper refactoring code. This will include some pretty major changes, and she's trying to get everything working with the current code in the engines plugin branch. Please, everyone hold off on committing any changes until Naomi tells us that she has committed hers. Then we'll all need to svn up and make sure any local changes we have in the pipe work with the new svn HEAD and then it will be safe to commit again. Naomi estimates she can have this commit done w/in the next couple of hours. Thanks, everyone! And thanks, Naomi! It's going to be so much easier to get BL running against institutional-specific indexes soon. Bess From ndushay at stanford.edu Mon Mar 16 11:46:33 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Mon, 16 Mar 2009 08:46:33 -0700 Subject: [Blacklight-development] refactoring for solr_helper - done? Message-ID: Fellow-blacklighters, I just committed the changes I've been working on. Main thrusts: 1. abstract out the solr request concepts; hide solr parameter details in solr_helper class, used by controllers as necessary 2. stop hardcoding solr field names to views; abstract them with solr.yml and display_fields class (see below) DO NOT hardcode solr field names, or the code will be brittle and annoying to get working with site indexes. 3. use document request handler (not details) when doing single document requests *** note that the "prev" and "next" buttons use the search handler. " All changes are in the plugin, except * solrconfig.xml <--- added document request handler (no need to reindex - just restart solr) * solr.yml (via solr.example) this is where the mapping of solr fields to abstract names occurs So, you should be able to get these updates working for you in three steps: 1. svn update 2. update solr.yml file in your rails app (restart rails app) 3. update solrconfig.xml in your solr instance (restart solr) Plugin changes: vendor/plugin/blacklight * init.rb modified to include display_fields .../lib * blacklight.rb <-- added document request handler * display_fields.rb added <-- assigns display info from solr.yml to variables .../app/controller <-- removed hardcoded solr field names, use solr_helper * catalog * home .../app/helper <-- removed hardcoded solr field names * catalog * solr_helper <-- THE STAR OF THE SHOW .../app/views/catalog <-- removed hardcoded solr field names * _document_list partial * show_index Specs: * catalog_controller: major changes; it should be less brittle now * home_controller: reworked; less brittle * view/catalog/show: reworked and expanded, less brittle added 2 specs * solr_helper * catalog/_document_list Did NOT do: 1. index.rss : I'm guessing this will be site specific, and will also involve enough fields used only for this purpose as to be silly for solr.yml 2. README updates: will be coming along soon. 3. giant purge of vestigial facet action from catalog_controller, and other related vestigial code. I took some of it out, but wasn't thorough. - Naomi p.s. it turns out that the world doesn't stop just because your coding hits a lot of snags From ndushay at stanford.edu Mon Mar 16 12:13:13 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Mon, 16 Mar 2009 09:13:13 -0700 Subject: [Blacklight-development] single document questions / concerns Message-ID: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> While refactoring the code, I found I had some questions about the single document display. The url for a single doc used to be: http://(blah):3000/catalog/show/00009103 Now the url that displays is http://searchworks-dev.stanford.edu:3000/catalog/00009103? counter=3&f[language_facet][]=English or something like it; it includes the query and whatever facets have been selected, and the position in the results. I understand that in order to do the "next" and "prev" links in the show/document view, we need to keep track of where we are in the search results, so we either need to retain the query information, or to cache the results. But ... we have lost those lovely clean urls. I'm aware that this has been discussed and it's a trade off ... but it makes me sad. Anyone have any brilliant solutions? Could we make the request "post" to hide the parameters? While coding, I frequently saw a :num_per_page parameter from single document requests (via next and prev buttons). i don't think it's needed. I may have already fixed this. Next, a rails n00b question: I can't find how do we get from the display url, which has no "show" in it, to the show action. - Naomi From rochkind at jhu.edu Mon Mar 16 12:20:05 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 16 Mar 2009 12:20:05 -0400 Subject: [Blacklight-development] refactoring for solr_helper - done? In-Reply-To: References: Message-ID: <49BE7C35.7040005@jhu.edu> These are awesome changes, thanks Naomi. Naomi Dushay wrote: > Fellow-blacklighters, > > I just committed the changes I've been working on. Main thrusts: > > 1. abstract out the solr request concepts; hide solr parameter > details in solr_helper class, used by controllers as necessary > 2. stop hardcoding solr field names to views; abstract them with > solr.yml and display_fields class (see below) > DO NOT hardcode solr field names, or the code will be brittle and > annoying to get working with site indexes. > 3. use document request handler (not details) when doing single > document requests > *** note that the "prev" and "next" buttons use the search handler. " > > > All changes are in the plugin, except > > * solrconfig.xml <--- added document request handler (no need to > reindex - just restart solr) > * solr.yml (via solr.example) this is where the mapping of solr > fields to abstract names occurs > > > So, you should be able to get these updates working for you in three > steps: > > 1. svn update > 2. update solr.yml file in your rails app (restart rails app) > 3. update solrconfig.xml in your solr instance (restart solr) > > > Plugin changes: > > vendor/plugin/blacklight > * init.rb modified to include display_fields > > .../lib > * blacklight.rb <-- added document request handler > * display_fields.rb added <-- assigns display info from > solr.yml to variables > > .../app/controller <-- removed hardcoded solr field names, use > solr_helper > * catalog > * home > > .../app/helper <-- removed hardcoded solr field names > * catalog > * solr_helper <-- THE STAR OF THE SHOW > > .../app/views/catalog <-- removed hardcoded solr field names > * _document_list partial > * show_index > > > Specs: > > * catalog_controller: major changes; it should be less brittle now > * home_controller: reworked; less brittle > * view/catalog/show: reworked and expanded, less brittle > > added 2 specs > * solr_helper > * catalog/_document_list > > > > Did NOT do: > > 1. index.rss : I'm guessing this will be site specific, and will > also involve enough fields used only for this purpose as to be silly > for solr.yml > > 2. README updates: will be coming along soon. > > 3. giant purge of vestigial facet action from catalog_controller, > and other related vestigial code. I took some of it out, but wasn't > thorough. > > > - Naomi > p.s. it turns out that the world doesn't stop just because your > coding hits a lot of snags > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From jamie at dang.com Mon Mar 16 12:21:41 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Mon, 16 Mar 2009 12:21:41 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> Message-ID: <17054D47-BE9D-45B7-9304-2ED93F2B9719@dang.com> Once you start posting to hide URLs, then you remove the ability of users to copy and paste URLs. It also makes navigation a bit more cumbersome for example, such as when a user hits the "delete" key to go back and now gets prompted with a "do you want to submit this form again" question. Uglier URLs are appropriate in this situation, IMO. Every one represents a unique search and can be copy and pasted. And the request is a simple GET. Jamie On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: > While refactoring the code, I found I had some questions about the > single document display. > > The url for a single doc used to be: > > http://(blah):3000/catalog/show/00009103 > > Now the url that displays is > > http://searchworks-dev.stanford.edu:3000/catalog/00009103? > counter=3&f[language_facet][]=English > > or something like it; it includes the query and whatever facets > have been selected, and the position in the results. I understand > that in order to do the "next" and "prev" links in the show/document > view, we need to keep track of where we are in the search results, > so we either need to retain the query information, or to cache the > results. But ... we have lost those lovely clean urls. I'm aware > that this has been discussed and it's a trade off ... but it makes > me sad. Anyone have any brilliant solutions? Could we make the > request "post" to hide the parameters? > > While coding, I frequently saw a :num_per_page parameter from > single document requests (via next and prev buttons). i don't think > it's needed. I may have already fixed this. > > Next, a rails n00b question: > I can't find how do we get from the display url, which has no > "show" in it, to the show action. > > - Naomi > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From rochkind at jhu.edu Mon Mar 16 12:55:28 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 16 Mar 2009 12:55:28 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <17054D47-BE9D-45B7-9304-2ED93F2B9719@dang.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <17054D47-BE9D-45B7-9304-2ED93F2B9719@dang.com> Message-ID: <49BE8480.5080601@jhu.edu> I'm confused. Why do you need anything other than the internal unique ID to represent the returned record in a way that can be copied and pasted? Jamie Orchard-Hays wrote: > Once you start posting to hide URLs, then you remove the ability of > users to copy and paste URLs. It also makes navigation a bit more > cumbersome for example, such as when a user hits the "delete" key to > go back and now gets prompted with a "do you want to submit this form > again" question. > > Uglier URLs are appropriate in this situation, IMO. Every one > represents a unique search and can be copy and pasted. And the request > is a simple GET. > > Jamie > > On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: > > >> While refactoring the code, I found I had some questions about the >> single document display. >> >> The url for a single doc used to be: >> >> http://(blah):3000/catalog/show/00009103 >> >> Now the url that displays is >> >> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >> counter=3&f[language_facet][]=English >> >> or something like it; it includes the query and whatever facets >> have been selected, and the position in the results. I understand >> that in order to do the "next" and "prev" links in the show/document >> view, we need to keep track of where we are in the search results, >> so we either need to retain the query information, or to cache the >> results. But ... we have lost those lovely clean urls. I'm aware >> that this has been discussed and it's a trade off ... but it makes >> me sad. Anyone have any brilliant solutions? Could we make the >> request "post" to hide the parameters? >> >> While coding, I frequently saw a :num_per_page parameter from >> single document requests (via next and prev buttons). i don't think >> it's needed. I may have already fixed this. >> >> Next, a rails n00b question: >> I can't find how do we get from the display url, which has no >> "show" in it, to the show action. >> >> - Naomi >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From ndushay at stanford.edu Mon Mar 16 12:58:55 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Mon, 16 Mar 2009 09:58:55 -0700 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <17054D47-BE9D-45B7-9304-2ED93F2B9719@dang.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <17054D47-BE9D-45B7-9304-2ED93F2B9719@dang.com> Message-ID: <36097358-5B10-4391-9D76-F5375CA1C6B7@stanford.edu> Right, BUT: if a person is copying and pasting a single document URL, then they probably won't want to "next / prev". And yeah, the "do you want to submit again" is very yucky. On Mar 16, 2009, at 9:21 AM, Jamie Orchard-Hays wrote: > Once you start posting to hide URLs, then you remove the ability of > users to copy and paste URLs. It also makes navigation a bit more > cumbersome for example, such as when a user hits the "delete" key to > go back and now gets prompted with a "do you want to submit this > form again" question. > > Uglier URLs are appropriate in this situation, IMO. Every one > represents a unique search and can be copy and pasted. And the > request is a simple GET. > > Jamie > > On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: > >> While refactoring the code, I found I had some questions about the >> single document display. >> >> The url for a single doc used to be: >> >> http://(blah):3000/catalog/show/00009103 >> >> Now the url that displays is >> >> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >> counter=3&f[language_facet][]=English >> >> or something like it; it includes the query and whatever facets >> have been selected, and the position in the results. I understand >> that in order to do the "next" and "prev" links in the show/ >> document view, we need to keep track of where we are in the search >> results, so we either need to retain the query information, or to >> cache the results. But ... we have lost those lovely clean urls. >> I'm aware that this has been discussed and it's a trade off ... but >> it makes me sad. Anyone have any brilliant solutions? Could we >> make the request "post" to hide the parameters? >> >> While coding, I frequently saw a :num_per_page parameter from >> single document requests (via next and prev buttons). i don't >> think it's needed. I may have already fixed this. >> >> Next, a rails n00b question: >> I can't find how do we get from the display url, which has no >> "show" in it, to the show action. >> >> - Naomi >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From rochkind at jhu.edu Mon Mar 16 13:00:31 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 16 Mar 2009 13:00:31 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <36097358-5B10-4391-9D76-F5375CA1C6B7@stanford.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <17054D47-BE9D-45B7-9304-2ED93F2B9719@dang.com> <36097358-5B10-4391-9D76-F5375CA1C6B7@stanford.edu> Message-ID: <49BE85AF.6010004@jhu.edu> Ah, I get it now. Thanks. Tricky. I agree with Naomi that it would be better to not have those there, but it may be too tricky. Naomi Dushay wrote: > Right, BUT: > > if a person is copying and pasting a single document URL, then they > probably won't want to "next / prev". > > And yeah, the "do you want to submit again" is very yucky. > > > On Mar 16, 2009, at 9:21 AM, Jamie Orchard-Hays wrote: > > >> Once you start posting to hide URLs, then you remove the ability of >> users to copy and paste URLs. It also makes navigation a bit more >> cumbersome for example, such as when a user hits the "delete" key to >> go back and now gets prompted with a "do you want to submit this >> form again" question. >> >> Uglier URLs are appropriate in this situation, IMO. Every one >> represents a unique search and can be copy and pasted. And the >> request is a simple GET. >> >> Jamie >> >> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >> >> >>> While refactoring the code, I found I had some questions about the >>> single document display. >>> >>> The url for a single doc used to be: >>> >>> http://(blah):3000/catalog/show/00009103 >>> >>> Now the url that displays is >>> >>> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >>> counter=3&f[language_facet][]=English >>> >>> or something like it; it includes the query and whatever facets >>> have been selected, and the position in the results. I understand >>> that in order to do the "next" and "prev" links in the show/ >>> document view, we need to keep track of where we are in the search >>> results, so we either need to retain the query information, or to >>> cache the results. But ... we have lost those lovely clean urls. >>> I'm aware that this has been discussed and it's a trade off ... but >>> it makes me sad. Anyone have any brilliant solutions? Could we >>> make the request "post" to hide the parameters? >>> >>> While coding, I frequently saw a :num_per_page parameter from >>> single document requests (via next and prev buttons). i don't >>> think it's needed. I may have already fixed this. >>> >>> Next, a rails n00b question: >>> I can't find how do we get from the display url, which has no >>> "show" in it, to the show action. >>> >>> - Naomi >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From erikhatcher at mac.com Mon Mar 16 13:37:02 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Mon, 16 Mar 2009 13:37:02 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> Message-ID: <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: > Now the url that displays is > > http://searchworks-dev.stanford.edu:3000/catalog/00009103? > counter=3&f[language_facet][]=English > > or something like it; it includes the query and whatever facets > have been selected, and the position in the results. I understand > that in order to do the "next" and "prev" links in the show/document > view, we need to keep track of where we are in the search results, > so we either need to retain the query information, or to cache the > results. But ... we have lost those lovely clean urls. I'm aware > that this has been discussed and it's a trade off ... but it makes > me sad. Anyone have any brilliant solutions? Brilliant?! No, but maybe too obvious... session scope. That's what it is for :) We use it on http://www.lucidimagination.com/search when you're looking at an e-mail detail after having searched. You'll see a "return to search results" link. If you were e-mailed the link and navigated to it directly, you wouldn't see that "return to..." link because you'd have no session with a search context. We could put previous/next links in there too with no change to the URL. Erik From erikhatcher at mac.com Mon Mar 16 13:39:25 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Mon, 16 Mar 2009 13:39:25 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <17054D47-BE9D-45B7-9304-2ED93F2B9719@dang.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <17054D47-BE9D-45B7-9304-2ED93F2B9719@dang.com> Message-ID: <85480226-8926-4B0D-86E5-20D4A9D563F1@mac.com> Another negative about putting search context in a detail URL like that is, IMO, it gives away information you might not really intend to share. Suppose I search for "idiot" and then go to a detail page, then send that link around. Folks know I found a particular hit because I was searching for "idiot", and also what facets I drilled into. It's too much info to share just to show a document detail. Erik On Mar 16, 2009, at 12:21 PM, Jamie Orchard-Hays wrote: > Once you start posting to hide URLs, then you remove the ability of > users to copy and paste URLs. It also makes navigation a bit more > cumbersome for example, such as when a user hits the "delete" key to > go back and now gets prompted with a "do you want to submit this > form again" question. > > Uglier URLs are appropriate in this situation, IMO. Every one > represents a unique search and can be copy and pasted. And the > request is a simple GET. > > Jamie > > On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: > >> While refactoring the code, I found I had some questions about the >> single document display. >> >> The url for a single doc used to be: >> >> http://(blah):3000/catalog/show/00009103 >> >> Now the url that displays is >> >> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >> counter=3&f[language_facet][]=English >> >> or something like it; it includes the query and whatever facets >> have been selected, and the position in the results. I understand >> that in order to do the "next" and "prev" links in the show/ >> document view, we need to keep track of where we are in the search >> results, so we either need to retain the query information, or to >> cache the results. But ... we have lost those lovely clean urls. >> I'm aware that this has been discussed and it's a trade off ... but >> it makes me sad. Anyone have any brilliant solutions? Could we >> make the request "post" to hide the parameters? >> >> While coding, I frequently saw a :num_per_page parameter from >> single document requests (via next and prev buttons). i don't >> think it's needed. I may have already fixed this. >> >> Next, a rails n00b question: >> I can't find how do we get from the display url, which has no >> "show" in it, to the show action. >> >> - Naomi >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From goodieboy at gmail.com Mon Mar 16 13:59:46 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Mon, 16 Mar 2009 13:59:46 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> Message-ID: AJAX! On Mon, Mar 16, 2009 at 1:37 PM, Erik Hatcher wrote: > > On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: > >> Now the url that displays is >> >> http://searchworks-dev.stanford.edu:3000/catalog/00009103 >> ?counter=3&f[language_facet][]=English >> >> or something like it; it includes the query and whatever facets have >> been selected, and the position in the results. I understand that in order >> to do the "next" and "prev" links in the show/document view, we need to keep >> track of where we are in the search results, so we either need to retain the >> query information, or to cache the results. But ... we have lost those >> lovely clean urls. I'm aware that this has been discussed and it's a trade >> off ... but it makes me sad. Anyone have any brilliant solutions? >> > > Brilliant?! No, but maybe too obvious... session scope. > > That's what it is for :) > > We use it on http://www.lucidimagination.com/search when you're looking at > an e-mail detail after having searched. You'll see a "return to search > results" link. If you were e-mailed the link and navigated to it directly, > you wouldn't see that "return to..." link because you'd have no session with > a search context. We could put previous/next links in there too with no > change to the URL. > > Erik > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erikhatcher at mac.com Mon Mar 16 14:02:33 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Mon, 16 Mar 2009 14:02:33 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> Message-ID: <1902E4B7-BC90-46E9-80B7-E9CD8C97872C@mac.com> On the caching front, note that Solr has a query cache for this very purpose, to allow recently used queries to get paged through. Also of note is Solr's use of HTTP caching headers, so you can put Solr behind a caching proxy if you like and have cache Solr responses properly and keep Solr from getting beat up to serve the same thing over and over. Erik On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: > > On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >> Now the url that displays is >> >> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >> counter=3&f[language_facet][]=English >> >> or something like it; it includes the query and whatever facets >> have been selected, and the position in the results. I understand >> that in order to do the "next" and "prev" links in the show/ >> document view, we need to keep track of where we are in the search >> results, so we either need to retain the query information, or to >> cache the results. But ... we have lost those lovely clean urls. >> I'm aware that this has been discussed and it's a trade off ... but >> it makes me sad. Anyone have any brilliant solutions? > > Brilliant?! No, but maybe too obvious... session scope. > > That's what it is for :) > > We use it on http://www.lucidimagination.com/search when you're > looking at an e-mail detail after having searched. You'll see a > "return to search results" link. If you were e-mailed the link and > navigated to it directly, you wouldn't see that "return to..." link > because you'd have no session with a search context. We could put > previous/next links in there too with no change to the URL. > > Erik > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From jkeck at stanford.edu Mon Mar 16 14:12:07 2009 From: jkeck at stanford.edu (Jessie Keck) Date: Mon, 16 Mar 2009 11:12:07 -0700 Subject: [Blacklight-development] HomeController fix Message-ID: <49BE9677.5080504@stanford.edu> Hi all, I'm just writing to let you know that I'll be committing a small fix to the HomeController to fix an issue that came up with Naomi's recent SolrHelper. All that is being done is using the helper method to bring the CatalogHelper into the HomeController. The HomeController renders facets, so it requires some of the functionality that the CatalogHelper provides to the facets. Let me know if you have any questions. -Jessie From jamie at dang.com Mon Mar 16 15:09:09 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Mon, 16 Mar 2009 15:09:09 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> Message-ID: <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> Once you click through to a detail page, the search doesn't really matter. That is, if you're just sharing a detail page, the only important part of the url is whatever gets you to that detail page. But for search results, I'd rather have the info in the URL than hidden in a session somewhere. On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: > > On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >> Now the url that displays is >> >> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >> counter=3&f[language_facet][]=English >> >> or something like it; it includes the query and whatever facets >> have been selected, and the position in the results. I understand >> that in order to do the "next" and "prev" links in the show/ >> document view, we need to keep track of where we are in the search >> results, so we either need to retain the query information, or to >> cache the results. But ... we have lost those lovely clean urls. >> I'm aware that this has been discussed and it's a trade off ... but >> it makes me sad. Anyone have any brilliant solutions? > > Brilliant?! No, but maybe too obvious... session scope. > > That's what it is for :) > > We use it on http://www.lucidimagination.com/search when you're > looking at an e-mail detail after having searched. You'll see a > "return to search results" link. If you were e-mailed the link and > navigated to it directly, you wouldn't see that "return to..." link > because you'd have no session with a search context. We could put > previous/next links in there too with no change to the URL. > > Erik > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From rochkind at jhu.edu Mon Mar 16 15:10:44 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 16 Mar 2009 15:10:44 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> Message-ID: <49BEA434.3080806@jhu.edu> Jamie Orchard-Hays wrote: > But for search results, I'd rather have the info in the URL than > hidden in a session somewhere. > Why? I'd rather have the info that's just about the context hidden somewhere, so that the URL in the browser bar, which may be bookmarked and shared, is context-independent and just represents the record you are looking at. Jonathan > > On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: > > >> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >> >>> Now the url that displays is >>> >>> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >>> counter=3&f[language_facet][]=English >>> >>> or something like it; it includes the query and whatever facets >>> have been selected, and the position in the results. I understand >>> that in order to do the "next" and "prev" links in the show/ >>> document view, we need to keep track of where we are in the search >>> results, so we either need to retain the query information, or to >>> cache the results. But ... we have lost those lovely clean urls. >>> I'm aware that this has been discussed and it's a trade off ... but >>> it makes me sad. Anyone have any brilliant solutions? >>> >> Brilliant?! No, but maybe too obvious... session scope. >> >> That's what it is for :) >> >> We use it on http://www.lucidimagination.com/search when you're >> looking at an e-mail detail after having searched. You'll see a >> "return to search results" link. If you were e-mailed the link and >> navigated to it directly, you wouldn't see that "return to..." link >> because you'd have no session with a search context. We could put >> previous/next links in there too with no change to the URL. >> >> Erik >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From jamie at dang.com Mon Mar 16 15:13:11 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Mon, 16 Mar 2009 15:13:11 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <49BEA434.3080806@jhu.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> Message-ID: <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> I think I've confused what I'm saying by not realizing that Naomi's original post for for a detail, not a search. For a document detail, it makes complete sense not to include the search info in the URL, so I agree with y'all on that. It's only for searches, even facet filters that I like simple GET requests with the info in the URL, even if they're a bit messy. Jamie On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: > Jamie Orchard-Hays wrote: >> But for search results, I'd rather have the info in the URL than >> hidden in a session somewhere. >> > Why? I'd rather have the info that's just about the context hidden > somewhere, so that the URL in the browser bar, which may be > bookmarked and shared, is context-independent and just represents > the record you are looking at. > > Jonathan > > >> >> On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: >> >> >>> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >>> >>>> Now the url that displays is >>>> >>>> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >>>> counter=3&f[language_facet][]=English >>>> >>>> or something like it; it includes the query and whatever >>>> facets have been selected, and the position in the results. I >>>> understand that in order to do the "next" and "prev" links in >>>> the show/ document view, we need to keep track of where we are in >>>> the search results, so we either need to retain the query >>>> information, or to cache the results. But ... we have lost >>>> those lovely clean urls. I'm aware that this has been discussed >>>> and it's a trade off ... but it makes me sad. Anyone have any >>>> brilliant solutions? >>>> >>> Brilliant?! No, but maybe too obvious... session scope. >>> >>> That's what it is for :) >>> >>> We use it on http://www.lucidimagination.com/search when you're >>> looking at an e-mail detail after having searched. You'll see a >>> "return to search results" link. If you were e-mailed the link >>> and navigated to it directly, you wouldn't see that "return >>> to..." link because you'd have no session with a search context. >>> We could put previous/next links in there too with no change to >>> the URL. >>> >>> Erik >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From rochkind at jhu.edu Mon Mar 16 15:25:08 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 16 Mar 2009 15:25:08 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> Message-ID: <49BEA794.6000908@jhu.edu> Agreed that a search URL should have everything in it neccessary to reconstruct the search and be looking at results for the same search if you bookmark it and come back later. Just not an item detail URL. :) Jamie Orchard-Hays wrote: > I think I've confused what I'm saying by not realizing that Naomi's > original post for for a detail, not a search. > > For a document detail, it makes complete sense not to include the > search info in the URL, so I agree with y'all on that. > > It's only for searches, even facet filters that I like simple GET > requests with the info in the URL, even if they're a bit messy. > > Jamie > > On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: > > >> Jamie Orchard-Hays wrote: >> >>> But for search results, I'd rather have the info in the URL than >>> hidden in a session somewhere. >>> >>> >> Why? I'd rather have the info that's just about the context hidden >> somewhere, so that the URL in the browser bar, which may be >> bookmarked and shared, is context-independent and just represents >> the record you are looking at. >> >> Jonathan >> >> >> >>> On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: >>> >>> >>> >>>> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >>>> >>>> >>>>> Now the url that displays is >>>>> >>>>> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >>>>> counter=3&f[language_facet][]=English >>>>> >>>>> or something like it; it includes the query and whatever >>>>> facets have been selected, and the position in the results. I >>>>> understand that in order to do the "next" and "prev" links in >>>>> the show/ document view, we need to keep track of where we are in >>>>> the search results, so we either need to retain the query >>>>> information, or to cache the results. But ... we have lost >>>>> those lovely clean urls. I'm aware that this has been discussed >>>>> and it's a trade off ... but it makes me sad. Anyone have any >>>>> brilliant solutions? >>>>> >>>>> >>>> Brilliant?! No, but maybe too obvious... session scope. >>>> >>>> That's what it is for :) >>>> >>>> We use it on http://www.lucidimagination.com/search when you're >>>> looking at an e-mail detail after having searched. You'll see a >>>> "return to search results" link. If you were e-mailed the link >>>> and navigated to it directly, you wouldn't see that "return >>>> to..." link because you'd have no session with a search context. >>>> We could put previous/next links in there too with no change to >>>> the URL. >>>> >>>> Erik >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From erikhatcher at mac.com Mon Mar 16 15:33:42 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Mon, 16 Mar 2009 15:33:42 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> Message-ID: <5D4006F0-9BA0-44D3-AC27-D64EFEFDA183@mac.com> On Mar 16, 2009, at 3:13 PM, Jamie Orchard-Hays wrote: > It's only for searches, even facet filters that I like simple GET > requests with the info in the URL, even if they're a bit messy. There's no reason that URLs need to be as messy as they are in Blacklight currently, is there? For example: Faceted into the solr project, and the email and wiki data sources with a full-text query of blacklight. Not too shabby. Erik From goodieboy at gmail.com Mon Mar 16 16:25:33 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Mon, 16 Mar 2009 16:25:33 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <49BEA794.6000908@jhu.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <49BEA794.6000908@jhu.edu> Message-ID: Wouldn't if be possible to get the next/previous items using faceting? Just curious. The reason we included the search params in the item-level url, is so you can click "<< back". It's interesting to think about someone coming to a bookmarked item for the first time, and having the ability to go back to the search from where the item came. Including the params in the item-level url gives us that possibility. If you don't want to see the next/previous params in the url... AJAX would solve the problem perfectly. You wouldn't need to track the current index in the url, and you wouldn't have to work out a POST solution. For the messy url param thing... we could (and there is code in the plugin's lib dir for doing this) write our own handler to make the urls pretty. I decided to stick with the Rails default "[]" funkiness to keep it simple though. Matt On Mon, Mar 16, 2009 at 3:25 PM, Jonathan Rochkind wrote: > Agreed that a search URL should have everything in it neccessary to > reconstruct the search and be looking at results for the same search if you > bookmark it and come back later. Just not an item detail URL. :) > > > Jamie Orchard-Hays wrote: > >> I think I've confused what I'm saying by not realizing that Naomi's >> original post for for a detail, not a search. >> >> For a document detail, it makes complete sense not to include the search >> info in the URL, so I agree with y'all on that. >> >> It's only for searches, even facet filters that I like simple GET >> requests with the info in the URL, even if they're a bit messy. >> >> Jamie >> >> On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: >> >> >> >>> Jamie Orchard-Hays wrote: >>> >>> >>>> But for search results, I'd rather have the info in the URL than >>>> hidden in a session somewhere. >>>> >>>> >>>> >>> Why? I'd rather have the info that's just about the context hidden >>> somewhere, so that the URL in the browser bar, which may be bookmarked and >>> shared, is context-independent and just represents the record you are >>> looking at. >>> >>> Jonathan >>> >>> >>> >>> >>>> On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: >>>> >>>> >>>> >>>> >>>>> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >>>>> >>>>> >>>>> >>>>>> Now the url that displays is >>>>>> >>>>>> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >>>>>> counter=3&f[language_facet][]=English >>>>>> >>>>>> or something like it; it includes the query and whatever facets >>>>>> have been selected, and the position in the results. I understand that >>>>>> in order to do the "next" and "prev" links in the show/ document view, we >>>>>> need to keep track of where we are in the search results, so we either >>>>>> need to retain the query information, or to cache the results. But ... we >>>>>> have lost those lovely clean urls. I'm aware that this has been discussed >>>>>> and it's a trade off ... but it makes me sad. Anyone have any brilliant >>>>>> solutions? >>>>>> >>>>>> >>>>>> >>>>> Brilliant?! No, but maybe too obvious... session scope. >>>>> >>>>> That's what it is for :) >>>>> >>>>> We use it on http://www.lucidimagination.com/search when you're >>>>> looking at an e-mail detail after having searched. You'll see a "return >>>>> to search results" link. If you were e-mailed the link and navigated to >>>>> it directly, you wouldn't see that "return to..." link because you'd have >>>>> no session with a search context. We could put previous/next links in >>>>> there too with no change to the URL. >>>>> >>>>> Erik >>>>> _______________________________________________ >>>>> Blacklight-development mailing list >>>>> Blacklight-development at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>>> >>>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rochkind at jhu.edu Mon Mar 16 16:53:52 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 16 Mar 2009 16:53:52 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <49BEA794.6000908@jhu.edu> Message-ID: <49BEBC60.9060205@jhu.edu> You see it as a good thing that someone can come to the detail URL for the first time and go back to the search it came from? I'm not sure this is a good thing. As someone else mentioned, it's even a potential privacy issue. I think it's preferable if the item detail doesn't carry that info both from a url-prettifying point of view, and because it's actually better functionality to me. I'm not sure I understand the AJAX-technique approach you're suggesting. I do think it's important that all the major features of Blacklight work even if the browser does not have javascript. (For a variety of reasons, including mobile device accessibility). It's generally possible to do any fancy AJAX things as an add on on top, such that even with no js present everything still works okay (if clunkier). I'm not sure that's compatible with what you're suggesting though. What do you have against Erik's session-storage suggestion? Jonathan Matt Mitchell wrote: > Wouldn't if be possible to get the next/previous items using faceting? Just curious. > > The reason we included the search params in the item-level url, is so you can click "<< back". It's interesting to think about someone coming to a bookmarked item for the first time, and having the ability to go back to the search from where the item came. Including the params in the item-level url gives us that possibility. > > If you don't want to see the next/previous params in the url... AJAX would solve the problem perfectly. You wouldn't need to track the current index in the url, and you wouldn't have to work out a POST solution. > > For the messy url param thing... we could (and there is code in the plugin's lib dir for doing this) write our own handler to make the urls pretty. I decided to stick with the Rails default "[]" funkiness to keep it simple though. > > Matt > > On Mon, Mar 16, 2009 at 3:25 PM, Jonathan Rochkind > wrote: > Agreed that a search URL should have everything in it neccessary to reconstruct the search and be looking at results for the same search if you bookmark it and come back later. Just not an item detail URL. :) > > > Jamie Orchard-Hays wrote: > I think I've confused what I'm saying by not realizing that Naomi's original post for for a detail, not a search. > > For a document detail, it makes complete sense not to include the search info in the URL, so I agree with y'all on that. > > It's only for searches, even facet filters that I like simple GET requests with the info in the URL, even if they're a bit messy. > > Jamie > > On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: > > > Jamie Orchard-Hays wrote: > > But for search results, I'd rather have the info in the URL than hidden in a session somewhere. > > > Why? I'd rather have the info that's just about the context hidden somewhere, so that the URL in the browser bar, which may be bookmarked and shared, is context-independent and just represents the record you are looking at. > > Jonathan > > > > On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: > > > > On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: > > > Now the url that displays is > > http://searchworks-dev.stanford.edu:3000/catalog/00009103? counter=3&f[language_facet][]=English > > or something like it; it includes the query and whatever facets have been selected, and the position in the results. I understand that in order to do the "next" and "prev" links in the show/ document view, we need to keep track of where we are in the search results, so we either need to retain the query information, or to cache the results. But ... we have lost those lovely clean urls. I'm aware that this has been discussed and it's a trade off ... but it makes me sad. Anyone have any brilliant solutions? > > > Brilliant?! No, but maybe too obvious... session scope. > > That's what it is for :) > > We use it on http://www.lucidimagination.com/search when you're looking at an e-mail detail after having searched. You'll see a "return to search results" link. If you were e-mailed the link and navigated to it directly, you wouldn't see that "return to..." link because you'd have no session with a search context. We could put previous/next links in there too with no change to the URL. > > Erik > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > > From rossfsinger at gmail.com Mon Mar 16 17:03:39 2009 From: rossfsinger at gmail.com (Ross Singer) Date: Mon, 16 Mar 2009 17:03:39 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <49BEBC60.9060205@jhu.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <49BEA794.6000908@jhu.edu> <49BEBC60.9060205@jhu.edu> Message-ID: <23b83f160903161403y42e5202dv2338415b48bfe6b1@mail.gmail.com> Yeah, to be honest, I've never really understood why this sort of functionality wasn't session based. -Ross. On Mon, Mar 16, 2009 at 4:53 PM, Jonathan Rochkind wrote: > You see it as a good thing that someone can come to the detail URL for the > first time and go back to the search it came from? > > I'm not sure this is a good thing. As someone else mentioned, it's even a > potential privacy issue. I think it's preferable if the item detail doesn't > carry that info both from a url-prettifying point of view, and because it's > actually better functionality to me. > > I'm not sure I understand the AJAX-technique approach you're suggesting. I > do think it's important that all the major features of Blacklight work even > if the browser does not have javascript. ?(For a variety of reasons, > including mobile device accessibility). It's generally possible to do any > fancy AJAX things as an add on on top, such that even with no js present > everything still works okay (if clunkier). I'm not sure that's compatible > with what you're suggesting though. > > What do you have against Erik's session-storage suggestion? > > Jonathan > > > > Matt Mitchell wrote: >> >> Wouldn't if be possible to get the next/previous items using faceting? >> Just curious. >> >> The reason we included the search params in the item-level url, is so you >> can click "<< back". It's interesting to think about someone coming to a >> bookmarked item for the first time, and having the ability to go back to the >> search from where the item came. Including the params in the item-level url >> gives us that possibility. >> >> If you don't want to see the next/previous params in the url... AJAX would >> solve the problem perfectly. You wouldn't need to track the current index in >> the url, and you wouldn't have to work out a POST solution. >> >> For the messy url param thing... we could (and there is code in the >> plugin's lib dir for doing this) write our own handler to make the urls >> pretty. I decided to stick with the Rails default "[]" funkiness to keep it >> simple though. >> >> Matt >> >> On Mon, Mar 16, 2009 at 3:25 PM, Jonathan Rochkind >> > wrote: >> Agreed that a search URL should have everything in it neccessary to >> reconstruct the search and be looking at results for the same search if you >> bookmark it and come back later. Just not an item detail URL. :) >> >> >> Jamie Orchard-Hays wrote: >> I think I've confused what I'm saying by not realizing that Naomi's >> ?original post for for a detail, not a search. >> >> For a document detail, it makes complete sense not to include the ?search >> info in the URL, so I agree with y'all on that. >> >> It's only for searches, even facet filters that I like simple GET >> ?requests with the info in the URL, even if they're a bit messy. >> >> Jamie >> >> On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: >> >> >> Jamie Orchard-Hays wrote: >> >> But for search results, I'd rather have the info in the URL than ? hidden >> in a session somewhere. >> >> >> Why? ?I'd rather have the info that's just about the context hidden >> ?somewhere, so that the URL in the browser bar, which may be ?bookmarked and >> shared, is context-independent and just represents ?the record you are >> looking at. >> >> Jonathan >> >> >> >> On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: >> >> >> >> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >> >> >> Now the url that displays is >> >> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >> ?counter=3&f[language_facet][]=English >> >> ?or something like it; ?it includes the query and whatever ?facets ?have >> been selected, and the position in the results. ? I ?understand ?that in >> order to do the "next" and "prev" links in ?the show/ document view, we need >> to keep track of where we are in ?the search ?results, so we either need to >> retain the query ?information, or to ?cache the results. ?But ... we have >> lost ?those lovely clean urls. ? I'm aware that this has been discussed ?and >> it's a trade off ... but ?it makes me sad. ? Anyone have any ?brilliant >> solutions? >> >> >> Brilliant?! ? No, but maybe too obvious... session scope. >> >> That's what it is for :) >> >> We use it on http://www.lucidimagination.com/search when you're ? looking >> at an e-mail detail after having searched. ?You'll see a ? "return to search >> results" link. ?If you were e-mailed the link ?and ?navigated to it >> directly, you wouldn't see that "return ?to..." link ?because you'd have no >> session with a search context. ? We could put ?previous/next links in there >> too with no change to ?the URL. >> >> ? ? ? Erik >> _______________________________________________ >> Blacklight-development mailing list >> >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> _______________________________________________ >> Blacklight-development mailing list >> >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> _______________________________________________ >> Blacklight-development mailing list >> >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> _______________________________________________ >> Blacklight-development mailing list >> >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From goodieboy at gmail.com Mon Mar 16 17:27:14 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Mon, 16 Mar 2009 17:27:14 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <49BEBC60.9060205@jhu.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <49BEA794.6000908@jhu.edu> <49BEBC60.9060205@jhu.edu> Message-ID: Hi Jonathan. The privacy issue is a good point, never thought about that actually. I don't necessarily think it's a good thing to see where an item came from, just an interesting concept. The ajax solution would involve making async requests to the catalog controller to get the previous/next solr doc id. Once for the page load, and additional requests per-pre/next click. The session way could work too, but there could be some problems. If you come to a details view with prev/next already set in the session then the prev/next links might not go to where you expect. I'd expect that when I go to a details page for the first time, the prev item would not exist, and the next would be #2 in the "list". If the prev/next are already set in a session from the day before (for example), I'd be a little thrown off when I first come to an item and see the prev link activated. Does that make sense? I tend to not use sessions. The only thing I really ever use them for is login info. Matt On Mon, Mar 16, 2009 at 4:53 PM, Jonathan Rochkind wrote: > You see it as a good thing that someone can come to the detail URL for the > first time and go back to the search it came from? > > I'm not sure this is a good thing. As someone else mentioned, it's even a > potential privacy issue. I think it's preferable if the item detail doesn't > carry that info both from a url-prettifying point of view, and because it's > actually better functionality to me. > > I'm not sure I understand the AJAX-technique approach you're suggesting. I > do think it's important that all the major features of Blacklight work even > if the browser does not have javascript. (For a variety of reasons, > including mobile device accessibility). It's generally possible to do any > fancy AJAX things as an add on on top, such that even with no js present > everything still works okay (if clunkier). I'm not sure that's compatible > with what you're suggesting though. > > What do you have against Erik's session-storage suggestion? > > Jonathan > > > > Matt Mitchell wrote: > >> Wouldn't if be possible to get the next/previous items using faceting? >> Just curious. >> >> The reason we included the search params in the item-level url, is so you >> can click "<< back". It's interesting to think about someone coming to a >> bookmarked item for the first time, and having the ability to go back to the >> search from where the item came. Including the params in the item-level url >> gives us that possibility. >> >> If you don't want to see the next/previous params in the url... AJAX would >> solve the problem perfectly. You wouldn't need to track the current index in >> the url, and you wouldn't have to work out a POST solution. >> >> For the messy url param thing... we could (and there is code in the >> plugin's lib dir for doing this) write our own handler to make the urls >> pretty. I decided to stick with the Rails default "[]" funkiness to keep it >> simple though. >> >> Matt >> >> On Mon, Mar 16, 2009 at 3:25 PM, Jonathan Rochkind > > wrote: >> Agreed that a search URL should have everything in it neccessary to >> reconstruct the search and be looking at results for the same search if you >> bookmark it and come back later. Just not an item detail URL. :) >> >> >> Jamie Orchard-Hays wrote: >> I think I've confused what I'm saying by not realizing that Naomi's >> original post for for a detail, not a search. >> >> For a document detail, it makes complete sense not to include the search >> info in the URL, so I agree with y'all on that. >> >> It's only for searches, even facet filters that I like simple GET >> requests with the info in the URL, even if they're a bit messy. >> >> Jamie >> >> On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: >> >> >> Jamie Orchard-Hays wrote: >> >> But for search results, I'd rather have the info in the URL than hidden >> in a session somewhere. >> >> >> Why? I'd rather have the info that's just about the context hidden >> somewhere, so that the URL in the browser bar, which may be bookmarked and >> shared, is context-independent and just represents the record you are >> looking at. >> >> Jonathan >> >> >> >> On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: >> >> >> >> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >> >> >> Now the url that displays is >> >> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >> counter=3&f[language_facet][]=English >> >> or something like it; it includes the query and whatever facets have >> been selected, and the position in the results. I understand that in >> order to do the "next" and "prev" links in the show/ document view, we need >> to keep track of where we are in the search results, so we either need to >> retain the query information, or to cache the results. But ... we have >> lost those lovely clean urls. I'm aware that this has been discussed and >> it's a trade off ... but it makes me sad. Anyone have any brilliant >> solutions? >> >> >> Brilliant?! No, but maybe too obvious... session scope. >> >> That's what it is for :) >> >> We use it on http://www.lucidimagination.com/search when you're looking >> at an e-mail detail after having searched. You'll see a "return to search >> results" link. If you were e-mailed the link and navigated to it >> directly, you wouldn't see that "return to..." link because you'd have no >> session with a search context. We could put previous/next links in there >> too with no change to the URL. >> >> Erik >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org> Blacklight-development at rubyforge.org> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> >> >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erikhatcher at mac.com Mon Mar 16 17:39:52 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Mon, 16 Mar 2009 17:39:52 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <49BEA794.6000908@jhu.edu> Message-ID: On Mar 16, 2009, at 4:25 PM, Matt Mitchell wrote: > Wouldn't if be possible to get the next/previous items using > faceting? Just curious. I don't think so. The next/previous was within the search results that led to the detail item, as I understand it. > The reason we included the search params in the item-level url, is > so you can click "<< back". It's interesting to think about someone > coming to a bookmarked item for the first time, and having the > ability to go back to the search from where the item came. Including > the params in the item-level url gives us that possibility. Yeah, it's kinda cool to have that level of reasoning of how someone got here, but I still think it adds details that probably aren't worth sharing with the world. It'd also have to encode a lot to ensure that going "<< back" leads you to a page where that actual document is listed, if you wanted to be that particular about it, and that could change over time as relevancy is tuned, etc. > If you don't want to see the next/previous params in the url... AJAX > would solve the problem perfectly. You wouldn't need to track the > current index in the url, and you wouldn't have to work out a POST > solution. You and your "Ajax solves all the worlds problems" :) Erik From rochkind at jhu.edu Mon Mar 16 18:10:49 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 16 Mar 2009 18:10:49 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <49BEA794.6000908@jhu.edu> <49BEBC60.9060205@jhu.edu>, Message-ID: <90FF863A96E1EC42B8B240D04C88FB1D73406AA4@JHEMTEXVS2.win.ad.jhu.edu> Yeah, session stuff can be tricky; you want to support people doing multiple searches simultaneously in different browser windows, can be hard to keep straight. I don't know the details of this code though, so I'll leave it to you to do what's best for now, and maybe some day propose changing it myself. But if there's sufficient information available, even without info in the url, for javascript in the page to get the prev/next solr doc id through an async request... why isn't there sufficient information even without info in the URL for the actual Rails controller to do this server side before delivering the page? I do think it's important that significant functionality like this works even without javascript in the browser. (If anyone is unsure why this is important, I can provide my extended argument, but I'll spare you for now. :) ). Jonathan ________________________________________ From: blacklight-development-bounces at rubyforge.org [blacklight-development-bounces at rubyforge.org] On Behalf Of Matt Mitchell [goodieboy at gmail.com] Sent: Monday, March 16, 2009 5:27 PM To: blacklight-development at rubyforge.org Subject: Re: [Blacklight-development] single document questions / concerns Hi Jonathan. The privacy issue is a good point, never thought about that actually. I don't necessarily think it's a good thing to see where an item came from, just an interesting concept. The ajax solution would involve making async requests to the catalog controller to get the previous/next solr doc id. Once for the page load, and additional requests per-pre/next click. The session way could work too, but there could be some problems. If you come to a details view with prev/next already set in the session then the prev/next links might not go to where you expect. I'd expect that when I go to a details page for the first time, the prev item would not exist, and the next would be #2 in the "list". If the prev/next are already set in a session from the day before (for example), I'd be a little thrown off when I first come to an item and see the prev link activated. Does that make sense? I tend to not use sessions. The only thing I really ever use them for is login info. Matt On Mon, Mar 16, 2009 at 4:53 PM, Jonathan Rochkind > wrote: You see it as a good thing that someone can come to the detail URL for the first time and go back to the search it came from? I'm not sure this is a good thing. As someone else mentioned, it's even a potential privacy issue. I think it's preferable if the item detail doesn't carry that info both from a url-prettifying point of view, and because it's actually better functionality to me. I'm not sure I understand the AJAX-technique approach you're suggesting. I do think it's important that all the major features of Blacklight work even if the browser does not have javascript. (For a variety of reasons, including mobile device accessibility). It's generally possible to do any fancy AJAX things as an add on on top, such that even with no js present everything still works okay (if clunkier). I'm not sure that's compatible with what you're suggesting though. What do you have against Erik's session-storage suggestion? Jonathan Matt Mitchell wrote: Wouldn't if be possible to get the next/previous items using faceting? Just curious. The reason we included the search params in the item-level url, is so you can click "<< back". It's interesting to think about someone coming to a bookmarked item for the first time, and having the ability to go back to the search from where the item came. Including the params in the item-level url gives us that possibility. If you don't want to see the next/previous params in the url... AJAX would solve the problem perfectly. You wouldn't need to track the current index in the url, and you wouldn't have to work out a POST solution. For the messy url param thing... we could (and there is code in the plugin's lib dir for doing this) write our own handler to make the urls pretty. I decided to stick with the Rails default "[]" funkiness to keep it simple though. Matt On Mon, Mar 16, 2009 at 3:25 PM, Jonathan Rochkind >> wrote: Agreed that a search URL should have everything in it neccessary to reconstruct the search and be looking at results for the same search if you bookmark it and come back later. Just not an item detail URL. :) Jamie Orchard-Hays wrote: I think I've confused what I'm saying by not realizing that Naomi's original post for for a detail, not a search. For a document detail, it makes complete sense not to include the search info in the URL, so I agree with y'all on that. It's only for searches, even facet filters that I like simple GET requests with the info in the URL, even if they're a bit messy. Jamie On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: Jamie Orchard-Hays wrote: But for search results, I'd rather have the info in the URL than hidden in a session somewhere. Why? I'd rather have the info that's just about the context hidden somewhere, so that the URL in the browser bar, which may be bookmarked and shared, is context-independent and just represents the record you are looking at. Jonathan On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: Now the url that displays is http://searchworks-dev.stanford.edu:3000/catalog/00009103? counter=3&f[language_facet][]=English or something like it; it includes the query and whatever facets have been selected, and the position in the results. I understand that in order to do the "next" and "prev" links in the show/ document view, we need to keep track of where we are in the search results, so we either need to retain the query information, or to cache the results. But ... we have lost those lovely clean urls. I'm aware that this has been discussed and it's a trade off ... but it makes me sad. Anyone have any brilliant solutions? Brilliant?! No, but maybe too obvious... session scope. That's what it is for :) We use it on http://www.lucidimagination.com/search when you're looking at an e-mail detail after having searched. You'll see a "return to search results" link. If you were e-mailed the link and navigated to it directly, you wouldn't see that "return to..." link because you'd have no session with a search context. We could put previous/next links in there too with no change to the URL. Erik _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org> http://rubyforge.org/mailman/listinfo/blacklight-development Blacklightopac Blog http://blacklightopac.org/ _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org> http://rubyforge.org/mailman/listinfo/blacklight-development Blacklightopac Blog http://blacklightopac.org/ _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org> http://rubyforge.org/mailman/listinfo/blacklight-development Blacklightopac Blog http://blacklightopac.org/ _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org> http://rubyforge.org/mailman/listinfo/blacklight-development Blacklightopac Blog http://blacklightopac.org/ _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org> http://rubyforge.org/mailman/listinfo/blacklight-development Blacklightopac Blog http://blacklightopac.org/ _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org http://rubyforge.org/mailman/listinfo/blacklight-development Blacklightopac Blog http://blacklightopac.org/ From jkeck at stanford.edu Mon Mar 16 18:49:00 2009 From: jkeck at stanford.edu (Jessie Keck) Date: Mon, 16 Mar 2009 15:49:00 -0700 Subject: [Blacklight-development] Bug in CatalogController Message-ID: <49BED75C.5090107@stanford.edu> Hi all, After installing and messing w/ all of the great mods made over the weekend I noticed a bug in the CatalogController. This bug was causing the @document variable to be overwritten by the setup_document_by_counter method. The affect: Once you clicked on an item record you would be given the correct doc w/ the correct id Once you started clicking on the next > link you would be given the correct doc in the search order, however the id in the URI would not be the id of the doc. This was caused by the setup_document_by_counter method returning an @document variable to the template while it just needed to return the document (no need to assign to a variable, just return the doc as the last line in the method). Also, there was a small error in the logic that checked to see if you were at the first item in a result. (counter <= 0 should have been counter < 1) I've made these two changes locally and all seems right in the world. I have also modified the CatalogController spec to test that no previous link shows up when the counter = 1 not 0 since the counter should never be 0. Let me know if you have any questions, or if I have fooched something because I didn't quite understand the Next/Previous linking methodology. -Jessie Keck Stanford University From ndushay at stanford.edu Mon Mar 16 19:09:48 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Mon, 16 Mar 2009 16:09:48 -0700 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> Message-ID: <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> Actually, now that we have a solution or two to choose from, there are some other good reasons to take the search info out of the URLs. The way it is now, there can be many urls for the same document. I'm not sure this is desirable. Also: what about when we have title searches, author searches and the like? then we'll have a qt param to add as well. I intend to explore Erik's suggestions for some solr magik to clean up our search urls; i'll experiment locally. Probably won't get to it for a bit. Wow! I guess I asked a good question! - Naomi On Mar 16, 2009, at 12:13 PM, Jamie Orchard-Hays wrote: > I think I've confused what I'm saying by not realizing that Naomi's > original post for for a detail, not a search. > > For a document detail, it makes complete sense not to include the > search info in the URL, so I agree with y'all on that. > > It's only for searches, even facet filters that I like simple GET > requests with the info in the URL, even if they're a bit messy. > > Jamie > > On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: > >> Jamie Orchard-Hays wrote: >>> But for search results, I'd rather have the info in the URL than >>> hidden in a session somewhere. >>> >> Why? I'd rather have the info that's just about the context hidden >> somewhere, so that the URL in the browser bar, which may be >> bookmarked and shared, is context-independent and just represents >> the record you are looking at. >> >> Jonathan >> >> >>> >>> On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: >>> >>> >>>> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >>>> >>>>> Now the url that displays is >>>>> >>>>> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >>>>> counter=3&f[language_facet][]=English >>>>> >>>>> or something like it; it includes the query and whatever >>>>> facets have been selected, and the position in the results. I >>>>> understand that in order to do the "next" and "prev" links in >>>>> the show/ document view, we need to keep track of where we are >>>>> in the search results, so we either need to retain the query >>>>> information, or to cache the results. But ... we have lost >>>>> those lovely clean urls. I'm aware that this has been >>>>> discussed and it's a trade off ... but it makes me sad. >>>>> Anyone have any brilliant solutions? >>>>> >>>> Brilliant?! No, but maybe too obvious... session scope. >>>> >>>> That's what it is for :) >>>> >>>> We use it on http://www.lucidimagination.com/search when you're >>>> looking at an e-mail detail after having searched. You'll see a >>>> "return to search results" link. If you were e-mailed the link >>>> and navigated to it directly, you wouldn't see that "return >>>> to..." link because you'd have no session with a search >>>> context. We could put previous/next links in there too with no >>>> change to the URL. >>>> >>>> Erik >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From ndushay at stanford.edu Mon Mar 16 20:10:14 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Mon, 16 Mar 2009 17:10:14 -0700 Subject: [Blacklight-development] a potential weakness to some of my specs Message-ID: <43F96BBE-3CD2-4DCC-9CC2-9A92911DB270@stanford.edu> I took the brittleness around solr field names out of the specs, but I left in a different type of brittleness. Some of the tests now expect a live solr instance, because they send queries to solr. I know that this external dependency in our tests is not ideal, but it seemed the lesser of two evils, to me. Truth is, the solr instances are NOT external lookups, in the sense of hammering a web service out of our control for testing. Sending search requests to a local solr, in order to make the testing code more functional (as in functional, not unit), was the choice I made. I tried to pick generic query strings (but, sadly, skewed to English language): empty query (should map to all docs query for all blacklight sites) user query: p (not in default stopwords and in a LOT of physical descriptions in MARC 300 field) facets: format Book, language English, user query p Affected specs: catalog_controller home_controller _document_list.html.erb (catalog view) show.html.erb (catalog view) solr_helper <-- I would argue that running this against a live solr index is worth it for sure. I'm NOT testing the solr functionality; I'm testing methods like "get_search_results" "get_document" I'm AM testing stuff like this: given a user query and/or some facets, we get results (documents and populated facets) given an empty query and no facets, we get results given a query for no docs, we get no results, but not an error. given a bad facet value, we get no results, but not an error given a known doc_id, we get a document given a non-existent doc_id, we get no document, but deal with it gracefully given no page number, we start with the first result given a page number, we skip the right number of results given a number-per-page, we get the right number of results This is all about knowing that the solr_helper methods will behave well when called. In essence, it's a way to test that changes to the solr gems (rsolr, or solr-ruby) have no ill effects. Which, I believe, is the point of the solr_helper class, and its spec. If folks disagree ... write yer own specs! ;-) Seriously: having a working spec trumps not having a spec. And I believe my specs are less brittle than mock documents with mock fields title_t, format_code_t, etc. We *could* have mock docs with the abstracted field names from solr.yml ... but I mindfully made my choices. - Naomi From goodieboy at gmail.com Mon Mar 16 20:52:40 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Mon, 16 Mar 2009 20:52:40 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> Message-ID: I've realized that there are two different things we're talking about here... The main problem being, url style/clutter/embedded search-context-params. The second one being the next/prev in single document mode. - search params in single-doc mode It seems that most of us agree on having search-context-params within a document url is not desirable. I now see how putting the search params into the session would solve the privacy issue. I'd imagine that the variable would be a hash; the key being a document id and the value being the query params. That could grow into a big hash though? Not sure if that's a problem or not... depends on the session store. The head scratcher for me is, how would you do it without sending a redirect for every document? - next/prev in single-doc mode I don't know the exact logic behind the next/prev code... but I'd imagine it could be done without a "counter" param by preparing the next/prev doc id's in the controller, and then simply linking to them in the view. The next/prev functionality is dependent on the search-context-params right? We can't know what will be next without knowing what search found the original document. But without passing the params to the url (GET style), we're left with a session based solution, which is going to require a redirect. POST could do it too, but this is not a POST request. Does that make sense? For what it's worth, my vote is that we either stick with the url params or remove this functionality alltogether. I'm probably missing something, does anyone else see a better solution to this? Matt On Mon, Mar 16, 2009 at 7:09 PM, Naomi Dushay wrote: > > Actually, now that we have a solution or two to choose from, there are some > other good reasons to take the search info out of the URLs. The way it is > now, there can be many urls for the same document. I'm not sure this is > desirable. > > Also: what about when we have title searches, author searches and the > like? then we'll have a qt param to add as well. > > I intend to explore Erik's suggestions for some solr magik to clean up our > search urls; i'll experiment locally. Probably won't get to it for a bit. > > Wow! I guess I asked a good question! > > - Naomi > > > On Mar 16, 2009, at 12:13 PM, Jamie Orchard-Hays wrote: > > I think I've confused what I'm saying by not realizing that Naomi's >> original post for for a detail, not a search. >> >> For a document detail, it makes complete sense not to include the search >> info in the URL, so I agree with y'all on that. >> >> It's only for searches, even facet filters that I like simple GET requests >> with the info in the URL, even if they're a bit messy. >> >> Jamie >> >> On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: >> >> Jamie Orchard-Hays wrote: >>> >>>> But for search results, I'd rather have the info in the URL than hidden >>>> in a session somewhere. >>>> >>>> Why? I'd rather have the info that's just about the context hidden >>> somewhere, so that the URL in the browser bar, which may be bookmarked and >>> shared, is context-independent and just represents the record you are >>> looking at. >>> >>> Jonathan >>> >>> >>> >>>> On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: >>>> >>>> >>>> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >>>>> >>>>> Now the url that displays is >>>>>> >>>>>> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >>>>>> counter=3&f[language_facet][]=English >>>>>> >>>>>> or something like it; it includes the query and whatever facets have >>>>>> been selected, and the position in the results. I understand that in >>>>>> order to do the "next" and "prev" links in the show/ document view, we need >>>>>> to keep track of where we are in the search results, so we either need to >>>>>> retain the query information, or to cache the results. But ... we have >>>>>> lost those lovely clean urls. I'm aware that this has been discussed and >>>>>> it's a trade off ... but it makes me sad. Anyone have any brilliant >>>>>> solutions? >>>>>> >>>>>> Brilliant?! No, but maybe too obvious... session scope. >>>>> >>>>> That's what it is for :) >>>>> >>>>> We use it on http://www.lucidimagination.com/search when you're >>>>> looking at an e-mail detail after having searched. You'll see a "return >>>>> to search results" link. If you were e-mailed the link and navigated to it >>>>> directly, you wouldn't see that "return to..." link because you'd have no >>>>> session with a search context. We could put previous/next links in there >>>>> too with no change to the URL. >>>>> >>>>> Erik >>>>> _______________________________________________ >>>>> Blacklight-development mailing list >>>>> Blacklight-development at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>> >>>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossfsinger at gmail.com Mon Mar 16 23:33:12 2009 From: rossfsinger at gmail.com (Ross Singer) Date: Mon, 16 Mar 2009 23:33:12 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> Message-ID: <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> Why would it require a redirect? It also doesn't need to store anything in the hash either, you can store all the session data on the server. Rails sessions are basically an ID and marshaled object. Basically you have your session id in the cookie, then you can basically: * persist an object around that session id -- to deal with next,previous the session could store an array of the ids in the result page (how does Blacklight deal with the "next" result from last result of a page, anyway?) * persist an object around the search, this would prevent the conflict from multiple tabs -- although obviously you'd want to be selective about what is actually being stored to prevent the session store from getting insanely massive. I guess at this point it would be useful to determine what, exactly, would be necessary to shove into the session object. Another option that would probably work ok for the full record view would be to "flash" the next/previous/back urls. This would actually be kind of nice since you wouldn't have to worry about dealing with maintenance in the session store. -Ross. 2009/3/16 Matt Mitchell : > I've realized that there are two different things we're talking about > here... The main problem being, url style/clutter/embedded > search-context-params. The second one being the next/prev in single document > mode. > > - search params in single-doc mode > It seems that most of us agree on having search-context-params within a > document url is not desirable. I now see how putting the search params into > the session would solve the privacy issue. I'd imagine that the variable > would be a hash; the key being a document id and the value being the query > params. That could grow into a big hash though? Not sure if that's a problem > or not... depends on the session store. The head scratcher for me is, how > would you do it without sending a redirect for every document? > > - next/prev in single-doc mode > I don't know the exact logic behind the next/prev code... but I'd imagine it > could be done without a "counter" param by preparing the next/prev doc id's > in the controller, and then simply linking to them in the view. > > The next/prev functionality is dependent on the search-context-params right? > We can't know what will be next without knowing what search found the > original document. > > But without passing the params to the url (GET style), we're left with a > session based solution, which is going to require a redirect. POST could do > it too, but this is not a POST request. > > Does that make sense? For what it's worth, my vote is that we either stick > with the url params or remove this functionality alltogether. I'm probably > missing something, does anyone else see a better solution to this? > > Matt > > On Mon, Mar 16, 2009 at 7:09 PM, Naomi Dushay wrote: >> >> Actually, now that we have a solution or two to choose from, there are >> some other good reasons to take the search info out of the URLs. ?The way it >> is now, there can be many urls for the same document. ? I'm not sure this is >> desirable. >> >> Also: ?what about when we have title searches, author searches and the >> like? ?then we'll have a qt param to add as well. >> >> I intend to explore Erik's suggestions for some solr magik to clean up our >> search urls; ?i'll experiment locally. ?Probably won't get to it for a bit. >> >> Wow! ?I guess I asked a good question! >> >> - Naomi >> >> On Mar 16, 2009, at 12:13 PM, Jamie Orchard-Hays wrote: >> >>> I think I've confused what I'm saying by not realizing that Naomi's >>> original post for for a detail, not a search. >>> >>> For a document detail, it makes complete sense not to include the search >>> info in the URL, so I agree with y'all on that. >>> >>> It's only for searches, even facet filters that I like simple GET >>> requests with the info in the URL, even if they're a bit messy. >>> >>> Jamie >>> >>> On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: >>> >>>> Jamie Orchard-Hays wrote: >>>>> >>>>> But for search results, I'd rather have the info in the URL than >>>>> ?hidden in a session somewhere. >>>>> >>>> Why? ?I'd rather have the info that's just about the context hidden >>>> somewhere, so that the URL in the browser bar, which may be bookmarked and >>>> shared, is context-independent and just represents the record you are >>>> looking at. >>>> >>>> Jonathan >>>> >>>> >>>>> >>>>> On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: >>>>> >>>>> >>>>>> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >>>>>> >>>>>>> Now the url that displays is >>>>>>> >>>>>>> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >>>>>>> counter=3&f[language_facet][]=English >>>>>>> >>>>>>> or something like it; ?it includes the query and whatever facets >>>>>>> ?have been selected, and the position in the results. ? I understand ?that >>>>>>> in order to do the "next" and "prev" links in the show/ document view, we >>>>>>> need to keep track of where we are in the search ?results, so we either need >>>>>>> to retain the query information, or to ?cache the results. ?But ... we have >>>>>>> lost those lovely clean urls. ? I'm aware that this has been discussed and >>>>>>> it's a trade off ... but ?it makes me sad. ? Anyone have any brilliant >>>>>>> solutions? >>>>>>> >>>>>> Brilliant?! ? No, but maybe too obvious... session scope. >>>>>> >>>>>> That's what it is for :) >>>>>> >>>>>> We use it on http://www.lucidimagination.com/search when you're >>>>>> ?looking at an e-mail detail after having searched. ?You'll see a ?"return >>>>>> to search results" link. ?If you were e-mailed the link and ?navigated to it >>>>>> directly, you wouldn't see that "return to..." link ?because you'd have no >>>>>> session with a search context. ?We could put ?previous/next links in there >>>>>> too with no change to the URL. >>>>>> >>>>>> ? ? ? ?Erik >>>>>> _______________________________________________ >>>>>> Blacklight-development mailing list >>>>>> Blacklight-development at rubyforge.org >>>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Blacklight-development mailing list >>>>> Blacklight-development at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From ndushay at stanford.edu Tue Mar 17 00:52:37 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Mon, 16 Mar 2009 21:52:37 -0700 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> Message-ID: <87D620CB-FADD-4BC7-B358-9F8FAE247DDF@stanford.edu> Currently, the blacklight code deals with 'next' and 'prev' by reusing the solr search request and asking for a single row at the position indicated by the counter. This approach allows "next" and "prev" to work beyond the single page of search results most recently displayed. Would it work for the session to store the doc ids returned in the search results page, AND store the query params in case the user scrolls past the search response? - Naomi On Mar 16, 2009, at 8:33 PM, Ross Singer wrote: > Why would it require a redirect? > > It also doesn't need to store anything in the hash either, you can > store all the session data on the server. Rails sessions are > basically an ID and marshaled object. > > Basically you have your session id in the cookie, then you can > basically: > > * persist an object around that session id -- to deal with > next,previous the session could store an array of the ids in the > result page (how does Blacklight deal with the "next" result from last > result of a page, anyway?) > * persist an object around the search, this would prevent the conflict > from multiple tabs -- although obviously you'd want to be selective > about what is actually being stored to prevent the session store from > getting insanely massive. > > I guess at this point it would be useful to determine what, exactly, > would be necessary to shove into the session object. > > Another option that would probably work ok for the full record view > would be to "flash" the next/previous/back urls. This would actually > be kind of nice since you wouldn't have to worry about dealing with > maintenance in the session store. > > -Ross. > > 2009/3/16 Matt Mitchell : >> I've realized that there are two different things we're talking about >> here... The main problem being, url style/clutter/embedded >> search-context-params. The second one being the next/prev in single >> document >> mode. >> >> - search params in single-doc mode >> It seems that most of us agree on having search-context-params >> within a >> document url is not desirable. I now see how putting the search >> params into >> the session would solve the privacy issue. I'd imagine that the >> variable >> would be a hash; the key being a document id and the value being >> the query >> params. That could grow into a big hash though? Not sure if that's >> a problem >> or not... depends on the session store. The head scratcher for me >> is, how >> would you do it without sending a redirect for every document? >> >> - next/prev in single-doc mode >> I don't know the exact logic behind the next/prev code... but I'd >> imagine it >> could be done without a "counter" param by preparing the next/prev >> doc id's >> in the controller, and then simply linking to them in the view. >> >> The next/prev functionality is dependent on the search-context- >> params right? >> We can't know what will be next without knowing what search found the >> original document. >> >> But without passing the params to the url (GET style), we're left >> with a >> session based solution, which is going to require a redirect. POST >> could do >> it too, but this is not a POST request. >> >> Does that make sense? For what it's worth, my vote is that we >> either stick >> with the url params or remove this functionality alltogether. I'm >> probably >> missing something, does anyone else see a better solution to this? >> >> Matt >> >> On Mon, Mar 16, 2009 at 7:09 PM, Naomi Dushay >> wrote: >>> >>> Actually, now that we have a solution or two to choose from, there >>> are >>> some other good reasons to take the search info out of the URLs. >>> The way it >>> is now, there can be many urls for the same document. I'm not >>> sure this is >>> desirable. >>> >>> Also: what about when we have title searches, author searches and >>> the >>> like? then we'll have a qt param to add as well. >>> >>> I intend to explore Erik's suggestions for some solr magik to >>> clean up our >>> search urls; i'll experiment locally. Probably won't get to it >>> for a bit. >>> >>> Wow! I guess I asked a good question! >>> >>> - Naomi >>> >>> On Mar 16, 2009, at 12:13 PM, Jamie Orchard-Hays wrote: >>> >>>> I think I've confused what I'm saying by not realizing that Naomi's >>>> original post for for a detail, not a search. >>>> >>>> For a document detail, it makes complete sense not to include the >>>> search >>>> info in the URL, so I agree with y'all on that. >>>> >>>> It's only for searches, even facet filters that I like simple GET >>>> requests with the info in the URL, even if they're a bit messy. >>>> >>>> Jamie >>>> >>>> On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: >>>> >>>>> Jamie Orchard-Hays wrote: >>>>>> >>>>>> But for search results, I'd rather have the info in the URL than >>>>>> hidden in a session somewhere. >>>>>> >>>>> Why? I'd rather have the info that's just about the context >>>>> hidden >>>>> somewhere, so that the URL in the browser bar, which may be >>>>> bookmarked and >>>>> shared, is context-independent and just represents the record >>>>> you are >>>>> looking at. >>>>> >>>>> Jonathan >>>>> >>>>> >>>>>> >>>>>> On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: >>>>>> >>>>>> >>>>>>> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >>>>>>> >>>>>>>> Now the url that displays is >>>>>>>> >>>>>>>> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >>>>>>>> counter=3&f[language_facet][]=English >>>>>>>> >>>>>>>> or something like it; it includes the query and whatever >>>>>>>> facets >>>>>>>> have been selected, and the position in the results. I >>>>>>>> understand that >>>>>>>> in order to do the "next" and "prev" links in the show/ >>>>>>>> document view, we >>>>>>>> need to keep track of where we are in the search results, so >>>>>>>> we either need >>>>>>>> to retain the query information, or to cache the results. >>>>>>>> But ... we have >>>>>>>> lost those lovely clean urls. I'm aware that this has been >>>>>>>> discussed and >>>>>>>> it's a trade off ... but it makes me sad. Anyone have any >>>>>>>> brilliant >>>>>>>> solutions? >>>>>>>> >>>>>>> Brilliant?! No, but maybe too obvious... session scope. >>>>>>> >>>>>>> That's what it is for :) >>>>>>> >>>>>>> We use it on http://www.lucidimagination.com/search when you're >>>>>>> looking at an e-mail detail after having searched. You'll >>>>>>> see a "return >>>>>>> to search results" link. If you were e-mailed the link and >>>>>>> navigated to it >>>>>>> directly, you wouldn't see that "return to..." link because >>>>>>> you'd have no >>>>>>> session with a search context. We could put previous/next >>>>>>> links in there >>>>>>> too with no change to the URL. >>>>>>> >>>>>>> Erik >>>>>>> From goodieboy at gmail.com Tue Mar 17 08:56:24 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 17 Mar 2009 08:56:24 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> Message-ID: Hey Ross, On Mon, Mar 16, 2009 at 11:33 PM, Ross Singer wrote: > Why would it require a redirect? > Good question. I guess you wouldn't have to do redirects, sorry. My thought was that when you click on an item link in the search list, you'd go to an action that set the session params, and then be redirected to the single doc without the search params in the url. The single doc view would then be able to use the session params to figure out what's next/prev. But your question made me realize that you could just store search params (as a single session variable) on every *search* request and get the same result. I still think there is the problem of coming to a single item view from a bookmark (for example). If you have search params in the session from the day before etc., the search params might be irrelevant. You could try to keep track of them by using a key/value pair in the session, but that could be weird to manage. Here's my attempt at sizing these two solutions up: going with a session: + nice urls + no privacy problems - problem of limiting how much gets stored (millions of documents) - weirdness when requesting a single doc where the referrer is NOT the search page - search params could lose sync with doc going with url + easy to manage + search params locked to doc when in url, very consistent behavior - ugly urls - privacy issue Wonder if this could be something that's abstracted out into something that's easily overridable? OK, I think you've all heard enough from me :) Matt > It also doesn't need to store anything in the hash either, you can > store all the session data on the server. Rails sessions are > basically an ID and marshaled object. > > Basically you have your session id in the cookie, then you can basically: > > * persist an object around that session id -- to deal with > next,previous the session could store an array of the ids in the > result page (how does Blacklight deal with the "next" result from last > result of a page, anyway?) > * persist an object around the search, this would prevent the conflict > from multiple tabs -- although obviously you'd want to be selective > about what is actually being stored to prevent the session store from > getting insanely massive. > > I guess at this point it would be useful to determine what, exactly, > would be necessary to shove into the session object. > > Another option that would probably work ok for the full record view > would be to "flash" the next/previous/back urls. This would actually > be kind of nice since you wouldn't have to worry about dealing with > maintenance in the session store. > > -Ross. > > 2009/3/16 Matt Mitchell : > > I've realized that there are two different things we're talking about > > here... The main problem being, url style/clutter/embedded > > search-context-params. The second one being the next/prev in single > document > > mode. > > > > - search params in single-doc mode > > It seems that most of us agree on having search-context-params within a > > document url is not desirable. I now see how putting the search params > into > > the session would solve the privacy issue. I'd imagine that the variable > > would be a hash; the key being a document id and the value being the > query > > params. That could grow into a big hash though? Not sure if that's a > problem > > or not... depends on the session store. The head scratcher for me is, how > > would you do it without sending a redirect for every document? > > > > - next/prev in single-doc mode > > I don't know the exact logic behind the next/prev code... but I'd imagine > it > > could be done without a "counter" param by preparing the next/prev doc > id's > > in the controller, and then simply linking to them in the view. > > > > The next/prev functionality is dependent on the search-context-params > right? > > We can't know what will be next without knowing what search found the > > original document. > > > > But without passing the params to the url (GET style), we're left with a > > session based solution, which is going to require a redirect. POST could > do > > it too, but this is not a POST request. > > > > Does that make sense? For what it's worth, my vote is that we either > stick > > with the url params or remove this functionality alltogether. I'm > probably > > missing something, does anyone else see a better solution to this? > > > > Matt > > > > On Mon, Mar 16, 2009 at 7:09 PM, Naomi Dushay > wrote: > >> > >> Actually, now that we have a solution or two to choose from, there are > >> some other good reasons to take the search info out of the URLs. The > way it > >> is now, there can be many urls for the same document. I'm not sure > this is > >> desirable. > >> > >> Also: what about when we have title searches, author searches and the > >> like? then we'll have a qt param to add as well. > >> > >> I intend to explore Erik's suggestions for some solr magik to clean up > our > >> search urls; i'll experiment locally. Probably won't get to it for a > bit. > >> > >> Wow! I guess I asked a good question! > >> > >> - Naomi > >> > >> On Mar 16, 2009, at 12:13 PM, Jamie Orchard-Hays wrote: > >> > >>> I think I've confused what I'm saying by not realizing that Naomi's > >>> original post for for a detail, not a search. > >>> > >>> For a document detail, it makes complete sense not to include the > search > >>> info in the URL, so I agree with y'all on that. > >>> > >>> It's only for searches, even facet filters that I like simple GET > >>> requests with the info in the URL, even if they're a bit messy. > >>> > >>> Jamie > >>> > >>> On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: > >>> > >>>> Jamie Orchard-Hays wrote: > >>>>> > >>>>> But for search results, I'd rather have the info in the URL than > >>>>> hidden in a session somewhere. > >>>>> > >>>> Why? I'd rather have the info that's just about the context hidden > >>>> somewhere, so that the URL in the browser bar, which may be bookmarked > and > >>>> shared, is context-independent and just represents the record you are > >>>> looking at. > >>>> > >>>> Jonathan > >>>> > >>>> > >>>>> > >>>>> On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: > >>>>> > >>>>> > >>>>>> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: > >>>>>> > >>>>>>> Now the url that displays is > >>>>>>> > >>>>>>> http://searchworks-dev.stanford.edu:3000/catalog/00009103? > >>>>>>> counter=3&f[language_facet][]=English > >>>>>>> > >>>>>>> or something like it; it includes the query and whatever facets > >>>>>>> have been selected, and the position in the results. I > understand that > >>>>>>> in order to do the "next" and "prev" links in the show/ document > view, we > >>>>>>> need to keep track of where we are in the search results, so we > either need > >>>>>>> to retain the query information, or to cache the results. But ... > we have > >>>>>>> lost those lovely clean urls. I'm aware that this has been > discussed and > >>>>>>> it's a trade off ... but it makes me sad. Anyone have any > brilliant > >>>>>>> solutions? > >>>>>>> > >>>>>> Brilliant?! No, but maybe too obvious... session scope. > >>>>>> > >>>>>> That's what it is for :) > >>>>>> > >>>>>> We use it on http://www.lucidimagination.com/search when you're > >>>>>> looking at an e-mail detail after having searched. You'll see a > "return > >>>>>> to search results" link. If you were e-mailed the link and > navigated to it > >>>>>> directly, you wouldn't see that "return to..." link because you'd > have no > >>>>>> session with a search context. We could put previous/next links in > there > >>>>>> too with no change to the URL. > >>>>>> > >>>>>> Erik > >>>>>> _______________________________________________ > >>>>>> Blacklight-development mailing list > >>>>>> Blacklight-development at rubyforge.org > >>>>>> http://rubyforge.org/mailman/listinfo/blacklight-development > >>>>>> Blacklightopac Blog http://blacklightopac.org/ > >>>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> Blacklight-development mailing list > >>>>> Blacklight-development at rubyforge.org > >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development > >>>>> Blacklightopac Blog http://blacklightopac.org/ > >>>>> > >>>> _______________________________________________ > >>>> Blacklight-development mailing list > >>>> Blacklight-development at rubyforge.org > >>>> http://rubyforge.org/mailman/listinfo/blacklight-development > >>>> Blacklightopac Blog http://blacklightopac.org/ > >>> > >>> _______________________________________________ > >>> Blacklight-development mailing list > >>> Blacklight-development at rubyforge.org > >>> http://rubyforge.org/mailman/listinfo/blacklight-development > >>> Blacklightopac Blog http://blacklightopac.org/ > >> > >> _______________________________________________ > >> Blacklight-development mailing list > >> Blacklight-development at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/blacklight-development > >> Blacklightopac Blog http://blacklightopac.org/ > > > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chapman.lists at gmail.com Tue Mar 17 09:35:14 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Tue, 17 Mar 2009 09:35:14 -0400 Subject: [Blacklight-development] Blacklight UI Message-ID: <49BFA712.7050104@gmail.com> Can someone on the BL UI teams explain how the expanding facet list works? I would really like to add that to my interface. If it is a trade secret I understand, if not then let me in on it. Pretty Please! Thanks Vernon From rossfsinger at gmail.com Tue Mar 17 09:40:58 2009 From: rossfsinger at gmail.com (Ross Singer) Date: Tue, 17 Mar 2009 09:40:58 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> Message-ID: <23b83f160903170640h263f6917v307a42effe9e7066@mail.gmail.com> Well, not so fast with your exit there, Matt :) I think you raise a good point about it being somewhat abstracted. I realize this is 2009 and vast swaths of the internet breaks for people who, for whatever tinfoil hatted reason, refuse to enable cookies, the fact of the matter is that BL's largest user base will probably be the university set, which seem to be a haven for the tinfoil types. Perhaps the abstraction should be a little more dynamic, in that it cascades down to the URL method (oh the irony of the privacy freaks having to share their search params with the world) if either cookies or sessions are disabled. This way, everybody wins :) Sort of. -Ross. 2009/3/17 Matt Mitchell : > Hey Ross, > > On Mon, Mar 16, 2009 at 11:33 PM, Ross Singer wrote: >> >> Why would it require a redirect? > > Good question. I guess you wouldn't have to do redirects, sorry. My thought > was that when you click on an item link in the search list, you'd go to an > action that set the session params, and then be redirected to the single doc > without the search params in the url. The single doc view would then be able > to use the session params to figure out what's next/prev. But your question > made me realize that you could just store search params (as a single session > variable) on every *search* request and get the same result. > > I still think there is the problem of coming to a single item view from a > bookmark (for example). If you have search params in the session from the > day before etc., the search params might be irrelevant. You could try to > keep track of them by using a key/value pair in the session, but that could > be weird to manage. > > Here's my attempt at sizing these two solutions up: > > going with a session: > + nice urls > + no privacy problems > - problem of limiting how much gets stored (millions of documents) > - weirdness when requesting a single doc where the referrer is NOT the > search page - search params could lose sync with doc > > going with url > + easy to manage > + search params locked to doc when in url, very consistent behavior > - ugly urls > - privacy issue > > Wonder if this could be something that's abstracted out into something > that's easily overridable? > > OK, I think you've all heard enough from me :) > > Matt > >> >> It also doesn't need to store anything in the hash either, you can >> store all the session data on the server. ?Rails sessions are >> basically an ID and marshaled object. >> >> Basically you have your session id in the cookie, then you can basically: >> >> * persist an object around that session id -- to deal with >> next,previous the session could store an array of the ids in the >> result page (how does Blacklight deal with the "next" result from last >> result of a page, anyway?) >> * persist an object around the search, this would prevent the conflict >> from multiple tabs -- although obviously you'd want to be selective >> about what is actually being stored to prevent the session store from >> getting insanely massive. >> >> I guess at this point it would be useful to determine what, exactly, >> would be necessary to shove into the session object. >> >> Another option that would probably work ok for the full record view >> would be to "flash" the next/previous/back urls. ?This would actually >> be kind of nice since you wouldn't have to worry about dealing with >> maintenance in the session store. >> >> -Ross. >> >> 2009/3/16 Matt Mitchell : >> > I've realized that there are two different things we're talking about >> > here... The main problem being, url style/clutter/embedded >> > search-context-params. The second one being the next/prev in single >> > document >> > mode. >> > >> > - search params in single-doc mode >> > It seems that most of us agree on having search-context-params within a >> > document url is not desirable. I now see how putting the search params >> > into >> > the session would solve the privacy issue. I'd imagine that the variable >> > would be a hash; the key being a document id and the value being the >> > query >> > params. That could grow into a big hash though? Not sure if that's a >> > problem >> > or not... depends on the session store. The head scratcher for me is, >> > how >> > would you do it without sending a redirect for every document? >> > >> > - next/prev in single-doc mode >> > I don't know the exact logic behind the next/prev code... but I'd >> > imagine it >> > could be done without a "counter" param by preparing the next/prev doc >> > id's >> > in the controller, and then simply linking to them in the view. >> > >> > The next/prev functionality is dependent on the search-context-params >> > right? >> > We can't know what will be next without knowing what search found the >> > original document. >> > >> > But without passing the params to the url (GET style), we're left with a >> > session based solution, which is going to require a redirect. POST could >> > do >> > it too, but this is not a POST request. >> > >> > Does that make sense? For what it's worth, my vote is that we either >> > stick >> > with the url params or remove this functionality alltogether. I'm >> > probably >> > missing something, does anyone else see a better solution to this? >> > >> > Matt >> > >> > On Mon, Mar 16, 2009 at 7:09 PM, Naomi Dushay >> > wrote: >> >> >> >> Actually, now that we have a solution or two to choose from, there are >> >> some other good reasons to take the search info out of the URLs. ?The >> >> way it >> >> is now, there can be many urls for the same document. ? I'm not sure >> >> this is >> >> desirable. >> >> >> >> Also: ?what about when we have title searches, author searches and the >> >> like? ?then we'll have a qt param to add as well. >> >> >> >> I intend to explore Erik's suggestions for some solr magik to clean up >> >> our >> >> search urls; ?i'll experiment locally. ?Probably won't get to it for a >> >> bit. >> >> >> >> Wow! ?I guess I asked a good question! >> >> >> >> - Naomi >> >> >> >> On Mar 16, 2009, at 12:13 PM, Jamie Orchard-Hays wrote: >> >> >> >>> I think I've confused what I'm saying by not realizing that Naomi's >> >>> original post for for a detail, not a search. >> >>> >> >>> For a document detail, it makes complete sense not to include the >> >>> search >> >>> info in the URL, so I agree with y'all on that. >> >>> >> >>> It's only for searches, even facet filters that I like simple GET >> >>> requests with the info in the URL, even if they're a bit messy. >> >>> >> >>> Jamie >> >>> >> >>> On Mar 16, 2009, at 3:10 PM, Jonathan Rochkind wrote: >> >>> >> >>>> Jamie Orchard-Hays wrote: >> >>>>> >> >>>>> But for search results, I'd rather have the info in the URL than >> >>>>> ?hidden in a session somewhere. >> >>>>> >> >>>> Why? ?I'd rather have the info that's just about the context hidden >> >>>> somewhere, so that the URL in the browser bar, which may be >> >>>> bookmarked and >> >>>> shared, is context-independent and just represents the record you are >> >>>> looking at. >> >>>> >> >>>> Jonathan >> >>>> >> >>>> >> >>>>> >> >>>>> On Mar 16, 2009, at 1:37 PM, Erik Hatcher wrote: >> >>>>> >> >>>>> >> >>>>>> On Mar 16, 2009, at 12:13 PM, Naomi Dushay wrote: >> >>>>>> >> >>>>>>> Now the url that displays is >> >>>>>>> >> >>>>>>> http://searchworks-dev.stanford.edu:3000/catalog/00009103? >> >>>>>>> counter=3&f[language_facet][]=English >> >>>>>>> >> >>>>>>> or something like it; ?it includes the query and whatever facets >> >>>>>>> ?have been selected, and the position in the results. ? I >> >>>>>>> understand ?that >> >>>>>>> in order to do the "next" and "prev" links in the show/ document >> >>>>>>> view, we >> >>>>>>> need to keep track of where we are in the search ?results, so we >> >>>>>>> either need >> >>>>>>> to retain the query information, or to ?cache the results. ?But >> >>>>>>> ... we have >> >>>>>>> lost those lovely clean urls. ? I'm aware that this has been >> >>>>>>> discussed and >> >>>>>>> it's a trade off ... but ?it makes me sad. ? Anyone have any >> >>>>>>> brilliant >> >>>>>>> solutions? >> >>>>>>> >> >>>>>> Brilliant?! ? No, but maybe too obvious... session scope. >> >>>>>> >> >>>>>> That's what it is for :) >> >>>>>> >> >>>>>> We use it on http://www.lucidimagination.com/search when you're >> >>>>>> ?looking at an e-mail detail after having searched. ?You'll see a >> >>>>>> ?"return >> >>>>>> to search results" link. ?If you were e-mailed the link and >> >>>>>> ?navigated to it >> >>>>>> directly, you wouldn't see that "return to..." link ?because you'd >> >>>>>> have no >> >>>>>> session with a search context. ?We could put ?previous/next links >> >>>>>> in there >> >>>>>> too with no change to the URL. >> >>>>>> >> >>>>>> ? ? ? ?Erik >> >>>>>> _______________________________________________ >> >>>>>> Blacklight-development mailing list >> >>>>>> Blacklight-development at rubyforge.org >> >>>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >> >>>>>> Blacklightopac Blog http://blacklightopac.org/ >> >>>>>> >> >>>>> >> >>>>> _______________________________________________ >> >>>>> Blacklight-development mailing list >> >>>>> Blacklight-development at rubyforge.org >> >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >> >>>>> Blacklightopac Blog http://blacklightopac.org/ >> >>>>> >> >>>> _______________________________________________ >> >>>> Blacklight-development mailing list >> >>>> Blacklight-development at rubyforge.org >> >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >> >>>> Blacklightopac Blog http://blacklightopac.org/ >> >>> >> >>> _______________________________________________ >> >>> Blacklight-development mailing list >> >>> Blacklight-development at rubyforge.org >> >>> http://rubyforge.org/mailman/listinfo/blacklight-development >> >>> Blacklightopac Blog http://blacklightopac.org/ >> >> >> >> _______________________________________________ >> >> Blacklight-development mailing list >> >> Blacklight-development at rubyforge.org >> >> http://rubyforge.org/mailman/listinfo/blacklight-development >> >> Blacklightopac Blog http://blacklightopac.org/ >> > >> > >> > _______________________________________________ >> > Blacklight-development mailing list >> > Blacklight-development at rubyforge.org >> > http://rubyforge.org/mailman/listinfo/blacklight-development >> > Blacklightopac Blog http://blacklightopac.org/ >> > >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From erikhatcher at mac.com Tue Mar 17 10:22:15 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 17 Mar 2009 10:22:15 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> Message-ID: <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> On Mar 17, 2009, at 8:56 AM, Matt Mitchell wrote: > I still think there is the problem of coming to a single item view > from a bookmark (for example). No problem... just don't show previous/next in that case. > Here's my attempt at sizing these two solutions up: > > going with a session: > - problem of limiting how much gets stored (millions of documents) I guess there is a big misunderstanding of how to use session scope. You most definitely would not store documents in session scope, just the users last search criteria (q, start, fq's, sort, and whatever else was sent to Solr). > going with url > + easy to manage Session is easy to manage as well, perhaps even more so than trying to carry state around on all URLs. > Wonder if this could be something that's abstracted out into > something that's easily overridable? Is there really a need for added complexity? I can't imagine Stanford and UVa have different needs here. Stay simple! Erik From jamie at dang.com Tue Mar 17 10:22:50 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Tue, 17 Mar 2009 10:22:50 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <49BEA794.6000908@jhu.edu> Message-ID: <69ACB4CA-B244-40B3-8E4A-3B3A7B643ECB@dang.com> I thought JRuby did that! :-D On Mar 16, 2009, at 5:39 PM, Erik Hatcher wrote: > You and your "Ajax solves all the worlds problems" :) > > Erik From ndushay at stanford.edu Tue Mar 17 16:59:02 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Tue, 17 Mar 2009 13:59:02 -0700 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> Message-ID: <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> Two things: 1. do we really want multiple URLs to get to the same document? For example: I search "cat" and get doc id 666 in my results. You search "kitten" and get doc id 666 in your results. If the search params appear in the urls, we get something like: (blacklightBaseUrl)/catalog/666/q="cat"&counter=8 vs. (blacklightBaseUrl)/catalog/666/q="kitten"&counter=5 2. Traditionally, libraries have been very concerned with user privacy: they deliberating do NOT keep records on who checked out which materials so they cannot be subpoenaed for that information. I suspect the library staff would be against exposing the searches used. These two reasons, taken together, have me voting for simplifying the single document URLs. So did I hear three options, or were there more? Were any of these rejected out of hand? 1. sessions for next/prev capability 2. AJAX for next/prev capability 3. leave it how it is - Naomi On Mar 17, 2009, at 7:22 AM, Erik Hatcher wrote: > > On Mar 17, 2009, at 8:56 AM, Matt Mitchell wrote: >> I still think there is the problem of coming to a single item view >> from a bookmark (for example). > > No problem... just don't show previous/next in that case. > >> Here's my attempt at sizing these two solutions up: >> >> going with a session: >> - problem of limiting how much gets stored (millions of documents) > > I guess there is a big misunderstanding of how to use session > scope. You most definitely would not store documents in session > scope, just the users last search criteria (q, start, fq's, sort, > and whatever else was sent to Solr). > >> going with url >> + easy to manage > > Session is easy to manage as well, perhaps even more so than trying > to carry state around on all URLs. > >> Wonder if this could be something that's abstracted out into >> something that's easily overridable? > > Is there really a need for added complexity? I can't imagine > Stanford and UVa have different needs here. Stay simple! > > Erik > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From goodieboy at gmail.com Tue Mar 17 17:33:20 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 17 Mar 2009 17:33:20 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> Message-ID: I'm all for simplifying too. I like the simpler urls and less code for handling the logic. If you wanted to implement this, it'd be easy enough to do in your own custom way. I don't think there were any other options? On Tue, Mar 17, 2009 at 4:59 PM, Naomi Dushay wrote: > Two things: > > 1. do we really want multiple URLs to get to the same document? For > example: I search "cat" and get doc id 666 in my results. You search > "kitten" and get doc id 666 in your results. If the search params appear in > the urls, we get something like: > > (blacklightBaseUrl)/catalog/666/q="cat"&counter=8 > > vs. > > (blacklightBaseUrl)/catalog/666/q="kitten"&counter=5 > > 2. Traditionally, libraries have been very concerned with user privacy: > they deliberating do NOT keep records on who checked out which materials so > they cannot be subpoenaed for that information. I suspect the library staff > would be against exposing the searches used. > > These two reasons, taken together, have me voting for simplifying the > single document URLs. > > So did I hear three options, or were there more? Were any of these > rejected out of hand? > > 1. sessions for next/prev capability > 2. AJAX for next/prev capability > 3. leave it how it is > > - Naomi > > > > On Mar 17, 2009, at 7:22 AM, Erik Hatcher wrote: > > >> On Mar 17, 2009, at 8:56 AM, Matt Mitchell wrote: >> >>> I still think there is the problem of coming to a single item view from a >>> bookmark (for example). >>> >> >> No problem... just don't show previous/next in that case. >> >> Here's my attempt at sizing these two solutions up: >>> >>> going with a session: >>> - problem of limiting how much gets stored (millions of documents) >>> >> >> I guess there is a big misunderstanding of how to use session scope. You >> most definitely would not store documents in session scope, just the users >> last search criteria (q, start, fq's, sort, and whatever else was sent to >> Solr). >> >> going with url >>> + easy to manage >>> >> >> Session is easy to manage as well, perhaps even more so than trying to >> carry state around on all URLs. >> >> Wonder if this could be something that's abstracted out into something >>> that's easily overridable? >>> >> >> Is there really a need for added complexity? I can't imagine Stanford and >> UVa have different needs here. Stay simple! >> >> Erik >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erikhatcher at mac.com Tue Mar 17 17:59:14 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 17 Mar 2009 17:59:14 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> Message-ID: On Mar 17, 2009, at 4:59 PM, Naomi Dushay wrote: > Two things: > > 1. do we really want multiple URLs to get to the same document? > For example: I search "cat" and get doc id 666 in my results. You > search "kitten" and get doc id 666 in your results. If the search > params appear in the urls, we get something like: > > (blacklightBaseUrl)/catalog/666/q="cat"&counter=8 > > vs. > > (blacklightBaseUrl)/catalog/666/q="kitten"&counter=5 Again, I'm -1 on putting the query context in the URL to a detail object. Lots of reasons, privacy is one that kinda irks me personally, and another is just the unnecessity of it since you can do it with session scope and if a user comes cold to a detail page just don't show prev/next for them. There'll be enough serendipitous value on a detail page with more-like-this wired in there and hyperlinks on the particular books genres, metadata context, etc. I just don't see where prev/next is at all helpful. But, as far as multiple URLs for the same resource, there's no harm in that. We opted to do that, for example, with LucidFind.... is the same as we did that because we wanted to leverage Google's crawler to find us easily on subject lines, but didn't want the subject line to be a key to a message. So we have a hash function that generates the hexadecimal key you see there, with anything else after the path. We can easily canonicalize this with redirects if the subject doesn't match, but we currently don't do that. Clicking on the subject line takes the user to the canonical URL (until/unless we change the suffix scheme). > 2. Traditionally, libraries have been very concerned with user > privacy: they deliberating do NOT keep records on who checked out > which materials so they cannot be subpoenaed for that information. > I suspect the library staff would be against exposing the searches > used. Indeed. I worked in a library long enough to get that feeling too. > So did I hear three options, or were there more? Were any of these > rejected out of hand? > > 1. sessions for next/prev capability > 2. AJAX for next/prev capability > 3. leave it how it is AJAX doesn't solve the issue at all... still gotta go to the detail page with some state somewhere. Either in a URL somewhere (even if hidden behind an AJAX call) or in session scope. So those options are not exactly comparable solutions to the problem. Sessions is the answer ;) Or creative cookie'ing, same diff. Erik From rochkind at jhu.edu Tue Mar 17 18:14:51 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Tue, 17 Mar 2009 18:14:51 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu>, Message-ID: <90FF863A96E1EC42B8B240D04C88FB1D73406AA7@JHEMTEXVS2.win.ad.jhu.edu> Does it make any sense to consider trying to sniff the HTTP referrer to discover context for next/previous? Rails 'flash' storage may be another solution, although Rails 'flash' always makes me nervous for some reason. Erik, when you say 'session scope', are you just talking about session storage in Rails, or are you talking about something SOLR-specific I might not understand? In general, when I've tried to do this kind of thing before, it gets a bit tricky in that you really want to support multiple searches going on at once in different windows of the same browser. Two different searches in different browser windows could even contain the same item, and when viewing the respective item in those two different windows, you really want the proper next/previous actions. But I agree with everyone that putting the context in the URL is a bad solution. And I agree with Erik that I'm still not seeing how AJAX provides any solution at all. If the context is available to the AJAX code, then couldn't it have been available to the controller to do the same thing without AJAX? The trick is making sure the context is available somehow; once it's available, doing something fancy with AJAX or doing something more ordinary before the html is generated, either one is fine. ________________________________________ From: blacklight-development-bounces at rubyforge.org [blacklight-development-bounces at rubyforge.org] On Behalf Of Erik Hatcher [erikhatcher at mac.com] Sent: Tuesday, March 17, 2009 5:59 PM To: blacklight-development at rubyforge.org Subject: Re: [Blacklight-development] single document questions / concerns On Mar 17, 2009, at 4:59 PM, Naomi Dushay wrote: > Two things: > > 1. do we really want multiple URLs to get to the same document? > For example: I search "cat" and get doc id 666 in my results. You > search "kitten" and get doc id 666 in your results. If the search > params appear in the urls, we get something like: > > (blacklightBaseUrl)/catalog/666/q="cat"&counter=8 > > vs. > > (blacklightBaseUrl)/catalog/666/q="kitten"&counter=5 Again, I'm -1 on putting the query context in the URL to a detail object. Lots of reasons, privacy is one that kinda irks me personally, and another is just the unnecessity of it since you can do it with session scope and if a user comes cold to a detail page just don't show prev/next for them. There'll be enough serendipitous value on a detail page with more-like-this wired in there and hyperlinks on the particular books genres, metadata context, etc. I just don't see where prev/next is at all helpful. But, as far as multiple URLs for the same resource, there's no harm in that. We opted to do that, for example, with LucidFind.... is the same as we did that because we wanted to leverage Google's crawler to find us easily on subject lines, but didn't want the subject line to be a key to a message. So we have a hash function that generates the hexadecimal key you see there, with anything else after the path. We can easily canonicalize this with redirects if the subject doesn't match, but we currently don't do that. Clicking on the subject line takes the user to the canonical URL (until/unless we change the suffix scheme). > 2. Traditionally, libraries have been very concerned with user > privacy: they deliberating do NOT keep records on who checked out > which materials so they cannot be subpoenaed for that information. > I suspect the library staff would be against exposing the searches > used. Indeed. I worked in a library long enough to get that feeling too. > So did I hear three options, or were there more? Were any of these > rejected out of hand? > > 1. sessions for next/prev capability > 2. AJAX for next/prev capability > 3. leave it how it is AJAX doesn't solve the issue at all... still gotta go to the detail page with some state somewhere. Either in a URL somewhere (even if hidden behind an AJAX call) or in session scope. So those options are not exactly comparable solutions to the problem. Sessions is the answer ;) Or creative cookie'ing, same diff. Erik _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org http://rubyforge.org/mailman/listinfo/blacklight-development Blacklightopac Blog http://blacklightopac.org/ From erikhatcher at mac.com Tue Mar 17 19:25:26 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 17 Mar 2009 19:25:26 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <90FF863A96E1EC42B8B240D04C88FB1D73406AA7@JHEMTEXVS2.win.ad.jhu.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> <90FF863A96E1EC42B8B240D04C88FB1D73406AA7@JHEMTEXVS2.win.ad.jhu.edu> Message-ID: On Mar 17, 2009, at 6:14 PM, Jonathan Rochkind wrote: > Does it make any sense to consider trying to sniff the HTTP referrer > to discover context for next/previous? referer [sic] is good info to capture and leverage, no question. for example, if you're getting a referer from a google search result you could use that for highlighting the terms that the user used on google. this technique is played out several sites i've clicked through on, but not currently LucidFind. > Rails 'flash' storage may be another solution, although Rails > 'flash' always makes me nervous for some reason. all flash is is a session scope cheat that gets cleared after it is next accessed, so it's a one time session message... say for some alert you want to put on another page that you're simply redirecting to (with a client-side GET). > Erik, when you say 'session scope', are you just talking about > session storage in Rails, or are you talking about something SOLR- > specific I might not understand? just in rails. solr is stateless across requests (other than building things into caches when serving requests). > In general, when I've tried to do this kind of thing before, it gets > a bit tricky in that you really want to support multiple searches > going on at once in different windows of the same browser. yeah, that can be an issue. session scope is shared between windows spawned from the same site in most natural cases. i imagine there are a number of solutions out there to address this sort of thing. > Two different searches in different browser windows could even > contain the same item, and when viewing the respective item in those > two different windows, you really want the proper next/previous > actions. i still don't see a prev/next as something needed on a detail page, but i'll go with the flow here and assume some usability tests said that was a good idea. personally, i know how to hit the back button and choose the next hit that is colored to indicate i haven't been there before. ajax could come in handy in a sense here, in that you could have a
that loads in parallel showing the current search result in context as succinct links and allowing clicking to the details. look at our mail threaded view perhaps for inspiration? also consider an approach where the search results actually has a detail viewer ajaxed into it... then a user can all results and a detail of one, and a link in that mini-viewer to open it full- browser. just thinking out loud usability wise here. ajax ain't evil... just use it gently and with care. Erik From goodieboy at gmail.com Tue Mar 17 20:39:00 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 17 Mar 2009 20:39:00 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> <90FF863A96E1EC42B8B240D04C88FB1D73406AA7@JHEMTEXVS2.win.ad.jhu.edu> Message-ID: Erik, that's exactly what I meant when I first said "AJAX!"... to handle the next/prev. But when I said that, I didn't understand the entire scope of this discussion. So no, ajax wouldn't really solve the problem where you're needing to pass around a param context and keep the url clean. There needs to be an initial way of getting the params to the request. No different from sessions. I love the idea behind ajax, but I've only used it a handful of times (I promise). Years ago I tried to make a CMS using PHP and ajax. Hierarchical pages, image uploads, and a public front-end. The whole thing was exposed through ONE url! It was a fun experiment, but had lots of problems. I know that a lot of these new javascript frameworks handle some of those bigger issues though. I thought about the referer value too. Seems that it could solve the problem of clearing your search context if you came from a bookmark or a page other than the "search" page. But yeah the multi-window thing is tough. If you provided a way to show a single doc in the search page (a 2-tabbed UI or something), a lot of these problems might go way. Left tab == search, right tab == doc details. There'd have to be a "bookmark this item" link in the details tab to access to the authentic, single-doc url of course (sans search params). You'd already have a list of doc ids for the next/prev thingy too. Here I am promoting ajax :) Matt On Tue, Mar 17, 2009 at 7:25 PM, Erik Hatcher wrote: > > On Mar 17, 2009, at 6:14 PM, Jonathan Rochkind wrote: > >> Does it make any sense to consider trying to sniff the HTTP referrer to >> discover context for next/previous? >> > > referer [sic] is good info to capture and leverage, no question. for > example, if you're getting a referer from a google search result you could > use that for highlighting the terms that the user used on google. this > technique is played out several sites i've clicked through on, but not > currently LucidFind. > > Rails 'flash' storage may be another solution, although Rails 'flash' >> always makes me nervous for some reason. >> > > all flash is is a session scope cheat that gets cleared after it is next > accessed, so it's a one time session message... say for some alert you want > to put on another page that you're simply redirecting to (with a client-side > GET). > > Erik, when you say 'session scope', are you just talking about session >> storage in Rails, or are you talking about something SOLR-specific I might >> not understand? >> > > just in rails. solr is stateless across requests (other than building > things into caches when serving requests). > > In general, when I've tried to do this kind of thing before, it gets a bit >> tricky in that you really want to support multiple searches going on at once >> in different windows of the same browser. >> > > yeah, that can be an issue. session scope is shared between windows > spawned from the same site in most natural cases. i imagine there are a > number of solutions out there to address this sort of thing. > > Two different searches in different browser windows could even contain >> the same item, and when viewing the respective item in those two different >> windows, you really want the proper next/previous actions. >> > > i still don't see a prev/next as something needed on a detail page, but > i'll go with the flow here and assume some usability tests said that was a > good idea. personally, i know how to hit the back button and choose the > next hit that is colored to indicate i haven't been there before. > > ajax could come in handy in a sense here, in that you could have a
> that loads in parallel showing the current search result in context as > succinct links and allowing clicking to the details. look at our mail > threaded view perhaps for inspiration? > > also consider an approach where the search results actually has a detail > viewer ajaxed into it... then a user can all results and a detail of one, > and a link in that mini-viewer to open it full-browser. just thinking out > loud usability wise here. > > ajax ain't evil... just use it gently and with care. > > > Erik > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erikhatcher at mac.com Tue Mar 17 20:44:41 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 17 Mar 2009 20:44:41 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> <90FF863A96E1EC42B8B240D04C88FB1D73406AA7@JHEMTEXVS2.win.ad.jhu.edu> Message-ID: <610ADEE1-993A-420E-8E14-730DF797A7F2@mac.com> On Mar 17, 2009, at 7:25 PM, Erik Hatcher wrote: > i still don't see a prev/next as something needed on a detail page, > but i'll go with the flow here and assume some usability tests said > that was a good idea. personally, i know how to hit the back button > and choose the next hit that is colored to indicate i haven't been > there before. further on this... one usability issue that i've fought hard on with LucidFind was targeting a new window when someone clicks on an external link. again, personally i know how to cmd(or ctrl)-click to open a link in a new tab and don't need a web page to do that for me and it is a bit annoying often when sites do that. but, it's something to consider usability-wise when showing search results and clicking off to details. Erik From erikhatcher at mac.com Tue Mar 17 21:25:16 2009 From: erikhatcher at mac.com (Erik Hatcher) Date: Tue, 17 Mar 2009 21:25:16 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> <90FF863A96E1EC42B8B240D04C88FB1D73406AA7@JHEMTEXVS2.win.ad.jhu.edu> Message-ID: <662D34E6-8093-4625-B9C4-A1D5FA70F1CF@mac.com> On Mar 17, 2009, at 8:39 PM, Matt Mitchell wrote: > If you provided a way to show a single doc in the search page (a 2- > tabbed UI or something), a lot of these problems might go way. Left > tab == search, right tab == doc details. There'd have to be a > "bookmark this item" link in the details tab to access to the > authentic, single-doc url of course (sans search params). You'd > already have a list of doc ids for the next/prev thingy too. Here I > am promoting ajax :) Check out Krugle's tabbed UI: http://krugle.org/kse/entcodespaces/DFHVeQ#2 click on a hit link. I find this a nice paradigm for search results, myself. Erik From rochkind at jhu.edu Wed Mar 18 10:43:39 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Wed, 18 Mar 2009 10:43:39 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <610ADEE1-993A-420E-8E14-730DF797A7F2@mac.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> <90FF863A96E1EC42B8B240D04C88FB1D73406AA7@JHEMTEXVS2.win.ad.jhu.edu> <610ADEE1-993A-420E-8E14-730DF797A7F2@mac.com> Message-ID: <49C1089B.2080401@jhu.edu> I can tell you with confidence that we have many users (including but not limited to librarians) that don't know how to right-click to get a new window, and don't even know how to use the browser back button. That doesn't neccesarily mean that opening a new window is a good idea (I can also tell you that often annoys people), or that next/previous buttons are neccesarily neccesary. More user evidence then I have would be required to determine those questions. There are trade-offs involved. But it's definitely not safe to assume all or even most users know how to do these things. Jonathan Erik Hatcher wrote: > On Mar 17, 2009, at 7:25 PM, Erik Hatcher wrote: > >> i still don't see a prev/next as something needed on a detail page, >> but i'll go with the flow here and assume some usability tests said >> that was a good idea. personally, i know how to hit the back button >> and choose the next hit that is colored to indicate i haven't been >> there before. >> > > further on this... one usability issue that i've fought hard on with > LucidFind was targeting a new window when someone clicks on an > external link. again, personally i know how to cmd(or ctrl)-click to > open a link in a new tab and don't need a web page to do that for me > and it is a bit annoying often when sites do that. > > but, it's something to consider usability-wise when showing search > results and clicking off to details. > > Erik > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From rochkind at jhu.edu Wed Mar 18 10:47:05 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Wed, 18 Mar 2009 10:47:05 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <610ADEE1-993A-420E-8E14-730DF797A7F2@mac.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> <90FF863A96E1EC42B8B240D04C88FB1D73406AA7@JHEMTEXVS2.win.ad.jhu.edu> <610ADEE1-993A-420E-8E14-730DF797A7F2@mac.com> Message-ID: <49C10969.6000505@jhu.edu> I can tell you with confidence that we have many users (including but not limited to librarians) that don't know how to right-click to get a new window, and don't even know how to use the browser back button. That doesn't neccesarily mean that opening a new window is a good idea (I can also tell you that often annoys people), or that next/previous buttons are neccesarily neccesary. More user evidence then I have would be required to determine those questions. There are trade-offs involved. But it's definitely not safe to assume all or even most users know how to do these things. Jonathan Erik Hatcher wrote: > On Mar 17, 2009, at 7:25 PM, Erik Hatcher wrote: > >> i still don't see a prev/next as something needed on a detail page, >> but i'll go with the flow here and assume some usability tests said >> that was a good idea. personally, i know how to hit the back button >> and choose the next hit that is colored to indicate i haven't been >> there before. >> > > further on this... one usability issue that i've fought hard on with > LucidFind was targeting a new window when someone clicks on an > external link. again, personally i know how to cmd(or ctrl)-click to > open a link in a new tab and don't need a web page to do that for me > and it is a bit annoying often when sites do that. > > but, it's something to consider usability-wise when showing search > results and clicking off to details. > > Erik > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From llc2w at virginia.edu Wed Mar 18 11:25:40 2009 From: llc2w at virginia.edu (L Campbell) Date: Wed, 18 Mar 2009 11:25:40 -0400 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <90FF863A96E1EC42B8B240D04C88FB1D73406AA7@JHEMTEXVS2.win.ad.jhu.edu> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> <90FF863A96E1EC42B8B240D04C88FB1D73406AA7@JHEMTEXVS2.win.ad.jhu.edu> Message-ID: <792298050903180825g3115b8e2u74923214d35e2dd9@mail.gmail.com> On Tue, Mar 17, 2009 at 6:14 PM, Jonathan Rochkind wrote: > Does it make any sense to consider trying to sniff the HTTP referrer to discover context for next/previous? I just wanted to +1 this. I think all of the other options are a bit excessive to simply have a prev/next link on the item detail page. Since those links only make sense within the context of a search, I think it's logical to simply grab that context from the referrer, when available. From eos8d at virginia.edu Thu Mar 19 11:58:03 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Thu, 19 Mar 2009 11:58:03 -0400 Subject: [Blacklight-development] blacklightopac.org downtime Message-ID: Hi, everyone. We had a power outage at Alderman library this morning and the server that runs blacklightopac.org went down. It should be back up now, but it might go down again briefly because I've asked our systems folks to move it to a UPS so this won't happen anymore. Sorry for the service interruption! Bess From ndushay at stanford.edu Thu Mar 19 12:27:12 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Thu, 19 Mar 2009 09:27:12 -0700 Subject: [Blacklight-development] blacklightopac.org downtime In-Reply-To: References: Message-ID: <2F23F8DE-AD94-4B45-A783-C0F23492DBBA@stanford.edu> actually, I believe it was down from about 7pm EDT yesterday ... - Naomi On Mar 19, 2009, at 8:58 AM, Bess Sadler wrote: > Hi, everyone. We had a power outage at Alderman library this morning > and the server that runs blacklightopac.org went down. It should be > back up now, but it might go down again briefly because I've asked > our systems folks to move it to a UPS so this won't happen anymore. > Sorry for the service interruption! > > Bess > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From jamie at dang.com Thu Mar 19 12:47:48 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Thu, 19 Mar 2009 12:47:48 -0400 Subject: [Blacklight-development] blacklightopac.org downtime In-Reply-To: References: Message-ID: <40174AF9-7266-418E-B334-23FF0818571B@dang.com> Looks like none of the java apps are set to startup with the server (JIRA and the demo instance of Jetty). On Mar 19, 2009, at 11:58 AM, Bess Sadler wrote: > Hi, everyone. We had a power outage at Alderman library this morning > and the server that runs blacklightopac.org went down. It should be > back up now, but it might go down again briefly because I've asked > our systems folks to move it to a UPS so this won't happen anymore. > Sorry for the service interruption! > > Bess > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From eos8d at virginia.edu Thu Mar 19 19:47:24 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Thu, 19 Mar 2009 19:47:24 -0400 Subject: [Blacklight-development] jira is back up Message-ID: <6FD159E0-ACB3-4BB3-AE2F-06672C67D1BF@virginia.edu> Sorry for all the blacklightopac.org server drama. I forgot to start tomcat6 automatically on boot, so jira was down all day. It's back up now, and I've fixed it so it will come back up automatically next time the box reboots. Bess From jamie at dang.com Fri Mar 20 11:41:09 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Fri, 20 Mar 2009 11:41:09 -0400 Subject: [Blacklight-development] specs passing Message-ID: <22A7694A-C603-4296-9BCC-9BD2D988D34E@dang.com> Specs are passing currently: http://cc.blacklightopac.org/ The index needed to be blown away and the sample data reindexed with the new schema. There was also one example that was failing: #TODO id => 1 is not in the sample data, so this fails it "should get document" do get :show, :id => 1 # assigns[:document].should_not be_nil end There's no item with that id in the sample data, so I've commented the assigns() call out for now. Right now specs that call out to a live Solr instance are quite fragile. We (Naomi, Bess and I?) need to discuss how to reduce this fragility so that we have reliable spec runs. Jamie From goodieboy at gmail.com Fri Mar 20 13:34:09 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Fri, 20 Mar 2009 13:34:09 -0400 Subject: [Blacklight-development] SolrHelper location Message-ID: I just noticed that the SolrHelper module is in the standard view helper directory, but it's only used to add behavior to the controller layer. Does anyone have a problem with me moving SolrHelper to: vendor/plugins/blacklight/lib/blacklight/controller/solr_helper.rb ? Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From jamie at dang.com Fri Mar 20 13:48:56 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Fri, 20 Mar 2009 13:48:56 -0400 Subject: [Blacklight-development] SolrHelper location In-Reply-To: References: Message-ID: <127B3A31-DEB7-4B30-981B-F812E8F2AD95@dang.com> Folks on the phone call say go for it. On Mar 20, 2009, at 1:34 PM, Matt Mitchell wrote: > I just noticed that the SolrHelper module is in the standard view > helper directory, but it's only used to add behavior to the > controller layer. > > Does anyone have a problem with me moving SolrHelper to: > > vendor/plugins/blacklight/lib/blacklight/controller/solr_helper.rb > > ? > > Matt > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From goodieboy at gmail.com Fri Mar 20 13:52:56 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Fri, 20 Mar 2009 13:52:56 -0400 Subject: [Blacklight-development] DisplayFields module vs Blacklight.solr_config Message-ID: Just checked out the DisplayFields module. I love this idea! A few questions/comments: At first glance, I thought that this should be in the Blacklight module: Blacklight::DisplayFields But after I saw how the code worked, I thought that this should actually be part of Blacklight.solr_config If you look at the Blacklight module, you can see that it's already loading the solr.yml file. For example, instead of DisplayFields.show_view, it'd be Blacklight.solr_config[:show_view] Any objections to me making this so? Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From ndushay at stanford.edu Fri Mar 20 14:10:25 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Fri, 20 Mar 2009 11:10:25 -0700 Subject: [Blacklight-development] SolrHelper location In-Reply-To: References: Message-ID: <352258F2-370D-49EB-A3E7-B12B28A2038B@stanford.edu> sounds fine. Good even. - Naomi On Mar 20, 2009, at 10:34 AM, Matt Mitchell wrote: > I just noticed that the SolrHelper module is in the standard view > helper directory, but it's only used to add behavior to the > controller layer. > > Does anyone have a problem with me moving SolrHelper to: > > vendor/plugins/blacklight/lib/blacklight/controller/solr_helper.rb > > ? > > Matt > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From ndushay at stanford.edu Fri Mar 20 14:50:19 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Fri, 20 Mar 2009 11:50:19 -0700 Subject: [Blacklight-development] DisplayFields module vs Blacklight.solr_config In-Reply-To: References: Message-ID: Discussed in IRC. Matt will be refactoring for this. Possibly making use of use of config/initializers/blacklight.rb as well. I already thought this var name was too long: DisplayFields.show_view Blacklight.solr_config[:show_view] is even longer. Mentioned to Matt. me: "to me, clarity trumps brevity" Matt: "maybe after some time, we'll start to identify ways to make things shorter?" me: "but i'm sure others can be clear and shorten my names. refactoring to the rescue!!!" Matt: "I love refactoring!" me: "e too ... but only with tests to make sure i didn't mess up!" - Naomi On Mar 20, 2009, at 10:52 AM, Matt Mitchell wrote: > Just checked out the DisplayFields module. I love this idea! A few > questions/comments: > > At first glance, I thought that this should be in the Blacklight > module: Blacklight::DisplayFields > > But after I saw how the code worked, I thought that this should > actually be part of Blacklight.solr_config > > If you look at the Blacklight module, you can see that it's already > loading the solr.yml file. > > For example, instead of DisplayFields.show_view, it'd be > Blacklight.solr_config[:show_view] > > Any objections to me making this so? > > Matt > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From goodieboy at gmail.com Fri Mar 20 15:31:56 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Fri, 20 Mar 2009 15:31:56 -0400 Subject: [Blacklight-development] please update, just committed rev. 1039 Message-ID: Lots of changes, be sure to upit! And read the log if your interested in the specifics. Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From goodieboy at gmail.com Fri Mar 20 15:51:29 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Fri, 20 Mar 2009 15:51:29 -0400 Subject: [Blacklight-development] please update, just committed rev. 1039 In-Reply-To: References: Message-ID: For convenience... here's the log: changed the require statements in the init.rb file to config.gem this allows us to be more specific about dependencies http://api.rubyonrails.org/classes/Rails/Configuration.html#M002480 moved the Blacklight.init call to the config.after_initialize block required to use config.gem removed calls to Blacklight.solr.commit in SolrHelper this was something I left in on accident when "testing" one day :) migrated to RSolr version 0.8.1 now using RSolr::Ext for request/response helpers version 0.5.9 Re-added the catalog controller #facet method this is used for paging through values of a single facet field - note: this is how the UVa facet menu works Updated specs fixed all differences that came from the rsolr upgrade fixed the home_controller spec, "should map default route to {:controller => 'home'}" Added a call to DisplayFields.init in the SolrHelper module * this is just a temporary fix to get the DisplayFields kicked into gear Changed the output of the solr param footer in the catalog/_solr_request.html.erb partial Added a blank blacklight initializer - will be throwing some stuff in there later -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony.zanella at gmail.com Fri Mar 20 21:27:42 2009 From: tony.zanella at gmail.com (Tony Zanella) Date: Fri, 20 Mar 2009 21:27:42 -0400 Subject: [Blacklight-development] including accordion effect for facets Message-ID: <1e4bf2cc0903201827g36693a8mb19261fd8868952a@mail.gmail.com> Hello all, Some folks have requested an accordion effect be included in the Blacklight demo. I decided to take a swing at writing one. While I was researching, I found that there are a number of different effects that go by the name "accordion". Since I wanted to keep swinging, I wrote a bare-bones javascript I think could be expanded or customized pretty easily. With the latest version of the demo, I added the javascript to /bl-demo/rails/public/plugin_assets/blacklight/javascripts and I linked it into the interface at /bl-demo/rails/vendor/plugins/blacklight/app/views/layouts and everything worked just fine. The question is: where should the addition and linkage really take place, assuming that it's a good idea to include this functionality in the first place? I guess the other option is to stay out of the plugin. If we were to stay out of the plugin, wouldn't it make sense to include a layout in, say, /bl-demo/rails/app/views ? and include the javascript in, say, /bl-demo/rails/public/javascripts ? Thanks! Tony Zanella From eos8d at virginia.edu Fri Mar 20 22:32:55 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Fri, 20 Mar 2009 22:32:55 -0400 Subject: [Blacklight-development] including accordion effect for facets In-Reply-To: <1e4bf2cc0903201827g36693a8mb19261fd8868952a@mail.gmail.com> References: <1e4bf2cc0903201827g36693a8mb19261fd8868952a@mail.gmail.com> Message-ID: <89FE55E0-8478-4B3B-92B3-433793EDA3D0@virginia.edu> I'm in favor of including this, particularly since we won't have much nice styling in the 2.0 release, and the accordion facets are one styling feature that people ask about a lot and that have done really well in usability testing. I can see an argument for including it in the plugin and for including it in the demo app, but I think I'd prefer to see it included in the demo app, where it can also serve as an example of how to overload behavior with local customization. Other thoughts? Bess On 20-Mar-09, at 9:27 PM, Tony Zanella wrote: > Hello all, > Some folks have requested an accordion effect be included in the > Blacklight demo. I decided to take a swing at writing one. While I was > researching, I found that there are a number of different effects that > go by the name "accordion". Since I wanted to keep swinging, I wrote a > bare-bones javascript I think could be expanded or customized pretty > easily. With the latest version of the demo, I added the javascript to > > /bl-demo/rails/public/plugin_assets/blacklight/javascripts > > and I linked it into the interface at > > /bl-demo/rails/vendor/plugins/blacklight/app/views/layouts > > and everything worked just fine. > > The question is: where should the addition and linkage really take > place, assuming that it's a good idea to include this functionality in > the first place? I guess the other option is to stay out of the > plugin. > > If we were to stay out of the plugin, wouldn't it make sense to > include a layout in, say, /bl-demo/rails/app/views ? and include the > javascript in, say, /bl-demo/rails/public/javascripts ? > > Thanks! > Tony Zanella > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From goodieboy at gmail.com Sat Mar 21 00:32:59 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Sat, 21 Mar 2009 00:32:59 -0400 Subject: [Blacklight-development] including accordion effect for facets In-Reply-To: <89FE55E0-8478-4B3B-92B3-433793EDA3D0@virginia.edu> References: <1e4bf2cc0903201827g36693a8mb19261fd8868952a@mail.gmail.com> <89FE55E0-8478-4B3B-92B3-433793EDA3D0@virginia.edu> Message-ID: I'm excited to see what you've come up with Tony. Demo or plugin... either way, it seems like the sort if thing that'd be easy to override as long as it'd non-obtrusive JS. Is it possible for the menu to maintain state on page loads? As in... keep the accordion items open if a value has been selected within? One thing to be careful of is the public/plugin_assets directory. When Rails starts up (when the Engine plugin inits), the contents of the vendor/plugin/blacklight/assets dir are copied into the public/plugin_assets dir. So any changes you make to public/plugin_assets will be lost when you restart. Matt On Fri, Mar 20, 2009 at 10:32 PM, Bess Sadler wrote: > I'm in favor of including this, particularly since we won't have much nice > styling in the 2.0 release, and the accordion facets are one styling feature > that people ask about a lot and that have done really well in usability > testing. I can see an argument for including it in the plugin and for > including it in the demo app, but I think I'd prefer to see it included in > the demo app, where it can also serve as an example of how to overload > behavior with local customization. > > Other thoughts? > > Bess > > > On 20-Mar-09, at 9:27 PM, Tony Zanella wrote: > > Hello all, >> Some folks have requested an accordion effect be included in the >> Blacklight demo. I decided to take a swing at writing one. While I was >> researching, I found that there are a number of different effects that >> go by the name "accordion". Since I wanted to keep swinging, I wrote a >> bare-bones javascript I think could be expanded or customized pretty >> easily. With the latest version of the demo, I added the javascript to >> >> /bl-demo/rails/public/plugin_assets/blacklight/javascripts >> >> and I linked it into the interface at >> >> /bl-demo/rails/vendor/plugins/blacklight/app/views/layouts >> >> and everything worked just fine. >> >> The question is: where should the addition and linkage really take >> place, assuming that it's a good idea to include this functionality in >> the first place? I guess the other option is to stay out of the >> plugin. >> >> If we were to stay out of the plugin, wouldn't it make sense to >> include a layout in, say, /bl-demo/rails/app/views ? and include the >> javascript in, say, /bl-demo/rails/public/javascripts ? >> >> Thanks! >> Tony Zanella >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eos8d at virginia.edu Sat Mar 21 16:20:37 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Sat, 21 Mar 2009 16:20:37 -0400 Subject: [Blacklight-development] re-writing some tests with mocks Message-ID: <1ECE0D1F-182D-4951-8569-052A5B923EE4@virginia.edu> Hi, folks. In keeping with good rspec practice, I'm going to try to re-write our rspec tests not to depend on a live solr instance to be running. Instead, I'm going to mock up the responses they would expect to get from solr, and include those in the rspec suite. This will offer a few advantages, but the number one reason is that we don't want all our tests to break when you point your Blacklight app at a new index that doesn't contain our sample records. I'm going to try this, and hope my rspec skills are up to the challenge. If there are any objections, or if I'm missing some reason why I shouldn't do this, let me know. Bess From eos8d at virginia.edu Sat Mar 21 18:19:12 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Sat, 21 Mar 2009 18:19:12 -0400 Subject: [Blacklight-development] re-writing some tests with mocks In-Reply-To: <1ECE0D1F-182D-4951-8569-052A5B923EE4@virginia.edu> References: <1ECE0D1F-182D-4951-8569-052A5B923EE4@virginia.edu> Message-ID: Okay, I made a first pass at this. I also commented out some tests that were failing, because I wanted to start with a 100% passing test suite before I made any major changes. I'll re-write those failing tests and add them back as I'm able. All of the tests I've changed so far are in solr_helper_spec.rb. It was running several queries, e.g., all_docs_query, no_docs_query, single_word_query, mult_word_query. I ran each of these against an index with all of our demo records in it, grabbed the response and put it in a mock method in mocks.rb, like this: def mock_no_docs_query %({"response"=> {"maxScore"=>0.0, "start"=>0, "docs"=>[], "numFound"=>0}, "facet_counts"=> { "facet_fields"=> { "subject_era_facet"=>[], "language_facet"=>[], "format_facet"=>[], "geographic_subject_facet"=>[]}, "facet_dates"=>{}, "facet_queries"=>{}}, "responseHeader"=> {"QTime"=>1, "params"=> {"qt"=>"search", "q"=>"zzzzzzzzzzzz", "wt"=>"ruby", "rows"=>"10"}, "status"=>0}}) end Then in the test, I create a RSolr::Ext::Response::Standard object, which is what the test is expecting, and populate it with that mocked response, like this: # use the mock response in mocks.rb raw_response = eval(mock_query_response) solr_response = RSolr::Ext::Response::Standard.new(raw_response) The rest of the tests remain the same. The only thing that has changed is that the response is coming from a text file instead of from solr. I left the old lines in place, but commented out and labeled, in case anyone wants to run against a live version of solr. Matt, I totally copied this technique from your rsolr-ext tests. Thank you! Bess On 21-Mar-09, at 4:20 PM, Bess Sadler wrote: > Hi, folks. > > In keeping with good rspec practice, I'm going to try to re-write our > rspec tests not to depend on a live solr instance to be running. > Instead, I'm going to mock up the responses they would expect to get > from solr, and include those in the rspec suite. This will offer a few > advantages, but the number one reason is that we don't want all our > tests to break when you point your Blacklight app at a new index that > doesn't contain our sample records. > > I'm going to try this, and hope my rspec skills are up to the > challenge. If there are any objections, or if I'm missing some reason > why I shouldn't do this, let me know. > > Bess > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From goodieboy at gmail.com Sat Mar 21 19:44:03 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Sat, 21 Mar 2009 19:44:03 -0400 Subject: [Blacklight-development] re-writing some tests with mocks In-Reply-To: References: <1ECE0D1F-182D-4951-8569-052A5B923EE4@virginia.edu> Message-ID: Nice. Hey you probably already know this but... if you want to get good "mock" responses, just go straight to solr and set the wt param to ruby. A lifetime supply available! On Sat, Mar 21, 2009 at 6:19 PM, Bess Sadler wrote: > Okay, I made a first pass at this. I also commented out some tests that > were failing, because I wanted to start with a 100% passing test suite > before I made any major changes. I'll re-write those failing tests and add > them back as I'm able. All of the tests I've changed so far are in > solr_helper_spec.rb. It was running several queries, e.g., all_docs_query, > no_docs_query, single_word_query, mult_word_query. I ran each of these > against an index with all of our demo records in it, grabbed the response > and put it in a mock method in mocks.rb, like this: > > def mock_no_docs_query > %({"response"=> > {"maxScore"=>0.0, "start"=>0, "docs"=>[], "numFound"=>0}, > "facet_counts"=> > { "facet_fields"=> > { "subject_era_facet"=>[], > "language_facet"=>[], > "format_facet"=>[], > "geographic_subject_facet"=>[]}, > "facet_dates"=>{}, > "facet_queries"=>{}}, > "responseHeader"=> > {"QTime"=>1, "params"=> > {"qt"=>"search", "q"=>"zzzzzzzzzzzz", "wt"=>"ruby", "rows"=>"10"}, > "status"=>0}}) > end > > Then in the test, I create a RSolr::Ext::Response::Standard object, which > is what the test is expecting, and populate it with that mocked response, > like this: > > # use the mock response in mocks.rb > raw_response = eval(mock_query_response) > solr_response = RSolr::Ext::Response::Standard.new(raw_response) > > The rest of the tests remain the same. The only thing that has changed is > that the response is coming from a text file instead of from solr. I left > the old lines in place, but commented out and labeled, in case anyone wants > to run against a live version of solr. > > Matt, I totally copied this technique from your rsolr-ext tests. Thank you! > > Bess > > > On 21-Mar-09, at 4:20 PM, Bess Sadler wrote: > > Hi, folks. >> >> In keeping with good rspec practice, I'm going to try to re-write our >> rspec tests not to depend on a live solr instance to be running. >> Instead, I'm going to mock up the responses they would expect to get >> from solr, and include those in the rspec suite. This will offer a few >> advantages, but the number one reason is that we don't want all our >> tests to break when you point your Blacklight app at a new index that >> doesn't contain our sample records. >> >> I'm going to try this, and hope my rspec skills are up to the >> challenge. If there are any objections, or if I'm missing some reason >> why I shouldn't do this, let me know. >> >> Bess >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eos8d at virginia.edu Sat Mar 21 20:34:55 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Sat, 21 Mar 2009 20:34:55 -0400 Subject: [Blacklight-development] re-writing some tests with mocks In-Reply-To: References: <1ECE0D1F-182D-4951-8569-052A5B923EE4@virginia.edu> Message-ID: After talking to Naomi I realized that I had to roll back these mocked tests. With the mocks in place, we would only be testing whether the solr_helper could properly handle responses from rsolr, but we would not be testing whether solr_helper knew how to create valid queries. I've rolled back my changes. We're going to need that local solr instance to test against after all. Hey, at least I learned a lot about rspec and mocks today! :) Bess On 21-Mar-09, at 6:19 PM, Bess Sadler wrote: > Okay, I made a first pass at this. I also commented out some tests > that were failing, because I wanted to start with a 100% passing test > suite before I made any major changes. I'll re-write those failing > tests and add them back as I'm able. All of the tests I've changed so > far are in solr_helper_spec.rb. It was running several queries, e.g., > all_docs_query, no_docs_query, single_word_query, mult_word_query. I > ran each of these against an index with all of our demo records in it, > grabbed the response and put it in a mock method in mocks.rb, like > this: > > def mock_no_docs_query > %({"response"=> > {"maxScore"=>0.0, "start"=>0, "docs"=>[], "numFound"=>0}, > "facet_counts"=> > { "facet_fields"=> > { "subject_era_facet"=>[], > "language_facet"=>[], > "format_facet"=>[], > "geographic_subject_facet"=>[]}, > "facet_dates"=>{}, > "facet_queries"=>{}}, > "responseHeader"=> > {"QTime"=>1, "params"=> > {"qt"=>"search", "q"=>"zzzzzzzzzzzz", "wt"=>"ruby", > "rows"=>"10"}, "status"=>0}}) > end > > Then in the test, I create a RSolr::Ext::Response::Standard object, > which is what the test is expecting, and populate it with that mocked > response, like this: > > # use the mock response in mocks.rb > raw_response = eval(mock_query_response) > solr_response = RSolr::Ext::Response::Standard.new(raw_response) > > The rest of the tests remain the same. The only thing that has changed > is that the response is coming from a text file instead of from solr. > I left the old lines in place, but commented out and labeled, in case > anyone wants to run against a live version of solr. > > Matt, I totally copied this technique from your rsolr-ext tests. Thank > you! > > Bess > > > On 21-Mar-09, at 4:20 PM, Bess Sadler wrote: > >> Hi, folks. >> >> In keeping with good rspec practice, I'm going to try to re-write our >> rspec tests not to depend on a live solr instance to be running. >> Instead, I'm going to mock up the responses they would expect to get >> from solr, and include those in the rspec suite. This will offer a >> few >> advantages, but the number one reason is that we don't want all our >> tests to break when you point your Blacklight app at a new index that >> doesn't contain our sample records. >> >> I'm going to try this, and hope my rspec skills are up to the >> challenge. If there are any objections, or if I'm missing some reason >> why I shouldn't do this, let me know. >> >> Bess >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From eos8d at Virginia.EDU Sun Mar 22 10:40:37 2009 From: eos8d at Virginia.EDU (Bess Sadler) Date: Sun, 22 Mar 2009 10:40:37 -0400 Subject: [Blacklight-development] storing marc21 in solr Message-ID: <9904BED0-6123-4AFB-B6AC-C447541FB1F2@Virginia.EDU> Hey, folks. Hopefully I'm just having a moment of stupidity, but I've been looking at this for awhile and I can't figure it out, so I'm asking the community. We want to store marc21 in our solr index, right? UVA has always stored marc-xml, but we had a talk with Stanford folks awhile ago about this, there seem to be lots of good reasons to store marc21 instead, and that's fine with me, so now I'm just trying to make it happen. BUT... when I store what I think is marc21 into solr, it doesn't come back out again quite right. I've isolated a really simple example of this: require 'rubygems' require 'marc' require 'rsolr' puts " ************* from file ****************** " reader = MARC::Reader.new('/usr/local/projects/bl-demo/data/ test_data.utf8.mrc') record2 = reader.first puts record2.to_s puts " ************* from solr ****************** " rsolr = RSolr.connect response = eval(rsolr.select(:q=>'*:*',:wt=>'ruby')) marc_display = response["response"]["docs"][0]["marc_display"] record = MARC::Record.new_from_marc(marc_display) puts record.to_s My output for that script looks like this: ************* from file ****************** LEADER 00799cam a2200241 a 4500 001 00282214 003 DLC 005 20090120022042.0 008 000417s1998 pk 000 0 urdo 010 $a 00282214 025 $a P-U-00282214; 05; 06 040 $a DLC $c DLC $d DLC 041 1 $a urd $h snd 042 $a lcode 050 00 $a PK2788.9.A9 $b F55 1998 100 1 $a Ayaz, Shaikh, $d 1923-1997. 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, Sha?h Muh?ammad Pi?rza?dah. 260 $a Kara?ci? : $b Da?niya?l, $c [1998] 300 $a 375 p. ; $c 23 cm. 546 $a In Urdu. 520 $a Selected poems and articles from the works of renowned Sindhi poet; chiefly translated from Sindhi. 700 1 $a Farruk?h?i?, A?s?if, $d 1959- 700 1 $a Pi?rza?dah, Sha?h Muh?ammad. ************* from solr ****************** LEADER 00799cam a2200241 a 4500 001 00282214 * 003 DLC* 005 20090120022042.0* 008 000417s1998 pk 000 0 urdo * So, when I take a file (test_data.utf.mrc) and read in the marc records from disk, it behaves as expected. But if I store it in solr first, then what I get back out of solr appears to be badly formed marc somehow. MARC::Record appears to read it in, but the record created is truncated. Naomi, how do you folks store marc21? Do you do anything special to it before putting it in the index? It occurred to me we could base64 encode it, but maybe there's something simpler that I'm missing? I'm storing the marc record at around line 96 of marc_mapper.rb: # _display is stored, but not indexed # don't store a string, store marc21 so we can read it back out # into a MARC::Record object map :marc_display do |rec,index| rec.to_marc end Thanks in advance for any advice, Bess From goodieboy at gmail.com Sun Mar 22 11:11:52 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Sun, 22 Mar 2009 11:11:52 -0400 Subject: [Blacklight-development] storing marc21 in solr In-Reply-To: <9904BED0-6123-4AFB-B6AC-C447541FB1F2@Virginia.EDU> References: <9904BED0-6123-4AFB-B6AC-C447541FB1F2@Virginia.EDU> Message-ID: <532B4FA0-B702-4127-824C-0AA2046932B4@gmail.com> One quick observation Bess... you don't need to eval the output from rsolr. It is already ruby. The mocks though, are strings so they must be eval'd. This might have something to do with it? Sent from my iPhone On Mar 22, 2009, at 10:40 AM, Bess Sadler wrote: > Hey, folks. > > Hopefully I'm just having a moment of stupidity, but I've been > looking at this for awhile and I can't figure it out, so I'm asking > the community. We want to store marc21 in our solr index, right? UVA > has always stored marc-xml, but we had a talk with Stanford folks > awhile ago about this, there seem to be lots of good reasons to > store marc21 instead, and that's fine with me, so now I'm just > trying to make it happen. > > BUT... when I store what I think is marc21 into solr, it doesn't > come back out again quite right. I've isolated a really simple > example of this: > > require 'rubygems' > require 'marc' > require 'rsolr' > > puts " ************* from file ****************** " > > reader = MARC::Reader.new('/usr/local/projects/bl-demo/data/ > test_data.utf8.mrc') > record2 = reader.first > puts record2.to_s > > puts " ************* from solr ****************** " > > rsolr = RSolr.connect > response = eval(rsolr.select(:q=>'*:*',:wt=>'ruby')) > marc_display = response["response"]["docs"][0]["marc_display"] > record = MARC::Record.new_from_marc(marc_display) > puts record.to_s > > My output for that script looks like this: > > ************* from file ****************** > LEADER 00799cam a2200241 a 4500 > 001 00282214 > 003 DLC > 005 20090120022042.0 > 008 000417s1998 pk 000 0 urdo > 010 $a 00282214 > 025 $a P-U-00282214; 05; 06 > 040 $a DLC $c DLC $d DLC > 041 1 $a urd $h snd > 042 $a lcode > 050 00 $a PK2788.9.A9 $b F55 1998 > 100 1 $a Ayaz, Shaikh, $d 1923-1997. > 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, > Sha?h Muh?ammad Pi?rza?dah. > 260 $a Kara?ci? : $b Da?niya?l, $c [1998] > 300 $a 375 p. ; $c 23 cm. > 546 $a In Urdu. > 520 $a Selected poems and articles from the works of renowned > Sindhi poet; chiefly translated from Sindhi. > 700 1 $a Farruk?h?i?, A?s?if, $d 1959- > 700 1 $a Pi?rza?dah, Sha?h Muh?ammad. > ************* from solr ****************** > LEADER 00799cam a2200241 a 4500 > 001 00282214 * > 003 DLC* > 005 20090120022042.0* > 008 000417s1998 pk 000 0 urdo * > > > So, when I take a file (test_data.utf.mrc) and read in the marc > records from disk, it behaves as expected. But if I store it in solr > first, then what I get back out of solr appears to be badly formed > marc somehow. MARC::Record appears to read it in, but the record > created is truncated. > > Naomi, how do you folks store marc21? Do you do anything special to > it before putting it in the index? It occurred to me we could base64 > encode it, but maybe there's something simpler that I'm missing? > > I'm storing the marc record at around line 96 of marc_mapper.rb: > > # _display is stored, but not indexed > # don't store a string, store marc21 so we can read it back out > # into a MARC::Record object > map :marc_display do |rec,index| > rec.to_marc > end > > Thanks in advance for any advice, > > Bess > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From eos8d at virginia.edu Sun Mar 22 13:16:24 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Sun, 22 Mar 2009 13:16:24 -0400 Subject: [Blacklight-development] storing marc21 in solr - base64 encoding? References: <4ADA3BBF-FD59-4C44-987D-6A33A750692C@eservices.virginia.edu> Message-ID: <6522F236-6B1B-4BEE-8AAA-AFAE039E94EB@virginia.edu> I decided to try my base64 encoding idea, and sure enough it works. So now the marc_mapper.rb looks like this: map :marc_display do |rec,index| Base64.encode64(rec.to_marc) end And my example code looks like this: require 'rubygems' require 'marc' require 'rsolr' require 'base64' puts " ************* from file ****************** " reader = MARC::Reader.new('/usr/local/projects/bl-demo/data/ test_data.utf8.mrc') record2 = reader.first puts record2.to_s puts " ************* from solr ****************** " rsolr = RSolr.connect response = rsolr.select(:q=>'*:*') marc_display = response["response"]["docs"][0]["marc_display"] record = MARC::Record.new_from_marc(Base64.decode64(marc_display)) puts record.to_s and sure enough, now the records are identical: ************* from file ****************** LEADER 00799cam a2200241 a 4500 001 00282214 003 DLC 005 20090120022042.0 008 000417s1998 pk 000 0 urdo 010 $a 00282214 025 $a P-U-00282214; 05; 06 040 $a DLC $c DLC $d DLC 041 1 $a urd $h snd 042 $a lcode 050 00 $a PK2788.9.A9 $b F55 1998 100 1 $a Ayaz, Shaikh, $d 1923-1997. 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, Sha?h Muh?ammad Pi?rza?dah. 260 $a Kara?ci? : $b Da?niya?l, $c [1998] 300 $a 375 p. ; $c 23 cm. 546 $a In Urdu. 520 $a Selected poems and articles from the works of renowned Sindhi poet; chiefly translated from Sindhi. 700 1 $a Farruk?h?i?, A?s?if, $d 1959- 700 1 $a Pi?rza?dah, Sha?h Muh?ammad. ************* from solr ****************** LEADER 00799cam a2200241 a 4500 001 00282214 003 DLC 005 20090120022042.0 008 000417s1998 pk 000 0 urdo 010 $a 00282214 025 $a P-U-00282214; 05; 06 040 $a DLC $c DLC $d DLC 041 1 $a urd $h snd 042 $a lcode 050 00 $a PK2788.9.A9 $b F55 1998 100 1 $a Ayaz, Shaikh, $d 1923-1997. 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, Sha?h Muh?ammad Pi?rza?dah. 260 $a Kara?ci? : $b Da?niya?l, $c [1998] 300 $a 375 p. ; $c 23 cm. 546 $a In Urdu. 520 $a Selected poems and articles from the works of renowned Sindhi poet; chiefly translated from Sindhi. 700 1 $a Farruk?h?i?, A?s?if, $d 1959- 700 1 $a Pi?rza?dah, Sha?h Muh?ammad. Maybe solr is losing some of the special chars or whatever from the marc record if you don't encode it first? Are there any objections to base64 encoding the stored marc record? It seems to me that base64 encoding and decoding won't use more processing overhead than encoding and decoding xml would. From the solr mailing list archives it does indeed look like base64 encoding a binary file is the answer to being able to store it and retrieve it from solr. (See, for example: http://www.lucidimagination.com/search/document/ba8f662decebfe9a/base64_support_containers) Bess > On Mar 22, 2009, at 10:40 AM, Bess Sadler wrote: > >> Hey, folks. >> >> Hopefully I'm just having a moment of stupidity, but I've been >> looking at this for awhile and I can't figure it out, so I'm asking >> the community. We want to store marc21 in our solr index, right? UVA >> has always stored marc-xml, but we had a talk with Stanford folks >> awhile ago about this, there seem to be lots of good reasons to >> store marc21 instead, and that's fine with me, so now I'm just >> trying to make it happen. >> >> BUT... when I store what I think is marc21 into solr, it doesn't >> come back out again quite right. I've isolated a really simple >> example of this: >> >> require 'rubygems' >> require 'marc' >> require 'rsolr' >> >> puts " ************* from file ****************** " >> >> reader = MARC::Reader.new('/usr/local/projects/bl-demo/data/ >> test_data.utf8.mrc') >> record2 = reader.first >> puts record2.to_s >> >> puts " ************* from solr ****************** " >> >> rsolr = RSolr.connect >> response = eval(rsolr.select(:q=>'*:*',:wt=>'ruby')) >> marc_display = response["response"]["docs"][0]["marc_display"] >> record = MARC::Record.new_from_marc(marc_display) >> puts record.to_s >> >> My output for that script looks like this: >> >> ************* from file ****************** >> LEADER 00799cam a2200241 a 4500 >> 001 00282214 >> 003 DLC >> 005 20090120022042.0 >> 008 000417s1998 pk 000 0 urdo >> 010 $a 00282214 >> 025 $a P-U-00282214; 05; 06 >> 040 $a DLC $c DLC $d DLC >> 041 1 $a urd $h snd >> 042 $a lcode >> 050 00 $a PK2788.9.A9 $b F55 1998 >> 100 1 $a Ayaz, Shaikh, $d 1923-1997. >> 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, >> Sha?h Muh?ammad Pi?rza?dah. >> 260 $a Kara?ci? : $b Da?niya?l, $c [1998] >> 300 $a 375 p. ; $c 23 cm. >> 546 $a In Urdu. >> 520 $a Selected poems and articles from the works of renowned >> Sindhi poet; chiefly translated from Sindhi. >> 700 1 $a Farruk?h?i?, A?s?if, $d 1959- >> 700 1 $a Pi?rza?dah, Sha?h Muh?ammad. >> ************* from solr ****************** >> LEADER 00799cam a2200241 a 4500 >> 001 00282214 * >> 003 DLC* >> 005 20090120022042.0* >> 008 000417s1998 pk 000 0 urdo * >> >> >> So, when I take a file (test_data.utf.mrc) and read in the marc >> records from disk, it behaves as expected. But if I store it in solr >> first, then what I get back out of solr appears to be badly formed >> marc somehow. MARC::Record appears to read it in, but the record >> created is truncated. >> >> Naomi, how do you folks store marc21? Do you do anything special to >> it before putting it in the index? It occurred to me we could base64 >> encode it, but maybe there's something simpler that I'm missing? >> >> I'm storing the marc record at around line 96 of marc_mapper.rb: >> >> # _display is stored, but not indexed >> # don't store a string, store marc21 so we can read it back out >> # into a MARC::Record object >> map :marc_display do |rec,index| >> rec.to_marc >> end >> >> Thanks in advance for any advice, >> >> Bess >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From ndushay at stanford.edu Sun Mar 22 13:47:23 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Sun, 22 Mar 2009 10:47:23 -0700 Subject: [Blacklight-development] storing marc21 in solr In-Reply-To: <9904BED0-6123-4AFB-B6AC-C447541FB1F2@Virginia.EDU> References: <9904BED0-6123-4AFB-B6AC-C447541FB1F2@Virginia.EDU> Message-ID: Bess, I create our index using solrmarc, which writes directly to the index (do not use solr, do not pass go, do not cellect $200.). So I can't be much help, sorry. However, I'll tell you that our marc21 doesn't look all nicely formatted like yours - it just appears as a single line and looks like this: 00651nam a22001575? 4500001000800000008004100008020001500049035002200064100001900086245006900105260003300174300001100207440005400218596000600272999021500278 a575946 910118s1990||||gw |||||||||||||||||ger|d a3412215880 a(CSt)notisACX2206 10aSeifert, Arno. 14aDer Ruckzug der biblischen Prophetie von der neueren Geschichte. 0 aKoln :bBohlau Verlag,c1990 a207 p. 0aBeihefte zum Archiv f?r Kulturgeschichte ;vv.31 a1 aCB3 .A6 SUPPL. V. 31  wLC  c1 i36105035087092d6/18/2008e6/18/2008kCHECKEDOUTlSTACKSmGREENn4p $40.00rMsYtSTKS-MONOu1/18/1991o.TECHSTAFF. c:alo.TECHSTAFF. PUB 1/22/91/pbo.TECHSTAFF. MIDSPINE.v.31 suppl. Maybe you don't have "marc21" but something else? - Naomi On Mar 22, 2009, at 7:40 AM, Bess Sadler wrote: > Hey, folks. > > Hopefully I'm just having a moment of stupidity, but I've been > looking at this for awhile and I can't figure it out, so I'm asking > the community. We want to store marc21 in our solr index, right? UVA > has always stored marc-xml, but we had a talk with Stanford folks > awhile ago about this, there seem to be lots of good reasons to > store marc21 instead, and that's fine with me, so now I'm just > trying to make it happen. > > BUT... when I store what I think is marc21 into solr, it doesn't > come back out again quite right. I've isolated a really simple > example of this: > > require 'rubygems' > require 'marc' > require 'rsolr' > > puts " ************* from file ****************** " > > reader = MARC::Reader.new('/usr/local/projects/bl-demo/data/ > test_data.utf8.mrc') > record2 = reader.first > puts record2.to_s > > puts " ************* from solr ****************** " > > rsolr = RSolr.connect > response = eval(rsolr.select(:q=>'*:*',:wt=>'ruby')) > marc_display = response["response"]["docs"][0]["marc_display"] > record = MARC::Record.new_from_marc(marc_display) > puts record.to_s > > My output for that script looks like this: > > ************* from file ****************** > LEADER 00799cam a2200241 a 4500 > 001 00282214 > 003 DLC > 005 20090120022042.0 > 008 000417s1998 pk 000 0 urdo > 010 $a 00282214 > 025 $a P-U-00282214; 05; 06 > 040 $a DLC $c DLC $d DLC > 041 1 $a urd $h snd > 042 $a lcode > 050 00 $a PK2788.9.A9 $b F55 1998 > 100 1 $a Ayaz, Shaikh, $d 1923-1997. > 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, > Sha?h Muh?ammad Pi?rza?dah. > 260 $a Kara?ci? : $b Da?niya?l, $c [1998] > 300 $a 375 p. ; $c 23 cm. > 546 $a In Urdu. > 520 $a Selected poems and articles from the works of renowned > Sindhi poet; chiefly translated from Sindhi. > 700 1 $a Farruk?h?i?, A?s?if, $d 1959- > 700 1 $a Pi?rza?dah, Sha?h Muh?ammad. > ************* from solr ****************** > LEADER 00799cam a2200241 a 4500 > 001 00282214 * > 003 DLC* > 005 20090120022042.0* > 008 000417s1998 pk 000 0 urdo * > > > So, when I take a file (test_data.utf.mrc) and read in the marc > records from disk, it behaves as expected. But if I store it in solr > first, then what I get back out of solr appears to be badly formed > marc somehow. MARC::Record appears to read it in, but the record > created is truncated. > > Naomi, how do you folks store marc21? Do you do anything special to > it before putting it in the index? It occurred to me we could base64 > encode it, but maybe there's something simpler that I'm missing? > > I'm storing the marc record at around line 96 of marc_mapper.rb: > > # _display is stored, but not indexed > # don't store a string, store marc21 so we can read it back out > # into a MARC::Record object > map :marc_display do |rec,index| > rec.to_marc > end > > Thanks in advance for any advice, > > Bess > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From rochkind at jhu.edu Sun Mar 22 16:14:30 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Sun, 22 Mar 2009 16:14:30 -0400 Subject: [Blacklight-development] storing marc21 in solr In-Reply-To: References: <9904BED0-6123-4AFB-B6AC-C447541FB1F2@Virginia.EDU>, Message-ID: <90FF863A96E1EC42B8B240D04C88FB1D73406AB8@JHEMTEXVS2.win.ad.jhu.edu> It makes sense to me to standardize 'what kind of MARC' we all store in SOLR. I guess you guys decided it made more sense for that to be MARC21 than MARC-XML? Curious what the motivation for that was? Since MARC21 is a binary format, you've got to make sure that what is going into the interface is binary identical to what should be, it should not be re-encoded or translated at all. MARC21 data can be in one of at least two character encodings, which one is used is theoretically mentioned in the MARC file (although it's often wrong). MARC21 file length (in bytes) is also noted in the MARC file header/leader. The whole thing needs to be byte-for-byte untouched when you stick it in your SOLR index, I don't know enough about SOLR or the indexing process to know about places that might introduce corruption there. MARC-XML is definitely a lot easier to work with, as it's just an ordinary XML file in UTF-8. I don't know a whole lot about MARC21, but it would not surprise me at all, if, like the charcter encoding issue, much of our MARC21 "in the wild" was actually illegal in undocumented various ways, but ways that our traditional ILS's are tolerant of. It's also possible there is a bug in the ruby MARC library. Jonathan ________________________________________ From: blacklight-development-bounces at rubyforge.org [blacklight-development-bounces at rubyforge.org] On Behalf Of Naomi Dushay [ndushay at stanford.edu] Sent: Sunday, March 22, 2009 1:47 PM To: blacklight-development at rubyforge.org Subject: Re: [Blacklight-development] storing marc21 in solr Bess, I create our index using solrmarc, which writes directly to the index (do not use solr, do not pass go, do not cellect $200.). So I can't be much help, sorry. However, I'll tell you that our marc21 doesn't look all nicely formatted like yours - it just appears as a single line and looks like this: 00651nam a22001575? 4500001000800000008004100008020001500049035002200064100001900086245006900105260003300174300001100207440005400218596000600272999021500278 a575946 910118s1990||||gw |||||||||||||||||ger|d a3412215880 a(CSt)notisACX2206 10aSeifert, Arno. 14aDer Ruckzug der biblischen Prophetie von der neueren Geschichte. 0 aKoln :bBohlau Verlag,c1990 a207 p. 0aBeihefte zum Archiv f?r Kulturgeschichte ;vv.31 a1 aCB3 .A6 SUPPL. V. 31  wLC  c1 i36105035087092d6/18/2008e6/18/2008kCHECKEDOUTlSTACKSmGREENn4p $40.00rMsYtSTKS-MONOu1/18/1991o.TECHSTAFF. c:alo.TECHSTAFF. PUB 1/22/91/pbo.TECHSTAFF. MIDSPINE.v.31 suppl. Maybe you don't have "marc21" but something else? - Naomi On Mar 22, 2009, at 7:40 AM, Bess Sadler wrote: > Hey, folks. > > Hopefully I'm just having a moment of stupidity, but I've been > looking at this for awhile and I can't figure it out, so I'm asking > the community. We want to store marc21 in our solr index, right? UVA > has always stored marc-xml, but we had a talk with Stanford folks > awhile ago about this, there seem to be lots of good reasons to > store marc21 instead, and that's fine with me, so now I'm just > trying to make it happen. > > BUT... when I store what I think is marc21 into solr, it doesn't > come back out again quite right. I've isolated a really simple > example of this: > > require 'rubygems' > require 'marc' > require 'rsolr' > > puts " ************* from file ****************** " > > reader = MARC::Reader.new('/usr/local/projects/bl-demo/data/ > test_data.utf8.mrc') > record2 = reader.first > puts record2.to_s > > puts " ************* from solr ****************** " > > rsolr = RSolr.connect > response = eval(rsolr.select(:q=>'*:*',:wt=>'ruby')) > marc_display = response["response"]["docs"][0]["marc_display"] > record = MARC::Record.new_from_marc(marc_display) > puts record.to_s > > My output for that script looks like this: > > ************* from file ****************** > LEADER 00799cam a2200241 a 4500 > 001 00282214 > 003 DLC > 005 20090120022042.0 > 008 000417s1998 pk 000 0 urdo > 010 $a 00282214 > 025 $a P-U-00282214; 05; 06 > 040 $a DLC $c DLC $d DLC > 041 1 $a urd $h snd > 042 $a lcode > 050 00 $a PK2788.9.A9 $b F55 1998 > 100 1 $a Ayaz, Shaikh, $d 1923-1997. > 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, > Sha?h Muh?ammad Pi?rza?dah. > 260 $a Kara?ci? : $b Da?niya?l, $c [1998] > 300 $a 375 p. ; $c 23 cm. > 546 $a In Urdu. > 520 $a Selected poems and articles from the works of renowned > Sindhi poet; chiefly translated from Sindhi. > 700 1 $a Farruk?h?i?, A?s?if, $d 1959- > 700 1 $a Pi?rza?dah, Sha?h Muh?ammad. > ************* from solr ****************** > LEADER 00799cam a2200241 a 4500 > 001 00282214 * > 003 DLC* > 005 20090120022042.0* > 008 000417s1998 pk 000 0 urdo * > > > So, when I take a file (test_data.utf.mrc) and read in the marc > records from disk, it behaves as expected. But if I store it in solr > first, then what I get back out of solr appears to be badly formed > marc somehow. MARC::Record appears to read it in, but the record > created is truncated. > > Naomi, how do you folks store marc21? Do you do anything special to > it before putting it in the index? It occurred to me we could base64 > encode it, but maybe there's something simpler that I'm missing? > > I'm storing the marc record at around line 96 of marc_mapper.rb: > > # _display is stored, but not indexed > # don't store a string, store marc21 so we can read it back out > # into a MARC::Record object > map :marc_display do |rec,index| > rec.to_marc > end > > Thanks in advance for any advice, > > Bess > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org http://rubyforge.org/mailman/listinfo/blacklight-development Blacklightopac Blog http://blacklightopac.org/ From bill at dueber.com Sun Mar 22 21:03:48 2009 From: bill at dueber.com (Bill Dueber) Date: Sun, 22 Mar 2009 21:03:48 -0400 Subject: [Blacklight-development] storing marc21 in solr In-Reply-To: <90FF863A96E1EC42B8B240D04C88FB1D73406AB8@JHEMTEXVS2.win.ad.jhu.edu> References: <9904BED0-6123-4AFB-B6AC-C447541FB1F2@Virginia.EDU> <90FF863A96E1EC42B8B240D04C88FB1D73406AB8@JHEMTEXVS2.win.ad.jhu.edu> Message-ID: solrmarc writes to the underlying lucene index using a binary field type that is unavailable in solr. This makes for easy storage of MARC, but makes round-tripping the data a little harder if that's your thing. Given that's the case, I wonder if a standard solr String field (probably compressed) of marcxml would be a better bet all around. Or push dchud into finishing his spec implementation of marc-json and store it as json :-) On Sun, Mar 22, 2009 at 4:14 PM, Jonathan Rochkind wrote: > It makes sense to me to standardize 'what kind of MARC' we all store in > SOLR. I guess you guys decided it made more sense for that to be MARC21 > than MARC-XML? Curious what the motivation for that was? > > Since MARC21 is a binary format, you've got to make sure that what is going > into the interface is binary identical to what should be, it should not be > re-encoded or translated at all. MARC21 data can be in one of at least two > character encodings, which one is used is theoretically mentioned in the > MARC file (although it's often wrong). MARC21 file length (in bytes) is > also noted in the MARC file header/leader. The whole thing needs to be > byte-for-byte untouched when you stick it in your SOLR index, I don't know > enough about SOLR or the indexing process to know about places that might > introduce corruption there. > > MARC-XML is definitely a lot easier to work with, as it's just an ordinary > XML file in UTF-8. > > I don't know a whole lot about MARC21, but it would not surprise me at all, > if, like the charcter encoding issue, much of our MARC21 "in the wild" was > actually illegal in undocumented various ways, but ways that our traditional > ILS's are tolerant of. > > It's also possible there is a bug in the ruby MARC library. > > Jonathan > > ________________________________________ > From: blacklight-development-bounces at rubyforge.org [ > blacklight-development-bounces at rubyforge.org] On Behalf Of Naomi Dushay [ > ndushay at stanford.edu] > Sent: Sunday, March 22, 2009 1:47 PM > To: blacklight-development at rubyforge.org > Subject: Re: [Blacklight-development] storing marc21 in solr > > Bess, > > I create our index using solrmarc, which writes directly to the index > (do not use solr, do not pass go, do not cellect $200.). So I can't > be much help, sorry. > > However, I'll tell you that our marc21 doesn't look all nicely > formatted like yours - it just appears as a single line and looks like > this: > > 00651nam a22001575? > > 4500001000800000008004100008020001500049035002200064100001900086245006900105260003300174300001100207440005400218596000600272999021500278 > a575946 910118s1990||||gw |||||||||||||||||ger|d a3412215880 > a(CSt)notisACX2206 10 aSeifert, Arno. 14 aDer Ruckzug der biblischen > Prophetie von der neueren Geschichte. 0 aKoln : bBohlau > Verlag, c1990 a207 p. 0 aBeihefte zum Archiv f?r > Kulturgeschichte ; vv.31 a1 aCB3 .A6 SUPPL. V. > 31 > > wLC > > c1 > i36105035087092 d6/18/2008 e6/18/2008 kCHECKEDOUT lSTACKS mGREEN n4 p > $40.00 rM sY tSTKS-MONO u1/18/1991 o.TECHSTAFF. c:al o.TECHSTAFF. PUB > 1/22/91/pb o.TECHSTAFF. MIDSPINE.v.31 suppl. > > Maybe you don't have "marc21" but something else? > > - Naomi > > On Mar 22, 2009, at 7:40 AM, Bess Sadler wrote: > > > Hey, folks. > > > > Hopefully I'm just having a moment of stupidity, but I've been > > looking at this for awhile and I can't figure it out, so I'm asking > > the community. We want to store marc21 in our solr index, right? UVA > > has always stored marc-xml, but we had a talk with Stanford folks > > awhile ago about this, there seem to be lots of good reasons to > > store marc21 instead, and that's fine with me, so now I'm just > > trying to make it happen. > > > > BUT... when I store what I think is marc21 into solr, it doesn't > > come back out again quite right. I've isolated a really simple > > example of this: > > > > require 'rubygems' > > require 'marc' > > require 'rsolr' > > > > puts " ************* from file ****************** " > > > > reader = MARC::Reader.new('/usr/local/projects/bl-demo/data/ > > test_data.utf8.mrc') > > record2 = reader.first > > puts record2.to_s > > > > puts " ************* from solr ****************** " > > > > rsolr = RSolr.connect > > response = eval(rsolr.select(:q=>'*:*',:wt=>'ruby')) > > marc_display = response["response"]["docs"][0]["marc_display"] > > record = MARC::Record.new_from_marc(marc_display) > > puts record.to_s > > > > My output for that script looks like this: > > > > ************* from file ****************** > > LEADER 00799cam a2200241 a 4500 > > 001 00282214 > > 003 DLC > > 005 20090120022042.0 > > 008 000417s1998 pk 000 0 urdo > > 010 $a 00282214 > > 025 $a P-U-00282214; 05; 06 > > 040 $a DLC $c DLC $d DLC > > 041 1 $a urd $h snd > > 042 $a lcode > > 050 00 $a PK2788.9.A9 $b F55 1998 > > 100 1 $a Ayaz, Shaikh, $d 1923-1997. > > 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, > > Sha?h Muh?ammad Pi?rza?dah. > > 260 $a Kara?ci? : $b Da?niya?l, $c [1998] > > 300 $a 375 p. ; $c 23 cm. > > 546 $a In Urdu. > > 520 $a Selected poems and articles from the works of renowned > > Sindhi poet; chiefly translated from Sindhi. > > 700 1 $a Farruk?h?i?, A?s?if, $d 1959- > > 700 1 $a Pi?rza?dah, Sha?h Muh?ammad. > > ************* from solr ****************** > > LEADER 00799cam a2200241 a 4500 > > 001 00282214 * > > 003 DLC* > > 005 20090120022042.0* > > 008 000417s1998 pk 000 0 urdo * > > > > > > So, when I take a file (test_data.utf.mrc) and read in the marc > > records from disk, it behaves as expected. But if I store it in solr > > first, then what I get back out of solr appears to be badly formed > > marc somehow. MARC::Record appears to read it in, but the record > > created is truncated. > > > > Naomi, how do you folks store marc21? Do you do anything special to > > it before putting it in the index? It occurred to me we could base64 > > encode it, but maybe there's something simpler that I'm missing? > > > > I'm storing the marc record at around line 96 of marc_mapper.rb: > > > > # _display is stored, but not indexed > > # don't store a string, store marc21 so we can read it back out > > # into a MARC::Record object > > map :marc_display do |rec,index| > > rec.to_marc > > end > > > > Thanks in advance for any advice, > > > > Bess > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -- Bill Dueber Library Systems Programmer University of Michigan Library -------------- next part -------------- An HTML attachment was scrubbed... URL: From ndushay at stanford.edu Mon Mar 23 01:56:04 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Sun, 22 Mar 2009 22:56:04 -0700 Subject: [Blacklight-development] storing marc21 in solr In-Reply-To: References: <9904BED0-6123-4AFB-B6AC-C447541FB1F2@Virginia.EDU> <90FF863A96E1EC42B8B240D04C88FB1D73406AB8@JHEMTEXVS2.win.ad.jhu.edu> Message-ID: Maybe it's too soon to worry about this. Is there any pressing need for all of us to store our marc data in our indexes in the same format? What's the real gain? Why should the blacklight code be rigid about this? Remember, a lot of our solr documents won't have marc. What if there was a blacklight config setting to say what format your marc was in, and the blacklight code accommodated the different formats? Jessie is already working on manipulating the marc21 in the Stanford index to create the UI features we want at Stanford. UVa has been using marcxml. Jessie should jump in and tell us if the work he's doing can be easily adapted to other marc formats -- hopefully the answer is "yes, it would be trivial." But let's get those features coded first, and refactor them later. I think we're doing great things here. UVa has put out a great body of code. The active Stanford adoption of the code is causing an excellent refactoring of the code, making it more configurable; additional features will be added as well. Further active adopters or folks that get down and dirty with the demo will provide more feedback, causing further improvements - refactoring or new features or whatever. The demo for release 2.0 is going to have some warts; I don't think there's a way around it. Usage driven methodology would say we have some folks take the demo out for a spin, and get their feedback. Have folks tried the current demo? Have they tried to go the next step, of getting their own data behind the blacklight front end? If not ... maybe we've gotten ahead of ourselves? Trying to guess a lot of this stuff before there's expressed demand from the users ... I think that can make extra work, and the results might not be useful, in the end. Okay, I've got my asbestos suit on - let the flames be thrown! - Naomi On Mar 22, 2009, at 6:03 PM, Bill Dueber wrote: > solrmarc writes to the underlying lucene index using a binary field > type that is unavailable in solr. This makes for easy storage of > MARC, but makes round-tripping the data a little harder if that's > your thing. Given that's the case, I wonder if a standard solr > String field (probably compressed) of marcxml would be a better bet > all around. > > Or push dchud into finishing his spec implementation of marc-json > and store it as json :-) > > On Sun, Mar 22, 2009 at 4:14 PM, Jonathan Rochkind > wrote: > It makes sense to me to standardize 'what kind of MARC' we all store > in SOLR. I guess you guys decided it made more sense for that to be > MARC21 than MARC-XML? Curious what the motivation for that was? > > Since MARC21 is a binary format, you've got to make sure that what > is going into the interface is binary identical to what should be, > it should not be re-encoded or translated at all. MARC21 data can > be in one of at least two character encodings, which one is used is > theoretically mentioned in the MARC file (although it's often > wrong). MARC21 file length (in bytes) is also noted in the MARC > file header/leader. The whole thing needs to be byte-for-byte > untouched when you stick it in your SOLR index, I don't know enough > about SOLR or the indexing process to know about places that might > introduce corruption there. > > MARC-XML is definitely a lot easier to work with, as it's just an > ordinary XML file in UTF-8. > > I don't know a whole lot about MARC21, but it would not surprise me > at all, if, like the charcter encoding issue, much of our MARC21 "in > the wild" was actually illegal in undocumented various ways, but > ways that our traditional ILS's are tolerant of. > > It's also possible there is a bug in the ruby MARC library. > > Jonathan > > ________________________________________ > From: blacklight-development-bounces at rubyforge.org [blacklight-development-bounces at rubyforge.org > ] On Behalf Of Naomi Dushay [ndushay at stanford.edu] > Sent: Sunday, March 22, 2009 1:47 PM > To: blacklight-development at rubyforge.org > Subject: Re: [Blacklight-development] storing marc21 in solr > > Bess, > > I create our index using solrmarc, which writes directly to the index > (do not use solr, do not pass go, do not cellect $200.). So I can't > be much help, sorry. > > However, I'll tell you that our marc21 doesn't look all nicely > formatted like yours - it just appears as a single line and looks like > this: > > 00651nam a22001575? > 4500001000800000008004100008020001500049035002200064100001900086245006900105260003300174300001100207440005400218596000600272999021500278 > a575946 910118s1990||||gw |||||||||||||||||ger|d a3412215880 > a(CSt)notisACX2206 10 aSeifert, Arno. 14 aDer Ruckzug der biblischen > Prophetie von der neueren Geschichte. 0 aKoln : bBohlau > Verlag, c1990 a207 p. 0 aBeihefte zum Archiv f?r > Kulturgeschichte ; vv.31 a1 aCB3 .A6 SUPPL. V. > 31 > > wLC > > c1 > i36105035087092 d6/18/2008 e6/18/2008 kCHECKEDOUT lSTACKS mGREEN n4 p > $40.00 rM sY tSTKS-MONO u1/18/1991 o.TECHSTAFF. c:al o.TECHSTAFF. PUB > 1/22/91/pb o.TECHSTAFF. MIDSPINE.v.31 suppl. > > Maybe you don't have "marc21" but something else? > > - Naomi > > On Mar 22, 2009, at 7:40 AM, Bess Sadler wrote: > > > Hey, folks. > > > > Hopefully I'm just having a moment of stupidity, but I've been > > looking at this for awhile and I can't figure it out, so I'm asking > > the community. We want to store marc21 in our solr index, right? UVA > > has always stored marc-xml, but we had a talk with Stanford folks > > awhile ago about this, there seem to be lots of good reasons to > > store marc21 instead, and that's fine with me, so now I'm just > > trying to make it happen. > > > > BUT... when I store what I think is marc21 into solr, it doesn't > > come back out again quite right. I've isolated a really simple > > example of this: > > > > require 'rubygems' > > require 'marc' > > require 'rsolr' > > > > puts " ************* from file ****************** " > > > > reader = MARC::Reader.new('/usr/local/projects/bl-demo/data/ > > test_data.utf8.mrc') > > record2 = reader.first > > puts record2.to_s > > > > puts " ************* from solr ****************** " > > > > rsolr = RSolr.connect > > response = eval(rsolr.select(:q=>'*:*',:wt=>'ruby')) > > marc_display = response["response"]["docs"][0]["marc_display"] > > record = MARC::Record.new_from_marc(marc_display) > > puts record.to_s > > > > My output for that script looks like this: > > > > ************* from file ****************** > > LEADER 00799cam a2200241 a 4500 > > 001 00282214 > > 003 DLC > > 005 20090120022042.0 > > 008 000417s1998 pk 000 0 urdo > > 010 $a 00282214 > > 025 $a P-U-00282214; 05; 06 > > 040 $a DLC $c DLC $d DLC > > 041 1 $a urd $h snd > > 042 $a lcode > > 050 00 $a PK2788.9.A9 $b F55 1998 > > 100 1 $a Ayaz, Shaikh, $d 1923-1997. > > 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, > > Sha?h Muh?ammad Pi?rza?dah. > > 260 $a Kara?ci? : $b Da?niya?l, $c [1998] > > 300 $a 375 p. ; $c 23 cm. > > 546 $a In Urdu. > > 520 $a Selected poems and articles from the works of renowned > > Sindhi poet; chiefly translated from Sindhi. > > 700 1 $a Farruk?h?i?, A?s?if, $d 1959- > > 700 1 $a Pi?rza?dah, Sha?h Muh?ammad. > > ************* from solr ****************** > > LEADER 00799cam a2200241 a 4500 > > 001 00282214 * > > 003 DLC* > > 005 20090120022042.0* > > 008 000417s1998 pk 000 0 urdo * > > > > > > So, when I take a file (test_data.utf.mrc) and read in the marc > > records from disk, it behaves as expected. But if I store it in solr > > first, then what I get back out of solr appears to be badly formed > > marc somehow. MARC::Record appears to read it in, but the record > > created is truncated. > > > > Naomi, how do you folks store marc21? Do you do anything special to > > it before putting it in the index? It occurred to me we could base64 > > encode it, but maybe there's something simpler that I'm missing? > > > > I'm storing the marc record at around line 96 of marc_mapper.rb: > > > > # _display is stored, but not indexed > > # don't store a string, store marc21 so we can read it back out > > # into a MARC::Record object > > map :marc_display do |rec,index| > > rec.to_marc > > end > > > > Thanks in advance for any advice, > > > > Bess > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > > > -- > Bill Dueber > Library Systems Programmer > University of Michigan Library > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossfsinger at gmail.com Mon Mar 23 10:39:00 2009 From: rossfsinger at gmail.com (Ross Singer) Date: Mon, 23 Mar 2009 10:39:00 -0400 Subject: [Blacklight-development] storing marc21 in solr In-Reply-To: References: <9904BED0-6123-4AFB-B6AC-C447541FB1F2@Virginia.EDU> <90FF863A96E1EC42B8B240D04C88FB1D73406AB8@JHEMTEXVS2.win.ad.jhu.edu> Message-ID: <23b83f160903230739h7d8961adweeb28fddecf4d85@mail.gmail.com> I realize I'm coming into this a little late -- not sure if Bess's other thread about Base64 encoding this "solved it". I agree with Naomi that we don't need to standardize on anything, but, obviously, the more people that do it in a certain way helps provide some traction. I would argue that storing as a marc21 string has the bigger upside for now. ruby-marc can only use REXML as its parser presently (although Ed and I had a conversation just last week about refactoring it to support Nokogiri, as well), so the binary marc reader (and, if needed, writer) is way faster. Even accounting for Base64 encoding/decoding. It would also keep the index a whole lot smaller, if anybody cares about that. BTW, the reason why you're seeing the line breaks in Bess's script is because .to_s pretty prints the record. If she called .to_marc it would look like Naomi's mushy blob of text. -Ross. 2009/3/23 Naomi Dushay : > Maybe it's too soon to worry about this. ?Is there any pressing need for all > of us to store our marc data in our indexes in the same format? ? What's the > real gain? ?Why should the blacklight code be rigid about this? ?Remember, a > lot of our solr documents won't have?marc. > What if there was a blacklight config setting to say what format your marc > was in, and the blacklight code accommodated the different formats? > Jessie is already working on manipulating the marc21 in the Stanford index > to create the UI features we want at Stanford. ?UVa has been using marcxml. > ??Jessie should jump in and tell us if the work he's doing can be easily > adapted to other marc formats -- hopefully the answer is "yes, it would be > trivial." ? But let's get those features coded first, and refactor them > later. > I think we're doing great things here. ?UVa has put out a great body of > code. ?The active Stanford adoption of the code is causing an?excellent > refactoring of the code, making it more configurable; ?additional features > will be added as well. ? Further active adopters or folks that get down and > dirty with the demo will provide more feedback, causing further improvements > - refactoring or new features or whatever. > The demo for release 2.0 is going to have some warts; ?I don't think there's > a way around it. ?Usage driven methodology would say we??have some folks > take the demo out for a spin, and get their feedback. ?Have folks tried the > current demo? ?Have they tried to go the next step, of getting their own > data behind the blacklight front end? ? If not ... maybe we've gotten ahead > of ourselves? > Trying to guess a lot of this stuff before there's expressed demand from the > users ... I think that can make extra work, and the results might not be > useful, in the end. > Okay, I've got my asbestos suit on - let the flames be thrown! > - Naomi > > On Mar 22, 2009, at 6:03 PM, Bill Dueber wrote: > > solrmarc writes to the underlying lucene index using a binary field type > that is unavailable in solr. This makes for easy storage of MARC, but makes > round-tripping the data a little harder if that's your thing. Given that's > the case, I wonder if a standard solr String field (probably compressed) of > marcxml would be a better bet all around. > > Or push dchud into finishing his spec implementation of marc-json and store > it as json :-) > > On Sun, Mar 22, 2009 at 4:14 PM, Jonathan Rochkind wrote: >> >> It makes sense to me to standardize 'what kind of MARC' we all store in >> SOLR. ?I guess you guys decided it made more sense for that to be MARC21 >> than MARC-XML? ?Curious what the motivation for that was? >> >> Since MARC21 is a binary format, you've got to make sure that what is >> going into the interface is binary identical to what should be, it should >> not be re-encoded or translated at all. ?MARC21 data can be in one of at >> least two character encodings, which one is used is theoretically mentioned >> in the MARC file (although it's often wrong). ? MARC21 file length (in >> bytes) is also noted in the MARC file header/leader. ?The whole thing needs >> to be byte-for-byte untouched when you stick it in your SOLR index, I don't >> know enough about SOLR or the indexing process to know about places that >> might introduce corruption there. >> >> MARC-XML is definitely a lot easier to work with, as it's just an ordinary >> XML file in UTF-8. >> >> I don't know a whole lot about MARC21, but it would not surprise me at >> all, if, like the charcter encoding issue, much of our MARC21 "in the wild" >> was actually illegal in undocumented various ways, but ways that our >> traditional ILS's are tolerant of. >> >> It's also possible there is a bug in the ruby MARC library. >> >> Jonathan >> >> ________________________________________ >> From: blacklight-development-bounces at rubyforge.org >> [blacklight-development-bounces at rubyforge.org] On Behalf Of Naomi Dushay >> [ndushay at stanford.edu] >> Sent: Sunday, March 22, 2009 1:47 PM >> To: blacklight-development at rubyforge.org >> Subject: Re: [Blacklight-development] storing marc21 in solr >> >> Bess, >> >> I create our index using solrmarc, which writes directly to the index >> (do not use solr, do not pass go, do not cellect $200.). ? So I can't >> be much help, sorry. >> >> However, I'll tell you that our marc21 doesn't look all nicely >> formatted like yours - it just appears as a single line and looks like >> this: >> >> 00651nam a22001575? >> >> 4500001000800000008004100008020001500049035002200064100001900086245006900105260003300174300001100207440005400218596000600272999021500278 >> ?a575946 910118s1990||||gw |||||||||||||||||ger|d ? ?a3412215880 >> ?a(CSt)notisACX2206 10 aSeifert, Arno. 14 aDer Ruckzug der biblischen >> Prophetie von der neueren Geschichte. 0 ?aKoln : bBohlau >> Verlag, c1990 ? ?a207 p. ?0 aBeihefte zum Archiv f?r >> Kulturgeschichte ; vv.31 ? ?a1 ? ?aCB3 .A6 SUPPL. V. >> 31 >> >> wLC >> >> c1 >> ?i36105035087092 d6/18/2008 e6/18/2008 kCHECKEDOUT lSTACKS mGREEN n4 p >> $40.00 rM sY tSTKS-MONO u1/18/1991 o.TECHSTAFF. c:al o.TECHSTAFF. PUB >> 1/22/91/pb o.TECHSTAFF. MIDSPINE.v.31 suppl. >> >> Maybe you don't have "marc21" but something else? >> >> - Naomi >> >> On Mar 22, 2009, at 7:40 AM, Bess Sadler wrote: >> >> > Hey, folks. >> > >> > Hopefully I'm just having a moment of stupidity, but I've been >> > looking at this for awhile and I can't figure it out, so I'm asking >> > the community. We want to store marc21 in our solr index, right? UVA >> > has always stored marc-xml, but we had a talk with Stanford folks >> > awhile ago about this, there seem to be lots of good reasons to >> > store marc21 instead, and that's fine with me, so now I'm just >> > trying to make it happen. >> > >> > BUT... when I store what I think is marc21 into solr, it doesn't >> > come back out again quite right. I've isolated a really simple >> > example of this: >> > >> > require 'rubygems' >> > require 'marc' >> > require 'rsolr' >> > >> > puts " ************* from file ****************** " >> > >> > reader = MARC::Reader.new('/usr/local/projects/bl-demo/data/ >> > test_data.utf8.mrc') >> > record2 = reader.first >> > puts record2.to_s >> > >> > puts " ************* from solr ****************** " >> > >> > rsolr = RSolr.connect >> > response = eval(rsolr.select(:q=>'*:*',:wt=>'ruby')) >> > marc_display = response["response"]["docs"][0]["marc_display"] >> > record = MARC::Record.new_from_marc(marc_display) >> > puts record.to_s >> > >> > My output for that script looks like this: >> > >> > ************* from file ****************** >> > LEADER 00799cam a2200241 a 4500 >> > 001 ? ?00282214 >> > 003 DLC >> > 005 20090120022042.0 >> > 008 000417s1998 ? ?pk ? ? ? ? ? ?000 0 urdo >> > 010 ? ?$a ? ?00282214 >> > 025 ? ?$a P-U-00282214; 05; 06 >> > 040 ? ?$a DLC $c DLC $d DLC >> > 041 1 ?$a urd $h snd >> > 042 ? ?$a lcode >> > 050 00 $a PK2788.9.A9 $b F55 1998 >> > 100 1 ?$a Ayaz, Shaikh, $d 1923-1997. >> > 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, >> > Sha?h Muh?ammad Pi?rza?dah. >> > 260 ? ?$a Kara?ci? : $b Da?niya?l, $c [1998] >> > 300 ? ?$a 375 p. ; $c 23 cm. >> > 546 ? ?$a In Urdu. >> > 520 ? ?$a Selected poems and articles from the works of renowned >> > Sindhi poet; chiefly translated from Sindhi. >> > 700 1 ?$a Farruk?h?i?, A?s?if, $d 1959- >> > 700 1 ?$a Pi?rza?dah, Sha?h Muh?ammad. >> > ************* from solr ****************** >> > LEADER 00799cam a2200241 a 4500 >> > 001 ? ?00282214 * >> > 003 DLC* >> > 005 20090120022042.0* >> > 008 000417s1998 ? ?pk ? ? ? ? ? ?000 0 urdo * >> > >> > >> > So, when I take a file (test_data.utf.mrc) and read in the marc >> > records from disk, it behaves as expected. But if I store it in solr >> > first, then what I get back out of solr appears to be badly formed >> > marc somehow. MARC::Record appears to read it in, but the record >> > created is truncated. >> > >> > Naomi, how do you folks store marc21? Do you do anything special to >> > it before putting it in the index? It occurred to me we could base64 >> > encode it, but maybe there's something simpler that I'm missing? >> > >> > I'm storing the marc record at around line 96 of marc_mapper.rb: >> > >> > ? # _display is stored, but not indexed >> > ? # don't store a string, store marc21 so we can read it back out >> > ? # into a MARC::Record object >> > ? map :marc_display do |rec,index| >> > ? ? rec.to_marc >> > ? end >> > >> > Thanks in advance for any advice, >> > >> > Bess >> > _______________________________________________ >> > Blacklight-development mailing list >> > Blacklight-development at rubyforge.org >> > http://rubyforge.org/mailman/listinfo/blacklight-development >> > Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > > -- > Bill Dueber > Library Systems Programmer > University of Michigan Library > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From rochkind at jhu.edu Mon Mar 23 10:50:32 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 23 Mar 2009 10:50:32 -0400 Subject: [Blacklight-development] including accordion effect for facets In-Reply-To: <89FE55E0-8478-4B3B-92B3-433793EDA3D0@virginia.edu> References: <1e4bf2cc0903201827g36693a8mb19261fd8868952a@mail.gmail.com> <89FE55E0-8478-4B3B-92B3-433793EDA3D0@virginia.edu> Message-ID: <49C7A1B8.1040002@jhu.edu> I'd rather have it included in the plugin, with an easy way to turn it on or off. But you know my opinion on what to include in the plugin tends to run to 'everything'. If lots of people are going to want it, and it's in the demo, then everyone winds up with their own seperate 'forked' copy. When in the future enhancements or bugfixes are made to it in one location, there are going to be difficulties sharing that code efficiently -- the whole reason we have the plugin in the first place. Jonathan Bess Sadler wrote: > I'm in favor of including this, particularly since we won't have much > nice styling in the 2.0 release, and the accordion facets are one > styling feature that people ask about a lot and that have done really > well in usability testing. I can see an argument for including it in > the plugin and for including it in the demo app, but I think I'd > prefer to see it included in the demo app, where it can also serve as > an example of how to overload behavior with local customization. > > Other thoughts? > > Bess > > On 20-Mar-09, at 9:27 PM, Tony Zanella wrote: > > >> Hello all, >> Some folks have requested an accordion effect be included in the >> Blacklight demo. I decided to take a swing at writing one. While I was >> researching, I found that there are a number of different effects that >> go by the name "accordion". Since I wanted to keep swinging, I wrote a >> bare-bones javascript I think could be expanded or customized pretty >> easily. With the latest version of the demo, I added the javascript to >> >> /bl-demo/rails/public/plugin_assets/blacklight/javascripts >> >> and I linked it into the interface at >> >> /bl-demo/rails/vendor/plugins/blacklight/app/views/layouts >> >> and everything worked just fine. >> >> The question is: where should the addition and linkage really take >> place, assuming that it's a good idea to include this functionality in >> the first place? I guess the other option is to stay out of the >> plugin. >> >> If we were to stay out of the plugin, wouldn't it make sense to >> include a layout in, say, /bl-demo/rails/app/views ? and include the >> javascript in, say, /bl-demo/rails/public/javascripts ? >> >> Thanks! >> Tony Zanella >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From eos8d at virginia.edu Mon Mar 23 11:48:33 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Mon, 23 Mar 2009 11:48:33 -0400 Subject: [Blacklight-development] including accordion effect for facets In-Reply-To: <49C7A1B8.1040002@jhu.edu> References: <1e4bf2cc0903201827g36693a8mb19261fd8868952a@mail.gmail.com> <89FE55E0-8478-4B3B-92B3-433793EDA3D0@virginia.edu> <49C7A1B8.1040002@jhu.edu> Message-ID: Hey, Tony. Let's go ahead and commit this into the plugin, which is where you have it already, I believe. Could you commit it today? I want to stick to our March 25 release date if we can, so I'd like us to have a couple of days to look at the code after you commit and before we release. Thanks! Bess On Mar 23, 2009, at 10:50 AM, Jonathan Rochkind wrote: > I'd rather have it included in the plugin, with an easy way to turn it > on or off. But you know my opinion on what to include in the plugin > tends to run to 'everything'. > > If lots of people are going to want it, and it's in the demo, then > everyone winds up with their own seperate 'forked' copy. When in the > future enhancements or bugfixes are made to it in one location, there > are going to be difficulties sharing that code efficiently -- the > whole > reason we have the plugin in the first place. > > Jonathan > > Bess Sadler wrote: >> I'm in favor of including this, particularly since we won't have much >> nice styling in the 2.0 release, and the accordion facets are one >> styling feature that people ask about a lot and that have done really >> well in usability testing. I can see an argument for including it in >> the plugin and for including it in the demo app, but I think I'd >> prefer to see it included in the demo app, where it can also serve as >> an example of how to overload behavior with local customization. >> >> Other thoughts? >> >> Bess >> >> On 20-Mar-09, at 9:27 PM, Tony Zanella wrote: >> >> >>> Hello all, >>> Some folks have requested an accordion effect be included in the >>> Blacklight demo. I decided to take a swing at writing one. While I >>> was >>> researching, I found that there are a number of different effects >>> that >>> go by the name "accordion". Since I wanted to keep swinging, I >>> wrote a >>> bare-bones javascript I think could be expanded or customized pretty >>> easily. With the latest version of the demo, I added the >>> javascript to >>> >>> /bl-demo/rails/public/plugin_assets/blacklight/javascripts >>> >>> and I linked it into the interface at >>> >>> /bl-demo/rails/vendor/plugins/blacklight/app/views/layouts >>> >>> and everything worked just fine. >>> >>> The question is: where should the addition and linkage really take >>> place, assuming that it's a good idea to include this >>> functionality in >>> the first place? I guess the other option is to stay out of the >>> plugin. >>> >>> If we were to stay out of the plugin, wouldn't it make sense to >>> include a layout in, say, /bl-demo/rails/app/views ? and include the >>> javascript in, say, /bl-demo/rails/public/javascripts ? >>> >>> Thanks! >>> Tony Zanella >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From jamie at dang.com Mon Mar 23 12:07:51 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Mon, 23 Mar 2009 12:07:51 -0400 Subject: [Blacklight-development] re-writing some tests with mocks In-Reply-To: References: <1ECE0D1F-182D-4951-8569-052A5B923EE4@virginia.edu> Message-ID: <2346C4B2-6A6C-48D9-A696-FE6E388F6AF4@dang.com> So then the question is, what is the best way for us to test if solr_helper is creating valid queries? Would it be simple, viable, etc to test the validity of the queries without connecting to a solr server? On Mar 21, 2009, at 8:34 PM, Bess Sadler wrote: > After talking to Naomi I realized that I had to roll back these > mocked tests. With the mocks in place, we would only be testing > whether the solr_helper could properly handle responses from rsolr, > but we would not be testing whether solr_helper knew how to create > valid queries. I've rolled back my changes. We're going to need that > local solr instance to test against after all. > > Hey, at least I learned a lot about rspec and mocks today! :) > > Bess > > On 21-Mar-09, at 6:19 PM, Bess Sadler wrote: > >> Okay, I made a first pass at this. I also commented out some tests >> that were failing, because I wanted to start with a 100% passing test >> suite before I made any major changes. I'll re-write those failing >> tests and add them back as I'm able. All of the tests I've changed so >> far are in solr_helper_spec.rb. It was running several queries, e.g., >> all_docs_query, no_docs_query, single_word_query, mult_word_query. I >> ran each of these against an index with all of our demo records in >> it, >> grabbed the response and put it in a mock method in mocks.rb, like >> this: >> >> def mock_no_docs_query >> %({"response"=> >> {"maxScore"=>0.0, "start"=>0, "docs"=>[], "numFound"=>0}, >> "facet_counts"=> >> { "facet_fields"=> >> { "subject_era_facet"=>[], >> "language_facet"=>[], >> "format_facet"=>[], >> "geographic_subject_facet"=>[]}, >> "facet_dates"=>{}, >> "facet_queries"=>{}}, >> "responseHeader"=> >> {"QTime"=>1, "params"=> >> {"qt"=>"search", "q"=>"zzzzzzzzzzzz", "wt"=>"ruby", >> "rows"=>"10"}, "status"=>0}}) >> end >> >> Then in the test, I create a RSolr::Ext::Response::Standard object, >> which is what the test is expecting, and populate it with that mocked >> response, like this: >> >> # use the mock response in mocks.rb >> raw_response = eval(mock_query_response) >> solr_response = RSolr::Ext::Response::Standard.new(raw_response) >> >> The rest of the tests remain the same. The only thing that has >> changed >> is that the response is coming from a text file instead of from solr. >> I left the old lines in place, but commented out and labeled, in case >> anyone wants to run against a live version of solr. >> >> Matt, I totally copied this technique from your rsolr-ext tests. >> Thank >> you! >> >> Bess >> >> >> On 21-Mar-09, at 4:20 PM, Bess Sadler wrote: >> >>> Hi, folks. >>> >>> In keeping with good rspec practice, I'm going to try to re-write >>> our >>> rspec tests not to depend on a live solr instance to be running. >>> Instead, I'm going to mock up the responses they would expect to get >>> from solr, and include those in the rspec suite. This will offer a >>> few >>> advantages, but the number one reason is that we don't want all our >>> tests to break when you point your Blacklight app at a new index >>> that >>> doesn't contain our sample records. >>> >>> I'm going to try this, and hope my rspec skills are up to the >>> challenge. If there are any objections, or if I'm missing some >>> reason >>> why I shouldn't do this, let me know. >>> >>> Bess >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From eos8d at virginia.edu Mon Mar 23 12:12:22 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Mon, 23 Mar 2009 12:12:22 -0400 Subject: [Blacklight-development] storing marc21 in solr In-Reply-To: <23b83f160903230739h7d8961adweeb28fddecf4d85@mail.gmail.com> References: <9904BED0-6123-4AFB-B6AC-C447541FB1F2@Virginia.EDU> <90FF863A96E1EC42B8B240D04C88FB1D73406AB8@JHEMTEXVS2.win.ad.jhu.edu> <23b83f160903230739h7d8961adweeb28fddecf4d85@mail.gmail.com> Message-ID: <10039AA7-1349-415C-B60D-279C62F692E9@virginia.edu> Hi, folks. Here are some of my thoughts on this topic. In summary: When I started this conversation I thought we needed to standardize on marc21, but now I think we can just let people use either one. Some goals of the project: - we want to access marc fields that aren't explicitly indexed - we want to be able to display the full record in a human-readable way - we want to provide RESTful access to our records for easy mash-up- ability (e.g., in xml, json, or mrc format) If we accept that those are among our goals, then we have to store the full marc record somewhere. And if we accept that those goals are going to be met by the plugin (i.e., if these are core functionality instead of an individual institution's job to implement) then we have to pick a way to write that core functionality. I want to be able to access marc as marc (I don't want to just parse marc-xml as if it were any other flavor of xml, and I don't want to parse a marc string) and I especially don't want to write any methods that act upon marc more than once (e.g., "Act this way if the stored marc record is xml, and act this other way if it's marc21.) I could, however, see core behavior where the CatalogController grabs whatever is in the stored_marc_display field, tests to see whether it's xml or marc21, and creates a MARC::Record object of it, and then all marc behaviors just act on that MARC::Record object. I really want to be writing code for MARC::Record objects as opposed to writing xml- or string-parsing code. My main motivation for thinking we had to store marc21 was that I thought that ruby-marc could only create MARC::Record objects from marc21, but thanks to Ross's email I have now discovered MARC::XMLReader. So I think I should now be able to go off and write the behaviors I described above, as well as tests for the validity of stored marc21 and marc-xml. Thank you, everyone. Bess On Mar 23, 2009, at 10:39 AM, Ross Singer wrote: > I realize I'm coming into this a little late -- not sure if Bess's > other thread about Base64 encoding this "solved it". > > I agree with Naomi that we don't need to standardize on anything, but, > obviously, the more people that do it in a certain way helps provide > some traction. > > I would argue that storing as a marc21 string has the bigger upside > for now. ruby-marc can only use REXML as its parser presently > (although Ed and I had a conversation just last week about refactoring > it to support Nokogiri, as well), so the binary marc reader (and, if > needed, writer) is way faster. Even accounting for Base64 > encoding/decoding. It would also keep the index a whole lot smaller, > if anybody cares about that. > > BTW, the reason why you're seeing the line breaks in Bess's script is > because .to_s pretty prints the record. If she called .to_marc it > would look like Naomi's mushy blob of text. > > -Ross. > > 2009/3/23 Naomi Dushay : >> Maybe it's too soon to worry about this. Is there any pressing >> need for all >> of us to store our marc data in our indexes in the same format? >> What's the >> real gain? Why should the blacklight code be rigid about this? >> Remember, a >> lot of our solr documents won't have marc. >> What if there was a blacklight config setting to say what format >> your marc >> was in, and the blacklight code accommodated the different formats? >> Jessie is already working on manipulating the marc21 in the >> Stanford index >> to create the UI features we want at Stanford. UVa has been using >> marcxml. >> Jessie should jump in and tell us if the work he's doing can be >> easily >> adapted to other marc formats -- hopefully the answer is "yes, it >> would be >> trivial." But let's get those features coded first, and refactor >> them >> later. >> I think we're doing great things here. UVa has put out a great >> body of >> code. The active Stanford adoption of the code is causing an >> excellent >> refactoring of the code, making it more configurable; additional >> features >> will be added as well. Further active adopters or folks that get >> down and >> dirty with the demo will provide more feedback, causing further >> improvements >> - refactoring or new features or whatever. >> The demo for release 2.0 is going to have some warts; I don't >> think there's >> a way around it. Usage driven methodology would say we have some >> folks >> take the demo out for a spin, and get their feedback. Have folks >> tried the >> current demo? Have they tried to go the next step, of getting >> their own >> data behind the blacklight front end? If not ... maybe we've >> gotten ahead >> of ourselves? >> Trying to guess a lot of this stuff before there's expressed demand >> from the >> users ... I think that can make extra work, and the results might >> not be >> useful, in the end. >> Okay, I've got my asbestos suit on - let the flames be thrown! >> - Naomi >> >> On Mar 22, 2009, at 6:03 PM, Bill Dueber wrote: >> >> solrmarc writes to the underlying lucene index using a binary field >> type >> that is unavailable in solr. This makes for easy storage of MARC, >> but makes >> round-tripping the data a little harder if that's your thing. Given >> that's >> the case, I wonder if a standard solr String field (probably >> compressed) of >> marcxml would be a better bet all around. >> >> Or push dchud into finishing his spec implementation of marc-json >> and store >> it as json :-) >> >> On Sun, Mar 22, 2009 at 4:14 PM, Jonathan Rochkind >> wrote: >>> >>> It makes sense to me to standardize 'what kind of MARC' we all >>> store in >>> SOLR. I guess you guys decided it made more sense for that to be >>> MARC21 >>> than MARC-XML? Curious what the motivation for that was? >>> >>> Since MARC21 is a binary format, you've got to make sure that what >>> is >>> going into the interface is binary identical to what should be, it >>> should >>> not be re-encoded or translated at all. MARC21 data can be in one >>> of at >>> least two character encodings, which one is used is theoretically >>> mentioned >>> in the MARC file (although it's often wrong). MARC21 file length >>> (in >>> bytes) is also noted in the MARC file header/leader. The whole >>> thing needs >>> to be byte-for-byte untouched when you stick it in your SOLR >>> index, I don't >>> know enough about SOLR or the indexing process to know about >>> places that >>> might introduce corruption there. >>> >>> MARC-XML is definitely a lot easier to work with, as it's just an >>> ordinary >>> XML file in UTF-8. >>> >>> I don't know a whole lot about MARC21, but it would not surprise >>> me at >>> all, if, like the charcter encoding issue, much of our MARC21 "in >>> the wild" >>> was actually illegal in undocumented various ways, but ways that our >>> traditional ILS's are tolerant of. >>> >>> It's also possible there is a bug in the ruby MARC library. >>> >>> Jonathan >>> >>> ________________________________________ >>> From: blacklight-development-bounces at rubyforge.org >>> [blacklight-development-bounces at rubyforge.org] On Behalf Of Naomi >>> Dushay >>> [ndushay at stanford.edu] >>> Sent: Sunday, March 22, 2009 1:47 PM >>> To: blacklight-development at rubyforge.org >>> Subject: Re: [Blacklight-development] storing marc21 in solr >>> >>> Bess, >>> >>> I create our index using solrmarc, which writes directly to the >>> index >>> (do not use solr, do not pass go, do not cellect $200.). So I >>> can't >>> be much help, sorry. >>> >>> However, I'll tell you that our marc21 doesn't look all nicely >>> formatted like yours - it just appears as a single line and looks >>> like >>> this: >>> >>> 00651nam a22001575? >>> >>> 4500001000800000008004100008020001500049035002200064100001900086245006900105260003300174300001100207440005400218596000600272999021500278 >>> a575946 910118s1990||||gw |||||||||||||||||ger|d a3412215880 >>> a(CSt)notisACX2206 10 aSeifert, Arno. 14 aDer Ruckzug der biblischen >>> Prophetie von der neueren Geschichte. 0 aKoln : bBohlau >>> Verlag, c1990 a207 p. 0 aBeihefte zum Archiv f?r >>> Kulturgeschichte ; vv.31 a1 aCB3 .A6 SUPPL. V. >>> 31 >>> >>> wLC >>> >>> c1 >>> i36105035087092 d6/18/2008 e6/18/2008 kCHECKEDOUT lSTACKS mGREEN >>> n4 p >>> $40.00 rM sY tSTKS-MONO u1/18/1991 o.TECHSTAFF. c:al o.TECHSTAFF. >>> PUB >>> 1/22/91/pb o.TECHSTAFF. MIDSPINE.v.31 suppl. >>> >>> Maybe you don't have "marc21" but something else? >>> >>> - Naomi >>> >>> On Mar 22, 2009, at 7:40 AM, Bess Sadler wrote: >>> >>>> Hey, folks. >>>> >>>> Hopefully I'm just having a moment of stupidity, but I've been >>>> looking at this for awhile and I can't figure it out, so I'm asking >>>> the community. We want to store marc21 in our solr index, right? >>>> UVA >>>> has always stored marc-xml, but we had a talk with Stanford folks >>>> awhile ago about this, there seem to be lots of good reasons to >>>> store marc21 instead, and that's fine with me, so now I'm just >>>> trying to make it happen. >>>> >>>> BUT... when I store what I think is marc21 into solr, it doesn't >>>> come back out again quite right. I've isolated a really simple >>>> example of this: >>>> >>>> require 'rubygems' >>>> require 'marc' >>>> require 'rsolr' >>>> >>>> puts " ************* from file ****************** " >>>> >>>> reader = MARC::Reader.new('/usr/local/projects/bl-demo/data/ >>>> test_data.utf8.mrc') >>>> record2 = reader.first >>>> puts record2.to_s >>>> >>>> puts " ************* from solr ****************** " >>>> >>>> rsolr = RSolr.connect >>>> response = eval(rsolr.select(:q=>'*:*',:wt=>'ruby')) >>>> marc_display = response["response"]["docs"][0]["marc_display"] >>>> record = MARC::Record.new_from_marc(marc_display) >>>> puts record.to_s >>>> >>>> My output for that script looks like this: >>>> >>>> ************* from file ****************** >>>> LEADER 00799cam a2200241 a 4500 >>>> 001 00282214 >>>> 003 DLC >>>> 005 20090120022042.0 >>>> 008 000417s1998 pk 000 0 urdo >>>> 010 $a 00282214 >>>> 025 $a P-U-00282214; 05; 06 >>>> 040 $a DLC $c DLC $d DLC >>>> 041 1 $a urd $h snd >>>> 042 $a lcode >>>> 050 00 $a PK2788.9.A9 $b F55 1998 >>>> 100 1 $a Ayaz, Shaikh, $d 1923-1997. >>>> 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, >>>> Sha?h Muh?ammad Pi?rza?dah. >>>> 260 $a Kara?ci? : $b Da?niya?l, $c [1998] >>>> 300 $a 375 p. ; $c 23 cm. >>>> 546 $a In Urdu. >>>> 520 $a Selected poems and articles from the works of renowned >>>> Sindhi poet; chiefly translated from Sindhi. >>>> 700 1 $a Farruk?h?i?, A?s?if, $d 1959- >>>> 700 1 $a Pi?rza?dah, Sha?h Muh?ammad. >>>> ************* from solr ****************** >>>> LEADER 00799cam a2200241 a 4500 >>>> 001 00282214 * >>>> 003 DLC* >>>> 005 20090120022042.0* >>>> 008 000417s1998 pk 000 0 urdo * >>>> >>>> >>>> So, when I take a file (test_data.utf.mrc) and read in the marc >>>> records from disk, it behaves as expected. But if I store it in >>>> solr >>>> first, then what I get back out of solr appears to be badly formed >>>> marc somehow. MARC::Record appears to read it in, but the record >>>> created is truncated. >>>> >>>> Naomi, how do you folks store marc21? Do you do anything special to >>>> it before putting it in the index? It occurred to me we could >>>> base64 >>>> encode it, but maybe there's something simpler that I'm missing? >>>> >>>> I'm storing the marc record at around line 96 of marc_mapper.rb: >>>> >>>> # _display is stored, but not indexed >>>> # don't store a string, store marc21 so we can read it back out >>>> # into a MARC::Record object >>>> map :marc_display do |rec,index| >>>> rec.to_marc >>>> end >>>> >>>> Thanks in advance for any advice, >>>> >>>> Bess >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> >> -- >> Bill Dueber >> Library Systems Programmer >> University of Michigan Library >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From eos8d at virginia.edu Mon Mar 23 12:22:01 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Mon, 23 Mar 2009 12:22:01 -0400 Subject: [Blacklight-development] (not) re-writing some tests with mocks In-Reply-To: <2346C4B2-6A6C-48D9-A696-FE6E388F6AF4@dang.com> References: <1ECE0D1F-182D-4951-8569-052A5B923EE4@virginia.edu> <2346C4B2-6A6C-48D9-A696-FE6E388F6AF4@dang.com> Message-ID: I've re-written all the tests now so that they only require a very small number of records be in the index, and all of those records are contained in test_data.utf8.mrc, which can index in only a few seconds. How would people feel about just requiring that those test records be indexed into the solr we distribute with the demo app in order to run the tests? I think it's pretty reasonable to require that the instance of solr we distribute is a test-only instance, and that when people want to start running for-real against their own data they need to set up a separate copy of solr for that. I know I was the one pushing mocks earlier, but I think Naomi has a really good point... we don't know if solr_helper is working properly unless we're actually running the queries it generates against an actual solr index. Bess On Mar 23, 2009, at 12:07 PM, Jamie Orchard-Hays wrote: > So then the question is, what is the best way for us to test if > solr_helper is creating valid queries? Would it be simple, viable, etc > to test the validity of the queries without connecting to a solr > server? > > > On Mar 21, 2009, at 8:34 PM, Bess Sadler wrote: > >> After talking to Naomi I realized that I had to roll back these >> mocked tests. With the mocks in place, we would only be testing >> whether the solr_helper could properly handle responses from rsolr, >> but we would not be testing whether solr_helper knew how to create >> valid queries. I've rolled back my changes. We're going to need that >> local solr instance to test against after all. >> >> Hey, at least I learned a lot about rspec and mocks today! :) >> >> Bess >> >> On 21-Mar-09, at 6:19 PM, Bess Sadler wrote: >> >>> Okay, I made a first pass at this. I also commented out some tests >>> that were failing, because I wanted to start with a 100% passing >>> test >>> suite before I made any major changes. I'll re-write those failing >>> tests and add them back as I'm able. All of the tests I've changed >>> so >>> far are in solr_helper_spec.rb. It was running several queries, >>> e.g., >>> all_docs_query, no_docs_query, single_word_query, mult_word_query. I >>> ran each of these against an index with all of our demo records in >>> it, >>> grabbed the response and put it in a mock method in mocks.rb, like >>> this: >>> >>> def mock_no_docs_query >>> %({"response"=> >>> {"maxScore"=>0.0, "start"=>0, "docs"=>[], "numFound"=>0}, >>> "facet_counts"=> >>> { "facet_fields"=> >>> { "subject_era_facet"=>[], >>> "language_facet"=>[], >>> "format_facet"=>[], >>> "geographic_subject_facet"=>[]}, >>> "facet_dates"=>{}, >>> "facet_queries"=>{}}, >>> "responseHeader"=> >>> {"QTime"=>1, "params"=> >>> {"qt"=>"search", "q"=>"zzzzzzzzzzzz", "wt"=>"ruby", >>> "rows"=>"10"}, "status"=>0}}) >>> end >>> >>> Then in the test, I create a RSolr::Ext::Response::Standard object, >>> which is what the test is expecting, and populate it with that >>> mocked >>> response, like this: >>> >>> # use the mock response in mocks.rb >>> raw_response = eval(mock_query_response) >>> solr_response = RSolr::Ext::Response::Standard.new(raw_response) >>> >>> The rest of the tests remain the same. The only thing that has >>> changed >>> is that the response is coming from a text file instead of from >>> solr. >>> I left the old lines in place, but commented out and labeled, in >>> case >>> anyone wants to run against a live version of solr. >>> >>> Matt, I totally copied this technique from your rsolr-ext tests. >>> Thank >>> you! >>> >>> Bess >>> >>> >>> On 21-Mar-09, at 4:20 PM, Bess Sadler wrote: >>> >>>> Hi, folks. >>>> >>>> In keeping with good rspec practice, I'm going to try to re-write >>>> our >>>> rspec tests not to depend on a live solr instance to be running. >>>> Instead, I'm going to mock up the responses they would expect to >>>> get >>>> from solr, and include those in the rspec suite. This will offer a >>>> few >>>> advantages, but the number one reason is that we don't want all our >>>> tests to break when you point your Blacklight app at a new index >>>> that >>>> doesn't contain our sample records. >>>> >>>> I'm going to try this, and hope my rspec skills are up to the >>>> challenge. If there are any objections, or if I'm missing some >>>> reason >>>> why I shouldn't do this, let me know. >>>> >>>> Bess >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From tony.zanella at gmail.com Mon Mar 23 12:29:43 2009 From: tony.zanella at gmail.com (Tony Zanella) Date: Mon, 23 Mar 2009 12:29:43 -0400 Subject: [Blacklight-development] including accordion effect for facets In-Reply-To: References: <1e4bf2cc0903201827g36693a8mb19261fd8868952a@mail.gmail.com> <89FE55E0-8478-4B3B-92B3-433793EDA3D0@virginia.edu> <49C7A1B8.1040002@jhu.edu> Message-ID: <1e4bf2cc0903230929w3778c3e5x6b67feb72f9436a3@mail.gmail.com> Hi Bess, I can commit what I have tonight. I think Matt's suggestion to have the menu maintain its state is vital. That feature isn't there yet, but Matt sent me a suggestion today that I think should work. I'll try to get it in there tonight, too. Tony On Mon, Mar 23, 2009 at 11:48 AM, Bess Sadler wrote: > Hey, Tony. Let's go ahead and commit this into the plugin, which is where > you have it already, I believe. Could you commit it today? I want to stick > to our March 25 release date if we can, so I'd like us to have a couple of > days to look at the code after you commit and before we release. > > Thanks! > > Bess > > On Mar 23, 2009, at 10:50 AM, Jonathan Rochkind wrote: > >> I'd rather have it included in the plugin, with an easy way to turn it >> on or off. ?But you know my opinion on what to include in the plugin >> tends to run to 'everything'. >> >> If lots of people are going to want it, and it's in the demo, then >> everyone winds up with their own seperate 'forked' copy. When in the >> future enhancements or bugfixes are made to it in one location, there >> are going to be difficulties sharing that code efficiently -- the whole >> reason we have the plugin in the first place. >> >> Jonathan >> >> Bess Sadler wrote: >>> >>> I'm in favor of including this, particularly since we won't have much >>> nice styling in the 2.0 release, and the accordion facets are one >>> styling feature that people ask about a lot and that have done really >>> well in usability testing. I can see an argument for including it in >>> the plugin and for including it in the demo app, but I think I'd >>> prefer to see it included in the demo app, where it can also serve as >>> an example of how to overload behavior with local customization. >>> >>> Other thoughts? >>> >>> Bess >>> >>> On 20-Mar-09, at 9:27 PM, Tony Zanella wrote: >>> >>> >>>> Hello all, >>>> Some folks have requested an accordion effect be included in the >>>> Blacklight demo. I decided to take a swing at writing one. While I was >>>> researching, I found that there are a number of different effects that >>>> go by the name "accordion". Since I wanted to keep swinging, I wrote a >>>> bare-bones javascript I think could be expanded or customized pretty >>>> easily. With the latest version of the demo, I added the javascript to >>>> >>>> /bl-demo/rails/public/plugin_assets/blacklight/javascripts >>>> >>>> and I linked it into the interface at >>>> >>>> /bl-demo/rails/vendor/plugins/blacklight/app/views/layouts >>>> >>>> and everything worked just fine. >>>> >>>> The question is: where should the addition and linkage really take >>>> place, assuming that it's a good idea to include this functionality in >>>> the first place? I guess the other option is to stay out of the >>>> plugin. >>>> >>>> If we were to stay out of the plugin, wouldn't it make sense to >>>> include a layout in, say, /bl-demo/rails/app/views ? and include the >>>> javascript in, say, /bl-demo/rails/public/javascripts ? >>>> >>>> Thanks! >>>> Tony Zanella >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From rochkind at jhu.edu Mon Mar 23 12:39:33 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 23 Mar 2009 12:39:33 -0400 Subject: [Blacklight-development] (not) re-writing some tests with mocks In-Reply-To: References: <1ECE0D1F-182D-4951-8569-052A5B923EE4@virginia.edu> <2346C4B2-6A6C-48D9-A696-FE6E388F6AF4@dang.com> Message-ID: <49C7BB45.8080400@jhu.edu> The pattern used by Rails tests is to have a test database, and to actually load the record set needed for the tests into it prior to running the tests. And then point the tests at that test database. Is there any easy/convenient way to do that with SOLR and rspec? Actually have rspec load those records into a SOLR index set up just for testing, prior to running the tests? Jonathan Bess Sadler wrote: > I've re-written all the tests now so that they only require a very > small number of records be in the index, and all of those records are > contained in test_data.utf8.mrc, which can index in only a few > seconds. How would people feel about just requiring that those test > records be indexed into the solr we distribute with the demo app in > order to run the tests? I think it's pretty reasonable to require that > the instance of solr we distribute is a test-only instance, and that > when people want to start running for-real against their own data they > need to set up a separate copy of solr for that. I know I was the one > pushing mocks earlier, but I think Naomi has a really good point... we > don't know if solr_helper is working properly unless we're actually > running the queries it generates against an actual solr index. > > Bess > > On Mar 23, 2009, at 12:07 PM, Jamie Orchard-Hays wrote: > > >> So then the question is, what is the best way for us to test if >> solr_helper is creating valid queries? Would it be simple, viable, etc >> to test the validity of the queries without connecting to a solr >> server? >> >> >> On Mar 21, 2009, at 8:34 PM, Bess Sadler wrote: >> >> >>> After talking to Naomi I realized that I had to roll back these >>> mocked tests. With the mocks in place, we would only be testing >>> whether the solr_helper could properly handle responses from rsolr, >>> but we would not be testing whether solr_helper knew how to create >>> valid queries. I've rolled back my changes. We're going to need that >>> local solr instance to test against after all. >>> >>> Hey, at least I learned a lot about rspec and mocks today! :) >>> >>> Bess >>> >>> On 21-Mar-09, at 6:19 PM, Bess Sadler wrote: >>> >>> >>>> Okay, I made a first pass at this. I also commented out some tests >>>> that were failing, because I wanted to start with a 100% passing >>>> test >>>> suite before I made any major changes. I'll re-write those failing >>>> tests and add them back as I'm able. All of the tests I've changed >>>> so >>>> far are in solr_helper_spec.rb. It was running several queries, >>>> e.g., >>>> all_docs_query, no_docs_query, single_word_query, mult_word_query. I >>>> ran each of these against an index with all of our demo records in >>>> it, >>>> grabbed the response and put it in a mock method in mocks.rb, like >>>> this: >>>> >>>> def mock_no_docs_query >>>> %({"response"=> >>>> {"maxScore"=>0.0, "start"=>0, "docs"=>[], "numFound"=>0}, >>>> "facet_counts"=> >>>> { "facet_fields"=> >>>> { "subject_era_facet"=>[], >>>> "language_facet"=>[], >>>> "format_facet"=>[], >>>> "geographic_subject_facet"=>[]}, >>>> "facet_dates"=>{}, >>>> "facet_queries"=>{}}, >>>> "responseHeader"=> >>>> {"QTime"=>1, "params"=> >>>> {"qt"=>"search", "q"=>"zzzzzzzzzzzz", "wt"=>"ruby", >>>> "rows"=>"10"}, "status"=>0}}) >>>> end >>>> >>>> Then in the test, I create a RSolr::Ext::Response::Standard object, >>>> which is what the test is expecting, and populate it with that >>>> mocked >>>> response, like this: >>>> >>>> # use the mock response in mocks.rb >>>> raw_response = eval(mock_query_response) >>>> solr_response = RSolr::Ext::Response::Standard.new(raw_response) >>>> >>>> The rest of the tests remain the same. The only thing that has >>>> changed >>>> is that the response is coming from a text file instead of from >>>> solr. >>>> I left the old lines in place, but commented out and labeled, in >>>> case >>>> anyone wants to run against a live version of solr. >>>> >>>> Matt, I totally copied this technique from your rsolr-ext tests. >>>> Thank >>>> you! >>>> >>>> Bess >>>> >>>> >>>> On 21-Mar-09, at 4:20 PM, Bess Sadler wrote: >>>> >>>> >>>>> Hi, folks. >>>>> >>>>> In keeping with good rspec practice, I'm going to try to re-write >>>>> our >>>>> rspec tests not to depend on a live solr instance to be running. >>>>> Instead, I'm going to mock up the responses they would expect to >>>>> get >>>>> from solr, and include those in the rspec suite. This will offer a >>>>> few >>>>> advantages, but the number one reason is that we don't want all our >>>>> tests to break when you point your Blacklight app at a new index >>>>> that >>>>> doesn't contain our sample records. >>>>> >>>>> I'm going to try this, and hope my rspec skills are up to the >>>>> challenge. If there are any objections, or if I'm missing some >>>>> reason >>>>> why I shouldn't do this, let me know. >>>>> >>>>> Bess >>>>> _______________________________________________ >>>>> Blacklight-development mailing list >>>>> Blacklight-development at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From rochkind at jhu.edu Mon Mar 23 12:43:10 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 23 Mar 2009 12:43:10 -0400 Subject: [Blacklight-development] storing marc21 in solr In-Reply-To: <10039AA7-1349-415C-B60D-279C62F692E9@virginia.edu> References: <9904BED0-6123-4AFB-B6AC-C447541FB1F2@Virginia.EDU> <90FF863A96E1EC42B8B240D04C88FB1D73406AB8@JHEMTEXVS2.win.ad.jhu.edu> <23b83f160903230739h7d8961adweeb28fddecf4d85@mail.gmail.com> <10039AA7-1349-415C-B60D-279C62F692E9@virginia.edu> Message-ID: <49C7BC1E.1060802@jhu.edu> Makes sense. I especially like your "RESTful access to MARC" goal, I'm going to need that to. There are definitely going to be some shared plugin functionalities that depend on MARC, I think (for records that HAVE marc, understood). Perhaps there's some easy way to actually record whether the thing stored is MARC-XML or marc21, instead of the plugin having to guess from the data? Either way, the plugin is going to read into a ruby marc document, so your methods won't have to be different for different storage, right? Just a question of figuring out whether you want to parse a marc21, or parse a MARC-XML. Jonathan Bess Sadler wrote: > Hi, folks. Here are some of my thoughts on this topic. In summary: > When I started this conversation I thought we needed to standardize on > marc21, but now I think we can just let people use either one. > > Some goals of the project: > - we want to access marc fields that aren't explicitly indexed > - we want to be able to display the full record in a human-readable way > - we want to provide RESTful access to our records for easy mash-up- > ability (e.g., in xml, json, or mrc format) > > If we accept that those are among our goals, then we have to store the > full marc record somewhere. And if we accept that those goals are > going to be met by the plugin (i.e., if these are core functionality > instead of an individual institution's job to implement) then we have > to pick a way to write that core functionality. I want to be able to > access marc as marc (I don't want to just parse marc-xml as if it were > any other flavor of xml, and I don't want to parse a marc string) and > I especially don't want to write any methods that act upon marc more > than once (e.g., "Act this way if the stored marc record is xml, and > act this other way if it's marc21.) > > I could, however, see core behavior where the CatalogController grabs > whatever is in the stored_marc_display field, tests to see whether > it's xml or marc21, and creates a MARC::Record object of it, and then > all marc behaviors just act on that MARC::Record object. I really want > to be writing code for MARC::Record objects as opposed to writing xml- > or string-parsing code. My main motivation for thinking we had to > store marc21 was that I thought that ruby-marc could only create > MARC::Record objects from marc21, but thanks to Ross's email I have > now discovered MARC::XMLReader. > > So I think I should now be able to go off and write the behaviors I > described above, as well as tests for the validity of stored marc21 > and marc-xml. Thank you, everyone. > > Bess > > > On Mar 23, 2009, at 10:39 AM, Ross Singer wrote: > > >> I realize I'm coming into this a little late -- not sure if Bess's >> other thread about Base64 encoding this "solved it". >> >> I agree with Naomi that we don't need to standardize on anything, but, >> obviously, the more people that do it in a certain way helps provide >> some traction. >> >> I would argue that storing as a marc21 string has the bigger upside >> for now. ruby-marc can only use REXML as its parser presently >> (although Ed and I had a conversation just last week about refactoring >> it to support Nokogiri, as well), so the binary marc reader (and, if >> needed, writer) is way faster. Even accounting for Base64 >> encoding/decoding. It would also keep the index a whole lot smaller, >> if anybody cares about that. >> >> BTW, the reason why you're seeing the line breaks in Bess's script is >> because .to_s pretty prints the record. If she called .to_marc it >> would look like Naomi's mushy blob of text. >> >> -Ross. >> >> 2009/3/23 Naomi Dushay : >> >>> Maybe it's too soon to worry about this. Is there any pressing >>> need for all >>> of us to store our marc data in our indexes in the same format? >>> What's the >>> real gain? Why should the blacklight code be rigid about this? >>> Remember, a >>> lot of our solr documents won't have marc. >>> What if there was a blacklight config setting to say what format >>> your marc >>> was in, and the blacklight code accommodated the different formats? >>> Jessie is already working on manipulating the marc21 in the >>> Stanford index >>> to create the UI features we want at Stanford. UVa has been using >>> marcxml. >>> Jessie should jump in and tell us if the work he's doing can be >>> easily >>> adapted to other marc formats -- hopefully the answer is "yes, it >>> would be >>> trivial." But let's get those features coded first, and refactor >>> them >>> later. >>> I think we're doing great things here. UVa has put out a great >>> body of >>> code. The active Stanford adoption of the code is causing an >>> excellent >>> refactoring of the code, making it more configurable; additional >>> features >>> will be added as well. Further active adopters or folks that get >>> down and >>> dirty with the demo will provide more feedback, causing further >>> improvements >>> - refactoring or new features or whatever. >>> The demo for release 2.0 is going to have some warts; I don't >>> think there's >>> a way around it. Usage driven methodology would say we have some >>> folks >>> take the demo out for a spin, and get their feedback. Have folks >>> tried the >>> current demo? Have they tried to go the next step, of getting >>> their own >>> data behind the blacklight front end? If not ... maybe we've >>> gotten ahead >>> of ourselves? >>> Trying to guess a lot of this stuff before there's expressed demand >>> from the >>> users ... I think that can make extra work, and the results might >>> not be >>> useful, in the end. >>> Okay, I've got my asbestos suit on - let the flames be thrown! >>> - Naomi >>> >>> On Mar 22, 2009, at 6:03 PM, Bill Dueber wrote: >>> >>> solrmarc writes to the underlying lucene index using a binary field >>> type >>> that is unavailable in solr. This makes for easy storage of MARC, >>> but makes >>> round-tripping the data a little harder if that's your thing. Given >>> that's >>> the case, I wonder if a standard solr String field (probably >>> compressed) of >>> marcxml would be a better bet all around. >>> >>> Or push dchud into finishing his spec implementation of marc-json >>> and store >>> it as json :-) >>> >>> On Sun, Mar 22, 2009 at 4:14 PM, Jonathan Rochkind >>> wrote: >>> >>>> It makes sense to me to standardize 'what kind of MARC' we all >>>> store in >>>> SOLR. I guess you guys decided it made more sense for that to be >>>> MARC21 >>>> than MARC-XML? Curious what the motivation for that was? >>>> >>>> Since MARC21 is a binary format, you've got to make sure that what >>>> is >>>> going into the interface is binary identical to what should be, it >>>> should >>>> not be re-encoded or translated at all. MARC21 data can be in one >>>> of at >>>> least two character encodings, which one is used is theoretically >>>> mentioned >>>> in the MARC file (although it's often wrong). MARC21 file length >>>> (in >>>> bytes) is also noted in the MARC file header/leader. The whole >>>> thing needs >>>> to be byte-for-byte untouched when you stick it in your SOLR >>>> index, I don't >>>> know enough about SOLR or the indexing process to know about >>>> places that >>>> might introduce corruption there. >>>> >>>> MARC-XML is definitely a lot easier to work with, as it's just an >>>> ordinary >>>> XML file in UTF-8. >>>> >>>> I don't know a whole lot about MARC21, but it would not surprise >>>> me at >>>> all, if, like the charcter encoding issue, much of our MARC21 "in >>>> the wild" >>>> was actually illegal in undocumented various ways, but ways that our >>>> traditional ILS's are tolerant of. >>>> >>>> It's also possible there is a bug in the ruby MARC library. >>>> >>>> Jonathan >>>> >>>> ________________________________________ >>>> From: blacklight-development-bounces at rubyforge.org >>>> [blacklight-development-bounces at rubyforge.org] On Behalf Of Naomi >>>> Dushay >>>> [ndushay at stanford.edu] >>>> Sent: Sunday, March 22, 2009 1:47 PM >>>> To: blacklight-development at rubyforge.org >>>> Subject: Re: [Blacklight-development] storing marc21 in solr >>>> >>>> Bess, >>>> >>>> I create our index using solrmarc, which writes directly to the >>>> index >>>> (do not use solr, do not pass go, do not cellect $200.). So I >>>> can't >>>> be much help, sorry. >>>> >>>> However, I'll tell you that our marc21 doesn't look all nicely >>>> formatted like yours - it just appears as a single line and looks >>>> like >>>> this: >>>> >>>> 00651nam a22001575? >>>> >>>> 4500001000800000008004100008020001500049035002200064100001900086245006900105260003300174300001100207440005400218596000600272999021500278 >>>> a575946 910118s1990||||gw |||||||||||||||||ger|d a3412215880 >>>> a(CSt)notisACX2206 10 aSeifert, Arno. 14 aDer Ruckzug der biblischen >>>> Prophetie von der neueren Geschichte. 0 aKoln : bBohlau >>>> Verlag, c1990 a207 p. 0 aBeihefte zum Archiv f?r >>>> Kulturgeschichte ; vv.31 a1 aCB3 .A6 SUPPL. V. >>>> 31 >>>> >>>> wLC >>>> >>>> c1 >>>> i36105035087092 d6/18/2008 e6/18/2008 kCHECKEDOUT lSTACKS mGREEN >>>> n4 p >>>> $40.00 rM sY tSTKS-MONO u1/18/1991 o.TECHSTAFF. c:al o.TECHSTAFF. >>>> PUB >>>> 1/22/91/pb o.TECHSTAFF. MIDSPINE.v.31 suppl. >>>> >>>> Maybe you don't have "marc21" but something else? >>>> >>>> - Naomi >>>> >>>> On Mar 22, 2009, at 7:40 AM, Bess Sadler wrote: >>>> >>>> >>>>> Hey, folks. >>>>> >>>>> Hopefully I'm just having a moment of stupidity, but I've been >>>>> looking at this for awhile and I can't figure it out, so I'm asking >>>>> the community. We want to store marc21 in our solr index, right? >>>>> UVA >>>>> has always stored marc-xml, but we had a talk with Stanford folks >>>>> awhile ago about this, there seem to be lots of good reasons to >>>>> store marc21 instead, and that's fine with me, so now I'm just >>>>> trying to make it happen. >>>>> >>>>> BUT... when I store what I think is marc21 into solr, it doesn't >>>>> come back out again quite right. I've isolated a really simple >>>>> example of this: >>>>> >>>>> require 'rubygems' >>>>> require 'marc' >>>>> require 'rsolr' >>>>> >>>>> puts " ************* from file ****************** " >>>>> >>>>> reader = MARC::Reader.new('/usr/local/projects/bl-demo/data/ >>>>> test_data.utf8.mrc') >>>>> record2 = reader.first >>>>> puts record2.to_s >>>>> >>>>> puts " ************* from solr ****************** " >>>>> >>>>> rsolr = RSolr.connect >>>>> response = eval(rsolr.select(:q=>'*:*',:wt=>'ruby')) >>>>> marc_display = response["response"]["docs"][0]["marc_display"] >>>>> record = MARC::Record.new_from_marc(marc_display) >>>>> puts record.to_s >>>>> >>>>> My output for that script looks like this: >>>>> >>>>> ************* from file ****************** >>>>> LEADER 00799cam a2200241 a 4500 >>>>> 001 00282214 >>>>> 003 DLC >>>>> 005 20090120022042.0 >>>>> 008 000417s1998 pk 000 0 urdo >>>>> 010 $a 00282214 >>>>> 025 $a P-U-00282214; 05; 06 >>>>> 040 $a DLC $c DLC $d DLC >>>>> 041 1 $a urd $h snd >>>>> 042 $a lcode >>>>> 050 00 $a PK2788.9.A9 $b F55 1998 >>>>> 100 1 $a Ayaz, Shaikh, $d 1923-1997. >>>>> 245 10 $a Fikr-i Aya?z / $c murattibi?n, A?s?if Farruk?h?i?, >>>>> Sha?h Muh?ammad Pi?rza?dah. >>>>> 260 $a Kara?ci? : $b Da?niya?l, $c [1998] >>>>> 300 $a 375 p. ; $c 23 cm. >>>>> 546 $a In Urdu. >>>>> 520 $a Selected poems and articles from the works of renowned >>>>> Sindhi poet; chiefly translated from Sindhi. >>>>> 700 1 $a Farruk?h?i?, A?s?if, $d 1959- >>>>> 700 1 $a Pi?rza?dah, Sha?h Muh?ammad. >>>>> ************* from solr ****************** >>>>> LEADER 00799cam a2200241 a 4500 >>>>> 001 00282214 * >>>>> 003 DLC* >>>>> 005 20090120022042.0* >>>>> 008 000417s1998 pk 000 0 urdo * >>>>> >>>>> >>>>> So, when I take a file (test_data.utf.mrc) and read in the marc >>>>> records from disk, it behaves as expected. But if I store it in >>>>> solr >>>>> first, then what I get back out of solr appears to be badly formed >>>>> marc somehow. MARC::Record appears to read it in, but the record >>>>> created is truncated. >>>>> >>>>> Naomi, how do you folks store marc21? Do you do anything special to >>>>> it before putting it in the index? It occurred to me we could >>>>> base64 >>>>> encode it, but maybe there's something simpler that I'm missing? >>>>> >>>>> I'm storing the marc record at around line 96 of marc_mapper.rb: >>>>> >>>>> # _display is stored, but not indexed >>>>> # don't store a string, store marc21 so we can read it back out >>>>> # into a MARC::Record object >>>>> map :marc_display do |rec,index| >>>>> rec.to_marc >>>>> end >>>>> >>>>> Thanks in advance for any advice, >>>>> >>>>> Bess >>>>> _______________________________________________ >>>>> Blacklight-development mailing list >>>>> Blacklight-development at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>> -- >>> Bill Dueber >>> Library Systems Programmer >>> University of Michigan Library >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ >> > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From jamie at dang.com Mon Mar 23 13:03:38 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Mon, 23 Mar 2009 13:03:38 -0400 Subject: [Blacklight-development] (not) re-writing some tests with mocks In-Reply-To: References: <1ECE0D1F-182D-4951-8569-052A5B923EE4@virginia.edu> <2346C4B2-6A6C-48D9-A696-FE6E388F6AF4@dang.com> Message-ID: <4966BE2B-5E33-420E-B14B-E607F69426A2@dang.com> I think we need to step back and look at how Rails deals with this sort of situation with regards to standard databases, as we are in an analogous situation, but with a solr server rather than a database server. * Rails doesn't include a database or a database server. * Rails includes database.yml for configuring the connection to the db or your choice. We have that covered with solr.yml. Check. * Rails uses scaffold files to populate the test database with test data. We don't have this. Right now are rails test data isn't in the rails app, it's in the higher level data/ folder. What we need is the equivalent scaffold data area for our Solr tests in rails. * Rails also has rake tasks related to the database and tests scoped in the namespace "db:test". Seems to me we could create a "solr" namespace and put parallel tasks there. rake solr:test:load Modeling our specs' use of solr in the way Rails uses databases means no one has to think too hard about how to deal with solr, if they already are familiar with Rails. What do you all think? Jamie On Mar 23, 2009, at 12:22 PM, Bess Sadler wrote: > I've re-written all the tests now so that they only require a very > small number of records be in the index, and all of those records > are contained in test_data.utf8.mrc, which can index in only a few > seconds. How would people feel about just requiring that those test > records be indexed into the solr we distribute with the demo app in > order to run the tests? I think it's pretty reasonable to require > that the instance of solr we distribute is a test-only instance, and > that when people want to start running for-real against their own > data they need to set up a separate copy of solr for that. I know I > was the one pushing mocks earlier, but I think Naomi has a really > good point... we don't know if solr_helper is working properly > unless we're actually running the queries it generates against an > actual solr index. > > Bess > > On Mar 23, 2009, at 12:07 PM, Jamie Orchard-Hays wrote: > >> So then the question is, what is the best way for us to test if >> solr_helper is creating valid queries? Would it be simple, viable, >> etc >> to test the validity of the queries without connecting to a solr >> server? >> >> >> On Mar 21, 2009, at 8:34 PM, Bess Sadler wrote: >> >>> After talking to Naomi I realized that I had to roll back these >>> mocked tests. With the mocks in place, we would only be testing >>> whether the solr_helper could properly handle responses from rsolr, >>> but we would not be testing whether solr_helper knew how to create >>> valid queries. I've rolled back my changes. We're going to need that >>> local solr instance to test against after all. >>> >>> Hey, at least I learned a lot about rspec and mocks today! :) >>> >>> Bess >>> >>> On 21-Mar-09, at 6:19 PM, Bess Sadler wrote: >>> >>>> Okay, I made a first pass at this. I also commented out some tests >>>> that were failing, because I wanted to start with a 100% passing >>>> test >>>> suite before I made any major changes. I'll re-write those failing >>>> tests and add them back as I'm able. All of the tests I've >>>> changed so >>>> far are in solr_helper_spec.rb. It was running several queries, >>>> e.g., >>>> all_docs_query, no_docs_query, single_word_query, >>>> mult_word_query. I >>>> ran each of these against an index with all of our demo records in >>>> it, >>>> grabbed the response and put it in a mock method in mocks.rb, like >>>> this: >>>> >>>> def mock_no_docs_query >>>> %({"response"=> >>>> {"maxScore"=>0.0, "start"=>0, "docs"=>[], "numFound"=>0}, >>>> "facet_counts"=> >>>> { "facet_fields"=> >>>> { "subject_era_facet"=>[], >>>> "language_facet"=>[], >>>> "format_facet"=>[], >>>> "geographic_subject_facet"=>[]}, >>>> "facet_dates"=>{}, >>>> "facet_queries"=>{}}, >>>> "responseHeader"=> >>>> {"QTime"=>1, "params"=> >>>> {"qt"=>"search", "q"=>"zzzzzzzzzzzz", "wt"=>"ruby", >>>> "rows"=>"10"}, "status"=>0}}) >>>> end >>>> >>>> Then in the test, I create a RSolr::Ext::Response::Standard object, >>>> which is what the test is expecting, and populate it with that >>>> mocked >>>> response, like this: >>>> >>>> # use the mock response in mocks.rb >>>> raw_response = eval(mock_query_response) >>>> solr_response = RSolr::Ext::Response::Standard.new(raw_response) >>>> >>>> The rest of the tests remain the same. The only thing that has >>>> changed >>>> is that the response is coming from a text file instead of from >>>> solr. >>>> I left the old lines in place, but commented out and labeled, in >>>> case >>>> anyone wants to run against a live version of solr. >>>> >>>> Matt, I totally copied this technique from your rsolr-ext tests. >>>> Thank >>>> you! >>>> >>>> Bess >>>> >>>> >>>> On 21-Mar-09, at 4:20 PM, Bess Sadler wrote: >>>> >>>>> Hi, folks. >>>>> >>>>> In keeping with good rspec practice, I'm going to try to re-write >>>>> our >>>>> rspec tests not to depend on a live solr instance to be running. >>>>> Instead, I'm going to mock up the responses they would expect to >>>>> get >>>>> from solr, and include those in the rspec suite. This will offer a >>>>> few >>>>> advantages, but the number one reason is that we don't want all >>>>> our >>>>> tests to break when you point your Blacklight app at a new index >>>>> that >>>>> doesn't contain our sample records. >>>>> >>>>> I'm going to try this, and hope my rspec skills are up to the >>>>> challenge. If there are any objections, or if I'm missing some >>>>> reason >>>>> why I shouldn't do this, let me know. >>>>> >>>>> Bess >>>>> _______________________________________________ >>>>> Blacklight-development mailing list >>>>> Blacklight-development at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From ndushay at stanford.edu Mon Mar 23 14:10:25 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Mon, 23 Mar 2009 11:10:25 -0700 Subject: [Blacklight-development] (not) re-writing some tests with mocks In-Reply-To: <4966BE2B-5E33-420E-B14B-E607F69426A2@dang.com> References: <1ECE0D1F-182D-4951-8569-052A5B923EE4@virginia.edu> <2346C4B2-6A6C-48D9-A696-FE6E388F6AF4@dang.com> <4966BE2B-5E33-420E-B14B-E607F69426A2@dang.com> Message-ID: <239631B4-BBCF-45D2-9677-C381D7BF7B16@stanford.edu> This sounds like a great approach. Jessie is out sick today; I would like to get his input, and understand better what he's doing. I'm definitely in favor of flexibility as well as elegance and DRY principles. Lynn McRae pointed out to me that we don't want blacklight dependent on marc - we're going to be using it as a front end for solr indexes that have no marc. But I don't have any problem with the demo and the tests using marc - that's going to be a serious use case, for sure. Also, we're already working on a bunch of marc specific behaviors; great to have testing infrastructure for those. This is so great! I wrote some tests, Bess improved them, and we're going to improve them even more. We rock! - Naomi On Mar 23, 2009, at 10:03 AM, Jamie Orchard-Hays wrote: > I think we need to step back and look at how Rails deals with this > sort of situation with regards to standard databases, as we are in > an analogous situation, but with a solr server rather than a > database server. > > * Rails doesn't include a database or a database server. > * Rails includes database.yml for configuring the connection to the > db or your choice. We have that covered with solr.yml. Check. > * Rails uses scaffold files to populate the test database with test > data. We don't have this. Right now are rails test data isn't in the > rails app, it's in the higher level data/ folder. What we need is > the equivalent scaffold data area for our Solr tests in rails. > * Rails also has rake tasks related to the database and tests scoped > in the namespace "db:test". Seems to me we could create a "solr" > namespace and put parallel tasks there. rake solr:test:load > > Modeling our specs' use of solr in the way Rails uses databases > means no one has to think too hard about how to deal with solr, if > they already are familiar with Rails. > > What do you all think? > > Jamie > > > On Mar 23, 2009, at 12:22 PM, Bess Sadler wrote: > >> I've re-written all the tests now so that they only require a very >> small number of records be in the index, and all of those records >> are contained in test_data.utf8.mrc, which can index in only a few >> seconds. How would people feel about just requiring that those test >> records be indexed into the solr we distribute with the demo app in >> order to run the tests? I think it's pretty reasonable to require >> that the instance of solr we distribute is a test-only instance, >> and that when people want to start running for-real against their >> own data they need to set up a separate copy of solr for that. I >> know I was the one pushing mocks earlier, but I think Naomi has a >> really good point... we don't know if solr_helper is working >> properly unless we're actually running the queries it generates >> against an actual solr index. >> >> Bess >> >> On Mar 23, 2009, at 12:07 PM, Jamie Orchard-Hays wrote: >> >>> So then the question is, what is the best way for us to test if >>> solr_helper is creating valid queries? Would it be simple, viable, >>> etc >>> to test the validity of the queries without connecting to a solr >>> server? >>> >>> >>> On Mar 21, 2009, at 8:34 PM, Bess Sadler wrote: >>> >>>> After talking to Naomi I realized that I had to roll back these >>>> mocked tests. With the mocks in place, we would only be testing >>>> whether the solr_helper could properly handle responses from rsolr, >>>> but we would not be testing whether solr_helper knew how to create >>>> valid queries. I've rolled back my changes. We're going to need >>>> that >>>> local solr instance to test against after all. >>>> >>>> Hey, at least I learned a lot about rspec and mocks today! :) >>>> >>>> Bess >>>> >>>> On 21-Mar-09, at 6:19 PM, Bess Sadler wrote: >>>> >>>>> Okay, I made a first pass at this. I also commented out some tests >>>>> that were failing, because I wanted to start with a 100% passing >>>>> test >>>>> suite before I made any major changes. I'll re-write those failing >>>>> tests and add them back as I'm able. All of the tests I've >>>>> changed so >>>>> far are in solr_helper_spec.rb. It was running several queries, >>>>> e.g., >>>>> all_docs_query, no_docs_query, single_word_query, >>>>> mult_word_query. I >>>>> ran each of these against an index with all of our demo records in >>>>> it, >>>>> grabbed the response and put it in a mock method in mocks.rb, like >>>>> this: >>>>> >>>>> def mock_no_docs_query >>>>> %({"response"=> >>>>> {"maxScore"=>0.0, "start"=>0, "docs"=>[], "numFound"=>0}, >>>>> "facet_counts"=> >>>>> { "facet_fields"=> >>>>> { "subject_era_facet"=>[], >>>>> "language_facet"=>[], >>>>> "format_facet"=>[], >>>>> "geographic_subject_facet"=>[]}, >>>>> "facet_dates"=>{}, >>>>> "facet_queries"=>{}}, >>>>> "responseHeader"=> >>>>> {"QTime"=>1, "params"=> >>>>> {"qt"=>"search", "q"=>"zzzzzzzzzzzz", "wt"=>"ruby", >>>>> "rows"=>"10"}, "status"=>0}}) >>>>> end >>>>> >>>>> Then in the test, I create a RSolr::Ext::Response::Standard >>>>> object, >>>>> which is what the test is expecting, and populate it with that >>>>> mocked >>>>> response, like this: >>>>> >>>>> # use the mock response in mocks.rb >>>>> raw_response = eval(mock_query_response) >>>>> solr_response = RSolr::Ext::Response::Standard.new(raw_response) >>>>> >>>>> The rest of the tests remain the same. The only thing that has >>>>> changed >>>>> is that the response is coming from a text file instead of from >>>>> solr. >>>>> I left the old lines in place, but commented out and labeled, in >>>>> case >>>>> anyone wants to run against a live version of solr. >>>>> >>>>> Matt, I totally copied this technique from your rsolr-ext tests. >>>>> Thank >>>>> you! >>>>> >>>>> Bess >>>>> >>>>> >>>>> On 21-Mar-09, at 4:20 PM, Bess Sadler wrote: >>>>> >>>>>> Hi, folks. >>>>>> >>>>>> In keeping with good rspec practice, I'm going to try to re-write >>>>>> our >>>>>> rspec tests not to depend on a live solr instance to be running. >>>>>> Instead, I'm going to mock up the responses they would expect >>>>>> to get >>>>>> from solr, and include those in the rspec suite. This will >>>>>> offer a >>>>>> few >>>>>> advantages, but the number one reason is that we don't want all >>>>>> our >>>>>> tests to break when you point your Blacklight app at a new index >>>>>> that >>>>>> doesn't contain our sample records. >>>>>> >>>>>> I'm going to try this, and hope my rspec skills are up to the >>>>>> challenge. If there are any objections, or if I'm missing some >>>>>> reason >>>>>> why I shouldn't do this, let me know. >>>>>> >>>>>> Bess >>>>>> _______________________________________________ >>>>>> Blacklight-development mailing list >>>>>> Blacklight-development at rubyforge.org >>>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>> >>>>> _______________________________________________ >>>>> Blacklight-development mailing list >>>>> Blacklight-development at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From eos8d at virginia.edu Mon Mar 23 14:39:28 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Mon, 23 Mar 2009 14:39:28 -0400 Subject: [Blacklight-development] (not) re-writing some tests with mocks References: Message-ID: I really like this approach. One question, though: How do we get our schema.xml file installed? I think that was the logic behind distributing a copy of solr, wasn't it? That our solr comes pre- configured? Do we just make installing the distributed schema.xml file a pre-requisite that someone has to do manually? Or... could we distribute a config of some kind? In jetty, when I type "java -jar start.jar --help" I see this: Usage: java [-DDEBUG] [-DSTART=start.config] [-Dmain.class=org.MyMain] -jar start.jar [--help|--stop|--version] [config ...] What could we do with -DSTART=start.config? Could it point at something we distribute? Or are we over-thinking this? It's pretty much assumed that if you want to run most rails apps you need to know enough to run mysql. Maybe we should just assume that if you want to run blacklight you should be able to follow instructions for downloading solr and installing a schema.xml file? Bess On Mar 23, 2009, at 1:03 PM, Jamie Orchard-Hays wrote: > I think we need to step back and look at how Rails deals with this > sort of situation with regards to standard databases, as we are in an > analogous situation, but with a solr server rather than a database > server. > > * Rails doesn't include a database or a database server. > * Rails includes database.yml for configuring the connection to the db > or your choice. We have that covered with solr.yml. Check. > * Rails uses scaffold files to populate the test database with test > data. We don't have this. Right now are rails test data isn't in the > rails app, it's in the higher level data/ folder. What we need is the > equivalent scaffold data area for our Solr tests in rails. > * Rails also has rake tasks related to the database and tests scoped > in the namespace "db:test". Seems to me we could create a "solr" > namespace and put parallel tasks there. rake solr:test:load > > Modeling our specs' use of solr in the way Rails uses databases means > no one has to think too hard about how to deal with solr, if they > already are familiar with Rails. > > What do you all think? > > Jamie > > > On Mar 23, 2009, at 12:22 PM, Bess Sadler wrote: > >> I've re-written all the tests now so that they only require a very >> small number of records be in the index, and all of those records >> are contained in test_data.utf8.mrc, which can index in only a few >> seconds. How would people feel about just requiring that those test >> records be indexed into the solr we distribute with the demo app in >> order to run the tests? I think it's pretty reasonable to require >> that the instance of solr we distribute is a test-only instance, and >> that when people want to start running for-real against their own >> data they need to set up a separate copy of solr for that. I know I >> was the one pushing mocks earlier, but I think Naomi has a really >> good point... we don't know if solr_helper is working properly >> unless we're actually running the queries it generates against an >> actual solr index. >> >> Bess >> >> On Mar 23, 2009, at 12:07 PM, Jamie Orchard-Hays wrote: >> >>> So then the question is, what is the best way for us to test if >>> solr_helper is creating valid queries? Would it be simple, viable, >>> etc >>> to test the validity of the queries without connecting to a solr >>> server? >>> >>> >>> On Mar 21, 2009, at 8:34 PM, Bess Sadler wrote: >>> >>>> After talking to Naomi I realized that I had to roll back these >>>> mocked tests. With the mocks in place, we would only be testing >>>> whether the solr_helper could properly handle responses from rsolr, >>>> but we would not be testing whether solr_helper knew how to create >>>> valid queries. I've rolled back my changes. We're going to need >>>> that >>>> local solr instance to test against after all. >>>> >>>> Hey, at least I learned a lot about rspec and mocks today! :) >>>> >>>> Bess >>>> >>>> On 21-Mar-09, at 6:19 PM, Bess Sadler wrote: >>>> >>>>> Okay, I made a first pass at this. I also commented out some tests >>>>> that were failing, because I wanted to start with a 100% passing >>>>> test >>>>> suite before I made any major changes. I'll re-write those failing >>>>> tests and add them back as I'm able. All of the tests I've >>>>> changed so >>>>> far are in solr_helper_spec.rb. It was running several queries, >>>>> e.g., >>>>> all_docs_query, no_docs_query, single_word_query, >>>>> mult_word_query. I >>>>> ran each of these against an index with all of our demo records in >>>>> it, >>>>> grabbed the response and put it in a mock method in mocks.rb, like >>>>> this: >>>>> >>>>> def mock_no_docs_query >>>>> %({"response"=> >>>>> {"maxScore"=>0.0, "start"=>0, "docs"=>[], "numFound"=>0}, >>>>> "facet_counts"=> >>>>> { "facet_fields"=> >>>>> { "subject_era_facet"=>[], >>>>> "language_facet"=>[], >>>>> "format_facet"=>[], >>>>> "geographic_subject_facet"=>[]}, >>>>> "facet_dates"=>{}, >>>>> "facet_queries"=>{}}, >>>>> "responseHeader"=> >>>>> {"QTime"=>1, "params"=> >>>>> {"qt"=>"search", "q"=>"zzzzzzzzzzzz", "wt"=>"ruby", >>>>> "rows"=>"10"}, "status"=>0}}) >>>>> end >>>>> >>>>> Then in the test, I create a RSolr::Ext::Response::Standard >>>>> object, >>>>> which is what the test is expecting, and populate it with that >>>>> mocked >>>>> response, like this: >>>>> >>>>> # use the mock response in mocks.rb >>>>> raw_response = eval(mock_query_response) >>>>> solr_response = RSolr::Ext::Response::Standard.new(raw_response) >>>>> >>>>> The rest of the tests remain the same. The only thing that has >>>>> changed >>>>> is that the response is coming from a text file instead of from >>>>> solr. >>>>> I left the old lines in place, but commented out and labeled, in >>>>> case >>>>> anyone wants to run against a live version of solr. >>>>> >>>>> Matt, I totally copied this technique from your rsolr-ext tests. >>>>> Thank >>>>> you! >>>>> >>>>> Bess >>>>> >>>>> >>>>> On 21-Mar-09, at 4:20 PM, Bess Sadler wrote: >>>>> >>>>>> Hi, folks. >>>>>> >>>>>> In keeping with good rspec practice, I'm going to try to re-write >>>>>> our >>>>>> rspec tests not to depend on a live solr instance to be running. >>>>>> Instead, I'm going to mock up the responses they would expect to >>>>>> get >>>>>> from solr, and include those in the rspec suite. This will >>>>>> offer a >>>>>> few >>>>>> advantages, but the number one reason is that we don't want all >>>>>> our >>>>>> tests to break when you point your Blacklight app at a new index >>>>>> that >>>>>> doesn't contain our sample records. >>>>>> >>>>>> I'm going to try this, and hope my rspec skills are up to the >>>>>> challenge. If there are any objections, or if I'm missing some >>>>>> reason >>>>>> why I shouldn't do this, let me know. >>>>>> >>>>>> Bess >>>>>> _______________________________________________ >>>>>> Blacklight-development mailing list >>>>>> Blacklight-development at rubyforge.org >>>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>> >>>>> _______________________________________________ >>>>> Blacklight-development mailing list >>>>> Blacklight-development at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From jamie at dang.com Mon Mar 23 14:45:03 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Mon, 23 Mar 2009 14:45:03 -0400 Subject: [Blacklight-development] (not) re-writing some tests with mocks In-Reply-To: References: Message-ID: <87E127D1-3211-42AD-9D41-CB1D15690C5A@dang.com> How about: rake solr:schema:install Or something like that. It could take an environment variable for the location. We could also make that part of solr.yml. Jamie On Mar 23, 2009, at 2:39 PM, Bess Sadler wrote: > I really like this approach. One question, though: How do we get our > schema.xml file installed? I think that was the logic behind > distributing a copy of solr, wasn't it? That our solr comes pre- > configured? Do we just make installing the distributed schema.xml file > a pre-requisite that someone has to do manually? Or... could we > distribute a config of some kind? In jetty, when I type "java -jar > start.jar --help" I see this: > > Usage: java [-DDEBUG] [-DSTART=start.config] [-Dmain.class=org.MyMain] > -jar start.jar [--help|--stop|--version] [config ...] > > What could we do with -DSTART=start.config? Could it point at > something we distribute? > > Or are we over-thinking this? It's pretty much assumed that if you > want to run most rails apps you need to know enough to run mysql. > Maybe we should just assume that if you want to run blacklight you > should be able to follow instructions for downloading solr and > installing a schema.xml file? > > Bess > > On Mar 23, 2009, at 1:03 PM, Jamie Orchard-Hays wrote: > >> I think we need to step back and look at how Rails deals with this >> sort of situation with regards to standard databases, as we are in an >> analogous situation, but with a solr server rather than a database >> server. >> >> * Rails doesn't include a database or a database server. >> * Rails includes database.yml for configuring the connection to the >> db >> or your choice. We have that covered with solr.yml. Check. >> * Rails uses scaffold files to populate the test database with test >> data. We don't have this. Right now are rails test data isn't in the >> rails app, it's in the higher level data/ folder. What we need is the >> equivalent scaffold data area for our Solr tests in rails. >> * Rails also has rake tasks related to the database and tests scoped >> in the namespace "db:test". Seems to me we could create a "solr" >> namespace and put parallel tasks there. rake solr:test:load >> >> Modeling our specs' use of solr in the way Rails uses databases means >> no one has to think too hard about how to deal with solr, if they >> already are familiar with Rails. >> >> What do you all think? >> >> Jamie >> >> >> On Mar 23, 2009, at 12:22 PM, Bess Sadler wrote: >> >>> I've re-written all the tests now so that they only require a very >>> small number of records be in the index, and all of those records >>> are contained in test_data.utf8.mrc, which can index in only a few >>> seconds. How would people feel about just requiring that those test >>> records be indexed into the solr we distribute with the demo app in >>> order to run the tests? I think it's pretty reasonable to require >>> that the instance of solr we distribute is a test-only instance, and >>> that when people want to start running for-real against their own >>> data they need to set up a separate copy of solr for that. I know I >>> was the one pushing mocks earlier, but I think Naomi has a really >>> good point... we don't know if solr_helper is working properly >>> unless we're actually running the queries it generates against an >>> actual solr index. >>> >>> Bess >>> >>> On Mar 23, 2009, at 12:07 PM, Jamie Orchard-Hays wrote: >>> >>>> So then the question is, what is the best way for us to test if >>>> solr_helper is creating valid queries? Would it be simple, viable, >>>> etc >>>> to test the validity of the queries without connecting to a solr >>>> server? >>>> >>>> >>>> On Mar 21, 2009, at 8:34 PM, Bess Sadler wrote: >>>> >>>>> After talking to Naomi I realized that I had to roll back these >>>>> mocked tests. With the mocks in place, we would only be testing >>>>> whether the solr_helper could properly handle responses from >>>>> rsolr, >>>>> but we would not be testing whether solr_helper knew how to create >>>>> valid queries. I've rolled back my changes. We're going to need >>>>> that >>>>> local solr instance to test against after all. >>>>> >>>>> Hey, at least I learned a lot about rspec and mocks today! :) >>>>> >>>>> Bess >>>>> >>>>> On 21-Mar-09, at 6:19 PM, Bess Sadler wrote: >>>>> >>>>>> Okay, I made a first pass at this. I also commented out some >>>>>> tests >>>>>> that were failing, because I wanted to start with a 100% passing >>>>>> test >>>>>> suite before I made any major changes. I'll re-write those >>>>>> failing >>>>>> tests and add them back as I'm able. All of the tests I've >>>>>> changed so >>>>>> far are in solr_helper_spec.rb. It was running several queries, >>>>>> e.g., >>>>>> all_docs_query, no_docs_query, single_word_query, >>>>>> mult_word_query. I >>>>>> ran each of these against an index with all of our demo records >>>>>> in >>>>>> it, >>>>>> grabbed the response and put it in a mock method in mocks.rb, >>>>>> like >>>>>> this: >>>>>> >>>>>> def mock_no_docs_query >>>>>> %({"response"=> >>>>>> {"maxScore"=>0.0, "start"=>0, "docs"=>[], "numFound"=>0}, >>>>>> "facet_counts"=> >>>>>> { "facet_fields"=> >>>>>> { "subject_era_facet"=>[], >>>>>> "language_facet"=>[], >>>>>> "format_facet"=>[], >>>>>> "geographic_subject_facet"=>[]}, >>>>>> "facet_dates"=>{}, >>>>>> "facet_queries"=>{}}, >>>>>> "responseHeader"=> >>>>>> {"QTime"=>1, "params"=> >>>>>> {"qt"=>"search", "q"=>"zzzzzzzzzzzz", "wt"=>"ruby", >>>>>> "rows"=>"10"}, "status"=>0}}) >>>>>> end >>>>>> >>>>>> Then in the test, I create a RSolr::Ext::Response::Standard >>>>>> object, >>>>>> which is what the test is expecting, and populate it with that >>>>>> mocked >>>>>> response, like this: >>>>>> >>>>>> # use the mock response in mocks.rb >>>>>> raw_response = eval(mock_query_response) >>>>>> solr_response = RSolr::Ext::Response::Standard.new(raw_response) >>>>>> >>>>>> The rest of the tests remain the same. The only thing that has >>>>>> changed >>>>>> is that the response is coming from a text file instead of from >>>>>> solr. >>>>>> I left the old lines in place, but commented out and labeled, in >>>>>> case >>>>>> anyone wants to run against a live version of solr. >>>>>> >>>>>> Matt, I totally copied this technique from your rsolr-ext tests. >>>>>> Thank >>>>>> you! >>>>>> >>>>>> Bess >>>>>> >>>>>> >>>>>> On 21-Mar-09, at 4:20 PM, Bess Sadler wrote: >>>>>> >>>>>>> Hi, folks. >>>>>>> >>>>>>> In keeping with good rspec practice, I'm going to try to re- >>>>>>> write >>>>>>> our >>>>>>> rspec tests not to depend on a live solr instance to be running. >>>>>>> Instead, I'm going to mock up the responses they would expect to >>>>>>> get >>>>>>> from solr, and include those in the rspec suite. This will >>>>>>> offer a >>>>>>> few >>>>>>> advantages, but the number one reason is that we don't want all >>>>>>> our >>>>>>> tests to break when you point your Blacklight app at a new index >>>>>>> that >>>>>>> doesn't contain our sample records. >>>>>>> >>>>>>> I'm going to try this, and hope my rspec skills are up to the >>>>>>> challenge. If there are any objections, or if I'm missing some >>>>>>> reason >>>>>>> why I shouldn't do this, let me know. >>>>>>> >>>>>>> Bess >>>>>>> _______________________________________________ >>>>>>> Blacklight-development mailing list >>>>>>> Blacklight-development at rubyforge.org >>>>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>>> >>>>>> _______________________________________________ >>>>>> Blacklight-development mailing list >>>>>> Blacklight-development at rubyforge.org >>>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>> >>>>> _______________________________________________ >>>>> Blacklight-development mailing list >>>>> Blacklight-development at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From jamie at dang.com Mon Mar 23 15:03:25 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Mon, 23 Mar 2009 15:03:25 -0400 Subject: [Blacklight-development] local specs failing Message-ID: <65B2E814-9178-4763-8AC5-8A20AE6A6283@dang.com> Folks, I've got two examples failing locally and I can't see what's wrong. I've got the latest from the repo and I have deleted my solr index and indexed the new test data. Here's what I'm getting (solr_helper_spec): 1) NoMethodError in 'SolrHelper Get Document By Id should have valid marc21 in the marc21_storage_field' You have a nil object when you didn't expect it! You might have expected an instance of ActiveRecord::Base. The error occurred while evaluating nil.[] /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/reader.rb:65:in `decode' /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/record.rb:88:in `new_from_marc' ./vendor/plugins/blacklight/spec/helpers/solr_helper_spec.rb:258: 2) 'SolrHelper Get Document By Id should have a non-nil value for marc21_storage_field' FAILED expected not: == nil, got: nil ./vendor/plugins/blacklight/spec/helpers/solr_helper_spec.rb:254: Finished in 1.48438 seconds 94 examples, 2 failures rake aborted! -------- From Textmate, the failures look like this: should have a non-nil value for marc21_storage_field expected not: == nil, got: nil /Users/jamieorc/dev/blacklight/git/rails/vendor/plugins/blacklight/ spec/helpers/solr_helper_spec.rb:254 252 253 it "should have a non-nil value for marc21_storage_field" do 254 marc21.should_not == nil 255 end 256 should have valid marc21 in the marc21_storage_field You have a nil object when you didn't expect it! You might have expected an instance of ActiveRecord::Base. The error occurred while evaluating nil.[] /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/reader.rb:65 :in `decode' /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/record.rb: 88 :in `new_from_marc' /Users/jamieorc/dev/blacklight/git/rails/vendor/ plugins/blacklight/spec/helpers/solr_helper_spec.rb:258 63 def self.decode(marc, params={}) 64 record = Record.new() 65 record.leader = marc[0..LEADER_LENGTH-1] 66 67 # where the field data starts From ndushay at stanford.edu Mon Mar 23 15:07:25 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Mon, 23 Mar 2009 12:07:25 -0700 Subject: [Blacklight-development] (not) re-writing some tests with mocks In-Reply-To: References: Message-ID: <71007C29-C7AF-4896-8D70-91201CE09388@stanford.edu> I think it's okay to assume some solr knowledge; blacklight is built on top of solr; solr is a core technology for blacklight. On Mar 23, 2009, at 11:39 AM, Bess Sadler wrote: > I really like this approach. One question, though: How do we get our > schema.xml file installed? I think that was the logic behind > distributing a copy of solr, wasn't it? That our solr comes pre- > configured? Do we just make installing the distributed schema.xml file > a pre-requisite that someone has to do manually? Or... could we > distribute a config of some kind? In jetty, when I type "java -jar > start.jar --help" I see this: > > Usage: java [-DDEBUG] [-DSTART=start.config] [-Dmain.class=org.MyMain] > -jar start.jar [--help|--stop|--version] [config ...] > > What could we do with -DSTART=start.config? Could it point at > something we distribute? > > Or are we over-thinking this? It's pretty much assumed that if you > want to run most rails apps you need to know enough to run mysql. > Maybe we should just assume that if you want to run blacklight you > should be able to follow instructions for downloading solr and > installing a schema.xml file? > > Bess > > On Mar 23, 2009, at 1:03 PM, Jamie Orchard-Hays wrote: > >> I think we need to step back and look at how Rails deals with this >> sort of situation with regards to standard databases, as we are in an >> analogous situation, but with a solr server rather than a database >> server. >> >> * Rails doesn't include a database or a database server. >> * Rails includes database.yml for configuring the connection to the >> db >> or your choice. We have that covered with solr.yml. Check. >> * Rails uses scaffold files to populate the test database with test >> data. We don't have this. Right now are rails test data isn't in the >> rails app, it's in the higher level data/ folder. What we need is the >> equivalent scaffold data area for our Solr tests in rails. >> * Rails also has rake tasks related to the database and tests scoped >> in the namespace "db:test". Seems to me we could create a "solr" >> namespace and put parallel tasks there. rake solr:test:load >> >> Modeling our specs' use of solr in the way Rails uses databases means >> no one has to think too hard about how to deal with solr, if they >> already are familiar with Rails. >> >> What do you all think? >> >> Jamie >> >> >> On Mar 23, 2009, at 12:22 PM, Bess Sadler wrote: >> >>> I've re-written all the tests now so that they only require a very >>> small number of records be in the index, and all of those records >>> are contained in test_data.utf8.mrc, which can index in only a few >>> seconds. How would people feel about just requiring that those test >>> records be indexed into the solr we distribute with the demo app in >>> order to run the tests? I think it's pretty reasonable to require >>> that the instance of solr we distribute is a test-only instance, and >>> that when people want to start running for-real against their own >>> data they need to set up a separate copy of solr for that. I know I >>> was the one pushing mocks earlier, but I think Naomi has a really >>> good point... we don't know if solr_helper is working properly >>> unless we're actually running the queries it generates against an >>> actual solr index. >>> >>> Bess >>> >>> On Mar 23, 2009, at 12:07 PM, Jamie Orchard-Hays wrote: >>> >>>> So then the question is, what is the best way for us to test if >>>> solr_helper is creating valid queries? Would it be simple, viable, >>>> etc >>>> to test the validity of the queries without connecting to a solr >>>> server? >>>> >>>> >>>> On Mar 21, 2009, at 8:34 PM, Bess Sadler wrote: >>>> >>>>> After talking to Naomi I realized that I had to roll back these >>>>> mocked tests. With the mocks in place, we would only be testing >>>>> whether the solr_helper could properly handle responses from >>>>> rsolr, >>>>> but we would not be testing whether solr_helper knew how to create >>>>> valid queries. I've rolled back my changes. We're going to need >>>>> that >>>>> local solr instance to test against after all. >>>>> >>>>> Hey, at least I learned a lot about rspec and mocks today! :) >>>>> >>>>> Bess >>>>> >>>>> On 21-Mar-09, at 6:19 PM, Bess Sadler wrote: >>>>> >>>>>> Okay, I made a first pass at this. I also commented out some >>>>>> tests >>>>>> that were failing, because I wanted to start with a 100% passing >>>>>> test >>>>>> suite before I made any major changes. I'll re-write those >>>>>> failing >>>>>> tests and add them back as I'm able. All of the tests I've >>>>>> changed so >>>>>> far are in solr_helper_spec.rb. It was running several queries, >>>>>> e.g., >>>>>> all_docs_query, no_docs_query, single_word_query, >>>>>> mult_word_query. I >>>>>> ran each of these against an index with all of our demo records >>>>>> in >>>>>> it, >>>>>> grabbed the response and put it in a mock method in mocks.rb, >>>>>> like >>>>>> this: >>>>>> >>>>>> def mock_no_docs_query >>>>>> %({"response"=> >>>>>> {"maxScore"=>0.0, "start"=>0, "docs"=>[], "numFound"=>0}, >>>>>> "facet_counts"=> >>>>>> { "facet_fields"=> >>>>>> { "subject_era_facet"=>[], >>>>>> "language_facet"=>[], >>>>>> "format_facet"=>[], >>>>>> "geographic_subject_facet"=>[]}, >>>>>> "facet_dates"=>{}, >>>>>> "facet_queries"=>{}}, >>>>>> "responseHeader"=> >>>>>> {"QTime"=>1, "params"=> >>>>>> {"qt"=>"search", "q"=>"zzzzzzzzzzzz", "wt"=>"ruby", >>>>>> "rows"=>"10"}, "status"=>0}}) >>>>>> end >>>>>> >>>>>> Then in the test, I create a RSolr::Ext::Response::Standard >>>>>> object, >>>>>> which is what the test is expecting, and populate it with that >>>>>> mocked >>>>>> response, like this: >>>>>> >>>>>> # use the mock response in mocks.rb >>>>>> raw_response = eval(mock_query_response) >>>>>> solr_response = RSolr::Ext::Response::Standard.new(raw_response) >>>>>> >>>>>> The rest of the tests remain the same. The only thing that has >>>>>> changed >>>>>> is that the response is coming from a text file instead of from >>>>>> solr. >>>>>> I left the old lines in place, but commented out and labeled, in >>>>>> case >>>>>> anyone wants to run against a live version of solr. >>>>>> >>>>>> Matt, I totally copied this technique from your rsolr-ext tests. >>>>>> Thank >>>>>> you! >>>>>> >>>>>> Bess >>>>>> >>>>>> >>>>>> On 21-Mar-09, at 4:20 PM, Bess Sadler wrote: >>>>>> >>>>>>> Hi, folks. >>>>>>> >>>>>>> In keeping with good rspec practice, I'm going to try to re- >>>>>>> write >>>>>>> our >>>>>>> rspec tests not to depend on a live solr instance to be running. >>>>>>> Instead, I'm going to mock up the responses they would expect to >>>>>>> get >>>>>>> from solr, and include those in the rspec suite. This will >>>>>>> offer a >>>>>>> few >>>>>>> advantages, but the number one reason is that we don't want all >>>>>>> our >>>>>>> tests to break when you point your Blacklight app at a new index >>>>>>> that >>>>>>> doesn't contain our sample records. >>>>>>> >>>>>>> I'm going to try this, and hope my rspec skills are up to the >>>>>>> challenge. If there are any objections, or if I'm missing some >>>>>>> reason >>>>>>> why I shouldn't do this, let me know. >>>>>>> >>>>>>> Bess >>>>>>> _______________________________________________ >>>>>>> Blacklight-development mailing list >>>>>>> Blacklight-development at rubyforge.org >>>>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>>> >>>>>> _______________________________________________ >>>>>> Blacklight-development mailing list >>>>>> Blacklight-development at rubyforge.org >>>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>>> Blacklightopac Blog http://blacklightopac.org/ >>>>> >>>>> _______________________________________________ >>>>> Blacklight-development mailing list >>>>> Blacklight-development at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>>> Blacklightopac Blog http://blacklightopac.org/ >>>> >>>> _______________________________________________ >>>> Blacklight-development mailing list >>>> Blacklight-development at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/blacklight-development >>>> Blacklightopac Blog http://blacklightopac.org/ >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From eos8d at virginia.edu Mon Mar 23 15:31:30 2009 From: eos8d at virginia.edu (Bess Sadler) Date: Mon, 23 Mar 2009 15:31:30 -0400 Subject: [Blacklight-development] local specs failing In-Reply-To: <65B2E814-9178-4763-8AC5-8A20AE6A6283@dang.com> References: <65B2E814-9178-4763-8AC5-8A20AE6A6283@dang.com> Message-ID: Hi, Jamie. It's weird that those specs are failing for you. I just downloaded a fresh copy and they all passed for me. I added a line to solr.example. Maybe you need to re-copy your solr.example to solr.yml if your solr.yml file pre-dates my marc21 updates? If that doesn't work try downloading a totally fresh copy and see if the specs pass. Bess On Mar 23, 2009, at 3:03 PM, Jamie Orchard-Hays wrote: > Folks, I've got two examples failing locally and I can't see what's > wrong. I've got the latest from the repo and I have deleted my solr > index and indexed the new test data. > > Here's what I'm getting (solr_helper_spec): > > 1) > NoMethodError in 'SolrHelper Get Document By Id should have valid > marc21 in the marc21_storage_field' > You have a nil object when you didn't expect it! > You might have expected an instance of ActiveRecord::Base. > The error occurred while evaluating nil.[] > /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/reader.rb:65:in > `decode' > /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/record.rb:88:in > `new_from_marc' > ./vendor/plugins/blacklight/spec/helpers/solr_helper_spec.rb:258: > > 2) > 'SolrHelper Get Document By Id should have a non-nil value for > marc21_storage_field' FAILED > expected not: == nil, > got: nil > ./vendor/plugins/blacklight/spec/helpers/solr_helper_spec.rb:254: > > Finished in 1.48438 seconds > > 94 examples, 2 failures > rake aborted! > > > -------- > From Textmate, the failures look like this: > > should have a non-nil value for marc21_storage_field > expected not: == nil, got: nil > /Users/jamieorc/dev/blacklight/git/rails/vendor/plugins/blacklight/ > spec/helpers/solr_helper_spec.rb:254 > 252 253 it "should have a non-nil value for marc21_storage_field" do > 254 marc21.should_not == nil 255 end 256 > should have valid marc21 in the marc21_storage_field > You have a nil object when you didn't expect it! You might have > expected an instance of ActiveRecord::Base. The error occurred while > evaluating nil.[] > /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/reader.rb:65 :in > `decode' /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/record.rb: > 88 :in `new_from_marc' /Users/jamieorc/dev/blacklight/git/rails/ > vendor/ > plugins/blacklight/spec/helpers/solr_helper_spec.rb:258 > 63 def self.decode(marc, params={}) 64 record = Record.new() 65 > record.leader = marc[0..LEADER_LENGTH-1] 66 67 # where the field data > starts > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From jvine at stanford.edu Mon Mar 23 15:38:09 2009 From: jvine at stanford.edu (Jennifer Vine) Date: Mon, 23 Mar 2009 12:38:09 -0700 Subject: [Blacklight-development] single document questions / concerns In-Reply-To: <610ADEE1-993A-420E-8E14-730DF797A7F2@mac.com> References: <67EB1694-5048-4293-B35A-80A38A94156A@stanford.edu> <1CDA8FA1-9DB2-4865-A171-5BC2B6A089C4@mac.com> <07DEA77B-B49D-4623-906D-58E5CD1E0A41@dang.com> <49BEA434.3080806@jhu.edu> <74EDA21B-0B17-44D8-A5A7-32775D3CFB1D@dang.com> <4628791D-671E-4ED2-9B53-32388D47AC7A@stanford.edu> <23b83f160903162033x3a6085d7ka0ed9fb0d1de97f8@mail.gmail.com> <850BF705-DA68-4CB4-B9AD-C8A795BC64A0@mac.com> <640FF925-D683-43EB-84D9-D6C20B5C6751@stanford.edu> <90FF863A96E1EC42B8B240D04C88FB1D73406AA7@JHEMTEXVS2.win.ad.jhu.edu> <610ADEE1-993A-420E-8E14-730DF797A7F2@mac.com> Message-ID: <005a01c9abee$e60bd0f0$b22372d0$@edu> Hi all - Belatedly catching up on some UI-related comments... > On Mar 17, 2009, at 7:25 PM, Erik Hatcher wrote: > > i still don't see a prev/next as something needed on a detail page, > > but i'll go with the flow here and assume some usability tests said > > that was a good idea. personally, i know how to hit the back button > > and choose the next hit that is colored to indicate i haven't been > > there before. > > further on this... one usability issue that i've fought hard on with > LucidFind was targeting a new window when someone clicks on an > external link. again, personally i know how to cmd(or ctrl)-click to > open a link in a new tab and don't need a web page to do that for me > and it is a bit annoying often when sites do that. > > but, it's something to consider usability-wise when showing search > results and clicking off to details. I think there are two different user groups with different scenarios here. We have some users who really want the prev/next option - it's kind of a passive approach to looking through the results. On the other end, we have users (students in particular) who have a more targeted approach and use ctrl-click to open selected items from the list in new tabs, then look through the tabs to decide which items are worth keeping. We definitely have users who aren't familiar with ctrl-click, but there are strong accessibility reasons to avoid opening a new window automatically: forcing a non-sighted user into a new window where the back button doesn't work is just ...bad. JV From ndushay at stanford.edu Mon Mar 23 15:47:40 2009 From: ndushay at stanford.edu (Naomi Dushay) Date: Mon, 23 Mar 2009 12:47:40 -0700 Subject: [Blacklight-development] local specs failing In-Reply-To: References: <65B2E814-9178-4763-8AC5-8A20AE6A6283@dang.com> Message-ID: <24903F94-9B41-458F-AC7F-42DF12BCEDC4@stanford.edu> The other possibility is that the data needs to be re-indexed in Jamie's solr server. - Naomi On Mar 23, 2009, at 12:31 PM, Bess Sadler wrote: > Hi, Jamie. > > It's weird that those specs are failing for you. I just downloaded a > fresh copy and they all passed for me. I added a line to > solr.example. Maybe you need to re-copy your solr.example to > solr.yml if your solr.yml file pre-dates my marc21 updates? If that > doesn't work try downloading a totally fresh copy and see if the > specs pass. > > Bess > > On Mar 23, 2009, at 3:03 PM, Jamie Orchard-Hays wrote: > >> Folks, I've got two examples failing locally and I can't see what's >> wrong. I've got the latest from the repo and I have deleted my solr >> index and indexed the new test data. >> >> Here's what I'm getting (solr_helper_spec): >> >> 1) >> NoMethodError in 'SolrHelper Get Document By Id should have valid >> marc21 in the marc21_storage_field' >> You have a nil object when you didn't expect it! >> You might have expected an instance of ActiveRecord::Base. >> The error occurred while evaluating nil.[] >> /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/reader.rb:65:in >> `decode' >> /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/record.rb:88:in >> `new_from_marc' >> ./vendor/plugins/blacklight/spec/helpers/solr_helper_spec.rb:258: >> >> 2) >> 'SolrHelper Get Document By Id should have a non-nil value for >> marc21_storage_field' FAILED >> expected not: == nil, >> got: nil >> ./vendor/plugins/blacklight/spec/helpers/solr_helper_spec.rb:254: >> >> Finished in 1.48438 seconds >> >> 94 examples, 2 failures >> rake aborted! >> >> >> -------- >> From Textmate, the failures look like this: >> >> should have a non-nil value for marc21_storage_field >> expected not: == nil, got: nil >> /Users/jamieorc/dev/blacklight/git/rails/vendor/plugins/blacklight/ >> spec/helpers/solr_helper_spec.rb:254 >> 252 253 it "should have a non-nil value for marc21_storage_field" do >> 254 marc21.should_not == nil 255 end 256 >> should have valid marc21 in the marc21_storage_field >> You have a nil object when you didn't expect it! You might have >> expected an instance of ActiveRecord::Base. The error occurred while >> evaluating nil.[] >> /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/reader.rb:65 :in >> `decode' /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/record.rb: >> 88 :in `new_from_marc' /Users/jamieorc/dev/blacklight/git/rails/ >> vendor/ >> plugins/blacklight/spec/helpers/solr_helper_spec.rb:258 >> 63 def self.decode(marc, params={}) 64 record = Record.new() 65 >> record.leader = marc[0..LEADER_LENGTH-1] 66 67 # where the field data >> starts >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From jvine at stanford.edu Mon Mar 23 16:08:45 2009 From: jvine at stanford.edu (Jennifer Vine) Date: Mon, 23 Mar 2009 13:08:45 -0700 Subject: [Blacklight-development] including accordion effect for facets In-Reply-To: <89FE55E0-8478-4B3B-92B3-433793EDA3D0@virginia.edu> References: <1e4bf2cc0903201827g36693a8mb19261fd8868952a@mail.gmail.com> <89FE55E0-8478-4B3B-92B3-433793EDA3D0@virginia.edu> Message-ID: <005e01c9abf3$2ca84e40$85f8eac0$@edu> I'm super in favour of including accordion facets in 2.0 - but I thought that was one of the already-completed UVa features that was going to be re-inserted into the demo app. Am I completely misremembering/misunderstanding? JV > -----Original Message----- > From: blacklight-development-bounces at rubyforge.org [mailto:blacklight- > development-bounces at rubyforge.org] On Behalf Of Bess Sadler > Sent: Friday, March 20, 2009 7:33 PM > To: blacklight-development at rubyforge.org > Subject: Re: [Blacklight-development] including accordion effect for > facets > > I'm in favor of including this, particularly since we won't have much > nice styling in the 2.0 release, and the accordion facets are one > styling feature that people ask about a lot and that have done really > well in usability testing. I can see an argument for including it in > the plugin and for including it in the demo app, but I think I'd > prefer to see it included in the demo app, where it can also serve as > an example of how to overload behavior with local customization. > > Other thoughts? > > Bess > > On 20-Mar-09, at 9:27 PM, Tony Zanella wrote: > > > Hello all, > > Some folks have requested an accordion effect be included in the > > Blacklight demo. I decided to take a swing at writing one. While I > was > > researching, I found that there are a number of different effects > that > > go by the name "accordion". Since I wanted to keep swinging, I wrote > a > > bare-bones javascript I think could be expanded or customized pretty > > easily. With the latest version of the demo, I added the javascript > to > > > > /bl-demo/rails/public/plugin_assets/blacklight/javascripts > > > > and I linked it into the interface at > > > > /bl-demo/rails/vendor/plugins/blacklight/app/views/layouts > > > > and everything worked just fine. > > > > The question is: where should the addition and linkage really take > > place, assuming that it's a good idea to include this functionality > in > > the first place? I guess the other option is to stay out of the > > plugin. > > > > If we were to stay out of the plugin, wouldn't it make sense to > > include a layout in, say, /bl-demo/rails/app/views ? and include the > > javascript in, say, /bl-demo/rails/public/javascripts ? > > > > Thanks! > > Tony Zanella > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From jamie at dang.com Mon Mar 23 16:33:21 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Mon, 23 Mar 2009 16:33:21 -0400 Subject: [Blacklight-development] local specs failing In-Reply-To: <24903F94-9B41-458F-AC7F-42DF12BCEDC4@stanford.edu> References: <65B2E814-9178-4763-8AC5-8A20AE6A6283@dang.com> <24903F94-9B41-458F-AC7F-42DF12BCEDC4@stanford.edu> Message-ID: <58C183AD-A347-47A0-A9BC-93CC0E7956B0@dang.com> Thanks all. I was missing a new config parameter from solr.example. On Mar 23, 2009, at 3:47 PM, Naomi Dushay wrote: > The other possibility is that the data needs to be re-indexed in > Jamie's solr server. - Naomi > > > On Mar 23, 2009, at 12:31 PM, Bess Sadler wrote: > >> Hi, Jamie. >> >> It's weird that those specs are failing for you. I just downloaded >> a fresh copy and they all passed for me. I added a line to >> solr.example. Maybe you need to re-copy your solr.example to >> solr.yml if your solr.yml file pre-dates my marc21 updates? If that >> doesn't work try downloading a totally fresh copy and see if the >> specs pass. >> >> Bess >> >> On Mar 23, 2009, at 3:03 PM, Jamie Orchard-Hays wrote: >> >>> Folks, I've got two examples failing locally and I can't see what's >>> wrong. I've got the latest from the repo and I have deleted my solr >>> index and indexed the new test data. >>> >>> Here's what I'm getting (solr_helper_spec): >>> >>> 1) >>> NoMethodError in 'SolrHelper Get Document By Id should have valid >>> marc21 in the marc21_storage_field' >>> You have a nil object when you didn't expect it! >>> You might have expected an instance of ActiveRecord::Base. >>> The error occurred while evaluating nil.[] >>> /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/reader.rb:65:in >>> `decode' >>> /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/record.rb:88:in >>> `new_from_marc' >>> ./vendor/plugins/blacklight/spec/helpers/solr_helper_spec.rb:258: >>> >>> 2) >>> 'SolrHelper Get Document By Id should have a non-nil value for >>> marc21_storage_field' FAILED >>> expected not: == nil, >>> got: nil >>> ./vendor/plugins/blacklight/spec/helpers/solr_helper_spec.rb:254: >>> >>> Finished in 1.48438 seconds >>> >>> 94 examples, 2 failures >>> rake aborted! >>> >>> >>> -------- >>> From Textmate, the failures look like this: >>> >>> should have a non-nil value for marc21_storage_field >>> expected not: == nil, got: nil >>> /Users/jamieorc/dev/blacklight/git/rails/vendor/plugins/blacklight/ >>> spec/helpers/solr_helper_spec.rb:254 >>> 252 253 it "should have a non-nil value for marc21_storage_field" do >>> 254 marc21.should_not == nil 255 end 256 >>> should have valid marc21 in the marc21_storage_field >>> You have a nil object when you didn't expect it! You might have >>> expected an instance of ActiveRecord::Base. The error occurred while >>> evaluating nil.[] >>> /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/reader.rb:65 :in >>> `decode' /Library/Ruby/Gems/1.8/gems/marc-0.2.2/lib/marc/record.rb: >>> 88 :in `new_from_marc' /Users/jamieorc/dev/blacklight/git/rails/ >>> vendor/ >>> plugins/blacklight/spec/helpers/solr_helper_spec.rb:258 >>> 63 def self.decode(marc, params={}) 64 record = Record.new() 65 >>> record.leader = marc[0..LEADER_LENGTH-1] 66 67 # where the field >>> data >>> starts >>> >>> _______________________________________________ >>> Blacklight-development mailing list >>> Blacklight-development at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/blacklight-development >>> Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From tony.zanella at gmail.com Mon Mar 23 17:23:28 2009 From: tony.zanella at gmail.com (Tony Zanella) Date: Mon, 23 Mar 2009 17:23:28 -0400 Subject: [Blacklight-development] including accordion effect for facets In-Reply-To: <005e01c9abf3$2ca84e40$85f8eac0$@edu> References: <1e4bf2cc0903201827g36693a8mb19261fd8868952a@mail.gmail.com> <89FE55E0-8478-4B3B-92B3-433793EDA3D0@virginia.edu> <005e01c9abf3$2ca84e40$85f8eac0$@edu> Message-ID: <1e4bf2cc0903231423h72d90962ra53dd97a8e43ccf8@mail.gmail.com> Hello all, I've just committed the accordion effect into the plugin. Please have a look. Thanks for the helpful discussion about locating it. And thanks, Matt, for the JQuery help. Jennifer, I'm not sure if this speaks to your question, but I recall that the last time I compared the sidebar faceting between the demo and a branch of the UVa instance, the HTML generated in the facet views for UVa used definition lists, while the demo uses list elements. I think a re-insertion into the demo would have had to include a lot of reconfiguring of the views. Tony On Mon, Mar 23, 2009 at 4:08 PM, Jennifer Vine wrote: > I'm super in favour of including accordion facets in 2.0 - but I thought > that was one of the already-completed UVa features that was going to be > re-inserted into the demo app. Am I completely > misremembering/misunderstanding? > > JV > > >> -----Original Message----- >> From: blacklight-development-bounces at rubyforge.org [mailto:blacklight- >> development-bounces at rubyforge.org] On Behalf Of Bess Sadler >> Sent: Friday, March 20, 2009 7:33 PM >> To: blacklight-development at rubyforge.org >> Subject: Re: [Blacklight-development] including accordion effect for >> facets >> >> I'm in favor of including this, particularly since we won't have much >> nice styling in the 2.0 release, and the accordion facets are one >> styling feature that people ask about a lot and that have done really >> well in usability testing. I can see an argument for including it in >> the plugin and for including it in the demo app, but I think I'd >> prefer to see it included in the demo app, where it can also serve as >> an example of how to overload behavior with local customization. >> >> Other thoughts? >> >> Bess >> >> On 20-Mar-09, at 9:27 PM, Tony Zanella wrote: >> >> > Hello all, >> > Some folks have requested an accordion effect be included in the >> > Blacklight demo. I decided to take a swing at writing one. While I >> was >> > researching, I found that there are a number of different effects >> that >> > go by the name "accordion". Since I wanted to keep swinging, I wrote >> a >> > bare-bones javascript I think could be expanded or customized pretty >> > easily. With the latest version of the demo, I added the javascript >> to >> > >> > /bl-demo/rails/public/plugin_assets/blacklight/javascripts >> > >> > and I linked it into the interface at >> > >> > /bl-demo/rails/vendor/plugins/blacklight/app/views/layouts >> > >> > and everything worked just fine. >> > >> > The question is: where should the addition and linkage really take >> > place, assuming that it's a good idea to include this functionality >> in >> > the first place? I guess the other option is to stay out of the >> > plugin. >> > >> > If we were to stay out of the plugin, wouldn't it make sense to >> > include a layout in, say, /bl-demo/rails/app/views ? and include the >> > javascript in, say, /bl-demo/rails/public/javascripts ? >> > >> > Thanks! >> > Tony Zanella >> > _______________________________________________ >> > Blacklight-development mailing list >> > Blacklight-development at rubyforge.org >> > http://rubyforge.org/mailman/listinfo/blacklight-development >> > Blacklightopac Blog http://blacklightopac.org/ >> >> _______________________________________________ >> Blacklight-development mailing list >> Blacklight-development at rubyforge.org >> http://rubyforge.org/mailman/listinfo/blacklight-development >> Blacklightopac Blog http://blacklightopac.org/ > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > From jamie at dang.com Mon Mar 23 18:58:29 2009 From: jamie at dang.com (Jamie Orchard-Hays) Date: Mon, 23 Mar 2009 18:58:29 -0400 Subject: [Blacklight-development] Clean(er) document urls Message-ID: <5268280C-2D43-488F-AC0B-4EA8077A4EDC@dang.com> Hey all: I just committed my refactoring of the the document URLs. In a nutshell, the solr query params (:q, :f, :page, and :per_page) get stored in session[:search] (I scoped them in :search to help avoid help avoid collisions with end users session use). Queries still use GET, but those four params get stored in the session on each request. When the user clicks through to a document, the url to the original search is built using the session[:search] values rather than params[] values as in the previous version. The reason the Subject in this email says "Clean(er)" is that I have left the counter param in the URL. I thought of putting it in the session, but I think that would get messy for the end user--as in easy to use the back/forward button a few times and get out of sync. Another option might be to allow the show action to accept a POST as well and hide the counter in a hidden variable. Play around with it and see if it works for you. A user's search criteria are now absent from any copy-pasting and the counter has no meaning on its own, so privacy is maintained WRT this. What happens when a user gets a copy-pasted URL with the counter in it? The search defaults to empty, or all documents. The only caveat is that the counter value in that copy-pasted URL will almost surely be inaccurate. Try it and you'll see what I mean. OK, that's pass one at this. Awaiting feedback! Jamie From jvine at stanford.edu Mon Mar 23 22:39:19 2009 From: jvine at stanford.edu (Jennifer Vine) Date: Mon, 23 Mar 2009 19:39:19 -0700 Subject: [Blacklight-development] including accordion effect for facets In-Reply-To: <1e4bf2cc0903231423h72d90962ra53dd97a8e43ccf8@mail.gmail.com> References: <1e4bf2cc0903201827g36693a8mb19261fd8868952a@mail.gmail.com> <89FE55E0-8478-4B3B-92B3-433793EDA3D0@virginia.edu> <005e01c9abf3$2ca84e40$85f8eac0$@edu> <1e4bf2cc0903231423h72d90962ra53dd97a8e43ccf8@mail.gmail.com> Message-ID: <000901c9ac29$bd54fcf0$37fef6d0$@edu> Ah, that makes sense Tony. Thanks. JV > -----Original Message----- > From: blacklight-development-bounces at rubyforge.org [mailto:blacklight- > development-bounces at rubyforge.org] On Behalf Of Tony Zanella > Sent: Monday, March 23, 2009 2:23 PM > To: blacklight-development at rubyforge.org > Subject: Re: [Blacklight-development] including accordion effect for > facets > > Hello all, > I've just committed the accordion effect into the plugin. Please have a > look. > > Thanks for the helpful discussion about locating it. > > And thanks, Matt, for the JQuery help. > > Jennifer, I'm not sure if this speaks to your question, but I recall > that the last time I compared the sidebar faceting between the demo > and a branch of the UVa instance, the HTML generated in the facet > views for UVa used definition lists, while the demo uses list > elements. I think a re-insertion into the demo would have had to > include a lot of reconfiguring of the views. > Tony > > On Mon, Mar 23, 2009 at 4:08 PM, Jennifer Vine > wrote: > > I'm super in favour of including accordion facets in 2.0 - but I > thought > > that was one of the already-completed UVa features that was going to > be > > re-inserted into the demo app. Am I completely > > misremembering/misunderstanding? > > > > JV > > > > > >> -----Original Message----- > >> From: blacklight-development-bounces at rubyforge.org > [mailto:blacklight- > >> development-bounces at rubyforge.org] On Behalf Of Bess Sadler > >> Sent: Friday, March 20, 2009 7:33 PM > >> To: blacklight-development at rubyforge.org > >> Subject: Re: [Blacklight-development] including accordion effect for > >> facets > >> > >> I'm in favor of including this, particularly since we won't have > much > >> nice styling in the 2.0 release, and the accordion facets are one > >> styling feature that people ask about a lot and that have done > really > >> well in usability testing. I can see an argument for including it in > >> the plugin and for including it in the demo app, but I think I'd > >> prefer to see it included in the demo app, where it can also serve > as > >> an example of how to overload behavior with local customization. > >> > >> Other thoughts? > >> > >> Bess > >> > >> On 20-Mar-09, at 9:27 PM, Tony Zanella wrote: > >> > >> > Hello all, > >> > Some folks have requested an accordion effect be included in the > >> > Blacklight demo. I decided to take a swing at writing one. While I > >> was > >> > researching, I found that there are a number of different effects > >> that > >> > go by the name "accordion". Since I wanted to keep swinging, I > wrote > >> a > >> > bare-bones javascript I think could be expanded or customized > pretty > >> > easily. With the latest version of the demo, I added the > javascript > >> to > >> > > >> > /bl-demo/rails/public/plugin_assets/blacklight/javascripts > >> > > >> > and I linked it into the interface at > >> > > >> > /bl-demo/rails/vendor/plugins/blacklight/app/views/layouts > >> > > >> > and everything worked just fine. > >> > > >> > The question is: where should the addition and linkage really take > >> > place, assuming that it's a good idea to include this > functionality > >> in > >> > the first place? I guess the other option is to stay out of the > >> > plugin. > >> > > >> > If we were to stay out of the plugin, wouldn't it make sense to > >> > include a layout in, say, /bl-demo/rails/app/views ? and include > the > >> > javascript in, say, /bl-demo/rails/public/javascripts ? > >> > > >> > Thanks! > >> > Tony Zanella > >> > _______________________________________________ > >> > Blacklight-development mailing list > >> > Blacklight-development at rubyforge.org > >> > http://rubyforge.org/mailman/listinfo/blacklight-development > >> > Blacklightopac Blog http://blacklightopac.org/ > >> > >> _______________________________________________ > >> Blacklight-development mailing list > >> Blacklight-development at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/blacklight-development > >> Blacklightopac Blog http://blacklightopac.org/ > > > > _______________________________________________ > > Blacklight-development mailing list > > Blacklight-development at rubyforge.org > > http://rubyforge.org/mailman/listinfo/blacklight-development > > Blacklightopac Blog http://blacklightopac.org/ > > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ From rochkind at jhu.edu Mon Mar 23 23:24:39 2009 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 23 Mar 2009 23:24:39 -0400 Subject: [Blacklight-development] Clean(er) document urls In-Reply-To: <5268280C-2D43-488F-AC0B-4EA8077A4EDC@dang.com> References: <5268280C-2D43-488F-AC0B-4EA8077A4EDC@dang.com> Message-ID: <90FF863A96E1EC42B8B240D04C88FB1D73406ABC@JHEMTEXVS2.win.ad.jhu.edu> Nice. I wish there was a way to keep counter in session storage too, but I'll wait until I'm (hopefully someday soon) actually working on the code, and see if I can figure a good one out. I think this solution will unfortunately mess up next/previous buttons if you try to have different searches open in different windows though? Might be a neccesary downside. IMO if the user gets a bookmarked URL (with or without a counter in it), and comes to the app without any search criteria in the session -- she simply should not get any next/previous buttons at all, rather than get next/previous buttons that default to all documents and a surely erroneous counter. This should be fairly easy to do, I think? Any reason not to do it? Jonathan ________________________________________ From: blacklight-development-bounces at rubyforge.org [blacklight-development-bounces at rubyforge.org] On Behalf Of Jamie Orchard-Hays [jamie at dang.com] Sent: Monday, March 23, 2009 6:58 PM To: blacklight-development at rubyforge.org Subject: [Blacklight-development] Clean(er) document urls Hey all: I just committed my refactoring of the the document URLs. In a nutshell, the solr query params (:q, :f, :page, and :per_page) get stored in session[:search] (I scoped them in :search to help avoid help avoid collisions with end users session use). Queries still use GET, but those four params get stored in the session on each request. When the user clicks through to a document, the url to the original search is built using the session[:search] values rather than params[] values as in the previous version. The reason the Subject in this email says "Clean(er)" is that I have left the counter param in the URL. I thought of putting it in the session, but I think that would get messy for the end user--as in easy to use the back/forward button a few times and get out of sync. Another option might be to allow the show action to accept a POST as well and hide the counter in a hidden variable. Play around with it and see if it works for you. A user's search criteria are now absent from any copy-pasting and the counter has no meaning on its own, so privacy is maintained WRT this. What happens when a user gets a copy-pasted URL with the counter in it? The search defaults to empty, or all documents. The only caveat is that the counter value in that copy-pasted URL will almost surely be inaccurate. Try it and you'll see what I mean. OK, that's pass one at this. Awaiting feedback! Jamie _______________________________________________ Blacklight-development mailing list Blacklight-development at rubyforge.org http://rubyforge.org/mailman/listinfo/blacklight-development Blacklightopac Blog http://blacklightopac.org/ From bill at dueber.com Tue Mar 24 09:33:03 2009 From: bill at dueber.com (Bill Dueber) Date: Tue, 24 Mar 2009 09:33:03 -0400 Subject: [Blacklight-development] Clean(er) document urls In-Reply-To: <90FF863A96E1EC42B8B240D04C88FB1D73406ABC@JHEMTEXVS2.win.ad.jhu.edu> References: <5268280C-2D43-488F-AC0B-4EA8077A4EDC@dang.com> <90FF863A96E1EC42B8B240D04C88FB1D73406ABC@JHEMTEXVS2.win.ad.jhu.edu> Message-ID: On Mon, Mar 23, 2009 at 11:24 PM, Jonathan Rochkind wrote: > I think this solution will unfortunately mess up next/previous buttons if > you try to have different searches open in different windows though? Might > be a neccesary downside. > If there's a *good* solution to this issue, I've never seen it. One option I've thought of is to use the search terms (which need to be on the page so they can go in the search box anyway) to key the session hash (e.g., session[:search]["search terms"]). Then the next/prev buttons could pass a munged version of the search terms, which could be used to pick the rest of the variables out of the session. The downside is that the next/prev URLs get a little uglier, but they're not supposed to be persistent, anyway. And, of course, a person could be doing two searches with the same search terms but different facets, but there are always pathological cases. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chapman.lists at gmail.com Tue Mar 24 10:48:07 2009 From: chapman.lists at gmail.com (Vernon Chapman) Date: Tue, 24 Mar 2009 10:48:07 -0400 Subject: [Blacklight-development] Latest Update (Version 1064) Message-ID: <49C8F2A7.5060203@gmail.com> Hello BL, I am trying to get the latest trunk up an running and I keep getting connection errors. Here is what I have done: 1 - checked out a fresh version (1064) of BL 2 - copied the database.example and solr.example files to .yml files 3 - changed all the urls in solr.yml to http://127.0.0.1:8080/solr/content to access my external multicore solr instance 4 - started the app using ./script/server Thanks to the extra logging that was added to blacklight.rb I see the following in the apps log file BLACKLIGHT: initialized with Blacklight.solr_config: {:adapter=>:http, :url=>"http://127.0.0.1:8080/solr/content"} BLACKLIGHT: initialized with Blacklight.solr: #:http, :url=>"http://127.0.0.1:8080/solr/content"}, @adapter=#, @opts={:adapter=>:curb, :url=>"http://127.0.0.1:8983/solr"}>> BLACKLIGHT: initialized with Blacklight.config: {} And here is the trace: /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/core_ext/pathname/clean_within.rb:7:in `clean_within' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/core_ext/exception.rb:11:in `clean_message' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/templates/rescues/diagnostics.erb:7 /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/renderable.rb:39:in `send' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/renderable.rb:39:in `render' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/template.rb:73:in `render_template' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/base.rb:256:in `render' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/rescue.rb:111:in `rescue_action_locally' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/rescue.rb:128:in `rescue_action_without_handler' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/rescue.rb:61:in `rescue_action' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/rescue.rb:138:in `perform_action_without_caching' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/caching/sql_cache.rb:13:in `perform_action' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/abstract/query_cache.rb:34:in `cache' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activerecord-2.2.2/lib/active_record/query_cache.rb:8:in `cache' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/caching/sql_cache.rb:12:in `perform_action' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:524:in `send' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:524:in `process_without_filters' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/filters.rb:606:in `process_without_session_management_support' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/session_management.rb:134:in `process' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:392:in `process' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:183:in `handle_request' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:110:in `dispatch_unlocked' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:123:in `dispatch' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:122:in `synchronize' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:122:in `dispatch' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:132:in `dispatch_cgi' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:39:in `dispatch' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/webrick_server.rb:103:in `handle_dispatch' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/webrick_server.rb:74:in `service' /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/httpserver.rb:104:in `service' /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/httpserver.rb:65:in `run' /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:173:in `start_thread' /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:162:in `start' /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:162:in `start_thread' /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:95:in `start' /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:92:in `each' /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:92:in `start' /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:23:in `start' /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:82:in `start' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/webrick_server.rb:60:in `dispatch' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/commands/servers/webrick.rb:66 /usr/local/ruby-1.8.7/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require' /usr/local/ruby-1.8.7/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/dependencies.rb:153:in `require' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/dependencies.rb:521:in `new_constants_in' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/dependencies.rb:153:in `require' /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/commands/server.rb:49 /usr/local/ruby-1.8.7/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require' /usr/local/ruby-1.8.7/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' script/server:3 Any ideas? Thanks Vernon From jronallo at gmail.com Tue Mar 24 10:57:03 2009 From: jronallo at gmail.com (Jason Ronallo) Date: Tue, 24 Mar 2009 10:57:03 -0400 Subject: [Blacklight-development] curb's solr port Message-ID: <763570460903240757j70e47c1bua6302e0678557b46@mail.gmail.com> Hi, Here's the long version of figuring out a problem I ran into, but if you want to get to the fix see below the cut. I'm playing with the blacklight demo app this morning and hit on a problem. I already had solr running on 8983 so I changed the ports in config/solr.yml to 8984. I then ran the db migrations, started up the packaged solr on 8984 (and can get to the admin interface and see the blacklight schema.xml) and tried to index the test data. It appeared that it was trying to post to 8983 still. I got an error about solr not having a format_facet field. When I inspected Blacklight.solr in app.rake I see the following: #, @opts={:url=>"http://127.0.0.1:8983/solr"}>, @opts={:adapter=>:http, :url=>"http://127.0.0.1:8984/solr"}> For some reason it looks like the correct port is picked up in one place but not the other? Thinking this might have been entered into the db before I made changes to the solr.yml, I removed the development sqlite db, changed the port in every rails file I could and ran the migrations again. I tried to search all the files in the bl-demo and change the port, but no luck. Is this a problem with the rsolr gem? I can reproduce this discrepancy between url port options in irb. > require 'rubygems' => true > require 'rsolr' => true > RSolr.connect(:url => 'http://127.0.0.1:8985/solr') => #"http://127.0.0.1:8985/solr", :adapter=>:http}, @adapter=#"http://127.0.0.1:8983/solr"}, @connector=#>> -------------------------- OK, I think I fixed this in my fork of rsolr: http://github.com/jronallo/rsolr/tree/master The fix was that if there is no adaptor option url passed in then it defaults to the standard port. I changed it so that it will default to the url given for connection options. Once I did that it indexed the demo records fine and I was able to start up rails. Now to figure out how to place blacklight over a preexisting solr index of some DC metadata. Jason Ronallo From goodieboy at gmail.com Tue Mar 24 11:09:30 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 24 Mar 2009 11:09:30 -0400 Subject: [Blacklight-development] Latest Update (Version 1064) In-Reply-To: <49C8F2A7.5060203@gmail.com> References: <49C8F2A7.5060203@gmail.com> Message-ID: Vernon, There is a bug in vendor/plugins/blacklight/lib/blacklight.rb that I found yesterday, and one that I created :) I haven't had a chance to fix/commit this yet. This line: # Create a global connection instance Blacklight.solr = RSolr.connect(Blacklight.solr_config) Needs to change to: # Create a global connection instance Blacklight.solr = RSolr.connect({}, Blacklight.solr_config) Give that a try and see what happens. If it works, would you mind committing? Matt On Tue, Mar 24, 2009 at 10:48 AM, Vernon Chapman wrote: > Hello BL, > > I am trying to get the latest trunk up an running and I keep getting > connection errors. > > Here is what I have done: > > 1 - checked out a fresh version (1064) of BL > 2 - copied the database.example and solr.example files to .yml files > 3 - changed all the urls in solr.yml to http://127.0.0.1:8080/solr/contentto access my external multicore solr instance > 4 - started the app using ./script/server > > Thanks to the extra logging that was added to blacklight.rb I see the > following in the apps log file > > BLACKLIGHT: initialized with Blacklight.solr_config: {:adapter=>:http, > :url=>"http://127.0.0.1:8080/solr/content"} > BLACKLIGHT: initialized with Blacklight.solr: # @opts={:adapter=>:http, :url=>"http://127.0.0.1:8080/solr/content"}, > @adapter=# @connector=#, > @opts={:adapter=>:curb, :url=>"http://127.0.0.1:8983/solr"}>> > BLACKLIGHT: initialized with Blacklight.config: {} > > And here is the trace: > > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/core_ext/pathname/clean_within.rb:7:in > `clean_within' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/core_ext/exception.rb:11:in > `clean_message' > > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/templates/rescues/diagnostics.erb:7 > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/renderable.rb:39:in > `send' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/renderable.rb:39:in > `render' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/template.rb:73:in > `render_template' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/base.rb:256:in > `render' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/rescue.rb:111:in > `rescue_action_locally' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/rescue.rb:128:in > `rescue_action_without_handler' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/rescue.rb:61:in > `rescue_action' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/rescue.rb:138:in > `perform_action_without_caching' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/caching/sql_cache.rb:13:in > `perform_action' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/abstract/query_cache.rb:34:in > `cache' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activerecord-2.2.2/lib/active_record/query_cache.rb:8:in > `cache' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/caching/sql_cache.rb:12:in > `perform_action' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:524:in > `send' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:524:in > `process_without_filters' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/filters.rb:606:in > `process_without_session_management_support' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/session_management.rb:134:in > `process' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:392:in > `process' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:183:in > `handle_request' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:110:in > `dispatch_unlocked' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:123:in > `dispatch' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:122:in > `synchronize' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:122:in > `dispatch' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:132:in > `dispatch_cgi' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:39:in > `dispatch' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/webrick_server.rb:103:in > `handle_dispatch' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/webrick_server.rb:74:in > `service' > /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/httpserver.rb:104:in `service' > /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/httpserver.rb:65:in `run' > /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:173:in `start_thread' > /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:162:in `start' > /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:162:in `start_thread' > /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:95:in `start' > /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:92:in `each' > /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:92:in `start' > /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:23:in `start' > /usr/local/ruby-1.8.7/lib/ruby/1.8/webrick/server.rb:82:in `start' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/webrick_server.rb:60:in > `dispatch' > > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/commands/servers/webrick.rb:66 > /usr/local/ruby-1.8.7/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in > `gem_original_require' > /usr/local/ruby-1.8.7/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in > `require' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/dependencies.rb:153:in > `require' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/dependencies.rb:521:in > `new_constants_in' > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/dependencies.rb:153:in > `require' > > /usr/local/ruby-1.8.7/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/commands/server.rb:49 > /usr/local/ruby-1.8.7/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in > `gem_original_require' > /usr/local/ruby-1.8.7/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in > `require' > script/server:3 > > Any ideas? > > Thanks > > Vernon > > _______________________________________________ > Blacklight-development mailing list > Blacklight-development at rubyforge.org > http://rubyforge.org/mailman/listinfo/blacklight-development > Blacklightopac Blog http://blacklightopac.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From goodieboy at gmail.com Tue Mar 24 11:28:55 2009 From: goodieboy at gmail.com (Matt Mitchell) Date: Tue, 24 Mar 2009 11:28:55 -0400 Subject: [Blacklight-development] curb's solr port In-Reply-To: <763570460903240757j70e47c1bua6302e0678557b46@mail.gmail.com> References: <763570460903240757j70e47c1bua6302e0678557b46@mail.gmail.com> Message-ID: Yes, the problem is that the options for the url are the second argument in RSolr.connect opts={} adapter_opts={:url=>'"} solr = RSolr.connect(opts, adapter_opts) For RSolr, there are options for the main connection class (the wrapper) and there are options for the adapters (http and jruby direct). Because we're using the http adapter, we need to set the :url option. Which means the second argument. I introduced this bug on Friday. I'll see if I can get a fix committed over lunch. Matt On Tue, Mar 24, 2009 at 10:57 AM, Jason Ronallo wrote: > Hi, > Here's the long version of figuring out a problem I ran into, but if > you want to get to the fix see below the cut. > > I'm playing with the blacklight demo app this morning and hit on a > problem. I already had solr running on 8983 so I changed the ports in > config/solr.yml to 8984. I then ran the db migrations, started up the > packaged solr on 8984 (and can get to the admin interface and see the > blacklight schema.xml) and tried to index the test data. It appeared > that it was trying to post to 8983 still. I got an error about