From rochkind at jhu.edu Thu Oct 13 15:26:41 2011 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Thu, 13 Oct 2011 15:26:41 -0400 Subject: [Umlaut-general] moving to git/github Message-ID: <4E973B71.3090901@jhu.edu> So I think I've got some time to actually start working on the Great Umlaut Migration. The outcome of this work will be an updated Umlaut that runs as a gem/plugin under Rails 3.1, and under ruby 1.9. (Probably requiring ruby 1.9, for it's better handling of threads). Probably also clean up some un-used code and do a bit of minor additional refactoring, etc. Let's call this the future Umlaut 3.0. I don't want to do this work in svn, I've gotten too used to the much better experience of git on github. The trick is that it will of course take some amount of time for Umlaut 3.0 to be finished -- in the meantime, our current Umlaut install instructions involve a checkout from svn, and in the meantime bug fixes or new features may have to be committed to Umlaut 2.0. I think I have a way to manage this where Umlaut 2.x development will still happen in svn (so we don't need to figure out a 2.x installation method based on git instead of svn), while my 3.x development happens in git, and if we fix bugs or add features in the 2.x svn, they can still easily be merged into my 3.x development (well, easily if they don't conflict). I've already imported the svn repo into github(https://github.com/team-umlaut/umlaut), using svn2git(https://github.com/nirvdrum/svn2git), which imported complete history/tags/branches too. svn2git allows you to periodically re-sync from svn -- it's a one-way operation, you can't go from git to svn. But my plan is, the 'master/trunk' in git (and other branches from svn) (git calls it 'master', svn calls it 'trunk', same thing), I will _never_ touch in git. I will do my development in a '3.x.dev' branch in git. Periodically, if there have been changes in the 2.x svn trunk, I will re-sync git to svn (run `svn2git --rebase`); then _merge- from the sync'd git master into my git feature branch, in order to get any bugfixes or new features into the 3.x branch too. That will work fine as long as there haven't been _major_ changes to the svn 2.x version, which conflict with the 3.x refactor, and require manual merging. (and assuming svn2git works as well as it says it does; I've done just minor tests to verify 'proof of concept' here). Eventually Umlaut 3.0 will be done and released, with a completely different better installation method, and at that point the rubyforge svn will go away, and we'll be all git/github all the time. So: 1. Does this sound like a good way to proceed? Any better ones? 2. Scot, I think you are currently working on, well, one or even several new features for Umlaut. The "institution" one could theoretically involve some fairly major refactoring, no? If you do major refactoring in 2.x after I start on this work, it could end up being a huge pain to merge it together. Do you have any timeline on when you think you'll commit this or any other features you're in the middle of? Ideally you'd finish them soon _before_ I start, OR perhaps they won't actually involve major refactors after all, OR you'd want until Umlaut 3.0 to do em. We can accomodate small to maybe medium sized code changes in 2.x merged over to git 3.x, for bug fixes, or new featuers. We can accomodate new adapters, or significant changes to existing adapters, no problem. It's significant refactoring to Umlaut's core architecture that could be problematic. 3. Any other questions about either this project, or what's envisioned for Umlaut 3.0? From std5 at nyu.edu Mon Oct 17 10:09:24 2011 From: std5 at nyu.edu (Scot Dalton) Date: Mon, 17 Oct 2011 10:09:24 -0400 Subject: [Umlaut-general] moving to git/github In-Reply-To: <4E973B71.3090901@jhu.edu> References: <4E973B71.3090901@jhu.edu> Message-ID: <431B9B38-415F-49E5-B1AB-A26E6FCD020C@nyu.edu> Hi Jonathan, Thanks for taking this on. Sounds great. As far as my development efforts go, I'd previously committed my changes for institutions to the auth-integration branch, which you moved to github. [1] There are a few identified bugs that I need to work out, but can commit to having those done by the end of this week and merging back to master/trunk. Should I just update the rubyforge svn trunk and then you'll use svn2git to move it over to github? Or I could just update github master and it could be a new feature of Umlaut 3.x. The only other enhancement in my queue is dealing with the SFX disambiguation page and trying to have Umlaut handle that more effectively, but that should only affect the SFX adapter and should be straightforward to manage. Let me know your thoughts. Thanks again, Scot [1] https://github.com/team-umlaut/umlaut/tree/auth-integration On Oct 13, 2011, at Oct 13, 3:26 PM, Jonathan Rochkind wrote: > So I think I've got some time to actually start working on the Great Umlaut Migration. The outcome of this work will be an updated Umlaut that runs as a gem/plugin under Rails 3.1, and under ruby 1.9. (Probably requiring ruby 1.9, for it's better handling of threads). Probably also clean up some un-used code and do a bit of minor additional refactoring, etc. Let's call this the future Umlaut 3.0. > > I don't want to do this work in svn, I've gotten too used to the much better experience of git on github. > > The trick is that it will of course take some amount of time for Umlaut 3.0 to be finished -- in the meantime, our current Umlaut install instructions involve a checkout from svn, and in the meantime bug fixes or new features may have to be committed to Umlaut 2.0. > > I think I have a way to manage this where Umlaut 2.x development will still happen in svn (so we don't need to figure out a 2.x installation method based on git instead of svn), while my 3.x development happens in git, and if we fix bugs or add features in the 2.x svn, they can still easily be merged into my 3.x development (well, easily if they don't conflict). > > I've already imported the svn repo into github(https://github.com/team-umlaut/umlaut), using svn2git(https://github.com/nirvdrum/svn2git), which imported complete history/tags/branches too. > > svn2git allows you to periodically re-sync from svn -- it's a one-way operation, you can't go from git to svn. But my plan is, the 'master/trunk' in git (and other branches from svn) (git calls it 'master', svn calls it 'trunk', same thing), I will _never_ touch in git. I will do my development in a '3.x.dev' branch in git. Periodically, if there have been changes in the 2.x svn trunk, I will re-sync git to svn (run `svn2git --rebase`); then _merge- from the sync'd git master into my git feature branch, in order to get any bugfixes or new features into the 3.x branch too. That will work fine as long as there haven't been _major_ changes to the svn 2.x version, which conflict with the 3.x refactor, and require manual merging. (and assuming svn2git works as well as it says it does; I've done just minor tests to verify 'proof of concept' here). > > Eventually Umlaut 3.0 will be done and released, with a completely different better installation method, and at that point the rubyforge svn will go away, and we'll be all git/github all the time. > > So: > > 1. Does this sound like a good way to proceed? Any better ones? > > 2. Scot, I think you are currently working on, well, one or even several new features for Umlaut. The "institution" one could theoretically involve some fairly major refactoring, no? If you do major refactoring in 2.x after I start on this work, it could end up being a huge pain to merge it together. Do you have any timeline on when you think you'll commit this or any other features you're in the middle of? Ideally you'd finish them soon _before_ I start, OR perhaps they won't actually involve major refactors after all, OR you'd want until Umlaut 3.0 to do em. > > We can accomodate small to maybe medium sized code changes in 2.x merged over to git 3.x, for bug fixes, or new featuers. We can accomodate new adapters, or significant changes to existing adapters, no problem. It's significant refactoring to Umlaut's core architecture that could be problematic. > > 3. Any other questions about either this project, or what's envisioned for Umlaut 3.0? > _______________________________________________ > Umlaut-general mailing list > Umlaut-general at rubyforge.org > http://rubyforge.org/mailman/listinfo/umlaut-general -- Scot Dalton Phone: (212) 998-2674 Web Services Division of Libraries New York University From rochkind at jhu.edu Mon Oct 17 11:21:51 2011 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Mon, 17 Oct 2011 11:21:51 -0400 Subject: [Umlaut-general] moving to git/github In-Reply-To: <431B9B38-415F-49E5-B1AB-A26E6FCD020C@nyu.edu> References: <4E973B71.3090901@jhu.edu> <431B9B38-415F-49E5-B1AB-A26E6FCD020C@nyu.edu> Message-ID: <4E9C480F.1030303@jhu.edu> Thanks Scot. Okay, I moved that branch to github, but my outlined plan is _not_ doing development you want to end up in 2.x on github -- there are no installation instructions for installing from github for 2.x. So it's up to you, whether you need this to be useable before we get to 3.x. If you do, then do it in svn, and I'll try to pull it over to github and merge it with my 3.x branch. If you can wait until 3.x for these features to appear, then you can, at your option, do it in github, and similarly I'll have to merge it into my 3.x branch. However, I'd also warn that I may end up needing or really wanting to change this stuff drastically as part of making Umlaut a Rails 3.x gem. Our experience with Blacklight was that auth stuff needs to change drastically when making the software a gem. So I also don't want to encourage to do lots of work that might end up wasted. As far as that goes, for making sure the 3.x version can do what you want with auth one way or another, I still really need a wiki page with an overview of what you are trying to accomplish, _why_ and _how_. What use cases you are trying to meet in what ways. Maybe that's already there somewhere I've forgotten and not seen? But I'll do my best to make sure there's a way to do what you need in 3.x; it may or may not end up being exactly what you've coded, alas. On 10/17/2011 10:09 AM, Scot Dalton wrote: > Hi Jonathan, > > Thanks for taking this on. Sounds great. > > As far as my development efforts go, I'd previously committed my changes for institutions to the auth-integration branch, which you moved to github. [1] There are a few identified bugs that I need to work out, but can commit to having those done by the end of this week and merging back to master/trunk. Should I just update the rubyforge svn trunk and then you'll use svn2git to move it over to github? Or I could just update github master and it could be a new feature of Umlaut 3.x. > > The only other enhancement in my queue is dealing with the SFX disambiguation page and trying to have Umlaut handle that more effectively, but that should only affect the SFX adapter and should be straightforward to manage. > > Let me know your thoughts. > > Thanks again, > Scot > > [1] https://github.com/team-umlaut/umlaut/tree/auth-integration > > > On Oct 13, 2011, at Oct 13, 3:26 PM, Jonathan Rochkind wrote: > >> So I think I've got some time to actually start working on the Great Umlaut Migration. The outcome of this work will be an updated Umlaut that runs as a gem/plugin under Rails 3.1, and under ruby 1.9. (Probably requiring ruby 1.9, for it's better handling of threads). Probably also clean up some un-used code and do a bit of minor additional refactoring, etc. Let's call this the future Umlaut 3.0. >> >> I don't want to do this work in svn, I've gotten too used to the much better experience of git on github. >> >> The trick is that it will of course take some amount of time for Umlaut 3.0 to be finished -- in the meantime, our current Umlaut install instructions involve a checkout from svn, and in the meantime bug fixes or new features may have to be committed to Umlaut 2.0. >> >> I think I have a way to manage this where Umlaut 2.x development will still happen in svn (so we don't need to figure out a 2.x installation method based on git instead of svn), while my 3.x development happens in git, and if we fix bugs or add features in the 2.x svn, they can still easily be merged into my 3.x development (well, easily if they don't conflict). >> >> I've already imported the svn repo into github(https://github.com/team-umlaut/umlaut), using svn2git(https://github.com/nirvdrum/svn2git), which imported complete history/tags/branches too. >> >> svn2git allows you to periodically re-sync from svn -- it's a one-way operation, you can't go from git to svn. But my plan is, the 'master/trunk' in git (and other branches from svn) (git calls it 'master', svn calls it 'trunk', same thing), I will _never_ touch in git. I will do my development in a '3.x.dev' branch in git. Periodically, if there have been changes in the 2.x svn trunk, I will re-sync git to svn (run `svn2git --rebase`); then _merge- from the sync'd git master into my git feature branch, in order to get any bugfixes or new features into the 3.x branch too. That will work fine as long as there haven't been _major_ changes to the svn 2.x version, which conflict with the 3.x refactor, and require manual merging. (and assuming svn2git works as well as it says it does; I've done just minor tests to verify 'proof of concept' here). >> >> Eventually Umlaut 3.0 will be done and released, with a completely different better installation method, and at that point the rubyforge svn will go away, and we'll be all git/github all the time. >> >> So: >> >> 1. Does this sound like a good way to proceed? Any better ones? >> >> 2. Scot, I think you are currently working on, well, one or even several new features for Umlaut. The "institution" one could theoretically involve some fairly major refactoring, no? If you do major refactoring in 2.x after I start on this work, it could end up being a huge pain to merge it together. Do you have any timeline on when you think you'll commit this or any other features you're in the middle of? Ideally you'd finish them soon _before_ I start, OR perhaps they won't actually involve major refactors after all, OR you'd want until Umlaut 3.0 to do em. >> >> We can accomodate small to maybe medium sized code changes in 2.x merged over to git 3.x, for bug fixes, or new featuers. We can accomodate new adapters, or significant changes to existing adapters, no problem. It's significant refactoring to Umlaut's core architecture that could be problematic. >> >> 3. Any other questions about either this project, or what's envisioned for Umlaut 3.0? >> _______________________________________________ >> Umlaut-general mailing list >> Umlaut-general at rubyforge.org >> http://rubyforge.org/mailman/listinfo/umlaut-general > > -- > Scot Dalton > Phone: (212) 998-2674 > Web Services > Division of Libraries > New York University > From rochkind at jhu.edu Tue Oct 18 12:47:29 2011 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Tue, 18 Oct 2011 12:47:29 -0400 Subject: [Umlaut-general] umlaut refactor ideas Message-ID: <4E9DADA1.2000002@jhu.edu> Okay, moving forward on the umlaut refactor, there are some tricks in there, but making steady progress. But it may be time to bring up some refactor ideas, so I'll have brought them up for discussion before I get to them. In general, I hope to remove a lot of cruft, un-used code, old ways of doing things, service adapters that haven't been supported/used for years, etc. Mostly this should be 'under the hood' and not effect configuration options (although HOW you do configuration will certainly change), or effect the 'contract' for Service Adapters much. (I also hope to add testing, including a testing harness for service adapters to ensure they comply with 'contract'). Here are a couple general architectural refactors I'd like to do, appreciate feedback. Both will end up simplifying umlaut code by removing some architectural flexibility that hasn't actually been used and/or has been problematic. 1. Remove Referent Caching and Re-Use Right now in Umlaut, if two different people request the 'same' OpenURL citation, both of their requests will use the same Referent object in the database. In theory, this would allow one to discover what were the most common citations requested, attach information or metadata to a citation (ie, a Referent object) that applies to anyone requesting it, etc. This has never actually been done in Umlaut 2+, but if I understand correctly that was Ross's idea. There are a couple problems here. One, it's in fact kind of tricky to tell when two OpenURLs are "the same thing", the OpenURL's can vary by still be the same thing. So hacky efforts are made to determine this. Two, sometimes an OpenURL is requested with _wrong_ information, say, the wrong OCLC number. The hacky code that tries to re-use referents for being 'nearly the same' ends up re-using a cached Referent with the wrong OCLCnum (for instance), and from them on until the Referent is cleaned from the database, the wrong OCLCnumber is there, causing false positives from catalog lookups, etc. Additional logic could be put in to try and prevent this. But the logic is getting more and more complicated, hacks on top of hacks, for flexibility that nobody's taken advantage of in the four years since Umlaut 2.x. I'd like to remove the Referent caching/re-use. Every Umlaut request will create it's own new Referent, never re-use one from another request (by the same user or different users). (The Request itself will still be cached on a per-session basis, same as it is now. But every Request will have it's own Referent, one-to-one). If later we want to add a feature to actually track 'most common' citations or attach metadata to them -- I think we'll have to use a different approach than the current one anyway, the current one is IMO a dead end. 2. Remove many-to-many ServiceResponse-to-Request, and ServiceResponse re-use Somewhat similarly, right now a ServiceResponse is attached with a many-to-many join table to an umlaut Request. This theoretically allows: A) A ServiceResponse to be re-sused among different Requests. I guess the idea here was caching, but we have never actually done that, and I don't think it's a good way to do caching. Because a ServiceResponse can end up really only applying to a certain user (A Google Books lookup can depend on the client's IP as to access). Caching of some kind is still something I want in Umlaut, but this isn't the right level/layer/way to do it, and has never been used for that. B) A single ServiceResponse can theoretically have multiple ServiceResponseTypes through the many-to-many join table. This has also not been used this way in Umlaut 2.0+ ..... a single Service can create multiple ServiceResponses, and that's what they do currently. There's no reason to make a single ServiceResponse have multiple types, and trying to do so would get complicated as to 'contracts', because what metadata belongs in a ServiceResponse depends on it's type. This is some slight performance hit using many-to-many db joins (not that big a deal if it was useful), added complexity to code (a bigger deal), and has never been used --I plan to remove this architectural feature, and instead make Request -> one-to-many -> ServiceResponse, and eliminate the join table. Both these changes shouldn't require rewriting of Service Adapters (ie Service sub-classes), as Service Adapters generally access all this stuff through particular method api, the underlying complexity was already hidden from them generally. From rochkind at jhu.edu Tue Oct 18 15:10:29 2011 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Tue, 18 Oct 2011 15:10:29 -0400 Subject: [Umlaut-general] Amazon Service change (Amazon API change) Message-ID: <4E9DCF25.7040309@jhu.edu> According to this announcement: https://affiliate-program.amazon.com/gp/advertising/api/detail/api-changes.html As of October 25, you are required to send an Affiliate ID with every request to the Amazon API that we use. However, for me at least, this has become true now (a week before October 25) -- I only know about this change in fact, because it just stopped working for me. I've updated (in Umlaut 2.x svn) the Amazon service adapter to require an "affiliate_tag" configuration parameter, and then send it on to Amazon so API requests will be accepted. It appears to me at the moment you can send any random garbage string there, so you can just make up an "affiliate_tag". For now. Who knows if it will stay that way, so probaby best to actually register for an Amazon Affiliate ID (same thing as "affiliate tag") here: https://affiliate-program.amazon.com/ I just went through the process, and it is possible to tell them that your employing library is the "beneficiary", as well as possible to simply refrain from setting up any payment options to get money from affiliate links. (I doubt it would be that much money for our uses, and I have no desire to go through the library beuracracy to figure out how to make that money to go to my employer for this chump change, plus I prefer avoiding a monetary incentive for our library to send traffic to amazon). From rochkind at jhu.edu Tue Oct 18 17:00:51 2011 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Tue, 18 Oct 2011 17:00:51 -0400 Subject: [Umlaut-general] umlaut refactoring: goodbye sru.rb and opac.rb Message-ID: <4E9DE903.4030107@jhu.edu> So opac.rb and sru.rb were an initial attempt to make a generic/abstract architecture for querying an OPAC. I don't think the files have been succesfully used or touched since Umlaut 2.0 four years ago. They probably no longer conform with Umlaut's architectural expectations. They also use some really old ruby plugins that I'd like to remove from Umlaut. And I don't want to go trying to replace the old plugins with more modern gems in code that I don't know how it works, don't know how to test it, and it probably has even bigger problems. I also think, based on my last few years experience with both Umlaut and Blacklight, that trying to write a generic opac adapter architecture like this is WAY harder than it seems, it's very hard to figure out what the right things are to standardize adn what the right things are to vary -- in actual reality, it seems so much varies that every opac adapter ends up needing to be written from scratch anyway. If we were to try to create such an abstract opac adapter framework in the future, I think we'd take an entirely different approach, perhaps trying to copy VuFind's architecture and logic into ruby. So goodbye opac.rb and sru.rb from Umalut 3.0, is the plan! From rochkind at jhu.edu Tue Oct 18 17:43:25 2011 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Tue, 18 Oct 2011 17:43:25 -0400 Subject: [Umlaut-general] umlaut refactoring: goodbye sru.rb and opac.rb In-Reply-To: <4E9DE903.4030107@jhu.edu> References: <4E9DE903.4030107@jhu.edu> Message-ID: <4E9DF2FD.30302@jhu.edu> Drat, turns out this effects the voyager.rb service adapter too, which is based on code that is based on old ruby libraries. I have no idea if the voyager adapter currently works or not either, so far as I know nobody has used it or edited it since Umlaut 2.0 four years ago. I think voyager.rb will probably also not make it into Umlaut 3.0, unless there's someone that wants to take on testing it and updating it. (Updating it will require getting rid of dependency on old style "plugins", making those plugins into gems or just putting them raw in ./lib; also ideally removing dependency on the old REXML library that nobody uses anymore using nokogiri instead, although that may not be strictly neccesary). I'm really trying to clear out code that hasn't been edited or used in four years, even if it could _hypothetically_ be useful in the future, if it hasn't been used so far... it's like throwing the clothes out of your closet you haven't worn in four years, heh. Code is a growing organism, if it's not maintained (and especially when it has no automated tests!), it's not gonna work years later when the environment around it has changed. > So opac.rb and sru.rb were an initial attempt to make a > generic/abstract architecture for querying an OPAC. > > > I don't think the files have been succesfully used or touched since > Umlaut 2.0 four years ago. They probably no longer conform with > Umlaut's architectural expectations. They also use some really old > ruby plugins that I'd like to remove from Umlaut. And I don't want to > go trying to replace the old plugins with more modern gems in code > that I don't know how it works, don't know how to test it, and it > probably has even bigger problems. > > I also think, based on my last few years experience with both Umlaut > and Blacklight, that trying to write a generic opac adapter > architecture like this is WAY harder than it seems, it's very hard to > figure out what the right things are to standardize adn what the right > things are to vary -- in actual reality, it seems so much varies that > every opac adapter ends up needing to be written from scratch anyway. > > If we were to try to create such an abstract opac adapter framework in > the future, I think we'd take an entirely different approach, perhaps > trying to copy VuFind's architecture and logic into ruby. > > So goodbye opac.rb and sru.rb from Umalut 3.0, is the plan! From m.e.phillips at durham.ac.uk Wed Oct 19 12:30:48 2011 From: m.e.phillips at durham.ac.uk (PHILLIPS M.E.) Date: Wed, 19 Oct 2011 17:30:48 +0100 Subject: [Umlaut-general] Introduction Message-ID: <1F5DB00D61AF1A479A6F8572FAC9ED8002E90C15@DURMAIL4.mds.ad.dur.ac.uk> Hello, I thought I'd introduce myself on this list. I'm the Systems Librarian at Durham University, north-east England, and we're looking into whether to implement Umlaut as a replacement front-end to WebBridge (from Innovative). We're also looking at potentially replacing the back-end resolver too, maybe with CUFTS from Simon Fraser University. So far we have Umlaut running on a test server, connecting to a couple of SFX servers. I have quite a bit of SFX experience as until a year ago I was Systems Librarian at University of Dundee, north-east Scotland. I contributed a number of bits of code to EL Commons as well as tinkering around generally improving SFX's target URL generation. Our developer, Sebastian Palucha, who mainly deals with e-prints and Fedora, is providing a lot of backup to this project, such as getting the virtual server set up properly for Ruby and so on. My next step is to look at writing a connector to hook Umlaut up to WebBridge. WebBridge doesn't actually have a suitable API to hook into, so I've spent the last couple of days writing one! The WebBridge system is fairly primitive underneath so it was possible to replicate its essential functionality in 642 lines of PHP (406 without blanks and comments) by accessing its underlying database direct. Here's an example of the result: http://bit.ly/rlZSXx Obviously, as I wrote the API myself I have full control over it, so I may find I need to adapt it once I understand more about Umlaut. We're also playing with CUFTS, which, as well as being open source already has an API. Here is an example of its output: http://bit.ly/nR5ZMQ The CUFTS system is rather nicer underneath than WebBridge, I have to say, but even so is not quite as well structured at the database level as SFX is. I've no experience of Ruby as yet: my main languages are Perl, PHP and C, so this will be an interesting experience. Obviously my first challenge will be to get the connector fetching the correct URL: I may need a bit of help with that. Is it OK if I pop up from time to time and ask initially stupid questions about, for example, how to generate debugging output so I can see what I'm doing? I'll try to Google first.... Matthew -- Matthew Phillips Electronic Systems Librarian, Durham University Durham University Library, Stockton Road, Durham, DH1 3LY +44 (0)191 334 2941 From rochkind at jhu.edu Wed Oct 19 12:55:22 2011 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Wed, 19 Oct 2011 12:55:22 -0400 Subject: [Umlaut-general] Introduction In-Reply-To: <1F5DB00D61AF1A479A6F8572FAC9ED8002E90C15@DURMAIL4.mds.ad.dur.ac.uk> References: <1F5DB00D61AF1A479A6F8572FAC9ED8002E90C15@DURMAIL4.mds.ad.dur.ac.uk> Message-ID: <4E9F00FA.8090104@jhu.edu> Hi Michael, nice to hear of your interest. Sure, ask whatever questions you want. For general ruby questions to other library coders, the #code4lib IRC channel on freenode or the code4lib listserv may also be (possibly more) helpful. I haven't actually heard of Web Bridge before, what is that? It's a link resolver? Beware, that as you've caught on, we're in the middle of migrated Umlaut to a new 3.0 version, which will entail some significant changes, once 3.0 is released you won't be able to automatically update a 2.0 Umlaut to 3.0, you'll have to re-install and re-configure. The service plug-ins should be basically the same though, a service plugin you write for 2.0 should be able to be used in 3.0 with perhaps only minor changes, is the idea. And "ruby-debug" is the gem you want for an interactive ruby debugger! Jonathan > Hello, > > I thought I'd introduce myself on this list. I'm the Systems Librarian > at Durham University, north-east England, and we're looking into whether > to implement Umlaut as a replacement front-end to WebBridge (from > Innovative). We're also looking at potentially replacing the back-end > resolver too, maybe with CUFTS from Simon Fraser University. > > So far we have Umlaut running on a test server, connecting to a couple > of SFX servers. I have quite a bit of SFX experience as until a year > ago I was Systems Librarian at University of Dundee, north-east > Scotland. I contributed a number of bits of code to EL Commons as well > as tinkering around generally improving SFX's target URL generation. > > Our developer, Sebastian Palucha, who mainly deals with e-prints and > Fedora, is providing a lot of backup to this project, such as getting > the virtual server set up properly for Ruby and so on. > > My next step is to look at writing a connector to hook Umlaut up to > WebBridge. WebBridge doesn't actually have a suitable API to hook into, > so I've spent the last couple of days writing one! The WebBridge system > is fairly primitive underneath so it was possible to replicate its > essential functionality in 642 lines of PHP (406 without blanks and > comments) by accessing its underlying database direct. > > Here's an example of the result: > > http://bit.ly/rlZSXx > > Obviously, as I wrote the API myself I have full control over it, so I > may find I need to adapt it once I understand more about Umlaut. > > We're also playing with CUFTS, which, as well as being open source > already has an API. Here is an example of its output: > > http://bit.ly/nR5ZMQ > > The CUFTS system is rather nicer underneath than WebBridge, I have to > say, but even so is not quite as well structured at the database level > as SFX is. > > I've no experience of Ruby as yet: my main languages are Perl, PHP and > C, so this will be an interesting experience. Obviously my first > challenge will be to get the connector fetching the correct URL: I may > need a bit of help with that. > > Is it OK if I pop up from time to time and ask initially stupid > questions about, for example, how to generate debugging output so I can > see what I'm doing? I'll try to Google first.... > > Matthew > From m.e.phillips at durham.ac.uk Thu Oct 20 05:02:55 2011 From: m.e.phillips at durham.ac.uk (PHILLIPS M.E.) Date: Thu, 20 Oct 2011 10:02:55 +0100 Subject: [Umlaut-general] Introduction In-Reply-To: <4E9F00FA.8090104@jhu.edu> References: <1F5DB00D61AF1A479A6F8572FAC9ED8002E90C15@DURMAIL4.mds.ad.dur.ac.uk> <4E9F00FA.8090104@jhu.edu> Message-ID: <1F5DB00D61AF1A479A6F8572FAC9ED8002E90C52@DURMAIL4.mds.ad.dur.ac.uk> It's Matthew, actually: I'm the same guy that was in touch with you direct a few days ago. WebBridge is Innovative Interfaces' OpenURL resolver which is an optional add-on to Millennium. While it integrates very well with the Millennium ERM system, as an OpenURL resolver it is fairly basic. Some folk have done great things with it by heavy use of Javascript, but I'm not particularly keen to go down that road! I'll look into ruby-debug today. Matthew > -----Original Message----- > From: umlaut-general-bounces at rubyforge.org [mailto:umlaut-general- > bounces at rubyforge.org] On Behalf Of Jonathan Rochkind > Sent: 19 October 2011 17:55 > To: umlaut-general at rubyforge.org > Subject: Re: [Umlaut-general] Introduction > > Hi Michael, nice to hear of your interest. Sure, ask whatever questions > you want. For general ruby questions to other library coders, the > #code4lib IRC channel on freenode or the code4lib listserv may also be > (possibly more) helpful. > > I haven't actually heard of Web Bridge before, what is that? It's a link > resolver? From jronallo at gmail.com Thu Oct 20 16:44:50 2011 From: jronallo at gmail.com (Jason Ronallo) Date: Thu, 20 Oct 2011 16:44:50 -0400 Subject: [Umlaut-general] umlaut refactor ideas In-Reply-To: <4E9DADA1.2000002@jhu.edu> References: <4E9DADA1.2000002@jhu.edu> Message-ID: Jonathan, It has been a while since I looked at Umlaut, so this might be a poor question. Does this in any way impact being able to have an early service adapter get more information and then allow a later service adapter to use that information? For instance if you're given and ISBN then you use something like xisbn to look up ISBNs for the same work, then later service adapters could use that fuller list of ISBNs to do lookups. Would something like that still be possible with these refactors? Jason On Tuesday, October 18, 2011, Jonathan Rochkind wrote: > Okay, moving forward on the umlaut refactor, there are some tricks in > there, but making steady progress. > > But it may be time to bring up some refactor ideas, so I'll have brought > them up for discussion before I get to them. > > In general, I hope to remove a lot of cruft, un-used code, old ways of > doing things, service adapters that haven't been supported/used for years, > etc. Mostly this should be 'under the hood' and not effect configuration > options (although HOW you do configuration will certainly change), or effect > the 'contract' for Service Adapters much. (I also hope to add testing, > including a testing harness for service adapters to ensure they comply with > 'contract'). > > > Here are a couple general architectural refactors I'd like to do, > appreciate feedback. Both will end up simplifying umlaut code by removing > some architectural flexibility that hasn't actually been used and/or has > been problematic. > > > 1. Remove Referent Caching and Re-Use > > Right now in Umlaut, if two different people request the 'same' OpenURL > citation, both of their requests will use the same Referent object in the > database. > > In theory, this would allow one to discover what were the most common > citations requested, attach information or metadata to a citation (ie, a > Referent object) that applies to anyone requesting it, etc. This has never > actually been done in Umlaut 2+, but if I understand correctly that was > Ross's idea. > > There are a couple problems here. One, it's in fact kind of tricky to tell > when two OpenURLs are "the same thing", the OpenURL's can vary by still be > the same thing. So hacky efforts are made to determine this. > > Two, sometimes an OpenURL is requested with _wrong_ information, say, the > wrong OCLC number. The hacky code that tries to re-use referents for being > 'nearly the same' ends up re-using a cached Referent with the wrong OCLCnum > (for instance), and from them on until the Referent is cleaned from the > database, the wrong OCLCnumber is there, causing false positives from > catalog lookups, etc. > > Additional logic could be put in to try and prevent this. But the logic is > getting more and more complicated, hacks on top of hacks, for flexibility > that nobody's taken advantage of in the four years since Umlaut 2.x. > > I'd like to remove the Referent caching/re-use. Every Umlaut request will > create it's own new Referent, never re-use one from another request (by the > same user or different users). (The Request itself will still be cached on a > per-session basis, same as it is now. But every Request will have it's own > Referent, one-to-one). > > If later we want to add a feature to actually track 'most common' citations > or attach metadata to them -- I think we'll have to use a different approach > than the current one anyway, the current one is IMO a dead end. > > > 2. Remove many-to-many ServiceResponse-to-Request, and ServiceResponse > re-use > > Somewhat similarly, right now a ServiceResponse is attached with a > many-to-many join table to an umlaut Request. > > This theoretically allows: > A) A ServiceResponse to be re-sused among different Requests. I guess the > idea here was caching, but we have never actually done that, and I don't > think it's a good way to do caching. Because a ServiceResponse can end up > really only applying to a certain user (A Google Books lookup can depend on > the client's IP as to access). Caching of some kind is still something I > want in Umlaut, but this isn't the right level/layer/way to do it, and has > never been used for that. > > B) A single ServiceResponse can theoretically have multiple > ServiceResponseTypes through the many-to-many join table. This has also not > been used this way in Umlaut 2.0+ ..... a single Service can create multiple > ServiceResponses, and that's what they do currently. There's no reason to > make a single ServiceResponse have multiple types, and trying to do so would > get complicated as to 'contracts', because what metadata belongs in a > ServiceResponse depends on it's type. > > This is some slight performance hit using many-to-many db joins (not that > big a deal if it was useful), added complexity to code (a bigger deal), and > has never been used --I plan to remove this architectural feature, and > instead make Request -> one-to-many -> ServiceResponse, and eliminate the > join table. > > > > Both these changes shouldn't require rewriting of Service Adapters (ie > Service sub-classes), as Service Adapters generally access all this stuff > through particular method api, the underlying complexity was already hidden > from them generally. > > > > > > ______________________________**_________________ > Umlaut-general mailing list > Umlaut-general at rubyforge.org > http://rubyforge.org/mailman/**listinfo/umlaut-general > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rossfsinger at gmail.com Thu Oct 20 18:26:31 2011 From: rossfsinger at gmail.com (Ross Singer) Date: Thu, 20 Oct 2011 23:26:31 +0100 Subject: [Umlaut-general] umlaut refactor ideas In-Reply-To: References: <4E9DADA1.2000002@jhu.edu> Message-ID: I would definitely vote for keeping the "waves of threaded services" approach Umlaut currently has. -Ross. On Thu, Oct 20, 2011 at 9:44 PM, Jason Ronallo wrote: > Jonathan, > It has been a while since I looked at Umlaut, so this might be a poor > question. Does this in any way impact being able to have an early service > adapter get more information and then allow a later service adapter to use > that information? For instance if you're given and ISBN then you use > something like xisbn to look up ISBNs for the same work, then later service > adapters could use that fuller list of ISBNs to do lookups. Would something > like that still be possible with these refactors? > Jason > > On Tuesday, October 18, 2011, Jonathan Rochkind wrote: >> >> Okay, moving forward on the umlaut refactor, there are some tricks in >> there, but making steady progress. >> >> But it may be time to bring up some refactor ideas, so I'll have brought >> them up for discussion before I get to them. >> >> In general, I hope to remove a lot of cruft, un-used code, old ways of >> doing things, service adapters that haven't been supported/used for years, >> etc. ?Mostly this should be 'under the hood' and not effect configuration >> options (although HOW you do configuration will certainly change), or effect >> the 'contract' for Service Adapters much. ?(I also hope to add testing, >> including a testing harness for service adapters to ensure they comply with >> 'contract'). >> >> >> Here are a couple general architectural refactors I'd like to do, >> appreciate feedback. Both will end up simplifying umlaut code by removing >> some architectural flexibility that hasn't actually been used and/or has >> been problematic. >> >> >> 1. Remove Referent Caching and Re-Use >> >> Right now in Umlaut, if two different people request the 'same' OpenURL >> citation, both of their requests will use the same Referent object in the >> database. >> >> In theory, this would allow one to discover what were the most common >> citations requested, attach information or metadata to a citation (ie, a >> Referent object) that applies to anyone requesting it, etc. ?This has never >> actually been done in Umlaut 2+, but if I understand correctly that was >> Ross's idea. >> >> There are a couple problems here. ?One, it's in fact kind of tricky to >> tell when two OpenURLs are "the same thing", the OpenURL's can vary by still >> be the same thing. ?So hacky efforts are made to determine this. >> >> Two, sometimes an OpenURL is requested with _wrong_ information, say, the >> wrong OCLC number. ?The hacky code that tries to re-use referents for being >> 'nearly the same' ends up re-using a cached Referent with the wrong OCLCnum >> (for instance), and from them on until the Referent is cleaned from the >> database, the wrong OCLCnumber is there, causing false positives from >> catalog lookups, etc. >> >> Additional logic could be put in to try and prevent this. But the logic is >> getting more and more complicated, hacks on top of hacks, for flexibility >> that nobody's taken advantage of in the four years since Umlaut 2.x. >> >> I'd like to remove the Referent caching/re-use. ?Every Umlaut request will >> create it's own new Referent, never re-use one from another request (by the >> same user or different users). (The Request itself will still be cached on a >> per-session basis, same as it is now. But every Request will have it's own >> Referent, one-to-one). >> >> If later we want to add a feature to actually track 'most common' >> citations or attach metadata to them -- I think we'll have to use a >> different approach than the current one anyway, the current one is IMO a >> dead end. >> >> >> 2. ?Remove many-to-many ServiceResponse-to-Request, and ServiceResponse >> re-use >> >> Somewhat similarly, right now a ServiceResponse is attached with a >> many-to-many join table to an umlaut Request. >> >> This theoretically allows: >> A) A ServiceResponse to be re-sused among different Requests. ?I guess the >> idea here was caching, but we have never actually done that, and I don't >> think it's a good way to do caching. Because a ServiceResponse can end up >> really only applying to a certain user (A Google Books lookup can depend on >> the client's IP as to access). Caching of some kind is still something I >> want in Umlaut, but this isn't the right level/layer/way to do it, and has >> never been used for that. >> >> B) A single ServiceResponse can theoretically have multiple >> ServiceResponseTypes through the many-to-many join table. ?This has also not >> been used this way in Umlaut 2.0+ ..... a single Service can create multiple >> ServiceResponses, and that's what they do currently. ?There's no reason to >> make a single ServiceResponse have multiple types, and trying to do so would >> get complicated as to 'contracts', because what metadata belongs in a >> ServiceResponse depends on it's type. >> >> This is some slight performance hit using many-to-many db joins (not that >> big a deal if it was useful), added complexity to code (a bigger deal), and >> has never been used --I plan to remove this architectural feature, and >> instead make Request -> one-to-many -> ServiceResponse, and eliminate the >> join table. >> >> >> >> Both these changes shouldn't require rewriting of Service Adapters (ie >> Service sub-classes), as Service Adapters generally access all this stuff >> through particular method api, the underlying complexity was already hidden >> from them generally. >> >> >> >> >> >> _______________________________________________ >> Umlaut-general mailing list >> Umlaut-general at rubyforge.org >> http://rubyforge.org/mailman/listinfo/umlaut-general > > _______________________________________________ > Umlaut-general mailing list > Umlaut-general at rubyforge.org > http://rubyforge.org/mailman/listinfo/umlaut-general > > From m.e.phillips at durham.ac.uk Fri Oct 21 07:05:19 2011 From: m.e.phillips at durham.ac.uk (PHILLIPS M.E.) Date: Fri, 21 Oct 2011 12:05:19 +0100 Subject: [Umlaut-general] Progress on connecting WebBridge Message-ID: <1F5DB00D61AF1A479A6F8572FAC9ED8002E90CDA@DURMAIL4.mds.ad.dur.ac.uk> I have a little service adaptor which connects to our home-made WebBridge API now. If you look at the following example: http://bit.ly/pP42PG you will see four electronic full text versions of the article. The names of the resources and the URLs are pulled from our live WebBridge OpenURL resolver. Is this the first time Umlaut has been hooked up to a resolver back-end which is not SFX? Now for a quick question: My WebBridge API does not at present do any CrossRef API lookup for DOIs. On the whole I would prefer this were handled in Umlaut so that WebBridge is passed fuller metadata. I see there is a crossref.rb service adaptor but it says at the top of the code that it is not working. If I want to go down this road, would adapting this code to make it work be a good way of going about it? I'd have to make CrossRef a higher-priority service than WebBridge, I suppose. Matthew -- Matthew Phillips Electronic Systems Librarian, Durham University Durham University Library, Stockton Road, Durham, DH1 3LY +44 (0)191 334 2941 From rochkind at jhu.edu Fri Oct 21 16:16:23 2011 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Fri, 21 Oct 2011 20:16:23 +0000 Subject: [Umlaut-general] umlaut refactor ideas In-Reply-To: References: <4E9DADA1.2000002@jhu.edu>, Message-ID: <665DBC51D0250A47B4F9306CE71E5FB70795C51A@JHEMTEBEX1.win.ad.jhu.edu> Yes, the "waves of threaded services" model definitely isn't going anywhere, and the way services work will remain substantially the same. Services can add information to the Referent object, or as ServiceResponses, and later services can use that information, and you can schedule services in waves -- none of this will be changing. For Jason's particular example, we don't have a standard data model to determine where a service would _put_ "ohter ISBNs for the same work" right now where later services would find it. I mean, we have a standard data model for information that is included in an OpenURL, but 'other ISSNs for the same work' isn't one of em. So if you wanted your particular example to work flexibly, someone would have to design how this would work, where/how the data would be stored. But that's true now, and will still be true in Umlaut 3.0, no changes with regard to your scenario are anticipated as part of Umlaut 3.0. (There are various arbitrary custom places you could put it now, but if someone else were writing a service intended to use it, they wouldn't know where to find it -- we'd need a documented standard place for this info that made sense for your scenario to work well). Also Jason, i think you are a committer on umlaut, but I coudln't figure out if you had a github account. If you do, or if you create one, let me know what it is, and I'll add you as a committer on the umlaut 3.0 github repo. ________________________________ From: umlaut-general-bounces at rubyforge.org [umlaut-general-bounces at rubyforge.org] on behalf of Jason Ronallo [jronallo at gmail.com] Sent: Thursday, October 20, 2011 4:44 PM To: umlaut-general at rubyforge.org Subject: Re: [Umlaut-general] umlaut refactor ideas Jonathan, It has been a while since I looked at Umlaut, so this might be a poor question. Does this in any way impact being able to have an early service adapter get more information and then allow a later service adapter to use that information? For instance if you're given and ISBN then you use something like xisbn to look up ISBNs for the same work, then later service adapters could use that fuller list of ISBNs to do lookups. Would something like that still be possible with these refactors? Jason On Tuesday, October 18, 2011, Jonathan Rochkind wrote: Okay, moving forward on the umlaut refactor, there are some tricks in there, but making steady progress. But it may be time to bring up some refactor ideas, so I'll have brought them up for discussion before I get to them. In general, I hope to remove a lot of cruft, un-used code, old ways of doing things, service adapters that haven't been supported/used for years, etc. Mostly this should be 'under the hood' and not effect configuration options (although HOW you do configuration will certainly change), or effect the 'contract' for Service Adapters much. (I also hope to add testing, including a testing harness for service adapters to ensure they comply with 'contract'). Here are a couple general architectural refactors I'd like to do, appreciate feedback. Both will end up simplifying umlaut code by removing some architectural flexibility that hasn't actually been used and/or has been problematic. 1. Remove Referent Caching and Re-Use Right now in Umlaut, if two different people request the 'same' OpenURL citation, both of their requests will use the same Referent object in the database. In theory, this would allow one to discover what were the most common citations requested, attach information or metadata to a citation (ie, a Referent object) that applies to anyone requesting it, etc. This has never actually been done in Umlaut 2+, but if I understand correctly that was Ross's idea. There are a couple problems here. One, it's in fact kind of tricky to tell when two OpenURLs are "the same thing", the OpenURL's can vary by still be the same thing. So hacky efforts are made to determine this. Two, sometimes an OpenURL is requested with _wrong_ information, say, the wrong OCLC number. The hacky code that tries to re-use referents for being 'nearly the same' ends up re-using a cached Referent with the wrong OCLCnum (for instance), and from them on until the Referent is cleaned from the database, the wrong OCLCnumber is there, causing false positives from catalog lookups, etc. Additional logic could be put in to try and prevent this. But the logic is getting more and more complicated, hacks on top of hacks, for flexibility that nobody's taken advantage of in the four years since Umlaut 2.x. I'd like to remove the Referent caching/re-use. Every Umlaut request will create it's own new Referent, never re-use one from another request (by the same user or different users). (The Request itself will still be cached on a per-session basis, same as it is now. But every Request will have it's own Referent, one-to-one). If later we want to add a feature to actually track 'most common' citations or attach metadata to them -- I think we'll have to use a different approach than the current one anyway, the current one is IMO a dead end. 2. Remove many-to-many ServiceResponse-to-Request, and ServiceResponse re-use Somewhat similarly, right now a ServiceResponse is attached with a many-to-many join table to an umlaut Request. This theoretically allows: A) A ServiceResponse to be re-sused among different Requests. I guess the idea here was caching, but we have never actually done that, and I don't think it's a good way to do caching. Because a ServiceResponse can end up really only applying to a certain user (A Google Books lookup can depend on the client's IP as to access). Caching of some kind is still something I want in Umlaut, but this isn't the right level/layer/way to do it, and has never been used for that. B) A single ServiceResponse can theoretically have multiple ServiceResponseTypes through the many-to-many join table. This has also not been used this way in Umlaut 2.0+ ..... a single Service can create multiple ServiceResponses, and that's what they do currently. There's no reason to make a single ServiceResponse have multiple types, and trying to do so would get complicated as to 'contracts', because what metadata belongs in a ServiceResponse depends on it's type. This is some slight performance hit using many-to-many db joins (not that big a deal if it was useful), added complexity to code (a bigger deal), and has never been used --I plan to remove this architectural feature, and instead make Request -> one-to-many -> ServiceResponse, and eliminate the join table. Both these changes shouldn't require rewriting of Service Adapters (ie Service sub-classes), as Service Adapters generally access all this stuff through particular method api, the underlying complexity was already hidden from them generally. _______________________________________________ Umlaut-general mailing list Umlaut-general at rubyforge.org http://rubyforge.org/mailman/listinfo/umlaut-general From rossfsinger at gmail.com Fri Oct 21 18:39:15 2011 From: rossfsinger at gmail.com (Ross Singer) Date: Fri, 21 Oct 2011 23:39:15 +0100 Subject: [Umlaut-general] umlaut refactor ideas In-Reply-To: <665DBC51D0250A47B4F9306CE71E5FB70795C51A@JHEMTEBEX1.win.ad.jhu.edu> References: <4E9DADA1.2000002@jhu.edu> <665DBC51D0250A47B4F9306CE71E5FB70795C51A@JHEMTEBEX1.win.ad.jhu.edu> Message-ID: On Fri, Oct 21, 2011 at 9:16 PM, Jonathan Rochkind wrote: > For Jason's particular example, we don't have a standard data model to determine where a service would _put_ "ohter ISBNs for the same work" right now where later services would find it. ?I mean, we have a standard data model for information that is included in an OpenURL, but 'other ISSNs for the same work' isn't one of em. ?So if you wanted your particular example to work flexibly, someone would have to design how this would work, where/how the data would be stored. But that's true now, and will still be true in Umlaut 3.0, no changes with regard to your scenario are anticipated as part of Umlaut 3.0. (There are various arbitrary custom places you could put it now, but if someone else were writing a service intended to use it, they wouldn't know where to find it -- we'd need a documented standard place for this info that made sense for your scenario to work well). Eh, I'm not sure if your "other ISSNs for the same work" was a typo for ISBN or not. Umlaut 1.0 did check xISBN before getting jiggy with the library catalogs (it did no ISSN concordance, which seems far less necessary), but, admittedly, I have no recollection on how that worked in practice (i.e. how it passed the ISBN list on). I'm glad the "waves of services" are going to continue and an x-identifier service is definitely one of those early level services, like Crossref. -Ross. From rochkind at jhu.edu Fri Oct 21 18:55:20 2011 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Fri, 21 Oct 2011 22:55:20 +0000 Subject: [Umlaut-general] umlaut refactor ideas In-Reply-To: References: <4E9DADA1.2000002@jhu.edu> <665DBC51D0250A47B4F9306CE71E5FB70795C51A@JHEMTEBEX1.win.ad.jhu.edu>, Message-ID: <665DBC51D0250A47B4F9306CE71E5FB70795C5E7@JHEMTEBEX1.win.ad.jhu.edu> It was a typo, although one might want to do the same thing with ISSN too. That functionality never really made it into Umlaut 2.0 (which is four years ago now, don't forget!). I'm not sure if there even is an xisbn service in umlaut 2.0,but if there is, that particular service might not make it into umlaut 3.0. (Someone else is welcome to port it, but I'm not porting things that nobody's been using in four years if they don't come over cleanly, or if I can't even figure out how to test to see if they came over cleanly. It's kind of a huge amount of work just getting the stuff people are using over to Rails 3.0! I'm finding Rails 1.0 code that was deprecated in 2.0 and stopped working in 2.3 and i've got to get it working in rails 3.1 now, heh.). I think in Umlaut 1.0, xisbn just stuffed the fetched ISBN's into a ServiceResponse in a kind of custom/undocumented way, and then other services (whcih were after all all written by you) looked for it there. In Umlaut 2.0, ServiceResponses are more clearly intended to describe things that show up on the screen, not arbitrary metadata, so it's not neccesarily appropriate to just shove it in a ServiceResponse. Although if someone really wants to, there'd be nothing stopping them from doing so, but it'd want to be docmented so someone else could write a later service to use this info and know where to look at it and not have their later service break if xisbn changes it's mind, etc. (Or if someone else wants to write a service that fetches altenrate ISBN's from a diferent source, etc). I think it probably makes more sense to hang it off a Referent somehow, then shove it in a ServiceResponse, since it's describing the referent. I wouldn't be opposed to actually making a to-many from Referent to 'other ISBNs'. Except then I started thinking, gee, you might not just want to do this with ISBN, but with ISSN, DOI, Amazon ID or OpenLibrary ID, any identifier. And then I started thinking, if you DO do that, rather than a big bucket of other identifiers, you'd really want ot say "Okay, there's this other edition X with ISBN a and Amazon ID b (those both refer to the same edition), and there's this OTHER other edition Y with ISBN m and Amazon id n".... and then it started to become unclear to me what the data model should look like exactly. One idea i had was actually linking one Referent object to another Referent object with an 'altenrate edition' relationship, since a Referent object can already hold whatever... but I dunno about that. The current Referent data model is a bit messy too. . But as with everything, if someone wants to work it out and write some code, you are welcome to! And I will if I ever end up needing it for a local priority. ________________________________________ From: umlaut-general-bounces at rubyforge.org [umlaut-general-bounces at rubyforge.org] on behalf of Ross Singer [rossfsinger at gmail.com] Sent: Friday, October 21, 2011 6:39 PM To: umlaut-general at rubyforge.org Subject: Re: [Umlaut-general] umlaut refactor ideas On Fri, Oct 21, 2011 at 9:16 PM, Jonathan Rochkind wrote: > For Jason's particular example, we don't have a standard data model to determine where a service would _put_ "ohter ISBNs for the same work" right now where later services would find it. I mean, we have a standard data model for information that is included in an OpenURL, but 'other ISSNs for the same work' isn't one of em. So if you wanted your particular example to work flexibly, someone would have to design how this would work, where/how the data would be stored. But that's true now, and will still be true in Umlaut 3.0, no changes with regard to your scenario are anticipated as part of Umlaut 3.0. (There are various arbitrary custom places you could put it now, but if someone else were writing a service intended to use it, they wouldn't know where to find it -- we'd need a documented standard place for this info that made sense for your scenario to work well). Eh, I'm not sure if your "other ISSNs for the same work" was a typo for ISBN or not. Umlaut 1.0 did check xISBN before getting jiggy with the library catalogs (it did no ISSN concordance, which seems far less necessary), but, admittedly, I have no recollection on how that worked in practice (i.e. how it passed the ISBN list on). I'm glad the "waves of services" are going to continue and an x-identifier service is definitely one of those early level services, like Crossref. -Ross. _______________________________________________ Umlaut-general mailing list Umlaut-general at rubyforge.org http://rubyforge.org/mailman/listinfo/umlaut-general From rossfsinger at gmail.com Fri Oct 21 19:16:20 2011 From: rossfsinger at gmail.com (Ross Singer) Date: Sat, 22 Oct 2011 00:16:20 +0100 Subject: [Umlaut-general] umlaut refactor ideas In-Reply-To: <665DBC51D0250A47B4F9306CE71E5FB70795C5E7@JHEMTEBEX1.win.ad.jhu.edu> References: <4E9DADA1.2000002@jhu.edu> <665DBC51D0250A47B4F9306CE71E5FB70795C51A@JHEMTEBEX1.win.ad.jhu.edu> <665DBC51D0250A47B4F9306CE71E5FB70795C5E7@JHEMTEBEX1.win.ad.jhu.edu> Message-ID: Hmm, so Umlaut currently does no ISBN concordance? -Ross. On Fri, Oct 21, 2011 at 11:55 PM, Jonathan Rochkind wrote: > It was a typo, although one might want to do the same thing with ISSN too. > > That functionality never really made it into Umlaut 2.0 (which is four years ago now, don't forget!). I'm not sure if there even is an xisbn service in umlaut 2.0,but if there is, that particular service might not make it into umlaut 3.0. (Someone else is welcome to port it, but I'm not porting things that nobody's been using in four years if they don't come over cleanly, or if I can't even figure out how to test to see if they came over cleanly. It's kind of a huge amount of work just getting the stuff people are using over to Rails 3.0! ?I'm finding Rails 1.0 code that was deprecated in 2.0 and stopped working in 2.3 and i've got to get it working in rails 3.1 now, heh.). > > I think in Umlaut 1.0, xisbn just stuffed the fetched ISBN's into a ServiceResponse in a kind of custom/undocumented way, and then other services (whcih were after all all written by you) looked for it there. > > In Umlaut 2.0, ServiceResponses are more clearly intended to describe things that show up on the screen, not arbitrary metadata, so it's not neccesarily appropriate to just shove it in a ServiceResponse. Although if someone really wants to, there'd be nothing stopping them from doing so, but it'd want to be docmented so someone else could write a later service to use this info and know where to look at it and not have their later service break if xisbn changes it's mind, etc. (Or if someone else wants to write a service that fetches altenrate ISBN's from a diferent source, etc). ?I think it probably makes more sense to hang it off a Referent somehow, then shove it in a ServiceResponse, since it's describing the referent. > > I wouldn't be opposed to actually making a to-many from Referent to 'other ISBNs'. ?Except then I started thinking, gee, you might not just want to do this with ISBN, but with ISSN, DOI, Amazon ID or OpenLibrary ID, any identifier. And then I started thinking, if you DO do that, rather than a big bucket of other identifiers, you'd really want ot say "Okay, there's this other edition X with ISBN a and Amazon ID b (those both refer to the same edition), and there's this OTHER other edition Y with ISBN m and Amazon id n".... and then it started to become unclear to me what the data model should look like exactly. One idea i had was actually linking one Referent object to another Referent object with an 'altenrate edition' relationship, since a Referent object can already hold whatever... but I dunno about that. The current Referent data model is a bit messy too. . > > But as with everything, if someone wants to work it out and write some code, you are welcome to! And I will if I ever end up needing it for a local priority. > ________________________________________ > From: umlaut-general-bounces at rubyforge.org [umlaut-general-bounces at rubyforge.org] on behalf of Ross Singer [rossfsinger at gmail.com] > Sent: Friday, October 21, 2011 6:39 PM > To: umlaut-general at rubyforge.org > Subject: Re: [Umlaut-general] umlaut refactor ideas > > On Fri, Oct 21, 2011 at 9:16 PM, Jonathan Rochkind wrote: > >> For Jason's particular example, we don't have a standard data model to determine where a service would _put_ "ohter ISBNs for the same work" right now where later services would find it. ?I mean, we have a standard data model for information that is included in an OpenURL, but 'other ISSNs for the same work' isn't one of em. ?So if you wanted your particular example to work flexibly, someone would have to design how this would work, where/how the data would be stored. But that's true now, and will still be true in Umlaut 3.0, no changes with regard to your scenario are anticipated as part of Umlaut 3.0. (There are various arbitrary custom places you could put it now, but if someone else were writing a service intended to use it, they wouldn't know where to find it -- we'd need a documented standard place for this info that made sense for your scenario to work well). > > Eh, I'm not sure if your "other ISSNs for the same work" was a typo > for ISBN or not. ?Umlaut 1.0 did check xISBN before getting jiggy with > the library catalogs (it did no ISSN concordance, which seems far less > necessary), but, admittedly, I have no recollection on how that worked > in practice (i.e. how it passed the ISBN list on). > > I'm glad the "waves of services" are going to continue and an > x-identifier service is definitely one of those early level services, > like Crossref. > > -Ross. > _______________________________________________ > Umlaut-general mailing list > Umlaut-general at rubyforge.org > http://rubyforge.org/mailman/listinfo/umlaut-general > _______________________________________________ > Umlaut-general mailing list > Umlaut-general at rubyforge.org > http://rubyforge.org/mailman/listinfo/umlaut-general > From rochkind at jhu.edu Fri Oct 21 19:30:39 2011 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Fri, 21 Oct 2011 23:30:39 +0000 Subject: [Umlaut-general] umlaut refactor ideas In-Reply-To: References: <4E9DADA1.2000002@jhu.edu> <665DBC51D0250A47B4F9306CE71E5FB70795C51A@JHEMTEBEX1.win.ad.jhu.edu> <665DBC51D0250A47B4F9306CE71E5FB70795C5E7@JHEMTEBEX1.win.ad.jhu.edu>, Message-ID: <665DBC51D0250A47B4F9306CE71E5FB70795C63C@JHEMTEBEX1.win.ad.jhu.edu> > Hmm, so Umlaut currently does no ISBN concordance? Sadly, no, and hasn't since Umlaut 2.0 some years ago. I mean, the xisbn service might have been there in 2.0, and might or might not have stopped working some time after that, but whatever data it generates and stores wherever was never used by any other services that worked in Umlaut 2.0 so far as I know. Unless the voyager one used it, that may have worked in 2.0 but probably stopped working sometime between then and now, and so far as I know nobody's used it. Sadly, several pieces of functionality never made it from Umlaut 1.0 to 2.0, just how it goes, they would have been needed to be refactored/redesigned/maintained to keep up with Umlaut, and nobody spent the time to do so, those features not being a priority for me or any other theoretical developers. ________________________________________ From: umlaut-general-bounces at rubyforge.org [umlaut-general-bounces at rubyforge.org] on behalf of Ross Singer [rossfsinger at gmail.com] Sent: Friday, October 21, 2011 7:16 PM To: umlaut-general at rubyforge.org Subject: Re: [Umlaut-general] umlaut refactor ideas Hmm, so Umlaut currently does no ISBN concordance? -Ross. On Fri, Oct 21, 2011 at 11:55 PM, Jonathan Rochkind wrote: > It was a typo, although one might want to do the same thing with ISSN too. > > That functionality never really made it into Umlaut 2.0 (which is four years ago now, don't forget!). I'm not sure if there even is an xisbn service in umlaut 2.0,but if there is, that particular service might not make it into umlaut 3.0. (Someone else is welcome to port it, but I'm not porting things that nobody's been using in four years if they don't come over cleanly, or if I can't even figure out how to test to see if they came over cleanly. It's kind of a huge amount of work just getting the stuff people are using over to Rails 3.0! I'm finding Rails 1.0 code that was deprecated in 2.0 and stopped working in 2.3 and i've got to get it working in rails 3.1 now, heh.). > > I think in Umlaut 1.0, xisbn just stuffed the fetched ISBN's into a ServiceResponse in a kind of custom/undocumented way, and then other services (whcih were after all all written by you) looked for it there. > > In Umlaut 2.0, ServiceResponses are more clearly intended to describe things that show up on the screen, not arbitrary metadata, so it's not neccesarily appropriate to just shove it in a ServiceResponse. Although if someone really wants to, there'd be nothing stopping them from doing so, but it'd want to be docmented so someone else could write a later service to use this info and know where to look at it and not have their later service break if xisbn changes it's mind, etc. (Or if someone else wants to write a service that fetches altenrate ISBN's from a diferent source, etc). I think it probably makes more sense to hang it off a Referent somehow, then shove it in a ServiceResponse, since it's describing the referent. > > I wouldn't be opposed to actually making a to-many from Referent to 'other ISBNs'. Except then I started thinking, gee, you might not just want to do this with ISBN, but with ISSN, DOI, Amazon ID or OpenLibrary ID, any identifier. And then I started thinking, if you DO do that, rather than a big bucket of other identifiers, you'd really want ot say "Okay, there's this other edition X with ISBN a and Amazon ID b (those both refer to the same edition), and there's this OTHER other edition Y with ISBN m and Amazon id n".... and then it started to become unclear to me what the data model should look like exactly. One idea i had was actually linking one Referent object to another Referent object with an 'altenrate edition' relationship, since a Referent object can already hold whatever... but I dunno about that. The current Referent data model is a bit messy too. . > > But as with everything, if someone wants to work it out and write some code, you are welcome to! And I will if I ever end up needing it for a local priority. > ________________________________________ > From: umlaut-general-bounces at rubyforge.org [umlaut-general-bounces at rubyforge.org] on behalf of Ross Singer [rossfsinger at gmail.com] > Sent: Friday, October 21, 2011 6:39 PM > To: umlaut-general at rubyforge.org > Subject: Re: [Umlaut-general] umlaut refactor ideas > > On Fri, Oct 21, 2011 at 9:16 PM, Jonathan Rochkind wrote: > >> For Jason's particular example, we don't have a standard data model to determine where a service would _put_ "ohter ISBNs for the same work" right now where later services would find it. I mean, we have a standard data model for information that is included in an OpenURL, but 'other ISSNs for the same work' isn't one of em. So if you wanted your particular example to work flexibly, someone would have to design how this would work, where/how the data would be stored. But that's true now, and will still be true in Umlaut 3.0, no changes with regard to your scenario are anticipated as part of Umlaut 3.0. (There are various arbitrary custom places you could put it now, but if someone else were writing a service intended to use it, they wouldn't know where to find it -- we'd need a documented standard place for this info that made sense for your scenario to work well). > > Eh, I'm not sure if your "other ISSNs for the same work" was a typo > for ISBN or not. Umlaut 1.0 did check xISBN before getting jiggy with > the library catalogs (it did no ISSN concordance, which seems far less > necessary), but, admittedly, I have no recollection on how that worked > in practice (i.e. how it passed the ISBN list on). > > I'm glad the "waves of services" are going to continue and an > x-identifier service is definitely one of those early level services, > like Crossref. > > -Ross. > _______________________________________________ > Umlaut-general mailing list > Umlaut-general at rubyforge.org > http://rubyforge.org/mailman/listinfo/umlaut-general > _______________________________________________ > Umlaut-general mailing list > Umlaut-general at rubyforge.org > http://rubyforge.org/mailman/listinfo/umlaut-general > _______________________________________________ Umlaut-general mailing list Umlaut-general at rubyforge.org http://rubyforge.org/mailman/listinfo/umlaut-general From rochkind at jhu.edu Fri Oct 21 19:49:13 2011 From: rochkind at jhu.edu (Jonathan Rochkind) Date: Fri, 21 Oct 2011 23:49:13 +0000 Subject: [Umlaut-general] umlaut refactor ideas In-Reply-To: <665DBC51D0250A47B4F9306CE71E5FB70795C63C@JHEMTEBEX1.win.ad.jhu.edu> References: <4E9DADA1.2000002@jhu.edu> <665DBC51D0250A47B4F9306CE71E5FB70795C51A@JHEMTEBEX1.win.ad.jhu.edu> <665DBC51D0250A47B4F9306CE71E5FB70795C5E7@JHEMTEBEX1.win.ad.jhu.edu>, , <665DBC51D0250A47B4F9306CE71E5FB70795C63C@JHEMTEBEX1.win.ad.jhu.edu> Message-ID: <665DBC51D0250A47B4F9306CE71E5FB70795C657@JHEMTEBEX1.win.ad.jhu.edu> There's no xisbn service in current umlaut 2.x, it turns out. I tried to look through repo history to see when there last was one, but I couldn't find it. It may never have even made it into initial umlaut 2.0 at all. ________________________________________ From: Jonathan Rochkind Sent: Friday, October 21, 2011 7:30 PM To: umlaut-general at rubyforge.org Subject: RE: [Umlaut-general] umlaut refactor ideas > Hmm, so Umlaut currently does no ISBN concordance? Sadly, no, and hasn't since Umlaut 2.0 some years ago. I mean, the xisbn service might have been there in 2.0, and might or might not have stopped working some time after that, but whatever data it generates and stores wherever was never used by any other services that worked in Umlaut 2.0 so far as I know. Unless the voyager one used it, that may have worked in 2.0 but probably stopped working sometime between then and now, and so far as I know nobody's used it. Sadly, several pieces of functionality never made it from Umlaut 1.0 to 2.0, just how it goes, they would have been needed to be refactored/redesigned/maintained to keep up with Umlaut, and nobody spent the time to do so, those features not being a priority for me or any other theoretical developers. ________________________________________ From: umlaut-general-bounces at rubyforge.org [umlaut-general-bounces at rubyforge.org] on behalf of Ross Singer [rossfsinger at gmail.com] Sent: Friday, October 21, 2011 7:16 PM To: umlaut-general at rubyforge.org Subject: Re: [Umlaut-general] umlaut refactor ideas Hmm, so Umlaut currently does no ISBN concordance? -Ross. On Fri, Oct 21, 2011 at 11:55 PM, Jonathan Rochkind wrote: > It was a typo, although one might want to do the same thing with ISSN too. > > That functionality never really made it into Umlaut 2.0 (which is four years ago now, don't forget!). I'm not sure if there even is an xisbn service in umlaut 2.0,but if there is, that particular service might not make it into umlaut 3.0. (Someone else is welcome to port it, but I'm not porting things that nobody's been using in four years if they don't come over cleanly, or if I can't even figure out how to test to see if they came over cleanly. It's kind of a huge amount of work just getting the stuff people are using over to Rails 3.0! I'm finding Rails 1.0 code that was deprecated in 2.0 and stopped working in 2.3 and i've got to get it working in rails 3.1 now, heh.). > > I think in Umlaut 1.0, xisbn just stuffed the fetched ISBN's into a ServiceResponse in a kind of custom/undocumented way, and then other services (whcih were after all all written by you) looked for it there. > > In Umlaut 2.0, ServiceResponses are more clearly intended to describe things that show up on the screen, not arbitrary metadata, so it's not neccesarily appropriate to just shove it in a ServiceResponse. Although if someone really wants to, there'd be nothing stopping them from doing so, but it'd want to be docmented so someone else could write a later service to use this info and know where to look at it and not have their later service break if xisbn changes it's mind, etc. (Or if someone else wants to write a service that fetches altenrate ISBN's from a diferent source, etc). I think it probably makes more sense to hang it off a Referent somehow, then shove it in a ServiceResponse, since it's describing the referent. > > I wouldn't be opposed to actually making a to-many from Referent to 'other ISBNs'. Except then I started thinking, gee, you might not just want to do this with ISBN, but with ISSN, DOI, Amazon ID or OpenLibrary ID, any identifier. And then I started thinking, if you DO do that, rather than a big bucket of other identifiers, you'd really want ot say "Okay, there's this other edition X with ISBN a and Amazon ID b (those both refer to the same edition), and there's this OTHER other edition Y with ISBN m and Amazon id n".... and then it started to become unclear to me what the data model should look like exactly. One idea i had was actually linking one Referent object to another Referent object with an 'altenrate edition' relationship, since a Referent object can already hold whatever... but I dunno about that. The current Referent data model is a bit messy too. . > > But as with everything, if someone wants to work it out and write some code, you are welcome to! And I will if I ever end up needing it for a local priority. > ________________________________________ > From: umlaut-general-bounces at rubyforge.org [umlaut-general-bounces at rubyforge.org] on behalf of Ross Singer [rossfsinger at gmail.com] > Sent: Friday, October 21, 2011 6:39 PM > To: umlaut-general at rubyforge.org > Subject: Re: [Umlaut-general] umlaut refactor ideas > > On Fri, Oct 21, 2011 at 9:16 PM, Jonathan Rochkind wrote: > >> For Jason's particular example, we don't have a standard data model to determine where a service would _put_ "ohter ISBNs for the same work" right now where later services would find it. I mean, we have a standard data model for information that is included in an OpenURL, but 'other ISSNs for the same work' isn't one of em. So if you wanted your particular example to work flexibly, someone would have to design how this would work, where/how the data would be stored. But that's true now, and will still be true in Umlaut 3.0, no changes with regard to your scenario are anticipated as part of Umlaut 3.0. (There are various arbitrary custom places you could put it now, but if someone else were writing a service intended to use it, they wouldn't know where to find it -- we'd need a documented standard place for this info that made sense for your scenario to work well). > > Eh, I'm not sure if your "other ISSNs for the same work" was a typo > for ISBN or not. Umlaut 1.0 did check xISBN before getting jiggy with > the library catalogs (it did no ISSN concordance, which seems far less > necessary), but, admittedly, I have no recollection on how that worked > in practice (i.e. how it passed the ISBN list on). > > I'm glad the "waves of services" are going to continue and an > x-identifier service is definitely one of those early level services, > like Crossref. > > -Ross. > _______________________________________________ > Umlaut-general mailing list > Umlaut-general at rubyforge.org > http://rubyforge.org/mailman/listinfo/umlaut-general > _______________________________________________ > Umlaut-general mailing list > Umlaut-general at rubyforge.org > http://rubyforge.org/mailman/listinfo/umlaut-general > _______________________________________________ Umlaut-general mailing list Umlaut-general at rubyforge.org http://rubyforge.org/mailman/listinfo/umlaut-general