[Alexandria-list] Contributing

costanti at science.unitn.it costanti at science.unitn.it
Fri Mar 30 09:55:03 EDT 2007


Dear all

Joseph Method <tristil at gmail.com> wrote:

> I was holding off on describing what my ideas for Alexandria's needs
> are because I wanted to get a pure bugfix release out first.

I think that the main problem to solve before releasing is the problem with
charset encoding, and after having fixed that, Alexandria 0.7 can be released;
however, I invite everyone to have a look at the bugs at
http://rubyforge.org/tracker/?atid=863&group_id=205&func=browse , and try to
fix them. There is for instance [#9666], an harmless problem, which may be
confusing for the new user.

Now Alexandria uses internally only UTF-8, however, if a user has, in the
~/.alexandria directory, some data saved by a previous version of Alexandria,
there may be strings with a different encoding, which cause Alexandria to crash
when a book is manually entered or modified, because it tries to provide
autocompletion.
In order to solve this problem, I propose the following.

* Use the ruby function GLib.UTF8_Validate , see
http://ruby-gnome2.sourceforge.jp/hiki.cgi?GLib%3A%3AUTF8 to check whether the
existing  strings are valid UTF-8 strings. Otherwise, the strings should
considered ISO-8859-1 (which is the most used by the books providers) and
converted to UTF-8.

* Make the autocompletion optional, by adding among the options, the one that
enables autocompletion.

* Add a version number in the ~/.alexandria/*/*.yaml files, I mean something
like

creator: "Alexandria"
version: "7.0"

Thereafter, Alexandria can check whether the data was saved by an older version,
and perform the check only in that case.
Also the "creator" may be useful, because Tellico too, or another program, can
export its data to Alexandria format.
By the way, I wrote about the encoding of the ~/.alexandria/*/*.yaml files to
Robby Stephenson, the author of Tellico, a program similar to Alexandria that
can import the Alexandria data. He asked me:

 "Would it be feasible to add some sort of version string in each data file?
 That way, you could easily check to see if the file was written by an old
 version of Alexandria, and deal accordingly."

See also http://www.catb.org/~esr/writings/taoup/html/ch01s06.html#id2879112
about having a version number in the files.

> I'll send
> that post to the mailing list soon.  Basically, Alexandria itself in
> the short term needs to handle data migration (moving books en masse
> over to the new 13-digit isbns) in a way that preserves cover images
> and doesn't destroy people's libraries, and provide for a migration
> path for future versions.

For that, it is necessary to manually check each occurrence of
Library.canonicalise_isbn and in case replace it with Library.canonicalise_ean.
There are some other problems, for instance, when exporting to ONIX, the XML
output contains the ISBN-13 in the ISBN field, which doesn't conform to the
ONIX standard (see Feature request [#862]).


> In the long term, we need to change to a
> sqlite database backend and do major code re-factoring (for example,
> code needs to get out of main_app.MainApp.initialize). Then we should
> focus on taking functionality *out* of Alexandria and putting it into
> standalone targeted applications that operate on a common database. We
> should also work to make Alexandria's data accessible and useful in
> other applications, like Tomboy and beagle/deskbar-applet. Anyways,
> that's just my idea, and I'll expand on the details later.

In my opinion, changing to sqlite is not so necessary nor urgent; consider also
that the Yaml format is easy, simple, textual, and already works well. See also
"The Importance of Being Textual" at
http://www.catb.org/~esr/writings/taoup/html/ch05s01.html . Requiring Sqlite is
another dependency.


The main task in the long term is, in my opinion, to separate the core from the
graphical user interface (GUI), which is probably the same idea behind "code
needs to get out of main_app.MainApp.initialize".

There should be a core as a standalone library, which handles getting the data
from the providers, saving it in yaml or sqlite, and so on.
Then this library can be accessed by the GUI, or by a command line interface, or
by a web interface, or by scripts or by other programs, such as Tellico or
GCstar. These other programs could then use Alexandria for searching the book
providers.

The attached image should explain this idea; I also strongly encourage to read
http://www.catb.org/~esr/writings/taoup/html/ch11s07.html about this. In this
way, the standalone library would not depend on Gnome or Gtk, but will be
available one each machine on which Ruby runs, that is, even on Windows.

Feature request [#1774] asks for a "Web interface: Is there any chance of
creating a web interface, rather than requiring a graphical toolkit?"
See http://www.catb.org/~esr/writings/taoup/html/ch11s08.html about having a web
interface

Alexandria core can be already used by other programs. For instance, the
attached (rudimental) script, which is called as

ruby al.rb 9782213617268

searches the book with ISBN 9782213617268 using the Alexandria providers, and
returns data as a piece of Tellico format. Some work will be required to adapt
the already implemented  code for exporting to Tellico format, and thereafter
Alexandria core can be used by other programs such as Tellico and CGstar,
greatly increasing the number of users of (this part of) Alexandria.

The Alexandria core could even be a Z39.50 server, so that one can let other
people check his/her books.


On the other hand, the Alexandria GUI could access not only the standalone
library, but also other programs.

Bugs item [#4888] asks " I want to ask if I could use the GUI etc. for a own
version using pyAmazon, which I know to work perfectly with all amazon
websites.. Would be a nice project :) ..?"

Among other programs that could be accessed, there is Thokbook. For lists of
possible other programs, see
http://discuss.joelonsoftware.com/default.asp?joel.3.378798.142 and
http://periapsis.org/tellico/ after ALTERNATIVES .

One of these programs, GCstar http://www.gcstar.org/ , can be accessed quickly
and soon. GCstar can already be used as a standalone library, and this command:

gcstar -x -c GCbooks -e Tellico -w Fnac --download  9782213617268

search the book with ISBN 9782213617268 at provider Fnac, and return the data
about in the Tellico format (not compressed). Alexandria can already use the
Tellico format (zipped), hence with some minor modifications in Alexandria we
can get all the book providers already implemented in GCstar. Tellico can
already use GCstar.


After having reorganized the code, having separated the GUI and the core, and
having separated the part that handles the saved data from the rest, it will be
possible and useful to provide alternatives to saving the data as yaml files.

This could be done by letting the user choose among the yaml files (as now),
Sqlite (as Joseph proposes) MySQL (as requested by some users) and so on
(PostgreSQL, Tellico format, ...).

Feature request [#2551] says that "would be nice to have the books stored in a
SQL/MySQL database instead of files."
Feature request [#1099] says: "Another option is to talk to a MySQL DB, but now
I'm just dreaming. :)"

For all these tasks, volunteers are very welcome.

Cheers,
Marco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: polyvalent.png
Type: image/png
Size: 5426 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/alexandria-list/attachments/20070330/e8ac4cd9/attachment.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: al.rb
Type: application/x-ruby
Size: 811 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/alexandria-list/attachments/20070330/e8ac4cd9/attachment.bin 


More information about the Alexandria-list mailing list