[Mongrel] [ANN] e c30ac536947f7330943f8de9c33f70ef2d5994e7

cr _ at whats-your.name
Thu Aug 16 05:18:59 EDT 2007


e, elemental, is a stack for the data web

there are 4 components: earth, air, fire, water

earth is a pure-ruby RDF triple-store, with a fs backend. no dependencies on 3rd party databases, just add filesystem (tm). theres also a ram backend built with the Mongrel URI-classifier trie as the primitive datastructure. from these two it should be easy to extrapolate how to write a memcached/hadoop backend, or whatever is your cup of tea. ive found reiser4 works quite well.. the main inspiration behind the store design is git.

the API for the store should be familiar to users of jQuery.. as with one class wrapping a DOM node, you get one class wrapping a RDF resource. the resource can be an actual file on the filesystem - eg throw a dir of jpgs off your cam into a web dir and start annotating them.. literals are any valid JSON data, and resource versioning is available.

a JSON query engine is included, which queries like this:

  {  type:Type,
    'type=':'http://weblog.com/story',
     date:Modified,
    'abs?':dc+'terms/abstract',
    'title?':dc+'elements/1.1/title',
    'content?':sioc+'content',
     limit:7,
     offset:0,
     order: {p:'date',d:'desc'}}

as well as plain keyword search of literals. the underlying indices are visible in and implemented using the store itself.

designed with optimizability /  hackability in mind, to invent the partitioning/distribution to meet your needs, adjust the editing method to send all edits to an Utu chan or RDF-over-JSON-over-Jabber to viewers, mirror data to a beastly SPARQL engine, etc.

  http://whats-your.name/e/e.tar.gz


water is the mongrel layer. it provides a default HTML view for crawlers, links, dillo, and pays attention to the headers to export RDF to tools that want it like tabulator, triplr, virtuoso. everything is cached for both server (serialized resource) and client (if-modified/etag). the base URI is mapped to the root of the server for your domain.

embedded in the oldschool HTML is RDFa which is used to bootstrap a base javascript GUI aka 'fire'. you'll likely want to customize this for your app, although i do plan on providing a more complete set of base tools - graphical query editor and some sample scripts as a seperate package for things like blogs, wikis, bug tracs. facilitating GUIs is a small set of jQuery extensions, 'air' to get form fields auto-watched for changes and updating triples on the server, JS DOM creation + bindings (closures and js vars instead of jQuery selectors to avoid the fragility and traversal, if you want). ive tested the base gUI and proven it can at least load and make edits in firefox2/3, opera, ie(s4linux), and webkit(qt/gdk), but not this week.

theres no global lock preventing concurrency, no class vars and only a couple instance vars (could make these go away, but this isnt haskell) . so far the only thing somewhat slow is ruby's sort_by. taking about 0.1 second for 30-50 mb of blog post data sorting on date. query results can be cached, but im going to investigate the performance and mem-hoggage of a RAM store for the index before rewriting everything in Lua.

the roadmap is add a few missing features: recursive queries (eg, get the 10 newest blog posts, and the 10 most recent comments for each of these), and OpenID-based accounts (local-only is supported currently). is there a mongrel-openID gem?

oh yeah, docs/tutorials will come some day, maybe..


More information about the Mongrel-users mailing list