From lewis at grasscommons.org Thu Aug 21 02:02:22 2008 From: lewis at grasscommons.org (Lewis Hoffman) Date: Thu, 21 Aug 2008 01:02:22 -0500 Subject: [Wagn-developers] wagn on github Message-ID: <7c8bd97e0808202302q4a936ba2k2d93ecc585168b03@mail.gmail.com> Hi folks, wagn is on github: http://github.com/wagn/wagn I've finally gotten on the git bandwagn with another project and love it, so I'll be switching my Wagn development over to git and pushing out to the rubyforge svn repo periodically. Lewis -- Lewis Hoffman 312.560.0786 lewis at grasscommons.org Technical Director :: Grass Commons http://grasscommons.org :: http://wagn.org :: http://hooze.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lewis at grasscommons.org Wed Aug 27 16:23:18 2008 From: lewis at grasscommons.org (Lewis Hoffman) Date: Wed, 27 Aug 2008 15:23:18 -0500 Subject: [Wagn-developers] performance, caching, testing, and refactoring. Message-ID: <7c8bd97e0808271323n2ce54b4x2fd9e2399758327a@mail.gmail.com> Hey folks, I wanted to share some of my recent thoughts on wagn performance, caching, testing, and refactoring. performance Wagn needs to get a lot faster. The challenge of performance in Wagn is that on a given page there may be lists of cards and deep nesting of cards and basically lots and lots of cards. Currently we cache the content of individual cards-- *before* processing transclusions (I'll refer to this as "model caching"). Each cache entry is atomic, so to render the page you have to process the transclusions and go back and forth to the cache for each card one at a time. So it can take a long time to get a page ready to deliver. view caching view caching is caching big chunks of content at a time so that when accessing a given view most of the transclusions are already processed. We've done a good chunk of design in this area; but Ethan and I both feel that it is scary. Managing the cache expiry is a complex problem; transclusions, templates, virtual cards, searches all have to be taken into account. It's complex and it cross-cuts most of the operations in the system. testing wagn behavior One way to reduce the scariness of view caching is good testing. To approximate BDD in a view caching implementation we would first creating a thorough behaviour spec at a high level that exercises the complete application stack (which the current wagn should mostly pass) and then build the caching layer so that it doesn't break that spec. I've been reading up on rspec stories ( http://rspec.info/documentation/stories.html ) and think they might be a good implementation medium for this. refactoring rendering & model caching Another way to reduce scariness is to increase the robustness of the current codebase. I think part of what is scary about adding the complexity of view-caching is that our current codebase is approaching it's complexity carrying capacity. By way of framing, I want to say that the underlying model API of Wagn feels pretty solid. It's a fairly clear design, reasonably well tested, and has proven itself by not needing much tweaking or bugfixing for some time. The rendering and caching layers however, feel somewhat ad-hoc and tangled, and are poorly tested. A quick run of flog [1] confirms that these areas are in some of the worst pain, and they have certainly been a source of bugs. semi-conclusion i think that an implementation of view-caching is most likely to succeed if we lay the proper ground-work for it and break things into phases: 1. refactoring and testing the rendering & model caching layers to reduce complexity and brittleness 2. building a comprehensive behavioural spec for wagn 3. implementing view caching each of these is a big job. the two groundwork phases each could have signification benefit in and of themselves. alternative to view caching : browser side processing A potential alternative to view caching that Ethan and I have discussed is browser side transclusion processing. This would not approach the potential speed of a view-cached system, but could be a big improvement over the current user experience-- the basic frame of the page would come up quickly, and things would gradually be filled in by ajax calls to the server. In this approach, I believe the comprehensive behavioral spec would not be as crucial, and obviously the implementation of view caching is unnecessary. So that leaves us with just refactoring the rendering & model caching layers. However, many of the rendering duties would migrate into javascript, which in my experience is a significantly slower development environment than ruby on the server. what do you think? Ethan is coming to visit Chicago next week so we should have a chance to haggle over some of these issues in person. If you have thoughts, questions, or comments that could feed into our haggling (or if you'd like to be involved in some way) that would be great! thanks! Lewis 1. flog ( http://ruby.sadi.st/Flog.html ) is a utility that gives a measure of code complexity. results of flogging wagn: lhoffman at spud:~/dev/wagn/wagn$ flog app | egrep "\#| = " | head [..several warnings omitted..] Total score = 6447.06767882241 Slot#render: (147.0) User#create_with_card: (133.3) CachedCard#get: (124.3) Slot#wrap: (84.3) WagnHelper#remote_function: (75.4) StubSlot#render_transclusion_view: (68.5) CardController#remove: (66.5) validates_each#name: (62.4) TrackedAttributes#set_permissions: (60.8) -- Lewis Hoffman 312.560.0786 lewis at grasscommons.org Technical Director :: Grass Commons http://grasscommons.org :: http://wagn.org :: http://hooze.org -------------- next part -------------- An HTML attachment was scrubbed... URL: