[webgen-devel] [TL] some doubts

Thomas Leitner t_leitner at gmx.at
Wed Sep 19 08:46:29 EDT 2007

From: Thomas Leitner <t_leitner at gmx.at>
Date: 23. August 2007 10:08:35 GMT+02:00
To: Andrea Ferro
Subject: Re: some doubts

> *) Perplexity one: logging
> The WebGen logging system currently implements logging as a method  
> that is added to plugin objects after they are initialized. And the  
> logging level is "globally set". This has several consequences.
> -) logging can only happen in the code of a plugin. What if a  
> plugin defines a "specialized node" and wants to log from the node?
> -) logging cannot happen from the initialize method (or from  
> methods called by initialize) of a plugin, as the log method is  
> "mixed in" after the plugin is initialized. This may be ok since  
> "real" plugin initialization should be delayed to the call to  
> init_plugin.
> -) logging cannot happen from code that is not in a plugin, hence  
> not (at least not with the usual syntax) from code of WebSite,  
> PluginManager etc.
> I think this is somehow suboptimal. I think Webgen should provide a  
> globally usefull logging facility. Logging from code in WebSite and  
> PluginManager (and even from the CommandParser) should be possible  
> (I know there's no logging statement as of now, but still it should  
> be possible). And more importantly it should be possible plugins  
> code that is not in the plugin class itself (for example in the  
> specialized node code).
> Ideally (at least as a future enhancement) it should be possible to  
> set different logging levels. Like info level logging for the main  
> system and debug level for a subsystem (for example for all File/*  
> plugins, or even for only one specific plugin).
> What do you think?

The logging code has also changed from each major version to the  
next. I think I was using some external logging library before I  
switched to the logger provided by the Ruby stdlib. The problem with  
logging is that you can either add a global logger to each object or  
that you need to provide the logger in an other way. Since I tried to  
make it possible to use several different webgen instances in one  
Ruby process, I don't like the 'global logger' approach. The other  
thing with logging is the problem with setting the logging level  
(also see your next point). When should it be set? It needs to be set  
after all configurators are in place and the correct value can be  
determined. This means that some code needs to run before setting the  
logging level.

It would be perfect if we can get the logging system to work with not  
plugin objects AND ensure proper logging level initialisation.  
However, I do not want to go the Rails approach - one process, one  
instance. It should be possible to create multiple Webgen::WebSite  
objects for different webgen websites within one instance.

One additional problem with the logging level: all parameters set for  
plugins are only needed on demand, ie. the plugins specifically ask  
for the value of a parameter. However, the logger never asks for the  
logging level parameter since then it needs to ask every now and then  
to recognize any changes in the parameter. So a facility that  
automatically executes code when a parameters value changes, would be  
perfect for setting the logging level. No need anymore for explicitly  
setting the logging level when the configurators have been assigned  
to the plugin manger instance. Good/bad idea?

> *) Perplexity two: initialization sequence
> Currently CommandParser::parse in cli.rb creates a WebSite object  
> then adds configurators to it's plugin_manager then sets the log  
> level on the plugin manager logger and finally goes for command  
> execution.
> But WebSite initializator (actually reset, called by the  
> initializator) calls load_all_plugin_bundles on it's plugin_manager  
> after creating it.
> This means that the search for plugins and the loading of plugin  
> bundles is done before all configurators are added to the  
> plugin_manager and before the final log level is set.
> This logic is fine only if:
> - logging cannot happen during the initial setup. And that is true  
> now, but not if we accept my previous comments on logging.
> - configurators should not be used untill the end of the initial  
> setup. This may imply some limitation on what the initial setup  
> code can do an on what information it has access to. For example if  
> in the future we want to support specification of custom names for  
> the site's "src" and "plugin" directories in the configuration file  
> for a a given site, the current logic will fail.
> A good example of the way the current logic may fail in unwanted/ 
> unexpected ways is in, one of the new features you pushed this  
> weekend: support for plugin autoloading.
> In the way it's done right now, autoload plugins will be  
> initialized during the initial setup (hence before the  
> configurators are in place) while other plugins will be initialized  
> later. This means the plugin's specific init_plugin method will be  
> invoked under different conditions depending if the plugin is  
> autoloaded or loaded on demand. Specifically for autoloaded plugins  
> the init_plugin should not do anything that requires the presence  
> of the configurators in the plugin manager as they have not yet  
> been set up by the CommandParse::parse method. And they should not  
> do logging as the correct logging level was not yet setup.
> Am I right or am I missing something?

Yes, you are right, that's a problem. The configurators should be in  
place before any plugin gets loaded and initialized. However, this  
should be easily fixable by adpating Webgen::WebSite and the code  
part in cli.rb.

The cli.rb is the current interface for running webgen. However,  
since everything related to a webgen website is/should be  
encapsulated in a Webgen::WebSite instance, it should also be  
possible to create other front ends for webgen, a Rake task being of  

> *) Perplexity three: testing
> I see you do unit tests. I've given only a cursory look at the test  
> code (and at all the plugins code. So far I'm concentrating to the  
> files under lib/webgen except for test.rb).
> I'm not sure if your development strategy is to write test cases  
> before or after the actual code and if you test your code by only  
> doing the tests (with rake?) or by also running the program from  
> the command line (bin/webgen).
> Also I'm not sure if the tests do a complete coverage or not.
> I've been working on many different kinds of projects. Sometimes  
> with no unit testing at all. Sometimes with unit tests and some  
> testing support code (this seems current state for webgen). Once  
> even with some effort toward Test Driven Development.
> I'm personally ok with whatever strategy the project does  
> implement. But of course I need to know. So I'm asking you: what's  
> the current strategy? How do you want me to handle the testing?

I have ported about half of the test code from the webgen 0.4.x  
series to 0.5.0 and I'm using rake to execute the test (rake test).  
The file lib/webgen/test.rb is used to provide some testing support  
code which also allows to run a single test.

I have the following bash function in my .bashrc:

webgen-devel ()
     export RUBYOPT="-rubygems -I~/work/webgen/devel/lib";
     cd ~/work/webgen/devel

After executing this function the following is possible and will work  

   rake test
   cd data/webgen/plugins/
   ruby contentprocessor/blocks.plugin/test/unittests/tc_blocks.rb

This makes running individual test easier.

The tests don't do a complete coverage currently. I have tried to  
cover every important method in lib/webgen/*.rb and some important  
plugin methods. Also, I have no fixed order when I write a test case,  
sometimes before the actual code, sometimes after it. When fixing a  
bug, I always try to modify the test code first to catch the bug and  
then I fix the code until the tests go green. Since the tests don't  
cover everything I'm also regularly running webgen from the command  
line to ensure it still works properly (which isn't such a good thing  
but I didn't have time to write tests for everything).

> *) what I think I can do now
> Rethink the logging support and change the code accordingly. And of  
> course document the new logging facility (besides that's probably  
> an aspect that even now is not very well documented).
> Rethink and change the way the initialization/loading is done,  
> making each piece of information/functionality available as soon as  
> possible, meanwhile checking/updating and improving the  
> documentation of these aspects of the system.
> Either keep the testing files updated according to the current  
> logic (whatever that is). Or begin steering toward BDD my  
> integrating rSpec (see my last mail) and start writing rSpec  
> specifications (they also are valid unit tests) for existing code.  
> In this latter case we are not actually doing things in a proper  
> BDD (Behaviour Driven Development) way as we were supposed to write  
> the behavior specification before writing the implementation code.  
> But integrating rSpec and beginning to add specifications is still  
> a first (and necessary) step if you want to "steer" webgen toward  
> Of course I'll also fix any bugs I'll happen to stumble upon.

I have looked at RSpec a month or so ago and it really looks  
promising. If you think that using RSpec will make testing easier or  
that it is a better approach, I'm okay with it and will have a look  
at it. So just go ahead with BDD :)

Concerning documentation: I want to separate developer and user  
documentation, meaning that a webgen or plugin developer should find  
everything in the RDoc documentation and the user should find  
everything he needs in the user documentation generated by webgen and  
available on webgen.rubyforge.org. I'm still struggling a little bit  
with the general structure of the user documentation, however, I will  
try to add a first draft as soon as possible.

> Sorry if I did not write you earlier, but this period is a vacancy  
> period and as I told you there's a friend that is here for a few  
> days and I'm spending most time with him (but in this very moment  
> I'm backupping his PowerBook internal disk in preparation for an  
> upgrade to ... Tiger :)

Why Tiger when Leopard is on the door step?! ;-)

-- Thomas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/webgen-devel/attachments/20070919/45724069/attachment-0001.html 

More information about the webgen-devel mailing list