[rspec-users] [rspec] loading specs as an object model without running them

[Marty Andrews]

On 29/04/2009, at 9:41 PM, David Chelimsky wrote:

> On Wed, Apr 29, 2009 at 6:09 AM, Marty Andrews  
> <marty at martyandrews.net> wrote:
>>
>> On 29/04/2009, at 8:54 PM, David Chelimsky wrote:
>>
>>> On Wed, Apr 29, 2009 at 1:23 AM, Marty Andrews <marty at martyandrews.net 
>>> >
>>> wrote:
>>>>
>>>> I'd like to load all of my examples into memory without actually
>>>> running them, so I can traverse the object tree and get  
>>>> descriptions
>>>> etc.  I'm finding it tricky to figure out how to do that though.
>>>> There seems to be an inherent assumption in the framework that that
>>>> examples will actually run.
>>>>
>>>> So basically, I want to do something like this:
>>>>
>>>> examples = load_examples_without_running
>>>> examples.each {|example| puts example.description}
>>
>> [snip]
>>
>>> As for getting a handle on the objects, this is something that is  
>>> not
>>> really baked into RSpec, but will definitely be baked into  
>>> rspec-2.0.
>>> Work on this will start later this year (probably not until after  
>>> the
>>> rspec book is off to print). Goals for it include a very clean
>>> separation of the DSL and the underlying object model and a very
>>> lightweight runner.
>>>
>>> So in the short run, your best option is to use --dry-run and
>>> understand that you only get value out of it if you write docstrings
>>> for all of your examples.
>>
>> Ok.  It's actually a bit more complicate than that.  Let me give  
>> you an
>> insight into where I'm headed.
>>
>> I'm working into a big corporate in Australia, and they are demanding
>> traceability of tests back to requirements.  I'm already using  
>> Mingle to
>> manage story cards, and I'm hoping to let the testers build in the
>> traceability via code.  So I've got a Mixin to the examples so that  
>> I can
>> declare the id of the story that the test maps to.  It looks  
>> something like
>> this:
>>
>> describe FunctionalArea do
>>  it "automates some acceptance test declared on a story" do
>>    traces_to_story 35
>>    # test implementation
>>  end
>> end
>>
>> So now, I want to be able to walk the object tree, getting the  
>> descriptions
>> *and* the story id's.  Mingle has a nice RESTful API that I've  
>> already used
>> to slurp up detail about stories.  Now I just want to wire in this  
>> last bit
>> of information to tick the traceability box without any human  
>> needing to do
>> the work.
>>
>> I'll have a look at the --dry-run option and see where it gets me.   
>> If
>> you've got any more tips, let me know.  I'll send in a code snippet  
>> if I
>> manage to get it going :)
>
> There *is* something you can *almost* use for this, but it's
> incomplete at the moment. RSpec let's you write your own output
> formatters, which receive messages for every example group and example
> as they are run. Currently, the object passed to example_started (and
> a few other methods) is an ExampleGroupProxy, which includes an
> options hash passed to the example. So if you do this:
>
> it "automates some acceptance test declared on a story",
>    :traces_to_story => 25 do
> # ...
> end
>
> Then you could do stuff like:
>
> def example_started(example_proxy)
>  traces_to_story(example_proxy.options[:traces_to_story]
> end
>
> This should solve your problem assuming that you have docstrings for
> all of your examples. The big missing piece at this moment is that the
> ExampleGroupProxy (passed to example_group_started) doesn't include
> the options hash, but I'll happily add that tonight.

Yep - no problems there.  I reckon about 98% have docstrings.  I can  
easily add the remainders.



> To see this in action, try this:
>
> * in options_formatter.rb
>
> require 'spec/runner/formatter/base_text_formatter'
>
> class OptionsFormatter < Spec::Runner::Formatter::BaseTextFormatter
>  def example_started(proxy)
>    puts proxy.options.inspect unless proxy.options.empty?
>  end
> end
>
> * in example_spec.rb
>
> require 'spec'
>
> describe "example" do
>  it "supports options", :like => 'these' do
>    # no need for anything here
>  end
> end
>
> Now, on the command line:
>
> $ spec --require options_formatter.rb example_spec.rb --format  
> OptionsFormatter
> {:like => 'these'}
>
> Formatter API is here:
> http://rspec.rubyforge.org/rspec/1.2.4/classes/Spec/Runner/Formatter/BaseFormatter.html
>
> Let me know if this is a good direction for you and I'll add the
> options hash to the example group proxy.

I reckon this would be great *if* it will still work with the --dry- 
run option.  Most of the examples are actually integration tests that  
call out to other tools as they run, so the build is S..L..O..W (over  
an hour).  I want to be able to generate the report without actually  
running the tests.

-- Marty