[rspec-users] customize description and failure messages when writing rspec matcher dsl

David Chelimsky dchelimsky at gmail.com
Sun Nov 27 12:43:47 EST 2011


On Nov 27, 2011, at 7:22 AM, Anran Yang wrote:

> On Nov 24, 2011, at 12:48 AM, Anran Yang wrote:
> 
> > Dear all,
> >
> > I recently tried to write a custom matcher using rspec dsl, which I'd like to use as the following:
> >
> > ==========hpgc_spec.rb===================
> >
> > require "spec_helper"
> >
> > describe Hpgc do
> >   specify do
> >     should be_structured_as(
> >     { :app => {
> >         :has => [
> >             :name,
> >             :author,
> >             :org,
> >             :version,
> >             :abstract,
> >             :uri
> >         ],
> >
> >         :can_be_a => {
> >             :dataset => {
> >                 :can_be_a => {
> >                     :vector => {},
> >                     :raster => {}
> >                 }
> >             },
> >
> >             :program => {
> >                 :has => [
> >                     :language,
> >                     :parallel_programming_model,
> >                     :parallel_performance_overview,
> >                     :category
> >                 ]
> >             }
> >         },
> >
> >         :has_many => {
> >             :app_option => {
> >                 :has => [
> >                     :switch,
> >                     :description
> >                 ]
> >             },
> >
> >             :app_io => {
> >                 :has => [
> >                     :name,
> >                     :description,
> >                     :is_in
> >                 ]
> >             }
> >         }
> >     }})
> >   end
> > end
> >
> > ===========================================
> >
> > The purpose is to test the entire model structure(attributes, inheritance and association).
> 
> Then term Behavior Driven Development came into being, in part, to discourage this approach to testing. We believe testing should be about behavior, and that testing structures leads to highly coupled tests that are hard to change.
> 
> I'd recommend thinking about the need for each of these attributes, come up with examples of how they are used and write those instead. That said, see below for more on how matchers work.
> 
> ---------------------------------
> It made me thinking a while. But at last I found that I've thought about it, though it may only happen unconsciously. So below is why I need this kind of spec.
> I must admit the Spec is against BDD's spirit. However, I think BDD may not be enough for all kinds of projects in all kinds of circumstance. In my project, the model itself is also a product, not only the "surface" of the software. Maybe the best way to accomplish this is to separate this kind of test and the "real" BDD test. But rspec can support them perfectly so I'm lazy this time. 
> Maybe there are some other projects involved in this kind of situation? I mean, some projects with a concrete domain model, so to some extent the development is driven by both behavior and knowledge?

The specific problem is that tests about structure are more highly coupled to structure than behavior, and therefore more brittle. This is true whether you wrote the test first or wrote it after, and regardless of what methodology you are thinking of when you write the tests.

I don't understand what you mean when you say 'In my project, the model itself is also a product, not only the "surface" of the software.'. Can you elaborate?

> > Then I wrote following code to implement it:
> >
> > ========be_structured_as.rb===================
> >
> > # This is a matcher for test hpgc model structure
> >
> > RSpec::Matchers.define :be_structured_as do |structure|
> >   def check(namespace, klass, desc)
> >     (desc[:has_many] ||= {}).keys.each { |item| klass.to_s.camelize.constantize.new.should have_many item.to_s.pluralize.to_sym }
> >     if !desc[:can_be_a] || desc[:can_be_a].empty?
> >       obj = Factory.create(klass)
> >       obj.should have_these_attributes (desc[:has] ||= [])
> >     else
> >       desc[:can_be_a].keys.map { |sub| sub.to_s }.should == namespace.subclasses_of(klass.to_s)
> >       desc[:can_be_a].each { |key, value| check namespace, key, value }
> >     end
> >   end
> >
> >   match do |namespace|
> >     self.class.send :include, namespace
> >     structure.each { |key, value| check namespace, key, value }
> >   end
> > end
> >
> > ===============================================
> >
> > To simplify the matcher definition,  I used some other machers like "obj.should have_these_attributes (desc[:has] ||= [])", this works fine, thanks to the excellent new matcher dsl syntax. Howerver, I got trouble when running the test using "rspec spec/models/hpgc_spec.rb -fd". The test passed but print
> >
> > <Screenshot.png>
> >
> > instead of something like "Hpgc should be structured as ...". I think it may due to the other matcher I used because the message is from the matcher "have_these_attributes".
> 
> That's correct. The default behavior is that the block passed to the "match" method in the matcher DSL is expected to return a boolean. If it raises an exception instead, that exception bubbles up and becomes the source of the failure message.
> 
> > I try to add method "description" but seems it doesn't work.
> 
> "description" is used for the documentation formatter, not for failure messages. If you run "rspec spec --format documentation" you'll see names for all your examples.
> 
> --------------------------
> Maybe I failed express my problem clearly, but the question is for pass message, not failure message. Knowing how rspec raise exception help little to get my spec say "Hpgc should be structured as... " when it passes. The method "description" failed to present itself even when the spec passed, as long as there is an inner "should" clause.

You can define your own description method in the same way:

description do |actual|
  #  ....
end

> > And the failure message is also a problem. I search the web and come across some solutions, but all of them need manually build the message. I've not come out a clean way to do this(I must add some invasive clauses to track the error) and think maybe the best behaviour would be that it "throw" the failure message of inner matchers or false-value expressions. But this time it just told me something like "Hpgc is expected to be structured as ...".
> 
> Instead of using the "match" method in the matcher, you can use "match_unless_raises", which returns false if it captures an exception, letting the matcher control the failure message (which is what I think you're looking for):
> 
> RSpec::Matchers.define :be_structured_as do |structure|
>  def check(namespace, klass, desc)
>    # ....
>  end
> 
>  match_unless_raises do |namespace|
>    self.class.send :include, namespace
>    structure.each { |key, value| check namespace, key, value }
>  end
> end
> 
> HTH,
> David
> 
> 
> ----------------------------------------------------------------
> Is the method in rspec-expectation? I searched the local doc but didn't find it.

It is in rspec-expectations but it is not documented. I added it to support matchers in rspec-rails that delegate to minitest assertions (which raise their own exception), and never took the time to formalize it as a public API. That said, it should be considered public, will not change without appropriate warning, and will be documented in the rdoc for the 2.8 release.

> Also I don't quite understand how the method can help track the failing message. It return false, but I still cannot figure out which line fails, isn't it?

It stores the exception from the failure in an instance variable named @rescued_exception, which you can access through a method named rescued_exception e.g.

failure_messsage_for_should do |actual|
  "Expected .... but something happened\n#{rescued_exception.message}\n#{rescued_exception.backtrace}"
end

Another approach would be to keep track of each thing individually using the standard matches method:

match do |actual|
  @reasons == []
  @reasons << "no foo" unless foo
  @reasons << "no bar" unless bar
  @reasons << "no baz" unless baz
  @reasons.empty?
end

That returns false if any conditions fail, but doesn't raise an exceptions. Then you can use @reasons in the failure message, e.g.

failure_message_for_should do |actual|
  "Failed because #{@reasons.join(', ')}"
end

> Thank you for your inspiring reply and tolerance to my poor English :-)

Your English is perfectly understandable.

Cheers,
David

> Regards,
> YANG
> 
> > Any idea?
> >
> > Best Regards,
> >
> > YANG Anran


More information about the rspec-users mailing list