Feature Requests: Browse | Submit New | Admin

[#28767] pluggable parsing for meta-programming

Date:
2010-12-05 16:34
Priority:
3
Submitted By:
Aidan Cully (aidancully)
Assigned To:
Nobody (None)
Category:
None
State:
Open
Summary:
pluggable parsing for meta-programming

Detailed description
It should be possible for gems that expose meta-programming operations to clients to automatically allow those clients
to document the meta-programmed results.  For example, consider the Struct class: it ought to be possible for the parser
to automatically detect that a user has derived from Struct, and document the attr_accessor fields in the resulting
class:

#!/usr/bin/ruby

require 'pp'
require 'rubygems'
gem 'rdoc'
require 'rdoc/options'
require 'rdoc/parser/ruby'
require 'rdoc/stats'

RBTXT = <<END
  class Foo
    attr_accessor :bar
  end

  Struct.new("Bar", :baz)
END

RDoc::TopLevel.reset
top_level = RDoc::TopLevel.new "foo.rb"
stats = RDoc::Stats.new 0
parser = RDoc::Parser::Ruby.new(top_level, "foo.rb", RBTXT, [], stats)
parser.parse_statements(top_level)
pp RDoc::TopLevel.find_class_named("Foo").attributes
# => [#<RDoc::Attr:0x2d76e8 Foo.attr_accessor :bar>]
pp RDoc::TopLevel.find_class_named("Struct::Bar")
# => nil
eval RBTXT
pp Foo
# => Foo
pp Struct::Bar
# => Struct::Bar

I also have a gem that meta-programs methods as follows:

class Foo
  class Bar
    Safer::IVar.instance_variable(self, :baz)
    # equivalent to attr_accessor :foo_bar__baz
  end
end

Documenting these fields relies on the :attr_accessor: markup in the comment, but I don't want to require that, as users
must duplicate the name of the attr in the markup, which hurts the documentation process.  (In fact, there's a bug in
current rdoc that makes the :attr_accessor: markup not work, in that it attempts to parse Safer::IVar(self, :baz) as
a constant, fails, and discards the comment.  This is filed as bug #27793.)

A way to support both of these cases is to:
1. allow the parser infrastructure to have infinite look-ahead and rollback when attempting to parse expressions;
2. change the parser so that, whenever it is going to make a decision about what element is getting documented, it attempts
all possibilities, using the best match; and
3. allow other gems to add to the set of possible parse methods.

To match the "Struct" case above, we would define an expression rule that expects "Struct" ; (method
arguments), and for the first method argument to be a string.  To match my Safer::IVar, define an expression rule that
expects (Const && const.text == "Safer") COLON2 (Const && const.text == "IVar")
PERIOD (method arguments), and method argument[0] == "self".

To be determined is how to decide which "match" is best, when there are multiple possible matches...  I don't
think this decision necessarily needs to be made now.

I am willing to do the work to implement this, but would like to know if there is any interest, first.

Add A Comment: Notepad

Please login


Followup

Message
Date: 2011-01-05 11:40
Sender: Aidan Cully

FWIW, I also noticed that Forwardable's def_delegator (from the
standard
library) won't be supported by RDoc.  (Neither will Delegator,
but that's a
tougher nut to crack.)  So that's at least two places in the
standard library
which have a readily recognizable syntax, which aren't handled
as
transparently by RDoc as straight (that is, non-meta-programmed)
Ruby.

I agree that metaprogrammed methods support allows such things
to be
documented.  But the fact that users have to use a different,
higher-effort
documentation scheme for metaprogrammed methods will inhibit
their use of
metaprogramming, or their documentation of the metaprogrammed
methods.
What I'd like to see is the ability for gem authors that provide
metaprogramming facilities to take that additional documentation
burden
onto themselves, away from users.
Date: 2011-01-02 23:10
Sender: Eric Hodel

I think the latter case is adequately covered by metaprogrammed
methods handling.  Struct handling should be added.

Attached Files:

Name Description Download
No Files Currently Attached

Changes:

No Changes Have Been Made to This Item