From t_leitner at gmx.at  Fri Oct  1 02:55:07 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Fri, 1 Oct 2010 08:55:07 +0200
Subject: [kramdown-users] [ANN] kramdown 0.11.0 released
Message-ID: <20101001085507.30f864d4@noweto>

## About kramdown

kramdown (sic, not Kramdown or KramDown, just kramdown) is a *free*
GPL-licensed [Ruby](http://www.ruby-lang.org) library for parsing a
superset of Markdown. It is completely written in Ruby, supports
standard Markdown (with some minor modifications) and various
extensions that have been made popular by the [PHP Markdown Extra]
package and [Maruku].

Homepage for installation instructions and documentation:
   http://kramdown.rubyforge.org


## kramdown 0.11.0 released

The biggest change in this release is the implementation of the "lazy
syntax" which allows one to not use the correct indent or block marker
and still continue a paragraph, blockquote, ... The original Markdown
syntax allows this and it was requested that kramdown allows this, too.
However, the main reason for adding this syntax to kramdown is not to
encourage authors to be lazy but to allow kramdown texts to be
hard-wrapped by other applications (think, for example, email
programs). Therefore you shouldn't make active use of this feature when
creating a kramdown document!

Another important, though minor, change is that invalid HTML tags and
extensions are not removed anymore. This is done because of the general
rule that unrecognized elements are treated as simple text.

**Note**: The "internal" API (which is currently everything except the
`Kramdown::Document` class) has changed and developers may therefore
need to update their extensions!


## Changes

* 3 major changes:

  - Line wrapping a.k.a. "lazy syntax" is now supported (requested by
    Shawn Van Ittersum)
  - Link URLs in inline links and link definitions may now also contain
    spaces, even if not enclosed in angle brackets (requested by Matt
    Neuburg)
  - The kramdown converter produces nicer output, using the new option
    `line_width`

* 9 minor changes:

  - The HTML converter does not escape the quotation mark in code
    blocks anymore (requested by Matt Neuburg)
  - The order of HTML attributes and attributes defined via IALs and
    ALDs is now preserved (requested by Matt Neuburg)
  - Syntax highlighting is now supported in code spans when using the
    HTML converter (requested by Josh Cheek)
  - Updated `nomarkdown` extension and converters to support
    restricting the output to certain or all converters
  - Colons are now allowed in ID names for ALDs and IALs
  - Tables and math blocks now have to start and end on block boundaries
  - The table syntax was relaxed to allow table lines that don't start
    with a pipe character (to be more compatible with PHP Markdown
    Extra tables)
  - HTML elements `<b>` and `<i>` are now converted to `<strong>` and
    `<em>` when using HTML-to-native conversion
  - The `document.html` template now uses the text of the first
    not-nested header as title text

* 9 bug fixes:

  - The LaTeX converter now removes trailing whitespace in footnotes
    (reported by Michael Franzl)
  - Fixed bug RF#28429: HTML output of `iframe` HTML element was
    invalid (reported by Matthew Riley)
  - Fixed bug RF#28420: LaTeX converter shouldn't escape the content of
    the `nomarkdown` extension (reported by Bj Wilson)
  - Fixed bug RF#28469: HTML "document" template did not work (reported
    by Vofa Ethe)
  - Fixed bug: HTML/kramdown output of `textarea` HTML element was
    invalid (reported by John Muhl)
  - Invalid or unknown extension tags are now left alone and not
    removed anymore
  - Invalid HTML tags are now left alone and not removed anymore
  - Fixed a minor problem in list parsing which arised due to compact
    nested list detection
  - Link/Abbreviation/Footnote definitions as well as extensions, ALDs
    and block IALs now work correctly as block separators

* 1 deprecation note:

  - The option `numeric_entities` has been removed


From matt at tidbits.com  Sat Oct  9 17:55:28 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Sat, 09 Oct 2010 14:55:28 -0700
Subject: [kramdown-users] block inline attribute list problem
Message-ID: <C8D632E0.1DFD%matt@tidbits.com>

This works as I expect:

require 'kramdown'
s = <<END
{: #myid}
Testing
END
puts Kramdown::Document.new(s).to_html


<p id="myid">Testing</p>

This, however, does nothing:

require 'kramdown'
s = <<END
{: #myid}
<div>
Testing
</div>
END
puts Kramdown::Document.new(s).to_html


<div>
Testing
</div>

I expect the div to gain the id value "myid", but it doesn't. Okay, so
maybe the problem is that markdown is not turned on for this div. Let's
try it:


require 'kramdown'
s = <<END
{: #myid}
<div markdown="1">
Testing
</div>
END
puts Kramdown::Document.new(s).to_html


<div>
  <p id="myid">Testing</p>
</div>


Hmmm. We've skipped the div and attached the id to the internal p.

On the other hand, it all works perfectly if the ALD is *after* the
closing </div>, regardless of whether markdown="1" or not. So, there's a
good workaround, but there is some expectation here that it will work the
same whether the ALD is before or after its block, and it doesn't. m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com





From matt at tidbits.com  Sun Oct 10 11:34:04 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Sun, 10 Oct 2010 08:34:04 -0700
Subject: [kramdown-users] kramdown table-making has gone completely insane
Message-ID: <C8D72AFC.1EA4%matt@tidbits.com>

require 'kramdown'
s = <<END
The operator `||` is called logical-or.
END
puts Kramdown::Document.new(s).to_html


What I expect:

<p>The operator....

What I get:

<table>
  <tbody>
    <tr>
      <td>The operator `</td>
      <td>&nbsp;</td>
      <td>` is called logical-or.</td>
    </tr>
  </tbody>
</table>

There are actually two errors here. One is that &nbsp; must never be
emitted, since I am using kramdown in an XSLT-processing context and
&nbsp; is not a legal XML entity. Asking for :numeric output of entities
makes no difference.


However, the obvious larger error is that the table-parser has suddenly
taken over and thinks this is a table. This is a new error (i.e. since the
last release I was using). m.

Workarounds: none found! I can't think what to do aside from trying to
revert to an earlier version of kramdown. m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com





From svicalifornia at gmail.com  Sun Oct 10 12:03:38 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Mon, 11 Oct 2010 03:03:38 +1100
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <C8D72AFC.1EA4%matt@tidbits.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
Message-ID: <20101010230338632089.15573510@gmail.com>

More generally, I think that kramdown should ignore the contents of any code block or span.  I can't think of a good reason for kramdown to interpret the contents of code blocks.  If it didn't, then this table creation problem would go away.

Shawn

On Sun, 10 Oct 2010 08:34:04 -0700, Matt Neuburg wrote:
> require 'kramdown'
> s = <<END
> The operator `||` is called logical-or.
> END
> puts Kramdown::Document.new(s).to_html
> 
> 
> What I expect:
> 
> <p>The operator....
> 
> What I get:
> 
> <table>
>   <tbody>
>     <tr>
>       <td>The operator `</td>
>       <td>&nbsp;</td>
>       <td>` is called logical-or.</td>
>     </tr>
>   </tbody>
> </table>
> 
> There are actually two errors here. One is that &nbsp; must never be
> emitted, since I am using kramdown in an XSLT-processing context and
> &nbsp; is not a legal XML entity. Asking for :numeric output of entities
> makes no difference.
> 
> 
> However, the obvious larger error is that the table-parser has suddenly
> taken over and thinks this is a table. This is a new error (i.e. since the
> last release I was using). m.
> 
> Workarounds: none found! I can't think what to do aside from trying to
> revert to an earlier version of kramdown. m.
> 
> -- 
> matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
> pantes anthropoi tou eidenai oregontai phusei
> Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
> AppleScript: the Definitive Guide, 2nd edition
> http://www.tidbits.com/matt/default.html#applescriptthings
> Take Control of Exploring & Customizing Snow Leopard
> http://tinyurl.com/kufyy8
> RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
> TidBITS, Mac news and reviews since 1990, http://www.tidbits.com
> 
> 
> 
> 
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From matt at tidbits.com  Mon Oct 11 01:18:49 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Sun, 10 Oct 2010 22:18:49 -0700
Subject: [kramdown-users] kramdown table-making has gone completely
	insane
In-Reply-To: <20101010230338632089.15573510@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
Message-ID: <362DA23E-6F4E-469F-93A5-22BCB347F04F@tidbits.com>

Presumably, that is what used to happen, since I've had that line in my page for months and many versions of kramdown. This behavior is evidently new with this version of kramdown. (I'm sorry I can't be more specific to say what I mean by "this version", but the previous two versions that I was using were both called 0.11.0 and so is this one, even though they are all three quite different from each other.) 

The docs do say, in fact, "Note that literal pipe characters need to be escaped, even if they occur in code spans!" Now, maybe that was never there before - if it was there, I never noticed it. But in fact I could not find any way of escaping these pipe characters so as to solve the problem. I mean, look, there are only three possibilities: escape the first, escape the second, and escape both. But none of those prevents the problem. m.


On Oct 10, 2010, at 9:03 AM, Shawn Van Ittersum wrote:

> More generally, I think that kramdown should ignore the contents of any code block or span.  I can't think of a good reason for kramdown to interpret the contents of code blocks.  If it didn't, then this table creation problem would go away.
> 
> Shawn
> 
> On Sun, 10 Oct 2010 08:34:04 -0700, Matt Neuburg wrote:
>> require 'kramdown'
>> s = <<END
>> The operator `||` is called logical-or.
>> END
>> puts Kramdown::Document.new(s).to_html
>> 
>> 
>> What I expect:
>> 
>> <p>The operator....
>> 
>> What I get:
>> 
>> <table>
>>  <tbody>
>>    <tr>
>>      <td>The operator `</td>
>>      <td>&nbsp;</td>
>>      <td>` is called logical-or.</td>
>>    </tr>
>>  </tbody>
>> </table>
>> 
>> There are actually two errors here. One is that &nbsp; must never be
>> emitted, since I am using kramdown in an XSLT-processing context and
>> &nbsp; is not a legal XML entity. Asking for :numeric output of entities
>> makes no difference.
>> 
>> 
>> However, the obvious larger error is that the table-parser has suddenly
>> taken over and thinks this is a table. This is a new error (i.e. since the
>> last release I was using). m.
>> 
>> Workarounds: none found! I can't think what to do aside from trying to
>> revert to an earlier version of kramdown. m.
>> 
>> -- 
>> matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
>> pantes anthropoi tou eidenai oregontai phusei
>> Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
>> AppleScript: the Definitive Guide, 2nd edition
>> http://www.tidbits.com/matt/default.html#applescriptthings
>> Take Control of Exploring & Customizing Snow Leopard
>> http://tinyurl.com/kufyy8
>> RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
>> TidBITS, Mac news and reviews since 1990, http://www.tidbits.com
>> 
>> 
>> 
>> 
>> _______________________________________________
>> kramdown-users mailing list
>> kramdown-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/kramdown-users
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users


From sunshine at sunshineco.com  Mon Oct 11 01:38:26 2010
From: sunshine at sunshineco.com (Eric Sunshine)
Date: Mon, 11 Oct 2010 01:38:26 -0400
Subject: [kramdown-users] kramdown table-making has gone
	completely	insane
In-Reply-To: <362DA23E-6F4E-469F-93A5-22BCB347F04F@tidbits.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>	<20101010230338632089.15573510@gmail.com>
	<362DA23E-6F4E-469F-93A5-22BCB347F04F@tidbits.com>
Message-ID: <4CB2A2D2.3060501@sunshineco.com>

The leading '|' on table lines became optional here: 
http://github.com/gettalong/kramdown/commit/5354385d6bec8c89e4d8c2c2b76f6d07433800c7

-- ES


On 10/11/2010 1:18 AM, Matt Neuburg wrote:
> Presumably, that is what used to happen, since I've had that line in my page for months and many versions of kramdown. This behavior is evidently new with this version of kramdown. (I'm sorry I can't be more specific to say what I mean by "this version", but the previous two versions that I was using were both called 0.11.0 and so is this one, even though they are all three quite different from each other.)
>
> The docs do say, in fact, "Note that literal pipe characters need to be escaped, even if they occur in code spans!" Now, maybe that was never there before - if it was there, I never noticed it. But in fact I could not find any way of escaping these pipe characters so as to solve the problem. I mean, look, there are only three possibilities: escape the first, escape the second, and escape both. But none of those prevents the problem. m.
>
>
> On Oct 10, 2010, at 9:03 AM, Shawn Van Ittersum wrote:
>
>> More generally, I think that kramdown should ignore the contents of any code block or span.  I can't think of a good reason for kramdown to interpret the contents of code blocks.  If it didn't, then this table creation problem would go away.
>>
>> Shawn
>>
>> On Sun, 10 Oct 2010 08:34:04 -0700, Matt Neuburg wrote:
>>> require 'kramdown'
>>> s =<<END
>>> The operator `||` is called logical-or.
>>> END
>>> puts Kramdown::Document.new(s).to_html
>>>
>>>
>>> What I expect:
>>>
>>> <p>The operator....
>>>
>>> What I get:
>>>
>>> <table>
>>>   <tbody>
>>>     <tr>
>>>       <td>The operator `</td>
>>>       <td>&nbsp;</td>
>>>       <td>` is called logical-or.</td>
>>>     </tr>
>>>   </tbody>
>>> </table>
>>>
>>> There are actually two errors here. One is that&nbsp; must never be
>>> emitted, since I am using kramdown in an XSLT-processing context and
>>> &nbsp; is not a legal XML entity. Asking for :numeric output of entities
>>> makes no difference.
>>>
>>>
>>> However, the obvious larger error is that the table-parser has suddenly
>>> taken over and thinks this is a table. This is a new error (i.e. since the
>>> last release I was using). m.
>>>
>>> Workarounds: none found! I can't think what to do aside from trying to
>>> revert to an earlier version of kramdown. m.
>>>
>>> --
>>> matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
>>> pantes anthropoi tou eidenai oregontai phusei
>>> Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
>>> AppleScript: the Definitive Guide, 2nd edition
>>> http://www.tidbits.com/matt/default.html#applescriptthings
>>> Take Control of Exploring&  Customizing Snow Leopard
>>> http://tinyurl.com/kufyy8
>>> RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
>>> TidBITS, Mac news and reviews since 1990, http://www.tidbits.com
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> kramdown-users mailing list
>>> kramdown-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/kramdown-users
>> _______________________________________________
>> kramdown-users mailing list
>> kramdown-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/kramdown-users
>
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users
>

From matt at tidbits.com  Mon Oct 11 02:01:12 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Sun, 10 Oct 2010 23:01:12 -0700
Subject: [kramdown-users] kramdown table-making has gone
	completely	insane
In-Reply-To: <4CB2A2D2.3060501@sunshineco.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>	<20101010230338632089.15573510@gmail.com>
	<362DA23E-6F4E-469F-93A5-22BCB347F04F@tidbits.com>
	<4CB2A2D2.3060501@sunshineco.com>
Message-ID: <FE67E9F9-B72D-440A-851A-5EE9A180ABB5@tidbits.com>

Thanks, Eric!

I guess, taking all of this into consideration, my suggestion on this would be (1) escaping should work (2) nbsp; should not be emitted if I've asked for numeric entities (3) it might be really cool if it were possible optionally to turn OFF this new feature, i.e. table lines without leading pipes.

Let's see what Thomas thinks. m.

On Oct 10, 2010, at 10:38 PM, Eric Sunshine wrote:

> The leading '|' on table lines became optional here: http://github.com/gettalong/kramdown/commit/5354385d6bec8c89e4d8c2c2b76f6d07433800c7
> 
> -- ES
> 
> 
> On 10/11/2010 1:18 AM, Matt Neuburg wrote:
>> Presumably, that is what used to happen, since I've had that line in my page for months and many versions of kramdown. This behavior is evidently new with this version of kramdown. (I'm sorry I can't be more specific to say what I mean by "this version", but the previous two versions that I was using were both called 0.11.0 and so is this one, even though they are all three quite different from each other.)
>> 
>> The docs do say, in fact, "Note that literal pipe characters need to be escaped, even if they occur in code spans!" Now, maybe that was never there before - if it was there, I never noticed it. But in fact I could not find any way of escaping these pipe characters so as to solve the problem. I mean, look, there are only three possibilities: escape the first, escape the second, and escape both. But none of those prevents the problem. m.
>> 
>> 
>> On Oct 10, 2010, at 9:03 AM, Shawn Van Ittersum wrote:
>> 
>>> More generally, I think that kramdown should ignore the contents of any code block or span.  I can't think of a good reason for kramdown to interpret the contents of code blocks.  If it didn't, then this table creation problem would go away.
>>> 
>>> Shawn
>>> 
>>> On Sun, 10 Oct 2010 08:34:04 -0700, Matt Neuburg wrote:
>>>> require 'kramdown'
>>>> s =<<END
>>>> The operator `||` is called logical-or.
>>>> END
>>>> puts Kramdown::Document.new(s).to_html
>>>> 
>>>> 
>>>> What I expect:
>>>> 
>>>> <p>The operator....
>>>> 
>>>> What I get:
>>>> 
>>>> <table>
>>>>  <tbody>
>>>>    <tr>
>>>>      <td>The operator `</td>
>>>>      <td>&nbsp;</td>
>>>>      <td>` is called logical-or.</td>
>>>>    </tr>
>>>>  </tbody>
>>>> </table>
>>>> 
>>>> There are actually two errors here. One is that&nbsp; must never be
>>>> emitted, since I am using kramdown in an XSLT-processing context and
>>>> &nbsp; is not a legal XML entity. Asking for :numeric output of entities
>>>> makes no difference.
>>>> 
>>>> 
>>>> However, the obvious larger error is that the table-parser has suddenly
>>>> taken over and thinks this is a table. This is a new error (i.e. since the
>>>> last release I was using). m.
>>>> 
>>>> Workarounds: none found! I can't think what to do aside from trying to
>>>> revert to an earlier version of kramdown. m.
>>>> 
>>>> --
>>>> matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
>>>> pantes anthropoi tou eidenai oregontai phusei
>>>> Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
>>>> AppleScript: the Definitive Guide, 2nd edition
>>>> http://www.tidbits.com/matt/default.html#applescriptthings
>>>> Take Control of Exploring&  Customizing Snow Leopard
>>>> http://tinyurl.com/kufyy8
>>>> RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
>>>> TidBITS, Mac news and reviews since 1990, http://www.tidbits.com
>>>> 
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> kramdown-users mailing list
>>>> kramdown-users at rubyforge.org
>>>> http://rubyforge.org/mailman/listinfo/kramdown-users
>>> _______________________________________________
>>> kramdown-users mailing list
>>> kramdown-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/kramdown-users
>> 
>> _______________________________________________
>> kramdown-users mailing list
>> kramdown-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/kramdown-users
>> 
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users


From svicalifornia at gmail.com  Mon Oct 11 03:25:00 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Mon, 11 Oct 2010 18:25:00 +1100
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <FE67E9F9-B72D-440A-851A-5EE9A180ABB5@tidbits.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<362DA23E-6F4E-469F-93A5-22BCB347F04F@tidbits.com>
	<4CB2A2D2.3060501@sunshineco.com>
	<FE67E9F9-B72D-440A-851A-5EE9A180ABB5@tidbits.com>
Message-ID: <20101011142500000874.8156cf15@gmail.com>

As Eric pointed out, and Thomas announced before, the change occurred in 0.11 by making the leading '|' optional in table definitions.

I suggest again that this be solved by making kramdown ignore the contents of code blocks.  Then code blocks will, in a sense, be automatically "escaped."  Other instances of '|' to be ignored should be escaped by leading backslashes (\).

Shawn

On Sun, 10 Oct 2010 23:01:12 -0700, Matt Neuburg wrote:
> Thanks, Eric!
> 
> I guess, taking all of this into consideration, my suggestion on this 
> would be (1) escaping should work (2) nbsp; should not be emitted if 
> I've asked for numeric entities (3) it might be really cool if it 
> were possible optionally to turn OFF this new feature, i.e. table 
> lines without leading pipes.
> 
> Let's see what Thomas thinks. m.
> 
> On Oct 10, 2010, at 10:38 PM, Eric Sunshine wrote:
> 
>> The leading '|' on table lines became optional here: 
>> http://github.com/gettalong/kramdown/commit/5354385d6bec8c89e4d8c2c2b76f6d07433800c7
>> 
>> -- ES
>> 
>> 
>> On 10/11/2010 1:18 AM, Matt Neuburg wrote:
>>> Presumably, that is what used to happen, since I've had that line 
>>> in my page for months and many versions of kramdown. This behavior 
>>> is evidently new with this version of kramdown. (I'm sorry I can't 
>>> be more specific to say what I mean by "this version", but the 
>>> previous two versions that I was using were both called 0.11.0 and 
>>> so is this one, even though they are all three quite different from 
>>> each other.)
>>> 
>>> The docs do say, in fact, "Note that literal pipe characters need 
>>> to be escaped, even if they occur in code spans!" Now, maybe that 
>>> was never there before - if it was there, I never noticed it. But 
>>> in fact I could not find any way of escaping these pipe characters 
>>> so as to solve the problem. I mean, look, there are only three 
>>> possibilities: escape the first, escape the second, and escape 
>>> both. But none of those prevents the problem. m.
>>> 
>>> 
>>> On Oct 10, 2010, at 9:03 AM, Shawn Van Ittersum wrote:
>>> 
>>>> More generally, I think that kramdown should ignore the contents 
>>>> of any code block or span.  I can't think of a good reason for 
>>>> kramdown to interpret the contents of code blocks.  If it didn't, 
>>>> then this table creation problem would go away.
>>>> 
>>>> Shawn
>>>> 
>>>> On Sun, 10 Oct 2010 08:34:04 -0700, Matt Neuburg wrote:
>>>>> require 'kramdown'
>>>>> s =<<END
>>>>> The operator `||` is called logical-or.
>>>>> END
>>>>> puts Kramdown::Document.new(s).to_html
>>>>> 
>>>>> 
>>>>> What I expect:
>>>>> 
>>>>> <p>The operator....
>>>>> 
>>>>> What I get:
>>>>> 
>>>>> <table>
>>>>>  <tbody>
>>>>>    <tr>
>>>>>      <td>The operator `</td>
>>>>>      <td>&nbsp;</td>
>>>>>      <td>` is called logical-or.</td>
>>>>>    </tr>
>>>>>  </tbody>
>>>>> </table>
>>>>> 
>>>>> There are actually two errors here. One is that&nbsp; must never be
>>>>> emitted, since I am using kramdown in an XSLT-processing context and
>>>>> &nbsp; is not a legal XML entity. Asking for :numeric output of entities
>>>>> makes no difference.
>>>>> 
>>>>> 
>>>>> However, the obvious larger error is that the table-parser has suddenly
>>>>> taken over and thinks this is a table. This is a new error (i.e. 
>>>>> since the
>>>>> last release I was using). m.
>>>>> 
>>>>> Workarounds: none found! I can't think what to do aside from trying to
>>>>> revert to an earlier version of kramdown. m.
>>>>> 
>>>>> --
>>>>> matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
>>>>> pantes anthropoi tou eidenai oregontai phusei
>>>>> Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
>>>>> AppleScript: the Definitive Guide, 2nd edition
>>>>> http://www.tidbits.com/matt/default.html#applescriptthings
>>>>> Take Control of Exploring&  Customizing Snow Leopard
>>>>> http://tinyurl.com/kufyy8
>>>>> RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
>>>>> TidBITS, Mac news and reviews since 1990, http://www.tidbits.com
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> kramdown-users mailing list
>>>>> kramdown-users at rubyforge.org
>>>>> http://rubyforge.org/mailman/listinfo/kramdown-users
>>>> _______________________________________________
>>>> kramdown-users mailing list
>>>> kramdown-users at rubyforge.org
>>>> http://rubyforge.org/mailman/listinfo/kramdown-users
>>> 
>>> _______________________________________________
>>> kramdown-users mailing list
>>> kramdown-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/kramdown-users
>>> 
>> _______________________________________________
>> kramdown-users mailing list
>> kramdown-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/kramdown-users
> 
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From t_leitner at gmx.at  Mon Oct 11 03:27:08 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Mon, 11 Oct 2010 09:27:08 +0200
Subject: [kramdown-users] block inline attribute list problem
In-Reply-To: <C8D632E0.1DFD%matt@tidbits.com>
References: <C8D632E0.1DFD%matt@tidbits.com>
Message-ID: <20101011092708.58a5a6b2@noweto>

On 2010-10-09 14:55 -0700 Matt Neuburg wrote:
> This, however, does nothing:
> 
> require 'kramdown'
> s = <<END
> {: #myid}
> <div>
> Testing
> </div>
> END
> puts Kramdown::Document.new(s).to_html
> 
> 
> <div>
> Testing
> </div>
> 
> I expect the div to gain the id value "myid", but it doesn't. Okay, so
> maybe the problem is that markdown is not turned on for this div.

Okay, it's a bit unusual to use a block IAL to set attributes on HTML
tags but it should certainly work.

> Let's try it:
> 
> 
> require 'kramdown'
> s = <<END
> {: #myid}
> <div markdown="1">
> Testing
> </div>
> END
> puts Kramdown::Document.new(s).to_html
> 
> 
> <div>
>   <p id="myid">Testing</p>
> </div>
> 
> 
> Hmmm. We've skipped the div and attached the id to the internal p.

Another bug you stumbled upon.

I have added these to my TODO list! Thanks for reporting!

-- Thomas

From svicalifornia at gmail.com  Mon Oct 11 03:32:01 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Mon, 11 Oct 2010 18:32:01 +1100
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <FE67E9F9-B72D-440A-851A-5EE9A180ABB5@tidbits.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<362DA23E-6F4E-469F-93A5-22BCB347F04F@tidbits.com>
	<4CB2A2D2.3060501@sunshineco.com>
	<FE67E9F9-B72D-440A-851A-5EE9A180ABB5@tidbits.com>
Message-ID: <20101011143201827588.432e4028@gmail.com>

I also agree with Matt on requiring the leading pipe character for table definitions.  I always thought that was an improvement in kramdown syntax over PHPMarkdownExtra/Maruku.  That said, I don't strongly object to making it optional, as long as there is an easy way to escape pipe characters.

And the contents code blocks really should be ignored by kramdown automatically.

Shawn

On Sun, 10 Oct 2010 23:01:12 -0700, Matt Neuburg wrote:
> Thanks, Eric!
> 
> I guess, taking all of this into consideration, my suggestion on this 
> would be (1) escaping should work (2) nbsp; should not be emitted if 
> I've asked for numeric entities (3) it might be really cool if it 
> were possible optionally to turn OFF this new feature, i.e. table 
> lines without leading pipes.
> 
> Let's see what Thomas thinks. m.
> 
> On Oct 10, 2010, at 10:38 PM, Eric Sunshine wrote:
> 
>> The leading '|' on table lines became optional here: 
>> http://github.com/gettalong/kramdown/commit/5354385d6bec8c89e4d8c2c2b76f6d07433800c7
>> 
>> -- ES
>> 
>> 
>> On 10/11/2010 1:18 AM, Matt Neuburg wrote:
>>> Presumably, that is what used to happen, since I've had that line 
>>> in my page for months and many versions of kramdown. This behavior 
>>> is evidently new with this version of kramdown. (I'm sorry I can't 
>>> be more specific to say what I mean by "this version", but the 
>>> previous two versions that I was using were both called 0.11.0 and 
>>> so is this one, even though they are all three quite different from 
>>> each other.)
>>> 
>>> The docs do say, in fact, "Note that literal pipe characters need 
>>> to be escaped, even if they occur in code spans!" Now, maybe that 
>>> was never there before - if it was there, I never noticed it. But 
>>> in fact I could not find any way of escaping these pipe characters 
>>> so as to solve the problem. I mean, look, there are only three 
>>> possibilities: escape the first, escape the second, and escape 
>>> both. But none of those prevents the problem. m.
>>> 
>>> 
>>> On Oct 10, 2010, at 9:03 AM, Shawn Van Ittersum wrote:
>>> 
>>>> More generally, I think that kramdown should ignore the contents 
>>>> of any code block or span.  I can't think of a good reason for 
>>>> kramdown to interpret the contents of code blocks.  If it didn't, 
>>>> then this table creation problem would go away.
>>>> 
>>>> Shawn
>>>> 
>>>> On Sun, 10 Oct 2010 08:34:04 -0700, Matt Neuburg wrote:
>>>>> require 'kramdown'
>>>>> s =<<END
>>>>> The operator `||` is called logical-or.
>>>>> END
>>>>> puts Kramdown::Document.new(s).to_html
>>>>> 
>>>>> 
>>>>> What I expect:
>>>>> 
>>>>> <p>The operator....
>>>>> 
>>>>> What I get:
>>>>> 
>>>>> <table>
>>>>>  <tbody>
>>>>>    <tr>
>>>>>      <td>The operator `</td>
>>>>>      <td>&nbsp;</td>
>>>>>      <td>` is called logical-or.</td>
>>>>>    </tr>
>>>>>  </tbody>
>>>>> </table>
>>>>> 
>>>>> There are actually two errors here. One is that&nbsp; must never be
>>>>> emitted, since I am using kramdown in an XSLT-processing context and
>>>>> &nbsp; is not a legal XML entity. Asking for :numeric output of entities
>>>>> makes no difference.
>>>>> 
>>>>> 
>>>>> However, the obvious larger error is that the table-parser has suddenly
>>>>> taken over and thinks this is a table. This is a new error (i.e. 
>>>>> since the
>>>>> last release I was using). m.
>>>>> 
>>>>> Workarounds: none found! I can't think what to do aside from trying to
>>>>> revert to an earlier version of kramdown. m.
>>>>> 
>>>>> --
>>>>> matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
>>>>> pantes anthropoi tou eidenai oregontai phusei
>>>>> Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
>>>>> AppleScript: the Definitive Guide, 2nd edition
>>>>> http://www.tidbits.com/matt/default.html#applescriptthings
>>>>> Take Control of Exploring&  Customizing Snow Leopard
>>>>> http://tinyurl.com/kufyy8
>>>>> RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
>>>>> TidBITS, Mac news and reviews since 1990, http://www.tidbits.com
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> kramdown-users mailing list
>>>>> kramdown-users at rubyforge.org
>>>>> http://rubyforge.org/mailman/listinfo/kramdown-users
>>>> _______________________________________________
>>>> kramdown-users mailing list
>>>> kramdown-users at rubyforge.org
>>>> http://rubyforge.org/mailman/listinfo/kramdown-users
>>> 
>>> _______________________________________________
>>> kramdown-users mailing list
>>> kramdown-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/kramdown-users
>>> 
>> _______________________________________________
>> kramdown-users mailing list
>> kramdown-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/kramdown-users
> 
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From t_leitner at gmx.at  Mon Oct 11 03:43:58 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Mon, 11 Oct 2010 09:43:58 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <C8D72AFC.1EA4%matt@tidbits.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
Message-ID: <20101011094358.7b093b1b@noweto>

On 2010-10-10 08:34 -0700 Matt Neuburg wrote:
> require 'kramdown'
> s = <<END
> The operator `||` is called logical-or.
> END
> puts Kramdown::Document.new(s).to_html

This is now converted to a table since the table syntax has been
adjusted to be more flexible and more compatible with the PHP Markdown
Extra tables.

The difference to the prior table syntax is basically that 

* tables must now start and stop on [block boundaries][1]
* *any* line that contains an unescaped pipe is potentially a table
  line, not only lines starting with a pipe

So since the above example consists only of one line, the block
boundary condition is met and since it contains unescaped pipes, the
other condition is also met. Note that you always needed to escape
pipes, even in code spans, which was the case even with the prior table
syntax!

To tell kramdown to treat the line as a simple paragraph, you can do
this:

    The operator `\|\|` is called logical-or.

This will give you a paragraph instead of a table.

> What I get:
> 
> <table>
>   <tbody>
>     <tr>
>       <td>The operator `</td>
>       <td>&nbsp;</td>
>       <td>` is called logical-or.</td>
>     </tr>
>   </tbody>
> </table>
> 
> There are actually two errors here. One is that &nbsp; must never be
> emitted, since I am using kramdown in an XSLT-processing context and
> &nbsp; is not a legal XML entity. Asking for :numeric output of
> entities makes no difference.

The entity `&nbsp;` is a valid HTML and XHTML entity, so outputting it
in the HTML converter is certainly okay. However, you are right that
the `entity_output` option should be respected, so this is a bug - I
have added this to my TODO list!
 
> Workarounds: none found! I can't think what to do aside from trying to
> revert to an earlier version of kramdown. m.

As stated above you need to escape the pipes for the line to be
treated as a paragraph.

-- Thomas 

[1]: http://kramdown.rubyforge.org/syntax.html#block-boundaries

From t_leitner at gmx.at  Mon Oct 11 03:50:00 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Mon, 11 Oct 2010 09:50:00 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101010230338632089.15573510@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
Message-ID: <20101011095000.217afd84@noweto>

On 2010-10-11 03:03 +1100 Shawn Van Ittersum wrote:
> More generally, I think that kramdown should ignore the contents of
> any code block or span.  I can't think of a good reason for kramdown
> to interpret the contents of code blocks.  If it didn't, then this
> table creation problem would go away.

kramdown does ignore the contents of code spans and code blocks.
However, the parsing code for tables would be much more complex and
slower if we were to check for pipes in code spans. For example, what
about this:

    cell 1 | cell 2 with a `code | span
    ending here` | cell 4

It is actually rather hard to trigger a table since *each* line needs
to have at least one pipe character *and* the table has to be on block
boundaries. For example, the following is not converted to a table:

    This is just some paragraph, showing the `|` character
    and explaining why this is not a table.

So the problem with the example from Matt is that the paragraph is so
short that it fits on one line. Therefore you need to escape the pipe
characters in such cases.

-- Thomas

From t_leitner at gmx.at  Mon Oct 11 04:03:05 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Mon, 11 Oct 2010 10:03:05 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <362DA23E-6F4E-469F-93A5-22BCB347F04F@tidbits.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<362DA23E-6F4E-469F-93A5-22BCB347F04F@tidbits.com>
Message-ID: <20101011100305.2f669383@noweto>

On 2010-10-10 22:18 -0700 Matt Neuburg wrote:
> Presumably, that is what used to happen, since I've had that line in
> my page for months and many versions of kramdown. This behavior is
> evidently new with this version of kramdown. (I'm sorry I can't be
> more specific to say what I mean by "this version", but the previous
> two versions that I was using were both called 0.11.0 and so is this
> one, even though they are all three quite different from each other.) 

Please don't state that there are multiple different versions of
kramdown with a version number of 0.11.0 when there is in fact only
one! If you clone the kramdown repository after the release of 0.10.0
and before the release of 0.11.0, you certainly don't have version
0.11.0 of kramdown *even* if the version number has already been bumped
to 0.11.0 in the repository. You just used *unreleased* code and that's
about it. How you refer to the unreleased code is up to you - I
recommend using the commit hash because then everyone knows about what
code state you are talking about.
 
> The docs do say, in fact, "Note that literal pipe characters need to
> be escaped, even if they occur in code spans!" Now, maybe that was
> never there before - if it was there, I never noticed it. But in fact
> I could not find any way of escaping these pipe characters so as to
> solve the problem. I mean, look, there are only three possibilities:
> escape the first, escape the second, and escape both. But none of
> those prevents the problem. m.

As stated in the other mail, this text extract has been in the syntax
documentation for the table syntax even before the 0.11.0 version! And
escaping the pipes does work to prevent kramdown from treating the text
as table.

However, I have just seen that this introduces a bug (the pipes remain
escaped although they shouldn't) when the line is not treated as a table
line - will have to fix this.

-- Thomas

From t_leitner at gmx.at  Mon Oct 11 06:07:06 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Mon, 11 Oct 2010 12:07:06 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <FE67E9F9-B72D-440A-851A-5EE9A180ABB5@tidbits.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<362DA23E-6F4E-469F-93A5-22BCB347F04F@tidbits.com>
	<4CB2A2D2.3060501@sunshineco.com>
	<FE67E9F9-B72D-440A-851A-5EE9A180ABB5@tidbits.com>
Message-ID: <20101011120706.486f03d7@noweto>

On 2010-10-10 23:01 -0700 Matt Neuburg wrote:
> Thanks, Eric!
> 
> I guess, taking all of this into consideration, my suggestion on this
> would be (1) escaping should work (2) nbsp; should not be emitted if
> I've asked for numeric entities (3) it might be really cool if it
> were possible optionally to turn OFF this new feature, i.e. table
> lines without leading pipes.
> 
> Let's see what Thomas thinks. m.

ad 1) Escaping works in that it is possible to prevent kramdown from
      interpreting a line as table line. However, as written before in
      another mail, the resulting paragraph line may not be correct.
      This is a bug and I will look into it.

ad 2) This is also a bug as outlined in another mail and it will be
      fixed.

ad 3) As written in another mail, it is rather hard to unintentionally
      start a table when a paragraph consists of multiple lines. Do you
      agree that pipe characters do not often appear in normal text? If
      so, there is no real reason for allowing one to turn off this
      feature. Your example will work if I can figure out a nice (and
      hopefully fast) way so that pipes in code spans do not need to be
      escaped. As I have said before I would like the parser to be
      non-configurable as much as possible.

-- Thomas

From svicalifornia at gmail.com  Mon Oct 11 07:06:20 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Mon, 11 Oct 2010 22:06:20 +1100
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101011095000.217afd84@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
Message-ID: <20101011180620364641.c9e86a24@gmail.com>

On Mon, 11 Oct 2010 09:50:00 +0200, Thomas Leitner wrote:
> On 2010-10-11 03:03 +1100 Shawn Van Ittersum wrote:
>> More generally, I think that kramdown should ignore the contents of
>> any code block or span.  I can't think of a good reason for kramdown
>> to interpret the contents of code blocks.  If it didn't, then this
>> table creation problem would go away.
> 
> kramdown does ignore the contents of code spans and code blocks.
> However, the parsing code for tables would be much more complex and
> slower if we were to check for pipes in code spans. For example, what
> about this:
> 
>     cell 1 | cell 2 with a `code | span
>     ending here` | cell 4

First, I think that this example should yield:

<table>
<tr>
<td>cell 1</td>
<td>cell 2 with a <code>code | span
ending here</code></td>
<td>cell 4</td>
</tr>
</table>

This can be accomplished fairly easily by first parsing for ` delimiters to find and isolate code blocks.  Once a block is identified as code, then kramdown will not apply to it.

GitHub-flavored Markdown has a clever method of replacing certain blocks with hash digests, so that the block contents cannot be interpreted by Markdown.  For example:

    def gfm_extraction(text)
      md5 = Digest::MD5.hexdigest(text)
      @extractions[md5] = text
      "{gfm-extraction-#{md5}}"
    end

    # convert code blocks to hashed extractions
    html.gsub!(%r{\`(.*)?\`}m) { gfm_extraction("<code>#{$1}</code>") ) }

After applying Markdown processing, the hashed blocked are expanded again:

    # Insert extractions
    html.gsub!(/\{gfm-extraction-([0-9a-f]{32})\}/) { @extractions[$1] }

This is very fast.

Shawn

From svicalifornia at gmail.com  Mon Oct 11 07:09:00 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Mon, 11 Oct 2010 22:09:00 +1100
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101011095000.217afd84@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
Message-ID: <20101011180900604973.3e8d4e59@gmail.com>

For whole-line code blocks (lines indented by spaces so that the entire line is considered a code-block), the solution is easy: don't apply kramdown processing to the contents of the line.

Is there any other syntax besides ` characters and leading spaces to trigger a code block?  If not, then I've covered both cases with simple, fast solutions to disable kramdown processing of the contents.

Shawn

On Mon, 11 Oct 2010 09:50:00 +0200, Thomas Leitner wrote:
> On 2010-10-11 03:03 +1100 Shawn Van Ittersum wrote:
>> More generally, I think that kramdown should ignore the contents of
>> any code block or span.  I can't think of a good reason for kramdown
>> to interpret the contents of code blocks.  If it didn't, then this
>> table creation problem would go away.
> 
> kramdown does ignore the contents of code spans and code blocks.
> However, the parsing code for tables would be much more complex and
> slower if we were to check for pipes in code spans. For example, what
> about this:
> 
>     cell 1 | cell 2 with a `code | span
>     ending here` | cell 4
> 
> It is actually rather hard to trigger a table since *each* line needs
> to have at least one pipe character *and* the table has to be on block
> boundaries. For example, the following is not converted to a table:
> 
>     This is just some paragraph, showing the `|` character
>     and explaining why this is not a table.
> 
> So the problem with the example from Matt is that the paragraph is so
> short that it fits on one line. Therefore you need to escape the pipe
> characters in such cases.
> 
> -- Thomas
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From matt at tidbits.com  Mon Oct 11 08:54:48 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Mon, 11 Oct 2010 05:54:48 -0700
Subject: [kramdown-users] kramdown table-making has gone completely
	insane
In-Reply-To: <20101011094358.7b093b1b@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com> <20101011094358.7b093b1b@noweto>
Message-ID: <51ECF118-6EDA-40ED-9DD7-1D39D5A0EFC1@tidbits.com>



> To tell kramdown to treat the line as a simple paragraph, you can do
> this:
> 
>    The operator `\|\|` is called logical-or.
> 
> This will give you a paragraph instead of a table.
> 

Have you tried it? I did, and it still gives a table. m.

From matt at tidbits.com  Mon Oct 11 09:23:59 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Mon, 11 Oct 2010 06:23:59 -0700
Subject: [kramdown-users] kramdown table-making has gone completely
	insane
In-Reply-To: <51ECF118-6EDA-40ED-9DD7-1D39D5A0EFC1@tidbits.com>
Message-ID: <C8D85DC9.1ED5%matt@tidbits.com>

On 10/11/10 5:54 AM, thus spake "Matt Neuburg" <matt at tidbits.com>:

>
>
>> To tell kramdown to treat the line as a simple paragraph, you can do
>> this:
>> 
>>    The operator `\|\|` is called logical-or.
>> 
>> This will give you a paragraph instead of a table.
>> 
>
>Have you tried it? I did, and it still gives a table. m.
>

Oh, and I forgot to mention: The trick that *does* work is to split the
paragraph into multiple lines. This was a very good tip; thank you! m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com





From t_leitner at gmx.at  Mon Oct 11 09:54:52 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Mon, 11 Oct 2010 15:54:52 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101011180900604973.3e8d4e59@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
Message-ID: <20101011155452.75c8f672@noweto>

On 2010-10-11 22:09 +1100 Shawn Van Ittersum wrote:
> For whole-line code blocks (lines indented by spaces so that the
> entire line is considered a code-block), the solution is easy: don't
> apply kramdown processing to the contents of the line.
> 
> Is there any other syntax besides ` characters and leading spaces to
> trigger a code block?  If not, then I've covered both cases with
> simple, fast solutions to disable kramdown processing of the contents.

First, you shouldn't confuse [code spans][1] and [code blocks][2].
Second, kramdown does neither process the contents of code spans nor
the contents of code blocks!

-- Thomas

[1]: http://kramdown.rubyforge.org/syntax.html#code-spans
[2]: http://kramdown.rubyforge.org/syntax.html#code-blocks

From t_leitner at gmx.at  Mon Oct 11 10:03:54 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Mon, 11 Oct 2010 16:03:54 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <51ECF118-6EDA-40ED-9DD7-1D39D5A0EFC1@tidbits.com>
References: <C8D72AFC.1EA4%matt@tidbits.com> <20101011094358.7b093b1b@noweto>
	<51ECF118-6EDA-40ED-9DD7-1D39D5A0EFC1@tidbits.com>
Message-ID: <20101011160354.2b87f84f@noweto>

On 2010-10-11 05:54 -0700 Matt Neuburg wrote:
> 
> 
> > To tell kramdown to treat the line as a simple paragraph, you can do
> > this:
> > 
> >    The operator `\|\|` is called logical-or.
> > 
> > This will give you a paragraph instead of a table.
> > 
> 
> Have you tried it? I did, and it still gives a table. m.

Yes, I tried it and it works for me. Here is the output:

    $ ruby -v; kramdown -v; cat mytest; kramdown mytest
    ruby 1.8.7 (2010-01-10 patchlevel 249) [x86_64-linux]\
    0.11.0
    The operator `\|\|` is called logical-or.
    <p>The operator <code>\|\|</code> is called logical-or.</p>

-- Thomas

From matt at tidbits.com  Mon Oct 11 10:48:09 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Mon, 11 Oct 2010 07:48:09 -0700
Subject: [kramdown-users] kramdown table-making has gone completely
	insane
In-Reply-To: <20101011160354.2b87f84f@noweto>
Message-ID: <C8D87159.1F0C%matt@tidbits.com>

On 10/11/10 7:03 AM, thus spake "Thomas Leitner" <t_leitner at gmx.at>:

>Yes, I tried it and it works for me. Here is the output:
>
>    $ ruby -v; kramdown -v; cat mytest; kramdown mytest
>    ruby 1.8.7 (2010-01-10 patchlevel 249) [x86_64-linux]\
>    0.11.0
>    The operator `\|\|` is called logical-or.
>    <p>The operator <code>\|\|</code> is called logical-or.</p>
>
>

You're right and I'm wrong. I forgot that in my script I needed to
double-escape in order to pass a single-escape character to kramdown.

Your test nicely exposes the bug with the escape character not being
removed.

Meanwhile the trick with splitting the line works perfectly for me, so I'm
back in business. Thx -

m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com





From t_leitner at gmx.at  Mon Oct 11 14:28:57 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Mon, 11 Oct 2010 20:28:57 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101011180620364641.c9e86a24@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180620364641.c9e86a24@gmail.com>
Message-ID: <20101011202857.7161a803@noweto>

On 2010-10-11 22:06 +1100 Shawn Van Ittersum wrote:
> >     cell 1 | cell 2 with a `code | span
> >     ending here` | cell 4
> 
> First, I think that this example should yield:
> 
> <table>
> <tr>
> <td>cell 1</td>
> <td>cell 2 with a <code>code | span
> ending here</code></td>
> <td>cell 4</td>
> </tr>
> </table>

This will certainly never be the case with the current table syntax
since the current syntax only allows cells on one line. I.e. each line
is a row and cells are separated from each other by pipe characters.
Therefore no span level element can span across two or more lines!

> GitHub-flavored Markdown has a clever method of replacing certain
> blocks with hash digests, so that the block contents cannot be
> interpreted by Markdown.  For example:
> 
>     def gfm_extraction(text)
>       md5 = Digest::MD5.hexdigest(text)
>       @extractions[md5] = text
>       "{gfm-extraction-#{md5}}"
>     end
> 
>     # convert code blocks to hashed extractions
>     html.gsub!(%r{\`(.*)?\`}m)
> { gfm_extraction("<code>#{$1}</code>") ) }
> 
> After applying Markdown processing, the hashed blocked are expanded
> again:
> 
>     # Insert extractions
>     html.gsub!(/\{gfm-extraction-([0-9a-f]{32})\}/)
> { @extractions[$1] }
> 
> This is very fast.

This may be so but the above code is not very robust and probably based
on the code of the original Markdown.pl implementation because it does
exactly this. I don't know what else GFM does but the above code for
code spans won't work in all situations. For example, consider this:

    This is an <span title="some ` title">element `codespan`</span>

The regexp will destroy the HTML `span` element and the code span it
contains.

It also probably won't work correctly for code spans using multiple
backticks (which is allowed by standard Markdown):

    This is a `` ` `` code span and a literal ` backtick.

-- Thomas

From svicalifornia at gmail.com  Mon Oct 11 14:47:29 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Tue, 12 Oct 2010 05:47:29 +1100
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101011202857.7161a803@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180620364641.c9e86a24@gmail.com>
	<20101011202857.7161a803@noweto>
Message-ID: <20101012014729722101.59363a66@gmail.com>

On Mon, 11 Oct 2010 20:28:57 +0200, Thomas Leitner wrote:
> This may be so but the above code is not very robust and probably based
> on the code of the original Markdown.pl implementation because it does
> exactly this. I don't know what else GFM does but the above code for
> code spans won't work in all situations. For example, consider this:
> 
>     This is an <span title="some ` title">element `codespan`</span>
> 
> The regexp will destroy the HTML `span` element and the code span it
> contains.

True, but is that legal kramdown?  Shouldn't the backtick in the title attribute be escaped?

> It also probably won't work correctly for code spans using multiple
> backticks (which is allowed by standard Markdown):
> 
>     This is a `` ` `` code span and a literal ` backtick.

Hmm, that's true.

My main thought for this thread is that the pipe characters in Matt's original example should not have been interpreted as table delimiters.  They shouldn't need to be escaped, as they were already in a code span.

Shawn

From svicalifornia at gmail.com  Mon Oct 11 14:41:40 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Tue, 12 Oct 2010 05:41:40 +1100
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101011155452.75c8f672@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
Message-ID: <20101012014140273990.efe1a504@gmail.com>

> First, you shouldn't confuse [code spans][1] and [code blocks][2].
> Second, kramdown does neither process the contents of code spans nor
> the contents of code blocks!

Sorry for the confusion about code spans and code blocks.  But if kramdown does not process the contents of code spans, then how do the pipe characters in Matt's example get interpreted as table cell delimiters?  They're inside a code span:

require 'kramdown'
s = <<END
The operator `||` is called logical-or.
END
puts Kramdown::Document.new(s).to_html

Shawn

From t_leitner at gmx.at  Tue Oct 12 03:27:12 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Tue, 12 Oct 2010 09:27:12 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101012014729722101.59363a66@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180620364641.c9e86a24@gmail.com>
	<20101011202857.7161a803@noweto>
	<20101012014729722101.59363a66@gmail.com>
Message-ID: <20101012092712.481e244e@noweto>

On 2010-10-12 05:47 +1100 Shawn Van Ittersum wrote:
> > 
> >     This is an <span title="some ` title">element `codespan`</span>
> > 
> > The regexp will destroy the HTML `span` element and the code span it
> > contains.
> 
> True, but is that legal kramdown?  Shouldn't the backtick in the
> title attribute be escaped?

Why shouldn't it be legal in kramdown? Backticks aren't special
characters in HTML and therefore valid in attribute values. The above
example works fine in kramdown.
 
> > It also probably won't work correctly for code spans using multiple
> > backticks (which is allowed by standard Markdown):
> > 
> >     This is a `` ` `` code span and a literal ` backtick.
> 
> Hmm, that's true.
> 
> My main thought for this thread is that the pipe characters in Matt's
> original example should not have been interpreted as table
> delimiters.  They shouldn't need to be escaped, as they were already
> in a code span.

Yes, I know. However, it is hard to recognize the code span without
actually parsing the line with the span level parser which is what I
tried to avoid. But since escaping the pipe in the code span obviously
also doesn't work as expected, I will have to find another solution
which probably will include the use of the span level parser.

-- Thomas

From t_leitner at gmx.at  Tue Oct 12 03:34:22 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Tue, 12 Oct 2010 09:34:22 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101012014140273990.efe1a504@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
Message-ID: <20101012093422.4381ad35@noweto>

On 2010-10-12 05:41 +1100 Shawn Van Ittersum wrote:
> Sorry for the confusion about code spans and code blocks.  But if
> kramdown does not process the contents of code spans, then how do the
> pipe characters in Matt's example get interpreted as table cell
> delimiters?  They're inside a code span:
> 
> require 'kramdown'
> s = <<END
> The operator `||` is called logical-or.
> END
> puts Kramdown::Document.new(s).to_html

This is easily explained: The kramdown parser has basically two stages.
The first stage parses the document line by line and builds the tree of
all *block level elements*. After that the text in the block level
elements is parsed using the span level parser which returns text and,
possibly, span level elements.

So your assumption that the above document line contains a code span is
only correct *if* the line is parsed as paragraph. However, the table
parser is invoked before the paragraph parser (because everything that
is not any valid other kramdown block level element is a paragraph),
finds the pipes, concludes that it is a table line, splits the line
into cells using the pipes as separators and boom: no more code span!

-- Thomas

From svicalifornia at gmail.com  Tue Oct 12 04:48:01 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Tue, 12 Oct 2010 19:48:01 +1100
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101012093422.4381ad35@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
Message-ID: <20101012154801867549.f68e3490@gmail.com>

On Tue, 12 Oct 2010 09:34:22 +0200, Thomas Leitner wrote:
> On 2010-10-12 05:41 +1100 Shawn Van Ittersum wrote:
>> Sorry for the confusion about code spans and code blocks.  But if
>> kramdown does not process the contents of code spans, then how do the
>> pipe characters in Matt's example get interpreted as table cell
>> delimiters?  They're inside a code span:
>> 
>> require 'kramdown'
>> s = <<END
>> The operator `||` is called logical-or.
>> END
>> puts Kramdown::Document.new(s).to_html
> 
> This is easily explained: The kramdown parser has basically two stages.
> The first stage parses the document line by line and builds the tree of
> all *block level elements*. After that the text in the block level
> elements is parsed using the span level parser which returns text and,
> possibly, span level elements.
> 
> So your assumption that the above document line contains a code span is
> only correct *if* the line is parsed as paragraph. However, the table
> parser is invoked before the paragraph parser (because everything that
> is not any valid other kramdown block level element is a paragraph),
> finds the pipes, concludes that it is a table line, splits the line
> into cells using the pipes as separators and boom: no more code span!

Thomas, please know that I have tremendous respect for the very powerful software you created.  However, your explanation of this particular issue is bordering on ridiculous.  Any human reading that example would interpret `||` as a code span, not as a table-cell delimiter.  Regardless of the underlying implementation details, kramdown's faulty interpretation of this code is a bug or design flaw, not a feature.  `||` is a code span, and kramdown's failure to recognize it as such doesn't change that fact.

As for your prior claim, "kramdown does neither process the contents of code spans nor the contents of code blocks," this is clearly not true, since you are applying the table interpreter before the block interpreter and thus processing the contents of code blocks.  Please revisit the implementation design so kramdown can reliably satisfy the claim not to process code spans.

Thanks,
Shawn

From t_leitner at gmx.at  Tue Oct 12 09:21:11 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Tue, 12 Oct 2010 15:21:11 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101012154801867549.f68e3490@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
Message-ID: <20101012152111.4f22eabe@noweto>

On 2010-10-12 19:48 +1100 Shawn Van Ittersum wrote:
> Thomas, please know that I have tremendous respect for the very
> powerful software you created.  However, your explanation of this
> particular issue is bordering on ridiculous.  Any human reading that
> example would interpret `||` as a code span, not as a table-cell
> delimiter.  Regardless of the underlying implementation details,
> kramdown's faulty interpretation of this code is a bug or design
> flaw, not a feature.  `||` is a code span, and kramdown's failure to
> recognize it as such doesn't change that fact.

Yes, you are right. Any human would probably see that but what about
these examples:

    This is a *cell | with some | data* here and here*

or

    Of course, you`re good | You are not so good | You`re bad!!!

Can you definitely say that these are *not* table lines?

> As for your prior claim, "kramdown does neither process the contents
> of code spans nor the contents of code blocks," this is clearly not
> true, since you are applying the table interpreter before the block
> interpreter and thus processing the contents of code blocks.  Please
> revisit the implementation design so kramdown can reliably satisfy
> the claim not to process code spans.

You need to know that the table parser *is* a block parser. There has
to be a certain ordering so that parsing block level elements works
correctly. Here is the ordering for the block level parsers:

    :blank_line, :codeblock, :codeblock_fenced, :blockquote, :table,
    :atx_header, :setext_header, :horizontal_rule, :list,
    :definition_list, :link_definition, :block_html,
    :footnote_definition, :abbrev_definition, :block_extensions,
    :block_math, :eob_marker, :paragraph

As you can see, the parser for code blocks is tried before the parser
for tables! So there is no issue here.

If you look at the implementation of the code span parser and the HTML
converter method for code span (or that of the code block
parser/converter), you will see that kramdown does no processing! So my
claim is correct.

In fact, the problem exists only because of how the table parser
currently works (and, naturally, because the code span parser does
*not* process the contents of code spans). As I said before I'm
working on correcting the problem!

-- Thomas

From matt at tidbits.com  Tue Oct 12 14:27:07 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Tue, 12 Oct 2010 11:27:07 -0700
Subject: [kramdown-users] kramdown table-making has gone completely
	insane
In-Reply-To: <20101012154801867549.f68e3490@gmail.com>
Message-ID: <C8D9ADC3.1F98%matt@tidbits.com>

>    Of course, you`re good | You are not so good | You`re bad!!!
>
> Can you definitely say that these are *not* table lines?

I can (or at least, I could until Sept. 24): They don't begin with a pipe
character so they are not table lines!

In other words, the problem is simply that the rule introduced on Sept. 24
violates the basic nature of blocks in Markdown / kramdown, which is that
a block is identified by how the line starts.

It is not clear to me what problem the Sept. 24 change is meant to
solve, but perhaps that problem could be solved some other way that *does*
involve a pipe a character at the start of the line.

Or, as I requested before, give me a way to "opt out" of this way of doing
things - a way to switch the rule off myself so that I don't have to worry
about it. I'm already using an option when I call kramdown (:numeric) and
I'm perfectly happy to use additional options. I'd be glad to say :tables
=> :simpleminded or whatever you like to call this. :)

What I *don't* want to have to do is look for all my pipe characters and
escape them.

m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com





From svicalifornia at gmail.com  Tue Oct 12 14:33:07 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Wed, 13 Oct 2010 05:33:07 +1100
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101012152111.4f22eabe@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
Message-ID: <20101013013307002686.5a84a3d2@gmail.com>

On Tue, 12 Oct 2010 15:21:11 +0200, Thomas Leitner wrote:
> On 2010-10-12 19:48 +1100 Shawn Van Ittersum wrote:
>> Thomas, please know that I have tremendous respect for the very
>> powerful software you created.  However, your explanation of this
>> particular issue is bordering on ridiculous.  Any human reading that
>> example would interpret `||` as a code span, not as a table-cell
>> delimiter.  Regardless of the underlying implementation details,
>> kramdown's faulty interpretation of this code is a bug or design
>> flaw, not a feature.  `||` is a code span, and kramdown's failure to
>> recognize it as such doesn't change that fact.
> 
> Yes, you are right. Any human would probably see that but what about
> these examples:
> 
>     This is a *cell | with some | data* here and here*

*...* does not delimit a code span, thus kramdown is free to process the contents.

>     Of course, you`re good | You are not so good | You`re bad!!!

According to typographic standards, backticks (`) should never be used as apostrophes.  However, nested quotes make a similar case:

   "He said pointedly, `Well, | isn't this a pickle? | I admitted, `Yes, I'm afraid it is.'"

However, even in this example, I would interpret the contents of the backticks as a code span, *unless* they had been escaped with backslashes.

I still believe that code spans should be found and extracted first, before looking for table cell delimiters.

> In fact, the problem exists only because of how the table parser
> currently works (and, naturally, because the code span parser does
> *not* process the contents of code spans). As I said before I'm
> working on correcting the problem!

You can solve this by finding the code spans on each line and extracting them (replacing them with hash markers, as in the example GFM code I sent previously) before applying the other parsers, including the table cell parser.

Shawn

From svicalifornia at gmail.com  Tue Oct 12 14:44:34 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Wed, 13 Oct 2010 05:44:34 +1100
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <C8D9ADC3.1F98%matt@tidbits.com>
References: <C8D9ADC3.1F98%matt@tidbits.com>
Message-ID: <20101013014434535120.baae6b0e@gmail.com>

On Tue, 12 Oct 2010 11:27:07 -0700, Matt Neuburg wrote:
> What I *don't* want to have to do is look for all my pipe characters and
> escape them.

Agreed.  Especially when those pipe characters are already inside code spans (backticks) and, as such, should already be ignored.

Giving pipe characters the ability to delimit table-cells mid-line as in PHP Markdown Extra is okay, but let's not give pipes so much power that than they can break up code spans.  That's going way too far.  Code spans (backticks) should take precedence.

As for backtick ambiguity, it's rare to see backticks in text when code spans aren't desired.  Any backticks that aren't meant to start a code span should be properly escaped with backslashes.

Shawn


From t_leitner at gmx.at  Wed Oct 13 04:10:42 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Wed, 13 Oct 2010 10:10:42 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101013013307002686.5a84a3d2@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
Message-ID: <20101013101042.1fb1d6ae@noweto>

On 2010-10-13 05:33 +1100 Shawn Van Ittersum wrote:
> You can solve this by finding the code spans on each line and
> extracting them (replacing them with hash markers, as in the example
> GFM code I sent previously) before applying the other parsers,
> including the table cell parser.

Extracting the parts and using hashes is not necessary, this can be
done in nicer ways.

However, think about the following example:

    This is *not* a `code | span, it continues on the
    second line and ends` here | some other text here

If I parse for code spans line by line, the code span which continues
to the second line, is not found and therefore this is interpreted as a
table with two rows and two cells each. However, a human reader would
probably see the code span...

I have come to the conclusion that tables should probably be created
out of paragraphs in a later step (after a whole paragraph is read) but
I have to think a bit more about this...

-- Thomas

From t_leitner at gmx.at  Wed Oct 13 04:08:55 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Wed, 13 Oct 2010 10:08:55 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101013014434535120.baae6b0e@gmail.com>
References: <C8D9ADC3.1F98%matt@tidbits.com>
	<20101013014434535120.baae6b0e@gmail.com>
Message-ID: <20101013100855.3b0187b4@noweto>

On 2010-10-13 05:44 +1100 Shawn Van Ittersum wrote:
> Giving pipe characters the ability to delimit table-cells mid-line as
> in PHP Markdown Extra is okay, but let's not give pipes so much power
> that than they can break up code spans.  That's going way too far.
> Code spans (backticks) should take precedence.

Please try some examples with PHP Markdown Extra, you will find numerous
cases where it does something odd, for example:

    table `||` test
    - | -
    table | som` test

The first line does contain a pipe but in a code span. So it should
probably not start a table (as defined in
http://michelf.com/projects/php-markdown/extra/#table). However, it
does start a table!

> As for backtick ambiguity, it's rare to see backticks in text when
> code spans aren't desired.  Any backticks that aren't meant to start
> a code span should be properly escaped with backslashes.

I agree. And we basically only have the problem with the current syntax
because the content of code spans is not processed by kramdown.

-- Thomas

From t_leitner at gmx.at  Wed Oct 13 04:15:15 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Wed, 13 Oct 2010 10:15:15 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <C8D9ADC3.1F98%matt@tidbits.com>
References: <20101012154801867549.f68e3490@gmail.com>
	<C8D9ADC3.1F98%matt@tidbits.com>
Message-ID: <20101013101515.03257876@noweto>

On 2010-10-12 11:27 -0700 Matt Neuburg wrote:
> >    Of course, you`re good | You are not so good | You`re bad!!!
> >
> > Can you definitely say that these are *not* table lines?
> 
> I can (or at least, I could until Sept. 24): They don't begin with a
> pipe character so they are not table lines!
> 
> In other words, the problem is simply that the rule introduced on
> Sept. 24 violates the basic nature of blocks in Markdown / kramdown,
> which is that a block is identified by how the line starts.

The above rule is true for most block level elements but not all, for
example, setext headers are defined differently.
 
> It is not clear to me what problem the Sept. 24 change is meant to
> solve, but perhaps that problem could be solved some other way that
> *does* involve a pipe a character at the start of the line.

The change was introduced so that the kramdown syntax is more
compatible to PHP Markdown Extra/Maruku and how they parse tables.
Since PHP Markdown Extra allows one to omit the leading pipe, I changed
the table syntax to also use an optional instead of a mandatory leading
pipe.

> Or, as I requested before, give me a way to "opt out" of this way of
> doing things - a way to switch the rule off myself so that I don't
> have to worry about it. I'm already using an option when I call
> kramdown (:numeric) and I'm perfectly happy to use additional
> options. I'd be glad to say :tables => :simpleminded or whatever you
> like to call this. :)
> 
> What I *don't* want to have to do is look for all my pipe characters
> and escape them.

As I said before (and as Shawn said regarding backticks), I think the
pipe characters aren't used that often in documents. And therefore I
don't think we need such an option.

-- Thomas

From svicalifornia at gmail.com  Wed Oct 13 08:18:12 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Wed, 13 Oct 2010 19:18:12 +0700
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101013101042.1fb1d6ae@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
	<20101013101042.1fb1d6ae@noweto>
Message-ID: <20101013191812071179.fb33691e@gmail.com>

> However, think about the following example:
> 
>     This is *not* a `code | span, it continues on the
>     second line and ends` here | some other text here

Why not interpret backticks across multiple lines as multi-line code spans?

> If I parse for code spans line by line, the code span which continues
> to the second line, is not found and therefore this is interpreted as a
> table with two rows and two cells each. However, a human reader would
> probably see the code span...

Right, which is why kramdown should interpret it as a code span, the same way a human reader would.  Principle of least surprise.

Shawn

From svicalifornia at gmail.com  Wed Oct 13 08:21:12 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Wed, 13 Oct 2010 19:21:12 +0700
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101013100855.3b0187b4@noweto>
References: <C8D9ADC3.1F98%matt@tidbits.com>
	<20101013014434535120.baae6b0e@gmail.com>
	<20101013100855.3b0187b4@noweto>
Message-ID: <20101013192112056362.24e11847@gmail.com>

On Wed, 13 Oct 2010 10:08:55 +0200, Thomas Leitner wrote:
> Please try some examples with PHP Markdown Extra, you will find numerous
> cases where it does something odd, for example:
> 
>     table `||` test
>     - | -
>     table | som` test
> 
> The first line does contain a pipe but in a code span. So it should
> probably not start a table (as defined in
> http://michelf.com/projects/php-markdown/extra/#table). However, it
> does start a table!

We should we do what's right, not copy the mistakes of some other implementation, especially one written in PHP. :)

> I agree. And we basically only have the problem with the current syntax
> because the content of code spans is not processed by kramdown.

It's not really a matter of *processing* the contents of code spans, but (1) recognizing code spans as a human would, and then (2) *sheltering* their contents to prevent kramdown processing.

Shawn

From t_leitner at gmx.at  Wed Oct 13 10:00:29 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Wed, 13 Oct 2010 16:00:29 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101013191812071179.fb33691e@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
	<20101013101042.1fb1d6ae@noweto>
	<20101013191812071179.fb33691e@gmail.com>
Message-ID: <20101013160029.747ee2e6@noweto>

On 2010-10-13 19:18 +0700 Shawn Van Ittersum wrote:
> > However, think about the following example:
> > 
> >     This is *not* a `code | span, it continues on the
> >     second line and ends` here | some other text here
> 
> Why not interpret backticks across multiple lines as multi-line code
> spans?

This is currently done *if* the above is parsed as paragraph and not as
table.

> > If I parse for code spans line by line, the code span which
> > continues to the second line, is not found and therefore this is
> > interpreted as a table with two rows and two cells each. However, a
> > human reader would probably see the code span...
> 
> Right, which is why kramdown should interpret it as a code span, the
> same way a human reader would.  Principle of least surprise.

I know, I just wanted to let you know that the GFM approach with line by
line parsing does not work. Another example:

    This is *not* a code span, | it `continues on the
    second line and ends` here | some other text here

How would you interpret this? It looks like a table but the code span
seems to be continued on the second line... So I think the only way to
resolve this would be to post-process paragraphs which also may not
work correctly (I'm thinking of definition lists...). Regarding the
above example, I would treat it as a table with two rows and two
columns.

So what about this:

* A line that contains a pipe potentially starts a table if found on a
  block boundary.
* Read the line with the table parser and continue with the next lines
  until (a) a block boundary is found or (b) a line without a pipe is
  found.
* If we are in case (b), then this is definitely not a table and we go
  back to the first line and run the rest of the block parsers on the
  line.
* If we are in case (a), the whole text (i.e. all parsed lines
  together) is parsed with the code span parser. Then we have again
  multiple possibilities:

  - (a1) No code span contains a pipe
  - (a2) A code span that is on one line contains a pipe and there is
    another pipe on the same line
  - (a3) A code span that is on one line contains a pipe and there is no
    other pipe on the same line
  - (a4) A code span spanning two or more lines contains a pipe and
    there is another pipe on the line where the pipe is
  - (a5) A code span spanning two or more lines contains a pipe and
    there is no other pipe on the line where the pipe is

  In case (a1) everything is fine and we parse the lines as table lines
  even if a code span spans multiple lines.
  In case (a2) we have to account for the pipe character in the code
  span and not split the table line there but aside from this we can
  parse the lines as table lines.
  In cases (a3) and (a5) we have a line without a pipe (except the one
  inside the code span) and therefore we go back to the first line and
  run the rest of the block parsers on this first line.
  Case (a4) is difficult, here is an example:

      bla bla | bla bla `bla bla | bla bla
      bla bla bla bla` bla | bla bla bla

  Since this is similar to case (a2) I would treat this as a table.
  However, since the code span spans multiple lines, treating each line
  as a table line will destroy the code span and the second pipe on the
  first line will become another table cell separator since it is not
  enclosed in a code span. I don't think that this is a major problem,
  though.

-- Thomas

From matt at tidbits.com  Wed Oct 13 14:45:17 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Wed, 13 Oct 2010 11:45:17 -0700
Subject: [kramdown-users] feature suggestion: styling in code blocks
Message-ID: <C8DB4C28.203E%matt@tidbits.com>

I suggest that italics (emphasis) and bold (strong) be permitted inside
code blocks. To allow this, we need to use delimiters that are otherwise
extremely unlikely. For example:

~~~~
if (____condition____)
    ****// do stuff****
~~~~

.... where the four underscores and four asterisks are my suggested
delimiters for emphasis and bold. To make assurance doubly sure, I would
add a rule that these delimiters must occur in pairs on the same line, as
here.

m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com





From svicalifornia at gmail.com  Wed Oct 13 16:07:58 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Thu, 14 Oct 2010 03:07:58 +0700
Subject: [kramdown-users] feature suggestion: styling in code blocks
In-Reply-To: <C8DB4C28.203E%matt@tidbits.com>
References: <C8DB4C28.203E%matt@tidbits.com>
Message-ID: <20101014030758633186.23cd3634@gmail.com>

Sorry, Matt, but I really don't think it's a good idea to interpret code spans/blocks, even for emphasis and italics.  Besides, the underscore sequence you used might show up in Python, which makes the point that it's very difficult to devise an emphasis syntax that won't be valid code in some other language.  Code spans and blocks should be untouchable.

Shawn

On Wed, 13 Oct 2010 11:45:17 -0700, Matt Neuburg wrote:
> I suggest that italics (emphasis) and bold (strong) be permitted inside
> code blocks. To allow this, we need to use delimiters that are otherwise
> extremely unlikely. For example:
> 
> ~~~~
> if (____condition____)
>     ****// do stuff****
> ~~~~
> 
> .... where the four underscores and four asterisks are my suggested
> delimiters for emphasis and bold. To make assurance doubly sure, I would
> add a rule that these delimiters must occur in pairs on the same line, as
> here.
> 
> m.
> 
> -- 
> matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
> pantes anthropoi tou eidenai oregontai phusei
> Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
> AppleScript: the Definitive Guide, 2nd edition
> http://www.tidbits.com/matt/default.html#applescriptthings
> Take Control of Exploring & Customizing Snow Leopard
> http://tinyurl.com/kufyy8
> RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
> TidBITS, Mac news and reviews since 1990, http://www.tidbits.com
> 
> 
> 
> 
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From svicalifornia at gmail.com  Wed Oct 13 16:15:58 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Thu, 14 Oct 2010 03:15:58 +0700
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101013160029.747ee2e6@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
	<20101013101042.1fb1d6ae@noweto>
	<20101013191812071179.fb33691e@gmail.com>
	<20101013160029.747ee2e6@noweto>
Message-ID: <20101014031558463737.4493d9b4@gmail.com>

What about the possibility that automatic line-wrapping (as by mail agents) will break a code span across multiple lines?  Should it not still be considered a code span?  To fully support lazy indentation, code spans should be allowed to span multiple lines.

And if so, then this example:

>     This is *not* a code span, | it `continues on the
>     second line and ends` here | some other text here

is really like this:

>     This is *not* a code span, | it `continues on the second line and ends` here | some other text here

which yields a table with one row, three cells, the middle of which contains a code span.

This is really simple: regexp search the document for backtick pairs, and replace each pair's contents with a hash, GFM-style.  Then apply your other kramdown interpreters, line-by-line.  Then expand the hashes at the end.

If someone puts a couple of unescaped backticks in the document, then there's going to be a huge code span.  But that's what is supposed to happen.  Backticks wrap code spans, pure and simple.  If users don't want code spans, they should escape their backticks.

Shawn

On Wed, 13 Oct 2010 16:00:29 +0200, Thomas Leitner wrote:
> On 2010-10-13 19:18 +0700 Shawn Van Ittersum wrote:
>>> However, think about the following example:
>>> 
>>>     This is *not* a `code | span, it continues on the
>>>     second line and ends` here | some other text here
>> 
>> Why not interpret backticks across multiple lines as multi-line code
>> spans?
> 
> This is currently done *if* the above is parsed as paragraph and not as
> table.
> 
>>> If I parse for code spans line by line, the code span which
>>> continues to the second line, is not found and therefore this is
>>> interpreted as a table with two rows and two cells each. However, a
>>> human reader would probably see the code span...
>> 
>> Right, which is why kramdown should interpret it as a code span, the
>> same way a human reader would.  Principle of least surprise.
> 
> I know, I just wanted to let you know that the GFM approach with line by
> line parsing does not work. Another example:
> 
>     This is *not* a code span, | it `continues on the
>     second line and ends` here | some other text here
> 
> How would you interpret this? It looks like a table but the code span
> seems to be continued on the second line... So I think the only way to
> resolve this would be to post-process paragraphs which also may not
> work correctly (I'm thinking of definition lists...). Regarding the
> above example, I would treat it as a table with two rows and two
> columns.
> 
> So what about this:
> 
> * A line that contains a pipe potentially starts a table if found on a
>   block boundary.
> * Read the line with the table parser and continue with the next lines
>   until (a) a block boundary is found or (b) a line without a pipe is
>   found.
> * If we are in case (b), then this is definitely not a table and we go
>   back to the first line and run the rest of the block parsers on the
>   line.
> * If we are in case (a), the whole text (i.e. all parsed lines
>   together) is parsed with the code span parser. Then we have again
>   multiple possibilities:
> 
>   - (a1) No code span contains a pipe
>   - (a2) A code span that is on one line contains a pipe and there is
>     another pipe on the same line
>   - (a3) A code span that is on one line contains a pipe and there is no
>     other pipe on the same line
>   - (a4) A code span spanning two or more lines contains a pipe and
>     there is another pipe on the line where the pipe is
>   - (a5) A code span spanning two or more lines contains a pipe and
>     there is no other pipe on the line where the pipe is
> 
>   In case (a1) everything is fine and we parse the lines as table lines
>   even if a code span spans multiple lines.
>   In case (a2) we have to account for the pipe character in the code
>   span and not split the table line there but aside from this we can
>   parse the lines as table lines.
>   In cases (a3) and (a5) we have a line without a pipe (except the one
>   inside the code span) and therefore we go back to the first line and
>   run the rest of the block parsers on this first line.
>   Case (a4) is difficult, here is an example:
> 
>       bla bla | bla bla `bla bla | bla bla
>       bla bla bla bla` bla | bla bla bla
> 
>   Since this is similar to case (a2) I would treat this as a table.
>   However, since the code span spans multiple lines, treating each line
>   as a table line will destroy the code span and the second pipe on the
>   first line will become another table cell separator since it is not
>   enclosed in a code span. I don't think that this is a major problem,
>   though.
> 
> -- Thomas
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From svicalifornia at gmail.com  Wed Oct 13 16:15:58 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Thu, 14 Oct 2010 03:15:58 +0700
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101013160029.747ee2e6@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
	<20101013101042.1fb1d6ae@noweto>
	<20101013191812071179.fb33691e@gmail.com>
	<20101013160029.747ee2e6@noweto>
Message-ID: <20101014031558463737.4493d9b4@gmail.com>

What about the possibility that automatic line-wrapping (as by mail agents) will break a code span across multiple lines?  Should it not still be considered a code span?  To fully support lazy indentation, code spans should be allowed to span multiple lines.

And if so, then this example:

>     This is *not* a code span, | it `continues on the
>     second line and ends` here | some other text here

is really like this:

>     This is *not* a code span, | it `continues on the second line and ends` here | some other text here

which yields a table with one row, three cells, the middle of which contains a code span.

This is really simple: regexp search the document for backtick pairs, and replace each pair's contents with a hash, GFM-style.  Then apply your other kramdown interpreters, line-by-line.  Then expand the hashes at the end.

If someone puts a couple of unescaped backticks in the document, then there's going to be a huge code span.  But that's what is supposed to happen.  Backticks wrap code spans, pure and simple.  If users don't want code spans, they should escape their backticks.

Shawn

On Wed, 13 Oct 2010 16:00:29 +0200, Thomas Leitner wrote:
> On 2010-10-13 19:18 +0700 Shawn Van Ittersum wrote:
>>> However, think about the following example:
>>> 
>>>     This is *not* a `code | span, it continues on the
>>>     second line and ends` here | some other text here
>> 
>> Why not interpret backticks across multiple lines as multi-line code
>> spans?
> 
> This is currently done *if* the above is parsed as paragraph and not as
> table.
> 
>>> If I parse for code spans line by line, the code span which
>>> continues to the second line, is not found and therefore this is
>>> interpreted as a table with two rows and two cells each. However, a
>>> human reader would probably see the code span...
>> 
>> Right, which is why kramdown should interpret it as a code span, the
>> same way a human reader would.  Principle of least surprise.
> 
> I know, I just wanted to let you know that the GFM approach with line by
> line parsing does not work. Another example:
> 
>     This is *not* a code span, | it `continues on the
>     second line and ends` here | some other text here
> 
> How would you interpret this? It looks like a table but the code span
> seems to be continued on the second line... So I think the only way to
> resolve this would be to post-process paragraphs which also may not
> work correctly (I'm thinking of definition lists...). Regarding the
> above example, I would treat it as a table with two rows and two
> columns.
> 
> So what about this:
> 
> * A line that contains a pipe potentially starts a table if found on a
>   block boundary.
> * Read the line with the table parser and continue with the next lines
>   until (a) a block boundary is found or (b) a line without a pipe is
>   found.
> * If we are in case (b), then this is definitely not a table and we go
>   back to the first line and run the rest of the block parsers on the
>   line.
> * If we are in case (a), the whole text (i.e. all parsed lines
>   together) is parsed with the code span parser. Then we have again
>   multiple possibilities:
> 
>   - (a1) No code span contains a pipe
>   - (a2) A code span that is on one line contains a pipe and there is
>     another pipe on the same line
>   - (a3) A code span that is on one line contains a pipe and there is no
>     other pipe on the same line
>   - (a4) A code span spanning two or more lines contains a pipe and
>     there is another pipe on the line where the pipe is
>   - (a5) A code span spanning two or more lines contains a pipe and
>     there is no other pipe on the line where the pipe is
> 
>   In case (a1) everything is fine and we parse the lines as table lines
>   even if a code span spans multiple lines.
>   In case (a2) we have to account for the pipe character in the code
>   span and not split the table line there but aside from this we can
>   parse the lines as table lines.
>   In cases (a3) and (a5) we have a line without a pipe (except the one
>   inside the code span) and therefore we go back to the first line and
>   run the rest of the block parsers on this first line.
>   Case (a4) is difficult, here is an example:
> 
>       bla bla | bla bla `bla bla | bla bla
>       bla bla bla bla` bla | bla bla bla
> 
>   Since this is similar to case (a2) I would treat this as a table.
>   However, since the code span spans multiple lines, treating each line
>   as a table line will destroy the code span and the second pipe on the
>   first line will become another table cell separator since it is not
>   enclosed in a code span. I don't think that this is a major problem,
>   though.
> 
> -- Thomas
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From t_leitner at gmx.at  Thu Oct 14 02:30:14 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 14 Oct 2010 08:30:14 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101014031558463737.4493d9b4@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
	<20101013101042.1fb1d6ae@noweto>
	<20101013191812071179.fb33691e@gmail.com>
	<20101013160029.747ee2e6@noweto>
	<20101014031558463737.4493d9b4@gmail.com>
Message-ID: <20101014083014.67c48965@noweto>

On 2010-10-14 03:15 +0700 Shawn Van Ittersum wrote:
> What about the possibility that automatic line-wrapping (as by mail
> agents) will break a code span across multiple lines?  Should it not
> still be considered a code span?  To fully support lazy indentation,
> code spans should be allowed to span multiple lines.

This is already allowed...
 
> And if so, then this example:
> 
> >     This is *not* a code span, | it `continues on the
> >     second line and ends` here | some other text here
> 
> is really like this:
> 
> >     This is *not* a code span, | it `continues on the second line
> > and ends` here | some other text here
> 
> which yields a table with one row, three cells, the middle of which
> contains a code span.

Yeah, but if you think a bit further the whole table syntax consists of
lines separated from other block level elements by blank lines (in most
cases). So if we follow your thoughts then each table could only have
one table row...

Most elements in kramdown work fine when wrapped, headers, fenced code
blocks and tables do not!

> This is really simple: regexp search the document for backtick pairs,
> and replace each pair's contents with a hash, GFM-style.  Then apply
> your other kramdown interpreters, line-by-line.  Then expand the
> hashes at the end.

Really simple? Just look at all the Markdown processors that use the
hash-extract-replace mechanism... They all have serious problems when
it comes down to produce valid HTML (although, one has to say, they are
perfectly compatible to Markdown.pl since Markdown.pl also uses this
approach). 

Your above idea will obviously not work correctly, for example:

    This is a paragraph `with a lone backtick.

    This is another paragraph with a` lone backtick.

The intention of the writer is clear: There are two paragraphs each
containing a backtick. Since there is only one backtick in each
paragraph, the writer correctly thought that he can escape it but he
does not have to - and chose the latter.
 
> If someone puts a couple of unescaped backticks in the document, then
> there's going to be a huge code span.  But that's what is supposed to
> happen.  Backticks wrap code spans, pure and simple.  If users don't
> want code spans, they should escape their backticks.

No, that's not supposed to happen since code span are span level
elements and cannot cross, per definition, the boundaries of block
level elements! And don't forget the rule about how markup in
Markdown/kramdown works: if something is marked up correctly (e.g. two
backticks in a paragraph delimiting a code span), it is correctly
parsed and used, otherwise (e.g. only one backtick) it is output as is.
So one does *not* have to escape a single backtick or a single asterisk
or a single opening bracket or ...

-- Thomas

From t_leitner at gmx.at  Thu Oct 14 02:34:48 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 14 Oct 2010 08:34:48 +0200
Subject: [kramdown-users] feature suggestion: styling in code blocks
In-Reply-To: <20101014030758633186.23cd3634@gmail.com>
References: <C8DB4C28.203E%matt@tidbits.com>
	<20101014030758633186.23cd3634@gmail.com>
Message-ID: <20101014083448.52c9f548@noweto>

On 2010-10-14 03:07 +0700 Shawn Van Ittersum wrote:
> Sorry, Matt, but I really don't think it's a good idea to interpret
> code spans/blocks, even for emphasis and italics.  Besides, the
> underscore sequence you used might show up in Python, which makes the
> point that it's very difficult to devise an emphasis syntax that
> won't be valid code in some other language.  Code spans and blocks
> should be untouchable.

I have to agree with Shawn, I don't think that this is a good idea. In
any case, you can use pure HTML to work around this:

    <pre>
    if (<em>condition</em>)
    <strong>// do stuff</strong>
    </pre>

Or a mixture of HTML and kramdown syntax:

    <pre markdown="span">
    if (_condition_)
    **// do stuff**
    </pre>

-- Thomas

From svicalifornia at gmail.com  Thu Oct 14 06:24:42 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Thu, 14 Oct 2010 17:24:42 +0700
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101014083014.67c48965@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
	<20101013101042.1fb1d6ae@noweto>
	<20101013191812071179.fb33691e@gmail.com>
	<20101013160029.747ee2e6@noweto>
	<20101014031558463737.4493d9b4@gmail.com>
	<20101014083014.67c48965@noweto>
Message-ID: <20101014172442504996.61348503@gmail.com>



Shawn

On Thu, 14 Oct 2010 08:30:14 +0200, Thomas Leitner wrote:
> Your above idea will obviously not work correctly, for example:
> 
>     This is a paragraph `with a lone backtick.
> 
>     This is another paragraph with a` lone backtick.
> 
> The intention of the writer is clear: There are two paragraphs each
> containing a backtick. Since there is only one backtick in each
> paragraph, the writer correctly thought that he can escape it but he
> does not have to - and chose the latter.

But I would interpret it as a single paragraph block containing a multi-line code span.  The failure to escape the backticks is not kramdown's problem to fix.  I am strongly advocating that backticks are for code spans only, and they *must* be escaped if they are not intended to open code spans.

Shawn

>> If someone puts a couple of unescaped backticks in the document, then
>> there's going to be a huge code span.  But that's what is supposed to
>> happen.  Backticks wrap code spans, pure and simple.  If users don't
>> want code spans, they should escape their backticks.
> 
> No, that's not supposed to happen since code span are span level
> elements and cannot cross, per definition, the boundaries of block
> level elements! And don't forget the rule about how markup in
> Markdown/kramdown works: if something is marked up correctly (e.g. two
> backticks in a paragraph delimiting a code span), it is correctly
> parsed and used, otherwise (e.g. only one backtick) it is output as is.
> So one does *not* have to escape a single backtick or a single asterisk
> or a single opening bracket or ...

Okay, but if a code span is opened on one line and closed several lines later, it is still contained inside one block.  The so-called block boundaries (blank lines, etc.) inside the code span are irrelevant, because kramdown is supposed to be ignoring the contents of the code span.

Back to your example:

    This is a paragraph `with a lone backtick.

    This is another paragraph with a` lone backtick.

This would yield a single paragraph block:

    <p>This is a paragraph <code>with a lone backtick.

    This is another paragraph with a</code> lone backtick.</p>

The code span is wholly contained by the paragraph block.

Shawn

From svicalifornia at gmail.com  Thu Oct 14 06:27:45 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Thu, 14 Oct 2010 17:27:45 +0700
Subject: [kramdown-users] feature suggestion: styling in code blocks
In-Reply-To: <20101014083448.52c9f548@noweto>
References: <C8DB4C28.203E%matt@tidbits.com>
	<20101014030758633186.23cd3634@gmail.com>
	<20101014083448.52c9f548@noweto>
Message-ID: <20101014172745431873.80d082a4@gmail.com>

At issue here is whether kramdown shall allow multi-line code spans.  I say it should.  And if it does, then it should also allow blank lines in the middle of code spans.  And then code spans will not be able to cross blocks, because any block delimiters crossed will be contained by the code span and ignored by kramdown, and thus are not considered block delimiters by virtue of being contained by the code span.

Shawn

From matt at tidbits.com  Thu Oct 14 10:17:45 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Thu, 14 Oct 2010 07:17:45 -0700
Subject: [kramdown-users] feature suggestion: styling in code blocks
In-Reply-To: <20101014083448.52c9f548@noweto>
Message-ID: <C8DC5EFE.2094%matt@tidbits.com>

On 10/13/10 11:34 PM, thus spake "Thomas Leitner" <t_leitner at gmx.at>:

>On 2010-10-14 03:07 +0700 Shawn Van Ittersum wrote:
>>Sorry, Matt, but I really don't think it's a good idea to interpret
>>code spans/blocks, even for emphasis and italics.  Besides, the
>>underscore sequence you used might show up in Python, which makes the
>>point that it's very difficult to devise an emphasis syntax that
>>won't be valid code in some other language.  Code spans and blocks
>>should be untouchable.
>
>I have to agree with Shawn, I don't think that this is a good idea.

No problem, I can implement this myself in post-processing. m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com





From t_leitner at gmx.at  Thu Oct 14 14:53:58 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 14 Oct 2010 20:53:58 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101014172442504996.61348503@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
	<20101013101042.1fb1d6ae@noweto>
	<20101013191812071179.fb33691e@gmail.com>
	<20101013160029.747ee2e6@noweto>
	<20101014031558463737.4493d9b4@gmail.com>
	<20101014083014.67c48965@noweto>
	<20101014172442504996.61348503@gmail.com>
Message-ID: <20101014205358.0e21e630@noweto>

On 2010-10-14 17:24 +0700 Shawn Van Ittersum wrote:
> On Thu, 14 Oct 2010 08:30:14 +0200, Thomas Leitner wrote:
> > Your above idea will obviously not work correctly, for example:
> > 
> >     This is a paragraph `with a lone backtick.
> > 
> >     This is another paragraph with a` lone backtick.
> > 
> > The intention of the writer is clear: There are two paragraphs each
> > containing a backtick. Since there is only one backtick in each
> > paragraph, the writer correctly thought that he can escape it but he
> > does not have to - and chose the latter.
> 
> But I would interpret it as a single paragraph block containing a
> multi-line code span.  The failure to escape the backticks is not
> kramdown's problem to fix.  I am strongly advocating that backticks
> are for code spans only, and they *must* be escaped if they are not
> intended to open code spans.

A paragraph in Markdown/kramdown is, by definition, separated from
other paragraphs by blank lines (or eventually some other block level
elements in kramdown). So there is no way the above example can be
interpreted as only one paragraph! And, as I said, the author did
nothing wrong in *not* escaping the backticks since only one backtick
is used in each paragraph (which is allowed by Markdown/kramdown).

If we follow your train of thought, then invalid (note that there
basically does not exist invalid markup in kramdown/Markdown since no
errors are produced) markup for any special character that is handled by
kramdown will run amok on texts.

> >> If someone puts a couple of unescaped backticks in the document,
> >> then there's going to be a huge code span.  But that's what is
> >> supposed to happen.  Backticks wrap code spans, pure and simple.
> >> If users don't want code spans, they should escape their backticks.
> > 
> > No, that's not supposed to happen since code span are span level
> > elements and cannot cross, per definition, the boundaries of block
> > level elements! And don't forget the rule about how markup in
> > Markdown/kramdown works: if something is marked up correctly (e.g.
> > two backticks in a paragraph delimiting a code span), it is
> > correctly parsed and used, otherwise (e.g. only one backtick) it is
> > output as is. So one does *not* have to escape a single backtick or
> > a single asterisk or a single opening bracket or ...
> 
> Okay, but if a code span is opened on one line and closed several
> lines later, it is still contained inside one block.  The so-called
> block boundaries (blank lines, etc.) inside the code span are
> irrelevant, because kramdown is supposed to be ignoring the contents
> of the code span.

No, it is not since paragraphs, as stated above, are *closed* by blank
lines. This is one of the most important features of the original
Markdown syntax! Since code *spans* are **span level elements** they
cannot cross block boundaries and therefore cannot contain block
boundaries.

As for the "ignoring the contents of the code spans": kramdown *is*
ignoring the contents of code spans, just look at the implementation
(`lib/kramdown/parser/kramdown/codespan.rb` and
`lib/kramdown/converter/html.rb`). So please don't use this invalid
argument anymore!

The fact is (and the problem which lead to this very thread) that the
current table syntax necessitates a syntax which sometimes produces not
wanted code spans. And this is not the failure of the code span
implementation but of the table syntax!

-- Thomas

From t_leitner at gmx.at  Thu Oct 14 15:37:49 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 14 Oct 2010 21:37:49 +0200
Subject: [kramdown-users] feature suggestion: styling in code blocks
In-Reply-To: <20101014172745431873.80d082a4@gmail.com>
References: <C8DB4C28.203E%matt@tidbits.com>
	<20101014030758633186.23cd3634@gmail.com>
	<20101014083448.52c9f548@noweto>
	<20101014172745431873.80d082a4@gmail.com>
Message-ID: <20101014213749.25ccca27@noweto>

On 2010-10-14 17:27 +0700 Shawn Van Ittersum wrote:
> At issue here is whether kramdown shall allow multi-line code spans.
> I say it should.  And if it does, then it should also allow blank
> lines in the middle of code spans.  And then code spans will not be
> able to cross blocks, because any block delimiters crossed will be
> contained by the code span and ignored by kramdown, and thus are not
> considered block delimiters by virtue of being contained by the code
> span.

No, this is not at issue: span level elements can only occur *within*
block level elements and blank lines separate consecutive paragraphs.
Ergo: code spans cannot include blank lines when used in paragraphs.
The same is true for all other block level elements that can contain
code spans.

-- Thomas

From svicalifornia at gmail.com  Thu Oct 14 15:57:08 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Fri, 15 Oct 2010 02:57:08 +0700
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101014205358.0e21e630@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
	<20101013101042.1fb1d6ae@noweto>
	<20101013191812071179.fb33691e@gmail.com>
	<20101013160029.747ee2e6@noweto>
	<20101014031558463737.4493d9b4@gmail.com>
	<20101014083014.67c48965@noweto>
	<20101014172442504996.61348503@gmail.com>
	<20101014205358.0e21e630@noweto>
Message-ID: <20101015025708553584.8b32d2f2@gmail.com>

Hi Thomas,

Are multi-line code spans allowed in kramdown?  Must they be written as indented code blocks?

Shawn

On Thu, 14 Oct 2010 20:53:58 +0200, Thomas Leitner wrote:
> No, it is not since paragraphs, as stated above, are *closed* by blank
> lines. This is one of the most important features of the original
> Markdown syntax! Since code *spans* are **span level elements** they
> cannot cross block boundaries and therefore cannot contain block
> boundaries.
> 
> As for the "ignoring the contents of the code spans": kramdown *is*
> ignoring the contents of code spans, just look at the implementation
> (`lib/kramdown/parser/kramdown/codespan.rb` and
> `lib/kramdown/converter/html.rb`). So please don't use this invalid
> argument anymore!
> 
> The fact is (and the problem which lead to this very thread) that the
> current table syntax necessitates a syntax which sometimes produces not
> wanted code spans. And this is not the failure of the code span
> implementation but of the table syntax!

From t_leitner at gmx.at  Fri Oct 15 04:04:14 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Fri, 15 Oct 2010 10:04:14 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101015025708553584.8b32d2f2@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
	<20101013101042.1fb1d6ae@noweto>
	<20101013191812071179.fb33691e@gmail.com>
	<20101013160029.747ee2e6@noweto>
	<20101014031558463737.4493d9b4@gmail.com>
	<20101014083014.67c48965@noweto>
	<20101014172442504996.61348503@gmail.com>
	<20101014205358.0e21e630@noweto>
	<20101015025708553584.8b32d2f2@gmail.com>
Message-ID: <20101015100414.38457171@noweto>

On 2010-10-15 02:57 +0700 Shawn Van Ittersum wrote:
> Are multi-line code spans allowed in kramdown?  Must they be written
> as indented code blocks?

If you think of something like this:

    `this is a codespan
    which spans
    serveral lines`

then yes, multi-line code spans are available in kramdown. However, if
you view the output in a browser you will see that whitespace
normalization is applied to the `<code>` element (multiple whitespace
characters are compressed into one space character).

-- Thomas

From t_leitner at gmx.at  Sat Oct 16 03:24:47 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Sat, 16 Oct 2010 09:24:47 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <C8D72AFC.1EA4%matt@tidbits.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
Message-ID: <20101016092447.4a07430d@noweto>

On 2010-10-10 08:34 -0700 Matt Neuburg wrote:
> require 'kramdown'
> s = <<END
> The operator `||` is called logical-or.
> END
> puts Kramdown::Document.new(s).to_html

I have just pushed the latest changes to the Github repository. Pipe
characters in code spans in tables do *not* need to be escaped anymore.
I hope the change fixes all the discussed problems. The above example
works now correctly!

-- Thomas

From svicalifornia at gmail.com  Sun Oct 17 13:06:34 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Mon, 18 Oct 2010 00:06:34 +0700
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101016092447.4a07430d@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com> <20101016092447.4a07430d@noweto>
Message-ID: <20101018000634586799.2073c92f@gmail.com>

Thank you, Thomas!  And thanks for being attentive to feedback and patient through our discussions.  Though we don't always agree, I am very appreciative of your thoughtful concern for each issue.  You are doing a great job managing this project.

Shawn

On Sat, 16 Oct 2010 09:24:47 +0200, Thomas Leitner wrote:
> On 2010-10-10 08:34 -0700 Matt Neuburg wrote:
>> require 'kramdown'
>> s = <<END
>> The operator `||` is called logical-or.
>> END
>> puts Kramdown::Document.new(s).to_html
> 
> I have just pushed the latest changes to the Github repository. Pipe
> characters in code spans in tables do *not* need to be escaped anymore.
> I hope the change fixes all the discussed problems. The above example
> works now correctly!
> 
> -- Thomas
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From svicalifornia at gmail.com  Sun Oct 17 13:10:32 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Mon, 18 Oct 2010 00:10:32 +0700
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101015100414.38457171@noweto>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
	<20101013101042.1fb1d6ae@noweto>
	<20101013191812071179.fb33691e@gmail.com>
	<20101013160029.747ee2e6@noweto>
	<20101014031558463737.4493d9b4@gmail.com>
	<20101014083014.67c48965@noweto>
	<20101014172442504996.61348503@gmail.com>
	<20101014205358.0e21e630@noweto>
	<20101015025708553584.8b32d2f2@gmail.com>
	<20101015100414.38457171@noweto>
Message-ID: <20101018001032030270.4b89856a@gmail.com>

On Fri, 15 Oct 2010 10:04:14 +0200, Thomas Leitner wrote:
> However, if
> you view the output in a browser you will see that whitespace
> normalization is applied to the `<code>` element (multiple whitespace
> characters are compressed into one space character).

This brings up another question: will Kramdown leave code indentation intact in code blocks?  (I'm pretty sure it does, from my own experience.)  What about multiple spaces in the middle of a line?  It would be strange if code indentation or comment alignment was lost in parsing a code block.

Does kramdown treat code spans different than code blocks, with regard to spaces?

Shawn

From t_leitner at gmx.at  Mon Oct 18 04:49:18 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Mon, 18 Oct 2010 10:49:18 +0200
Subject: [kramdown-users] kramdown table-making has gone completely
 insane
In-Reply-To: <20101018001032030270.4b89856a@gmail.com>
References: <C8D72AFC.1EA4%matt@tidbits.com>
	<20101010230338632089.15573510@gmail.com>
	<20101011095000.217afd84@noweto>
	<20101011180900604973.3e8d4e59@gmail.com>
	<20101011155452.75c8f672@noweto>
	<20101012014140273990.efe1a504@gmail.com>
	<20101012093422.4381ad35@noweto>
	<20101012154801867549.f68e3490@gmail.com>
	<20101012152111.4f22eabe@noweto>
	<20101013013307002686.5a84a3d2@gmail.com>
	<20101013101042.1fb1d6ae@noweto>
	<20101013191812071179.fb33691e@gmail.com>
	<20101013160029.747ee2e6@noweto>
	<20101014031558463737.4493d9b4@gmail.com>
	<20101014083014.67c48965@noweto>
	<20101014172442504996.61348503@gmail.com>
	<20101014205358.0e21e630@noweto>
	<20101015025708553584.8b32d2f2@gmail.com>
	<20101015100414.38457171@noweto>
	<20101018001032030270.4b89856a@gmail.com>
Message-ID: <20101018104918.36b900ef@noweto>

On 2010-10-18 00:10 +0700 Shawn Van Ittersum wrote:
> On Fri, 15 Oct 2010 10:04:14 +0200, Thomas Leitner wrote:
> > However, if
> > you view the output in a browser you will see that whitespace
> > normalization is applied to the `<code>` element (multiple
> > whitespace characters are compressed into one space character).
> 
> This brings up another question: will Kramdown leave code indentation
> intact in code blocks?  (I'm pretty sure it does, from my own
> experience.)  What about multiple spaces in the middle of a line?  It
> would be strange if code indentation or comment alignment was lost in
> parsing a code block.

Yes, kramdown does not alter indentation in code blocks (aside from
stripping the first four spaces or one tab from the code block since
this is just the syntax part of the code block) nor the spaces in the
middle of a line.
 
> Does kramdown treat code spans different than code blocks, with
> regard to spaces?

No, but if you use multiple backticks for a code span, the first and
last space are removed. E.g.

    ``  test  ``

becomes

    <p><code> test </code></p>

-- Thomas

From t_leitner at gmx.at  Sat Oct 23 05:45:53 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Sat, 23 Oct 2010 11:45:53 +0200
Subject: [kramdown-users] API changes
Message-ID: <20101023114553.2d87bf31@noweto>

Hi everyone,

I have just pushed the latest changes to the Github repository. Aside
from some bug fixes and documentation updates the API has been changed.

Parsers and converters can now be used with a Kramdown::Document
instance. This implies that all the needed information is stored in the
element tree itself. The changes decouple the parser and converter
classes from the Kramdown::Document class and make the latter lighter.
Until now the Document class stored parsing and conversion information
that is of no use to a normal user of the kramdown library.

The parsing information that needs to be available for converters is
now stored in the options hash of the root element. The conversion
information is now only used by the converters themselves.

I have also updated the API documentation to be more useful, some more
documentation updates will still come before the 0.12.0 release.

The current API will probably be the final API for the stable 1.0.0
release. If you find the API lacking, please tell me now so that we
can fix it *before* a 1.0.0 release!

Best regards,
  Thomas

From t_leitner at gmx.at  Thu Oct 28 04:05:06 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 28 Oct 2010 10:05:06 +0200
Subject: [kramdown-users] Performance optimizations
Message-ID: <20101028100506.294ae3b9@noweto>

Hi everybody,

there was a major performance drop in the 0.11.0 version of kramdown
due to the preservation of the ordering of the element attributes.

I did some investigations and found that turning off the garbage
collector makes kramdown run much faster. So it seems that kramdown
creates too much objects and the garbage collector has to do too much.
After some tweaking the performance of the next release should be much
better again. Also, 15% less objects are created in comparison with the
0.11.0 release.

Here are some performance numbers for various kramdown versions on
different ruby versions. All benchmarks are run on an Intel Core i5
M520 with 4GB memory on Ubuntu 10.10 with 64bit. The data is generated
with the new `benchmark/benchmark.sh` script. You can reproduce the
data by cloning the git repository and running

    bash benchmark/benchmark.sh -k "REL_0_1_0 REL_0_6_0 REL_0_9_0 REL_0_10_0 REL_0_11_0 master"

The generated data can be found in /tmp/kramdown-benchmark. By default
all tags and the master version are used for the kramdown versions if
not overridden with the `-k` option. The script uses rvm to switch
between ruby versions, so you need that too! You will also need gnuplot
so that the images can be generated.

jruby-1.5.0RC1
#       REL_0_1_0 ||  REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || REL_0_11_0 ||     master
  1       0.23100       0.21300       0.24600       0.17700       0.18600       0.25500
  2       0.15100       0.23300       0.21500       0.21300       0.24600       0.21600
  4       0.17200       0.17400       0.19600       0.18700       0.33900       0.23600
  8       0.29300       0.28600       0.31200       0.29600       0.37600       0.31800
 16       0.37400       0.35600       0.52100       0.50500       0.42200       0.45600
 32       0.59000       0.55700       0.59900       0.59900       0.70300       0.64900

ruby 1.8.5-p231
#       REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || REL_0_11_0 ||     master
  1       0.01000       0.01051       0.01136       0.01195       0.01175
  2       0.02109       0.02174       0.02119       0.02277       0.02039
  4       0.03730       0.04732       0.04732       0.04573       0.04850
  8       0.09734       0.08701       0.08335       0.11130       0.08702
 16       0.18256       0.20505       0.19535       0.23545       0.20143
 32       0.41248       0.44407       0.46074       0.49541       0.46472

ruby 1.8.6-p399
#       REL_0_1_0 ||  REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || REL_0_11_0 ||     master
  1       0.01241       0.01014       0.01191       0.01134       0.01222       0.01117
  2       0.01608       0.02513       0.02176       0.02733       0.02321       0.02155
  4       0.03024       0.03929       0.04743       0.04753       0.04654       0.04751
  8       0.06593       0.07937       0.08792       0.08623       0.09291       0.08353
 16       0.13082       0.17244       0.19194       0.18539       0.22715       0.18366
 32       0.27855       0.38740       0.42560       0.42649       0.47899       0.49377

ruby 1.8.7-p249
#       REL_0_1_0 ||  REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || REL_0_11_0 ||     master
  1       0.00895       0.01051       0.01264       0.01274       0.01242       0.01141
  2       0.01722       0.02107       0.02299       0.02229       0.02382       0.02318
  4       0.03070       0.04586       0.04909       0.04872       0.04764       0.04919
  8       0.06811       0.08272       0.09249       0.09098       0.11013       0.08745
 16       0.16567       0.17962       0.19554       0.18920       0.22599       0.18849
 32       0.28729       0.39177       0.53765       0.44901       0.49453       0.39598

ruby 1.8.7-p302
#       REL_0_1_0 ||  REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || REL_0_11_0 ||     master
  1       0.00966       0.01171       0.01291       0.01286       0.01387       0.01261
  2       0.01778       0.02483       0.02535       0.02476       0.03128       0.02728
  4       0.03422       0.05098       0.05400       0.05377       0.05474       0.05435
  8       0.07529       0.09204       0.09825       0.09899       0.12087       0.09575
 16       0.18189       0.19847       0.21057       0.25462       0.24744       0.20743
 32       0.31704       0.43067       0.51242       0.51385       0.53427       0.44922

ruby 1.9.2-p0
#       REL_0_1_0 ||  REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || REL_0_11_0 ||     master
  1       0.01357       0.00870       0.00934       0.00935       0.01056       0.00968
  2       0.02199       0.01758       0.01957       0.01947       0.02030       0.01938
  4       0.04581       0.03701       0.04082       0.04215       0.04545       0.04122
  8       0.09261       0.07473       0.08601       0.08608       0.08778       0.08847
 16       0.18589       0.14448       0.15985       0.16125       0.18634       0.15448
 32       0.38448       0.31029       0.35264       0.35203       0.37317       0.34102

-- Thomas

From svicalifornia at gmail.com  Thu Oct 28 05:45:17 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Thu, 28 Oct 2010 16:45:17 +0700
Subject: [kramdown-users] Performance optimizations
In-Reply-To: <20101028100506.294ae3b9@noweto>
References: <20101028100506.294ae3b9@noweto>
Message-ID: <20101028164517782845.e98ea417@gmail.com>

Hi Thomas,

How do you implement preservation of element attribute order?  Would you please point me to it in the code?

Thanks,
Shawn

On Thu, 28 Oct 2010 10:05:06 +0200, Thomas Leitner wrote:
> Hi everybody,
> 
> there was a major performance drop in the 0.11.0 version of kramdown
> due to the preservation of the ordering of the element attributes.
> 
> I did some investigations and found that turning off the garbage
> collector makes kramdown run much faster. So it seems that kramdown
> creates too much objects and the garbage collector has to do too much.
> After some tweaking the performance of the next release should be much
> better again. Also, 15% less objects are created in comparison with the
> 0.11.0 release.
> 
> Here are some performance numbers for various kramdown versions on
> different ruby versions. All benchmarks are run on an Intel Core i5
> M520 with 4GB memory on Ubuntu 10.10 with 64bit. The data is generated
> with the new `benchmark/benchmark.sh` script. You can reproduce the
> data by cloning the git repository and running
> 
>     bash benchmark/benchmark.sh -k "REL_0_1_0 REL_0_6_0 REL_0_9_0 
> REL_0_10_0 REL_0_11_0 master"
> 
> The generated data can be found in /tmp/kramdown-benchmark. By default
> all tags and the master version are used for the kramdown versions if
> not overridden with the `-k` option. The script uses rvm to switch
> between ruby versions, so you need that too! You will also need gnuplot
> so that the images can be generated.
> 
> jruby-1.5.0RC1
> #       REL_0_1_0 ||  REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || 
> REL_0_11_0 ||     master
>   1       0.23100       0.21300       0.24600       0.17700       
> 0.18600       0.25500
>   2       0.15100       0.23300       0.21500       0.21300       
> 0.24600       0.21600
>   4       0.17200       0.17400       0.19600       0.18700       
> 0.33900       0.23600
>   8       0.29300       0.28600       0.31200       0.29600       
> 0.37600       0.31800
>  16       0.37400       0.35600       0.52100       0.50500       
> 0.42200       0.45600
>  32       0.59000       0.55700       0.59900       0.59900       
> 0.70300       0.64900
> 
> ruby 1.8.5-p231
> #       REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || REL_0_11_0 ||     master
>   1       0.01000       0.01051       0.01136       0.01195       0.01175
>   2       0.02109       0.02174       0.02119       0.02277       0.02039
>   4       0.03730       0.04732       0.04732       0.04573       0.04850
>   8       0.09734       0.08701       0.08335       0.11130       0.08702
>  16       0.18256       0.20505       0.19535       0.23545       0.20143
>  32       0.41248       0.44407       0.46074       0.49541       0.46472
> 
> ruby 1.8.6-p399
> #       REL_0_1_0 ||  REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || 
> REL_0_11_0 ||     master
>   1       0.01241       0.01014       0.01191       0.01134       
> 0.01222       0.01117
>   2       0.01608       0.02513       0.02176       0.02733       
> 0.02321       0.02155
>   4       0.03024       0.03929       0.04743       0.04753       
> 0.04654       0.04751
>   8       0.06593       0.07937       0.08792       0.08623       
> 0.09291       0.08353
>  16       0.13082       0.17244       0.19194       0.18539       
> 0.22715       0.18366
>  32       0.27855       0.38740       0.42560       0.42649       
> 0.47899       0.49377
> 
> ruby 1.8.7-p249
> #       REL_0_1_0 ||  REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || 
> REL_0_11_0 ||     master
>   1       0.00895       0.01051       0.01264       0.01274       
> 0.01242       0.01141
>   2       0.01722       0.02107       0.02299       0.02229       
> 0.02382       0.02318
>   4       0.03070       0.04586       0.04909       0.04872       
> 0.04764       0.04919
>   8       0.06811       0.08272       0.09249       0.09098       
> 0.11013       0.08745
>  16       0.16567       0.17962       0.19554       0.18920       
> 0.22599       0.18849
>  32       0.28729       0.39177       0.53765       0.44901       
> 0.49453       0.39598
> 
> ruby 1.8.7-p302
> #       REL_0_1_0 ||  REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || 
> REL_0_11_0 ||     master
>   1       0.00966       0.01171       0.01291       0.01286       
> 0.01387       0.01261
>   2       0.01778       0.02483       0.02535       0.02476       
> 0.03128       0.02728
>   4       0.03422       0.05098       0.05400       0.05377       
> 0.05474       0.05435
>   8       0.07529       0.09204       0.09825       0.09899       
> 0.12087       0.09575
>  16       0.18189       0.19847       0.21057       0.25462       
> 0.24744       0.20743
>  32       0.31704       0.43067       0.51242       0.51385       
> 0.53427       0.44922
> 
> ruby 1.9.2-p0
> #       REL_0_1_0 ||  REL_0_6_0 ||  REL_0_9_0 || REL_0_10_0 || 
> REL_0_11_0 ||     master
>   1       0.01357       0.00870       0.00934       0.00935       
> 0.01056       0.00968
>   2       0.02199       0.01758       0.01957       0.01947       
> 0.02030       0.01938
>   4       0.04581       0.03701       0.04082       0.04215       
> 0.04545       0.04122
>   8       0.09261       0.07473       0.08601       0.08608       
> 0.08778       0.08847
>  16       0.18589       0.14448       0.15985       0.16125       
> 0.18634       0.15448
>  32       0.38448       0.31029       0.35264       0.35203       
> 0.37317       0.34102
> 
> -- Thomas
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From t_leitner at gmx.at  Thu Oct 28 08:04:05 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 28 Oct 2010 14:04:05 +0200
Subject: [kramdown-users] Performance optimizations
In-Reply-To: <20101028164517782845.e98ea417@gmail.com>
References: <20101028100506.294ae3b9@noweto>
	<20101028164517782845.e98ea417@gmail.com>
Message-ID: <20101028140405.4bb383f1@noweto>

On 2010-10-28 16:45 +0700 Shawn Van Ittersum wrote:
> Hi Thomas,
> 
> How do you implement preservation of element attribute order?  Would
> you please point me to it in the code?

Have a look at the file lib/kramdown/utils/ordered_hash.rb. The class
OrderedHash implements the needed methods to mimick a basic Hash class
that preserves the insertion order.

Note, however, that this is *not* needed under Ruby 1.9 since the
built-in Hash does automatically preserve the insertion order. One of
the performance improvements was to alias `OrderedHash = Hash` in case
of Ruby 1.9.

-- Thomas