tag, this is a bit heavy-handed. I'd like to be able to say <title smartypants="1">, or maybe <title markdown="smarty">, so that kramdown will apply its *only* curly-quoting smartification inside the <title> tag without wasting time looking for *any* other Markdown material. Indeed, asterisks inside the <title> tag should be legal without triggering an incorrect additional tag inside the tag. So for example: <title markdown="smarty">Isn't this a *test*?

, but it dives into
all other displayed content and makes curly quotes.

Now, kramdown lacks this; it is all or nothing. If you say
<...markdown="1">, everything inside that tag gets the ENTIRE markdown
treatment, plus smartypants treatment. But for something like the 
tag, this is a bit heavy-handed. I'd like to be able to say <title
smartypants="1">, or maybe <title markdown="smarty">, so that kramdown will
apply its *only* curly-quoting smartification inside the <title> tag without
wasting time looking for *any* other Markdown material.

Indeed, asterisks inside the <title> tag should be legal without triggering
an incorrect additional tag inside the tag. So for example:

<title markdown="smarty">Isn't this a *test*?

Would yield:

Isn’t this a *test*?

But perhaps there is already a better way to do this and I've forgotten what
it is...?

m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com

From t_leitner at gmx.at  Fri Aug  6 15:03:59 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Fri, 6 Aug 2010 21:03:59 +0200
Subject: [kramdown-users] Line breaks
In-Reply-To: <4C5BE04C.6080908@gmx.at>
References: <4C5BE04C.6080908@gmx.at>
Message-ID: <20100806210359.6e181243@noweto>

On 2010-08-06 12:13 +0200 Michael Franzl wrote:
> Kramdown source
> 
>      \\
>      \\
>      \\
> 
> renders the HTML
> 
>      

>      

>      \
> 
> I am not sure if this is the right way to generate line breaks, but
> at least there is one 
 missing and one backslash too much.

This is actually the correct behavior: a line break on the last line
of a paragraph is not allowed, therefore the characters `\\` are
interpreted as an escaped backslash.

-- Thomas

From t_leitner at gmx.at  Sun Aug  8 14:15:24 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Sun, 8 Aug 2010 20:15:24 +0200
Subject: [kramdown-users] is this rule necessary?
In-Reply-To: 
References: <20100806084449.705b0cbe@noweto> 
Message-ID: <20100808201524.7a8e7368@noweto>

> I also have an unreleased experimental version called 1.0.2b7:
> 
>  $VERSION = '1.0.2b7';
>  # Tue 29 Aug 2006
> 
> The same document comes out the same way. Again, I'll send you the
> Perl script if you wish.
> 
> So, that's two for two in my favor. :) I have literally thousands of
> pages that depend upon this behavior.

I'm always using Markdown.pl version 1.0.2b8 where this behaviour seems
to be changed...

> Moreover, I have never seen *any* account of the original Markdown
> syntax requiring or even mentioning the (<...>) construct. It is
> something that kramdown introduces. The documentation does describe
> it but does not point out that this is a difference from Markdown.

The original Markdown.pl implementation allows angle brackets in inline
and reference links. This is the reason why added them to kramdown,
too. However, Markdown.pl does not seem to support spaces in links when
using angle brackets...

> Plus, I would suggest that (<...>) is a possible candidate for Occam's
> razor, since what does it accomplish that simple (...) does not? That
> is, what are we trying to distinguish by addition of the angle
> brackets? Under what circumstances would [...](...) *not* be intended
> as an inline link? And in any case, if it *were* intended as not a
> link, that would be a case for escaping the first square bracket.
> That is what the original Markdown expects. So this:

As mentioned above the reason for having the angle brackets is because
Markdown.pl supports them... So I would rather not remove them.
However, as said in my previous mail, I will try to change the link
syntax to support spaces without using angle brackets.

-- Thomas

From t_leitner at gmx.at  Sun Aug  8 14:19:17 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Sun, 8 Aug 2010 20:19:17 +0200
Subject: [kramdown-users] smartypants-only switch
In-Reply-To: 
References: 
Message-ID: <20100808201917.2ce027d1@noweto>

On 2010-08-06 10:48 -0700 Matt Neuburg wrote:
> This may have already been answered and I may have dropped the ball;
> if so, I apologize.
> 
> In Frontier, and in RubyFrontier, my Web site framework has applied
> Markdown and SmartyPants in two separate stages. The reason is that
> SmartyPants is smart about making quotes and apostrophes curly
> throughout the *whole* page
> - it stays out of tags and the insides of  and , but it
> dives into all other displayed content and makes curly quotes.
> 
> Now, kramdown lacks this; it is all or nothing. If you say
> <...markdown="1">, everything inside that tag gets the ENTIRE markdown
> treatment, plus smartypants treatment. But for something like the
>  tag, this is a bit heavy-handed. I'd like to be able to say
> <title smartypants="1">, or maybe <title markdown="smarty">, so that
> kramdown will apply its *only* curly-quoting smartification inside
> the <title> tag without wasting time looking for *any* other Markdown
> material.
> 
> Indeed, asterisks inside the <title> tag should be legal without
> triggering an incorrect additional tag inside the tag. So for example:
> 
> <title markdown="smarty">Isn't this a *test*?
> 
> Would yield:
> 
> Isn’t this a *test*?
> 
> But perhaps there is already a better way to do this and I've
> forgotten what it is...?

No, there is currently no way to do this except by subclassing the
kramdown parser and removing the parsers for the smart quotes. The you
can use kramdown without in-built smartpants support and use rubypants
to do the smartypants part.

-- Thomas

From matt at tidbits.com  Sun Aug  8 15:34:24 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Sun, 8 Aug 2010 12:34:24 -0700
Subject: [kramdown-users] smartypants-only switch
In-Reply-To: <20100808201917.2ce027d1@noweto>
References:  <20100808201917.2ce027d1@noweto>
Message-ID: <33DAA665-23B4-41D7-9933-3991046BA748@tidbits.com>

On Aug 8, 2010, at 11:19 AM, Thomas Leitner wrote:

>> Isn't this a *test*?
>> 
>> Would yield:
>> 
>> Isn’t this a *test*?
>> 
>> But perhaps there is already a better way to do this and I've
>> forgotten what it is...?
> 
> No, there is currently no way to do this except by subclassing the
> kramdown parser and removing the parsers for the smart quotes. The you
> can use kramdown without in-built smartpants support and use rubypants
> to do the smartypants part.

I don't want to remove the smart quote parser at his point. I've adjusted RubyFrontier so that I don't need to do that.

(RubyFrontier is now officially supporting kramdown instead of Markdown, and I have been switching all my own sites from Markdown to kramdown. I really like kramdown's predictability and well-defined behavior, plus I use many of its added features such as inline span attributes, definition lists, and simple tables. Plus kramdown is faster because I don't have to context-switch to Perl. The RubyFrontier docs - see http://www.apeth.com/RubyFrontierDocs/ - are now built using kramdown instead of Markdown; they also use LESS, Haml, and other cool Ruby tools. RubyFrontier has made several changes to accommodate kramdown's behavior; right now the most serious issue facing me is that kramdown changes attribute order inside literal HTML tags, as I've pointed out in a different thread.)

I just want a way to access the smart quote parser only (and no other parser) in certain tags. I'd like to request this formally as a kramdown feature - in addition to markdown="0", markdown="1", markdown="span", I'm wondering if we could have markdown="smarty". This would be only for rare tags like the . Of course I can work around this in other ways, but I'm thinking that markdown="smarty" might be faster than markdown="span" because it doesn't have to look for asterisks and backticks and other span-level markup. If it's too hard to do I can live without it; but I thought you might see this as a useful idea.

m.

From t_leitner at gmx.at  Mon Aug  9 04:26:18 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Mon, 9 Aug 2010 10:26:18 +0200
Subject: [kramdown-users] more on attribute rearrangment
In-Reply-To: <8D021729-C0E9-4C79-AB78-2604C114146C@tidbits.com>
References: <8D021729-C0E9-4C79-AB78-2604C114146C@tidbits.com>
Message-ID: <20100809102618.375ec3b4@noweto>

> (2) This is wrong, I think:
> 
>   <div markdown="0">
>   <a href="test" class="ha">hey</a>
>   </div>
> 
> #=>
> 
> <div>
>   <a class="ha" href="test">hey</a>
>   </div>
> 
> The output should be the same. Attribute order should be left alone.
> This is kind of a disaster; RubyFrontier absolutely requires the href
> attribute to be first in the <a> tag in order to do certain
> postprocessing on links. It looks like kramdown must be disassembling
> tags and reassembling them them with attributes in alphabetical
> order. This seems to violate the contract in the docs, which says
> "parsed as raw HTML block and left alone". This is not "left alone"!
> m.

I have pushed the latest changes to the github repo, including the
change to preserve the order of HTML/IAL/ALD attributes. Please try out
and let me know if this works for you!

-- Thomas

From matt at tidbits.com  Mon Aug  9 13:21:13 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Mon, 09 Aug 2010 10:21:13 -0700
Subject: [kramdown-users] more on attribute rearrangment
In-Reply-To: <20100809102618.375ec3b4@noweto>
Message-ID: <C8858719.54A2D%matt@tidbits.com>

On or about 8/9/10 1:26 AM, thus spake "Thomas Leitner" <t_leitner at gmx.at>:

> I have pushed the latest changes to the github repo, including the
> change to preserve the order of HTML/IAL/ALD attributes. Please try out
> and let me know if this works for you!

Looks great! I've been doing some comparative builds using Markdown vs.
kramdown and I'm not finding any differences in attribute order. So I
conclude that kramdown really is now leaving the attribute order alone. This
is great - thanks!

m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com

From matt at tidbits.com  Mon Aug  9 21:52:56 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Mon, 09 Aug 2010 18:52:56 -0700
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
Message-ID: <C885FF08.54A74%matt@tidbits.com>

require 'kramdown'
s = <<END
<div>
</div>
<br />
<div>
</div>
END
puts Kramdown::Document.new(s).to_html

The result is:

<div>
</div>
<p><br /></p>
<div>
</div>

This feels like a possible bug. m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com

From t_leitner at gmx.at  Tue Aug 10 05:28:21 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Tue, 10 Aug 2010 11:28:21 +0200
Subject: [kramdown-users] smartypants-only switch
In-Reply-To: <33DAA665-23B4-41D7-9933-3991046BA748@tidbits.com>
References: <C88198F4.549AE%matt@tidbits.com> <20100808201917.2ce027d1@noweto>
	<33DAA665-23B4-41D7-9933-3991046BA748@tidbits.com>
Message-ID: <20100810112821.2b1b9aa9@noweto>

> I just want a way to access the smart quote parser only (and no other
> parser) in certain tags. I'd like to request this formally as a
> kramdown feature - in addition to markdown="0", markdown="1",
> markdown="span", I'm wondering if we could have markdown="smarty".
> This would be only for rare tags like the <title>. Of course I can
> work around this in other ways, but I'm thinking that
> markdown="smarty" might be faster than markdown="span" because it
> doesn't have to look for asterisks and backticks and other span-level
> markup. If it's too hard to do I can live without it; but I thought
> you might see this as a useful idea.

I don't think that putting this feature into kramdown would be the
correct way to go. The request means that you basically want to use
some parts of the parser and not another ones. If we would implement
this, why shouldn't we also implement another exception, for example,
so that only paragraphs and block quotes are parsed instead of all
block level elements?

Isn't it easier in your case to run kramdown over the whole document
and the rubypants afterwards? You said that rubypants knows that it
only has to look inside HTML tags and don't mess with attributes. So,
in the case you mentioned, running kramdown and then rubypants should
be sufficient, shouldn't it?

-- Thomas

From t_leitner at gmx.at  Tue Aug 10 05:31:07 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Tue, 10 Aug 2010 11:31:07 +0200
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <C885FF08.54A74%matt@tidbits.com>
References: <C885FF08.54A74%matt@tidbits.com>
Message-ID: <20100810113107.08ac97b0@noweto>

On 2010-08-09 18:52 -0700 Matt Neuburg wrote:
> require 'kramdown'
> s = <<END
> <div>
> </div>
> <br />
> <div>
> </div>
> END
> puts Kramdown::Document.new(s).to_html
> 
> The result is:
> 
> <div>
> </div>
> <p><br /></p>
> <div>
> </div>
> 
> This feels like a possible bug. m.

It isn't. The `<div></div>` parts before and after the `<br />` are
parsed with the block HTML parser, the `<br />` is no block HTML
element (like `<a>`), and therefore this line is parsed with the
paragraph parser.

-- Thomas

From t_leitner at gmx.at  Tue Aug 10 09:57:10 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Tue, 10 Aug 2010 15:57:10 +0200
Subject: [kramdown-users] LaTeX Footnote inside of emphasized text
In-Reply-To: <4C5911D1.4050502@gmx.at>
References: <4C41C842.1070903@gmx.at> <20100718124110.1efb9b05@noweto>
	<4C5911D1.4050502@gmx.at>
Message-ID: <20100810155710.38d3a073@noweto>

On 2010-08-04 09:08 +0200 Michael Franzl wrote:
> On 07/18/2010 12:41 PM, Thomas Leitner wrote:
> > The only other solution would be to output only those footnotes in
> > the LaTeX converter that contain a single paragraph.
> 
> Or: Join multiple paragraphs into one paragraph. It's a limitation of 
> LaTeX, and users will need to avoid it anyway. This solution would
> help only making the rendered LaTeX code error free.

The document would be error free but not what the user would expect...
So I would rather go with the "let LaTeX fail and let the user correct
the problem" approach. This way the user knows that the source document
is not correct and needs to fix it himself which has the advantage that
the user can choose how to fix it, rather than forcing a probably bad
solution onto him.

> But, regarding footnotes, there is another thing I stumbled across:
> 
>      This is text[^note] with a footnote.
> 
>      [^note]: A footnote.
> 
> Translates into
> 
>      This is text\footnote{A footnote.
> 
>      } with a footnote.
> 
> This is an empty paragraph for LaTeX, and sometimes, if it is wrapped
> to the next page, LaTeX will render a footnote line with no text
> below it, which is very ugly. So, the correct translation would be:
> 
>      This is text\footnote{A footnote.} with a footnote.

I have fixed this one. The whitespace before the closing brace is now
always stripped away.

-- Thomas

From t_leitner at gmx.at  Tue Aug 10 10:36:43 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Tue, 10 Aug 2010 16:36:43 +0200
Subject: [kramdown-users] Inline Code Highlighting
In-Reply-To: <AANLkTikuDhEu2EJbkyAe9mHj7ndG-D1QLYyN81dIhvOd@mail.gmail.com>
References: <AANLkTikuDhEu2EJbkyAe9mHj7ndG-D1QLYyN81dIhvOd@mail.gmail.com>
Message-ID: <20100810163643.334eea24@noweto>

On 2010-07-20 06:01 -0500 Josh Cheek wrote:
> When selecting a language for inline code, I can do
> 
> {: lang='ruby'}`puts "Hello world"`
> 
> {: lang='ruby'}
>     puts "Hello world"
> 
> The second one gets syntax highlighted, but the first does not. I
> can't seem to find any examples of an inline code highlight.

Inline code highlighting is currently not supported. I just fixed this
and it will be available with the next release.

-- Thomas

From matt at tidbits.com  Tue Aug 10 12:18:47 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Tue, 10 Aug 2010 09:18:47 -0700
Subject: [kramdown-users] smartypants-only switch
In-Reply-To: <20100810112821.2b1b9aa9@noweto>
Message-ID: <C886C9F7.54AA8%matt@tidbits.com>

On or about 8/10/10 2:28 AM, thus spake "Thomas Leitner" <t_leitner at gmx.at>:

> Isn't it easier in your case to run kramdown over the whole document
> and the rubypants afterwards? You said that rubypants knows that it
> only has to look inside HTML tags and don't mess with attributes. So,
> in the case you mentioned, running kramdown and then rubypants should
> be sufficient, shouldn't it?

I haven't tried rubypants. I have been using Jon Gruber's SmartyPants, which
is Perl like Markdown; I'd like to drop it. My design is based around the
fact that Markdown and SmartPants were two different things. It seems otiose
to pass through a second stage of evaluation to do something that kramdown
already knows how to do, and could easily do since it parses the whole
document looking for "markdown=" attributes. However, of course I can live
without the suggested addition. m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com

From matt at tidbits.com  Tue Aug 10 12:22:14 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Tue, 10 Aug 2010 09:22:14 -0700
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <20100810113107.08ac97b0@noweto>
Message-ID: <C886CAC6.54AAA%matt@tidbits.com>

On or about 8/10/10 2:31 AM, thus spake "Thomas Leitner" <t_leitner at gmx.at>:

> It isn't. The `<div></div>` parts before and after the `<br />` are
> parsed with the block HTML parser, the `<br />` is no block HTML
> element (like `<a>`), and therefore this line is parsed with the
> paragraph parser.

Good answer - sorry about that. m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com

From t_leitner at gmx.at  Tue Aug 10 14:07:46 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Tue, 10 Aug 2010 20:07:46 +0200
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <C886CAC6.54AAA%matt@tidbits.com>
References: <20100810113107.08ac97b0@noweto> <C886CAC6.54AAA%matt@tidbits.com>
Message-ID: <20100810200746.0ad44039@noweto>

On 2010-08-10 09:22 -0700 Matt Neuburg wrote:
> On or about 8/10/10 2:31 AM, thus spake "Thomas Leitner"
> <t_leitner at gmx.at>:
> 
> > It isn't. The `<div></div>` parts before and after the `<br />` are
> > parsed with the block HTML parser, the `<br />` is no block HTML
> > element (like `<a>`), and therefore this line is parsed with the
> > paragraph parser.
> 
> Good answer - sorry about that. m.

No problem, nothing to be sorry about.

-- Thomas

From svicalifornia at gmail.com  Wed Aug 11 09:26:51 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Wed, 11 Aug 2010 20:26:51 +0700
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <20100810113107.08ac97b0@noweto>
References: <C885FF08.54A74%matt@tidbits.com> <20100810113107.08ac97b0@noweto>
Message-ID: <20100811202651338294.b8fec964@gmail.com>

I thought that there was a requirement for a blank line before all paragraphs?  This is clearly a contiguous block of HTML, and kramdown should leave HTML alone, as Markdown does.

Matt, what does Markdown do when parsing this input?

Shawn

On Tue, 10 Aug 2010 11:31:07 +0200, Thomas Leitner wrote:
> On 2010-08-09 18:52 -0700 Matt Neuburg wrote:
>> require 'kramdown'
>> s = <<END
>> <div>
>> </div>
>> <br />
>> <div>
>> </div>
>> END
>> puts Kramdown::Document.new(s).to_html
>> 
>> The result is:
>> 
>> <div>
>> </div>
>> <p><br /></p>
>> <div>
>> </div>
>> 
>> This feels like a possible bug. m.
> 
> It isn't. The `<div></div>` parts before and after the `<br />` are
> parsed with the block HTML parser, the `<br />` is no block HTML
> element (like `<a>`), and therefore this line is parsed with the
> paragraph parser.
> 
> -- Thomas
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From matt at tidbits.com  Wed Aug 11 11:37:03 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Wed, 11 Aug 2010 08:37:03 -0700
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <20100811202651338294.b8fec964@gmail.com>
Message-ID: <C88811AF.54AE1%matt@tidbits.com>

Actually, now that I am a firm and enthusiastic convert to kramdown, I
believe (thanks to Thomas's hint) that I understand kramdown's behavior
here, and can defend it.

So, here's how I understand what kramdown is doing in this example. Yes,
kramdown stays out of a <div>. So if I wrap the *entire* expression in a
<div>, it is untouched:

<div>
<div>
</div>
<br />
<div>
</div>
</div>

[Output is identical to input.]

But in my actual example, kramdown divides the input into three regions: the
div pairs, and the stuff between them. The "stuff between them" is not
inside a div, so it is not protected. Thus it is a target for block-level
processing.

<div>
</div>
<br /> <-- this is unprotected, it is a "kramdown target"
<div>
</div>

The "contiguous" issue that you raise also misled me at first, but now that
I understand it, I actually like what kramdown is doing. The fact is that
"contiguous" is irrelevant! kramdown isolates the line "<br />" as a target
for processing, not on the basis of what it is touching or not touching, but
on the basis of the overall *structure* - it is a region unprotected by a
surrounding <div>. This is actually very good, because it means that, unlike
Markdown, I don't have to add extra spacing as I embed the document into its
surrounding templated HTML.

Here is another way to think of it. What should kramdown do here?

require 'kramdown'
s = "test"
puts Kramdown::Document.new(s).to_html

I think you will agree that it should surround "test" with <p> tags. And
that is just what it does. Yet there is no space before or after "test";
there isn't even a newline. Well, the "<br />" line in my example is exactly
equivalent to that. It is a single-line processing "document" on its own.

If I am serious about protecting the "<br />", I have ways to do it. I can
surround the entire document with a <div> and just isolate the sections to
be processed with an inner <div markdown="1">. Or I can write:

<div>
</div>
{::nomarkdown}
<br />
{:/}
<div>
</div>

If you're going to argue about an edge case, here is an edge case to argue
about:

<div>
</div>
{::nomarkdown}<br />{:/}
<div>
</div>

I think what kramdown does in that case is a bit more surprising (it *does*
surround the <br /> with <p> tags), but I can live with it. It is an edge
case! Edge cases are always difficult. I can envision kramdown applying what
amounts to two conflicting rules here, and I can easily see how the <p> rule
might win. The same applies here:

require 'kramdown'
s = "{::nomarkdown}<br />{:/}"
puts Kramdown::Document.new(s).to_html

Same edge case, same result. But I don't think it is worth trying to "fix",
since the workaround is clear; there is syntax to protect the <br /> if
desired.

m.

PS. Original Markdown's behavior in this regard is irrelevant, because in
fact this is one of the many areas in which Markdown was somewhat
indeterminate. Markdown did *not* always leave HTML alone. Markdown
sometimes used to wrap <div></div> in <p> tags, wrongly (giving
<p><div></div></p>), and I had to add extra post-processing code to detect
and fix these. But kramdown aims to be consistent and predictable (besides
adding power that original Markdown lacks). The chief difference is in
overall approach: Markdown was just doing find-and-replace, but kramdown is
actually ***parsing the HTML structure of the document***.

This fact originally gave me some trouble in switching to kramdown, you may
recall. I was able to hand the document to Markdown at a time when it did
not in fact consist of just text and legal HTML (it also contained ERB
expressions, for example - you may recall that I complained that kramdown
was altering my ERB so that when the ERB parser actually came along, the ERB
itself was wrong). But I have compensated by changing the order of
processing in RubyFrontier so that this is no longer an issue.

My chief remaining issue with kramdown was that it was not merely treating
its HTML parsing as a pass-through: it was disassembling the HTML and
reassembling it, and it was reassembling it differently. Thus for example,
<div manny="moe" hey="ho"> was turning into <div hey="ho" manny="moe">. And
since RubyFrontier was using these sorts of details to pass secret messages
to itself across the stages of processing, kramdown was disrupting those
messages. But Thomas made the order of attributes stable, and so this
problem went away. kramdown *still* does not merely treat its HTML parsing
as a pass-through (for example, it will alter <div hey='ho'> so that the
quotes are double instead of single), and I still think this is wrong, but I
can live with it.

On or about 8/11/10 6:26 AM, thus spake "Shawn Van Ittersum"
<svicalifornia at gmail.com>:

> I thought that there was a requirement for a blank line before all paragraphs?
> This is clearly a contiguous block of HTML, and kramdown should leave HTML
> alone, as Markdown does.
> 
> Matt, what does Markdown do when parsing this input?
> 
> Shawn
> 
> On Tue, 10 Aug 2010 11:31:07 +0200, Thomas Leitner wrote:
>> On 2010-08-09 18:52 -0700 Matt Neuburg wrote:
>>> require 'kramdown'
>>> s = <<END
>>> <div>
>>> </div>
>>> <br />
>>> <div>
>>> </div>
>>> END
>>> puts Kramdown::Document.new(s).to_html
>>> 
>>> The result is:
>>> 
>>> <div>
>>> </div>
>>> <p><br /></p>
>>> <div>
>>> </div>

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com

From svicalifornia at gmail.com  Wed Aug 11 12:15:02 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Wed, 11 Aug 2010 23:15:02 +0700
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <C88811AF.54AE1%matt@tidbits.com>
References: <C88811AF.54AE1%matt@tidbits.com>
Message-ID: <20100811231502075477.202aedcf@gmail.com>

I disagree that contiguous is irrelevant.  I also see what kramdown is doing, and it's something that developers can figure out.  However, when you or I use kramdown in some other product, its end users are non-developers, who are not going to figure this out.  Non-developers who are going to copy in a block of HTML from somewhere, expecting it to be untouched as it would be in Markdown, and being very frustrated to see kramdown messing with it.

Contiguous blocks of HTML should be left alone.  kramdown results should not diverge from Markdown results except to add clear benefit, and I fail to see the benefit if this particular divergence.

With regard to your example input of "test": the rule we discussed before was that the first line and any line that followed a blank line would be treated as the start of a new block.  Following this rule, it is fitting to wrap "test" in p tags.  It is not appropriate to apply p tags in the middle of a block of HTML, regardless of the opening and closing of DOM elements within that HTML block.  kramdown should respect the rules of Markdown first.

I still don't understand why kramdown has an HTML parser... HTML tags should simply pass through untouched.

Shawn

On Wed, 11 Aug 2010 08:37:03 -0700, Matt Neuburg wrote:
> Actually, now that I am a firm and enthusiastic convert to kramdown, I
> believe (thanks to Thomas's hint) that I understand kramdown's behavior
> here, and can defend it.
> 
> So, here's how I understand what kramdown is doing in this example. Yes,
> kramdown stays out of a <div>. So if I wrap the *entire* expression in a
> <div>, it is untouched:
> 
> <div>
> <div>
> </div>
> <br />
> <div>
> </div>
> </div>
> 
> [Output is identical to input.]
> 
> But in my actual example, kramdown divides the input into three regions: the
> div pairs, and the stuff between them. The "stuff between them" is not
> inside a div, so it is not protected. Thus it is a target for block-level
> processing.
> 
> <div>
> </div>
> <br /> <-- this is unprotected, it is a "kramdown target"
> <div>
> </div>
> 
> The "contiguous" issue that you raise also misled me at first, but now that
> I understand it, I actually like what kramdown is doing. The fact is that
> "contiguous" is irrelevant! kramdown isolates the line "<br />" as a target
> for processing, not on the basis of what it is touching or not touching, but
> on the basis of the overall *structure* - it is a region unprotected by a
> surrounding <div>. This is actually very good, because it means that, unlike
> Markdown, I don't have to add extra spacing as I embed the document into its
> surrounding templated HTML.
> 
> Here is another way to think of it. What should kramdown do here?
> 
> require 'kramdown'
> s = "test"
> puts Kramdown::Document.new(s).to_html
> 
> I think you will agree that it should surround "test" with <p> tags. And
> that is just what it does. Yet there is no space before or after "test";
> there isn't even a newline. Well, the "<br />" line in my example is exactly
> equivalent to that. It is a single-line processing "document" on its own.
> 
> If I am serious about protecting the "<br />", I have ways to do it. I can
> surround the entire document with a <div> and just isolate the sections to
> be processed with an inner <div markdown="1">. Or I can write:
> 
> <div>
> </div>
> {::nomarkdown}
> <br />
> {:/}
> <div>
> </div>
> 
> If you're going to argue about an edge case, here is an edge case to argue
> about:
> 
> <div>
> </div>
> {::nomarkdown}<br />{:/}
> <div>
> </div>
> 
> I think what kramdown does in that case is a bit more surprising (it *does*
> surround the <br /> with <p> tags), but I can live with it. It is an edge
> case! Edge cases are always difficult. I can envision kramdown applying what
> amounts to two conflicting rules here, and I can easily see how the <p> rule
> might win. The same applies here:
> 
> require 'kramdown'
> s = "{::nomarkdown}<br />{:/}"
> puts Kramdown::Document.new(s).to_html
> 
> Same edge case, same result. But I don't think it is worth trying to "fix",
> since the workaround is clear; there is syntax to protect the <br /> if
> desired.
> 
> m.
> 
> PS. Original Markdown's behavior in this regard is irrelevant, because in
> fact this is one of the many areas in which Markdown was somewhat
> indeterminate. Markdown did *not* always leave HTML alone. Markdown
> sometimes used to wrap <div></div> in <p> tags, wrongly (giving
> <p><div></div></p>), and I had to add extra post-processing code to detect
> and fix these. But kramdown aims to be consistent and predictable (besides
> adding power that original Markdown lacks). The chief difference is in
> overall approach: Markdown was just doing find-and-replace, but kramdown is
> actually ***parsing the HTML structure of the document***.
> 
> This fact originally gave me some trouble in switching to kramdown, you may
> recall. I was able to hand the document to Markdown at a time when it did
> not in fact consist of just text and legal HTML (it also contained ERB
> expressions, for example - you may recall that I complained that kramdown
> was altering my ERB so that when the ERB parser actually came along, the ERB
> itself was wrong). But I have compensated by changing the order of
> processing in RubyFrontier so that this is no longer an issue.
> 
> My chief remaining issue with kramdown was that it was not merely treating
> its HTML parsing as a pass-through: it was disassembling the HTML and
> reassembling it, and it was reassembling it differently. Thus for example,
> <div manny="moe" hey="ho"> was turning into <div hey="ho" manny="moe">. And
> since RubyFrontier was using these sorts of details to pass secret messages
> to itself across the stages of processing, kramdown was disrupting those
> messages. But Thomas made the order of attributes stable, and so this
> problem went away. kramdown *still* does not merely treat its HTML parsing
> as a pass-through (for example, it will alter <div hey='ho'> so that the
> quotes are double instead of single), and I still think this is wrong, but I
> can live with it.
> 
> On or about 8/11/10 6:26 AM, thus spake "Shawn Van Ittersum"
> <svicalifornia at gmail.com>:
> 
>> I thought that there was a requirement for a blank line before all 
>> paragraphs?
>> This is clearly a contiguous block of HTML, and kramdown should leave HTML
>> alone, as Markdown does.
>> 
>> Matt, what does Markdown do when parsing this input?
>> 
>> Shawn
>> 
>> On Tue, 10 Aug 2010 11:31:07 +0200, Thomas Leitner wrote:
>>> On 2010-08-09 18:52 -0700 Matt Neuburg wrote:
>>>> require 'kramdown'
>>>> s = <<END
>>>> <div>
>>>> </div>
>>>> <br />
>>>> <div>
>>>> </div>
>>>> END
>>>> puts Kramdown::Document.new(s).to_html
>>>> 
>>>> The result is:
>>>> 
>>>> <div>
>>>> </div>
>>>> <p><br /></p>
>>>> <div>
>>>> </div>
> 
> -- 
> matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
> pantes anthropoi tou eidenai oregontai phusei
> Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
> AppleScript: the Definitive Guide, 2nd edition
> http://www.tidbits.com/matt/default.html#applescriptthings
> Take Control of Exploring & Customizing Snow Leopard
> http://tinyurl.com/kufyy8
> RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
> TidBITS, Mac news and reviews since 1990, http://www.tidbits.com
> 
> 
> 
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From matt at tidbits.com  Wed Aug 11 14:39:04 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Wed, 11 Aug 2010 11:39:04 -0700
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <20100811231502075477.202aedcf@gmail.com>
Message-ID: <C8883C58.54AED%matt@tidbits.com>

On or about 8/11/10 9:15 AM, thus spake "Shawn Van Ittersum"
<svicalifornia at gmail.com>:

> I still don't understand why kramdown has an HTML parser... HTML tags should
> simply pass through untouched.

I am presuming that kramdown has an HTML parser because it needs to be able
to cycle down through the HTML looking for a <div> with a "markdown"
attribute. I agree with you, however, in that I'd prefer if this parser were
read-only, so that the HTML found would be then identical in the output to
what it was in the input. The source of this issue, I think, is that
kramdown is using REXML. As I've hinted before, I think that if kramdown
were willing depend on nokogiri it could just jump straight to those divs
without parsing its way down to them. And nokogiri does not rewrite
attribute order.

m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com

From svicalifornia at gmail.com  Wed Aug 11 14:42:19 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Thu, 12 Aug 2010 01:42:19 +0700
Subject: [kramdown-users] Table Column Alignment
Message-ID: <20100812014219674956.fa4d01e4@gmail.com>

Hi Thomas,

I see that kramdown is accomplishing center and right alignment in table cells via colgroup tags.  It seems colgroup align attributes are being trumped in modern browsers (such as Safari) by CSS text-align properties on containing divs.  Can you change kramdown to put 'align' or 'style' attributes directly on the table cells?

Thanks,
Shawn

From josh.cheek at gmail.com  Wed Aug 11 17:29:20 2010
From: josh.cheek at gmail.com (Josh Cheek)
Date: Wed, 11 Aug 2010 16:29:20 -0500
Subject: [kramdown-users] Inline Code Highlighting
Message-ID: <AANLkTikaSYXF3N4c2JfZPvrRmBRKXoaitV0akLH-FB+n@mail.gmail.com>

> Inline code highlighting is currently not supported. I just fixed this
> and it will be available with the next release.

Thank you very much!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/kramdown-users/attachments/20100811/1268c9ce/attachment.html>

From jim at saturnflyer.com  Wed Aug 11 20:00:05 2010
From: jim at saturnflyer.com (Jim Gay)
Date: Wed, 11 Aug 2010 20:00:05 -0400
Subject: [kramdown-users] Inline Code Highlighting
In-Reply-To: <AANLkTikaSYXF3N4c2JfZPvrRmBRKXoaitV0akLH-FB+n@mail.gmail.com>
References: <AANLkTikaSYXF3N4c2JfZPvrRmBRKXoaitV0akLH-FB+n@mail.gmail.com>
Message-ID: <AANLkTikD-u84EvXr9GUJ7gbwY3DVX3hScJARSaCXrhzR@mail.gmail.com>

On Wed, Aug 11, 2010 at 5:29 PM, Josh Cheek <josh.cheek at gmail.com> wrote:

> > Inline code highlighting is currently not supported. I just fixed this
> > and it will be available with the next release.
>
> Thank you very much!

I didn't realize code highlighting was built-in. Is there any documentation
about this feature?

-- 
Jim Gay
Saturn Flyer LLC
http://www.saturnflyer.com
571-403-0338
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/kramdown-users/attachments/20100811/351e0792/attachment.html>

From t_leitner at gmx.at  Thu Aug 12 05:46:07 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 12 Aug 2010 11:46:07 +0200
Subject: [kramdown-users] Inline Code Highlighting
In-Reply-To: <AANLkTikD-u84EvXr9GUJ7gbwY3DVX3hScJARSaCXrhzR@mail.gmail.com>
References: <AANLkTikaSYXF3N4c2JfZPvrRmBRKXoaitV0akLH-FB+n@mail.gmail.com>
	<AANLkTikD-u84EvXr9GUJ7gbwY3DVX3hScJARSaCXrhzR@mail.gmail.com>
Message-ID: <20100812114607.21f50f3d@noweto>

On 2010-08-11 20:00 -0400 Jim Gay wrote:
> On Wed, Aug 11, 2010 at 5:29 PM, Josh Cheek <josh.cheek at gmail.com>
> wrote:
> 
> > > Inline code highlighting is currently not supported. I just fixed
> > > this and it will be available with the next release.
> >
> > Thank you very much!
> 
> 
> I didn't realize code highlighting was built-in. Is there any
> documentation about this feature?

Yes, it is documented in on the documentation pages of each converter,
for example, for the HTML converter you will find the information at
http://kramdown.rubyforge.org/converter/html.html#syntax-highlighting

-- Thomas

From t_leitner at gmx.at  Thu Aug 12 05:52:39 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 12 Aug 2010 11:52:39 +0200
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <20100811202651338294.b8fec964@gmail.com>
References: <C885FF08.54A74%matt@tidbits.com> <20100810113107.08ac97b0@noweto>
	<20100811202651338294.b8fec964@gmail.com>
Message-ID: <20100812115239.3c5c1f57@noweto>

On 2010-08-11 20:26 +0700 Shawn Van Ittersum wrote:
> I thought that there was a requirement for a blank line before all
> paragraphs?  This is clearly a contiguous block of HTML, and kramdown
> should leave HTML alone, as Markdown does.

No, there is no such requirement. The only requirement is that there is
at least one blank line *between* two consecutive paragraphs.

> Matt, what does Markdown do when parsing this input?

It does the same, it wraps the `<br />` tag in a paragraph.

-- Thomas

From t_leitner at gmx.at  Thu Aug 12 05:57:42 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 12 Aug 2010 11:57:42 +0200
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <C88811AF.54AE1%matt@tidbits.com>
References: <20100811202651338294.b8fec964@gmail.com>
	<C88811AF.54AE1%matt@tidbits.com>
Message-ID: <20100812115742.45b5f1ef@noweto>

On 2010-08-11 08:37 -0700 Matt Neuburg wrote:
> Actually, now that I am a firm and enthusiastic convert to kramdown, I
> believe (thanks to Thomas's hint) that I understand kramdown's
> behavior here, and can defend it.
> 
> <SNIP />

Your explanation is completely right, kramdown does it exactly in this
way.

> If I am serious about protecting the "<br />", I have ways to do it.
> I can surround the entire document with a <div> and just isolate the
> sections to be processed with an inner <div markdown="1">. Or I can
> write:
> 
> <div>
> </div>
> {::nomarkdown}
> <br />
> {:/}
> <div>
> </div>
> 
> If you're going to argue about an edge case, here is an edge case to
> argue about:
> 
> <div>
> </div>
> {::nomarkdown}<br />{:/}
> <div>
> </div>

This can also be explained: in the first case the extension tag is the
only thing on the line, therefore it is parsed by the block level
parser as the start of an extension block. In the second case there are
non-whitespace characters after the extension start tag, therefore it
is *not* processed by the block level extension parser but is converted
to a paragraph. Then the span level parser parses the extension.

-- Thomas

From t_leitner at gmx.at  Thu Aug 12 06:14:04 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 12 Aug 2010 12:14:04 +0200
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <20100811231502075477.202aedcf@gmail.com>
References: <C88811AF.54AE1%matt@tidbits.com>
	<20100811231502075477.202aedcf@gmail.com>
Message-ID: <20100812121404.77553e2b@noweto>

On 2010-08-11 23:15 +0700 Shawn Van Ittersum wrote:
> I disagree that contiguous is irrelevant.  I also see what kramdown
> is doing, and it's something that developers can figure out.
> However, when you or I use kramdown in some other product, its end
> users are non-developers, who are not going to figure this out.
> Non-developers who are going to copy in a block of HTML from
> somewhere, expecting it to be untouched as it would be in Markdown,
> and being very frustrated to see kramdown messing with it.
>
> Contiguous blocks of HTML should be left alone.  kramdown results
> should not diverge from Markdown results except to add clear benefit,
> and I fail to see the benefit if this particular divergence.

The Markdown implementation by Gruber does actually the same in this
case, as does PHP Markdown.

> With regard to your example input of "test": the rule we discussed
> before was that the first line and any line that followed a blank
> line would be treated as the start of a new block.  Following this
> rule, it is fitting to wrap "test" in p tags.  It is not appropriate
> to apply p tags in the middle of a block of HTML, regardless of the
> opening and closing of DOM elements within that HTML block.  kramdown
> should respect the rules of Markdown first.

No, this is not correct. For example, you can have a blank line between
two parts of one code block, like this:

    This is the first part of the code block.

    This is still the same code block.

Markdown is more or less a line oriented markup language: each line is
processed on its own and assigned a certain meaning. If it starts with
a hash, it becomes a header; if it starts with four spaces or one tab,
it is a code block. This is also how kramdown does the parsing: one
line after another.

There are some syntax elements that span multiple lines, like setext
headers, but they also don't stop in the middle of the line but need
whole lines.

The only block level parser that is different regarding this is... the
HTML parser but there are good reasons for being different!

> I still don't understand why kramdown has an HTML parser... HTML tags
> should simply pass through untouched.

HTML parsing is hard, although it seems easy. This is the reason why
there have been several completely different HTML parsers in kramdown.
The first incarnation did something very simple and wasn't very
powerful (ie. no "markdown" attribute and no markdown-in-html
parsing, many problematic edge cases).

The current parser is needed to ensure that everything works as good as
possible:

* There are clearly defined rules how HTML is parsed in kramdown. If
  you follow the rules, you know what you will get as output. This is
  documented at http://kramdown.rubyforge.org/syntax.html#html-blocks
  If you find a difference to what kramdown actually does, please
  report!

* kramdown-in-HTML is supported as well as the "markdown" attribute.

* Edge cases are reduced to a minimum.

* Adding HTML to a kramdown document is easier for end users because the
  kramdown HTML parser is more intelligent and tries to do the correct
  thing.

* I did not develop the rules for the HTML parser alone but was helped
  and often corrected by people on this ML.

-- Thomas

From t_leitner at gmx.at  Thu Aug 12 06:20:04 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 12 Aug 2010 12:20:04 +0200
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <C8883C58.54AED%matt@tidbits.com>
References: <20100811231502075477.202aedcf@gmail.com>
	<C8883C58.54AED%matt@tidbits.com>
Message-ID: <20100812122004.3ca55ba1@noweto>

On 2010-08-11 11:39 -0700 Matt Neuburg wrote:
> On or about 8/11/10 9:15 AM, thus spake "Shawn Van Ittersum"
> <svicalifornia at gmail.com>:
> 
> > I still don't understand why kramdown has an HTML parser... HTML
> > tags should simply pass through untouched.
> 
> I am presuming that kramdown has an HTML parser because it needs to
> be able to cycle down through the HTML looking for a <div> with a
> "markdown" attribute. I agree with you, however, in that I'd prefer
> if this parser were read-only, so that the HTML found would be then
> identical in the output to what it was in the input. The source of
> this issue, I think, is that kramdown is using REXML. As I've hinted
> before, I think that if kramdown were willing depend on nokogiri it
> could just jump straight to those divs without parsing its way down
> to them. And nokogiri does not rewrite attribute order.

Yes, for anything than a naive implementation you will need a good HTML
parser.

However, kramdown does not use any library for parsing HTML, neither
REXML nor nokogiri! I once tried to use REXML but was very slow and I
needed to hack it in order to get certain functionality. There were
some many complications that I decided to write the HTML parser myself.
This lead to increased speed and better support for HTML.

Nokogiri does not have the speed problem as REXML but it is also not
suitable for kramdown. For example, consider the following document:

    This is a para

    <div markdown="1">
        This is a code block with a </div>
    </div>

    This is a para

How should nokogiri be able to parse the div without the knowledge of
the kramdown parser?

This and similar cases lead to the creation of the current parser.

-- Thomas

From svicalifornia at gmail.com  Thu Aug 12 08:01:00 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Thu, 12 Aug 2010 19:01:00 +0700
Subject: [kramdown-users] possible bug: kramdown wrapping <br> in <p>
In-Reply-To: <20100812121404.77553e2b@noweto>
References: <C88811AF.54AE1%matt@tidbits.com>
	<20100811231502075477.202aedcf@gmail.com>
	<20100812121404.77553e2b@noweto>
Message-ID: <20100812190100286830.72e3dc54@gmail.com>

Hi Thomas,

Thanks for the thorough explanation of kramdown's behavior.  I did not realize that Markdown's output was the same.  So even though is it different than what I expected, I must say again, great work on kramdown!

Shawn

On Thu, 12 Aug 2010 12:14:04 +0200, Thomas Leitner wrote:
> On 2010-08-11 23:15 +0700 Shawn Van Ittersum wrote:
>> I disagree that contiguous is irrelevant.  I also see what kramdown
>> is doing, and it's something that developers can figure out.
>> However, when you or I use kramdown in some other product, its end
>> users are non-developers, who are not going to figure this out.
>> Non-developers who are going to copy in a block of HTML from
>> somewhere, expecting it to be untouched as it would be in Markdown,
>> and being very frustrated to see kramdown messing with it.
>> 
>> Contiguous blocks of HTML should be left alone.  kramdown results
>> should not diverge from Markdown results except to add clear benefit,
>> and I fail to see the benefit if this particular divergence.
> 
> The Markdown implementation by Gruber does actually the same in this
> case, as does PHP Markdown.
>  
>> With regard to your example input of "test": the rule we discussed
>> before was that the first line and any line that followed a blank
>> line would be treated as the start of a new block.  Following this
>> rule, it is fitting to wrap "test" in p tags.  It is not appropriate
>> to apply p tags in the middle of a block of HTML, regardless of the
>> opening and closing of DOM elements within that HTML block.  kramdown
>> should respect the rules of Markdown first.
> 
> No, this is not correct. For example, you can have a blank line between
> two parts of one code block, like this:
> 
>     This is the first part of the code block.
> 
>     This is still the same code block.
> 
> Markdown is more or less a line oriented markup language: each line is
> processed on its own and assigned a certain meaning. If it starts with
> a hash, it becomes a header; if it starts with four spaces or one tab,
> it is a code block. This is also how kramdown does the parsing: one
> line after another.
> 
> There are some syntax elements that span multiple lines, like setext
> headers, but they also don't stop in the middle of the line but need
> whole lines.
> 
> The only block level parser that is different regarding this is... the
> HTML parser but there are good reasons for being different!
> 
>> I still don't understand why kramdown has an HTML parser... HTML tags
>> should simply pass through untouched.
> 
> HTML parsing is hard, although it seems easy. This is the reason why
> there have been several completely different HTML parsers in kramdown.
> The first incarnation did something very simple and wasn't very
> powerful (ie. no "markdown" attribute and no markdown-in-html
> parsing, many problematic edge cases).
> 
> The current parser is needed to ensure that everything works as good as
> possible:
> 
> * There are clearly defined rules how HTML is parsed in kramdown. If
>   you follow the rules, you know what you will get as output. This is
>   documented at http://kramdown.rubyforge.org/syntax.html#html-blocks
>   If you find a difference to what kramdown actually does, please
>   report!
> 
> * kramdown-in-HTML is supported as well as the "markdown" attribute.
> 
> * Edge cases are reduced to a minimum.
> 
> * Adding HTML to a kramdown document is easier for end users because the
>   kramdown HTML parser is more intelligent and tries to do the correct
>   thing.
> 
> * I did not develop the rules for the HTML parser alone but was helped
>   and often corrected by people on this ML.
> 
> -- Thomas
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From svicalifornia at gmail.com  Mon Aug 16 13:42:25 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Tue, 17 Aug 2010 00:42:25 +0700
Subject: [kramdown-users] Bug in list parsing
Message-ID: <20100817004225975069.566dae5e@gmail.com>

There's a problem with kramdown's recognition of list items with "wrapped" or "lazy" indentation, whereby kramdown's output does not match Markdown's.

Gruber's Markdown syntax guide:

To make lists look nice, you can wrap items with hanging indents:

*   Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
    Aliquam hendrerit mi posuere lectus. Vestibulum enim wisi,
    viverra nec, fringilla in, laoreet vitae, risus.
*   Donec sit amet nisl. Aliquam semper ipsum sit amet velit.
    Suspendisse id sem consectetuer libero luctus adipiscing.

But if you want to be lazy, you don?t have to:

*   Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
Aliquam hendrerit mi posuere lectus. Vestibulum enim wisi,
viverra nec, fringilla in, laoreet vitae, risus.
*   Donec sit amet nisl. Aliquam semper ipsum sit amet velit.
Suspendisse id sem consectetuer libero luctus adipiscing.

Taking the last block above as input, Markdown outputs a nice unordered list:

<ul>
<li>Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
Aliquam hendrerit mi posuere lectus. Vestibulum enim wisi,
viverra nec, fringilla in, laoreet vitae, risus.</li>
<li>Donec sit amet nisl. Aliquam semper ipsum sit amet velit.
Suspendisse id sem consectetuer libero luctus adipiscing.</li>
</ul>

However, kramdown doesn't recognize anything beyond the first line as an unordered list, and instead wraps the remainder in paragraph tags.  It outputs:

<ul>
  <li>Lorem ipsum dolor sit amet, consectetuer adipiscing elit.</li>
</ul>
<p>Aliquam hendrerit mi posuere lectus. Vestibulum enim wisi,
viverra nec, fringilla in, laoreet vitae, risus.
*   Donec sit amet nisl. Aliquam semper ipsum sit amet velit.
Suspendisse id sem consectetuer libero luctus adipiscing.</p>

I'm working with Markdown-syntax text in email messages, which get wrapped by mail transports beyond user control.  As Gruber says in the Philosophy of Markdown, "the single biggest source of inspiration for Markdown?s syntax is the format of plain text email."  So it would be great if kramdown would handle this "wrapped" indentation of lists as gracefully as Markdown does.

Please let me know if this is something you can fix, and when I can test it again. :)

Thanks,
Shawn

From t_leitner at gmx.at  Mon Aug 16 17:23:44 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Mon, 16 Aug 2010 23:23:44 +0200
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <20100817004225975069.566dae5e@gmail.com>
References: <20100817004225975069.566dae5e@gmail.com>
Message-ID: <20100816232344.541859c1@noweto>

On 2010-08-17 00:42 +0700 Shawn Van Ittersum wrote:
> There's a problem with kramdown's recognition of list items with
> "wrapped" or "lazy" indentation, whereby kramdown's output does not
> match Markdown's.
> 
> <SNIP />
> 
> I'm working with Markdown-syntax text in email messages, which get
> wrapped by mail transports beyond user control.  As Gruber says in
> the Philosophy of Markdown, "the single biggest source of inspiration
> for Markdown?s syntax is the format of plain text email."  So it
> would be great if kramdown would handle this "wrapped" indentation of
> lists as gracefully as Markdown does.

See the syntax documentation for lists regarding this - it is one place
where kramdown syntax is different from Markdown syntax.

-- Thomas

From svicalifornia at gmail.com  Mon Aug 16 18:04:15 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Tue, 17 Aug 2010 05:04:15 +0700
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <20100816232344.541859c1@noweto>
References: <20100817004225975069.566dae5e@gmail.com>
	<20100816232344.541859c1@noweto>
Message-ID: <20100817050415296913.04761f6f@gmail.com>

Hi Thomas,

Given that this difference causes problems for email-wrapped text, and "the single biggest source of inspiration for Markdown's syntax is the format of plain text email," would you please reconsider the kramdown syntax?  What are the technical issues involved?

Shawn

On Mon, 16 Aug 2010 23:23:44 +0200, Thomas Leitner wrote:
> On 2010-08-17 00:42 +0700 Shawn Van Ittersum wrote:
>> There's a problem with kramdown's recognition of list items with
>> "wrapped" or "lazy" indentation, whereby kramdown's output does not
>> match Markdown's.
>> 
>> <SNIP />
>> 
>> I'm working with Markdown-syntax text in email messages, which get
>> wrapped by mail transports beyond user control.  As Gruber says in
>> the Philosophy of Markdown, "the single biggest source of inspiration
>> for Markdown?s syntax is the format of plain text email."  So it
>> would be great if kramdown would handle this "wrapped" indentation of
>> lists as gracefully as Markdown does.
> 
> See the syntax documentation for lists regarding this - it is one place
> where kramdown syntax is different from Markdown syntax.
> 
> -- Thomas
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From t_leitner at gmx.at  Tue Aug 17 10:14:04 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Tue, 17 Aug 2010 16:14:04 +0200
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <20100817050415296913.04761f6f@gmail.com>
References: <20100817004225975069.566dae5e@gmail.com>
	<20100816232344.541859c1@noweto>
	<20100817050415296913.04761f6f@gmail.com>
Message-ID: <20100817161404.0e8b3247@noweto>

On 2010-08-17 05:04 +0700 Shawn Van Ittersum wrote:
> Hi Thomas,
> 
> Given that this difference causes problems for email-wrapped text,
> and "the single biggest source of inspiration for Markdown's syntax
> is the format of plain text email," would you please reconsider the
> kramdown syntax?  What are the technical issues involved?

The inspiration may be email but not how bad mail programs garble
mails...

Markdown supports paragraphs with hard-wrapped lines, ie. one or more
continuous lines are one paragraph even if they contain single line
breaks. If you write your paragraphs using single, long lines, then
mail programs will probably garble the text and the result is harder to
parse and to understand semantically. This is not only a problem for
computers but also for humans who have to decipher a garbled email text
(although it is vastly easier for humans because they understand the
text).

Lazy paragraphs are not only supported within lists in Markdown but
also within blockquotes and probably other block level elements. What
do you say to the following:

    > ... but this is wrong!
    No, it is not!

Is this one blockquote or a blockquote followed by a paragraph? Since
Markdown supports lazy blockquotes, it considers the above text to be
one blockquote. However, I think it is more natural to assume that this
is a blockquote followed by a paragraph which is how kramdown
interprets it.

So if we change the behaviour for lists, we should probably also change
the behaviour for blockquotes, footnote definitions and all other
similar block level elements to remain consistent.

-- Thomas

From matt at tidbits.com  Tue Aug 17 12:52:51 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Tue, 17 Aug 2010 09:52:51 -0700
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <20100817161404.0e8b3247@noweto>
Message-ID: <C8900C73.54C0B%matt@tidbits.com>

On or about 8/17/10 7:14 AM, thus spake "Thomas Leitner" <t_leitner at gmx.at>:

> So if we change the behaviour for lists, we should probably also change
> the behaviour for blockquotes, footnote definitions and all other
> similar block level elements to remain consistent.

Well, I hope you don't change it. I find kramdown's behavior rational and
deterministic (i.e. predictable and consistent), as opposed to Markdown
where you never quite knew what would happen with a list, how your indenting
would be interpreted, etc. And I find the documentation's explanation of how
kramdown will behave to be clear and correct.

In the one particular case in question, if it is known that the input
represents a list, it is trivial to preprocess it before kramdown sees it.
It is up to the user to prepare the input for kramdown; it is not up to
kramdown to adapt itself to incorrect input.

m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com

From svicalifornia at gmail.com  Tue Aug 17 13:43:24 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Wed, 18 Aug 2010 00:43:24 +0700
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <C8900C73.54C0B%matt@tidbits.com>
References: <C8900C73.54C0B%matt@tidbits.com>
Message-ID: <20100818004324877115.7b83aa9a@gmail.com>

Hi Matt,

The contents of the input are not known in advance; it is only assumed to be valid Markdown.

I continue to advocate the use of white-space lines to separate blocks.  Contiguous lines of text, regardless of indentation, belong to the same block.  Thus:

    > ... but this is wrong!
    No, it is not!

is a single blockquote, while

    > ... but this is wrong!

    No, it is not!

results in one blockquote and one paragraph.

The primary exception I'd make to the above rule is for list items:

    1. One
    2. Two
lazy wrapped text ok
    3. Three

would yield
    <ol>
        <li>One</li>
        <li>Two lazy wrapped text ok</li>
        <li>Three</li>
    </ol>

kramdown would look for a list item marker /\d+\./ at the start of each line, to find new list item blocks.

By these rules, a nested list would need to be wrapped in white-space lines:

    1. One
	2. Two

         a. Hello
         b. Goodbye

    3. Three

A list block following a list block at the same indentation is converted to a single list block with paragraph list items:

    1. One
    2. Two

    3. Three

    <ol>
       <li><p>One</p></li>
       <li><p>Two</p></li>
       <li><p>Three</p></li>
    </ol>

Tables and math blocks have their own internal syntax, but tables must be bordered by white-space lines and/or file boundaries:

    Here is a paragraph
    | with | table code | too close |

    <p>Here is a paragraph | with | table code | too close |</p>

    Here is a paragraph

    | with | table code | properly separated |

	<p>Here is a paragraph</p>
	<table>
		...
	</table>

These rules are summarized simply as:

  * All blocks must be bordered top/bottom by white-space lines and/or file boundaries, with the following exceptions.

  * List items are started by line boundary, optional whitespace, and list item marker /(\*|-|\+|\d+\.)/.  The first item of a list must be at the start of the file or have a white-space line directly above it.  The last item of a list must be at the end of the file or have a white-space line directly below it.

  * Table rows are started by line boundary, optional whitespace, and pipe character /\|/.  The first row of a table must be at the start of the file or have a white-space line directly above it.  The last row of a table must be at the end of the file or have a white-space line directly below it.

  * Math blocks start and end with $$ and have their own internal syntax.

If you read the text/plain part of this email, you'll likely see that the preceding text has been wrapped.  kramdown should still be able to recognize it as a list.  This is a core feature of Markdown, and it needs to be supported for kramdown to be acceptable by mainstream end users, who do not have patience to indent lines as precisely as you have advocated.  Markdown was created for its simplicity, to be usable by regular people, and support for lazy-indentation is a key part of that simplicity.  If kramdown does not support such a key feature of Markdown, then it really can't be called a superset of Markdown anymore.

Shawn

On Tue, 17 Aug 2010 09:52:51 -0700, Matt Neuburg wrote:
> On or about 8/17/10 7:14 AM, thus spake "Thomas Leitner" <t_leitner at gmx.at>:
> 
>> So if we change the behaviour for lists, we should probably also change
>> the behaviour for blockquotes, footnote definitions and all other
>> similar block level elements to remain consistent.
> 
> Well, I hope you don't change it. I find kramdown's behavior rational and
> deterministic (i.e. predictable and consistent), as opposed to Markdown
> where you never quite knew what would happen with a list, how your indenting
> would be interpreted, etc. And I find the documentation's explanation of how
> kramdown will behave to be clear and correct.
> 
> In the one particular case in question, if it is known that the input
> represents a list, it is trivial to preprocess it before kramdown sees it.
> It is up to the user to prepare the input for kramdown; it is not up to
> kramdown to adapt itself to incorrect input.
> 
> m.
> 
> -- 
> matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
> pantes anthropoi tou eidenai oregontai phusei
> Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
> AppleScript: the Definitive Guide, 2nd edition
> http://www.tidbits.com/matt/default.html#applescriptthings
> Take Control of Exploring & Customizing Snow Leopard
> http://tinyurl.com/kufyy8
> RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
> TidBITS, Mac news and reviews since 1990, http://www.tidbits.com
> 
> 
> 
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From matt at tidbits.com  Tue Aug 17 15:27:27 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Tue, 17 Aug 2010 12:27:27 -0700
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <20100818004324877115.7b83aa9a@gmail.com>
Message-ID: <C89030AF.54C18%matt@tidbits.com>

On or about 8/17/10 10:43 AM, thus spake "Shawn Van Ittersum"
<svicalifornia at gmail.com>:

> If kramdown does not support such a key feature of Markdown, then it really
> can't be called a superset of Markdown anymore

Agreed. The use of "superset" in the kramdown documentation is wrong.
kramdown isn't Markdown, and never was. It's what Markdown should have been.
:)

As I've said here before, I've got thousands of Web pages that are generated
using Markdown, and I've been converting them to use kramdown instead.
Basically *none* of them gives the desired output without alteration; I'm
having to rewrite *every* page in some way. However, that's fine with me;
it's the price I'm willing to pay to take advantage of kramdown's speed and
predictability. m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com

From svicalifornia at gmail.com  Tue Aug 17 16:42:57 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Wed, 18 Aug 2010 03:42:57 +0700
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <C89030AF.54C18%matt@tidbits.com>
References: <C89030AF.54C18%matt@tidbits.com>
Message-ID: <20100818034257351050.6db496ad@gmail.com>

Hi Matt and others,

In its current state, kramdown is not what Markdown should have been, unless you believe that kramdown should only be used by engineers.  Markdown was designed to make document markup more accessible to the common man.  kramdown, as evidenced by its many obtuse rules (just read the kramdown syntax page) has become anything but simple.  It has lost its simplicity and flexibility, imposing a rigid structure of precise indentation, taking it further and further away from the common man.  This is clearly not in the spirit of Markdown (see the Markdown philosophy page).

My users -- regular people, not computer experts -- won't understand why text wrapping by email clients makes a difference to their messages.  They will not follow kramdown's precise instructions for indentation.  They won't bother to learn other workarounds.  Instead, they will say that the software doesn't work, or it's too hard, and in any case, they won't use it.

I don't believe it is Thomas's intention to abandon such core principles of Markdown and make it difficult to use by the common man.  It appears that this has happened by accident, and that there is still a chance to simplify kramdown and make it easier for end users.

I have proposed how to retain kramdown's powerful superset features while keeping Markdown's simplicity.  The blank-line block separator is consistent with a core tenet of Markdown's design -- from Markdown's "Basics" page:

   > A paragraph is simply one or more consecutive lines of text, separated by one or more blank lines.

The parsing behavior I've described is completely predictable.  The syntax is much simpler for users whose text editors, email clients, or web browsers are wrapping their text without their knowledge.  So again I ask, what is the problem with separating blocks by white space, and then supporting lazy indentation within blocks?

Shawn

On Tue, 17 Aug 2010 12:27:27 -0700, Matt Neuburg wrote:
> On or about 8/17/10 10:43 AM, thus spake "Shawn Van Ittersum"
> <svicalifornia at gmail.com>:
> 
>> If kramdown does not support such a key feature of Markdown, then it really
>> can't be called a superset of Markdown anymore
> 
> Agreed. The use of "superset" in the kramdown documentation is wrong.
> kramdown isn't Markdown, and never was. It's what Markdown should have been.
> :)
> 
> As I've said here before, I've got thousands of Web pages that are generated
> using Markdown, and I've been converting them to use kramdown instead.
> Basically *none* of them gives the desired output without alteration; I'm
> having to rewrite *every* page in some way. However, that's fine with me;
> it's the price I'm willing to pay to take advantage of kramdown's speed and
> predictability. m.
> 
> -- 
> matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
> pantes anthropoi tou eidenai oregontai phusei
> Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
> AppleScript: the Definitive Guide, 2nd edition
> http://www.tidbits.com/matt/default.html#applescriptthings
> Take Control of Exploring & Customizing Snow Leopard
> http://tinyurl.com/kufyy8
> RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
> TidBITS, Mac news and reviews since 1990, http://www.tidbits.com
> 
> 
> 
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From t_leitner at gmx.at  Thu Aug 19 13:29:09 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 19 Aug 2010 19:29:09 +0200
Subject: [kramdown-users] is this rule necessary?
In-Reply-To: <20100808201524.7a8e7368@noweto>
References: <20100806084449.705b0cbe@noweto> <C88102BC.549AA%matt@tidbits.com>
	<20100808201524.7a8e7368@noweto>
Message-ID: <20100819192909.75f5739e@noweto>

> As mentioned above the reason for having the angle brackets is because
> Markdown.pl supports them... So I would rather not remove them.
> However, as said in my previous mail, I will try to change the link
> syntax to support spaces without using angle brackets.

I have just added support for spaces in inline links and link
definitions and pushed the latest version of kramdown to github. Please
try this out to see if it fits your needs and if any side effects
arise! Thanks!

-- Thomas

From t_leitner at gmx.at  Thu Aug 19 13:53:05 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 19 Aug 2010 19:53:05 +0200
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <20100818004324877115.7b83aa9a@gmail.com>
References: <C8900C73.54C0B%matt@tidbits.com>
	<20100818004324877115.7b83aa9a@gmail.com>
Message-ID: <20100819195305.70ec2c76@noweto>

> I continue to advocate the use of white-space lines to separate
> blocks.  Contiguous lines of text, regardless of indentation, belong
> to the same block.  Thus:
> 
>     > ... but this is wrong!
>     No, it is not!
> 
> is a single blockquote, while

The problem with the above example is that it can be interpreted in
both ways and we have to decide what's the better one.

> 
>     > ... but this is wrong!
> 
>     No, it is not!
> 
> results in one blockquote and one paragraph.

This works in Markdown and in kramdown. If you separate your block
level elements by blank lines you will always get the correct result.
However, blank lines between *all* block level elements is not enforced
- see the syntax documentation where blank lines are needed.

> A list block following a list block at the same indentation is
> converted to a single list block with paragraph list items:
> 
>     1. One
>     2. Two
> 
>     3. Three
> 
>     <ol>
>        <li><p>One</p></li>
>        <li><p>Two</p></li>
>        <li><p>Three</p></li>
>     </ol>

This is already done, however, conversion of simple list texts to
paragraphs follows other rules. In the above case, only the second item
would be wrapped in a paragraph.

> Tables and math blocks have their own internal syntax, but tables
> must be bordered by white-space lines and/or file boundaries:
> 
>     Here is a paragraph
>     | with | table code | too close |
> 
>     <p>Here is a paragraph | with | table code | too close |</p>

This is not enforced in kramdown, ie. you can follow a paragraph
directly with a table.

> If you read the text/plain part of this email, you'll likely see that
> the preceding text has been wrapped.  kramdown should still be able
> to recognize it as a list.  This is a core feature of Markdown, and
> it needs to be supported for kramdown to be acceptable by mainstream
> end users, who do not have patience to indent lines as precisely as
> you have advocated.  Markdown was created for its simplicity, to be
> usable by regular people, and support for lazy-indentation is a key
> part of that simplicity.  If kramdown does not support such a key
> feature of Markdown, then it really can't be called a superset of
> Markdown anymore.

Yes, I *see* that the text is wrapped by my email program. However,
when I copy and paste the text, each paragraph is still copied as
one long line and therefore there are no problems with kramdown
processing the content. So if you just *write* long lines, everything
is fine with kramdown. However, if you *hard-wrap* lines yourself, then
you are in trouble.

Aside from the specific discussion about list syntax we can talk about
whether it would be useful to enforce the use of blank lines for some
more block level elements. For example, headers have to be preceded by a
blank line already. We could discuss if it would be useful to do
something similar with other block elements like math block and
tables... 

-- Thomas

From t_leitner at gmx.at  Thu Aug 19 13:55:07 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 19 Aug 2010 19:55:07 +0200
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <C89030AF.54C18%matt@tidbits.com>
References: <20100818004324877115.7b83aa9a@gmail.com>
	<C89030AF.54C18%matt@tidbits.com>
Message-ID: <20100819195507.1bca21ba@noweto>

On 2010-08-17 12:27 -0700 Matt Neuburg wrote:
> On or about 8/17/10 10:43 AM, thus spake "Shawn Van Ittersum"
> <svicalifornia at gmail.com>:
> 
> > If kramdown does not support such a key feature of Markdown, then
> > it really can't be called a superset of Markdown anymore
> 
> Agreed. The use of "superset" in the kramdown documentation is wrong.
> kramdown isn't Markdown, and never was. It's what Markdown should
> have been. :)

:-) Yes, you are right, kramdown is neither a correct subset nor a
correct superset of Markdown. However, since many Markdown documents
just work fine with kramdown, I think it is okay to use the word
"superset". And recall that we haven't reached one-oh-oh yet ;-)

-- Thomas

From t_leitner at gmx.at  Thu Aug 19 14:45:35 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Thu, 19 Aug 2010 20:45:35 +0200
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <20100818034257351050.6db496ad@gmail.com>
References: <C89030AF.54C18%matt@tidbits.com>
	<20100818034257351050.6db496ad@gmail.com>
Message-ID: <20100819204535.627b17b8@noweto>

> In its current state, kramdown is not what Markdown should have been,
> unless you believe that kramdown should only be used by engineers.
> Markdown was designed to make document markup more accessible to the
> common man.  kramdown, as evidenced by its many obtuse rules (just
> read the kramdown syntax page) has become anything but simple.  It
> has lost its simplicity and flexibility, imposing a rigid structure
> of precise indentation, taking it further and further away from the
> common man.  This is clearly not in the spirit of Markdown (see the
> Markdown philosophy page).

I consider having a complete syntax documentation a plus because you
can read it and know what output you can expect. Contrast that with the
documentation of Markdown...

However, you are right that there may be some rules that only exist
because they simplify parsing or they allow much more fine grained
control. The syntax and its documentation grew with the new features
and both should probably be revised before the 1.0 release. Help with
that would be very appreciated!

> My users -- regular people, not computer experts -- won't understand
> why text wrapping by email clients makes a difference to their
> messages.  They will not follow kramdown's precise instructions for
> indentation.  They won't bother to learn other workarounds.  Instead,
> they will say that the software doesn't work, or it's too hard, and
> in any case, they won't use it.

Text wrapping aside, what is the problem regarding indentation?

> I don't believe it is Thomas's intention to abandon such core
> principles of Markdown and make it difficult to use by the common
> man.  It appears that this has happened by accident, and that there
> is still a chance to simplify kramdown and make it easier for end
> users.

I'm all for simplifying but I don't want to loose advanced functionality
in the process. So if we come up with a different syntax for something
that seems to be too difficult to use by the common man, I'm willing to
implement it.

> The parsing behavior I've described is completely predictable.  The
> syntax is much simpler for users whose text editors, email clients,
> or web browsers are wrapping their text without their knowledge.  So
> again I ask, what is the problem with separating blocks by white
> space, and then supporting lazy indentation within blocks?

There is nothing wrong with separating block level elements by blank
lines, you can do that already - but it is not enforced.

Lazy indentation is another beast that we can tackle if it is really
needed. I will write something up based on your previous email (and
probably after combing through the Markdown ML archives) and start a new
thread where we can discuss this further. However, I really think that
it is not that easy to get it right...

-- Thomas

From svicalifornia at gmail.com  Thu Aug 19 16:43:30 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Fri, 20 Aug 2010 03:43:30 +0700
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <20100819204535.627b17b8@noweto>
References: <C89030AF.54C18%matt@tidbits.com>
	<20100818034257351050.6db496ad@gmail.com>
	<20100819204535.627b17b8@noweto>
Message-ID: <20100820034330254159.9758928f@gmail.com>

Hi Thomas,

Thanks for your reply.

Gmail sends messages with two MIME parts: one text/plain, and one HTML.  When you copied my list, I think you may have been looking at the HTML part, instead of the text/plain part.  The text/plain part is usually wrapped to less than 78 characters per line, according to RFC 2822:

> There are two limits that this standard places on the number of characters in a line. Each line of characters MUST be no more than 998 characters, and SHOULD be no more than 78 characters, excluding the CRLF.

For more info, see: http://mailformat.dan.info/body/linelength.html

I want for my users to be able to send Markdown-format messages by email to an automated system that reads the Markdown and renders HTML.  However, if the mail transport systems wrap the Markdown text, following RFC 2822, then kramdown does not parse it correctly.  So the problem with the indentation, in this case, is that it is outside my control and my users' control, because the mail system is wrapping the lines.

John Gruber's syntax for Markdown allows this kind of wrapping.  kramdown does not.  Even if my users are careful about indenting paragraphs as kramdown prescribes, they must also wrap their text to less than 78 characters per line to prevent the mail system from doing it for them and messing up the indentation.  These users should not have to worry about things like this, because Markdown says that lazy indentation is okay, and kramdown claimed to be a superset of Markdown.

If we make a rule that blocks are separated by white-space lines (with the exceptions that I've already explained), then kramdown could be made to handle both lazy indentation and email-wrapping.  And I believe it would make the kramdown syntax much, much simpler to parse and to explain.

It will be much easier to explain to my users that paragraphs, lists, and tables should be separated by blank lines, than to tell them that they need a specialized text editor (Notepad won't cut it) to count characters and wrap lines.  No text editor, even Textmate, will wrap to kramdown's specifications out-of-the-box.  They'd have to wrap one line at a time: indent, wrap, indent, wrap, etc.  This should not be a requirement, because kramdown can and should allow lazy indentation as Markdown does, by enforcing blank lines between blocks.

Thanks,
Shawn

On Thu, 19 Aug 2010 20:45:35 +0200, Thomas Leitner wrote:
>> In its current state, kramdown is not what Markdown should have been,
>> unless you believe that kramdown should only be used by engineers.
>> Markdown was designed to make document markup more accessible to the
>> common man.  kramdown, as evidenced by its many obtuse rules (just
>> read the kramdown syntax page) has become anything but simple.  It
>> has lost its simplicity and flexibility, imposing a rigid structure
>> of precise indentation, taking it further and further away from the
>> common man.  This is clearly not in the spirit of Markdown (see the
>> Markdown philosophy page).
> 
> I consider having a complete syntax documentation a plus because you
> can read it and know what output you can expect. Contrast that with the
> documentation of Markdown...
> 
> However, you are right that there may be some rules that only exist
> because they simplify parsing or they allow much more fine grained
> control. The syntax and its documentation grew with the new features
> and both should probably be revised before the 1.0 release. Help with
> that would be very appreciated!
> 
>> My users -- regular people, not computer experts -- won't understand
>> why text wrapping by email clients makes a difference to their
>> messages.  They will not follow kramdown's precise instructions for
>> indentation.  They won't bother to learn other workarounds.  Instead,
>> they will say that the software doesn't work, or it's too hard, and
>> in any case, they won't use it.
> 
> Text wrapping aside, what is the problem regarding indentation?
> 
>> I don't believe it is Thomas's intention to abandon such core
>> principles of Markdown and make it difficult to use by the common
>> man.  It appears that this has happened by accident, and that there
>> is still a chance to simplify kramdown and make it easier for end
>> users.
> 
> I'm all for simplifying but I don't want to loose advanced functionality
> in the process. So if we come up with a different syntax for something
> that seems to be too difficult to use by the common man, I'm willing to
> implement it.
> 
>> The parsing behavior I've described is completely predictable.  The
>> syntax is much simpler for users whose text editors, email clients,
>> or web browsers are wrapping their text without their knowledge.  So
>> again I ask, what is the problem with separating blocks by white
>> space, and then supporting lazy indentation within blocks?
> 
> There is nothing wrong with separating block level elements by blank
> lines, you can do that already - but it is not enforced.
> 
> Lazy indentation is another beast that we can tackle if it is really
> needed. I will write something up based on your previous email (and
> probably after combing through the Markdown ML archives) and start a new
> thread where we can discuss this further. However, I really think that
> it is not that easy to get it right...
> 
> -- Thomas
> _______________________________________________
> kramdown-users mailing list
> kramdown-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/kramdown-users

From matt at tidbits.com  Thu Aug 19 22:55:11 2010
From: matt at tidbits.com (Matt Neuburg)
Date: Thu, 19 Aug 2010 19:55:11 -0700
Subject: [kramdown-users] is this rule necessary?
In-Reply-To: <20100819192909.75f5739e@noweto>
Message-ID: <C8933C9F.54C8D%matt@tidbits.com>

On or about 8/19/10 10:29 AM, thus spake "Thomas Leitner"
<t_leitner at gmx.at>:

>> As mentioned above the reason for having the angle brackets is because
>> Markdown.pl supports them... So I would rather not remove them.
>> However, as said in my previous mail, I will try to change the link
>> syntax to support spaces without using angle brackets.
> 
> I have just added support for spaces in inline links and link
> definitions and pushed the latest version of kramdown to github. Please
> try this out to see if it fits your needs and if any side effects
> arise! Thanks!

Perfect. I generated the html pages of a large web site using the previous
version of kramdown. Then I changed all instances of [...](<...>) throughout
my web site to [...](...) and regenerated the html pages using the current
(repository) version of kramdown. The resulting pages were all identical. So
there are no side effects whatever in any of those pages (hundreds of them).
Thank you! m.

-- 
matt neuburg, phd = matt at tidbits.com, http://www.tidbits.com/matt/
pantes anthropoi tou eidenai oregontai phusei
Among the 2007 MacTech Top 25, http://tinyurl.com/2rh4pf
AppleScript: the Definitive Guide, 2nd edition
http://www.tidbits.com/matt/default.html#applescriptthings
Take Control of Exploring & Customizing Snow Leopard
http://tinyurl.com/kufyy8
RubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html
TidBITS, Mac news and reviews since 1990, http://www.tidbits.com

From t_leitner at gmx.at  Fri Aug 20 02:48:25 2010
From: t_leitner at gmx.at (Thomas Leitner)
Date: Fri, 20 Aug 2010 08:48:25 +0200
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <20100820034330254159.9758928f@gmail.com>
References: <C89030AF.54C18%matt@tidbits.com>
	<20100818034257351050.6db496ad@gmail.com>
	<20100819204535.627b17b8@noweto>
	<20100820034330254159.9758928f@gmail.com>
Message-ID: <20100820084825.7bf8fe6b@noweto>

> Gmail sends messages with two MIME parts: one text/plain, and one
> HTML.  When you copied my list, I think you may have been looking at
> the HTML part, instead of the text/plain part.  The text/plain part
> is usually wrapped to less than 78 characters per line, according to
> RFC 2822:
> 
> > There are two limits that this standard places on the number of
> > characters in a line. Each line of characters MUST be no more than
> > 998 characters, and SHOULD be no more than 78 characters, excluding
> > the CRLF.
> 
> For more info, see: http://mailformat.dan.info/body/linelength.html

No, I just get the text/plain part from the mailing list. And my email
client does not perform hard-wrapping but only soft-wrapping of text
messages. I.e. if I copy the text I won't get the newline characters.
So again, this  will work in kramdown.

Have you skimmed the article you linked to until the end? There they say
that you should wrap your text *before* sending, ie. you should
configure your mail program like I did with mine to wrap the text so
that there won't be any problem in any mail client! You could do this
with/for your users - then the sent messages will contain nicely
formatted text. The only problems that remains would be when quoting
another email...

> I want for my users to be able to send Markdown-format messages by
> email to an automated system that reads the Markdown and renders
> HTML.  However, if the mail transport systems wrap the Markdown text,
> following RFC 2822, then kramdown does not parse it correctly.  So
> the problem with the indentation, in this case, is that it is outside
> my control and my users' control, because the mail system is wrapping
> the lines.

As said above, just correctly configure the mail clients (if you have
access to them!).

> If we make a rule that blocks are separated by white-space lines
> (with the exceptions that I've already explained), then kramdown
> could be made to handle both lazy indentation and email-wrapping.
> And I believe it would make the kramdown syntax much, much simpler to
> parse and to explain.

There has been some discussion on this on the Markdown ML, for example,
http://six.pairlist.net/pipermail/markdown-discuss/2006-August/000151.html
http://www.mail-archive.com/markdown-discuss at six.pairlist.net/msg00643.html
http://www.mail-archive.com/markdown-discuss at six.pairlist.net/msg00692.html
http://www.mail-archive.com/markdown-discuss at six.pairlist.net/msg00698.html
http://www.mail-archive.com/markdown-discuss at six.pairlist.net/msg00704.html

> It will be much easier to explain to my users that paragraphs, lists,
> and tables should be separated by blank lines, than to tell them that
> they need a specialized text editor (Notepad won't cut it) to count
> characters and wrap lines.  No text editor, even Textmate, will wrap
> to kramdown's specifications out-of-the-box.  They'd have to wrap one
> line at a time: indent, wrap, indent, wrap, etc.  This should not be
> a requirement, because kramdown can and should allow lazy indentation
> as Markdown does, by enforcing blank lines between blocks.

Just tell them to use blank lines between block level elements and one
worry will just go away! As for the lazy indentation, I'm still
skimming the Markdown ML on this.

-- Thomas

From svicalifornia at gmail.com  Fri Aug 20 04:23:55 2010
From: svicalifornia at gmail.com (Shawn Van Ittersum)
Date: Fri, 20 Aug 2010 15:23:55 +0700
Subject: [kramdown-users] Bug in list parsing
In-Reply-To: <20100820084825.7bf8fe6b@noweto>
References: <C89030AF.54C18%matt@tidbits.com>
	<20100818034257351050.6db496ad@gmail.com>
	<20100819204535.627b17b8@noweto>
	<20100820034330254159.9758928f@gmail.com>
	<20100820084825.7bf8fe6b@noweto>
Message-ID: <20100820152355450702.05d4706c@gmail.com>

> Have you skimmed the article you linked to until the end? There they say
> that you should wrap your text *before* sending, ie. you should
> configure your mail program like I did with mine to wrap the text so
> that there won't be any problem in any mail client! You could do this
> with/for your users - then the sent messages will contain nicely
> formatted text. The only problems that remains would be when quoting
> another email...

If the email client wraps long list items, such as this:

  * Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
  * Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

It will be wrapped as:

   * Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore
et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut
aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa
qui officia deserunt mollit anim id est laborum.
   * Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore
et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut
aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse
cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa
qui officia deserunt mollit anim id est laborum.

See the problem?  Markdown handles this gracefully, because it supports lazy indentation.  kramdown does not.

> Just tell them to use blank lines between block level elements and one
> worry will just go away! As for the lazy indentation, I'm still
> skimming the Markdown ML on this.

Markdown supports lazy indentation throughout the syntax.  It's explicitly mentioned as a key part of its design, on the "Basics" page and on the "Syntax" page.

Shawn