What is meta? ×

Have a look at the revisions of http://stackoverflow.com/revisions/1230618/list, in particular the formatting-related edit I made (edit 2). The formatted text contains spurious newlines and indentation errors (not to speak of the randomly inserted HTML escape sequence). Even deleting the newlines and re-inserting them didn’t help.


Minimal example that still exhibits the buggy behaviour:

  1. Item


Source (displays exactly right):

1. Item


Another example, from an old posting of mine that worked before, but is now broken: http://stackoverflow.com/questions/244959/coalescing-regular-expressions-in-php/248361#248361

Notice how the posting’s source code ends on “<strong&”. Unfortunately, I can’t reproduce this bug here since the code looks fine in preview (click “edit” to look at it there); only the normal display is wrong, suggesting that the server-side Markdown processor has an additional bug.

share|improve this question
Some of these errors can be worked around (while a permanent solution is not ready) by using the workaround mentioned here: meta.stackexchange.com/questions/29425/… –  Abel Nov 13 '09 at 15:14

3 Answers 3

up vote 33 down vote accepted

The culprit was this tiny line in the Markdown specification:

If list items are separated by blank lines, Markdown will wrap the items in <p> tags in the HTML output.

Besides the fact that this is actually ambiguous (Which item is wrapped in <p> tags? The one before the blank line? The one after it? Both? All of them, once there's a blank line anywhere?) and that different implementations actually handle this differently, here's how the original implementation does it:

The first element is wrapped in <p> if it's followed by a blank line, the last element when it's preceded by a blank line, and all other ones when there's a blank line at least on one side. All items are wrapped if they contain blank lines.

All well, there's some consistency to this. The implementation, however, tries to express this logic in a regular expression:

    (\n)?                            # leading line = $1
    (^[ \t]*)                        # leading whitespace = $2
    ($marker_any) [ \t]+             # list marker = $3
    ((?s:.+?)                        # list item text   = $4
    (?= \n* (\z | \2 ($marker_any) [ \t]+))

When a Markdown list is separated into items, this preserves one or two newline characters at the end (one meaning there's no blank line, two meaning there is), and one newline character at the beginning.

The presence of the double newline at the end means this item is followed by a blank line. The presence of a leading newline on top means this item is preceded by a blank line.

Well, almost. Because now we need three newline characters between the items, two to be captured at the end, and one to be captured at the top.

Well, that's easy:

# Turn double returns into triple returns, so that we can make a
# paragraph for the last item in a list, if necessary:
$list =~ s/\n{2,}/\n\n\n/g;

– and that's why code in lists started growing newlines.

Many Markdown ports to other languages do it the same way – including Attacklab Showdown (SO's client-side Markdown renderer) and MarkdownSharp (the server-side version).

After removing this piece of logic from the regex and instead keeping a little bit of extra state* between the list item processing steps, we could remove the double-triple-conversion.

* appropriately called last_item_had_a_double_newline

share|improve this answer
Amazing. I know from experience how hard debugging regular expressions is. And then there’s the complexity of parsing Markdown at all … William Shields’ attempt at parsing Markdown with a formal parser (as far as I see) hasn’t been fruitful either. –  Konrad Rudolph Jan 6 '11 at 17:21
amazing work Ben! –  Jeff Atwood Jan 6 '11 at 21:30

There have been ZERO changes to any of the markdown code in months. FYI. Be happy to share SVN logs with you (privately) if you don't believe me.

I think this is simply a condition we hadn't discovered before. It seems to be based on spacing, like so:

  1. uno

  2. dos

  3. tres

  4. cuatro


Conclusion: this is not a new bug, surprisingly -- both the client and server side markdown renderers do the same thing here! That leads me to believe it's a narrow condition around code within numbered lists.

Bottom line, if you have code (8-space-indent) inside a bulleted list, you cannot have blank lines inside the code. Bizarre, but true.

Checking in http://babelmark.bobtfish.net verifies this is a bug in multiple markdown engines. Click here to see the above sample text in multiple markdown engines. There's definitely a disturbance in the force here..

share|improve this answer
Whitespaceisforwimpsanyway. –  Marc Gravell Sep 2 '09 at 22:14
I could have sworn that I’d already used this construct – and it’s not that unusual, after all. Well, if the logs say differently, that’s fair enough. –  Konrad Rudolph Sep 3 '09 at 10:14
I was surprised too but I can't think of anything that has changed -- it's duplicated on both client and server, and the babelfish Markdown processor (some of them anyway) show the SAME error. –  Jeff Atwood Sep 3 '09 at 12:26
This is just righteous punishment for people who use 8-space-wide indenting. –  TM. Dec 23 '09 at 19:20
You have to do that in lists, TM. –  GManNickG Dec 23 '09 at 19:34
@GMan, i take his comment to be about 8-space-indenting in some random source code file, where actual tabs or 3-or-4-space-indenting is better. –  quack quixote Dec 23 '09 at 21:27
This would be awesome if it was fixed, since ctr-k doesn't work on code in a numbered list ( you have to hand indent the extra 4 lines ). –  Peter Ajtai Oct 3 '10 at 0:12

Also note that additional scroll bars appear if there are long lines in a code block containing empty lines:

  1. Create list
  2. Add code:

    function() {
    // here we do lot's of stuff, and because comments are evil we get strange scrollbars displayed

(See also the source view of this post)

share|improve this answer
this is actually a different (now fixed) issue, see meta.stackexchange.com/questions/73799/… –  balpha Jan 7 '11 at 12:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .