Sphinx 1.3.5: Could not parse literal_block as "language"

[stevepiercy]

The following might not be a bug, but it is a changed behavior that enforces stricter syntax, and should be noted.

How to reproduce

For this code sample (and several others):

.. code-block:: ini
   :linenos:

   [app:main]
   ...
   pyramid.includes =
       pyramid_debugtoolbar

Sphinx 1.3.4 yielded no errors. Upgrading to 1.3.5 and attempting to build HTML causes the following error:

/home/travis/build/Pylons/pyramid/docs/narr/project.rst:434: WARNING: Could not parse literal_block as "ini". highlighting skipped.

How to Fix/Workaround

Use a comment for elided output instead of a literal .... It appears that one can no longer use incorrect syntax for a given language for highlighting.

.. code-block:: ini
   :linenos:

   [app:main]
   # ... elided configuration
   pyramid.includes =
       pyramid_debugtoolbar

The screenshot compares two examples, where the first is highlighted but the second is not.

I noticed that this shows up for javascript or posh syntaxes, too.

Possibly related to #2262.

[birkenfeld]

The second block would have been unhighlighted in previous Sphinx versions too. Only the warning is new to make you aware of it.

[stevepiercy]

@birkenfeld is this a change in Pygments or Sphinx? Either way, it's a good change that helped me catch a couple of other mistakes.

[birkenfeld]

This was added to Sphinx in #1565.

[stevepiercy]

Would it be helpful to add to the change log something like this?

If the syntax is incorrect for the :language: option in a .. code-block:: directive, Sphinx will now emit a warning as follows.

WARNING: Could not parse literal_block as "<language>". highlighting skipped.

Reasoning: When searching the change log, I could not find anything relevant to "literal_block". This would link the error message to a specific change log entry and provide a clue for what changed in 1.3.5 and how to fix it.

Alternatively, the warning message could refer to the specific directive that literal_block refers to, e.g.,

WARNING: Could not parse code-block as "<language>". highlighting skipped.

I can submit a PR, but the change log is very terse, and I wouldn't want to break that pattern.

If not, then this issue can be closed, and at the least this discussion can be indexed by search engines.

[tk0miya]

OK, I'll update it later.

[tk0miya]

@stevepiercy This change also effects to other directives (cf. literalinclude, parsed-literal and so on).

So I rewrite changelog as follows:

• Add an option to throw a warning when Pygments throws an ErrorToken #1565: Sphinx will now emit a warning that highlighting was skipped if the syntax
  is incorrect for code-block, literalinclude and so on.

Thanks,

[stevepiercy]

Thank you!

[karelyatin]

If i use sphinx 1.6.2 or 1.6.3 with pygments<=2.1.3 the following error is reported:
Could not lex literal_block as "ini". Highlighting skipped.

But with pygments 2.2 The above error is not reported.

[jean]

Is it possible to log in more detail why lexing failed?

[tk0miya]

@jean could you file a new issue please? I'll take a look later.