Check for numbers after markers when setting fold level

[rhelder]

The Problem

Vim's built-in capability for folding code using markers (at least if one is using the default markers, as the Vim documentation recommends) is unsatisfactory when editing LaTeX documents, because it is not so unlikely that a line of LaTeX code will contain three closing braces and be misconstrued by Vim as the end of a fold. VimTeX's folding mechanism improves on this situation by only recognizing markers in comments (although if one uses the default markers, even this can sometimes have unexpected results, as dicussed in e.g. #1502 and #1515). However, VimTeX's folding mechanism falls short compared to Vim's in that VimTeX does not recognize numbers after markers, which are used in Vim to manually specify the fold level.

This difference between VimTeX's folding and Vim's native folding can have not only unexpected but also unfortunate consequences. I've uploaded an example file with various nested folds (It is supposed to imitate a personal sty file, although please note I had to upload it as a txt file, since GitHub wouldn't accept a tex file. Also, disclaimer: I don't necessarily endorse the folds in the example file with respect to style/organization; I just wanted to make sure there was some good nesting in there. :)) Here's how the file looks at first glance with Vim's built-in folding:

But here's how it looks with VimTeX's folding:

Because Vim allows you to manually specify the fold level, not every opening marker needs to be followed by a closing marker (which is nice, because it's less messy). My general policy, for example, is that I only add a closing marker when I want there to be white space visible after the fold. The result, in this file, is that no opening marker of fold level 1 is followed by a closing marker (except at the end of the document; that way, you can add things to the end of the document outside of the last fold), but every opening marker of fold level greater than 1 is followed by a corresponding closing fold marker. (I think you follow this sort of style in many, though not all, of your VimTeX source files). What happens, then, because VimTeX doesn't recognize the numbers following the fold markers, is that the fold level increases for each {{{1, without there being a closing marker to decrease the fold level again. So the fold level gets bigger and bigger, and everything gets subsumed under 'Options'.

A Solution

Essentially, my code just looks for a number after the marker specified by the user (or the default marker) by appending a capture group to the existing regexes and using matchlist to get the number. If there is a number, the the level function returns e.g. '>2' if the marker is an opening marker, and e.g. '<2' if the marker is a closing marker. So we get the expected, Vim-like behavior, and VimTeX users can specify the g:vimtex_fold_types.markers option in the way that they always have. Here's how it looks at first glance:

The foldtext

I personally feel dissatisfied with VimTeX's foldtext for markers. The markers themselves don't mean anything, so they shouldn't be part of the text, and there's also no way to see what level each fold is. I considered including an alternative foldtext in my PR, but then I decided I was getting myself involved in matters of taste... Others, including you, may well like the text returned by the current foldtext. And people like me, if they want e.g. the default Vim foldtext, can very easily get it like so:

" Make sure the g:vimtex_fold_types.markers dict is defined!
let g:vimtex_fold_types = {'markers': {}}

function! g:vimtex_fold_types.markers.text(line, level) abort dict
  return foldtext()
endfunction

The only recommendation I might make would be to inform people in the documentation that they can specify their own foldtext by adding a funcref to the appropriate dictionary in their g:vimtex_fold_types configuration, as above. Someone could only figure this out by reading the code (or reading this post, I suppose). If you want to add that tip to the documentation (I can see reasons not to), I would be happy to write something and submit it as a separate PR, if that would save you some time.

I appreciate all of your work on VimTeX, and I hope this code can be helpful. Thank you!

[lervag]

Thanks, this seems like a valid and good contribution! I had a couple of minor comments to the code.

Also, a couple of comments to your text:

However, VimTeX's folding mechanism falls short compared to Vim's in that VimTeX does not recognize numbers after markers, which are used in Vim to manually specify the fold level.

True.

Also, disclaimer: I don't necessarily endorse the folds in the example file with respect to style/organization; I just wanted to make sure there was some good nesting in there. :)

Haha, good to hear!

(I think you follow this sort of style in many, though not all, of your VimTeX source files).

Yes; although I've started to shift towards syntax based folding with tree-sitter (only possible with neovim, though).

I find marker based folding to be sort of a "poor man's" solution when a filetype does not have any very sensible default. With treesitter, it seems we are getting closer to good folding for many more syntaxes, which is something I like quite a lot.

The foldtext

I personally feel dissatisfied with VimTeX's foldtext for markers. The markers themselves don't mean anything, so they shouldn't be part of the text, and there's also no way to see what level each fold is. I considered including an alternative foldtext in my PR, but then I decided I was getting myself involved in matters of taste... Others, including you, may well like the text returned by the current foldtext. And people like me, if they want e.g. the default Vim foldtext, can very easily get it like so:

Thanks for recognizing the matter of taste here. Personally, I like having the markers in the foldtext to make it clear that this is indeed a marker based fold. I recognize that this is clearly a subjective choice, which is one of the reasons I've tried to make it relatively easy to customize for anyone who wants to.

Notice, I think you can make the customization even simpler:

let g:vimtex_fold_types = #{markers: #{
      \ text: { line, level -> foldtext() }
      \}}

The only recommendation I might make would be to inform people in the documentation that they can specify their own foldtext by adding a funcref to the appropriate dictionary in their g:vimtex_fold_types configuration, as above. Someone could only figure this out by reading the code (or reading this post, I suppose).

Good point.

If you want to add that tip to the documentation (I can see reasons not to), I would be happy to write something and submit it as a separate PR, if that would save you some time.

Yes, thank you, I would very much appreciate that!

I think the best place to add this is under the section "FOLDING" and/or under the docs for g:vimtex_fold_types.

I appreciate all of your work on VimTeX, and I hope this code can be helpful. Thank you!

Thanks! Glad you find VimTeX useful and I'm happy to see your contribution!

[rhelder]

Yes; although I've started to shift towards syntax based folding with tree-sitter (only possible with neovim, though).

I find marker based folding to be sort of a "poor man's" solution when a filetype does not have any very sensible default. With treesitter, it seems we are getting closer to good folding for many more syntaxes, which is something I like quite a lot.

That's helpful to hear your thoughts on marker-based folding – although I submit an example of a sty file with this PR, when you start doing package development with e.g. dtx files (where user comments become part of the syntax) marker folding becomes more and more problematic, so it's nice to hear your thoughts on alternatives.

Notice, I think you can make the customization even simpler:

Again, I wouldn't have thought of this – glad to learn something!

Yes, thank you, I would very much appreciate that!

I think the best place to add this is under the section "FOLDING" and/or under the docs for g:vimtex_fold_types.

Great, I will get to work on that. Thanks again!

[lervag]

That's helpful to hear your thoughts on marker-based folding – although I submit an example of a sty file with this PR, when you start doing package development with e.g. dtx files (where user comments become part of the syntax) marker folding becomes more and more problematic, so it's nice to hear your thoughts on alternatives.

Glad to share my thoughts :)

Notice, I think you can make the customization even simpler:

Again, I wouldn't have thought of this – glad to learn something!

My pleasure!

Yes, thank you, I would very much appreciate that!
I think the best place to add this is under the section "FOLDING" and/or under the docs for g:vimtex_fold_types.

Great, I will get to work on that. Thanks again!

Looking forward to seeing it. And thanks to you for this; I've merged it now with a minor fix on top.