Markdown mode is getting slow

[mrkkrp]

I'm trying to fight frightening typing latency in my Emacs. It LAGS WHEN I TYPE. It's slower than an IDE. It's become really slow.

Please think twice when you put heavy code into hooks that are effectively executed after every key-press. I'm already thinking about writing state-less Smartparens that does not skyrocket latency. I already have to invent something to deal with the same issue with Flyspell. Heck, I have other things to do except for constantly making sure that I can type in Emacs normally. Every package puts a little bit of not-very-fast code into hooks, and then it all sums up. This is a real problem for me now. I don't even know why the heck I need to scan and fontify wiki links. What sort of Wiki? Can you make a variable that would turn off all these hooks expect for very-very basic ones? Or should I rewrite parts of markdown-mode in my config in order to be able to use it?

[mrkkrp]

I'm sorry for being so harsh. I'm trying to investigate cause of this disaster. Smartparens executes a lot of stuff even when disabled. I'll need to remove it completely together with Flyspell to adequately profile the rest of offending packages. markdown-check-change-for-wiki-link was quite slow in one test, but now I cannot reliably reproduce it.

[mrkkrp]

Here we go:

- redisplay_internal (C function)                                3449  68%
 - jit-lock-function                                             3381  67%
  - jit-lock-fontify-now                                         3381  67%
   - jit-lock--run-functions                                     3381  67%
    - run-hook-wrapped                                           3381  67%
     - #<compiled 0xde4caf>                                      3381  67%
      - font-lock-fontify-region                                 3381  67%
       - font-lock-default-fontify-region                        3381  67%
        + font-lock-fontify-syntactically-region                 3195  63%
        - font-lock-fontify-keywords-region                       179   3%
         + markdown-match-code                                     32   0%
         + markdown-match-italic                                   26   0%
         + markdown-match-bold                                     25   0%
         + markdown-match-blockquotes                              10   0%
           markdown-match-hr                                        7   0%
         + markdown-match-multimarkdown-metadata                    6   0%
         + markdown-match-heading-2-setext                          3   0%
         + markdown-match-fenced-start-code-block                   3   0%
         + markdown-match-pandoc-metadata                           3   0%
 + eval                                                            55   1%
   file-remote-p                                                    4   0%
 + #<compiled 0x3e3d5f>                                             3   0%
- ...                                                            1025  20%
   Automatic GC                                                  1025  20%
- command-execute                                                 527  10%
 - call-interactively                                             527  10%
  - funcall-interactively                                         527  10%
   + counsel-M-x                                                  425   8%
   - self-insert-command                                           99   1%
    + markdown-check-change-for-wiki-link-after-change                 78   1%
    + markdown-gfm-checkbox-after-change-function                  14   0%
    + jit-lock-after-change                                         7   0%
  sml/generate-position-help                                        7   0%

When I hold a key, I can see how characters periodically slow down for GC. This is poor performance anyway — this should not happen and it does not happen in modes with more complicated syntax than markdown. Fontification is slow. When combined with other nasty things, it stops you from normal interaction with editor altogether. Can you imporve this?

[mrkkrp]

Edit: fontification is not slow, it's rather memory-inefficient if you spend 20% if time in GC. In Emacs lisp mode it's 5 % even with some plugins like aggressive indentation.

[jrblevin]

First, can you confirm that you are using the latest master? Much effort recently has gone towards efficiency. I have done similar profiling and have seen significant improvement since December or so. See for example #30, #42, and #44. For me, the lag while typing now is almost imperceptible in a complicated 5000+ line file.

Due to the complexity, parsing Markdown on the fly as a user edits is neither easy nor fast when compared to a very structured language, like most programming languages, which can use the Emacs syntax table for everything. Multi-line constructs, nesting lists and pre blocks, etc. mean that we have to sometime go back far in the document to determine the context. Recently, I have moved towards caching this context in text properties. That being said, I still care a lot about making markdown-mode as efficient as possible. If you have other suggestions for improvement they are most welcome, especially in the form of patches.

Many people use Markdown when editing wikis, including on GitHub, which is why the wiki features exist. However, I agree that most people do not need this and therefore adding flags to disable checking for wiki links is something I've already planned to do. We need two flags: one to globally disable wiki link fontification, say markdown-fontify-wiki-links, and a second one which determines whether to check for broken/missing wiki links (only when the former flag is enabled), say markdown-fontify-missing-wiki-links. Checking for and updating missing wiki links (whether target files were created or removed) is the source of the latency you're seeing.

[mrkkrp]

Yes, I update all packages regularly (every day) from MELPA, so my version of markdown-mode is up-to-date. Thanks for the answer, I'm looking forward to having these options to disable wiki link fontification.

[jrblevin]

Please see f8fd642.

[mrkkrp]

Thank you, I will try it.

[nitingupta910]

I'm running markdown-mode (20160722.445) with emacs 24.5.1. There is still good amount of lag even when just moving around with cursor keys or typing anything. Any ways to improve this?

[nitingupta910]

I see the lag was because I was editing a .md file over sshfs. There is no lag when I edit a local .md file. Sorry for false alarm.

[daveliepmann]

I'm on the most recent markdown-mode and still getting significant lag in some markdown-mode buffers when merely typing. Turning off font-lock-mode resolves the issue but this is a suboptimal solution.

[daveliepmann]

Could someone with permission please add the performance tag to this issue?

[jrblevin]

This is a closed issue, so no need for a tag. It will help if you open a new issue with a specific problem narrowed down. Try using the profiler, with a test document.

[daveliepmann]

This is a closed issue, so no need for a tag.

Except that it greatly helps people looking for similar issues to their own, which is the entire point of an issue tracker.

[daveliepmann]

Thanks.

Profiling clearly shows that markdown-mode (specifically markdown-match-code and markdown-match-bold) chokes on lines (copied from command-line output) that have dozens of asterisks in a row.

Moving these multi-asterisk lines into code blocks fixed the massive lag, I assume because markdown-mode no longer tried to calculate nested boldings. Issue resolved for now.

[jrblevin]

Thanks for following up with that. I am planning to add formatted yank commands (e.g., yank as code block) that will make this more efficient.

[libre-man]

I have the same issue for a file with a lot (~140) of GFM toggle boxes in a list. This is the report of the profiler:

- redisplay_internal (C function)                                4528  84%
 - jit-lock-function                                             4500  83%
  - jit-lock-fontify-now                                         4496  83%
   - jit-lock--run-functions                                     4492  83%
    - run-hook-wrapped                                           4492  83%
     - #<compiled 0x2a18acf>                                     4492  83%
      - font-lock-fontify-region                                 4492  83%
       - font-lock-default-fontify-region                        4488  83%
        - font-lock-fontify-keywords-region                      4404  81%
         + markdown-fontify-reference-links                      2068  38%
         + markdown-fontify-inline-links                         1984  36%
         + markdown-fontify-list-items                            140   2%
         + markdown-match-declarative-metadata                     36   0%
         + markdown-fontify-plain-uris                             28   0%
         + markdown-match-pandoc-metadata                          28   0%
         + markdown-match-code                                     16   0%
         + markdown-fontify-blockquotes                            16   0%
         + markdown-match-yaml-metadata-begin                      12   0%
         + markdown-match-includes                                  8   0%
         + markdown-match-yaml-metadata-key                         8   0%
         + markdown-match-inline-attributes                         8   0%
           markdown-match-math-single                               4   0%
         + markdown-fontify-sub-superscripts                        4   0%
         + markdown-match-gfm-open-code-blocks                      4   0%
         + markdown-fontify-angle-uris                              4   0%
         + markdown-match-italic                                    4   0%
        + font-lock-fontify-syntactically-region                   32   0%

[jrblevin]

@libre-man Thanks, the issue with GFM checkboxes should be fixed now (by commit c9955dc).