Docstring doctest failures missing if calling makedocs multiple times

[mortenpi]

Not sure what's up, but if you use the workflow where you include("make.jl") multiple times in a REPL session (to avoid recompilation) , then doctest errors that arise from docstrings are only shown the first time you include and are missing in later includes.

julia> include("docs/make.jl")
# doctest failures show up
julia> include("docs/make.jl")
# doctest failures don't show up anymore

[fredrikekre]

This only happens for doctests included in docstrings and not doctests from markdown files.

[mortenpi]

Hmm.. it might be because we modify the Markdown.MD object when we go over the doctest (replace jldoctest as a language with julia). The Markdown AST of the files gets re-parsed when makedocs runs again, but not the docstrings.

[fredrikekre]

Hmm.. it might be because we're modify the Markdown.MD object when we go over the doctest (replace jldoctest as a language with julia)

Yes, on the second run the language is julia-repl.

The Markdown AST of the files gets re-parsed when makedocs runs again, but not the docstrings.

How? Are thosed saved somewhere then?

[mortenpi]

Yeah, somewhere in the global state. I am not entirely sure, but I think each module has a variable that holds all the docstrings of that module. The Docs module in base should hold the answer.

[mortenpi]

We should probably deepcopy the docstrings before we transform them.

[johnnychen94]

Could it at least print some infomation or warning about this?