Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade pandoc to 2.14 and fix plugin breaks #427

Merged
merged 11 commits into from
Jun 10, 2021
25 changes: 17 additions & 8 deletions USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,14 +133,23 @@ Note that multiple citations must be semicolon separated.
Be careful not to cite the same study using identifiers from multiple sources.
For example, the following citations all refer to the same study, but will be treated as separate references: `[@doi:10.7717/peerj.705; @pmc:PMC4304851; @pubmed:25648772]`.

Citation keys must adhere to the syntax described in the [Pandoc manual](https://pandoc.org/MANUAL.html#citations):

> The citation key must begin with a letter, digit, or `_`, and may contain alphanumerics, `_`, and internal punctuation characters (`:.#$%&-+?<>~/`).

To evaluate whether a citation key fully matches this syntax, try [this online regex](https://regex101.com/r/mXZyY2/latest).
If the citation key is not valid, use the [citation aliases](#citation-aliases) workaround below.
This is required for citation keys that contain forbidden characters such as `;` or `=` or end with a non-alphanumeric character such as `/`.
<!-- See [jgm/pandoc#6026](https://github.com/jgm/pandoc/issues/6026) for progress on a more flexible Markdown citation key syntax. -->
The citation key syntax is described in the [Pandoc manual](https://pandoc.org/MANUAL.html#citation-syntax):

> Unless a citation key start with a letter, digit, or `_`,
> and contains only alphanumerics and internal punctuation characters (`:.#$%&-+?<>~/`),
> it must be surrounded by curly braces,
> which are not considered part of the key.
> In `@Foo_bar.baz.`, the key is `Foo_bar.baz`.
> The final period is not *internal* punctuation,
> so it is not included in the key.
> In `@{Foo_bar.baz.}`, the key is `Foo_bar.baz.`, including the final period.
> The curly braces are recommended if you use URLs as keys:
> `[@{https://example.com/bib?name=foobar&date=2000}, p. 33]`.

If a citation key does not fully match [this online regex](https://regex101.com/r/mXZyY2/latest)
(for example, contains characters such as `;` or `=` or end with a non-alphanumeric character such as `/`),
make sure to surround it with curly braces or use the [citation aliases](#citation-aliases) workaround below.
<!-- See [jgm/pandoc#6026](https://github.com/jgm/pandoc/issues/6026) on the curly-brace syntax for Markdown citation keys, which was introduced in Pandoc 2.14. -->

Prior to Rootstock commit [`6636b91`](https://github.com/manubot/rootstock/commit/6636b912c6b41593acd2041d34cd4158c1b317fb) on 2020-01-14, Manubot processed citations separately from Pandoc.
Switching to a Pandoc filter improved reliability on complex documents, but restricted the syntax of citation keys slightly.
Expand Down
2 changes: 1 addition & 1 deletion build/assets/style.csl
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
<text variable="citation-number"/>
</layout>
</citation>
<bibliography hanging-indent="true" second-field-align="flush">
<bibliography second-field-align="flush">
<layout>
<text variable="citation-number" suffix=". "/>
<group delimiter=" ">
Expand Down
16 changes: 8 additions & 8 deletions build/environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,10 @@ dependencies:
- cairo=1.14.12
- cairocffi=0.8.0
- cffi=1.12.3
- ghp-import=0.5.5
- ghp-import=1.0.0
- jinja2=2.11.2
- jsonschema=3.2.0
- pandoc=2.9.2
- panflute=1.12.5
- pandoc=2.14
- pango=1.40.14
- pip=20.2
- psutil=5.7.3
Expand All @@ -21,14 +20,15 @@ dependencies:
- yamllint=1.25.0
- pip:
- errorhandler==2.0.1
- git+https://github.com/manubot/manubot@553090a16ac24a9fdc3f8df241a91b790c140865
- git+https://github.com/manubot/manubot@2c028a7a87be24a76e2dec3ac7db6e8754ae6d4a
- isbnlib==3.10.3
- opentimestamps-client==0.7.0
- opentimestamps==0.4.1
- pandoc-eqnos==2.3.0
- pandoc-fignos==2.3.1
- pandoc-tablenos==2.2.2
- pandoc-xnos==2.4.3
- pandoc-eqnos==2.5.0
- pandoc-fignos==2.4.0
- pandoc-tablenos==2.3.0
- pandoc-xnos==2.5.0
- panflute==2.1.0
- pybase62==0.4.3
- pysha3==1.0.2
- python-bitcoinlib==0.10.2
Expand Down
2 changes: 1 addition & 1 deletion build/pandoc/defaults/common.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ filters:
- pandoc-eqnos
- pandoc-tablenos
- pandoc-manubot-cite
- pandoc-citeproc
- citeproc
wrap: preserve
metadata:
csl: build/assets/style.csl
Expand Down
1 change: 1 addition & 0 deletions build/pandoc/defaults/html.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ include-after-body:
- build/plugins/mathjax.html
#- build/plugins/scite.html
variables:
document-css: false
math: ""
html-math-method:
method: mathjax
3 changes: 2 additions & 1 deletion build/plugins/anchors.html
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@
<script type="module">
// which types of elements to add anchors next to, in "document.querySelector"
// format
const typesQuery = 'h1, h2, h3, [id^="fig:"], [id^="tbl:"], [id^="eq:"]';
const typesQuery =
'h1, h2, h3, div[id^="fig:"], div[id^="tbl:"], span[id^="eq:"]';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is div[id^="fig:"]the workaround for double table ids (on the div and table) mentioned at #425 (comment)? It works by only matching div ids that start with tbl: as opposed to all elements?

Just to confirm, it is okay for us to proceed despite the duplicate ids, because this PR handles it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is div[id^="fig:"]the workaround for double table ids (on the div and table) mentioned at #425 (comment)? It works by only matching div ids that start with tbl: as opposed to all elements?

That's correct. I've essentially made the queries more specific. As long as pandoc/table-nos is consistent and never outputs a table without the div wrapper, this should be fine.

Looked at the issue, 👍 FWIW id's being unique isn't just a suggestion, it's a requirement of the spec and can cause unexpected behavior. It shouldn't cause anything to crash though.

Copy link
Member

@dhimmel dhimmel Jun 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it.

As long as pandoc/table-nos is consistent and never outputs a table without the div wrapper, this should be fine.

Based on the early discussion in the issue, it seems like the most likely fix is that pandoc-tablenos stops adding the div wrapper for tables since the id is now set in <table>.

So if we upgrade pandoc-tablenos, we might have to tweak this, but that's okay. Let's wait to see what solution pandoc-tablenos adopts and merge this now.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh if that’s the case we’ll need to re-add the plug-in to wrap tables in divs for scrolling purposes 😞


// start script
function start() {
Expand Down
10 changes: 6 additions & 4 deletions build/plugins/jump-to-first.html
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,11 @@

// start script
function start() {
if (references !== "false") makeButtons("ref-", "p", "reference");
if (figures !== "false") makeButtons("fig:", "figcaption", "figure");
if (tables !== "false") makeButtons("tbl:", "caption", "table");
if (references !== "false")
makeButtons(`div[id^="ref-"]`, ".csl-left-margin", "reference");
if (figures !== "false")
makeButtons(`div[id^="fig:"]`, "figcaption", "figure");
if (tables !== "false") makeButtons(`div[id^="tbl:"]`, "caption", "table");
}

// when jump button clicked
Expand Down Expand Up @@ -45,7 +47,7 @@

// add button next to each reference entry, figure, or table
function makeButtons(query, containerQuery, subject) {
const elements = document.querySelectorAll(`[id^="${query}"`);
const elements = document.querySelectorAll(query);
for (const element of elements) {
const id = element.id;
const buttonContainer = element.querySelector(containerQuery);
Expand Down
8 changes: 4 additions & 4 deletions build/plugins/tooltips.html
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,7 @@
"data-selected",
"data-highlighted",
"data-glow",
"class",
];
for (const attribute of attributes) {
sourceCopy.removeAttribute(attribute);
Expand Down Expand Up @@ -290,9 +291,8 @@
prevButton.classList.add("icon_button");
nextButton.classList.add("icon_button");
prevButton.innerHTML = document.querySelector(".icon_caret_left").innerHTML;
nextButton.innerHTML = document.querySelector(
".icon_caret_right"
).innerHTML;
nextButton.innerHTML =
document.querySelector(".icon_caret_right").innerHTML;
navBar.appendChild(prevButton);
navBar.appendChild(document.createTextNode(text));
navBar.appendChild(nextButton);
Expand Down Expand Up @@ -336,7 +336,7 @@
if (!target) return;

// if ref or figure, modify target to get expected element
if (id.indexOf("ref-") === 0) target = target.querySelector("p");
if (id.indexOf("ref-") === 0) target = target.querySelector(":nth-child(2)");
else if (id.indexOf("fig:") === 0) target = target.querySelector("figure");

return target;
Expand Down
13 changes: 10 additions & 3 deletions build/themes/default.html
Original file line number Diff line number Diff line change
Expand Up @@ -267,8 +267,7 @@
}

/* table auto-number */
table > caption > span:first-of-type,
div.table_wrapper > table > caption > span:first-of-type {
table > caption > span:first-of-type {
font-weight: bold;
margin-right: 5px;
}
Expand Down Expand Up @@ -495,6 +494,15 @@
top: 0.125em;
}

/* -------------------------------------------------- */
/* references */
/* -------------------------------------------------- */

.csl-entry {
margin-top: 15px;
margin-bottom: 15px;
}

/* -------------------------------------------------- */
/* print control */
/* -------------------------------------------------- */
Expand Down Expand Up @@ -626,7 +634,6 @@

/* tablenos wrapper */
.tablenos {
/* show scrollbar on tables if necessary to prevent overflow */
width: 100%;
margin: 20px 0;
}
Expand Down
2 changes: 1 addition & 1 deletion content/02.delete-me.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ Citation by Wikidata ID [@wikidata:Q56458321].

Citation by ISBN [@isbn:9780262517638].

Citation by URL [@https://greenelab.github.io/meta-review/].
Citation by URL [@{https://greenelab.github.io/meta-review/}].

Citation by alias [@deep-review].

Expand Down