code blocks not passed to remark plugin

[shedali]

Was just testing a few remark plugins and found the AST passed to the remark plugin for graphviz /mermaid for example was not of the expected 'code' type

Vanilla js
{type: 'code'}

Through mdsvex
{type: 'html'}

Is there a way to pass just the original code block through to the plugin?

[pngwn]

Ah, at that point they've been transformed, we can probably apply the highlighting transformations after user provided plugins have run. I'll take a look at this.

[pngwn]

This is difficult, the graphviz and mermaid plugins add options directly to the vFile which mdsvex doesn't provide any direct access to, you would never be able to modify the destination directory of the SVG plugins which may be problematic depending on the usecase.

[pngwn]

I have a branch with the problem fixed, but I can't get mermaid-remark to run properly either way. It wants options to be passed via the vFile directly which just isn't remotely possible.

I'll make the change shortly but I don't know if those plugins will work.

[shedali]

I have a branch with the problem fixed,

nice one 🚀

the graphviz and mermaid plugins add options directly to the vFile which mdsvex doesn't provide any direct access to

the vFile data passing felt odd; but it appears to be the recommended way of passing data about for unifedjs plugins https://github.com/vfile/vfile#vfiledata.

It is often necessary to set destinationDir to 'static' or 'public' etc and also specify the resulting url path.

[pngwn]

That makes sense in the context of unified but less so for mdsvex. unified is essentially an implementation detail here and I don't particularly want to bind mdsvex to that. The only contract that mdsvex currently has with any external ecosystem is the remark and rehype plugin interface and the AST formats that are used for markdown and HTML.

[shedali]

Yes best to keep it simple. I have setup a pre-process step to do the necessary prior to mdsvex. Thanks for your work.

[pngwn]

This never really got fixed. At least the core issue of mdsvex transforming code nodes before a user-provided plugin could get to them.

[pngwn]

Now it is.

[michaeloliverx]

I have tried this but I cannot get the remark plugin to run, I have been trying to use https://shikijs.github.io/twoslash/ which offers a remark plugin, my config looks like this:

import { mdsvex } from "mdsvex";

import remarkShikiTwoslash from "remark-shiki-twoslash";

const MDSVEX_EXTENSIONS = [".svx"];

const mdsvexPreprocess = mdsvex({
  extensions: MDSVEX_EXTENSIONS,
  layout: {
    blog: "/src/layouts/blog.svelte",
  },
  remarkPlugins: [[remarkShikiTwoslash, { theme: "github-dark" }]],
});

export { mdsvexPreprocess, MDSVEX_EXTENSIONS };

But my outputted HTML looks to be using the built in highlighter:

<pre class="language-python"><code class="language-python"><span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"Hello, World"</span><span class="token punctuation">)</span></code></pre>

Any ideas?

BTW I am using svelte kit.

[michaeloliverx]

I got remark-shiki-twoslash to run, it was because of some common js / default export issue. Now I have created a mdsvex.config.cjs file with the following:

const remarkShikiTwoslash = require("remark-shiki-twoslash").default;

const config = {
  extensions: [".svx", ".md"],
  remarkPlugins: [[remarkShikiTwoslash, { theme: "github-dark" }]],
  highlight: false,
};

module.exports = config;

The plugin is ran but there is an escaping issue happening.

Raw markdown

```tsx
import axios from "axios";
import { useQuery } from "react-query";

import type { UseQueryResult } from "react-query";

type Todo = {
  userId: number;
  id: number;
  title: string;
  completed: boolean;
};

function useTodos(): UseQueryResult<Todo[], Error> {
  return useQuery({
    queryKey: "groups",
    queryFn: async () => {
      const { data } = await axios.get(
        "https://jsonplaceholder.typicode.com/todos"
      );
      return data;
    },
  });
}

export function Todos() {
  const { data } = useTodos();
  if (!data) return null;
  return <div>{data}</div>;
}
```

Output:

Related to: #212 and #139

[pngwn]

I still don't know why the whitespace thing sometimes happens, it feels really strange. I haven't come up with a good isolate repro or test case for that yet.

Now, I have an idea why the HTML entity thing here is happening. It will have something to do with the fact that two code transformers are running on the same block. I'm guessing the first one also transforms < and > to entities, then when the highlighter sees those characters, if highlights them individually instead of as an 'entity.

The way the default highlighter works here is it highlights the raw {, }, <, > and then transforms them. so < -> <span class="thing"><</span> -> <span class="thing">&lt;</span>.

In this case we are probably seeing: < -> <span class="thing"><</span> -> <span class="thing">&lt;</span> -> something_messed_up.

I'm not sure what the best way to resolve this is, I'm starting to think that custom code transformations are going to require the user to also highlight them. I could package up the highlighter separately to make this easier to compose with your custom plugin, which will also be less error prone.

Almost all of the mechanics of this will be changing in v1 but I'll open a new issue to track this so I don't lose sight of the use case.