Code block treatment breaks compatibility with some rehype plugins?

[deklanw]

Was trying to get https://github.com/rsclarke/rehype-shiki to work. It lets you syntax highlight with any VSCode theme. It wasn't working, so I dug into it: it just walks the tree lookiing for pre tags wrapping a code tag.

But, it seems MDsvex is doing something special with @html that breaks this https://github.com/pngwn/MDsveX/blob/master/packages/mdsvex/test/it/code_highlighting.test.ts#L64

I'm just going to use the built-in syntax highlighting for now, but just making a note of this with this issue

[pngwn]

Yeah, this is the cause of #100 as well. Will be fixed in a future version. Highlighting will also be done by Shiki in a future version now that shikijs/shiki#49 is in.

[deklanw]

I actually ended up finding this https://github.com/SaraVieira/prism-theme-converter which lets you use VSCode themes with prism.

Why the switch to Shiki? More language support than prism with that issue you linked to?

[pngwn]

Yeah, you can apply any textmate grammar at all which is better than Prism's non-standard grammar (even though they have some similarities) as you don't need to write a new grammar when you're doing something funky.

The other reason is more selfish, the way Prism is built makes it incredibly difficult to work with in various contexts. It is really quite frustrating and makes a mess of my code due to the UMD format that it publishes. shiki is much friendlier in that regard.

[deklanw]

Makes sense. Thanks for the explanation.

[pngwn]

The rehype issue is still present (code blocks are transformed via a remark plugin and this would require significant refactoring) but highlighjting with shiki is now possible via a custom highlight function as async highlight functions are now supported.

See the parent post in #205 for an example of what this might look like.

[michaeloliverx]

I am trying to use https://github.com/shikijs/twoslash which provides syntax highlighting and enriches the JS and TS code with static types (see https://shikijs.github.io/twoslash/ for demo, its really nice).

It doesn't work because of some escaping issue I think.

Here is the raw markdown:

```tsx twoslash
import axios from "axios";
import { useQuery } from "react-query";

import type { UseQueryResult } from "react-query";

type Todo = {
  userId: number;
  id: number;
  title: string;
  completed: boolean;
};

function useTodos(): UseQueryResult<Todo[], Error> {
  return useQuery({
    queryKey: "groups",
    queryFn: async () => {
      const { data } = await axios.get(
        "https://jsonplaceholder.typicode.com/todos"
      );
      return data;
    },
  });
}

export function Todos() {
  const { data } = useTodos();
  if (!data) return null;
  return <div>{data}</div>;
}
```

I get an error coming from twoslash and it shows the code that it received:

'''tsx
import React from 'react'
// ---cut---
import axios from "axios";
import { useQuery } from "react-query";

import type { UseQueryResult } from "react-query";

type Todo = {
  userId: number;
  id: number;
  title: string;
  completed: boolean;
};

function useTodos(): UseQueryResult<Todo[], Error> {
  return useQuery({
    queryKey: "groups",
    queryFn: async () => {
      const { data } = await axios.get(
        "https://jsonplaceholder.typicode.com/todos"
      );
      return data;
    },
  });
}

export function Todos() {
  const { data } = useTodos();
  if (!data) return null;
  return <div>{data}</div>;
}
'''

We got around this in #212 by providing a custom highlight function and using escapeSvelte, it would be nice to be able to support code block plugins without using the custom function.

I created a barebones svelte kit project with this issue happening here: https://github.com/michael0liver/mdsvex-remark-codeblock-issue

[michaeloliverx]

Okay I got around it again by using the custom highlight function 😄

// svelte.config.js
import preprocess from "svelte-preprocess";
import { mdsvex } from "mdsvex";
import {
  createShikiHighlighter,
  runTwoSlash,
  renderCodeToHTML,
} from "shiki-twoslash";
import { readFileSync } from "fs";

// From https://github.com/whizkydee/vscode-palenight-theme/blob/master/themes/palenight.json
const palenightTheme = JSON.parse(readFileSync("./palenight-theme.json"));

const mdsvexPreprocess = mdsvex({
  extensions: [".svx", ".md"],
  highlight: {
    highlighter: async (code, lang, meta) => {
      const highlighter = await createShikiHighlighter({
        theme: palenightTheme,
      });

      let twoslashResults = null;
      if (meta && meta.includes("twoslash")) {
        twoslashResults = runTwoSlash(code, lang, {});
      }

      const html = renderCodeToHTML(
        code,
        lang,
        meta || [],
        {},
        highlighter,
        twoslashResults
      );

      return `{@html \`${html}\` }`;
    },
  },
});

/** @type {import('@sveltejs/kit').Config} */
const config = {
  preprocess: [mdsvexPreprocess, preprocess()],
  extensions: [".svelte", ".svx"],
  kit: {
    target: "#svelte",
  },
};

export default config;

Working great:

[pngwn]

I think this is the same as : #304 and more specifically this: #93 (comment)