Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

highlite: fix #17890 - tokenize Nim escape seq-s #17919

Merged
merged 2 commits into from
May 3, 2021

Conversation

a-mr
Copy link
Contributor

@a-mr a-mr commented May 1, 2021

Before:

image

After:

image

Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
case g.buf[pos]
of '\\':
g.kind = gtEscapeSequence
of 'x', 'X':
Copy link
Member

@timotheecour timotheecour May 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

proc main*()=
  ## \x123
  ## `\x123`
  runnableExamples:
    echo "abc\x1234"
    echo "abc\x1"
    echo "abc\uabcd3" # bug with this one
    echo "abc\u{12345}def" # and this one
    echo "abc\e"
main()

is there possibility to reuse code (possibly via a lib/std/private/lexerutils) to avoid duplication with what compiler already does?

(a plausible alternative is to not highlight things inside string literals, but see also #17722 which would specify the language with which to highlight)

supporting \u etc can also be defered to future work, since it's more rare

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just enabled the existing code by moving case of first \ into surrounding if — Github obviously shows more changes than has been really done.

I can hardly guess why highlite.nim was written with its own separate lexer.

@Araq ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lexer: Transforms "\n" into \10. Skips comments and whitespace (more or less).
Highlighter: Highlights "\n". Highlights comments and whitespace.

Different code for different things.

@Araq Araq merged commit 287f117 into nim-lang:devel May 3, 2021
PMunch pushed a commit to PMunch/Nim that referenced this pull request Mar 28, 2022
)

* highlite: fix nim-lang#17890 - tokenize Nim escape seq-s

* Update tests/stdlib/thighlite.nim

Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>

Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants