Skip to content

Fix issue with EOF parsing

Pre-release
Pre-release
Compare
Choose a tag to compare
@element-of-surprise element-of-surprise released this 14 Aug 18:23

Seems like I had a bug with parsing when we didn't have a blank line when we reached EOF.

That didn't seem to be much of a problem when using halfpike to lex/parse something like a router config, where you are sweeping over a lot of details. But when using it to parse an IDL language where every character is important, this was a problem.

So, we'd have something like this:

{
  a: "b"
}```

The last line would emit Item{Type: ItemEOF, Val: "}"}, when it should emit two values, one of ItemText and one of ItemEOF.

Also, I was doing some weird stuff with EOF.  There is no EOF character in UTF-8, we simply know how large the file is.  Because the lexer is run based, I was emitting a character to signal EOF. That character was not good, so I'm using the control character now of '\x01'.  But I was also emitting that character in Raw in some places. I've stopped doing that, as that isn't real.