Use a flat layout for TokenBuffer

[CAD97]

This looks like a small perf win:

flat-tokenbuffer

test baseline            ... bench:     751,638 ns/iter (+/- 37,662)   [0.714ms..0.789ms]
test create_token_buffer ... bench:   2,624,105 ns/iter (+/- 218,503)  [2.416ms..2.843ms]
test parse_file          ... bench:   6,546,490 ns/iter (+/- 285,880)  [6.261ms..6.832ms]

main

test baseline            ... bench:     766,913 ns/iter (+/- 35,923)   [0.731ms..0.802ms]
test create_token_buffer ... bench:   2,888,407 ns/iter (+/- 167,829)  [2.721ms..3.056ms]
test parse_file          ... bench:   6,737,680 ns/iter (+/- 492,060)  [6.316ms..7.160ms]

Replaces #1196. With a flat tokenbuffer, implementing Ord for Cursor will become trivial.

I did try a bit to make Cursor store &[Entry] directly, but I was having trouble doing so as part of this change. It might still be a worthwhile followup to reduce the necessary amount of unsafe, but doesn't need to be done immediately.

The biggest code quality question is probably how I handled changing bump.

[dtolnay]

Nice! Thank you.

[dtolnay]

Trying to figure out the purpose of this. Is this to support code that did fn f<T: Drop> and called f::<TokenBuffer>? If so, that's "you brought this upon yourself" territory and I wouldn't care about breaking it. Is there any plausible practical code that would need this impl to exist?

Or is it the reverse concern, that there might be code that breaks if we later need to add back a Drop impl?

[CAD97]

It's paranoia around letter-of-law and the former — I don't think there's any other impact, since there aren't any generics to capture potentially late bound lifetimes in.