Refactor date detection in lexer

[moorereason]

Describe the Issue

As of 5c94d86, 18-20% of memory consumption is occurring inside of the dateRegexp matching in lexRvalue when unmarshalling the benchmark.toml doc. There are very few dates in the TOML document.

sh$ go tool pprof mytest mem.pprof
(pprof) top1
Showing nodes accounting for 36.09kB, 17.95% of 201.03kB total
Dropped 81 nodes (cum <= 1kB)
Showing top 1 nodes out of 95
      flat  flat%   sum%        cum   cum%
   36.09kB 17.95% 17.95%    36.09kB 17.95%  regexp.(*bitState).reset

To Reproduce

Run the unmarshal happening in BenchmarkUnmarshalToml once with pprof profiling enabled. Here's the jist of it:

package main

import (
        "io/ioutil"
        "log"
        "time"

        "github.com/pelletier/go-toml"
        "github.com/pkg/profile"
)

func main() {
        defer profile.Start(
                profile.MemProfile,
                profile.MemProfileRate(1),
                profile.ProfilePath(".")).Stop()

        benchmarkUnmarshalToml()
}

func benchmarkUnmarshalToml() {
        bytes, err := ioutil.ReadFile("benchmark.toml")
        if err != nil {
                log.Fatal(err)
        }

        target := benchmarkDoc{}
        err = toml.Unmarshal(bytes, &target)
        if err != nil {
                log.Fatal(err)
        }
}

type benchmarkDoc struct {
// ... see benchmark/benchmark_test.go

Expected Behavior

No regular expressions in the lexer. Better memory performance.

Versions

• go-toml 5c94d86
• go version go1.15.2 linux/amd64

Additional Context

Part of the issue may be that the regexp is executed every time a number or date is expected.

Related to #167

[pelletier]

Totally right, that needs to go. I was lazy when I put that in back then, now it's biting us in the back. Sorry 😬

[moorereason]

As an additional side-effect, factoring out the regexp would get rid of init() usage in the lexer, which is generally frowned upon for libraries. The same issue applies to numberUnderscoreInvalidRegexp and hexNumberUnderscoreInvalidRegexp in the parser.

[pelletier]

With the two patches:

benchmark                    old ns/op     new ns/op     delta
BenchmarkUnmarshalToml-8     297364        257032        -13.56%

benchmark                    old allocs     new allocs     delta
BenchmarkUnmarshalToml-8     2746           2650           -3.50%

benchmark                    old bytes     new bytes     delta
BenchmarkUnmarshalToml-8     133359        127030        -4.75%