Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encode: support json.Number type #923

Merged
merged 8 commits into from
Jan 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 53 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,9 +98,9 @@ Given the following struct, let's see how to read it and write it as TOML:

```go
type MyConfig struct {
Version int
Name string
Tags []string
Version int
Name string
Tags []string
}
```

Expand All @@ -119,7 +119,7 @@ tags = ["go", "toml"]
var cfg MyConfig
err := toml.Unmarshal([]byte(doc), &cfg)
if err != nil {
panic(err)
panic(err)
}
fmt.Println("version:", cfg.Version)
fmt.Println("name:", cfg.Name)
Expand All @@ -140,14 +140,14 @@ as a TOML document:

```go
cfg := MyConfig{
Version: 2,
Name: "go-toml",
Tags: []string{"go", "toml"},
Version: 2,
Name: "go-toml",
Tags: []string{"go", "toml"},
}

b, err := toml.Marshal(cfg)
if err != nil {
panic(err)
panic(err)
}
fmt.Println(string(b))

Expand Down Expand Up @@ -175,40 +175,40 @@ the AST level. See https://pkg.go.dev/github.com/pelletier/go-toml/v2/unstable.
Execution time speedup compared to other Go TOML libraries:

<table>
<thead>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead>
<tbody>
<tr><td>Marshal/HugoFrontMatter-2</td><td>1.9x</td><td>2.2x</td></tr>
<tr><td>Marshal/ReferenceFile/map-2</td><td>1.7x</td><td>2.1x</td></tr>
<tr><td>Marshal/ReferenceFile/struct-2</td><td>2.2x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/HugoFrontMatter-2</td><td>2.9x</td><td>2.7x</td></tr>
<tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.6x</td><td>2.7x</td></tr>
<tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.6x</td><td>5.1x</td></tr>
</tbody>
<thead>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead>
<tbody>
<tr><td>Marshal/HugoFrontMatter-2</td><td>1.9x</td><td>2.2x</td></tr>
<tr><td>Marshal/ReferenceFile/map-2</td><td>1.7x</td><td>2.1x</td></tr>
<tr><td>Marshal/ReferenceFile/struct-2</td><td>2.2x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/HugoFrontMatter-2</td><td>2.9x</td><td>2.7x</td></tr>
<tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.6x</td><td>2.7x</td></tr>
<tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.6x</td><td>5.1x</td></tr>
</tbody>
</table>
<details><summary>See more</summary>
<p>The table above has the results of the most common use-cases. The table below
contains the results of all benchmarks, including unrealistic ones. It is
provided for completeness.</p>

<table>
<thead>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead>
<tbody>
<tr><td>Marshal/SimpleDocument/map-2</td><td>1.8x</td><td>2.7x</td></tr>
<tr><td>Marshal/SimpleDocument/struct-2</td><td>2.7x</td><td>3.8x</td></tr>
<tr><td>Unmarshal/SimpleDocument/map-2</td><td>3.8x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/SimpleDocument/struct-2</td><td>5.6x</td><td>4.1x</td></tr>
<tr><td>UnmarshalDataset/example-2</td><td>3.0x</td><td>3.2x</td></tr>
<tr><td>UnmarshalDataset/code-2</td><td>2.3x</td><td>2.9x</td></tr>
<tr><td>UnmarshalDataset/twitter-2</td><td>2.6x</td><td>2.7x</td></tr>
<tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.2x</td><td>2.3x</td></tr>
<tr><td>UnmarshalDataset/canada-2</td><td>1.8x</td><td>1.5x</td></tr>
<tr><td>UnmarshalDataset/config-2</td><td>4.1x</td><td>2.9x</td></tr>
<tr><td>geomean</td><td>2.7x</td><td>2.8x</td></tr>
</tbody>
<thead>
<tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>
</thead>
<tbody>
<tr><td>Marshal/SimpleDocument/map-2</td><td>1.8x</td><td>2.7x</td></tr>
<tr><td>Marshal/SimpleDocument/struct-2</td><td>2.7x</td><td>3.8x</td></tr>
<tr><td>Unmarshal/SimpleDocument/map-2</td><td>3.8x</td><td>3.0x</td></tr>
<tr><td>Unmarshal/SimpleDocument/struct-2</td><td>5.6x</td><td>4.1x</td></tr>
<tr><td>UnmarshalDataset/example-2</td><td>3.0x</td><td>3.2x</td></tr>
<tr><td>UnmarshalDataset/code-2</td><td>2.3x</td><td>2.9x</td></tr>
<tr><td>UnmarshalDataset/twitter-2</td><td>2.6x</td><td>2.7x</td></tr>
<tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.2x</td><td>2.3x</td></tr>
<tr><td>UnmarshalDataset/canada-2</td><td>1.8x</td><td>1.5x</td></tr>
<tr><td>UnmarshalDataset/config-2</td><td>4.1x</td><td>2.9x</td></tr>
<tr><td>geomean</td><td>2.7x</td><td>2.8x</td></tr>
</tbody>
</table>
<p>This table can be generated with <code>./ci.sh benchmark -a -html</code>.</p>
</details>
Expand All @@ -233,24 +233,24 @@ Go-toml provides three handy command line tools:

* `tomljson`: Reads a TOML file and outputs its JSON representation.

```
$ go install github.com/pelletier/go-toml/v2/cmd/tomljson@latest
$ tomljson --help
```
```
$ go install github.com/pelletier/go-toml/v2/cmd/tomljson@latest
$ tomljson --help
```

* `jsontoml`: Reads a JSON file and outputs a TOML representation.

```
$ go install github.com/pelletier/go-toml/v2/cmd/jsontoml@latest
$ jsontoml --help
```
```
$ go install github.com/pelletier/go-toml/v2/cmd/jsontoml@latest
$ jsontoml --help
```

* `tomll`: Lints and reformats a TOML file.

```
$ go install github.com/pelletier/go-toml/v2/cmd/tomll@latest
$ tomll --help
```
```
$ go install github.com/pelletier/go-toml/v2/cmd/tomll@latest
$ tomll --help
```

### Docker image

Expand Down Expand Up @@ -301,7 +301,7 @@ type doc struct {

d := doc{
A: inner{
B: "Before",
B: "Before",
},
}

Expand Down Expand Up @@ -565,10 +565,11 @@ complete solutions exist out there.

## Versioning

Go-toml follows [Semantic Versioning](https://semver.org). The supported version
of [TOML](https://github.com/toml-lang/toml) is indicated at the beginning of
this document. The last two major versions of Go are supported
(see [Go Release Policy](https://golang.org/doc/devel/release.html#policy)).
Expect for parts explicitely marked otherwise, go-toml follows [Semantic
Versioning](https://semver.org). The supported version of
[TOML](https://github.com/toml-lang/toml) is indicated at the beginning of this
document. The last two major versions of Go are supported (see [Go Release
Policy](https://golang.org/doc/devel/release.html#policy)).

## License

Expand Down
13 changes: 12 additions & 1 deletion cmd/jsontoml/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package main

import (
"encoding/json"
"flag"
"io"

"github.com/pelletier/go-toml/v2"
Expand All @@ -33,7 +34,11 @@ Reading from a file:
jsontoml file.json > file.toml
`

var useJsonNumber bool

func main() {
flag.BoolVar(&useJsonNumber, "use-json-number", false, "unmarshal numbers into `json.Number` type instead of as `float64`")

p := cli.Program{
Usage: usage,
Fn: convert,
Expand All @@ -45,11 +50,17 @@ func convert(r io.Reader, w io.Writer) error {
var v interface{}

d := json.NewDecoder(r)
e := toml.NewEncoder(w)

if useJsonNumber {
d.UseNumber()
e.SetMarshalJsonNumbers(true)
}

err := d.Decode(&v)
if err != nil {
return err
}

e := toml.NewEncoder(w)
return e.Encode(v)
}
23 changes: 19 additions & 4 deletions cmd/jsontoml/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,11 @@ import (

func TestConvert(t *testing.T) {
examples := []struct {
name string
input string
expected string
errors bool
name string
input string
expected string
errors bool
useJsonNumber bool
}{
{
name: "valid json",
Expand All @@ -26,6 +27,19 @@ func TestConvert(t *testing.T) {
}`,
expected: `[mytoml]
a = 42.0
`,
},
{
name: "use json number",
useJsonNumber: true,
input: `
{
"mytoml": {
"a": 42
}
}`,
expected: `[mytoml]
a = 42
`,
},
{
Expand All @@ -37,6 +51,7 @@ a = 42.0

for _, e := range examples {
b := new(bytes.Buffer)
useJsonNumber = e.useJsonNumber
err := convert(strings.NewReader(e.input), b)
if e.errors {
require.Error(t, err)
Expand Down
33 changes: 29 additions & 4 deletions marshaler.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package toml
import (
"bytes"
"encoding"
"encoding/json"
"fmt"
"io"
"math"
Expand Down Expand Up @@ -37,10 +38,11 @@ type Encoder struct {
w io.Writer

// global settings
tablesInline bool
arraysMultiline bool
indentSymbol string
indentTables bool
tablesInline bool
arraysMultiline bool
indentSymbol string
indentTables bool
marshalJsonNumbers bool
}

// NewEncoder returns a new Encoder that writes to w.
Expand Down Expand Up @@ -87,6 +89,17 @@ func (enc *Encoder) SetIndentTables(indent bool) *Encoder {
return enc
}

// SetMarshalJsonNumbers forces the encoder to serialize `json.Number` as a
// float or integer instead of relying on TextMarshaler to emit a string.
//
// *Unstable:* This method does not follow the compatiblity guarnatees of
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small typo: s/guarnatees/guarantees/

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦🏻‍♂️ good catch, thank you #927

// semver. It can be changed or removed without a new major version being
// issued.
func (enc *Encoder) SetMarshalJsonNumbers(indent bool) *Encoder {
enc.marshalJsonNumbers = indent
return enc
}

// Encode writes a TOML representation of v to the stream.
//
// If v cannot be represented to TOML it returns an error.
Expand Down Expand Up @@ -252,6 +265,18 @@ func (enc *Encoder) encode(b []byte, ctx encoderCtx, v reflect.Value) ([]byte, e
return append(b, x.String()...), nil
case LocalDateTime:
return append(b, x.String()...), nil
case json.Number:
if enc.marshalJsonNumbers {
if x == "" { /// Useful zero value.
return append(b, "0"...), nil
} else if v, err := x.Int64(); err == nil {
return enc.encode(b, ctx, reflect.ValueOf(v))
} else if f, err := x.Float64(); err == nil {
return enc.encode(b, ctx, reflect.ValueOf(f))
} else {
return nil, fmt.Errorf("toml: unable to convert %q to int64 or float64", x)
}
}
}

hasTextMarshaler := v.Type().Implements(textMarshalerType)
Expand Down
23 changes: 23 additions & 0 deletions marshaler_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -948,6 +948,29 @@ func TestEncoderSetIndentSymbol(t *testing.T) {
assert.Equal(t, expected, w.String())
}

func TestEncoderSetMarshalJsonNumbers(t *testing.T) {
var w strings.Builder
enc := toml.NewEncoder(&w)
enc.SetMarshalJsonNumbers(true)
err := enc.Encode(map[string]interface{}{
"A": json.Number("1.1"),
"B": json.Number("42e-3"),
"C": json.Number("42"),
"D": json.Number("0"),
"E": json.Number("0.0"),
"F": json.Number(""),
})
require.NoError(t, err)
expected := `A = 1.1
B = 0.042
C = 42
D = 0
E = 0.0
F = 0
`
assert.Equal(t, expected, w.String())
}

func TestEncoderOmitempty(t *testing.T) {
type doc struct {
String string `toml:",omitempty,multiline"`
Expand Down
Loading