Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(parsers.binary): Handle hex-encoded inputs #12232

Merged
merged 4 commits into from
Nov 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 17 additions & 8 deletions plugins/parsers/binary/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,22 @@ user-specified configurations.
## Do not error-out if none of the filter expressions below matches.
# allow_no_match = false

## Specify the endianess of the data.
## Specify the endianness of the data.
## Available values are "be" (big-endian), "le" (little-endian) and "host",
## where "host" means the same endianess as the machine running Telegraf.
## where "host" means the same endianness as the machine running Telegraf.
# endianess = "host"

## Interpret input as string containing hex-encoded data.
# hex_encoding = false

## Multiple parsing sections are allowed
[[inputs.file.binary]]
## Optional: Metric (measurement) name to use if not extracted from the data.
# metric_name = "my_name"

## Definition of the message format and the extracted data.
## Please note that you need to define all elements of the data in the
## correct order with the correct length as the data is parsed in the order
## correct order with the correct length as the data is parsed in the order
## given.
## An entry can have the following properties:
## name -- Name of the element (e.g. field or tag). Can be omitted
Expand Down Expand Up @@ -59,7 +62,7 @@ user-specified configurations.
{ name = "address", type = "uint16", assignment = "tag" },
{ name = "value", type = "float64" },
{ type = "unix", assignment = "time" },
]
]

## Optional: Filter evaluated before applying the configuration.
## This option can be used to mange multiple configuration specific for
Expand Down Expand Up @@ -101,14 +104,20 @@ By specifying `allow_no_match` you allow the parser to silently ignore data
that does not match _any_ given configuration filter. This can be useful if
you only want to collect a subset of the available messages.

#### `endianess` (optional)
#### `endianness` (optional)

This specifies the endianess of the data. If not specified, the parser will
fallback to the "host" endianess, assuming that the message and Telegraf
machine share the same endianess.
This specifies the endianness of the data. If not specified, the parser will
fallback to the "host" endianness, assuming that the message and Telegraf
machine share the same endianness.
Alternatively, you can explicitly specify big-endian format (`"be"`) or
little-endian format (`"le"`).

#### `hex_encoding` (optional)

If `true`, the input data is interpreted as a string containing hex-encoded
data like `C0 C7 21 A9`. The value is _case insensitive_ and can handle spaces,
however prefixes like `0x` or `x` are _not_ allowed.

### Non-byte aligned value extraction

In both, `filter` and `entries` definitions, values can be extracted at non-byte
Expand Down
17 changes: 16 additions & 1 deletion plugins/parsers/binary/parser.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@ package binary

import (
"encoding/binary"
"encoding/hex"
"errors"
"fmt"
"strings"
"time"

"github.com/influxdata/telegraf"
Expand All @@ -14,6 +16,7 @@ type Parser struct {
AllowNoMatch bool `toml:"allow_no_match"`
Endianess string `toml:"endianess"`
Configs []Config `toml:"binary"`
HexEncoding bool `toml:"hex_encoding"`
Log telegraf.Logger `toml:"-"`

metricName string
Expand Down Expand Up @@ -47,9 +50,21 @@ func (p *Parser) Init() error {
return nil
}

func (p *Parser) Parse(buf []byte) ([]telegraf.Metric, error) {
func (p *Parser) Parse(data []byte) ([]telegraf.Metric, error) {
t := time.Now()

// If the data is encoded in HEX, we need to decode it first
buf := data
if p.HexEncoding {
s := strings.ReplaceAll(string(data), " ", "")
s = strings.ReplaceAll(s, "\t", "")
var err error
buf, err = hex.DecodeString(s)
if err != nil {
return nil, fmt.Errorf("decoding hex failed: %w", err)
}
}

matches := 0
metrics := make([]telegraf.Metric, 0)
for i, cfg := range p.Configs {
Expand Down
33 changes: 33 additions & 0 deletions plugins/parsers/binary/parser_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package binary
import (
"bytes"
"encoding/binary"
"encoding/hex"
"fmt"
"os"
"path/filepath"
Expand Down Expand Up @@ -1465,3 +1466,35 @@ func TestCases(t *testing.T) {
})
}
}

func TestHexEncoding(t *testing.T) {
testdata := []interface{}{
uint64(0x01020304050607),
uint64(0x08090A0B0C0D0E),
uint64(0x0F101213141516),
uint64(0x1718191A1B1C1D),
uint64(0x1E1F2021222324),
}

parser := &Parser{
Endianess: "be",
HexEncoding: true,
Configs: []Config{
{
Entries: []Entry{dummyEntry},
},
},
Log: testutil.Logger{Name: "parsers.binary"},
metricName: "binary",
}
require.NoError(t, parser.Init())

// Generate the binary data and encode it to HEX
data, err := generateBinary(testdata, binary.BigEndian)
require.NoError(t, err)
encoded := hex.EncodeToString(data)

metrics, err := parser.Parse([]byte(encoded))
require.NoError(t, err)
require.NotEmpty(t, metrics)
}