Skip to content

Commit

Permalink
[processor/transform] Added parameter validation for truncate_all and…
Browse files Browse the repository at this point in the history
… limit (#9783)

* Added parameter validation for truncate_all and limit

* updated changelog
  • Loading branch information
TylerHelmuth authored May 6, 2022
1 parent 1ff279a commit fe12a90
Show file tree
Hide file tree
Showing 5 changed files with 52 additions and 37 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

- `datadogexporter`: Replace HistogramMode defined as string with enum.
- `pkg/translator/signalfx`: Change signalfx translator to expose To/From translator structs. (#9740)
- `transformprocessor`: Add parameter validation to `truncate_all` and `limit` functions. The `limit` parameter can no longer be negative. (#9783)

### 🚩 Deprecations 🚩

Expand Down
4 changes: 2 additions & 2 deletions processor/transformprocessor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ it references an unset map value, there will be no action.
- `keep_keys(target, string...)` - `target` is a path expression to a map type field. The map will be mutated to only contain
the fields specified by the list of strings. e.g., `keep_keys(attributes, "http.method")`, `keep_keys(attributes, "http.method", "http.route")`

- `truncate_all(target, limit)` - `target` is a path expression to a map type field. `limit` is an integer. The map will be mutated such that all string values are truncated to the limit. e.g., `truncate_all(attributes, 100)` will truncate all string values in `attributes` such that all string values have less than or equal to 100 characters. Non-string values are ignored.
- `truncate_all(target, limit)` - `target` is a path expression to a map type field. `limit` is a non-negative integer. The map will be mutated such that all string values are truncated to the limit. e.g., `truncate_all(attributes, 100)` will truncate all string values in `attributes` such that all string values have less than or equal to 100 characters. Non-string values are ignored.

- `limit(target, limit)` - `target` is a path expression to a map type field. `limit` is an integer. The map will be mutated such that the number of items does not exceed the limit. e.g., `limit(attributes, 100)` will limit `attributes` to no more than 100 items. Which items are dropped is random.
- `limit(target, limit)` - `target` is a path expression to a map type field. `limit` is a non-negative integer. The map will be mutated such that the number of items does not exceed the limit. e.g., `limit(attributes, 100)` will limit `attributes` to no more than 100 items. Which items are dropped is random.

Supported where operations:
- `==` - matches telemetry where the values are equal to each other
Expand Down
6 changes: 6 additions & 0 deletions processor/transformprocessor/internal/common/functions.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@ func keepKeys(target GetSetter, keys []string) (ExprFunc, error) {
}

func truncateAll(target GetSetter, limit int64) (ExprFunc, error) {
if limit < 0 {
return nil, fmt.Errorf("invalid limit for truncate_all function, %d cannot be negative", limit)
}
return func(ctx TransformContext) interface{} {
if limit < 0 {
return nil
Expand Down Expand Up @@ -104,6 +107,9 @@ func truncateAll(target GetSetter, limit int64) (ExprFunc, error) {
}

func limit(target GetSetter, limit int64) (ExprFunc, error) {
if limit < 0 {
return nil, fmt.Errorf("invalid limit for limit function, %d cannot be negative", limit)
}
return func(ctx TransformContext) interface{} {
val := target.Get(ctx)
if val == nil {
Expand Down
46 changes: 43 additions & 3 deletions processor/transformprocessor/internal/common/functions_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ func Test_newFunctionCall_invalid(t *testing.T) {
},
},
{
name: "not matching slice type",
name: "keep_keys not matching slice type",
inv: Invocation{
Function: "keep_keys",
Arguments: []Value{
Expand All @@ -114,7 +114,7 @@ func Test_newFunctionCall_invalid(t *testing.T) {
},
},
{
name: "not int",
name: "truncate_all not int",
inv: Invocation{
Function: "truncate_all",
Arguments: []Value{
Expand All @@ -134,7 +134,27 @@ func Test_newFunctionCall_invalid(t *testing.T) {
},
},
{
name: "not int",
name: "truncate_all negative limit",
inv: Invocation{
Function: "truncate_all",
Arguments: []Value{
{
Path: &Path{
Fields: []Field{
{
Name: "name",
},
},
},
},
{
Int: intp(-1),
},
},
},
},
{
name: "limit not int",
inv: Invocation{
Function: "limit",
Arguments: []Value{
Expand All @@ -153,6 +173,26 @@ func Test_newFunctionCall_invalid(t *testing.T) {
},
},
},
{
name: "limit negative limit",
inv: Invocation{
Function: "limit",
Arguments: []Value{
{
Path: &Path{
Fields: []Field{
{
Name: "name",
},
},
},
},
{
Int: intp(-1),
},
},
},
},
{
name: "function call returns error",
inv: Invocation{
Expand Down
32 changes: 0 additions & 32 deletions processor/transformprocessor/internal/traces/functions_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -316,38 +316,6 @@ func Test_newFunctionCall(t *testing.T) {
attrs.CopyTo(span.Attributes())
},
},
{
name: "truncate resource negative",
inv: common.Invocation{
Function: "truncate_all",
Arguments: []common.Value{
{
Path: &common.Path{
Fields: []common.Field{
{
Name: "resource",
},
{
Name: "attributes",
},
},
},
},
{
Int: intp(-1),
},
},
},
want: func(span ptrace.Span) {
input.CopyTo(span)
span.Attributes().Clear()
attrs := pcommon.NewMap()
attrs.InsertString("test", "hello world")
attrs.InsertInt("test2", 3)
attrs.InsertBool("test3", true)
attrs.CopyTo(span.Attributes())
},
},
{
name: "limit attributes",
inv: common.Invocation{
Expand Down

0 comments on commit fe12a90

Please sign in to comment.