[Filebeat][httpjson input] Unable to create body with single value list

[P1llus]

Follow-up from discussion here https://discuss.elastic.co/t/header-transform-concern-need-help/269064/19.

There does not seem to be any good way or workaround to create a body with a single value list.

The append documentation states:
https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_append
Appends a value to a list. If the field does not exist, the first entry will be a scalar value, and subsequent additions will convert the value to a list.

Relevant code is here: https://github.com/elastic/beats/blob/master/x-pack/filebeat/input/httpjson/internal/v2/transform_append.go#L136

[P1llus]

We could either change the default behavior, or we would need some config option for this, is there a reason why the first value is not a list when using append compared to set, WDYT @marc-gr ?

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[legoguy1000]

@P1llus the original poster in discuss had created an issue for this, #24952. I would close one so there's not duplicates.

[marc-gr]

We could either change the default behavior, or we would need some config option for this, is there a reason why the first value is not a list when using append compared to set, WDYT @marc-gr ?

Was a matter of consistency with append processor, which works like this. Truth is that the append transform is not only meant to send data to ES so this behavior might be not the expected one. I am fine with either (or both) changing the default behavior to always create a list, or to allow set to have list values as suggested in #24952. The first would be considered a breaking change though, so maybe is safer to err on the safe side if we were not considering the current behavior a bug.