Skip to content

Commit

Permalink
ES|QL: add MV_APPEND function (#107001)
Browse files Browse the repository at this point in the history
Adding `MV_APPEND(value1, value2)` function, that appends two values
creating a single multi-value. If one or both the inputs are
multi-values, the result is the concatenation of all the values, eg.

```
MV_APPEND([a, b], [c, d]) -> [a, b, c, d]
```

~I think for this specific case it makes sense to consider `null` values
as empty arrays, so that~ ~MV_APPEND(value, null) -> value~ ~It is
pretty uncommon for ESQL (all the other functions, apart from
`COALESCE`, short-circuit to `null` when one of the values is null), so
let's discuss this behavior.~

[EDIT] considering the feedback from Andrei, I changed this logic and
made it consistent with the other functions: now if one of the
parameters is null, the function returns null
  • Loading branch information
luigidellaquila authored Jun 4, 2024
1 parent 21952c7 commit 5f6e8f6
Show file tree
Hide file tree
Showing 24 changed files with 1,585 additions and 1 deletion.
5 changes: 5 additions & 0 deletions docs/reference/esql/functions/description/mv_append.asciidoc

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

242 changes: 242 additions & 0 deletions docs/reference/esql/functions/kibana/definition/mv_append.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 7 additions & 0 deletions docs/reference/esql/functions/kibana/docs/mv_append.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions docs/reference/esql/functions/layout/mv_append.asciidoc

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions docs/reference/esql/functions/parameters/mv_append.asciidoc

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions docs/reference/esql/functions/signature/mv_append.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 21 additions & 0 deletions docs/reference/esql/functions/types/mv_append.asciidoc

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
"keyword" : { "type" : "keyword" },
"date" : { "type" : "date" },
"date_nanos": { "type" : "date_nanos" },
"long" : { "type" : "long" },
"ip" : { "type" : "ip" },
"unsupported" : { "type" : "ip_range" },
"some" : {
"properties" : {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1076,6 +1076,23 @@ required_capability: agg_values
[1955-01-21T00:00:00Z, 1957-05-23T00:00:00Z, 1959-12-03T00:00:00Z] | null
;


mvAppendDates
required_capability: fn_mv_append

FROM employees
| WHERE emp_no == 10039 OR emp_no == 10040
| SORT emp_no
| EVAL dates = mv_append(birth_date, hire_date)
| KEEP emp_no, birth_date, hire_date, dates
;

emp_no:integer | birth_date:date | hire_date:date | dates:date
10039 | 1959-10-01T00:00:00Z | 1988-01-19T00:00:00Z | [1959-10-01T00:00:00Z, 1988-01-19T00:00:00Z]
10040 | null | 1993-02-14T00:00:00Z | null
;


implicitCastingNotEqual
required_capability: string_literal_auto_casting
from employees | where birth_date != "1957-05-23T00:00:00Z" | keep emp_no, birth_date | sort emp_no | limit 3;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -523,6 +523,25 @@ required_capability: agg_values
[1.7, 1.83, 2.05] | null
;


mvAppend
required_capability: fn_mv_append

FROM employees
| WHERE emp_no == 10008 OR emp_no == 10021
| EVAL d = mv_append(salary_change, salary_change),
i = mv_append(salary_change.int, salary_change.int),
i2 = mv_append(emp_no, salary_change.int),
i3 = mv_append(emp_no, emp_no),
s = mv_append(salary_change.keyword, salary_change.keyword)
| KEEP emp_no, salary_change, d, i, i2, i3, s;

emp_no:integer | salary_change:double | d:double | i:integer | i2:integer | i3:integer | s:keyword
10008 | [-2.92,0.75,3.54,12.68] | [-2.92,0.75,3.54,12.68,-2.92,0.75,3.54,12.68] | [-2,0,3,12,-2,0,3,12] | [10008,-2,0,3,12] | [10008, 10008] | [-2.92,0.75,12.68,3.54,-2.92,0.75,12.68,3.54]
10021 | null | null | null | null | [10021, 10021] | null
;


signumOfPositiveDouble#[skip:-8.13.99,reason:new scalar function added in 8.14]
row d = to_double(100) | eval s = signum(d);

Expand Down
Loading

0 comments on commit 5f6e8f6

Please sign in to comment.