Kibana: Array Values are by default sorting instead of printing as is.

[kanika-singla]

Kibana version: v7

Original install method (e.g. download page, yum, from source, etc.): I am using company infrastructure deployment.

Describe the bug: I am trying to show the field which is nested array (nestedResults), on Kibana (Lens) it is sorting the array values when displaying it on the dashboard (Lens visualization). I am breaking the nested array on the unique field values.

Steps to reproduce:

1. Have logs ready in elastic with array containing some values- repetitive, strings, numbers.
2. Create a dashboard in Kibana
3. Create a Lens visualization -> Data Table -> Select the field containing array values
4. It will show as sorted field value

Expected behavior: The values of array print as is on Kibana and should not sort the values.

Screenshots (if relevant):

Any additional context:
https://discuss.elastic.co/t/kibana-sorting-out-array-values/272266

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[flash1293]

This is a limitation of how the top hits agg config works today. It's happening because we are fetching these from the docvalues which seem to not preserve the original order in Elasticsearch.

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServices)

[flash1293]

This will be fixed by #103279

Right now in these cases docvalues are used which are not preserving the original order.

POST mytest/_doc
{
  "a": ["z", "y", "x"],
  "ts": "2021-06-28"
}

GET mytest/_search
{
  "size": 0,
  "aggs": {
    "h": {
      "top_hits": {
        "size": 1,
        "docvalue_fields": ["a.keyword"],
        "sort": [
          {
            "ts": {
              "order": "desc"
            }
          }
        ]
      }
    }
  }
}

returns

{
            "_index" : "mytest",
            "_id" : "VXmvUXoBaek9z0nub3ei",
            "_score" : null,
            "_source" : {
              "a" : [
                "z",
                "y",
                "x"
              ],
              "ts" : "2021-06-28"
            },
            "fields" : {
              "a.keyword" : [
                "x",
                "y",
                "z"
              ]
            },
            "sort" : [
              1624838400000
            ]
          }

However using fields like this:

GET mytest/_search
{
  "size": 0,
  "aggs": {
    "h": {
      "top_hits": {
        "size": 1,
        "fields": ["a.keyword"],
        "sort": [
          {
            "ts": {
              "order": "desc"
            }
          }
        ]
      }
    }
  }
}

will return this:

{
            "_index" : "mytest",
            "_id" : "VXmvUXoBaek9z0nub3ei",
            "_score" : null,
            "_source" : {
              "a" : [
                "z",
                "y",
                "x"
              ],
              "ts" : "2021-06-28"
            },
            "fields" : {
              "a.keyword" : [
                "z",
                "y",
                "x"
              ]
            },
            "sort" : [
              1624838400000
            ]
          }

[flash1293]

Fixed by #103279