Metricbeat Dashboard causes query exception in master/7.0

[beniwohli]

With Elasticsearch/Kibana on master, when running metricbeat with the system module active, opening the Host Overview dashboard leads to an error

"reason":"field expansion matches too many fields, limit: 1024, got: 1030"

The full error, as well as the metricbeat configuration I used can be found here: https://gist.github.com/beniwohli/bb1bfe63ed02480d08025a444d157376

[ruflin]

@simianhacker I assume these are the same issues? #5278

[simianhacker]

@ruflin I don't think so... I get that error when searching in Discover as well.

[simianhacker]

@elastic/beats This is related to elastic/elasticsearch#26541 which is going to be a problem for Metricbeat.

[Bargs]

I just caught up on some discussion of this in Slack from last week. I just wanted to note that this will still affect users when they switch to kuery since the limit also applies to multi_match queries, which kuery uses under the hood.

[simianhacker]

So the fix we decided on last week is to force the admin to add a list of 1024 fields to the index setting index.query.default_field. Since we are administering the metrcibeat-* indices we will be responsible for changing our index template to include index.query.default_field.

[Bargs]

That won't work for kuery since multi_match doesn't use default_field. But if I understood the conversation, what you described is meant to be a simple fix for the short term, so that's probably fine.

[simianhacker]

@Bargs Yeah... that's going to be a problem for multi_match. What do you guys put in the fields array now just *? It would suck to ask the user to fill that in for each index pattern that has more the 1024 searchable fields.

[Bargs]

Yep, just *. I asked if the ES guys have any advice in the other issue.

[rashmivkulkarni]

We hit the same issue as above while running the integration tests on master .

org.elasticsearch.transport.RemoteTransportException: [cQvsA1-][10.0.2.15:9300][indices:data/read/search[phase/query]]
Caused by: org.elasticsearch.index.query.QueryShardException: failed to create query: {
  "bool" : {
    "must" : [
      {
        "query_string" : {
          "query" : "*",
          "default_field" : "*",
          "fields" : [ ],
          "type" : "best_fields",
          "default_operator" : "or",
          "max_determinized_states" : 10000,
          "enable_position_increments" : true,
          "fuzziness" : "AUTO",
          "fuzzy_prefix_length" : 0,
          "fuzzy_max_expansions" : 50,
          "phrase_slop" : 0,
          "analyze_wildcard" : true,
          "escape" : false,
          "auto_generate_synonyms_phrase_query" : true,
          "fuzzy_transpositions" : true,
          "boost" : 1.0
        }
      },
      {
        "query_string" : {
          "query" : "*",
          "default_field" : "*",
          "fields" : [ ],
          "type" : "best_fields",
          "default_operator" : "or",
          "max_determinized_states" : 10000,
          "enable_position_increments" : true,
          "fuzziness" : "AUTO",
          "fuzzy_prefix_length" : 0,
          "fuzzy_max_expansions" : 50,
          "phrase_slop" : 0,
          "analyze_wildcard" : true,
          "escape" : false,
          "auto_generate_synonyms_phrase_query" : true,
          "fuzzy_transpositions" : true,
          "boost" : 1.0
        }
      },

can we be notified when this is fixed?
cc / @LeeDr

[bhavyarm]

@elastic/beats I logged this on Kibana and then found out from the viz team that this is an old bug. elastic/kibana#15863 Just making a note here.

[ruflin]

Tracking issue on the kibana side: elastic/kibana#16232

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.