_mget error, vector's dimensions must be <= [1024]

[li5m]

Elasticsearch Version

8.12

Installed Plugins

No response

Java Version

bundled

OS Version

Alibaba Cloud Linux 3.2104 LTS 64-bit

Problem Description

Occasionally, an exception is returned when using the _mget API to retrieve documents：
ErrorCause: {"type":"illegal_argument_exception","reason":"Field [textVector] vector's dimensions must be <= [1024]; got 1536","root_cause":[{"type":"illegal_argument_exception","reason":"Field [textVector] vector's dimensions must be <= [1024]; got 1536"}]}

"mappings": {
"dynamic": "strict",
"_source": {
"mode": "synthetic"
},
"properties": {
"actId": {
"type": "keyword"
},
"actPublishTime": {
"type": "date"
},
"activityType": {
"type": "keyword"
},
"docModifyTime": {
"type": "date"
},
"firstCategoryIdList": {
"type": "long"
},
"ghId": {
"type": "keyword"
},
"goodsFirstPic": {
"type": "keyword"
},
"pactId": {
"type": "keyword"
},
"primaryCategory": {
"type": "keyword"
},
"startActId": {
"type": "keyword"
},
"text": {
"type": "keyword",
"index": false
},
"textVector": {
"type": "dense_vector",
"dims": 1536,
"index": true,
"similarity": "l2_norm"
}
}
}

Steps to Reproduce

1、Use dense_vector field.
2、Add documents.
3、Use _mget api.
Can not stable reproduce

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[benwtrent]

@li5m this is a weird one!

Did you capture a stack trace for the failure? A way to do that is to add the url parameter error_trace, this will return a stack trace of the exception thrown in the response body and will help diagnose the bug.

[benwtrent]

@li5m one more question: do you set the realtime parameter? Or do you use the default value of true for the _mget request?

[benwtrent]

@elastic/es-distributed this is really weird.

In the DenseVectorFieldMapper we override the codec used so that this error isn't thrown. The value shown in the error indicates that the document indexing path was executed without our overridden codec settings.

I am digging into the various _mget paths and its starting to think it has something to do with us getting realtime values via the Translog.

Do we do anything there that simulates the indexing path for a document that relies on the default codecs being used?

[elasticsearchmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[li5m]

@li5m this is a weird one!

Did you capture a stack trace for the failure? A way to do that is to add the url parameter error_trace, this will return a stack trace of the exception thrown in the response body and will help diagnose the bug.

I couldn't find the error_trace parameter in the elasticsearch-java MgetRequest.Builder class, so I'm unable to capture the stack trace. Can you tell me how to set this parameter?

[li5m]

@li5m one more question: do you set the realtime parameter? Or do you use the default value of true for the _mget request?

I did not set the realtime parameter; it is using the default value of true.

[pxsalehi]

I am digging into the various _mget paths and its starting to think it has something to do with us getting realtime values via the Translog.

@benwtrent One change we've made a while ago was to provide a "translog-only" get. For that, I moved around some code (f02e9f9). I don't think we care about the codec nor have we touched it on that path (at least not purposefully) AFAICT. I am not aware of any changes that we have made to TranslogDirectoryReader for the real-time get/mget stuff.

[benwtrent]

@pxsalehi this may be a bug that has been around forever. Mind reviewing #104700 ?

[benwtrent]

@li5m

I couldn't find the error_trace parameter in the elasticsearch-java MgetRequest.Builder class, so I'm unable to capture the stack trace. Can you tell me how to set this parameter?

I think we found the place with the bug. However, to not leave you without an answer, its a query parameter. So, you should be able to set it in the RequestOptions object when building the client.