Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

migrations Actions.readWithPit must use allow_partial_search_results=false #136271

Closed
rudolf opened this issue Jul 13, 2022 · 2 comments · Fixed by #136419
Closed

migrations Actions.readWithPit must use allow_partial_search_results=false #136271

rudolf opened this issue Jul 13, 2022 · 2 comments · Fixed by #136419
Assignees
@rudolf
Labels
bug Fixes for quality problems that affect the customer experience Feature:Migrations Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc

Comments

@rudolf
Copy link
Contributor

rudolf commented Jul 13, 2022

REINDEX_SOURCE_TO_TEMP_READ and OUTDATED_DOCUMENTS_SEARCH_READ uses Actions.readWithPit to page through all the documents in an index. However, readWithPit doesn't set allow_partial_search_results so it defaults to true. This means the search will successfully complete with partial results if one of the shards aren't available. When upgrading Kibana from an previous version this could lead to data loss.

Because we always go through the WAIT_FOR_YELLOW_SOURCE step first, it's unlikely that the source index's shards aren't available, but this isn't a guarantee.

This potentially impacts all versions since 7.14 (#97222)
@rudolf rudolf added bug Fixes for quality problems that affect the customer experience Feature:Migrations labels Jul 13, 2022
@botelastic botelastic bot added the needs-team Issues missing a team label label Jul 13, 2022
@rudolf rudolf self-assigned this Jul 13, 2022
@rudolf rudolf added the Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc label Jul 13, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-core (Team:Core)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Jul 13, 2022
@gsoldevila
Copy link
Contributor

I'm working on a PR to fix this

@gsoldevila gsoldevila mentioned this issue Jul 14, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rudolf rudolf

Labels
bug Fixes for quality problems that affect the customer experience Feature:Migrations Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Don't allow_partial_search_results in readWithPit gsoldevila/kibana
3 participants
@rudolf @elasticmachine @gsoldevila