[Migrations] Add support of deferred migrations #153117
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
c0fb7fe to
e66bb7b
Compare
93151d1 to
ca900f7
Compare
751a795 to
e857f66
Compare
b338228 to
9ddfc05
Compare
|
@TinaHeiligers @rudolf @pgayvallet I've made some changes in the PR and also updated the documentation. That would be great if you could take another look at the updated files. |
rudolf
approved these changes
May 18, 2023
💚 Build Succeeded
Metrics [docs]Public APIs missing comments
Unknown metric groupsAPI count
ESLint disabled line counts
References to deprecated APIs
Total ESLint disabled count
History
To update your PR or re-run it, just comment with: |
pgayvallet
added a commit
that referenced
this pull request
May 23, 2023
## Summary Part of #150312 (next steps depend on #153117) **This PR does two things:** - introduce the concept of version persistence schema - adapt the document migrator to support downward migrations for documents of an higher version. In the follow-up, we will then update the calls from the SOR to the document migrator to allow downward conversions when we're using the ZDT migration algorithm (which requires #153117 to be merged) ### Model version persistence schema. *(This is what has also been named 'eviction schema' or 'known fields schema'.)* A new `SavedObjectsModelVersion.schemas.backwardConversion` property was added to the model version definition. This 'schema' can either be an arbitrary function, or a `schema.object` from `@kbn/config-schema` ```ts type SavedObjectModelVersionBackwardConversionSchema< InAttrs = unknown, OutAttrs = unknown > = ObjectType | SavedObjectModelVersionBackwardConversionFn<InAttrs, OutAttrs>; ``` When specified for a version, the document's attributes will go thought this schema during down conversions by the document migrator. ### Adapt the document migrator to support downward migrations for documents of an higher version. Add an `allowDowngrade` option to `DocumentMigrator.migrate` and `KibanaMigrator.migrateDocument`. When this option is set to `true`, the document migration will accept to 'downgrade' the document if necessary, instead of throwing an error as done when the option is `false` or unspecified (which was the only behavior prior to this PR's changes) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
delanni
pushed a commit
to delanni/kibana
that referenced
this pull request
May 25, 2023
* Add deferred migrations parameter. * Update outdated documents query to take into account deferred migrations. * Update outdated documents query to take into account the core migration version. * Update read operations in the saved objects repository to perform deferred migrations.
delanni
pushed a commit
to delanni/kibana
that referenced
this pull request
May 25, 2023
) ## Summary Part of elastic#150312 (next steps depend on elastic#153117) **This PR does two things:** - introduce the concept of version persistence schema - adapt the document migrator to support downward migrations for documents of an higher version. In the follow-up, we will then update the calls from the SOR to the document migrator to allow downward conversions when we're using the ZDT migration algorithm (which requires elastic#153117 to be merged) ### Model version persistence schema. *(This is what has also been named 'eviction schema' or 'known fields schema'.)* A new `SavedObjectsModelVersion.schemas.backwardConversion` property was added to the model version definition. This 'schema' can either be an arbitrary function, or a `schema.object` from `@kbn/config-schema` ```ts type SavedObjectModelVersionBackwardConversionSchema< InAttrs = unknown, OutAttrs = unknown > = ObjectType | SavedObjectModelVersionBackwardConversionFn<InAttrs, OutAttrs>; ``` When specified for a version, the document's attributes will go thought this schema during down conversions by the document migrator. ### Adapt the document migrator to support downward migrations for documents of an higher version. Add an `allowDowngrade` option to `DocumentMigrator.migrate` and `KibanaMigrator.migrateDocument`. When this option is set to `true`, the document migration will accept to 'downgrade' the document if necessary, instead of throwing an error as done when the option is `false` or unspecified (which was the only behavior prior to this PR's changes) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Resolves #147445.
Checklist
For maintainers