-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Fleet] Add package policy upgrade API #103017
[Fleet] Add package policy upgrade API #103017
Conversation
…ade-wizard # Conflicts: # x-pack/plugins/fleet/public/applications/integrations/screens/home/upgrade_callout.tsx # x-pack/plugins/fleet/public/applications/integrations/sections/epm/screens/home/index.tsx
…ade-wizard # Conflicts: # x-pack/plugins/fleet/public/applications/integrations/sections/epm/screens/home/index.tsx # x-pack/plugins/fleet/public/constants/page_paths.ts
Pinging @elastic/fleet (Feature:Fleet) |
Wow, this is awesome, I will take this out for a spin soon! We will want to hold off on merging this until past FF, to target 7.15, to account for design changes and testing. So I'm updating the label as such. |
I'm going to work on adding some test cases based on the test plan in the PR description, namely verifying that an error is reported when upgrading a package whose spec removes a previously existing input. |
Thanks for the amazing work here @Zacqary @kpollich I noticed a few things while testing:
Otherwise I did a few tests and looks it working well :) |
Thanks @nchaulet for taking a look. The dry run should contain a |
No there was no errors during the dry run, and a 500 during the real call with nothing inside errors |
Thank you I will take a look at that. |
@jen-huang @nchaulet - I've been working on adding a test case to test the error reporting process when upgrading a package to a version that incurs an error (see @Zacqary's test case in the PR description for what I'm trying to achieve) in a344796, but I'm having trouble setting up the text fixtures for the integrations. I'd like to to pair up with one or both of you and try to work through it. I've been combing through the Integrations docs and referencing other packages to try and build up a test package here, but I keep running into this error when attempting to create policies for this package. {
"statusCode": 500,
"error": "Internal Server Error",
"message": "Stream template not found, unable to find stream for input test_input"
} My request payload looks like this:
Let me know if either of you have some time on Monday to take a look at this with me, or maybe there's something obvious I'm missing that might unblock me getting this test case in place! Thank you 🙂 |
@kpollich for your test case you need to have an agent template (in handlebar) for each of the stream you declare inside the input, we used that template to build the policy send to the agent, (ping me if you need help on that) |
Just tested again and looks like it's working, maybe I was just not passing the good params. |
@elasticmachine merge upstream |
💚 Build Succeeded
Metrics [docs]Public APIs missing comments
Page load bundle
History
To update your PR or re-run it, just comment with: |
Tested locally and upgrade worked well 🚀 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚀
* Add Integrations page callout for package upgades * Fix props * Add missing file * Add integrations upgrade callout message * Add link to updates available tab * Fix merge * Upgrade ppolicies UI WIP * Initial upgrade dry run API * Add upgrade method * Move overridePackageInputs and use for upgrade method * Add new variables to dry run diff * Revert UI changes to uto upgrade wizard * Add vars and streams to error keys * Type fix * Fix jest * Fix types * Fix typecheck * Fix types * Add integration test for dry run API * Flesh out test cases * Clean up error responses for dry runs * Fix failing tests * WIP: Add (failing for now) test case for package upgrade w/ error * Add compiled_stream to test API payload * Fix failing test case for automatic upgrade * Fix compiled stream in package policy upgrade * Remove fleet and agent setup from integration test * Unload esarchiver fixtures in api integration test Co-authored-by: Kyle Pollich <kpollich1@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Add Integrations page callout for package upgades * Fix props * Add missing file * Add integrations upgrade callout message * Add link to updates available tab * Fix merge * Upgrade ppolicies UI WIP * Initial upgrade dry run API * Add upgrade method * Move overridePackageInputs and use for upgrade method * Add new variables to dry run diff * Revert UI changes to uto upgrade wizard * Add vars and streams to error keys * Type fix * Fix jest * Fix types * Fix typecheck * Fix types * Add integration test for dry run API * Flesh out test cases * Clean up error responses for dry runs * Fix failing tests * WIP: Add (failing for now) test case for package upgrade w/ error * Add compiled_stream to test API payload * Fix failing test case for automatic upgrade * Fix compiled stream in package policy upgrade * Remove fleet and agent setup from integration test * Unload esarchiver fixtures in api integration test Co-authored-by: Kyle Pollich <kpollich1@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Zacqary Adam Xeper <Zacqary@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Add Integrations page callout for package upgades * Fix props * Add missing file * Add integrations upgrade callout message * Add link to updates available tab * Fix merge * Upgrade ppolicies UI WIP * Initial upgrade dry run API * Add upgrade method * Move overridePackageInputs and use for upgrade method * Add new variables to dry run diff * Revert UI changes to uto upgrade wizard * Add vars and streams to error keys * Type fix * Fix jest * Fix types * Fix typecheck * Fix types * Add integration test for dry run API * Flesh out test cases * Clean up error responses for dry runs * Fix failing tests * WIP: Add (failing for now) test case for package upgrade w/ error * Add compiled_stream to test API payload * Fix failing test case for automatic upgrade * Fix compiled stream in package policy upgrade * Remove fleet and agent setup from integration test * Unload esarchiver fixtures in api integration test Co-authored-by: Kyle Pollich <kpollich1@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Summary
Part of #106048
(KP) I'm working on getting this PR over this finish line by doing the following
This adds an endpoint at
/api/fleet/package_policies/upgrade
that will upgrade an agent policy's integration to the latest installed version of its package.It can be used both to perform policy upgrades, or (using the
dryRun
flag) check to see if the policy can be upgraded automatically or requires user attention.The API takes a payload of:
packagePolicyIds
:Array<string> (required)
- each package policyid
passed in to be upgraded to the latest installed package version. Any newinputs
added in the most recent package version will pull their default values from the package's template.dryRun
:boolean
- iftrue
, instead of performing the upgrade operation, this will instead return a diff of what the package policy would look like before and after its upgrade. If anyinputs
were removed or had their names changed in the most recent package version, these will throw errors and set ahasErrors
flag in the returning payload totrue
.Response:
if
dryRun
is false:An array of:
id
:string
- the ID of the upgraded package policyname
:string
- the name of the upgraded package policysuccess
:boolean
- whether the upgrade was a success or notif
dryRun
is true:An array of:
name
:string
- the name of the upgraded package policydiff
:[PackagePolicy, DryRunPackagePolicy]
-[0]
is the JSON of the package policy before upgrade,[1]
is the JSON of the package policy after upgrade. If any errors were thrown,[1]
will also contain:errors
:Array<{ key: string; message: string}>
- an array of which inputs, streams, or their corresponding vars threw errors when trying to transfer them to the new package. As of this PR this will only occur if the input was removed from the integration in the target (new) version.hasErrors
:boolean
- If anyerrors
are present, this will betrue
Sample Request/Response Payloads
Request:
Response:
Request:
Response:
Testing
apache
elastic-package
to build this new packagerequired: true
and a default value. Bump the version on this package's top-levelmanifest.yml
. Then build it again withelastic-package
.xpack.fleet.registryUrl: http://localhost:8080
inkibana.yml
.kibana.yml
, usexpack.fleet.packages
to preinstall the previous version of your test package, without the new variable you added (e.g. if you bumped the version ofapache
from0.7.0
to0.7.1
, install0.7.0
). Usexpack.fleet.agentPolicies
to preconfigure an agent policy that uses this package.isEnterprise
on this line to always betrue
. (elastic-package
seems to be having trouble serving Elasticsearch with an Enterprise license right now.)/api/fleet/package_policies/upgrade
withdryRun
set to false.integrations
repo is also present, and has the expected default value.integrations
repo, remove a varaible. Bump the version again and build it. Restartelastic-package
./api/fleet/package_policies/upgrade
again withdryRun
set totrue
. Ensure that the payload includes a clear error pointing you to the removed variable.Checklist