[Bug] Pre-Built Detection Rules Package Versioning #2251
Comments
|
From our discussion, is this what we are looking to provide with the version lock file. A couple of things, we did not include the Example versions lock file: version.lock.json.txt Here is the output from doing a build with update version locks. If I understand correctly, we want to get a PR going with these version lock changes and backport this to 7.16. Then checkout 8.3 branch, checkout the locked versions commit previously used, build the package from 8.3 branch and follow the normal release process to integrations and then package-storage? Code used: |
|
After further discussion with @brokensound77 and @Mikaayenson we have decided that all rules will need to be hard-forked (add a min_stack_version) to them if a new build-time field is identified during build. This applies to new rules and existing. A buffer of ~100 will be given to forked rules in regards to versioning so these rules can still be updated whereas the current version starts at +100 therefore any changes that have to go directly to a forked rule will have versioning space available and not overlap the current. Once this space is up, we must deprecate the rule and create a new one. |
This was
linked to
pull requests
Aug 24, 2022
|
This has been solved and completed as noted by the tasks in the original comment. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Overview
Initial Conversation: https://elastic.slack.com/archives/C02USDK55AQ/p1660833389191059
We recently released 8.2.1 and 7.16.4 pre-built detection rule integration packages for 8.4. During 8.4 dev, @brokensound77 @Mikaayenson and myself were tasked with adding three new fields to rules.
The code was developed, tested and implemented which dynamically builds the three fields during the building of the package. This only occurs if the stack version referenced in
packages.yamlis version 8.3 or above.Our release process involves checking out each least-compatible stack for 7 and 8 series that we support and then building the package from there that is then released to Kibana and Integrations.
Locked commit used: #2236
Therefore, our recent process did the following in short:
Kibana Update
The Kibana update did receive the rules with the additional fields as expected and tested in the updated PR. This is because we built from the 8.4 branch so the 3 fields were added as they met the minimum stack requirement of 8.3.
OOB Integrations Update
7 Series
8 Series
Since we built from 7.16 and 8.2, the additional fields did not get added to the packages and went into production. This is not a bug as the code is working as expected, however, we will need to release an 8.3.1 package (building from 8.3 branch) for these fields to be added to the package.
Versioning
When version locking occurred and was backported, the process started at 7.16 going to main. If a detection rule had changes, a version bump was given and the changes continued to the next branch, doing the lock versions again. Once 8.3 branch was reached, the new fields were added and some rules received a double bump if they had a change during 8.4 dev. If a rule did not have a change, they still received a single bump because the fields were added in 8.3. This caused all rules to receive a version increase.
This causes an issue if we build with 8.3 if we were to release it immediately as there are no changes to the hash and thus no new versions. Thus we need to manually bump the versions and check forked (previous) versions in the version lock file.
Overall, the tasks are as follows:
min_stack_versionof 8.3 to all rules | min_stack all rules to 8.3 #2259version.lock.jsonfile to change current rules to 100The text was updated successfully, but these errors were encountered: