-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[ML] AIOps: Tweak log rate changes in log rate analysis results table. (
#188648) ## Summary Part of #187684. This moves functions related to log rate changes to the `@kbn/aiops_log_rate_analysis` package. - `getLogRateAnalysisType` was renamed to `getLogRateAnalysisTypeForHistogram` to indicate its use with histogram data. - `getLogRateAnalysisTypeForCounts` was added for cases where we don't have the histogram data available but just the doc counts for baseline an deviation time ranges. This isn't used yet as of this PR but will be in a follow up in combination with the o11y AI assistant. - `getSwappedWindowParameters` is a helper to consolidate inline code that's used to swap baseline and deviation when we detected a dip in log rate. - Rounding for the log rate change messages was tweaked. Changes below `10x` will now be rounded to one digit to avoid messages like `1x increase`. - Tweaked/Shortened the message for 0 in baseline or deviation to just `45 up from 0 in baseline` / `down to 0 from 45 in baseline`. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
- Loading branch information
Showing
23 changed files
with
551 additions
and
159 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
77 changes: 77 additions & 0 deletions
77
x-pack/packages/ml/aiops_log_rate_analysis/get_baseline_and_deviation_rates.test.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the Elastic License | ||
| * 2.0; you may not use this file except in compliance with the Elastic License | ||
| * 2.0. | ||
| */ | ||
|
|
||
| import { getBaselineAndDeviationRates } from './get_baseline_and_deviation_rates'; | ||
| import { LOG_RATE_ANALYSIS_TYPE } from './log_rate_analysis_type'; | ||
|
|
||
| describe('getBaselineAndDeviationRates', () => { | ||
| it('calculates rates for SPIKE analysis', () => { | ||
| const analysisType = LOG_RATE_ANALYSIS_TYPE.SPIKE; | ||
| const baselineBuckets = 10; | ||
| const deviationBuckets = 5; | ||
| const docCount = 100; | ||
| const bgCount = 50; | ||
| const expected = { | ||
| baselineBucketRate: 5, // 50 / 10 | ||
| deviationBucketRate: 20, // 100 / 5 | ||
| }; | ||
|
|
||
| const result = getBaselineAndDeviationRates( | ||
| analysisType, | ||
| baselineBuckets, | ||
| deviationBuckets, | ||
| docCount, | ||
| bgCount | ||
| ); | ||
|
|
||
| expect(result).toEqual(expected); | ||
| }); | ||
|
|
||
| it('calculates rates for DIP analysis', () => { | ||
| const analysisType = LOG_RATE_ANALYSIS_TYPE.DIP; | ||
| const baselineBuckets = 8; | ||
| const deviationBuckets = 4; | ||
| const docCount = 80; // Now represents baseline period in DIP | ||
| const bgCount = 40; // Now represents deviation period in DIP | ||
| const expected = { | ||
| baselineBucketRate: 10, // 80 / 8 | ||
| deviationBucketRate: 10, // 40 / 4 | ||
| }; | ||
|
|
||
| const result = getBaselineAndDeviationRates( | ||
| analysisType, | ||
| baselineBuckets, | ||
| deviationBuckets, | ||
| docCount, | ||
| bgCount | ||
| ); | ||
|
|
||
| expect(result).toEqual(expected); | ||
| }); | ||
|
|
||
| it('handles zero buckets without throwing error', () => { | ||
| const analysisType = LOG_RATE_ANALYSIS_TYPE.SPIKE; | ||
| const baselineBuckets = 0; | ||
| const deviationBuckets = 0; | ||
| const docCount = 100; | ||
| const bgCount = 50; | ||
| const expected = { | ||
| baselineBucketRate: 0, | ||
| deviationBucketRate: 0, | ||
| }; | ||
|
|
||
| const result = getBaselineAndDeviationRates( | ||
| analysisType, | ||
| baselineBuckets, | ||
| deviationBuckets, | ||
| docCount, | ||
| bgCount | ||
| ); | ||
|
|
||
| expect(result).toEqual(expected); | ||
| }); | ||
| }); |
47 changes: 47 additions & 0 deletions
47
x-pack/packages/ml/aiops_log_rate_analysis/get_baseline_and_deviation_rates.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the Elastic License | ||
| * 2.0; you may not use this file except in compliance with the Elastic License | ||
| * 2.0. | ||
| */ | ||
|
|
||
| import { LOG_RATE_ANALYSIS_TYPE, type LogRateAnalysisType } from './log_rate_analysis_type'; | ||
|
|
||
| /** | ||
| * Calculates the baseline and deviation rates for log rate analysis based on the specified analysis type. | ||
| * | ||
| * This function computes the rates by dividing the document count (docCount) and background count (bgCount) | ||
| * by the number of buckets allocated for baseline and deviation periods, respectively. The calculation | ||
| * method varies depending on whether the analysis type is a "spike" or a "dip". For a "spike", the baseline | ||
| * rate is derived from the background count and the deviation rate from the document count. For a "dip", | ||
| * the roles are reversed. | ||
| * | ||
| * @param analysisType The type of analysis to perform, can be either "spike" or "dip". | ||
| * @param baselineBuckets The number of buckets into which the baseline period is divided. | ||
| * @param deviationBuckets The number of buckets into which the deviation period is divided. | ||
| * @param docCount The total document count observed in the deviation period. | ||
| * @param bgCount The total background count observed in the baseline period. | ||
| * @returns An object containing the calculated baseline and deviation bucket rates. | ||
| */ | ||
| export function getBaselineAndDeviationRates( | ||
| analysisType: LogRateAnalysisType, | ||
| baselineBuckets: number, | ||
| deviationBuckets: number, | ||
| docCount: number, | ||
| bgCount: number | ||
| ): { baselineBucketRate: number; deviationBucketRate: number } { | ||
| if (baselineBuckets === 0 || deviationBuckets === 0) { | ||
| return { baselineBucketRate: 0, deviationBucketRate: 0 }; | ||
| } else if (analysisType === LOG_RATE_ANALYSIS_TYPE.SPIKE) { | ||
| return { | ||
| baselineBucketRate: Math.round(bgCount / baselineBuckets), | ||
| deviationBucketRate: Math.round(docCount / deviationBuckets), | ||
| }; | ||
| } else { | ||
| // For dip, the "doc count" refers to the amount of documents in the baseline time range so we set baselineBucketRate | ||
| return { | ||
| baselineBucketRate: Math.round(docCount / baselineBuckets), | ||
| deviationBucketRate: Math.round(bgCount / deviationBuckets), | ||
| }; | ||
| } | ||
| } |
54 changes: 54 additions & 0 deletions
54
x-pack/packages/ml/aiops_log_rate_analysis/get_log_rate_analysis_type_for_counts.test.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the Elastic License | ||
| * 2.0; you may not use this file except in compliance with the Elastic License | ||
| * 2.0. | ||
| */ | ||
|
|
||
| import { LOG_RATE_ANALYSIS_TYPE } from './log_rate_analysis_type'; | ||
| import { getLogRateAnalysisTypeForCounts } from './get_log_rate_analysis_type_for_counts'; | ||
|
|
||
| const windowParameters = { | ||
| baselineMin: 1654579807500, | ||
| baselineMax: 1654586107500, | ||
| deviationMin: 1654586400000, | ||
| deviationMax: 1654587007500, | ||
| }; | ||
|
|
||
| describe('getLogRateAnalysisTypeForCounts', () => { | ||
| it('returns SPIKE when normalized deviation count is higher than baseline count', () => { | ||
| const baselineCount = 100; | ||
| const deviationCount = 200; | ||
|
|
||
| const result = getLogRateAnalysisTypeForCounts(baselineCount, deviationCount, windowParameters); | ||
|
|
||
| expect(result).toEqual(LOG_RATE_ANALYSIS_TYPE.SPIKE); | ||
| }); | ||
|
|
||
| it('returns DIP when normalized deviation count is lower than baseline count', () => { | ||
| const baselineCount = 20000; | ||
| const deviationCount = 10; | ||
|
|
||
| const result = getLogRateAnalysisTypeForCounts(baselineCount, deviationCount, windowParameters); | ||
|
|
||
| expect(result).toEqual(LOG_RATE_ANALYSIS_TYPE.DIP); | ||
| }); | ||
|
|
||
| it('handles zero baseline count without throwing error', () => { | ||
| const baselineCount = 0; | ||
| const deviationCount = 100; | ||
|
|
||
| const result = getLogRateAnalysisTypeForCounts(baselineCount, deviationCount, windowParameters); | ||
|
|
||
| expect(result).toBe(LOG_RATE_ANALYSIS_TYPE.SPIKE); | ||
| }); | ||
|
|
||
| it('handles zero deviation count without throwing error', () => { | ||
| const baselineCount = 100; | ||
| const deviationCount = 0; | ||
|
|
||
| const result = getLogRateAnalysisTypeForCounts(baselineCount, deviationCount, windowParameters); | ||
|
|
||
| expect(result).toBe(LOG_RATE_ANALYSIS_TYPE.DIP); | ||
| }); | ||
| }); |
35 changes: 35 additions & 0 deletions
35
x-pack/packages/ml/aiops_log_rate_analysis/get_log_rate_analysis_type_for_counts.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the Elastic License | ||
| * 2.0; you may not use this file except in compliance with the Elastic License | ||
| * 2.0. | ||
| */ | ||
|
|
||
| import { LOG_RATE_ANALYSIS_TYPE, type LogRateAnalysisType } from './log_rate_analysis_type'; | ||
| import type { WindowParameters } from './window_parameters'; | ||
|
|
||
| /** | ||
| * Identify the log rate analysis type based on the baseline/deviation doc counts. | ||
| * | ||
| * @param baselineCount The baseline doc count. | ||
| * @param deviationCount The deviation doc count. | ||
| * @param windowParameters The window parameters with baseline and deviation time range. | ||
| * @returns The log rate analysis type. | ||
| */ | ||
| export function getLogRateAnalysisTypeForCounts( | ||
| baselineCount: number, | ||
| deviationCount: number, | ||
| windowParameters: WindowParameters | ||
| ): LogRateAnalysisType { | ||
| const { baselineMin, baselineMax, deviationMin, deviationMax } = windowParameters; | ||
|
|
||
| const deviationDuration = deviationMax - deviationMin; | ||
| const deviationPerBucket = deviationCount; | ||
|
|
||
| const baselineNormalizedDuration = (baselineMax - baselineMin) / deviationDuration; | ||
| const baselinePerBucket = baselineCount / baselineNormalizedDuration; | ||
|
|
||
| return deviationPerBucket >= baselinePerBucket | ||
| ? LOG_RATE_ANALYSIS_TYPE.SPIKE | ||
| : LOG_RATE_ANALYSIS_TYPE.DIP; | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.