Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Infra] Running processes missing from processes table #209076

Merged
merged 11 commits into from
Feb 4, 2025
Merged

[Infra] Running processes missing from processes table #209076

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
f69bed1
Fix missing processes issue
jennypavlova Jan 30, 2025
8e22f35
Fix charts size
jennypavlova Jan 31, 2025
1f08bf5
Merge branch 'main' into 184582-infra-running-processes-missing-from-…
jennypavlova Jan 31, 2025
ca96fec
Update test archive / match with e2e test
jennypavlova Jan 31, 2025
fe7e9fa
Merge branch 'main' into 184582-infra-running-processes-missing-from-…
jennypavlova Jan 31, 2025
5831445
Merge branch 'main' into 184582-infra-running-processes-missing-from-…
jennypavlova Feb 3, 2025
437ad05
Fix new archive mappings and tests
jennypavlova Feb 3, 2025
8d9ef5e
Merge branch 'main' into 184582-infra-running-processes-missing-from-…
jennypavlova Feb 3, 2025
8e84acf
Change index settings in mappings for the new archive
jennypavlova Feb 3, 2025
920a94b
Merge branch 'main' into 184582-infra-running-processes-missing-from-…
jennypavlova Feb 3, 2025
568ed71
Change index settings in mappings for the new archive
jennypavlova Feb 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion x-pack/solutions/observability/plugins/infra/common/constants.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ export const TIEBREAKER_FIELD = '_doc';
export const HOST_NAME_FIELD = 'host.name';
export const CONTAINER_ID_FIELD = 'container.id';
export const KUBERNETES_POD_UID_FIELD = 'kubernetes.pod.uid';
export const SYSTEM_PROCESS_CMDLINE_FIELD = 'system.process.cmdline';
export const PROCESS_COMMANDLINE_FIELD = 'process.command_line';
export const EVENT_MODULE = 'event.module';
export const METRICSET_MODULE = 'metricset.module';
export const METRICSET_NAME = 'metricset.name';
Expand Down
1 change: 1 addition & 0 deletions x-pack/solutions/observability/plugins/infra/common/http_api/host_details/process_list.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ export const ProcessListAPIQueryAggregationRT = rt.type({
_source: rt.type({
process: rt.type({
pid: rt.number,
command_line: rt.string,
}),
system: rt.type({
process: rt.type({
Expand Down
4 changes: 3 additions & 1 deletion ...ility/plugins/infra/public/components/asset_details/tabs/processes/parse_search_string.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
* 2.0.
*/

import { PROCESS_COMMANDLINE_FIELD } from '../../../../../common/constants';

export const parseSearchString = (query: string) => {
if (query.trim() === '') {
return [
Expand All @@ -22,7 +24,7 @@ export const parseSearchString = (query: string) => {
return [
...cmdlineFilters.map((clause) => ({
query_string: {
fields: ['system.process.cmdline'],
fields: [PROCESS_COMMANDLINE_FIELD],
query: `*${escapeReservedCharacters(clause)}*`,
minimum_should_match: 1,
},
Expand Down
70 changes: 33 additions & 37 deletions ...ility/plugins/infra/public/components/asset_details/tabs/processes/process_row_charts.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,44 +166,40 @@ const ProcessChart = ({ timeseries, color, label }: ProcessChartProps) => {
: { max: 0, min: 0 };

return (
<div
css={css`
{
width: 100%;
height: 140px;
}
`}
<Chart
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

size={{
height: '140px',
width: '100%',
}}
>
<Chart>
<MetricExplorerSeriesChart
type={MetricsExplorerChartType.area}
metric={chartMetric}
id="0"
series={timeseries}
stack={false}
/>
<Axis
id={'timestamp'}
position={Position.Bottom}
showOverlappingTicks={true}
tickFormat={dateFormatter}
/>
<Axis
id={'values'}
position={Position.Left}
tickFormat={yAxisFormatter}
domain={domain}
ticks={6}
gridLine={{ visible: true }}
/>
<Tooltip headerFormatter={({ value }) => moment(value).format('Y-MM-DD HH:mm:ss.SSS')} />
<Settings
baseTheme={chartTheme.baseTheme}
theme={chartTheme.theme}
locale={i18n.getLocale()}
/>
</Chart>
</div>
<MetricExplorerSeriesChart
type={MetricsExplorerChartType.area}
metric={chartMetric}
id="0"
series={timeseries}
stack={false}
/>
<Axis
id={'timestamp'}
position={Position.Bottom}
showOverlappingTicks={true}
tickFormat={dateFormatter}
/>
<Axis
id={'values'}
position={Position.Left}
tickFormat={yAxisFormatter}
domain={domain}
ticks={6}
gridLine={{ visible: true }}
/>
<Tooltip headerFormatter={({ value }) => moment(value).format('Y-MM-DD HH:mm:ss.SSS')} />
<Settings
baseTheme={chartTheme.baseTheme}
theme={chartTheme.theme}
locale={i18n.getLocale()}
/>
</Chart>
);
};

Expand Down
11 changes: 8 additions & 3 deletions x-pack/solutions/observability/plugins/infra/server/lib/host_details/process_list.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
* 2.0.
*/

import { TIMESTAMP_FIELD, SYSTEM_PROCESS_CMDLINE_FIELD } from '../../../common/constants';
import { TIMESTAMP_FIELD, PROCESS_COMMANDLINE_FIELD } from '../../../common/constants';
import type {
ProcessListAPIRequest,
ProcessListAPIQueryAggregation,
Expand Down Expand Up @@ -72,7 +72,7 @@ export const getProcessList = async (
aggs: {
filteredProcs: {
terms: {
field: SYSTEM_PROCESS_CMDLINE_FIELD,
field: PROCESS_COMMANDLINE_FIELD,
size: TOP_N,
order: {
[sortBy.name]: sortBy.isAscending ? 'asc' : 'desc',
Expand Down Expand Up @@ -104,7 +104,12 @@ export const getProcessList = async (
},
},
],
_source: ['system.process.state', 'user.name', 'process.pid'],
_source: [
'system.process.state',
'user.name',
'process.pid',
'process.command_line',
],
},
},
},
Expand Down
6 changes: 3 additions & 3 deletions x-pack/solutions/observability/plugins/infra/server/lib/host_details/process_list_chart.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
*/

import { first } from 'lodash';
import { TIMESTAMP_FIELD, SYSTEM_PROCESS_CMDLINE_FIELD } from '../../../common/constants';
import { TIMESTAMP_FIELD, PROCESS_COMMANDLINE_FIELD } from '../../../common/constants';
import type {
ProcessListAPIChartRequest,
ProcessListAPIChartQueryAggregation,
Expand Down Expand Up @@ -48,7 +48,7 @@ export const getProcessListChart = async (
must: [
{
match: {
[SYSTEM_PROCESS_CMDLINE_FIELD]: command,
[PROCESS_COMMANDLINE_FIELD]: command,
},
},
],
Expand All @@ -57,7 +57,7 @@ export const getProcessListChart = async (
aggs: {
filteredProc: {
terms: {
field: SYSTEM_PROCESS_CMDLINE_FIELD,
field: PROCESS_COMMANDLINE_FIELD,
size: 1,
},
aggs: {
Expand Down
13 changes: 9 additions & 4 deletions ...test/api_integration/deployment_agnostic/apis/observability/infra/metrics_process_list.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,20 @@ export default function ({ getService }: DeploymentAgnosticFtrProviderContext) {

describe('API /api/metrics/process_list', () => {
let supertestWithAdminScope: SupertestWithRoleScopeType;

before(async () => {
supertestWithAdminScope = await roleScopedSupertest.getSupertestWithRoleScope('admin', {
withInternalHeaders: true,
useCookieHeader: true,
});
await esArchiver.load('x-pack/test/functional/es_archives/infra/8.0.0/metrics_and_apm');
await esArchiver.load(
'x-pack/test/functional/es_archives/infra/8.0.0/metrics_hosts_processes'
);
});
after(async () => {
await esArchiver.unload('x-pack/test/functional/es_archives/infra/8.0.0/metrics_and_apm');
await esArchiver.unload(
'x-pack/test/functional/es_archives/infra/8.0.0/metrics_hosts_processes'
);
await supertestWithAdminScope.destroy();
});

Expand All @@ -42,7 +47,7 @@ export default function ({ getService }: DeploymentAgnosticFtrProviderContext) {
'host.name': 'gke-observability-8--observability-8--bc1afd95-nhhw',
},
sourceId: 'default',
to: 1564432800000,
to: 1680027660000,
sortBy: {
name: 'cpu',
isAscending: false,
Expand All @@ -59,7 +64,7 @@ export default function ({ getService }: DeploymentAgnosticFtrProviderContext) {
const { processList, summary } = decodeOrThrow(ProcessListAPIResponseRT)(response.body);

expect(processList.length).to.be(10);
expect(summary.total).to.be(178);
expect(summary.total).to.be(313);
});
});
}
13 changes: 9 additions & 4 deletions ...pi_integration/deployment_agnostic/apis/observability/infra/metrics_process_list_chart.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,14 @@ export default function ({ getService }: DeploymentAgnosticFtrProviderContext) {
withInternalHeaders: true,
useCookieHeader: true,
});
await esArchiver.load('x-pack/test/functional/es_archives/infra/8.0.0/metrics_and_apm');
await esArchiver.load(
'x-pack/test/functional/es_archives/infra/8.0.0/metrics_hosts_processes'
);
});
after(async () => {
await esArchiver.unload('x-pack/test/functional/es_archives/infra/8.0.0/metrics_and_apm');
await esArchiver.unload(
'x-pack/test/functional/es_archives/infra/8.0.0/metrics_hosts_processes'
);
await supertestWithAdminScope.destroy();
});

Expand All @@ -43,8 +47,9 @@ export default function ({ getService }: DeploymentAgnosticFtrProviderContext) {
'host.name': 'gke-observability-8--observability-8--bc1afd95-nhhw',
},
indexPattern: 'metrics-*,metricbeat-*',
to: 1564432800000,
command: '/usr/lib/systemd/systemd-journald',
to: 1680027660000,
command:
'/System/Library/CoreServices/NotificationCenter.app/Contents/MacOS/NotificationCenter',
})
)
.expect(200);
Expand Down
Binary file added x-pack/test/functional/es_archives/infra/8.0.0/metrics_hosts_processes/data.json.gz
View file
Open in desktop
Binary file not shown.
Loading