Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Endpoint] resolver v1 events #59233

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
a9fcd64
Unifying the test index name for resolver and alerts
jonathan-buttner Mar 2, 2020
4223a42
Endpoint isn't sending the agent field so check for it
jonathan-buttner Mar 2, 2020
8562b20
Merge remote-tracking branch 'buttner/feature/single-event-index' int…
kqualters-elastic Mar 3, 2020
940959e
Update resolver to use either legacy or ecs events
kqualters-elastic Mar 3, 2020
692a945
Use correct format for child events api
kqualters-elastic Mar 3, 2020
fa4c139
Poorly handle empty child events response
kqualters-elastic Mar 6, 2020
479f813
Adding string or array for category and type
jonathan-buttner Mar 6, 2020
d9db350
Merge pull request #1 from jonathan-buttner/add-array
kqualters-elastic Mar 6, 2020
ca85aa4
Add return types to process event models
kqualters-elastic Mar 6, 2020
c160677
Merge remote-tracking branch 'upstream/master' into task/resolver-v1-…
kqualters-elastic Mar 9, 2020
084ac79
Merge branch 'task/resolver-v1-events' of https://github.com/kqualter…
kqualters-elastic Mar 10, 2020
5e06e8d
Create a common/models.ts for common event logic
kqualters-elastic Mar 10, 2020
971866a
Merge remote-tracking branch 'upstream/master' into task/resolver-v1-…
kqualters-elastic Mar 10, 2020
e5c0a1d
Decrease resolver min height
kqualters-elastic Mar 10, 2020
ed6c269
Merge remote-tracking branch 'upstream/master' into task/resolver-v1-…
kqualters-elastic Mar 11, 2020
aa3912f
Merge branch 'master' into task/resolver-v1-events
kqualters-elastic Mar 12, 2020
3eea057
Merge branch 'master' into task/resolver-v1-events
kqualters-elastic Mar 13, 2020
074bba0
Merge branch 'master' into task/resolver-v1-events
kqualters-elastic Mar 16, 2020
7b13c14
Update types to match cli tool
kqualters-elastic Mar 16, 2020
7b13e0e
Merge branch 'master' into task/resolver-v1-events
kqualters-elastic Mar 17, 2020
9e4a829
Add a smoke test for resolver rendering nodes, remove unused selector
kqualters-elastic Mar 17, 2020
840911c
Make parent process name optional
kqualters-elastic Mar 17, 2020
f38635c
Remove parent.parent from endpoint event
kqualters-elastic Mar 17, 2020
ef7b568
Add common/models/event
kqualters-elastic Mar 18, 2020
bcf1035
Internationalize some strings, address pr comments
kqualters-elastic Mar 18, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions x-pack/plugins/endpoint/common/models/event.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { EndpointEvent, LegacyEndpointEvent } from '../types';

export function isLegacyEvent(
event: EndpointEvent | LegacyEndpointEvent
): event is LegacyEndpointEvent {
return (event as LegacyEndpointEvent).endgame !== undefined;
}

export function eventTimestamp(
event: EndpointEvent | LegacyEndpointEvent
): string | undefined | number {
if (isLegacyEvent(event)) {
return event.endgame.timestamp_utc;
} else {
return event['@timestamp'];
}
}

export function eventName(event: EndpointEvent | LegacyEndpointEvent): string {
if (isLegacyEvent(event)) {
return event.endgame.process_name ? event.endgame.process_name : '';
} else {
return event.process.name;
}
}
5 changes: 3 additions & 2 deletions x-pack/plugins/endpoint/common/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -311,8 +311,8 @@ export interface EndpointEvent {
version: string;
};
event: {
category: string;
type: string;
category: string | string[];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ Should this stuff get typed as readonly

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my suggestion is to leave the types in types.ts as plain as possible. this way, you could use them when constructing a EndpointEvent. If you need a readonly (or partial, or deepreadonly, or deeppartial, etc) type, you can use a generic type for it. e.g. function takesReadonlyEvent(event: DeepReadonly<EndpointEvent>) {

type: string | string[];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when would category and type be string arrays?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure exactly when this is the case, I believe some events can have more than one of each in the case where it was hard to bucket into exactly one existing ECS value. @jonathan-buttner could probably tell you in more detail, this comes from kqualters-elastic#1

id: string;
kind: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ At a glance, we have three things (category, type, and now kind) that are basically perfect synonyms. Can we do something to make this type more readable or useful? Either changing from string to the things that can actually be there, or documenting the differences?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bkimmel I can link you to the docs that describe the differences. We could use enums I suppose if we wanted but we'd have to update them as ecs changes. I think it'd probably go better the document the differences route.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bkimmel long term we need a way to keep these types in sync w/ ECS. Part of that should be getting descriptions of each field. In the meantime, getting comments on each field here is on my radar.

};
Expand All @@ -328,6 +328,7 @@ export interface EndpointEvent {
name: string;
parent?: {
entity_id: string;
name?: string;
};
};
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -165,15 +165,3 @@ export const hasSelectedAlert: (state: AlertListState) => boolean = createSelect
uiQueryParams,
({ selected_alert: selectedAlert }) => selectedAlert !== undefined
);

/**
* Determine if the alert event is most likely compatible with LegacyEndpointEvent.
*/
export const selectedAlertIsLegacyEndpointEvent: (
state: AlertListState
) => boolean = createSelector(selectedAlertDetailsData, function(event) {
if (event === undefined) {
return false;
}
return 'endgame' in event;
});
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
* you may not use this file except in compliance with the Elastic License.
*/
import React, { memo, useMemo } from 'react';
import styled from 'styled-components';
import { i18n } from '@kbn/i18n';
import { FormattedMessage } from '@kbn/i18n/react';
import {
Expand All @@ -19,87 +20,104 @@ import * as selectors from '../../../../store/alerts/selectors';
import { MetadataPanel } from './metadata_panel';
import { FormattedDate } from '../../formatted_date';
import { AlertDetailResolver } from '../../resolver';
import { ResolverEvent } from '../../../../../../../common/types';
import { TakeActionDropdown } from './take_action_dropdown';

export const AlertDetailsOverview = memo(() => {
const alertDetailsData = useAlertListSelector(selectors.selectedAlertDetailsData);
if (alertDetailsData === undefined) {
return null;
}
const selectedAlertIsLegacyEndpointEvent = useAlertListSelector(
selectors.selectedAlertIsLegacyEndpointEvent
);
export const AlertDetailsOverview = styled(
memo(() => {
const alertDetailsData = useAlertListSelector(selectors.selectedAlertDetailsData);
if (alertDetailsData === undefined) {
return null;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please remove the selectedAlertIsLegacyEndpointEvent selector if it's no longer used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


const tabs: EuiTabbedContentTab[] = useMemo(() => {
return [
{
id: 'overviewMetadata',
'data-test-subj': 'overviewMetadata',
name: i18n.translate(
'xpack.endpoint.application.endpoint.alertDetails.overview.tabs.overview',
{
defaultMessage: 'Overview',
}
),
content: (
<>
<EuiSpacer />
<MetadataPanel />
</>
),
},
{
id: 'overviewResolver',
name: i18n.translate(
'xpack.endpoint.application.endpoint.alertDetails.overview.tabs.resolver',
{
defaultMessage: 'Resolver',
}
),
content: (
<>
<EuiSpacer />
{selectedAlertIsLegacyEndpointEvent && <AlertDetailResolver />}
</>
),
},
];
}, [selectedAlertIsLegacyEndpointEvent]);
const tabs: EuiTabbedContentTab[] = useMemo(() => {
return [
{
id: 'overviewMetadata',
'data-test-subj': 'overviewMetadata',
name: i18n.translate(
'xpack.endpoint.application.endpoint.alertDetails.overview.tabs.overview',
{
defaultMessage: 'Overview',
}
),
content: (
<>
<EuiSpacer />
<MetadataPanel />
</>
),
},
{
id: 'overviewResolver',
'data-test-subj': 'overviewResolverTab',
name: i18n.translate(
'xpack.endpoint.application.endpoint.alertDetails.overview.tabs.resolver',
{
defaultMessage: 'Resolver',
}
),
content: (
<>
<EuiSpacer />
<AlertDetailResolver selectedEvent={(alertDetailsData as unknown) as ResolverEvent} />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this unsafe cast needed? can we work out a way to remove it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo we should get rid of the AlertData type, as it's really just a variation of an EndpointEvent

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does removing the cast change anything then?

Copy link
Contributor

@peluja1012 peluja1012 Mar 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oatkiller @kqualters-elastic @marshallmain ResolverEvent is defined as:

ResolverEvent = EndpointEvent | LegacyEndpointEvent;

AlertData, which alertDetailsData is a type of, is defined as:

export type AlertData = AlertEvent & AlertMetadata;

We have malware_event, network, process, and registry ECS schemas defined here: https://github.com/elastic/endpoint-app-team/tree/master/schemas/v1

Does it make sense to model our types after these schemas? If so, I think our AlertEvent type should follow the malware_event ECS and our EndpointEvent type should probably be a union of a new ProcessEvent type and our existing AlertEvent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes agreed, wouldn't hurt to support LegacyEndpointEvent as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that adds up. That being said, if malware_event, network, process, and registry are in the same index, I don't see why they shouldn't be the same schema. In fact, I don't see why ECS shouldn't just be a single schema. Its not Elastic Common Schemas is it?

</>
),
},
];
}, [alertDetailsData]);

return (
<>
<section className="details-overview-summary">
<EuiTitle size="s">
<h3>
return (
<>
<section className="details-overview-summary">
<EuiTitle size="s">
<h3>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ I think the h3 might be ( ❔ ) redundant but maybe you have to use the size="l" variant of EUITitle

<FormattedMessage
id="xpack.endpoint.application.endpoint.alertDetails.overview.title"
defaultMessage="Detected Malicious File"
/>
</h3>
</EuiTitle>
<EuiSpacer />
<EuiText>
<p>
<FormattedMessage
id="xpack.endpoint.application.endpoint.alertDetails.overview.summary"
defaultMessage="MalwareScore detected the opening of a document on {hostname} on {date}"
values={{
hostname: alertDetailsData.host.hostname,
date: <FormattedDate timestamp={alertDetailsData['@timestamp']} />,
}}
/>
</p>
</EuiText>
<EuiSpacer />
<EuiText>
Endpoint Status:{' '}
<EuiHealth color="success">
{' '}
<FormattedMessage
id="xpack.endpoint.application.endpoint.alertDetails.endpoint.status.online"
defaultMessage="Online"
/>
</EuiHealth>
</EuiText>
<EuiText>
{' '}
<FormattedMessage
id="xpack.endpoint.application.endpoint.alertDetails.overview.title"
defaultMessage="Detected Malicious File"
id="xpack.endpoint.application.endpoint.alertDetails.alert.status.open"
defaultMessage="Alert Status: Open"
/>
</h3>
</EuiTitle>
<EuiSpacer />
<EuiText>
<p>
<FormattedMessage
id="xpack.endpoint.application.endpoint.alertDetails.overview.summary"
defaultMessage="MalwareScore detected the opening of a document on {hostname} on {date}"
values={{
hostname: alertDetailsData.host.hostname,
date: <FormattedDate timestamp={alertDetailsData['@timestamp']} />,
}}
/>
</p>
</EuiText>
<EuiSpacer />
<EuiText>
Endpoint Status: <EuiHealth color="success">Online</EuiHealth>
</EuiText>
<EuiText>Alert Status: Open</EuiText>
<EuiSpacer />
<TakeActionDropdown />
<EuiSpacer />
</section>
<EuiTabbedContent tabs={tabs} initialSelectedTab={tabs[0]} />
</>
);
});
</EuiText>
<EuiSpacer />
<TakeActionDropdown />
<EuiSpacer />
</section>
<EuiTabbedContent tabs={tabs} initialSelectedTab={tabs[0]} />
</>
);
})
)`
height: 100%;
width: 100%;
`;
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ import { Provider } from 'react-redux';
import { useKibana } from '../../../../../../../../src/plugins/kibana_react/public';
import { Resolver } from '../../../../embeddables/resolver/view';
import { EndpointPluginServices } from '../../../../plugin';
import { LegacyEndpointEvent } from '../../../../../common/types';
import { ResolverEvent } from '../../../../../common/types';
import { storeFactory } from '../../../../embeddables/resolver/store';

export const AlertDetailResolver = styled(
React.memo(
({ className, selectedEvent }: { className?: string; selectedEvent?: LegacyEndpointEvent }) => {
({ className, selectedEvent }: { className?: string; selectedEvent?: ResolverEvent }) => {
const context = useKibana<EndpointPluginServices>();
const { store } = storeFactory(context);

Expand All @@ -33,4 +33,5 @@ export const AlertDetailResolver = styled(
width: 100%;
display: flex;
flex-grow: 1;
min-height: 500px;
`;
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@

import { uniquePidForProcess, uniqueParentPidForProcess } from './process_event';
import { IndexedProcessTree } from '../types';
import { LegacyEndpointEvent } from '../../../../common/types';
import { ResolverEvent } from '../../../../common/types';
import { levelOrder as baseLevelOrder } from '../lib/tree_sequencers';

/**
* Create a new IndexedProcessTree from an array of ProcessEvents
*/
export function factory(processes: LegacyEndpointEvent[]): IndexedProcessTree {
const idToChildren = new Map<number | undefined, LegacyEndpointEvent[]>();
const idToValue = new Map<number, LegacyEndpointEvent>();
export function factory(processes: ResolverEvent[]): IndexedProcessTree {
const idToChildren = new Map<string | undefined, ResolverEvent[]>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗 Cool, I didn't know you could use Type directions in constructors that way.

const idToValue = new Map<string, ResolverEvent>();

for (const process of processes) {
idToValue.set(uniquePidForProcess(process), process);
Expand All @@ -36,10 +36,7 @@ export function factory(processes: LegacyEndpointEvent[]): IndexedProcessTree {
/**
* Returns an array with any children `ProcessEvent`s of the passed in `process`
*/
export function children(
tree: IndexedProcessTree,
process: LegacyEndpointEvent
): LegacyEndpointEvent[] {
export function children(tree: IndexedProcessTree, process: ResolverEvent): ResolverEvent[] {
const id = uniquePidForProcess(process);
const processChildren = tree.idToChildren.get(id);
return processChildren === undefined ? [] : processChildren;
Expand All @@ -50,8 +47,8 @@ export function children(
*/
export function parent(
tree: IndexedProcessTree,
childProcess: LegacyEndpointEvent
): LegacyEndpointEvent | undefined {
childProcess: ResolverEvent
): ResolverEvent | undefined {
const uniqueParentPid = uniqueParentPidForProcess(childProcess);
if (uniqueParentPid === undefined) {
return undefined;
Expand All @@ -74,7 +71,7 @@ export function root(tree: IndexedProcessTree) {
if (size(tree) === 0) {
return null;
}
let current: LegacyEndpointEvent = tree.idToProcess.values().next().value;
let current: ResolverEvent = tree.idToProcess.values().next().value;
while (parent(tree, current) !== undefined) {
current = parent(tree, current)!;
}
Expand Down
Loading