From b952d183403044073c4dcd299305101e6cd2fa7f Mon Sep 17 00:00:00 2001 From: bui Date: Wed, 3 Apr 2024 17:53:15 +0200 Subject: [PATCH 1/4] fix most of #512 issues --- crowdsec-docs/docs/appsec/configuration.md | 6 +- crowdsec-docs/docs/appsec/hooks.md | 116 +++++++++++++-------- crowdsec-docs/docs/appsec/rules_syntax.md | 2 +- 3 files changed, 75 insertions(+), 49 deletions(-) diff --git a/crowdsec-docs/docs/appsec/configuration.md b/crowdsec-docs/docs/appsec/configuration.md index 0320f350..0ee19888 100644 --- a/crowdsec-docs/docs/appsec/configuration.md +++ b/crowdsec-docs/docs/appsec/configuration.md @@ -40,15 +40,15 @@ A supplementary list of rules can be loaded during the out-of-band phase. These ### `inband_rules` -An optional list of rules to be loaded in inband phase. In band rules are blocking and evaluated before answering to the remediation component. Useful for virtual patching, rules with no/low false positives. +An optional list of rules to be loaded in inband phase. In band rules are blocking and evaluated before answering the remediation component. Useful for virtual patching, rules with no/low false positives. ### `default_remediation` -An optional remediation for inband rules, defaults to `block`. +An optional remediation for inband rules, defaults to `ban`. If set to `allow`, remediation component won't block the request (even if it matched rules). Any other value (including `captcha`) is passed as-is back to the remediation component. ### `default_pass_action` -An optional remediation for requests that didn't match any rules (or rules with a pass action). Defaults to nothing. +An optional remediation for requests that didn't match any rules (or rules with a pass action). Defaults to `allow`. Any other value will be passed as-is to the remediation component. ### `blocked_http_code` diff --git a/crowdsec-docs/docs/appsec/hooks.md b/crowdsec-docs/docs/appsec/hooks.md index 9a5b0275..5c3fcd26 100644 --- a/crowdsec-docs/docs/appsec/hooks.md +++ b/crowdsec-docs/docs/appsec/hooks.md @@ -16,12 +16,27 @@ The three phases are: Hooks are configured in your `appsec-config` file. -Except for the `on_load` hook, all hooks support a `filter` parameter and an `apply` parameter (`on_load` only has `apply`). +`on_load` hook only supports `apply`, while other hooks support `filter` and `apply` parameters. Both `filter` and `apply` of the same phase have access to the same helpers. Except for `on_load`, hooks can be called twice per request: once for in-band processing and once for out-of-band processing, thus it is recommended to use the `IsInBand` and `IsOutBand` variables to filter the hook. + +Hooks have the following format: + +```yaml +filter: 1 == 1 +apply: + - valid expression + - valid expression +``` + +If the filter returns `true`, each of the expressions in the `apply` section are executed. + + + + ### `on_load` This hook is intended to be used to disable rules at loading (eg, to temporarily disable a rule that is causing false positives). @@ -29,17 +44,17 @@ This hook is intended to be used to disable rules at loading (eg, to temporarily #### Available helpers -| Helper Name | Type | Description | -| --- | --- | --- | -| `RemoveInBandRuleByName` | `func(tag str)` | Disable the named in-band rule | -| `RemoveInBandRuleByTag` | `func(tag str)` | Disable the in-band rule identified by the tag (multiple rules can have the same tag) | -| `RemoveInBandRuleByID` | `func(id int)` | Disable the in-band rule identified by the ID | -| `RemoveOutBandRuleByName` | `func(tag str)` | Disable the named out-of-band rule | -| `RemoveOutBandRuleByTag` | `func(tag str)` | Disable the out-of-band rule identified by the tag (multiple rules can have the same tag) | -| `RemoveOutBandRuleByID` | `func(id int)` | Disable the out-of-band rule identified by the ID | -| `SetRemediationByTag` | `func(tag str, remediation string)` | Change the remediation of the in-band rule identified by the tag (multiple rules can have the same tag) | -| `SetRemediationByID` | `func(id int, remediation string)` | Change the remediation of the in-band rule identified by the ID | -| `SetRemediationByName` | `func(name str, remediation string)` | Change the remediation of the in-band rule identified by the name | +| Helper Name | Type | Description | +| ------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------- | +| `RemoveInBandRuleByName` | `func(tag str)` | Disable the named in-band rule | +| `RemoveInBandRuleByTag` | `func(tag str)` | Disable the in-band rule identified by the tag (multiple rules can have the same tag) | +| `RemoveInBandRuleByID` | `func(id int)` | Disable the in-band rule identified by the ID | +| `RemoveOutBandRuleByName` | `func(tag str)` | Disable the named out-of-band rule | +| `RemoveOutBandRuleByTag` | `func(tag str)` | Disable the out-of-band rule identified by the tag (multiple rules can have the same tag) | +| `RemoveOutBandRuleByID` | `func(id int)` | Disable the out-of-band rule identified by the ID | +| `SetRemediationByTag` | `func(tag str, remediation string)` | Change the remediation of the in-band rule identified by the tag (multiple rules can have the same tag) | +| `SetRemediationByID` | `func(id int, remediation string)` | Change the remediation of the in-band rule identified by the ID | +| `SetRemediationByName` | `func(name str, remediation string)` | Change the remediation of the in-band rule identified by the name | ##### Example @@ -62,20 +77,20 @@ This hook is intended to be used to disable rules only for this particular reque #### Available helpers -| Helper Name | Type | Description | -| --- | --- | --- | -| `RemoveInBandRuleByName` | `func(tag str)` | Disable the named in-band rule | -| `RemoveInBandRuleByTag` | `func(tag str)` | Disable the in-band rule identified by the tag (multiple rules can have the same tag) | -| `RemoveInBandRuleByID` | `func(id int)` | Disable the in-band rule identified by the ID | -| `RemoveOutBandRuleByName` | `func(tag str)` | Disable the named out-of-band rule | -| `RemoveOutBandRuleByTag` | `func(tag str)` | Disable the out-of-band rule identified by the tag (multiple rules can have the same tag) | -| `RemoveOutBandRuleByID` | `func(id int)` | Disable the out-of-band rule identified by the ID | -| `IsInBand` | `bool` | `true` if the request is in the in-band processing phase | -| `IsOutBand` | `bool` | `true` if the request is in the out-of-band processing phase | -| `SetRemediationByTag` | `func(tag str, remediation string)` | Change the remediation of the in-band rule identified by the tag (multiple rules can have the same tag) | -| `SetRemediationByID` | `func(id int, remediation string)` | Change the remediation of the in-band rule identified by the ID | -| `SetRemediationByName` | `func(name str, remediation string)` | Change the remediation of the in-band rule identified by the name | -| `req` | `http.Request` | Original HTTP request received by the remediation component | +| Helper Name | Type | Description | +| ------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------- | +| `RemoveInBandRuleByName` | `func(tag str)` | Disable the named in-band rule | +| `RemoveInBandRuleByTag` | `func(tag str)` | Disable the in-band rule identified by the tag (multiple rules can have the same tag) | +| `RemoveInBandRuleByID` | `func(id int)` | Disable the in-band rule identified by the ID | +| `RemoveOutBandRuleByName` | `func(tag str)` | Disable the named out-of-band rule | +| `RemoveOutBandRuleByTag` | `func(tag str)` | Disable the out-of-band rule identified by the tag (multiple rules can have the same tag) | +| `RemoveOutBandRuleByID` | `func(id int)` | Disable the out-of-band rule identified by the ID | +| `IsInBand` | `bool` | `true` if the request is in the in-band processing phase | +| `IsOutBand` | `bool` | `true` if the request is in the out-of-band processing phase | +| `SetRemediationByTag` | `func(tag str, remediation string)` | Change the remediation of the in-band rule identified by the tag (multiple rules can have the same tag) | +| `SetRemediationByID` | `func(id int, remediation string)` | Change the remediation of the in-band rule identified by the ID | +| `SetRemediationByName` | `func(name str, remediation string)` | Change the remediation of the in-band rule identified by the name | +| `req` | `http.Request` | Original HTTP request received by the remediation component | #### Example @@ -96,12 +111,12 @@ pre_eval: This hook is mostly intended for debugging or threat-hunting purposes. #### Available helpers -| Helper Name | Type | Description | -| --- | --- | --- | -| `IsInBand` | `bool` | `true` if the request is in the in-band processing phase | -| `IsOutBand` | `bool` | `true` if the request is in the out-of-band processing phase | -| `DumpRequest` | `func()` | Dump the request to a file | -| `req` | `http.Request` | Original HTTP request received by the remediation component | +| Helper Name | Type | Description | +| ------------- | -------------- | ------------------------------------------------------------ | +| `IsInBand` | `bool` | `true` if the request is in the in-band processing phase | +| `IsOutBand` | `bool` | `true` if the request is in the out-of-band processing phase | +| `DumpRequest` | `func()` | Dump the request to a file | +| `req` | `http.Request` | Original HTTP request received by the remediation component | #### DumpRequest @@ -149,19 +164,19 @@ This hook is intended to be used to change the behavior of the engine after a ma #### Available helpers -| Helper Name | Type | Description | -| --- | --- | --- | +| Helper Name | Type | Description | +| ---------------- | -------------------------- | ------------------------------------------------------------------------- | | `SetRemediation` | `func(remediation string)` | Change the remediation that will be returned to the remediation component | -| `SetReturnCode` | `func(code int)` | Change the HTTP code that will be returned to the remediation component | -| `CancelAlert` | `func()` | Prevent the Application Security Component to create a crowdsec alert | -| `SendAlert` | `func()` | Force the Application Security Component to create a crowdsec alert | -| `CancelEvent` | `func()` | Prevent the Application Security Component to create a crowdsec event | -| `SendEvent` | `func()` | Force the Application Security Component to create a crowdsec event | -| `DumpRequest` | `func()` | Dump the request to a file (see previous section for detailed usage) | -| `IsInBand` | `bool` | `true` if the request is in the in-band processing phase | -| `IsOutBand` | `bool` | `true` if the request is in the out-of-band processing phase | -| `evt` | `types.Event` | The event that has been generated by the Application Security Component | -| `req` | `http.Request` | Original HTTP request received by the remediation component | +| `SetReturnCode` | `func(code int)` | Change the HTTP code that will be returned to the remediation component | +| `CancelAlert` | `func()` | Prevent the Application Security Component to create a crowdsec alert | +| `SendAlert` | `func()` | Force the Application Security Component to create a crowdsec alert | +| `CancelEvent` | `func()` | Prevent the Application Security Component to create a crowdsec event | +| `SendEvent` | `func()` | Force the Application Security Component to create a crowdsec event | +| `DumpRequest` | `func()` | Dump the request to a file (see previous section for detailed usage) | +| `IsInBand` | `bool` | `true` if the request is in the in-band processing phase | +| `IsOutBand` | `bool` | `true` if the request is in the out-of-band processing phase | +| `evt` | `types.Event` | The event that has been generated by the Application Security Component | +| `req` | `http.Request` | Original HTTP request received by the remediation component | #### Example @@ -176,4 +191,15 @@ post_eval: apply: - CancelAlert() - CancelEvent() -``` \ No newline at end of file +``` + +## Detailed Helpers Information + +### `SetRemediation*` + +When using `SetRemediation*` helpers, the only special value is `allow`: the remediation component won't block the request. +Any other values (including `ban` and `captcha`) are transmitted as-is to the remediation component. + + + + diff --git a/crowdsec-docs/docs/appsec/rules_syntax.md b/crowdsec-docs/docs/appsec/rules_syntax.md index 6688b5e8..4c8a3640 100644 --- a/crowdsec-docs/docs/appsec/rules_syntax.md +++ b/crowdsec-docs/docs/appsec/rules_syntax.md @@ -101,7 +101,7 @@ Match provides the pattern to match the target against, including optional trans - _(mandatory)_ `type` indicates the matching method, one of: - `regex`: matches _target_ against value (_value_ is a RE2 regexp) - - `equal`: _target_ is a string equal to _value_ + - `equals`: _target_ is a string equal to _value_ - `startsWith`: _target_ starts with _value_ - `endsWith`: _target_ ends with _value_ - `contains`: _target_ contains value From 10e99374cd9c15cefe9639de0d8e13ac28395e47 Mon Sep 17 00:00:00 2001 From: bui Date: Wed, 3 Apr 2024 18:11:07 +0200 Subject: [PATCH 2/4] clarification --- crowdsec-docs/docs/appsec/hooks.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/crowdsec-docs/docs/appsec/hooks.md b/crowdsec-docs/docs/appsec/hooks.md index 5c3fcd26..3385bd2e 100644 --- a/crowdsec-docs/docs/appsec/hooks.md +++ b/crowdsec-docs/docs/appsec/hooks.md @@ -26,10 +26,11 @@ Except for `on_load`, hooks can be called twice per request: once for in-band pr Hooks have the following format: ```yaml -filter: 1 == 1 -apply: - - valid expression - - valid expression +on_match: + - filter: 1 == 1 + apply: + - valid expression + - valid expression ``` If the filter returns `true`, each of the expressions in the `apply` section are executed. From 0d83b229c5471ac3ed9f792be39e14d498655565 Mon Sep 17 00:00:00 2001 From: bui Date: Thu, 4 Apr 2024 09:50:53 +0200 Subject: [PATCH 3/4] document user_blocked_http_code + user_passed_http_code --- crowdsec-docs/docs/appsec/configuration.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/crowdsec-docs/docs/appsec/configuration.md b/crowdsec-docs/docs/appsec/configuration.md index 0ee19888..753840a9 100644 --- a/crowdsec-docs/docs/appsec/configuration.md +++ b/crowdsec-docs/docs/appsec/configuration.md @@ -58,6 +58,14 @@ The HTTP code to return to the remediation component when a request should be bl The HTTP code to return to the remediation component when a request should not be blocked. Defaults to `200` +### `user_blocked_http_code` + +The HTTP code to return to the final client when a request should be blocked. Defaults to `403` + +### `user_passed_http_code` + +The HTTP code to return to the final client when a request should not be blocked. Defaults to `200` + ### `on_load` See the [dedicated doc](/docs/appsec/hooks.md#on_load) From e1222370dbb7e54fd96e99bf85e0ce17dfa5c77b Mon Sep 17 00:00:00 2001 From: bui Date: Thu, 4 Apr 2024 10:15:15 +0200 Subject: [PATCH 4/4] up --- crowdsec-docs/docs/appsec/hooks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crowdsec-docs/docs/appsec/hooks.md b/crowdsec-docs/docs/appsec/hooks.md index 3385bd2e..c338bf35 100644 --- a/crowdsec-docs/docs/appsec/hooks.md +++ b/crowdsec-docs/docs/appsec/hooks.md @@ -27,7 +27,7 @@ Hooks have the following format: ```yaml on_match: - - filter: 1 == 1 + - filter: IsInBand && 1 == 1 apply: - valid expression - valid expression