Update routing-by-type.rst (#1679)

docs/index.rst

@@ -88,7 +88,7 @@ Better Prometheus Alerts (and more) for Kubernetes
         * :ref:`AI Investigation <AI Analysis>` - Kickstart your alert investigations with AI (optional)
         * :ref:`Alert Enrichment <Automatically Investigate a Prometheus Alert>` - see pods log and other data alongside your alerts
         * :ref:`Self-Healing <Remediate Prometheus Alerts>` - define auto-remediation rules for faster fixes
-        * :ref:`Advanced Routing <Notification Basics>` by team, namespace, k8s metadata and more
+        * :ref:`Advanced Routing <Defining Sinks>` by team, namespace, k8s metadata and more
         * :ref:`K8s Problem-Detection <Triggers Reference>` - alert on OOMKills or failing Jobs without PromQL
         * :ref:`Change Tracking <Track Kubernetes Changes>` - correlate alerts and Kubernetes rollouts
         * :ref:`Auto-Resolve <Jira>` - send alerts, resolve them when updated (e.g. in Jira)

docs/notification-routing/configuring-sinks.rst

@@ -1,31 +1,47 @@
 .. _sinks-overview:
 
-Notification Basics
+Defining Sinks
 ==========================
 
-Robusta can send notifications to various destinations, called sinks. For a list of all sinks, refer to :ref:`Sinks Reference`.
+Robusta can send notifications to various destinations, called sinks.
 
-Defining Sinks
-^^^^^^^^^^^^^^^^^^
+For a list of all sinks, refer to :ref:`Sinks Reference`.
+
+A Simple Sink Configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Sinks are defined in Robusta's Helm chart, using the ``sinksConfig`` value.
 
-For example, lets add a :ref:`Microsoft Teams <MS Teams sink>`:
+For example, lets add a :ref:`Microsoft Teams Sink <MS Teams sink>`:
+
+.. code-block:: yaml
+
+    sinksConfig:
+    - ms_teams_sink:                  # sink type
+        name: my_teams_sink           # name that uniquely identifies this sink in Robusta
+        webhook_url: <placeholder>    # the webhook URL for MSTeams - each sink has different parameters like this
+
+More Sink Options
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Here are some more options you can set on sinks:
 
 .. code-block:: yaml
 
     sinksConfig:
     - ms_teams_sink:                  # sink type
-        name: my_teams_sink           # arbitrary name
-        webhook_url: <placeholder>    # a sink-specific parameter
-        stop: false                   # optional (see `Routing Alerts to only one Sink`)
-        scope: {}                     # optional routing rules
-        default: true                 # optional (see below)
+        name: my_teams_sink           # name that uniquely identifies this sink in Robusta
+        stop: false                   # optional - covered in the next tutorial
+        scope: {}                     # optional - covered in the next tutorial
+        default: true                 # optional - covered in the next tutorial
+
+        # sink-specific parameters
+        webhook_url: <placeholder>
 
-In addition to the above attributes, some sinks have additional attributes that can be found in their documentation.
+The ``stop``, ``scope``, and ``default`` fields are used for :ref:`routing (scopes) <sink-scope-matching>`.
 
 Defining Multiple Sinks
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-You can define multiple sinks and by default, notifications will be sent to all of them.
+You can define multiple sinks. By default, notifications will be sent to all of them.
 
 In the following example, we define a :ref:`Slack sink <Slack>` and a :ref:`MS Teams sink <MS Teams>` without any routing rules, so both sinks receive all notifications:
 
@@ -42,7 +58,8 @@ In the following example, we define a :ref:`Slack sink <Slack>` and a :ref:`MS T
 
 To selectively send notifications to different sinks, refer to :ref:`routing (scopes) <sink-scope-matching>`.
 
-See Also
+Learn More
 ^^^^^^^^^^^^
 
-🔔 :ref:`All Sinks <Sinks Reference>`
+* 🔔 :ref:`All Sinks <Sinks Reference>`
+* ↳ :ref:`Routing (scopes) <sink-scope-matching>`

docs/notification-routing/implementing-monitoring-shifts.rst

@@ -1,4 +1,4 @@
-Monitoring Shifts
+Route By Time of Day
 ####################
 
 In this example we'll route alerts to different support teams based on the time of day:
@@ -33,4 +33,4 @@ In this example we'll route alerts to different support teams based on the time
             - start: 12:00  # 12 PM
                 end: 23:59  # 11:59 PM
 
-For more information on time-based routing, :ref:`view <Route by Time>`
+For more information on time-based routing, view :ref:`Route by Time`.

docs/notification-routing/notification-routing-examples.rst

@@ -7,8 +7,8 @@ Routing Cookbook
 
    routing-by-namespace
    routing-by-type
-   routing-to-multiple-slack-channels
    implementing-monitoring-shifts
+   routing-to-multiple-slack-channels
    routing-exclusion
 
 In this section you'll find example configurations for common routing patterns.

docs/notification-routing/routing-by-namespace.rst

@@ -11,7 +11,7 @@ Prerequisites
 Setting Up Routing
 ----------------------
 
-Assume you have an existing sink, in this case Slack:
+Assume you have an existing sink:
 
 .. code-block:: yaml
 
@@ -21,7 +21,9 @@ Assume you have an existing sink, in this case Slack:
         slack_channel: app-notifications
         api_key: secret-key
 
-By default, the sink will receive notifications for all namespaces. Let's duplicate the sink and change only the Slack channel:
+By default, the sink will receive notifications for all namespaces.
+
+Let's create a 2nd copy of the sink and change the Slack channel:
 
 .. code-block:: yaml
 
@@ -35,7 +37,9 @@ By default, the sink will receive notifications for all namespaces. Let's duplic
         slack_channel: system-notifications
         api_key: secret-key
 
-We now have two sinks, both receiving all notifications. Restrict the notifications for each sink by adding :ref:`scopes <sink-matchers>`:
+Now we have two sinks, both receiving all notifications.
+
+The final step is to restrict notifications for each sink by adding :ref:`scopes <sink-scope-matching>`:
 
 .. code-block:: yaml
 
@@ -58,9 +62,13 @@ We now have two sinks, both receiving all notifications. Restrict the notificati
 
 Alerts will be now routed according to Kubernetes namespace.
 
-You can apply this method with as many sinks as you like. If the number of sinks is large, consider setting the channel dynamically. See instructions for :ref:`Slack <Dynamic Alert Routing>` or :ref:`MS Teams <Dynamically Route MS Teams Alerts>`.
+Routing for many namespaces
+-----------------------------------
+Using the above method, you can create one sink for each namespace.
+
+If you have a large number of namespaces, there is an alternative method you can consider: you can define a single sink and set the channel dynamically according to alert metadata. See instructions for :ref:`Slack <Dynamic Alert Routing>` or :ref:`MS Teams <Dynamically Route MS Teams Alerts>`.
 
-Troubleshooting Issues
-------------------------
+Fallback for alerts without a namespace
+--------------------------------------------------
 
-For this guide to work, alerts must be tagged with ``namespace`` metadata. It is recommended that you setup a :ref:`Fallback Sink <Stop Further Notifications>` to catch alerts that don't have a namespace.
+The above example assumes that all alerts have ``namespace`` metadata. It is recommended that you setup a :ref:`Fallback Sink <Stop Further Notifications>` to catch alerts that don't have a namespace.

docs/notification-routing/routing-by-type.rst

@@ -1,29 +1,29 @@
-Route By Type
+Route By Alert Name
 =============================
 
-By default, all Robusta notifications are sent to all :ref:`sinks <Sinks Reference>`.
+By default, Robusta notifications are sent to all :ref:`sinks <Sinks Reference>`.
 
-It can be useful to send alerts of a *specific type* to a dedicated channel.
+In this guide, you'll learn to route alerts by their name:
 
-In this guide, we'll show how to route notifications for crashing pods to a specific Slack channel. All other notifications
-will be sent to the usual channel.
+* *KubePodCrashLooping* alerts will be sent to a *#crash-alerts* channel
+* All other alerts will be sent to *#general-alerts*
 
 Prerequisites
 ----------------
 
-All least one existing :ref:`sink <Sinks Reference>` must be configured.
+All least one existing :ref:`sink <Sinks Reference>` must be configured. Below, we'll assume it's a Slack sink.
 
 Setting Up Routing
 ----------------------
 
-This guide applies to all sink types. For simplicity's sake we'll assume you have an existing Slack sink:
+Assume you have an existing Slack sink as follows:
 
 .. code-block:: yaml
 
     sinksConfig:
     - slack_sink:
-        name: slack_app_sink
-        slack_channel: main-notifications
+        name: sink1
+        slack_channel: general-alerts
         api_key: secret-key
 
 The first step is to duplicate your sink. You need two unique sinks - one for each channel:
@@ -32,43 +32,80 @@ The first step is to duplicate your sink. You need two unique sinks - one for ea
 
     sinksConfig:
     - slack_sink:
-        name: main_sink
-        slack_channel: main-notifications
+        name: sink2
+        slack_channel: crash-alerts
         api_key: secret-key
     - slack_sink:
-        name: crashloopbackoff_slack_sink
-        slack_channel: crashpod-notifications
+        name: sink1
+        slack_channel: general-alerts
         api_key: secret-key
 
-The sinks are nearly identical - only the ``name`` and ``slack_channel`` parameters vary.
+The sinks are nearly identical - only the ``name`` and ``slack_channel`` parameters vary:
 
-Now lets add a :ref:`matcher <sink-matchers>` to each sink, so it receives a subset of notifications:
+* The ``name`` field identifies this sink in Robusta and can be chosen arbitrarily - so long as it is unique between sinks
+* The ``slack_channel`` field should match a channel in your Slack account
+
+The next step is to update the configuration so that ``#crash-alerts`` receives a subset of alerts:
 
 .. code-block:: yaml
 
     sinksConfig:
     - slack_sink:
-        name: main_sink
-        slack_channel: main-notifications
+        name: sink2
+        slack_channel: crash-alerts
         api_key: secret-key
         - scope:
-            exclude:
-            # don't send notifications related to image pull backoff
-              - identifier: [ImagePullBackoff]
+            include:
+            # only send notifications for the KubePodCrashLooping alert
+            - identifier: [KubePodCrashLooping]
+    - slack_sink:
+        name: sink1
+        slack_channel: general-alerts
+        api_key: secret-key
+
+We added an :ref:`inclusion scope <sink-matchers>` for the ``#crash-alerts`` channel. To filter alerts by their name, use the ``identifier`` field which corresponds to the Prometheus alert name.
+
+One final step: we must update the default sink to exclude *KubePodCrashLooping*. You can do this two ways:
 
-   - slack_sink:
-        name: crashloopbackoff_slack_sink
-        slack_channel: crash-notifications
+**Option 1:** add an exclusion scope:
+
+.. code-block:: yaml
+
+    sinksConfig:
+    - slack_sink:
+        name: sink2
+        slack_channel: crash-alerts
         api_key: secret-key
         - scope:
             include:
-            # only send notifications related to crashing pods and CPU throttling
-              - identifier: [CrashLoopBackoff, CPUThrottlingHigh]
+            # only send notifications for the KubePodCrashLooping alert
+            - identifier: [KubePodCrashLooping]
+    - slack_sink:
+        name: sink1
+        slack_channel: general-alerts
+        api_key: secret-key
+        - scope:
+            exclude:
+            # don't send notifications for the KubePodCrashLooping alert
+            - identifier: [KubePodCrashLooping]
 
-.. note::
+**Option 2:** use ``stop: true`` to prevent alerts from propogating after a match:
 
-    For Prometheus alerts use the Alert name. Example, ``CPUThrottlingHigh``, ``KubeContainerWaiting``.
-    For other events, use the name as it appears on the Robusta timeline. Example, ``report_crash_loop`` and ``image_pull_backoff_reporter``
+.. code-block:: yaml
+
+    sinksConfig:
+    - slack_sink:
+        name: sink2
+        slack_channel: crash-alerts
+        api_key: secret-key
+        # add the following line!
+        stop: true
+        - scope:
+            include:
+            - identifier: [KubePodCrashLooping]
+    - slack_sink:
+        name: sink1
+        slack_channel: general-alerts
+        api_key: secret-key
 
-Now the ``crash-notifications`` channel will receive crashpod notifications and all other notifications will go to the
-``main-notifications`` channel.
+Whichever way you chose, now *KubePodCrashLooping* alerts are sent to ``#crash-alerts``. Other alerts go to ``#general-alerts``.

docs/notification-routing/routing-to-multiple-slack-channels.rst

@@ -22,3 +22,4 @@ In this example, we'll route alerts to different Slack channels depending on an
             include:
             - namespace: [backend]
 
+For more details, refer to :ref:`scopes <sink-scope-matching>`.