Merge branch 'master' into update-architecture-docs

docs/configuration/sinks/index.rst

@@ -41,7 +41,7 @@ By default, Robusta sends all notifications to all sinks.
 **Related Topics:**
 
 * :ref:`sinks-overview`
-* :ref:`Routing Alerts to Specific Sinks`
+* :ref:`sink-scope-matching`
 
 Available sinks
 ^^^^^^^^^^^^^^^^^^^^^

docs/configuration/sinks/slack.rst

@@ -1,24 +1,29 @@
 Slack
 #################
 
-Robusta can report issues and events in your Kubernetes cluster to Slack.
+Robusta can proxy Prometheus alerts to Slack, adding powerful features like :ref:`AI investigation <AI Analysis>`, :ref:`smart grouping <notification-grouping>` and more.
+
+.. image:: /images/robusta-slack.png
+   :width: 600px
+   :align: center
+
+Optionally, Robusta can monitor Kubernetes directly and send notifications on deployment changes or Kubernetes errors.
 
 .. warning::
 
-   If you are using the Slack sink and a Robusta version prior to 0.10.29, we **highly recommend [upgrading](https://docs.robusta.dev/master/setup-robusta/upgrade.html)**, as Slack has deprecated their older APIs. This older API will be sundown March 11, 2025. This will cause the slack sink to stop working for older versions of Robusta. 
+   If you are using the Slack sink and a Robusta version prior to 0.10.29, we **highly recommend** `upgrading <https://docs.robusta.dev/master/setup-robusta/upgrade.html>`_, as Slack has deprecated their older APIs. This older API will be sundown March 11, 2025. This will cause the slack sink to stop working for older versions of Robusta. 
 
-   Follow [these steps](https://docs.robusta.dev/master/setup-robusta/upgrade.html#helm-upgrade) to upgrade.
+   Follow `these steps <https://docs.robusta.dev/master/setup-robusta/upgrade.html#helm-upgrade>`_ to upgrade.
 
 Connecting Slack
 ------------------------------------------------
 
-When installing Robusta, run ``robusta gen-config`` and follow the prompts. This will use our `official
+When installing Robusta, run ``robusta gen-config`` and follow the prompt to create a Slack API key. This will use our `official
 Slack app <https://slack.com/apps/A0214S5PHB4-robusta?tab=more_info>`_.
 
 **Note: Robusta can only write messages and doesn't require read permissions.**
 
-Alternatively, generate a key by running ``robusta integrations slack`` and set the following Helm values in your
-``generated_values.yaml``:
+You can also generate a Slack API key by running ``robusta integrations slack`` and setting the following Helm values in your ``generated_values.yaml``:
 
 .. code-block:: yaml
 
@@ -32,28 +37,28 @@ Alternatively, generate a key by running ``robusta integrations slack`` and set
          channel_override: DYNAMIC SLACK CHANNEL OVERRIDE (Optional)
          investigate_link: true/false # optional, if false no investigate links/buttons will be included in Slack messages
 
-Then do a :ref:`Helm Upgrade <Simple Upgrade>`.
-
 .. warning::
 
     If you don't want to put your Slack key in Helm values, you can use a secret. See the :ref:`Managing Secrets <Managing Secrets>` section for more information.
 
+Then do a :ref:`Helm Upgrade <Simple Upgrade>` to apply the new configuration.
+
 Notification Grouping
 -----------------------------
-Slack allows grouping multiple notifications into summary messages and Slack threads. Refer to :ref:`Notification Grouping <notification-grouping>`.
+Slack allows grouping multiple notifications into summary messages and Slack threads. Refer to :ref:`Grouping <notification-grouping>` for details.
 
 .. image:: /images/notification-grouping.png
    :width: 600px
    :align: center
 
-Dynamic Slack Channels
+Dynamic Alert Routing
 -------------------------------------------------------------------
 
-You can set the ``Slack`` channel dynamically, based on the ``cluster name`` or a value of a specific ``label`` or ``annotation``.
+You can route alerts to different Slack channels by defining several Slack sinks. See :ref:`Route By Namespace` for an example.
 
-This can be done using the optional ``channel_override`` sink parameter.
+Alternatively, if the number of channels is large, you can define a single Slack sink and use the ``channel_override`` parameter to read read the destination channel from alert metadata.
 
-Allowed values for this parameter are:
+Allowed values for ``channel_override`` are:
 
 - ``cluster_name`` - The Slack channel will be the Robusta ``cluster_name``
 - ``labels.foo`` - The Slack channel will be taken from a ``label`` value with the key ``foo``. If no such label, the default channel will be used.
@@ -69,7 +74,7 @@ For example:
          name: main_slack_sink
          api_key: xoxb-112...
          slack_channel: my-fallback-channel
-         channel_override: "labels.slack"
+         channel_override: "labels.slack"   # read the 'slack' label from the alert and route to that channel
 
 A replacement pattern is also allowed, using ``$`` sign, before the variable.
 For cases where labels or annotations include special characters, such as ``${annotations.kubernetes.io/service-name}``, you can use the `${}` replacement pattern to represent the entire key, including special characters.
@@ -91,7 +96,6 @@ Example:
          channel_override: "$cluster_name-alerts-$labels.env-${annotations.kubernetes.io/service-name}"
 
 
-
 Using Private Channels
 -------------------------------------------------------------------
 

docs/index.rst

@@ -33,23 +33,21 @@
    :hidden:
 
    configuration/index
-   🆕 Notification Grouping <configuration/notification-grouping>
    🪄 AI Analysis - HolmesGPT <configuration/ai-analysis>
+   🔔 Sinks <configuration/sinks/index>
+   🔥 Prometheus/AlertManager <configuration/alertmanager-integration/index>
    Cost Savings - KRR <configuration/resource-recommender>
    K8s Misconfigurations - Popeye <configuration/cluster-misconfigurations>
-   configuration/configuring-sinks
-   🔔 Sinks Reference <configuration/sinks/index>
-   configuration/alertmanager-integration/index
    configuration/exporting/exporting-data
-   configuration/additional-settings
 
 .. toctree::
    :maxdepth: 4
    :caption: 🔔 Notifications & Routing
    :hidden:
 
    notification-routing/configuring-sinks
-   🆕 Notification Grouping <notification-routing/notification-grouping>
+   Routing (Scopes) <notification-routing/routing-with-scopes>
+   Grouping (Slack Threads) <notification-routing/notification-grouping>
    notification-routing/routing-by-namespace
    notification-routing/routing-by-type
    notification-routing/routing-by-time

docs/notification-routing/configuring-sinks.rst

@@ -1,15 +1,15 @@
 .. _sinks-overview:
 
-Notification 101
+Notification Basics
 ==========================
 
-Robusta can send notifications to various destinations, called sinks. These notifications are enriched with issue logs, graphs, :ref:`AI Analysis` and more.
-
-The following guide is explains general settings for all sinks, please take a look at the specific sink guide for detailed instructions. For a list of all sinks, refer to :ref:`Sinks Reference`.
+Robusta can send notifications to various destinations, called sinks. For a list of all sinks, refer to :ref:`Sinks Reference`.
 
 Defining Sinks
 ^^^^^^^^^^^^^^^^^^
-Sinks are defined in Robusta's Helm chart, using the ``sinksConfig`` value:
+Sinks are defined in Robusta's Helm chart, using the ``sinksConfig`` value.
+
+For example, lets add a :ref:`Microsoft Teams <MS Teams sink>`:
 
 .. code-block:: yaml
 
@@ -21,203 +21,28 @@ Sinks are defined in Robusta's Helm chart, using the ``sinksConfig`` value:
         scope: {}                     # optional routing rules
         default: true                 # optional (see below)
 
-To add a sink, update ``sinksConfig`` according to the instructions in :ref:`Sinks Reference`. Then do a :ref:`Helm Upgrade <Simple Upgrade>`.
-
-Integrate as many sinks as you like.
-
-.. _sink-matchers:
-
-Routing Alerts to Only One Sink
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Many sinks have unique parameters which can be found under :ref:`Sinks Reference`.
 
-By default, alerts are sent to all sinks that matches the alerts.
+Defining Multiple Sinks
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+You can define multiple sinks and by default, notifications will be sent to all of them.
 
-To prevent sending alerts to more sinks after the current one, you can specify ``stop: true``
+If you'd like to selectively send notifications to different sinks, you can define :ref:`routing rules <sink-scope-matching>`.
 
-The sinks evaluation order, is the order defined in ``generated_values.yaml``.
+In the following example, we define a :ref:`Slack sink <Slack>` and a :ref:`MS Teams sink <MS Teams>` without any routing rules, so both sinks receive all notifications:
 
 .. code-block:: yaml
 
     sinksConfig:
     - slack_sink:
-        name: production_sink
-        slack_channel: production-notifications
+        name: my_slack_sink
+        slack_channel: my-channel
         api_key: secret-key
-        scope:
-          include:
-            - namespace: production
-        stop: true
-
-.. _sink-scope-matching:
-
-Routing Alerts To Specific Sinks
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Define which messages a sink accepts using ``scope``.
-
-For example, **Slack**  can be integrated to receive high-severity messages in a specific namespace. Other messages will not be sent to this **Slack** sink.
-
-.. code-block:: yaml
-
-    sinksConfig:
-    - slack_sink:
-        name: test_sink
-        slack_channel: test-notifications
-        api_key: secret-key
-        scope:
-          include: # more options available - see below
-            - namespace: [prod]
-              severity: HIGH
-
-Each attribute expression used in the ``scope`` specification can be 1 item, or a list, where each is either a `regex <https://docs.python.org/3/library/re.html#re.match>`_ or an exact match.
-
-``Scope`` allows specifying a set of ``include`` and ``exclude`` sections:
-
-.. code-block:: yaml
-
-    sinksConfig:
-    - slack_sink:
-        name: prod_slack_sink
-        slack_channel: prod-notifications
-        api_key: secret-key
-        scope:
-        # AND between namespace and labels, but OR within each selector
-          include:
-            - namespace: default
-              labels: "instance=1,foo!=x.*"
-            - namespace: bla
-              name:
-              - foo
-              - qux
-          exclude:
-            - type: ISSUE
-              title: .*crash.*
-            - name: bar[a-z]*
-
-
-In order for a message to be sent to a ``Sink``, it must match **one of** the ``include`` sections, and **must not** match **all** the ``exclude`` sections.
-
-When multiple attributes conditions are present, all must be satisfied.
-
-.. tip::
-
-    If you're using the Robusta UI, you can test alert routing by `Simulating an alert <https://platform.robusta.dev/simulate-alert/>`_.
-
-The following attributes can be included in an ``include``/``excluded`` block:
-
-- ``title``: e.g. ``Crashing pod foo in namespace default``
-- ``name`` : the Kubernetes object name
-- ``namespace``: the Kubernetes object namespace
-- ``namespace_labels``: labels assigned to the namespace; matching these is done in the same way as matching ``labels`` (see below)
-- ``node`` : the Kubernetes node name
-- ``severity``: one of ``INFO``, ``LOW``, ``MEDIUM``, ``HIGH``
-- ``type``: one of ``ISSUE``, ``CONF_CHANGE``, ``HEALTH_CHECK``, ``REPORT``
-- ``kind``: one of ``deployment``, ``node``, ``pod``, ``job``, ``daemonset``
-- ``source``: one of ``NONE``, ``KUBERNETES_API_SERVER``, ``PROMETHEUS``, ``MANUAL``, ``CALLBACK``
-- ``identifier``: e.g. ``CrashLoopBackoff``
-- ``labels``: A comma separated list of ``key=val`` e.g. ``foo=bar,instance=123``
-- ``annotations``: A comma separated list of ``key=val`` e.g. ``app.kubernetes.io/name=prometheus``
-
-.. note::
-
-    ``labels`` and ``annotations`` are both the Kubernetes resource labels and annotations
-    (e.g. pod labels) and the Prometheus alert labels and annotations. If both contains the
-    same label/annotation, the value from the Prometheus alert is preferred.
-
-.. note::
-
-    For performance reasons, the namespace information used for matching ``namespace_labels``
-    is cached (with a default cache timeout of 30 minutes). If you change namespace labels
-    and want these changes to be immediately reflected in the sink ``scope`` matching
-    mechanism, you will need to manually restart the Robusta runner.
-
-.. details:: How do I find the ``identifier`` value to use in a match block? (deprecated)
-
-    For Prometheus alerts, it's always the alert name.
-
-    .. TODO: update after we finish our improvements here:
-    .. For builtin APIServer alerts, it can vary, but common values are ``CrashLoopBackoff``, ``ImagePullBackoff``, ``ConfigurationChange/KubernetesResource/Change``, and ``JobFailure``.
-
-    For custom playbooks, it's the value you set in :ref:`create_finding<create_finding>` under ``aggregation_key``.
-
-    Ask us in Slack if you need help.
-
-By default, every message is sent to every matching sink. To change this behaviour, you can mark a sink as :ref:`non-default <Non-default sinks>`.
-
-The top-level mechanism works as follows:
-
-#. If the notification is **excluded** by any of the sink ``scope`` excludes - drop it
-#. If the notification is **included** by any of the sink ``scope`` includes - accept it
-#. If the notification is **included** by any of the sink ``matchers`` - accept it (Deprecated)
-
-Any of (but not both) of the ``include`` and ``exclude`` may be left undefined or empty.
-An undefined/empty ``include`` section will effectively allow all alerts, and an
-undefined/empty ``exclude`` section will not exclude anything.
-
-Inside the ``include`` and ``exclude`` section, at the topmost level, the consecutive
-items act with the OR logic, meaning that it's enough to match a single item in the
-list in order to allow/reject a message. The same applies to the items listed under
-each attribute name.
-
-Within a specific ``labels`` or ``annotations`` expression, the logic is ``AND``
-
-.. code-block:: yaml
-
-    ....
-        scope:
-          include:
-            - labels: "instance=1,foo=x.*"
-    .....
-
-The above requires that the ``instance`` will have a value of ``1`` **AND** the ``foo`` label values starts with ``x``
-
-
-Alternative Routing Methods
-************************************************
-
-For :ref:`customPlaybooks <defining-playbooks>`, there is another option for routing notifications.
-
-Instead of using sink matchers, you can set the *sinks* attribute per playbook:
-
-.. code-block:: yaml
-
-    customPlaybooks:
-    - triggers:
-      - on_job_failure: {}
-      actions:
-      - create_finding:
-          aggregation_key: "JobFailure"
-          title: "Job Failed"
-      - job_info_enricher: {}
-      - job_events_enricher: {}
-      - job_pod_enricher: {}
-      sinks:
-        - "some_sink"
-        - "some_other_sink"
-
-Notifications generated this way are sent exclusively to the specified sinks. They will still be filtered by matchers.
-
-Non-Default Sinks
-*********************************
-
-To prevent a sink from receiving most notifications, you can set ``default: false``. In this case, notifications will be
-routed to the sink only from :ref:`customPlaybooks that explicitly name this sink <Alternative Routing Methods>`.
-
-Here too, matchers apply as usual and perform further filtering.
-
-
-Examples
-^^^^^^^^^^^
-
-🎓 :ref:`Route Alerts By Namespace`
-
-🎓 :ref:`Route Alerts By Type`
-
-🎓 :ref:`Routing with Exclusion Rules`
+    - ms_teams_sink:
+        name: my_teams_sink
+        webhook_url: <placeholder>
 
 See Also
 ^^^^^^^^^^^^
 
 🔔 :ref:`All Sinks <Sinks Reference>`
-
-🎓 :ref:`Silencing Alerts`

docs/notification-routing/notification-routing-examples.rst

@@ -1,8 +1,7 @@
-Examples
+Common Examples
 ===================================
 
-
-You can combine multiple routing methods to create an alerting system that gives your teams only alerts relevant to them.
+Here are examples of common routing patterns that teams use with Robusta. The goal is to send each team only the most relevant alerts.
 
 Use Case 1: Route Specific Alerts to Seperate Teams and Slack Channels
 **********************************************************************************
@@ -28,7 +27,7 @@ In this example we are going to consider two teams, with #frontend and #backend
             include:
             - namespace: [backend]
 
-For a complete list of filters, view the :ref:`Scopes documentation <Routing Alerts To Specific Sinks>`.
+For a complete list of filters, view the :ref:`Scopes documentation <sink-scope-matching>`.
 
 Use Case 2: Route Alerts To Different Support Teams Based On Time Of The Day
 **********************************************************************************
@@ -63,9 +62,9 @@ Let's see how we can route alerts between two teams with different Slack channel
                 end: 23:59  # 11:59 PM
 
 
-For more information on time-based routing, :ref:`view <Route Alerts by Time>`
+For more information on time-based routing, :ref:`view <Route by Time>`
 
-If you want to selectively apply time-based-routing to certain alerts only, then this method can be combined with :ref:`Scopes <Routing Alerts To Specific Sinks>`. For example:
+If you want to selectively apply time-based-routing to certain alerts only, then this method can be combined with :ref:`Scopes <sink-scope-matching>`. For example:
 
 .. code-block:: yaml