Direct paging of users

[Matvey-Kuk]

Today:

• we could run AG manually using slack and "/oncall"
• we could "invite" to specific AG in Slack.

I want us to be able to start new AG with Escalation on a specific person using Web, Slack, TG...

Tricky part:

• We want OnCall to help users to avoid "bad practices".
• Tagging specific user without on-call schedule is a bad practice.
• We want to make OnCall easy tagging users who are on-call and "hard" tagging users who are not.

I see it this way (Web):

• I want alongside with user picker to be a schedules picker at the very beginning.
• User picker + some message "why do I escalate to this person" (because we issue new AG.
• If user is in on-call rotations & is on-call now: we're ok with tagging.
• If user is in on-call rotations & is not on-call now: we say "this person is in schedule X, but not on-call now. Person B is on-call, do you want to tag him?".
• If user is not in on-call rotations & inside working hours: we tag but suggest using on-call schedules.
• If user is in on-call rotations & outside of working hours: we tag but complain a lot.

[Matvey-Kuk]

@vadimkerr I asked @matiasb to take this one to give you some more time to work on mobile API :)

[raphael-batte]

UI is ready

Prototype https://www.figma.com/proto/sJ4RvHvnUq6AZAiHYYiB5j/Grafana-OnCall?page-id=5155%3A107137&node-id=5168%3A106039&viewport=63%2C221%2C0.12&scaling=min-zoom&starting-point-node-id=5163%3A105248&show-proto-sidebar=1

Figma https://www.figma.com/proto/sJ4RvHvnUq6AZAiHYYiB5j/Grafana-OnCall?page-id=5155%3A107137&node-id=5168%3A106039&viewport=63%2C221%2C0.12&scaling=min-zoom&starting-point-node-id=5163%3A105248&show-proto-sidebar=1

[iskhakov]

• Consider adding option to escalate to next (other) escalation step from escalation chain
• @raphael-batte can you please add some description for @matiasb
• @raphael-batte can you add designs for slack and messengers
• is there way to choose between critical and default notifications when paging users

[raphael-batte]

Added 'escalation chain' as one of the option for paging
https://www.figma.com/proto/sJ4RvHvnUq6AZAiHYYiB5j/Grafana-OnCall?page-id=5155%3A107137&node-id=5197%3A109010&viewport=-2263%2C-3669%2C1&scaling=min-zoom&starting-point-node-id=5163%3A105248&show-proto-sidebar=1

[iskhakov]

@raphael-batte / @matiasb / @iskhakov call summary

• User list prioritisation is not going to be implemented in the first version
• User is checked for (oncall/working hours/set escalation rules) in the popup after the user is selected
• when selecting user to escalate use same component as in esclation chains user select
• when we hover over user being escalated we show tooltip with their notification policies (default or important)
• we can escalate to multiple users at the same time
• title can be edited by user when the escalation is created

[raphael-batte]

Multiple users UX/UI

• when we add multiple users all actions will be user-by-user
• we provide 2 users and additional label with counter at default component
  
• we have combined list view when component expands by click
  

[raphael-batte]

Modal

[vstpme]

Notes from our call with @matiasb:

Seems like implementing the backend for paging users and schedules should be pretty straightforward (not escalation chains though, see the first question below).

Questions:

• Do we want to support paging escalation chains in the first version? This could be tricky because there's a lot of existing logic that assumes only one escalation chain can be associated with a given alert group. So while it should be pretty easy to implement direct paging for a new alert group, it would require a lot more work to support this for existing alert groups.
• Do we want to implement direct paging for Slack, MS Teams and Telegram in the first version? This could be tricky and require some workarounds because of UI system differences for different messengers.

@matiasb feel free to comment in case I missed something 🙂

[matiasb]

Adding a few more notes from the previous discussion:

• Right now we allow to associate one escalation chain to an alert group (via channel filter), and because of how we manage snapshots/alert plan, the simple existing way to set an escalation chain for a page will be during creation (ie. adding more than one escalation chain, or setting it up after escalation is created, would be tricky)
• Enabling the "escalate to" option to any alert group should be possible, and it could eventually replace the existing invite option; makes sense?
• When adding support for messengers, besides considering their different limitations, we will need organization/team information (fyi, when designing the UX/UI)
• Since we are building more and more on top of working hours (schedules, paging), I think we should expose this in the user profile somehow.

[raphael-batte]

Combined all last updates to this UX https://www.figma.com/file/sJ4RvHvnUq6AZAiHYYiB5j/Grafana-OnCall?node-id=5520%3A116727&t=AsjOJ1E6XnGanZaw-1

[vstpme]

Here's a PR with backend and API for direct paging: #1098