Update slack group users to match the current on-call rotation.

[jghibiki]

Is your feature request related to a problem? Please describe:
Not a problem, however it would be nice if we could allow go-alert to manage the users in our support slack group, that way we could automatically match go-alert's on call rotation with who is assigned to the slack group.

Describe the solution you'd like:
When a go-alert rotation ends:

• go alert would add the new on-call users (primary and secondary rotations) to a configured slack group
• go alert would remove the old on-call users (primary and secordary rotations) from the same configured slack group

Describe alternatives you've considered:
Currently our team does this manually, however because our goalert rotation ends at midnight thursday evening. Because of this we tend to either update the slack group with the new on call rotation either the day before the rotation ends, or the morning after, potentially sending some of our slack based notifications to the wrong on call group (i.e. if someone asks for help from our @<team-name>-support group in slack.

Additional context:
N/A

Thanks so much!

[mastercactapus]

• We'd need a body of work to handle linking a user to a Slack user

Question:

• would this make sense to be per-service? (e.g., sync with Slack group for all users referenced by the EP) Or per schedule?

The schedule might be more clear/direct to implement but I could see a use-case for both.

[jghibiki]

If you were looking for my input (disregard if you weren't):

would this make sense to be per-service? (e.g., sync with Slack group for all users referenced by the EP) Or per schedule?

That seems to make a lot of sense, whoever shows up in the service's on call matches up perfectly with who we'd want in the slack group 👍

EDIT:

RE: this,

We'd need a body of work to handle linking a user to a Slack user

This looks like the endpoint you would need from the slack api: https://api.slack.com/methods/usergroups.users.update
However you would first need to lookup the slack group id by name, and then the on call rotation users slack uuids.
I am not familiar with what info you have on a go-alert user that could be used to tie to a slack user, but if the same email is shared between systems you could probably use this to lookup by email

[stale]

This issue has been automatically marked as stale because it has not had recent activity.

[bherbst]

Saw the bot marked this as stale, so I wanted to chime in and say that this is a feature my team is constantly asking for. We would love for low priority alerts sent via Slack or support requests targeted at a Slack group to go to the right on-call individual automatically.

[mastercactapus]

Since this was opened, we have added some basic slack->user linking along with on-call notifications as a schedule feature. There's still work to be done there.

It could be feasible to add the ability to update a slack group with that notification functionality.

That would put it more on the schedule vs. service side though.

[ande1mat]

our team would love this feature!

[AndersDJohnson]

We would love this feature as well! Today we end up having to manually maintain Slack groups to match GoAlert schedules on a rotating basis each week, which is a hassle and sometimes forgotten or at least delayed / out of sync. Or, we just fall back to any tools and integrations just mentioning full teams instead of only the subset of those on-call in a given week, which can be distracting and adds noise and dilutes the meaning of each ping.

[justintuchek]

Just looking at the project status here in GitHub - indicates that it's still Awaiting Discussion/Design

Has there been any discussion about delivery for this feature?

[ashwinkolla]

This would be a great feature my team would also love

[mastercactapus]

As an update, here are some notes from our recent design/planning discussion around this feature. In short, we're thinking of adding a couple of slash-commands to the GoAlert Slack bot for managing the groups and linking.

---

• Slash commands:

  • /goalert user-group-schedule <@user-group> <schedule id> (select dropdown instead of pasteing the ID can come later as a +1)

    • Success:

      @goalert-help is now assigned the on-call users from schedule "GoAlert"

    • Error, on-call users are not linked:

      One or more on-call users are not yet linked. Have them run /goalert link from the Slack app.

      • Bob
      • Joe

    • Success, unlinked users (on-call are linked):

      @goalert-help is now assigned the on-call users from schedule "GoAlert"

      Note: One or more users associated with this schedule are not yet linked. Have them run /goalert link from the Slack app to prevent issues later.

      • Bob
      • Joe

  • /goalert link

    • Responds with ephemeral or DM with link URL that behaves the same as alert-triggered account linking
• On-Call notification
  • Success: updates slack user group with on-call users
  • Error, one or more users are not linked (options)
    • Create a channel #goalert-schedule-errors and @ mention everyone on the schedule to bring them in

      Hey everyone! I couldn't update @goalert-help because one or more on-call users are not yet linked. Please have them run /goalert link from the Slack app (or update the schedule), and then I'll take care of the rest!
      Thank you!

      @linked-user-from-schedule-1, @linked-user-from-schedule-2

      Unlinked users:

      • Bob

[mastercactapus]

Update since it's been a while, and a few things have changed:

• The DM contact method stuff will be used for managing linked status
• Skip over adding the /goalert slash commands for now
• Slack group updates will happen as part of the On-Call Notifications flow (i.e., you can "notify" your channel and/or a user group)
• When adding a Slack user group "notification" you will be required to provide a "fallback" channel (instead of a global channel like above)
• The error message to that channel will include a button to initiate the linking flow as an alternative to adding a contact method manually (same as alert notification's flow)

Messaging and functionality will be the same otherwise.

[mastercactapus]

On the backend, a new notification channel type, SLACK_USRGRP, can be added, whose value is a combination of the user group ID and the channel ID.

When sending a notification to the group, a "linking" error message should be sent to the channel (including a button) if any users are not linked. Other API issues can be passed to the channel without the link functionality.

UI Wireframe Ideas

[mastercactapus]

This is available in the nightly builds via the slack-ug experimental flag. This is a summary of the next set of changes:

• Schedule on-call notifications will get a dropdown similar to contact methods
• Slack user group notification rules will still have a channel for errors, but will be de-coupled from channel notifications (meaning they will update silently)
• Slack channel notifications will remain unchanged
• Update in response to feedback in slack: lower slack cache time #3028 and slack: use usergroup handle instead of name #3037

[jghibiki]

So i have had some more time to play around with this recently, and I am just curious if as it's implemented now, the system supports this scenario (from my playing around, it seems like the answer is no):

1. Our team has two support rotations: Primary, Secondary
2. Our team has two schedules, one for the primary rotation one for the secondary rotation
3. We have two separate schedules because in our escalation policy, we want to alert the primary rotation first, then wait 5 min and alert the secondary rotation.

From my investigation, it seems like because the slack group assignments are at a "schedule" level, we could not have the slack group be managed by the combination of two schedules. i.e. We'd want those in the primary schedule and secondary schedule to be assigned to the slack group.

Does this appear to be correct for the current state?

[mastercactapus]

@jghibiki, that is correct in the current state.

To get what you're looking for, I imagine notifications at the service level (rather than schedule). For example, if you go to the service details page, the complete list of On-Call Users is at the bottom. Would that be accurate?

[jghibiki]

@mastercactapus We don't use the slack group for go-alert notifications, we use it for partner teams and stakeholders to get a hold of our on-call support people.

[mastercactapus]

Right, so the list of on-call users on the service details page would be the list of users you want to be assigned to the Slack group; is that correct?