integration: grafana 8.x support

[mastercactapus]

Description:
Adds support for Grafana 8.x webhooks. Fixes #2077

The new version supports multiple alerts in a single payload.
Details are generated via template, the summary is the summary annotation, falling back to the alertname label and using the fingerprint field for dedup.

A README.md file has been added to the grafana/ directory with documentation on testing the Grafana integration.

Context:
Grafana 8 has changed the webhook payload compared to Grafana 7. Here's a comparison of the payloads for the alert firing event & ok event:

Grafana 7.1.4

"firing":
{
  "dashboardId": 1,
  "evalMatches": [
    {
      "value": 2,
      "metric": "A-series",
      "tags": {}
    }
  ],
  "orgId": 1,
  "panelId": 2,
  "ruleId": 1,
  "ruleName": "xxspenceralert",
  "ruleUrl": "http://localhost:3000/d/D0iP3l0nz/new-dashboard-copy?tab=alert&editPanel=2&orgId=1",
  "state": "alerting",
  "tags": {},
  "title": "[Alerting] xxspenceralert"
}

"ok":

{
  "dashboardId": 1,
  "evalMatches": [],
  "orgId": 1,
  "panelId": 2,
  "ruleId": 1,
  "ruleName": "xxspenceralert",
  "ruleUrl": "http://localhost:3000/d/D0iP3l0nz/new-dashboard-copy?tab=alert&editPanel=2&orgId=1",
  "state": "ok",
  "tags": {},
  "title": "[OK] xxspenceralert"
}

Grafana 8.x.x
"firing":

{
  "receiver": "GoAlert - spencertest",
  "status": "firing",
  "alerts": [
    {
      "status": "firing",
      "labels": {
        "alertname": "spencer-goalert-3-testt"
      },
      "annotations": {},
      "startsAt": "2022-01-04T12:03:32.588089-06:00",
      "endsAt": "0001-01-01T00:00:00Z",
      "generatorURL": "http://localhost:3000/alerting/M7wzIEA7z/edit",
      "fingerprint": "a6cfa83908c8f7cb",
      "silenceURL": "http://localhost:3000/alerting/silence/new?alertmanager=grafana&matchers=alertname%3Dspencer-goalert-3-testt",
      "dashboardURL": "http://localhost:3000/d/A_mLME0nk",
      "panelURL": "http://localhost:3000/d/A_mLME0nk?viewPanel=6",
      "valueString": "[ metric='A-series' labels={} value=2 ]"
    }
  ],
  "groupLabels": {},
  "commonLabels": {
    "alertname": "spencer-goalert-3-testt"
  },
  "commonAnnotations": {},
  "externalURL": "http://localhost:3000/",
  "version": "1",
  "groupKey": "{}:{}",
  "truncatedAlerts": 0,
  "orgId": 1,
  "title": "[FIRING:1]  (spencer-goalert-3-testt)",
  "state": "alerting",
  "message": "**Firing**\n\nValue: [ metric='A-series' labels={} value=2 ]\nLabels:\n - alertname = spencer-goalert-3-testt\nAnnotations:\nSource: http://localhost:3000/alerting/M7wzIEA7z/edit\nSilence: http://localhost:3000/alerting/silence/new?alertmanager=grafana&matchers=alertname%3Dspencer-goalert-3-testt\nDashboard: http://localhost:3000/d/A_mLME0nk\nPanel: http://localhost:3000/d/A_mLME0nk?viewPanel=6\n"
}

"ok":

{
  "receiver": "GoAlert - spencertest",
  "status": "resolved",
  "alerts": [
    {
      "status": "resolved",
      "labels": {
        "alertname": "spencer-goalert-3-testt"
      },
      "annotations": {},
      "startsAt": "2022-01-04T12:06:32.588089-06:00",
      "endsAt": "2022-01-04T12:08:42.588089-06:00",
      "generatorURL": "http://localhost:3000/alerting/M7wzIEA7z/edit",
      "fingerprint": "a6cfa83908c8f7cb",
      "silenceURL": "http://localhost:3000/alerting/silence/new?alertmanager=grafana&matchers=alertname%3Dspencer-goalert-3-testt",
      "dashboardURL": "http://localhost:3000/d/A_mLME0nk",
      "panelURL": "http://localhost:3000/d/A_mLME0nk?viewPanel=6",
      "valueString": "[ metric='A-series' labels={} value=2 ]"
    }
  ],
  "groupLabels": {},
  "commonLabels": {
    "alertname": "spencer-goalert-3-testt"
  },
  "commonAnnotations": {},
  "externalURL": "http://localhost:3000/",
  "version": "1",
  "groupKey": "{}:{}",
  "truncatedAlerts": 0,
  "orgId": 1,
  "title": "[RESOLVED]  (spencer-goalert-3-testt)",
  "state": "ok",
  "message": "**Resolved**\n\nValue: [ metric='A-series' labels={} value=2 ]\nLabels:\n - alertname = spencer-goalert-3-testt\nAnnotations:\nSource: http://localhost:3000/alerting/M7wzIEA7z/edit\nSilence: http://localhost:3000/alerting/silence/new?alertmanager=grafana&matchers=alertname%3Dspencer-goalert-3-testt\nDashboard: http://localhost:3000/d/A_mLME0nk\nPanel: http://localhost:3000/d/A_mLME0nk?viewPanel=6\n"
}

This PR essentially checks the API "version" then reads the data in differently depending on the webhook format. To test this you'll want to install Grafana 8 and make an alert. @spencerpauly can help you set that up if you want.

Some things to test:

• An alert firing in Grafana should create an alert in GoAlert
• An alert status changing to "ok" should close the alert in GoAlert
  ^ Would be best to test these cases in grafana 8 as-well as grafana 7

[dctalbot]

Functionally validated both versions, and code looks great. One question - how did you come up with the versioning? Would it make sense to use the minimum granfana version e.g. 8 or 8.3.4 instead of 1?

[spencerpauly]

Functionally validated both versions, and code looks great. One question - how did you come up with the versioning? Would it make sense to use the minimum granfana version e.g. 8 or 8.3.4 instead of 1?

Grafana 8 added new alerting (v1), but they kept around the original alerting as "legacy alerts". So checking for version 1 in the webhook request just checks that it's sent in the new Grafana alert format. But, I believe you can still use the old alerts in grafana 8, but you need to opt-in and those alerts are slated to be removed in the future. That might be a good thing to comment though.

[dctalbot]

@spencerpauly Ah I see, that makes total sense then, I just read too fast. Thanks!

[spencerpauly]

Looks good and working on my end 👍.