Kibana Alerting Integration Guide
Kibana is an easy to use and powerful status dashboard for the ELK Stack. It allows you to visualize, explore and get insights of the data processed by your ELK Stack. TaskCall integrates with Kibana's Alerting to help you keep track of incidents raised by anomalies in your system. Readily set up alerts with thresholds to monitor your stack and address issues faster than ever before with TaskCall's notification system.
Pointers
- The integration is between a TaskCall service and Kibana.
- This integration only receives alerts in TaskCall from Kibana.
- TaskCall will automatically identify alerts that are the same and group them together into one incident.
- Incidents will automatically resolve when the metrics come back within threshold on Kibana.
In TaskCall
- Go to Configurations > Integrations and click New Integration.
- Select the Kibana Alerting integration.
- Give the integration a name and configure how incidents will be created. Only the Default Service field is required. The rest are optional.
- Incidents (Optional)
- TaskCall automatically maps incoming payloads with internal attributes to create incidents. However, this can be customized in the Incidents tab. Constant values can be included alongside placeholders to dictate which attributes should map to the Incident Title, Description, Tags, Snapshots and Vendor URL. To insert a placeholder use double curly braces.
For example, if you enter {{ alert.title }} in the Incident Title field, TaskCall will look for an attribute called "title" nested inside another attribute called "alert" in the incoming payload. If the nested attribute exists, its corresponding value will be pulled from the payload and inserted. If the nested attribute does not resolve successfully, TaskCall will fall back to the default mapping.
If you are unsure, please leave the fields empty. TaskCall will use the default mapping to create the incident for you. - Select the Urgency level the incident should be created at.
- Select the option to Group similar alerts together if you would like follow up triggers from the same alert source to be grouped into the same incident. If you leave this option unchecked, then a new incident will be created for every trigger. (This behavior is specific to new triggers; not updates to an alert.)
- Click Next to move on to the Services tab.
- TaskCall automatically maps incoming payloads with internal attributes to create incidents. However, this can be customized in the Incidents tab. Constant values can be included alongside placeholders to dictate which attributes should map to the Incident Title, Description, Tags, Snapshots and Vendor URL. To insert a placeholder use double curly braces.
- Services (Required)
- Select the Default Service incidents should be sent to. To route incidents to different services instead, set Map With Attribute to the attribute to route by using a placeholder. Then, map each value it can take to the service that should receive it.
For example, to route by a region attribute sent in the payload, set Map With Attribute to {{ region }}, then map US to one service and EU to another, and so on.
- Click Next to move on to the Authorization tab.
- Select the Default Service incidents should be sent to. To route incidents to different services instead, set Map With Attribute to the attribute to route by using a placeholder. Then, map each value it can take to the service that should receive it.
- Authorization (Optional)
- The Authorization tab allows a Bearer Token and an IP Allowlist be added if you would like to restrict who is able to send alerts to this integration. If you are unsure, this can be skipped.
- Once done, click Save.
- Copy the Integration Url that is issued for the integration. This will be used to set up the notification in Kibana Alerting.
In Kibana
- Log in to your Kibana dashboard.
- Click on the menu icon on the top right corner. Then go to Stack Management > Rules and Connectors .
- Go to the Connector tab. Click Create connector.
- Select the Webhook connector.
- Name the connector TaskCall. Make sure the webhook method is set to POST. Paste the Integration Url you copied over from Taskcall in the URL field. Unselect the Require authentication for this webhook option. Select the Add HTTP header option and add the Content-Type header. Set the value of the Content-Type to application/json. Please make sure that you add the application/json header; the integration will fail without it.
- Click Save.
- We have set up the connector. Now we have to add this connector to alert rules to receive the alerts. We will walk you through an example. We will create a CPU Usage alert here.
- On the same Rules and Connectors page, go to the Rules tab.
- Click Create rule.
- Give the alert a name and configure how it should run. Select the Alert Type that you want (we are using the CPU Usage alert here) and set the trigger threshold.
- Select the Webhook Action and then choose the TaskCall connector your created above.
- Select when the action will run.
- In the Body section paste the below JSON payload.
{
"alert_id": "{{alert.id}}",
"alert_action_group": "{{alert.actionGroup}}",
"alert_action_subgroup": "{{alert.actionSubgroup}}",
"alert_action_group_name": "{{alert.actionGroupName}}",
"kibana_base_url": "{{kibanaBaseUrl}}",
"rule_id": "{{rule.id}}",
"rule_name": "{{rule.name}}",
"rule_description": "{{rule.description}}"
}
You can add more fields to the json payload and change the values as you wish, but the ones shown above must be included. If any of the fields are not provided, then TaskCall will fail to create incidents. The rule_name and the alert_action_group_name fields are used to set the incident title.
- Here is a snapshot of what the whole step would look like.
- If you want TaskCall to automatically resolve the alert when it resolves on Kibana, then please configure another Action to this rule and set the Run when option to Recovery.
- Click Save when you are done.
- You have set up a Connector and associated it to a Rule. Now you are all good to go. You should be receiving alerts in TaskCall from here on.