Elasticsearch Watcher Integration Guide

Pointers

• The integration happens between a TaskCall service and Elasticsearch Watcher.
• Incidents will automatically resolve in TaskCall when issues are resolved in Elasticsearch.
• At the moment this integration only supports alerts to be received from Elasticsearch Watcher to TaskCall.

In TaskCall

1. Go to Configurations > Integrations and click New Integration.
2. Select the Elasticsearch Watcher integration.
3. Give the integration a name and configure how incidents will be created. Only the Default Service field is required. The rest are optional.
4. Incidents (Optional)
   1. TaskCall automatically maps incoming payloads with internal attributes to create incidents. However, this can be customized in the Incidents tab. Constant values can be included alongside placeholders to dictate which attributes should map to the Incident Title, Description, Tags, Snapshots and Vendor URL. To insert a placeholder use double curly braces.
      
      
      
      For example, if you enter {{ alert.title }} in the Incident Title field, TaskCall will look for an attribute called "title" nested inside another attribute called "alert" in the incoming payload. If the nested attribute exists, its corresponding value will be pulled from the payload and inserted. If the nested attribute does not resolve successfully, TaskCall will fall back to the default mapping.
      
      If you are unsure, please leave the fields empty. TaskCall will use the default mapping to create the incident for you.
   2. Select the Urgency level the incident should be created at.
   3. Select the option to Group similar alerts together if you would like follow up triggers from the same alert source to be grouped into the same incident. If you leave this option unchecked, then a new incident will be created for every trigger. (This behavior is specific to new triggers; not updates to an alert.)
   4. Click Next to move on to the Services tab.
5. Services (Required)
   1. Select the Default Service incidents should be sent to. To route incidents to different services instead, set Map With Attribute to the attribute to route by using a placeholder. Then, map each value it can take to the service that should receive it.
      
      For example, to route by a region attribute sent in the payload, set Map With Attribute to {{ region }}, then map US to one service and EU to another, and so on.
      
      
   2. Click Next to move on to the Authorization tab.
6. Authorization (Optional)
   1. The Authorization tab allows a Bearer Token and an IP Allowlist be added if you would like to restrict who is able to send alerts to this integration. If you are unsure, this can be skipped.
7. Once done, click Save.
8. Copy the Integration Url that is issued for the integration. This will be used to set up the notification in Elasticsearch Watcher.

On Watcher

1. Go to your Elasticsearch server.
2. Make sure that you have Watcher set up and running on the server. You can verify by running the following curl command:
   
   
   curl -XGET 'http://localhost:9200/_watcher/stats?pretty'
   
   
   This should produce an output stating that Watcher has been started.
   
   
   {
      "_nodes" : {
        "total" : 1,
        "successful" : 1,
        "failed" : 0
      },
      "cluster_name" : "elasticsearch",
      "manually_stopped" : false,
      "stats" : [
        {
          "node_id" : "GOf0zK6FSEG-kxpAK5wh0g",
          "watcher_state" : "started",
          "watch_count" : 0,
          "execution_thread_pool" : {
            "queue_size" : 0,
            "max_size" : 0
          }
        }
     ]
   }
   
   
   
3. Set up a watch with a Webhook for TaskCall and use a PUT request to add it to Watcher. Replace the <INTEGRATION_KEY> market in the body of attribute with the the Integration Key you copied over from TaskCall.
   
   
   curl -XPUT 'http://localhost:9200/_watcher/watch/cluster_health_watch' -H 'Content-Type: application/json' -d '{
      "trigger" : {
        "schedule" : { "interval" : "30m" }
      },
      "input" : {
        "http" : {
          "request" : {
            "host" : "localhost",
            "port" : 9200,
            "path" : "/_cluster/health"
          }
        }
      },
      "condition" : {
        "always" : {}
      },
      "actions" : {
        "taskcall": {
          "webhook": {
            "scheme": "https",
            "method": "POST",
            "host": "integrations.taskcallapp.com",
            "port": 443,
            "path": "/elasticsearch-watcher/<INTEGRATION_KEY>",
            "body": "{\"cluster\": \"{{ctx.payload.cluster_name}}\",\"watch_id\": \"{{ctx.watch_id}}\",\"status\": \"{{ctx.payload.status}}\",\"description\": \"Issue in elastic watcher\",\"message\": \"{{ctx.watch_id}}:{{ctx.payload.hits.total}}\"}",
            "headers": {"Content-type": "application/json"}
          }
        }
      }
   }'
   
   
   
4. Please schedule the watch time, interval and conditions as per your need. The above is a minimal example of a configuration.
5. The body attribute can be customized to include more information. However, it must always at least contain the following fields -- cluster, watch_id, status, description and message. TaskCall uses the description attribute as the incident title . while all the attributes of the body are displayed on the incident
6. TaskCall only triggers incidents when the status of the alert is red. It automatically resolves the incident when the alert status changes to green.