Set Up Heartbeat Monitoring in TaskCall Application

What is Heartbeat Monitoring?

Heartbeat monitoring is a passive form of monitoring to ensure your systems are running as expected or your jobs are completing within their scheduled times. As opposed to traditional monitoring where an external system pings a known public IP address on your system, heartbeat monitoring works in reverse where your system periodically pings a known external endpoint to let it know that everything is functioning as expected.

It is useful when your system is unreachable from the public internet. Perhaps it is hosted within an intranet or has restricted access from certain ports. In such cases a heartbeat monitor can be set up to run within a given interval as a sanity check.

Heartbeat Types

TaskCall offers two types of heartbeat monitoring:

1. HTTP ping: Ping a URL endpoint through a HTTP request.
2. Email ping: Send an email ping to a given email address.

Create Heartbeat Monitor

Heartbeats can be set up by users who have edit permissions on the platform.

1. Go to Automation > Heartbeats. Click New Heartbeat.
2. Give the heartbeat a name and a description.
3. Select the Service this heartbeat is for. An incident will be created on this service when the check fails.
4. Configure how the ping will be received:
   • Ping Type: HTTP or email.
   • Interval: The interval sets the frequency of pings.
   • Grace Period: It allows you to set some variability margin for the ping. If the ping is not received within the set interval, this additional grace period will be taken into account.
5. Set the details of the incident that should be created if a ping is not received within the set interval (+ grace period):
   • Title: Title of the incident.
   • Description: Description of what the incident is for and how it could be addressed.
   • Tags: Tags that should be added to the incident.
6. Set the restrictions for the source of the pings if you would like. This will bar false positive pings from unknown actors from being processed. For HTTP pings, a whitelist of IP addresses (IPV4) can be set while for email pings a whitelist of email (sender) addresses can be set.
7. Once you are done, click Save.

As soon as the heartbeat check is created, pings will be expected within the set intervals. Please ensure that you have the pinger set up in due time. You can also disable the ping temporarily while you set up the pinger.

Incidents that are created from failed checks will automatically resolve when the next valid ping is received.

How to ping a heartbeat check?

HTTP pings can be made through a GET or POST request to the ping endpoint. If valid IP addresses are set, then pings will be recognized as valid only if they originate from one of the whitelisted IP addresses.

Email pings can be made by sending an email to the given email address for the check. If a whitelist is set, then the emails must be sent from one of the whitelisted email addresses.

PLEASE NOTE: You can only ping using the endpoint that has been assigned to your specific check based on the ping type you have set. If the ping type is set as HTTP, email pings will not be allowed. If the ping type is set to Email, then HTTP pings will not be allowed.

How are ping timers reset?

Every time a valid heartbeat is received, the timer is reset by the set interval. The next heartbeat must be received before this interval in addition to any grace period that may be set.

Here is a timeline example for a heartbeat check that has an interval of 60 minutes and a grace period of 10 minutes.

• First ping is received at 09:00 am. This sets the timer for the next ping to 10:00 am as per the interval.
• At 09:45 am the next ping is received. Although the ping has arrived earlier than expected, it is still considered as a valid ping and the check passes. The timer is reset to 10:45 am.
• By 10:45 am the next ping is not received. The set interval has passed, but there is a grace period of 10 minutes. The system waits till 10:55 am.
• By 10:55 am the ping is still not received. The check fails and an incident is created on the service the check is for. The timer is reset to expect the next ping at 11:55 am.
• At 10:58 am the ping is finally received. The check passes and the timer is reset to 11:58 am. The incident that was created from the failed check is automatically resolved.