KEPLIN Docs

Email alerts

What triggers an alert, what the email says, who receives it, and how to turn the warnings on and tune the thresholds.

The Radar tells what is going on to whoever opens the screen. Alerts do the opposite: they warn without anyone having to look. It is the difference between knowing an app was down and knowing it is.

An alert only arrives when someone needs to act, and it says what broke, where, how many times, to how many people and since when, with a link to see it in the Radar.

They are turned on and tuned in the platform Settings, at the bottom of the sidebar.

Note

These settings belong to whoever administers the installation. With a developer profile, the Settings menu does not appear.

What triggers an alert

Situation When it warns
New problem A problem appears in a published app: an error in a screen, a workflow or a server route, an API that fails on the server, or a test that fails in a scheduled run (see Running and reading results).
Problem that came back A problem marked as resolved happens again.
Open problem that came back An open problem that nobody had been warned about happens again.
Widespread failure The percentage of requests that fail on the server crosses the threshold, with at least 5 failures and the minimum number of requests.
Failed scripts Failed script runs cross the threshold.
Index queue and Disk The index queue falls behind or the disk runs low on free space.
Antivirus The antivirus rejects an uploaded file, or puts a file already stored in quarantine (see Antivirus).
Provider secret about to expire The secret of a Microsoft Entra ID provider expires within 30 days, or has already expired (see Sign in with authentication providers).

Problems follow the state they have in the Radar (see Investigating a problem):

  • While a problem is open, no more emails arrive about it.
  • To Resolve a problem closes the alert without an email: whoever resolved it already knows. If the problem happens again, a new email says that it came back.
  • To Ignore a problem silences it: it never triggers alerts.
  • What happens in the studio, while someone builds or tests the app, warns nobody.

For the widespread failure, only server failures count (status 500 or above). Permission refusals, requests cancelled by whoever made them and validation errors in what someone typed do not count. When an app does not have enough traffic to decide, the alert stays as it was: it does not close just because people stopped using the app.

What an email says

Each person receives one email with the alerts that concern them, in the language they use the platform in. For a problem, the email says:

  • The app name and the error message, in the subject.
  • Where: the API, the screen, the workflow or the server route.
  • How many occurrences there were in the window, and how many in total.
  • How many users were affected.
  • When the problem was first seen and, for a problem that came back, when it had been marked as resolved.
  • Open in Radar: the link that opens the problem in the app's Radar.

Widespread failure and script alerts say which APIs or scripts failed, how many times, and the latest error message of each one.

Turning the warnings on

  1. Open Settings in the sidebar.
  2. Choose Alerts, in the Warnings group.
  3. Turn on Email warnings. The help says what is at stake: "Warns when a new problem appears in an app, when a resolved problem happens again and when something crosses the limits below. Without it, you only find out when you open the screen."
  4. Click Save, in the header. The confirmation says "Settings saved."

The Alerts section, with the Email warnings switch
The Alerts section, with the Email warnings switch

The Processes section, in the Platform group, holds three other switches worth knowing, because half of the "alerts that never arrive" come from one of them being off:

Switch What it does
Scheduler Runs the schedules, the scheduled reports and the workflow timers.
Observability index Moves what the apps record into the central index and applies each app's retention periods. Off, the screens show stale data.
Real-time notifications Real-time server for the notifications. Without it, notifications arrive on the next visit.

Where the warnings go

The Email section, in the Warnings group, holds the outgoing server and the recipients: "The server platform warnings go out through. Separate from each app's email."

The Email section, with the outgoing server and the recipients
The Email section, with the outgoing server and the recipients

Field What it is
Server The address of the outgoing email server.
Port Its port.
Security SSL/TLS (port 465), STARTTLS if the server offers it (port 587) or None (port 25).
User The authentication account on the server.
Password "Stored encrypted and never shown again. Leave blank to keep the current one."
From The sender's address. It is required.
Recipients Who receives every warning, instead of the automatic choice.

With the Recipients blank, the platform chooses whom to warn:

  • An app's alerts go to the people assigned to that app.
  • The alerts of an app with nobody assigned, and the platform's own alerts (the index queue and the disk), go to the active administrators.

Tip

Leave the Recipients blank. Each app warns the people who work on it, and the list updates itself when someone joins or leaves the team. A hand-written list receives everything from every app and goes stale the day someone changes jobs.

The password's state appears below the field: Set — type to replace or Not set.

When to warn

The Thresholds are in the Alerts section, below the switch: "New problems always send a warning. The values below decide when a widespread failure, scripts, the index queue or the disk send an email."

The thresholds of the Alerts section, with the window
The thresholds of the Alerts section, with the window

Threshold What it measures Note
API server failures (%) "Percentage of requests that fail on the server (5xx), per app, with at least 5 failures in the window." Counted app by app, not over the sum of all of them.
Minimum requests How many calls are needed for the percentage to count "Below this the percentage says nothing: 1 error in 3 is 33%."
Failed scripts How many script runs may fail before warning
Queue lag (s) How far behind the index queue may fall "While the queue is behind, screens show stale data."
Free disk (%) Below what percentage of free disk to warn
Window (min) The period all of this is counted over "The period the numbers above are counted over."

The Minimum requests keep an app with little traffic from firing widespread failure alerts: with three requests and one failure, the percentage is 33% and means nothing. Each server failure already raises its own problem alert; the percentage is for the day many things fail at once.

How to choose the values

There are no universal numbers, but there is a method that works:

  1. Let the platform run for a week with the default values.
  2. Open Observability → Overview with the period on 7d and look at the real Error rate and Response (p95).
  3. Set the API server failures (%) threshold a little above what is normal on that installation — the goal is to catch the abnormal, not to confirm the normal.
  4. Adjust the Minimum requests to the traffic of the quietest app.

Warning

A threshold that is too tight is worse than none. Alerts that fire every day stop being read within a week — and the day the important one arrives, it will arrive in the middle of the others.

Confirming that the alerts really work

The State section of the Observability has, at the top, a sentence that tells the truth about the alerts on this installation. It is the place to check after touching the settings.

The alert state notice, at the top of the State section of the Observability
The alert state notice, at the top of the State section of the Observability

There are three possible sentences:

What it says What it means
"Alerts on: notifies the people working on each app when a new problem appears and when a resolved problem happens again, and also when more than 5% of requests fail on the server, from 20 requests upwards." Everything working — with your thresholds in plain sight.
"Alerts are off. Nobody is notified when something goes wrong — this page only tells whoever opens it." Email warnings still needs turning on.
"Alerts are on, but email cannot be sent: … Alarms are recorded and nobody is notified." The warning fires, but does not go out — the email server still needs configuring.

The third is the most treacherous, because everything looks configured. If you see it, go back to the Email section and complete what is missing.

Sessions and limits

Still in the Settings, the Limits section, in the Platform group, holds two values that are not alerts but tend to be looked for at the same time:

  • Session length (h) — "An active session renews itself; this is the idle time that expires it."
  • Max file (MB) — the maximum size of an uploaded file.

Why don't I see…?

  • …alerts arriving, with everything on? Check the State panel: if it says "email cannot be sent", the outgoing server is missing. If it says "off", you still need to save.
  • …the Alerts section? You are in the settings of an app, not of the platform. The platform's open through the Settings menu in the sidebar.
  • …my thresholds taking effect? Without Save, in the header, nothing is kept. The confirmation is "Settings saved."
  • …an app's alerts? An app's alerts go to the people assigned to it. Check in Users that you are assigned to the app.
  • …another email about the same problem? An open problem only warns once. It warns again if it is marked as resolved and happens again.
  • …the alert for an error I saw in the studio? What happens in the studio, to whoever is building the app, warns nobody. Only the published app raises alerts.