Engineering services
Monitoring & Alerts
We watch your live product day and night, so we know something has broken — and have started fixing it — before your customers ring you about it.
Part of your product’s journey
Of the eight stages we run, this service covers Monitor. We also run the others.
What this makes possible
There is a version of monitoring that exists to be pointed at during a meeting. A screen of green squares, a check that the homepage loads, and an alert channel everyone has muted because it goes off forty times a day. It gives the feeling of visibility without any of the substance, and it fails in one particular way: you find out about an outage from a customer’s email.
Our approach in more detail
Useful monitoring answers questions rather than displaying numbers. When something is slow, which part is slow. When errors rise, whether they started with a release. When one customer complains, what actually happened to their request. That needs three kinds of information joined together. Measurements taken over time. The detailed records the software writes as it works. And a step-by-step timeline of individual requests as they pass through the system. A spike on a chart should lead to the timelines behind it, and those to the records that explain them — in one place, rather than three tools nobody has connected.
The other half is alerting on the things people actually feel: slowness, failures, a system running out of room — not causes that may not matter. Alerts have to be rare enough to be believed, and every alert needs a defined action or it is just a notification. We tune towards fewer, more meaningful alerts, each linked to written instructions, so whoever is woken at three in the morning is not starting from nothing.
What you get
- Numbers, records and error trails, all in one place
- A single screen showing whether everything is healthy
- Alerts for the problems a customer would notice
- Automated checks running continuously that the site is up and working
- Error reports, linked to the release that caused them
- Written steps for handling a failure, and who to call
How we get there.
- 01
Define what matters
Before any tools, we work out what healthy means for your product. Which journeys must work. How slow is too slow. What rate of errors is normal. These become the measures everything else is judged against. Skip this and you get screens full of numbers with no threshold anyone can defend.
- 02
Wiring up the measurements
Measurements, structured records and request timelines added across the application and the servers it runs on, using open standards so the data is not trapped inside one supplier’s product. Every record carries an identifier that lets a single request be followed across every part of the system. Releases are recorded too, which is what lets you see a change and its consequence on the same chart.
- 03
Dashboards worth opening
One overview per service, answering whether it is healthy right now, with more detail available behind it for when it is not. Deliberately few charts. A screen with sixty charts is one nobody reads under pressure, and pressure is the only time it matters.
- 04
Alerting and on-call
Alerts on what users notice, thresholds taken from measured behaviour rather than round numbers, and severity levels that mean something — one wakes a person, another waits until morning. Automated checks walk real journeys from outside the system, since a status check reporting all clear while login is broken is a common and embarrassing failure.
- 05
Written instructions and review
Every alert links to written instructions: what it means, what to check first, how to limit the damage, when to escalate. After an incident, a review that looks for changes to make rather than someone to blame. We revisit the thresholds after a few weeks of real data, because the first set is always an estimate.
A little more detail.
The process, the fit, and the questions worth asking before we begin.
Is this the right service for me?
- Not for you if nobody will respond to the alerts. Monitoring without an owner produces a louder version of the same outage. If there is no rota for who is on call and no appetite to create one, buy support instead.
- Not for you if the application cannot be changed. Useful monitoring needs measurement points added inside the code. With a supplier’s product you cannot modify, you are limited to checking it from the outside, and we will say plainly what that does and does not tell you.
- Not for you if the goal is a screen to show a board. It will be accurate and it will not be the point, and there are cheaper ways to produce a report.
- Not for you if the tooling budget is the objection. Collecting all three kinds of information carries a real monthly cost that grows with volume. We size it honestly before you commit, and it is not small.
- Not for you if the system is falling over daily for reasons you already know. Fix those first. Monitoring tells you what is wrong; it does not shorten a queue of problems you already have.
A worked example
Hypothetical scenario
Illustrative: intermittent slowness nobody can pin down
Say your application is usually fast, but a few times a week it crawls for twenty minutes and then recovers before anyone can look. Uptime monitoring shows nothing, because the site was up. Work of this shape starts with request timelines, so a slow request can be broken down into where its time actually went. Very often the timeline shows the application waiting on the database, and the database waiting on a scheduled report that is holding everything else up — invisible from outside, obvious from inside. The measurement is the deliverable, not the fix. Afterwards you have response times reported not as an average but by how the slowest requests behave: the worst one in twenty and the worst one in a hundred. An average hides exactly this. You have an alert that fires when those slow requests cross a threshold, rather than only when the site is down. You have release markers on the chart, so a new problem can be tied to a release. And you have written instructions naming that report as the first thing to check. Two to four weeks for a system of moderate size.
An illustration of our approach, not a delivered client project.
How long does setup take?
Two to four weeks for a single application, covering the measurement points, the dashboards, a first set of alerts and the written instructions behind them. Longer for several services, or where request timelines have to be added to code that has none. Plan for a tuning pass three to four weeks after go-live: the first thresholds are estimates, and the pass exists to remove the alerts that turned out to be noise.
Which tools do you use?
We collect the data using open standards so it stays portable, then choose where it is stored to fit your volume and budget. Hosted platforms are quicker to run and cost more as the data grows. Running it yourself is cheaper at volume, and somebody has to look after it. We price both against your actual data volume first, because how much you send and how long you keep it drive the bill, and both are easy to underestimate.
Do you provide the on-call rota, and does the timezone difference matter?
We can hold first-line response under a support arrangement with a written response time, or set everything up for your own team to run. Our engineers work Indian hours, which covers most of the European and Asia-Pacific working day. For a client in the Americas we agree explicitly which severity levels get a response outside those hours, and what the target is. It becomes a number in a contract rather than an assumption.
How do you avoid alert fatigue?
Alert on what users notice, not on every number that can be charted. Set thresholds from measured behaviour rather than round figures. Give every alert written instructions and a required action, and delete any alert that fires without one. We review how many alerts fired after the first month and remove the ones nobody acted on. A few alerts people believe beat complete coverage everyone has muted.
Can you add this to a system you did not build?
Yes, and it is a common starting point when a product has been inherited or the original team has moved on. We read the code and the setup it runs on, add the measurement points without changing how the product behaves, and document what we find. The write-up is often as valuable as the monitoring itself, because it names the weak points nobody had written down.
Bring the pieces together
Tell us what you need to achieve. We’ll help you scope the work and see how it fits your wider product.