Why Our Alerts for Critical Links Are Meant to Wake Us Up
Network Monitoring • Operational Resilience • 24/7 Support
Strategic Summary: Network monitoring is only useful if it’s set to the standard of the service it protects. For clients whose operations must stay up around the clock, we tightened packet loss alerts on their critical links, accepting more night-time call-outs as the price of catching real problems early.
Eight Links, One Moment
Last Sunday night, eight of the internet links we monitor started reporting packet loss at exactly the same moment. These weren’t ordinary office lines. They were links serving clients whose operations need to stay up around the clock. Our on-call engineer was alerted, and he spent the rest of the night in inSight working the problem.
When we logged it with the carriers, the first response was that they could see nothing wrong on their side. Their circuits looked healthy.
We didn’t accept that as the end of it, and we didn’t treat our own alerts as a false alarm either. Eight links failing in the same way at the same moment is not a coincidence. It points to something further upstream, in a shared part of a provider’s network, that the people answering the phone may simply not have known about yet. Our monitoring had seen a real event, even though the carrier’s first-line view had not.
Setting Network Monitoring to the Service Promise
That night is a fair illustration of a choice we made recently. For the clients who rely on us most, we tightened the packet loss thresholds on their critical links. Before the change, the thresholds were set loosely enough that brief but meaningful loss could pass without anyone being told. For a business whose trading, payments or customer systems depend on a link, short bursts of packet loss are exactly the early warning you want. So we brought the thresholds into line with what those clients expect from us.
The Real Cost of 24/7 Network Monitoring
There’s a cost to that, and it’s carried by our engineers. Tighter thresholds mean more alerts, and on a 24-hour service more of those alerts arrive at night. We’ve had a conversation internally about whether some types of alert could follow business hours instead. For most services that’s a reasonable design. For links that must stay up around the clock, it isn’t. Those clients pay for us to be awake when something goes wrong, so that’s what we do.
It’s also worth keeping in proportion. This was the first event like it in about three months, and the thresholds are tuned to catch real problems rather than every tiny wobble. Where an alert pattern starts to fire without good reason, we refine it. We don’t switch it off.
Your Provider Monitors Their Network, Not Your Business
There’s a practical lesson here for any business that depends on its connectivity. Your provider’s monitoring is designed around their network, not your business. When something goes wrong, the fastest route to a fix is often independent evidence: what happened, on which circuits, at what time and for how long. It also helps to have exact details ready, such as the precise address where each circuit is handed over, because carriers’ support teams often need that before they can pass a fault on to the people who can fix it.
Network Monitoring That Works at 3am
Monitoring is only useful if it’s set to the standard of the service it protects. A link that has to work at 3am deserves alerting that works at 3am too, and people prepared to answer it.
Our Trusted Response Centre watches critical links around the clock as part of our managed connectivity services. Discuss your needs with our engineers.
