Anomaly Detection Without a Data Science Team

Start with rules, not models. Why explainable thresholds beat a classifier for most of what organisations actually want, where a model does earn its place, and the four reasons these systems get switched off, none of which is detection quality.

Most organisations do not need a model to find anomalies. They need somebody to write down what normal is, and an owner for the alert. The statistical part is the easy part, and it is not the reason these systems fail.

What people mean when they ask for this

Three different things people mean by anomaly detection

Frequently bundled into one request. Separating them is usually the first useful hour of the engagement.

Source: Thinklytics AI automation practice, 2026.

Three different things, usually at once:

• Fraud. Deliberate, adversarial, rare, and expensive when missed.
• Error. Accidental, more common, cheaper individually and often larger in aggregate: the duplicate payment, the miskeyed unit, the invoice posted to the wrong period.
• Operational break. A feed stopped, a system started sending nulls, a store's numbers went flat because a till was never reconciled.

They need different responses and are frequently bundled into one request. Separating them is usually the first useful hour of the engagement, because operational break is often the majority of the value and the easiest to deliver.

Start with rules, not models

Rules first, model second

• Written rules. Explainable and auditable. The alert says which rule fired, so the person receiving it knows what to do. A regulator can read the rule. Too many alerts, change the threshold today. No training data required.
• A statistical model. Three situations justify it. Normal is multidimensional, normal changes on its own in a seasonal or growing business, or an adversary adapts and a fixed rule becomes a published boundary.

Starting with a model means you cannot tell whether it is earning its keep, because you never established what a page of rules would have caught for nothing. Even where a model is justified, it sits behind the rules and the alert still carries a human-readable reason.

Source: Thinklytics AI automation practice, 2026.

This is unfashionable advice and it is right more often than not.

A rule is a written statement of a condition somebody already knows is wrong. Payment over a threshold to a supplier added this month. Two invoices with the same amount, supplier and date. A meter reading that fell to zero and stayed there. Expenses submitted on a weekend by someone who does not work weekends.

Rules have properties a model does not:

• Explainable. The alert says which rule fired. The person receiving it knows what to do.
• Auditable. A regulator can read the rule. This matters more than it sounds.
• Immediately tunable. Too many alerts, change the threshold, today.
• Free of training data. You can write them before you have labelled a single historical case.

Their limit is real. They only catch what somebody thought of, and a determined adversary learns where the thresholds are. But a large share of what organisations want from anomaly detection is error and operational break, not adversaries, and rules handle those well.

The sensible path is rules first, measure what they catch and what they miss, then add statistical detection specifically for the gap. Starting with a model means you cannot tell whether it is earning its keep, because you never established what a page of rules would have caught for nothing.

Where a model earns its place

Three situations justify it.

The normal pattern is multidimensional. Not amount is high, but amount is high for this supplier, in this month, given this contract. Nobody can write that as rules at scale.

Normal changes on its own. Seasonal businesses, growing accounts, post-migration volumes. Fixed thresholds are wrong within a quarter and everyone stops trusting the alerts.

The adversary adapts. Where somebody is actively probing for the threshold, a fixed rule is a published boundary.

Even then, the model should sit behind the rules rather than replace them, and the alert should still carry a human-readable reason.

Why these systems get switched off

What keeps an anomaly system alive, and what switches it off

Almost never is detection quality the reason one of these gets turned off.

• Alert volume tuned to what the team can actually work. Start deliberately conservative and loosen, rather than tuning for recall in a demonstration.
• One named owner, a service level, and a visible ageing report. Somebody whose job it is to work the queue, with the backlog in view.
• A disposition loop that changes the system. When the reviewer marks an alert as a false positive, that has to go somewhere that changes the thresholds.
• Periodic sampling of what was not flagged. Audit a random slice on a schedule. Without it you know only that the system is producing output.
• Tuned for sensitivity in the pilot. Four hundred alerts in week one, nobody triages them, and within a month everyone ignores the queue.
• Alerts into a shared mailbox. Shared mailboxes are where alerts go to die.
• False positives that disappear. The false positive rate becomes permanent and the team learns the system is wrong.
• Counting detections only. Missed anomalies are invisible by construction, so the count tells you nothing about whether it works.

Underneath all of it is data quality. A system watching a feed that silently changed its unit of measure will faithfully report that everything is normal.

Source: Thinklytics AI automation practice, 2026.

Almost never because the detection was poor.

Alert volume. A system tuned for sensitivity in a pilot produces four hundred alerts in week one, nobody triages them, and within a month everyone ignores the queue. Tune for a volume the actual team can work, starting deliberately conservative and loosening, rather than for recall in a demonstration.

No owner. The alert arrives in a shared mailbox. Shared mailboxes are where alerts go to die. One named person, a service level, and a visible ageing report.

No disposition loop. When the reviewer marks an alert as a false positive, that has to go somewhere that changes the system. If it does not, the false positive rate is permanent and the team learns the system is wrong.

No measure of what is missed. Detected anomalies are counted. Missed ones are invisible by construction. Without periodic sampling, auditing a random slice of what was not flagged, you have no idea whether the system works, only that it is producing output.

What we would do in the first month

1. Separate fraud, error and operational break, and pick one. 2. Take twelve months of history and find the incidents that actually happened and hurt. Ask for the list of things that went wrong, there is always a list. 3. Write the rules that would have caught them and run those against the history. 4. Measure the alert volume that produces. If it exceeds what the team can work, the thresholds are wrong before anything is built. 5. Name the owner and the escalation path, then agree the sampling regime for what is missed.

Only after that is there a sensible conversation about a model, and by then it is a narrow conversation about a specific gap rather than a platform purchase.

The thing underneath all of it

Anomaly detection is a data quality problem wearing a more interesting hat. A system watching a feed that silently changed its unit of measure will faithfully report that everything is normal. Before detection is worth building, the inputs need to be monitored for freshness, completeness and schema change, which is unglamorous, cheap, and catches a substantial share of what people hoped the anomaly system would find.

Frequently asked questions

Do we need machine learning for anomaly detection?

Usually not at first. Most of what organisations want from anomaly detection is error and operational break rather than a determined adversary, and written rules handle those well. Rules are explainable, auditable, tunable the same day, and need no training data. Start there, measure what they catch and miss, then add statistical detection for the gap.

What is the difference between fraud, error and operational break?

Fraud is deliberate, adversarial, rare and expensive when missed. Error is accidental and more common, cheaper individually and often larger in aggregate: the duplicate payment, the miskeyed unit. Operational break is a feed that stopped or a system that started sending nulls. They need different responses and separating them is usually the first useful hour.

When does a model earn its place over rules?

Three situations. When normal is multidimensional, such as high for this supplier in this month given this contract, which nobody can write as rules at scale. When normal changes on its own, in seasonal or growing businesses where fixed thresholds go wrong within a quarter. And when an adversary adapts, because a fixed rule is a published boundary.

Why do anomaly detection systems get switched off?

Almost never because detection was poor. Alert volume tuned for sensitivity in a pilot, so nobody triages. No owner, because alerts go to a shared mailbox. No disposition loop, so false positives never improve. And no measure of what is missed, because detected anomalies are counted and missed ones are invisible by construction.

How do we avoid alert fatigue?

Tune for the volume the actual team can work rather than for recall in a demonstration. Start deliberately conservative and loosen. Give the queue one named owner, a service level and a visible ageing report, and make every false-positive disposition change the system rather than disappear.

How do we know what the system is missing?

Sample it. Audit a random slice of what was not flagged, on a schedule. Without that you have no idea whether the system works, only that it is producing output. Detected anomalies are counted automatically, missed ones never are.

What should the first month look like?

Separate fraud, error and operational break and pick one. Take twelve months of history and find the incidents that actually happened and hurt. Write the rules that would have caught them and run those against history. Measure the alert volume that produces. Name the owner and escalation path, then agree the sampling regime for misses.