Logging and detection

Record enough to explain an incident, and nothing that turns the logs into one.

13of 16
≈ 11Min read
5Parts

Logs are a copy of your data with worse access control

Application logs get shipped to a third party, kept for a year, searched by anyone on the team, and backed up somewhere nobody remembers. Every field you put in one is a field that now exists in all of those places.

So the first rule is subtractive: a password, a token, a session id, a card number, a full national id or an authorisation header must never reach a log line, not even at debug level, not even for five minutes in production.

What to record

structured, so it can be queried
log.info({
  event: "order.cancelled",
  requestId: req.id,
  userId: req.user.id,
  orderId: order.id,
  outcome: "ok",
});
FieldWhy
timestamp, in UTC, ISOso two systems can be lined up
a request idso one user’s journey can be followed across services
who (a user id, not a name or an email)enough to investigate, not a directory
what happened, as a stable event namelogin.failed, not "Login failed for…" — searchable
the outcome, and why403 ownership, not just 403
source IP and user agentfor the security events — and see the privacy lesson

The security events worth having

  • Authentication: success, failure, logout, password change, second-factor enrolment.
  • Authorisation failures — especially the ones where a logged-in user asked for something that was not theirs. A burst of those is an attack in progress.
  • Changes to accounts: email, role, permissions. Who changed it, and to what.
  • Administrative actions, exports and bulk reads.
  • Rate limits hit, and input rejected by validation — a spike in either is reconnaissance.

Masking, not deleting

Support needs to confirm "the card ending 4242" and you need to see that the same address tried forty times — neither needs the value itself. A keyed hash keeps grouping and correlation while removing the personal data, and is the usual answer for IP addresses in analytics.

keep enough to recognise it, not enough to use it
mask("4242424242424242")   // "•••• 4242"    — last four
mask("ulvi@doors.test")     // "u•••@doors.test"
mask(token)                 // "sk_live_…" + the first 6, never more
hashOf(ip)                  // a keyed hash, so repeats still group together

Noticing

A log nobody reads is a filing cabinet. Alert on the small number of things that mean something: a spike in authorisation failures, a login from a new country for an admin, a rate limit hit at scale, an error rate above a baseline, a deployment followed by a rise in 500s.

Alert fatigue is the failure mode. Five alerts that always mean something beat five hundred that mostly do not — and each one should say what to do next.

Log injection is real

A user-supplied value containing a newline can forge a second log line; one containing ANSI escapes can lie to a terminal; one in a log viewer that renders HTML is XSS. Structured logging — a JSON object, not string concatenation — removes all three.

Practice