Nuage Solutions All articles
Cloud Strategy

The Retention You Can Defend: Choosing Log Lifetimes Before an Incident Chooses Them for You

Nuage Solutions
The Retention You Can Defend: Choosing Log Lifetimes Before an Incident Chooses Them for You

Photo: server room log data storage archive UK business technology, via media.geeksforgeeks.org

Most cloud estates do not choose a log retention period. They inherit one. A console presents thirty, ninety or perhaps 365 days, a project team accepts the default to keep the service live, and the setting quietly becomes the organisation’s memory. It is only months later — during an incident review, a subject access request, a regulatory query or an insurer’s questionnaire — that someone asks whether the evidence still exists. By then, the answer is often no.

The problem is not that thirty or ninety days is always wrong. For some sources it is proportionate and sensible. The problem is that the period was never decided. A defensible retention policy begins with the questions the organisation expects its logs to answer, then works backwards to the sources, the storage tiers and the cost. That sequence matters because log retention is not primarily a technical preference. It is an evidence-governance decision with financial consequences.

Begin with the questions, not the console

A useful retention review starts with a short list of questions that the business, legal, security and finance teams can agree on. For example: can we reconstruct who signed in to a privileged account on a given date? Can we show whether a file was downloaded before an employee left? Can we identify which application made a particular API call? Can we evidence that a network flow was blocked at the perimeter? Can we respond to a subject access request that arrives six months after the event?

Each question points to a different log source. Microsoft Entra ID sign-in logs answer access questions, but they do not answer data-export questions. Microsoft 365 unified audit logs may show mailbox and file activity, but they do not show network traffic. Cloud provider activity logs record control-plane changes, while flow logs and web application firewall logs record traffic. Endpoint, DNS and email gateway logs fill other gaps. No single source is a complete record.

This is why the default retention period is such a fragile anchor. A ninety-day sign-in log may be adequate for routine troubleshooting but insufficient for a long-running insider investigation. A thirty-day network flow log may satisfy operational needs but fail an insurer’s request for evidence covering the preceding twelve months. The organisation needs to know which source supports which question, and for how long.

Separate the expensive from the merely old

The second decision is where each log lives over time. Hot search — the tier that supports immediate queries, dashboards and alerts — is the most expensive place to keep data. Archive storage is far cheaper, but retrieval can take hours and may involve a separate query tool. Many UK estates overspend because they keep everything hot for convenience, then cut retention everywhere when the bill rises. A better approach is tiered by purpose.

Operational logs that feed live detection and incident response may justify thirty to ninety days hot. Investigative sources that answer access, data movement and administrative-change questions may need twelve months hot or warm, with a longer archive. Regulatory or contractual evidence may need several years in archive, subject to the organisation’s retention schedule and legal advice. The point is not to prescribe a single number. It is to make the distinction explicit: hot for search, cold for proof.

Finance should be in the room for this conversation. Log volume grows with user activity, workload count and diagnostic verbosity. A retention extension that looks modest in a console can become a material run-rate cost. Conversely, deleting logs early to save money can create a far larger cost when an incident response team has to reconstruct events from incomplete evidence. The review should include an estimated cost per source per tier, even if the figures are rough.

Put the schedule in writing

A retention schedule does not need to be a fifty-page policy. It does need to be written, owned and reviewed. At minimum, it should record the log source, the business or legal question it supports, the hot retention period, the archive period, the storage location, the owner and the review date. It should also state how legal hold works: if a dispute or investigation is anticipated, the organisation must be able to prevent routine deletion.

UK organisations should consider the storage-limitation principle under UK GDPR alongside their evidence needs. Keeping logs indefinitely “just in case” is not a neutral position; it creates cost, privacy risk and a larger breach surface. The schedule should therefore distinguish between data retained for a defined purpose and data retained because no one has decided what else to do with it. Legal and finance should sign off the periods, and IT should confirm that the configuration actually matches the schedule. A policy that says twelve months while the tenant is set to thirty days is not a policy; it is a hope.

A one-page template for the next cloud review

Before the next cloud review, take a single page with five columns: log source, question answered, hot period, archive period and owner. Add a final column for review date. Populate it with the sources that matter most to the estate — identity, productivity, infrastructure, network and security tooling — and mark any source where the current setting differs from the agreed period. That gap list is the practical output. It gives technical teams a clear change request, finance a cost estimate, and legal a record of the decision.

The console will always offer a default. A UK organisation’s job is to decide whether that default is good enough for the questions it may one day have to answer. The right moment to make that decision is before the incident, not after the logs have expired. Nuage Solutions advises treating retention as part of the cloud strategy record: a documented, costed and reviewable choice rather than an inherited setting.

All Articles

Related Articles

The Billing Trail Is Not an Asset Register: Owning Every Cloud Subscription in a UK Estate

The Billing Trail Is Not an Asset Register: Owning Every Cloud Subscription in a UK Estate

Counting the Minutes You Can Prove: A Service-Credit Readiness Routine for UK Cloud Buyers

Counting the Minutes You Can Prove: A Service-Credit Readiness Routine for UK Cloud Buyers

Before the Request Arrives: Rehearsing Subject Access Retrieval Across a UK Cloud Estate

Before the Request Arrives: Rehearsing Subject Access Retrieval Across a UK Cloud Estate