The Platform PM

Rules and risks

Field Guide

Observability: metrics, logs and traces

07

Rules and risks

3 min read · Chapter 7 of 10

How the rules work

There's no observability regulator and no licence, so most of this section is about data rules and buyer rules that happen to land on telemetry. I'd still take them seriously, because the fines come from what's in the logs.

Personal data in logs. Telemetry carries IP addresses, user IDs, emails in URLs, auth tokens in headers and now LLM prompts. The largest GDPR fine I found about data sitting in internal systems is Meta's €91 million from Ireland's regulator in September 2024, for passwords stored in plaintext, partly for failing to notify and document the breach. France's CNIL recommends keeping access logs six months to a year, minimizing personal data in them, and storing them separately and append-only. Deleting one person's data from append-only stores is hard, so the common practice is to redact in the pipeline and keep retention short. The customer is the controller and carries the liability; the vendor is a processor.

Diagnostic files are the leak people forget. In 2023 an attacker reached support files for 134 Okta customers; some were browser recordings (HAR files) holding session tokens, and five customers' sessions were hijacked. Those files never went through a pipeline. Session replay in RUM tools has its own exposure: since a court struck down part of HHS's tracking guidance in 2024, claims over replay and tracking pixels have moved to state wiretap lawsuits against website operators.

Retention floors.

PCI DSS v4.0.1, 10.5.1

Who it covers
Systems in the card-data environment
Retention
12 months, the last 3 immediately available
Status (Oct 2026)
In force; automated log review required since March 31, 2025

OMB M-26-14

Who it covers
US federal civilian agencies
Retention
6 months actively searchable, 12 months retrievable
Status (Oct 2026)
In force since May 22, 2026

OMB M-21-31

Who it covers
US federal agencies
Retention
12 months hot plus 18 cold
Status (Oct 2026)
Rescinded May 22, 2026

CNIL recommendation

Who it covers
France, under GDPR
Retention
6-12 months for access logs
Status (Oct 2026)
Guidance

HIPAA audit controls

Who it covers
Covered entities and their vendors
Retention
No period for logs; policies kept 6 years
Status (Oct 2026)
Update proposed, stalled

California cybersecurity audits

Who it covers
Large businesses under CCPA
Retention
Audits cover audit-log management
Status (Oct 2026)
Effective January 1, 2026; first certifications 2028

The federal change is the clearest example of rules following the bill. M-26-14 cut the floor from 30 months to 12, and the memo itself says keeping "vast quantities of logging data without clear utility" was neither operationally feasible nor cost-effective. Its top maturity level grades signal: logs should generate actionable alerts covering at least 95% of baseline requirements. PCI's 12-and-3 maps neatly onto vendors' hot and archive tiers.

Buyer rules. FedRAMP decides who can sell to US federal agencies. Datadog for Government and Grafana's federal cloud say they're authorized at High, Dynatrace holds Moderate and said in July 2026 it would pursue High, and FedRAMP's new 20x process becomes mandatory on January 1, 2027. Listed US companies must file an 8-K within four business days of deciding a cyber incident is material, under SEC Item 1.05; trade groups want it rescinded, but I found no proposal as of October 2026. Telemetry is the evidence for scoping, and "we can't determine the scope" is an answer regulators read badly.

Residency and switching. Regions are fixed at setup. Datadog's sites are separate ("you cannot share data across sites"), and New Relic says the customer is "solely responsible" for choosing its region. Moving is a re-implementation. From January 12, 2027, the EU Data Act bars cloud and SaaS providers, observability vendors included, from charging EU customers switching or data-egress fees when they move.

SLAs and contracts. Most of the customer's protection sits here, and it's thin; see Liability allocation. EU financial firms have their own operational resilience rules (DORA), which I haven't covered.

What mistakes cost

  • Bill shock: a tag or a debug flag can add tens of thousands of dollars a month within days, and overage history raises the next commitment.
  • Ignored alerts: users find the outage first. In Splunk's UK cut of its 2025 survey, 54% said false alerts hurt morale.
  • Cost-cutting blind spots: dropped logs and sampled traces lengthen the next investigation, and rehydrating the archive mid-incident costs time you don't have.
  • Secrets in logs: Meta's €91 million fine was about passwords in internal systems, and Okta's 2023 breach ran through troubleshooting files.
  • Logs you didn't keep: after Storm-0558, the US Cyber Safety Review Board found Microsoft had "no evidence or logs" for parts of its explanation.
  • Vendor and agent failures: Datadog's 2023 outage and CrowdStrike's 2024 update, both in Liability allocation, left customers carrying most of the cost.
  • The wrong region: moving an account to another region is a rebuild, with dashboards and alerts recreated by hand.