Certificates Expire on Weekends

A 398-day certificate lifetime is 56 weeks and six days. A 90-day lifetime is 12 weeks and six days. Any certificate lifetime that is not a multiple of seven drifts backward across the calendar. When a certificate is issued during a weekend maintenance window, the day of the week walks backward until it lands on a Saturday or Sunday. The certificate did not fail. The arithmetic did.

The industry spent two decades trying to make certificate revocation work. It did not. The response was to shrink certificate lifetimes instead. The operational bill for that decision arrives on a Saturday at 2:00 a.m.

When the pager goes off, the on-call engineer looks at the monitoring dashboard. The dashboard says the certificate expired fourteen minutes ago. The engineer looks at the CMDB. The CMDB lists the owner as a web services team that was dissolved in a reorganization two years ago. The runbook says to contact the PKI distribution list. The list has one member, a contractor who rotated off the account in March. Every certificate has an owner. Most of them are wrong.

Certificate authorities send renewal warnings at thirty, fifteen, and seven days before expiration. These emails route to shared mailboxes created during the initial project deployment. Over time, the original engineers leave, the team reorganizes, and the mailbox rules route external alerts to a folder nobody checks. The reminder is not a control. It is a hope with a timestamp.

An inventory of issued certificates is not an inventory of certificates in use. The CMDB records what someone remembered to enter. The network scan finds what is currently listening. The load balancer API shows what is actively terminating traffic. None of them find the certificate embedded in a Java truststore, the one bought by an external agency on a corporate card, or the one sitting on a vendor appliance that requires a support ticket to update. The truth is the intersection of these sources, plus the surprises.

When the maximum validity for public TLS certificates drops to 90 days, manual renewal stops being a chore and becomes a mathematical impossibility. You cannot run a procurement process, generate a CSR, route the approval through a weekly Change Advisory Board, wait for the CA, and deploy the artifact four times a year for ten thousand endpoints. If your renewal process takes three weeks and the certificate lifetime is 47 days, the process is the outage. The queue never drains.

Automation is the only way out, but automation collides with change management. Every automated renewal generates a change record. If the CAB meets weekly and reviews every change, the automation bottlenecks at the same human gate it was supposed to bypass. The fix is a pre-approved change class for automated renewal. The automation itself is the change. The change record is the automation’s existence and scope, approved once and audited continuously.

Even when the renewal job reports success, the outage can still happen. The unit of work is not the certificate file. It is the path from issuance to a verified deployment.

A certificate can renew successfully while the load balancer, ingress controller, or application server keeps serving the old one. The new certificate sits in a secret store, but the application still has the previous one loaded in memory because it reads the keystore only at startup. The load balancer presents the new certificate to the outside world, but the backend presents the old one to clients validating the chain. Everything looks fine until a strict client connects.

In an active and standby load balancer pair, the renewal script updates the active node. The standby node does not get the new certificate. The failover happens at 4:00 a.m. on a Sunday, and the standby presents an expired certificate.

Expired certificates rarely announce themselves clearly to the monitoring system. They present as application timeouts, dropped database connections, opaque API failures, or a third-party webhook silently failing to post. The mean time to resolution is dominated by the time spent proving it is a certificate issue.

The 80 percent of certificates that run on modern platforms with ACME clients can be automated. They stop being interesting. The 20 percent that live on legacy appliances, proprietary vendor boxes, and older network gear require a vendor ticket, a remote session, and a scheduled outage. That long tail is where the outages live.

To avoid the manual effort of managing that tail, engineers buy wildcard certificates. That wildcard file is then passed around on shared drives, pasted into chat channels, and installed across fifty disparate systems. Wildcard certificates are just shared passwords disguised as infrastructure. When the wildcard expires, or when it needs to be revoked, the blast radius is the entire company, and validating the replacement takes a quarter.

Cloud migration and zero trust architectures did not reduce the certificate count. Workload identity and mutual TLS meshes increase the count by an order of magnitude. Inside the mesh, rotation is automatic. At ingress and egress, someone still owns a certificate manually.

Internal PKI sprawls alongside the public one. Developers find the internal CA slow to respond to tickets, so they generate self-signed certificates and force their applications to ignore TLS validation errors. Security scanners eventually flag the self-signed certificates, generating thousands of low-priority alerts that mask actual vulnerabilities.

Then there is the internal root certificate. A root with a twenty-year life is the root nobody has a plan for. When it nears expiration, the rotation is a multi-year program. It requires a client inventory, compatibility testing, cross-signing plans, and tracking down the kiosks, badge readers, and legacy Java apps that only trust the old root. It is not a ticket.

The test of a certificate program is whether the team can find a certificate, name the person who operates its endpoint, replace it, and confirm what the clients actually see. The date is on the certificate. The date is in the CMDB. The date is in the email. The only question is whether the renewal process is shorter than the lifetime.