Dead Man's ROA: Outages Caused by Expired ROAs and Stale Manifests in a Self-Hosted CA

Running your own RPKI certificate authority gives you control that hosted RPKI doesn't. You can issue ROAs on your own schedule, automate them from your IPAM, and keep signing keys inside your own HSM. The cost is that you now own a publication system with hard expiry deadlines, and nothing in BGP will warn you when one is missed.

This article covers the failure we call the dead man's ROA. A prefix that was correctly authorized yesterday becomes unreachable today, because a manifest, CRL, or ROA lapsed and nobody noticed.

How the failure happens

A delegated RPKI setup publishes a bundle of signed objects into a repository directory, called a publication point:

• ROAs authorize an origin AS for a prefix, up to a maximum length.
• A manifest (.mft) lists every file in the directory with its hash.
• A CRL (.crl) lists revoked EE certificates.
• The CA certificate is issued by your parent, which is usually the RIR or an upstream LIR.

Each object has its own validity window, and relying-party software such as Routinator, rpki-client, or OctoRPKI enforces every one of them. The ones that bite are short-lived:

Manifests and CRLs are the dangerous ones. Software like Krill or rpkid re-signs them automatically, well before expiry. So the failure is almost never "someone forgot." Usually the re-signing process has stopped:

• The Krill daemon crashed or was OOM-killed, and nothing restarted it.
• The HSM or PKCS#11 token went offline, so signing fails silently.
• The disk holding the data directory filled up.
• The publication server (rsync or RRDP) is up but serving a stale snapshot.
• The CA's own certificate wasn't renewed because the parent-child exchange broke after a firewall change.

Everything keeps serving. The files are still there and the repository answers requests. But once the manifest's nextUpdate passes, validators discard the directory.

What the network sees

When a validator drops your ROAs, the prefixes they covered lose their validated ROA payloads (VRPs). What happens next depends on what else exists in the RPKI:

• Not found: if no other ROA covers the prefix, its state goes from valid to not-found. Most ROV-enforcing networks still accept not-found, so you may see only a quiet loss of protection.
• Invalid: if a covering ROA remains, for example one for a less-specific aggregate or a different origin, your more-specific announcement becomes invalid. Networks that enforce ROV drop it, and the routes vanish.

The second case is the outage. It is common when you announce /24s from an aggregate and only the aggregate's ROA survived, or when a stale max-length leaves a more-specific uncovered.

Detection is hard because the failure is partial and uneven. Large transit providers and IXP route servers that enforce ROV will drop the routes. A smaller network without ROV will not. Your own looking glass shows everything fine, and customers on affected paths report "your site is down from some places."

Validators also cache. Routinator and rpki-client keep the last good data for a limited period, and RTR sessions to routers refresh on their own timers. The symptoms can appear hours after the expiry, so they look unrelated to any change you made.

Why an attacker or auditor cares

A lapse is not an attack, but it creates an opening. When your legitimate ROA disappears and your prefix becomes not-found, a hijacker announcing it faces no ROV filtering at all. An origin hijack that would have been invalid and dropped everywhere is now accepted by networks that are strict about ROV. The window lasts as long as your objects stay expired.

An auditor will flag this as a control failure: RPKI is a claimed protection, but there is no evidence anyone monitors its health. In compliance reviews of routing security, such as those following MANRS-aligned programs, the question is not just "do you have ROAs?" but "how would you know if they stopped being valid?"

What it costs

• Direct outage for any customer or service reached over ROV-enforcing paths. A single missed manifest on a Friday evening can last the weekend.
• Slow recovery. Even after you fix the signer, validators re-fetch on their own schedules, and RTR propagation to routers adds time. Plan for 30 minutes to a few hours.
• Incident response across several teams, because the symptom looks like a network problem and the cause sits in the PKI.
• Damaged trust. Upstreams who see repeated invalid announcements from you may be slower to help the next time.

How monitoring catches it

You need to watch the objects themselves, not only the daemon. Three checks cover most failures.

1. Manifest and CRL freshness. Fetch your publication point from outside your network and read nextUpdate. You can use rpki-client's output or a plain openssl check:

# Print manifest validity from a downloaded .mft
openssl cms -verify -inform DER -in your-ca.mft -noverify -out /dev/null
rpki-client -vv -f your-ca.mft 2>&1 | grep -E "Next update|Valid until"

Alert when remaining validity falls below half the normal lifetime. If you re-sign every 8 hours with a 24-hour window, alert when under 12 hours remain.

1. VRP presence in an independent validator. Query a public validator or run your own, and assert that each expected prefix and origin appears:

curl -s http://localhost:9556/api/v1/validity/AS64500/192.0.2.0/24
# Expect: "state": "valid"

Do this for every announced prefix, from a validator you do not operate through your own CA. A stale invalid or not-found result is the earliest signal that routers are about to act.

1. Chain checks. Verify the CA certificate expiry and that the parent-child exchange succeeds. A cert that expires in 30 days should page someone today.

Compare your announced routes against validated state continuously. A route that is announced but not valid is the condition to alert on.

Mitigation and implementation

1. Monitor from outside. Run a validator in a separate environment and probe your prefixes at least every 15 minutes. Use a different failure domain than the CA.
2. Alert on remaining lifetime, not on expiry. Set thresholds at 50% and 25% of the object lifetime for manifests and CRLs, and at 60 and 30 days for the CA certificate.
3. Supervise the signer. Run Krill or your CA under systemd with Restart=on-failure, and add a watchdog that checks the last-signed timestamp rather than process liveness.
4. Monitor the HSM path. Test a signing operation periodically so a dead token is noticed before the next re-sign.
5. Lengthen manifest validity moderately. If your operations are weak, a longer window gives you time to react. Be careful, though: longer windows also extend how long a compromised or stale state stays valid.
6. Keep a runbook and run a drill. Document how to regenerate the manifest, force a parent sync, and publish. Practice it once.
7. Consider hosted RPKI as a fallback. If you cannot staff the monitoring, the RIR's hosted service moves expiry handling to someone whose full-time job it is.
8. Review max-length. Make sure the more-specifics you announce are covered by an exact ROA, so a surviving aggregate ROA does not turn them invalid.

Key takeaways

• A self-hosted CA turns RPKI from a set-and-forget record into a service with deadlines. Manifests and CRLs expire in days, not months.
• Expiry makes validators discard whole publication points. Prefixes become not-found (lost protection) or invalid (dropped by ROV networks).
• The outage is partial and delayed, which makes it hard to diagnose and easy to miss.
• A lapse also opens a window for hijackers, since not-found routes are not filtered.
• Monitor object lifetimes and validated state from an independent vantage point, and alert on remaining validity rather than on failure.
• Supervise the signing path, including the HSM, and rehearse recovery before you need it.