Our deepest apologies for the downtime today between 13:17 and 14:18 UTC.

Tens of thousands of businesses trust Polar with their billing, and an incident like this should not happen. We'll publish a full post-mortem in the coming days and take actions to make sure it does not happen again. In the meantime, here's exactly what happened.

We rotate secrets frequently as a security policy. Today we rotated our database credentials – but a flaw in our config, where the credentials were tied to the database's Terraform resource, caused Terraform to trigger a full resource replacement. Our production database was replaced. It took us an hour to restore and bring everything back.

Was any data lost? No. We run point-in-time recovery with remote backups, hosted on Render with enterprise support. We immediately began a PIT recovery, replaying WAL up to the moment before the Terraform change applied. Recovery completed at 14:16 UTC; we confirmed full data integrity and brought infrastructure back online at 14:18 UTC.

A few edge cases we're reconciling now – none involve lost data:

• Our API stayed up for ~50s after the latest point in time recovery point and kept accepting event ingestion (which we double-write into Tinybird). A few hundred events landed there; we're going to replay them.
• We received 3 Stripe webhooks for in-flight transactions as we went down. We're resolving these and reaching out to the affected customers directly.
• After restore, our webhook worker was properly restarted to point to the recovered database. That's fixed; it's catching up now, so webhook delivery is delayed but recovering.

Polar is fully operational. The only residual effect was delayed webhook delivery while the worker catched up, which it has now. No action is required on your end.

Our post-mortem will detail the full timeline and the guardrails we're adding so this class of failure can't recur, and we’re truly sorry it happened in the first place.

Everything should be back to normal.

We're having trouble connecting to our database. We're investigating the issue.