a month ago
Project: Every AI (e732bf93-fc6e-4cb1-b2a9-7bfbf37838bd)
Summary
Since ~24h ago your edge serves the WRONG TLS cert for several of our custom domains: a wildcard cert that does NOT cover the requested host, instead of that host's own (already-issued, valid) cert. Browsers reject with ERR_CERT_COMMON_NAME_INVALID. This is hitting 3 independent services at once, started with NO deploy on our side, and recurs after temporary fixes. It is NOT an issuance problem (issuance succeeds — CT proof below); it's an edge cert-SELECTION/binding problem.
The bug
A *.X wildcard cert cannot cover its parent X. For each broken host the edge serves exactly such a non-covering wildcard:
Host Railway target Cert SERVED Covers? HTTPS
every.ai (apex) 9fq42l1d *.every.ai NO FAIL
staging.every.ai ls8itskw *.staging.every.ai NO FAIL
staging.api.every.ai vql6o0b8 *.every.ai NO FAIL
www.every.ai (control) zgp01yde *.every.ai yes OK
app.every.ai (control) 4rsv7xyv *.every.ai yes OK
app.staging.every.ai (control) at5ylvn0 *.staging.every.ai yes OK
staging.every.ai vs app.staging.every.ai: both served *.staging.every.ai — it covers app.staging but NOT the bare staging (same as apex).Issuance works; selection is broken
crt.sh shows valid bare-apex every.ai certs were issued, incl. two yesterday:
CN=every.ai issued 2026-06-03T18:32:02Z exp 2026-09-01
CN=every.ai issued 2026-06-03T18:31:24Z exp 2026-09-01
Yet the edge serves the WILDCARD .every.ai (issued 2026-05-02, exp 2026-07-31). Both wildcards involved are ~a month old (.every.ai 05-02 18:52:57; *.staging.every.ai 05-02 18:52:06) — not new.
All DNS verified, cert still wrong (clincher)
Every affected domain shows verified/green in Networking (CNAME + _railway-verify TXT both checkmarked), yet the edge still serves a non-covering wildcard. DNS is satisfied; edge cert selection is not. This is not fixable from our DNS side.
Not isolated
Identical report for apex certified.one: https://station.railway.com/questions/prod-outage-certified-one-apex-tls-brok-cdefbeab — same "provisioned but not served at edge," same window, recurring, spreading to nested subdomains. Two customers = platform-side regression.
Likely upstream trigger: Cloudflare incident https://new.cloudflarestatus.com/incidents/j17t8xz91xs0 ("TLS certs using Let's Encrypt CA," SSL Certificate Provisioning, Jun 1 8:25PM–Jun 3 1:01AM EDT). Resolved on CF's side + they auto-rebuilt their chains — but our Railway-served certs are left wrong-at-edge.
Timeline / ruled out
Onset ~2026-06-03, stable for months prior. Broke with NO deploy. Only deploys in window were unrelated app-code (no DNS/cert/nginx config). Remove+re-add only fixes it temporarily then reverts (and risks Let's Encrypt rate limits).
What we need
Why is the edge serving a non-covering wildcard for every.ai / staging.every.ai / staging.api.every.ai instead of each host's own issued cert? Please rebuild/re-bind edge cert selection.
Is this a known incident, related to the CF cert incident? Was Railway's cert pipeline impacted, and are affected certs being rebuilt?
How do we make this durable so re-issuing isn't a recurring manual step?
This currently takes our production marketing domain (every.ai) down for all visitors.
1 Replies
a month ago
This thread has been opened as a public bounty so the community can help solve it. The thread and any further activity are now visible to everyone.
Status changed to Open Railway • about 1 month ago
a month ago
This is an excellent bug report and you're absolutely correct — this is a Railway edge cert-selection regression, not a user-side issue.
What's happening:
Railway's edge provisioned valid apex certs (confirmed via crt.sh) but the edge proxy is still serving the old wildcard cert instead. The cert binding/selection layer is broken — issuance and serving are decoupled and out of sync.
What you can try as a temporary workaround:
Remove and re-add the affected custom domains in Railway service settings — this sometimes forces the edge to re-bind the correct cert
If that reverts, don't keep removing/re-adding — you'll hit Let's Encrypt rate limits (5 certs per domain per week)
What actually needs to happen:
This is a Railway infrastructure fix. The edge needs to flush its cert-selection cache and re-bind the already-issued apex certs to the correct services. No amount of DNS changes or redeployments will fix a broken cert-binding layer.
Action needed:
You've already linked the identical report from certified.one — that confirms it's a platform regression affecting multiple customers
The likely trigger (Cloudflare/Let's Encrypt incident on Jun 1–3) left Railway's edge with stale cert bindings even after CF resolved their side
Escalate directly to Railway support with your project ID e732bf93-fc6e-4cb1-b2a9-7bfbf37838bd and reference both threads — this needs an engineer to force a cert re-binding on the edge nodes