a month ago
Project: qisstpay (id 71932004-50be-46ce-be77-8631393f96bf), environment production. Service: consumer (Next.js).
Two custom domains on the consumer service return HTTP 404 with header "x-railway-fallback: true" on every path, even though the certificate shows VALID and DNS is fully propagated:
- app.qisstpay.com (CNAME -> ix0492fy.up.railway.app, PROPAGATED, cert VALID)
-
- docs.qisstpay.com (CNAME -> q89pk9sz.up.railway.app, PROPAGATED, cert VALID)
The root domain qisstpay.com on the SAME consumer service works perfectly (200), and other services' custom domains (e.g. api.qisstpay.com on the api service) also work. The default *.up.railway.app URL for consumer serves fine too. So the app is healthy — the edge simply is not binding these two hostnames to the consumer service.
Example (404 fallback, not the app):
curl -sI https://app.qisstpay.com/ -> HTTP/2 404, server: railway-hikari, x-railway-fallback: true, x-railway-edge: lax1
Already tried, no effect: redeployed the consumer service; deleted + re-added the custom domain (updating DNS to the new CNAME target each time); triggered certificate issuance (cert went VALID, routing still 404s). Control plane reports the domain VALID with correct/propagated DNS, but the edge returns the fallback 404 — looks like a stuck edge binding for the consumer service's non-root custom domains.
Could you please re-bind app.qisstpay.com and docs.qisstpay.com to the consumer service on the edge? Happy to provide any additional IDs. Thanks!
Pinned Solution
a month ago
Your TXT record on _railway-verify.app.qisstpay.com is malformed. There's two railway-verify=. (Though I'm somehow able to access app.qisstpay.com just fine)
7 Replies
a month ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • 30 days ago
a month ago
Follow-up: verification done to pre-empt the usual user-side suspects, so you can go straight to the edge state.
-
Duplicate registrations: audited every project x environment x service in our account via the API — each qisstpay.com hostname is registered exactly once (9 domains, all in project qisstpay / production). No duplicates anywhere.
-
DNS/registrar: zone is served solely by Cloudflare (fay + rodney NS at the .com registry), DNSSEC unsigned (no stale DS from the Google Domains -> Squarespace migration), and Google/Cloudflare/Quad9 resolvers all return identical answers. No wildcard records.
-
Clean reproducer — docs.qisstpay.com (DNS untouched, NOT proxied, grey-cloud CNAME straight to q89pk9sz.up.railway.app): at the same moment, openssl shows your edge serving a valid cert CN=docs.qisstpay.com (issued Jul 18) while HTTP returns 404 with x-railway-fallback: true (x-railway-edge: lax1). Control: merchant.qisstpay.com — identical DNS-only CNAME pattern, same zone, different service — serves its cert AND routes 200.
-
So the edge holds the certificate for the hostname (SNI table knows it) but routes it to fallback (routing table doesn't) — an internally inconsistent state we can't produce from our side. Possibly related: control plane shows the working domains on edgeId edge-6705e5cc... and both broken ones on edge-500b32d4..., and neither broken domain obtained a cert until we manually called customDomainIssueCertificate — they appear stuck since creation.
Note: app.qisstpay.com currently 301s to the root via Cloudflare as a user-facing workaround, so please test against docs.qisstpay.com — it still exhibits the raw symptom.
a month ago
Your TXT record on _railway-verify.app.qisstpay.com is malformed. There's two railway-verify=. (Though I'm somehow able to access app.qisstpay.com just fine)
a month ago
Also, it may take up to a few hours (maybe even a day) for Railway to validate the DNS and issue certificates. If one isn't issued by then, try removing the domain from Railway and add it back after ~10-15 mins. Update DNS records as necessary.
a month ago
Good catch, thank you! Fixed and in progress:
-
TXT on _railway-verify.app corrected to a single railway-verify= prefix (propagated on 1.1.1.1 and 8.8.8.8). Worth noting where the doubled prefix came from: the
railway domainCLI prints the Value column as "railway-verify=railway-verify=" — it got pasted faithfully at setup time. Might be worth a CLI bug report. -
Heads up: you're able to access app.qisstpay.com because we put a Cloudflare 301 -> qisstpay.com in front of it as a user-facing workaround (it's proxied now, which is also why its CNAME check shows REQUIRES_UPDATE). The raw symptom is still reproducible on docs.qisstpay.com — grey-cloud CNAME straight to your edge, valid cert served, 404 x-railway-fallback.
-
Interesting datapoint: docs' TXT was already well-formed (single prefix), and docs shows the same routing failure — so the malformed TXT may not be the whole story.
-
Following your advice, I've removed docs.qisstpay.com and will re-add after ~15 minutes, then update DNS to the new target and report back here once it's had time to validate.
axiomstate
Good catch, thank you! Fixed and in progress: 1. TXT on _railway-verify.app corrected to a single railway-verify= prefix (propagated on 1.1.1.1 and 8.8.8.8). Worth noting where the doubled prefix came from: the `railway domain` CLI prints the Value column as "railway-verify=railway-verify=<token>" — it got pasted faithfully at setup time. Might be worth a CLI bug report. 2. Heads up: you're able to access app.qisstpay.com because we put a Cloudflare 301 -> qisstpay.com in front of it as a user-facing workaround (it's proxied now, which is also why its CNAME check shows REQUIRES_UPDATE). The raw symptom is still reproducible on docs.qisstpay.com — grey-cloud CNAME straight to your edge, valid cert served, 404 x-railway-fallback. 3. Interesting datapoint: docs' TXT was already well-formed (single prefix), and docs shows the same routing failure — so the malformed TXT may not be the whole story. 4. Following your advice, I've removed docs.qisstpay.com and will re-add after ~15 minutes, then update DNS to the new target and report back here once it's had time to validate.
a month ago
For 3, the docs domain seemed to be configured just fine. Probably just needed a longer wait or a remove/readd process.
a month ago
Confirmed fixed for docs — the remove/re-add cycle worked. Removed the domain, waited ~12 minutes, re-added, pointed the CNAME at the new target (ghza56mb), and docs.qisstpay.com now serves 200 from the app with no x-railway-fallback, on the same lax1 edge that was 404ing. Running the identical cycle for app.qisstpay.com now; once it binds I'll drop the Cloudflare redirect workaround and report back.
One small thing for the team: the railway domain CLI reproducibly prints the TXT Value column as railway-verify=railway-verify= (doubled prefix) — it did it again on this re-add. Easy for users to paste verbatim and end up with a malformed record.
a month ago
All resolved — app.qisstpay.com is also fixed via the same cycle (remove → wait ~12 min → re-add → point CNAME at the new target, grey-cloud). Both domains now serve 200 directly from the edge with their own valid certs, no fallback:
- app.qisstpay.com → 200 (CN=app.qisstpay.com)
-
- docs.qisstpay.com → 200 (CN=docs.qisstpay.com)
Cloudflare redirect workaround retired (record un-proxied, so the rule no longer applies). Accepted your remove/re-add answer as the solution — thanks for the fast help. Only open item for the team is the CLI's doubled railway-verify= prefix in the TXT output, which reproduced on all three re-adds today.
Status changed to Solved axiomstate • 30 days ago