Production outage - Apex domain certificate missing on Railway edge (shop-orders.com)
justinb470
PROOP

a month ago

Hello Railway Support,

We are experiencing a production outage affecting our apex domain:

shop-orders.com

Cloudflare is returning Error 526 while running in Full (Strict) mode.

After investigation, we found evidence that the issue appears to be an edge certificate propagation problem within Railway rather than a DNS, application, or certificate issuance problem.

What we observed:

  • Railway control plane reports:

    • shop-orders.com certificateStatus = VALID
    • *.shop-orders.com certificateStatus = VALID
  • The apex certificate exists and is valid.

  • The application itself is healthy and returns HTTP 200 when reached directly.

  • All wildcard subdomains remain healthy.

Certificate behavior appears inconsistent across Railway edge infrastructure:

Old Railway edge:

  • SNI: shop-orders.com
  • Presents CN=shop-orders.com
  • Hostname verification succeeds

Current Railway edge serving traffic:

  • SNI: shop-orders.com
  • Presents CN=*.shop-orders.com
  • Hostname verification fails because the wildcard does not cover the apex domain
  • Cloudflare Full (Strict) correctly rejects the connection and returns 526

Additional findings:

  • shop-orders.com worked previously.
  • The outage began suddenly without DNS or certificate changes on our side.
  • The behavior appears consistent with traffic being moved to a newer Railway edge where the apex certificate is not loaded.
  • Wildcard subdomains continue functioning because the wildcard certificate is present on that edge.

Request:

Can you please investigate whether the valid apex certificate for shop-orders.com has failed to propagate to the current Railway edge infrastructure serving the domain?

Specifically:

  1. Verify which edge fleet is currently serving shop-orders.com.
  2. Verify that the shop-orders.com certificate is attached to all active edge nodes.
  3. Re-provision or re-push the apex certificate to the active edge infrastructure if necessary.
  4. Confirm whether this is related to any recent Railway edge migrations or certificate propagation events.

This is currently impacting production access to our primary platform domain.

Thank you.

Temporary mitigation: We can switch Cloudflare from Full (Strict) to Full if necessary, but we would like to understand and resolve the underlying Railway certificate propagation issue rather than leave the platform in a non-strict configuration.

$20 Bounty

3 Replies

Railway
BOT

a month ago

This thread has been opened as a public bounty so the community can help solve it. The thread and any further activity are now visible to everyone.

Status changed to Open Railway about 1 month ago


I can access your domain just fine. There doesn't seem an issue with it currently. Have you tried to access it from an incognito tab, or use a different network?

image.png

Attachments


darseen

I can access your domain just fine. There doesn't seem an issue with it currently. Have you tried to access it from an incognito tab, or use a different network? ![image.png](https://station-server.railway.com/attachments/att_01kt7egmn6fk38vxbthpxp1j69)

justinb470
PROOP

a month ago

It's only up right now because I changed my Settings in Cloudflare to Full instead of Full Strict. This is a temporary fix. Railway is still causing the items I listed in the first support ticket. It being up at this exact moment has nothing to do with what is actually happening behind scenes with Railway. That is the problem I need railway to investigate and fix. I Also am having SSL Provisioning Stuck for Days at a time which is completely unacceptable. I was told give it 24hrs and it's been 48hrs now.


justinb470

It's only up right now because I changed my Settings in Cloudflare to Full instead of Full Strict. This is a temporary fix. Railway is still causing the items I listed in the first support ticket. It being up at this exact moment has nothing to do with what is actually happening behind scenes with Railway. That is the problem I need railway to investigate and fix. I Also am having SSL Provisioning Stuck for Days at a time which is completely unacceptable. I was told give it 24hrs and it's been 48hrs now.

It's supposed to be on Full. Using Full Strict may cause intended behavior.


Welcome!

Sign in to your Railway account to join the conversation.

Loading...