Single client being routed to multiple different regions within seconds
fourth-wall
PROOP

3 months ago

Hello everyone, I have deployed an astro js site using multiple regions, specifically 1 for each of the 4 available in railway, 1 replica each. I noticed that the same user connection was randomly being rerouted to different regions. I added logs in my middleware so every time a navigate between pages it looks like this:

const region = process.env.RAILWAY_REPLICA_REGION || "default";
console.log(`[REGION] - ${region}`);

with result:

[REGION] - us-west2
[REGION] - us-west2
[REGION] - europe-west4-drams3a
[REGION] - europe-west4-drams3a
[REGION] - europe-west4-drams3a
[REGION] - us-west2
[REGION] - asia-southeast1-eqsg3a
[REGION] - us-east4-eqdc4a

As far as I understand, railway edge networking automatically detects and routes to the nearest edge location. In fact when i try to curl my site for grep -i X-Railway-Edge I always get the EU west edge which is closest to my location. In my site however it almost looks like it's swapping replicas per request. That wasn't the case at all when I first deployed.

The problem emerged unexpectedly without me changing any deployments in any environment or service. This leads me to believe that it is related to infrastructure, and having tested with different networks and users, seems to yield the same issue. Is this normal, or maybe something wrong with my system ? Is RAILWAY_REPLICA_REGION the correct variable to at least verify which region is being hit ? What could be causing this all of a sudden ?

FYI i noticed the issue first around 17:00 EST, March 6.

Thank you in advance.

Solved

4 Replies

sam-a
EMPLOYEE

3 months ago

Yes, RAILWAY_REPLICA_REGION is the correct variable to verify which region is serving a request. What you're seeing is not expected behavior. With multi-region replicas, requests entering through an edge location should be routed to the nearest deployment region, so an EU client hitting the EU west edge should consistently land on the europe-west4-drams3a replica. We're tracking this as a platform issue for the team to investigate.

Do let us know if you see it again.


Status changed to Awaiting User Response Railway 3 months ago


sam-a

Yes, `RAILWAY_REPLICA_REGION` is the correct variable to verify which region is serving a request. What you're seeing is not expected behavior. With multi-region replicas, requests entering through an edge location should be routed to the nearest deployment region, so an EU client hitting the EU west edge should consistently land on the `europe-west4-drams3a` replica. We're tracking this as a platform issue for the team to investigate. Do let us know if you see it again.

fourth-wall
PROOP

2 months ago

As of now I'm still seeing the same switching behaviour. I also tried promoting an older preprod deployment temporarily but still nothing. If this changes or when it gets fixed I will let you know thanks.


Status changed to Awaiting Railway Response Railway 2 months ago


sam-a
EMPLOYEE

2 months ago

Thanks - we'll let you know when the underlying issue is resolved.


Status changed to Awaiting User Response Railway 2 months ago


sam-a
EMPLOYEE

2 months ago

The issue should be resolved. Redeploying the service will make it take effect. Let us know if you see any further issues.


Status changed to Solved sam-a 2 months ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...