14 days ago
app is hosted in us-west and i'm in new zealand, which means i should be getting 150-210ms but i'm seeing:
when hitting API routes:
400ms from NZ through railway domain
http logs have an upstreamRqDuration of roughly 250ms
600-1200ms from NZ when hitting a cloudflare cache miss
50ms from a digitalocean server in california through the railway domain
100ms from a digitalocean server in california when hitting a cloudflare cache miss
50-110ms from a friend in california when going through cloudflare
rendering the root document:
1000ms from NZ through railway domain
http logs have an upstreamRqDuration of roughly 700ms
1200-2000ms from NZ through cloudflare
for context, the app is a tanstack start app using bun. in the logs, i've been logging how long it's taking the loaders to execute, and they're all between a couple ms and at most 40-70ms for some heavier ones.
running on my local machine the root document returns in 50-75ms and API routes return in around 5-10ms, so there's a massive disconnect between actual render time and network performance. while i do have a 5700X3D in my PC, i don't believe the hardware here would be causing significantly worse performance, certainly not when considering the railway render performance benchmarks that were floating around the last couple weeks wrt cloudflare workers performance.
traceroute to railway domain:
```
1 2 ms 1 ms 1 ms 192.168.1.254
2 5 ms 5 ms 5 ms 222-152-80-1-fibre.sparkbb.co.nz [222.152.80.1]
3 * 29 ms 29 ms 122.56.119.217
4 29 ms 29 ms 29 ms 122.56.119.216
5 54 ms 55 ms 69 ms et5-0-2.sgbr3.global-gateway.net.nz [122.56.119.26]
6 52 ms 55 ms 53 ms 103.13.80.125
7 55 ms 63 ms 53 ms ae-3.r21.sydnau06.au.bb.gin.ntt.net [129.250.3.158]
8 149 ms ae-10.r24.sngpsi07.sg.bb.gin.ntt.net [129.250.6.149]
9 147 ms 145 ms 148 ms ae-4.a02.sngpsi07.sg.bb.gin.ntt.net [129.250.6.63]
10 145 ms 145 ms 145 ms ce-2-0-0.a02.sngpsi07.sg.ce.gin.ntt.net [116.51.16.19]
11 145 ms 145 ms 145 ms 66.33.22.111
```
traceroute to my california digitalocean server:
```
1 2 ms 2 ms 2 ms 192.168.1.254
2 6 ms 6 ms 5 ms 222-152-80-1-fibre.sparkbb.co.nz [222.152.80.1]
3 * * * Request timed out.
4 30 ms 29 ms 30 ms 122.56.119.216
5 31 ms 29 ms 30 ms ae10-10.tkbr12.global-gateway.net.nz [202.50.232.29]
6 155 ms 156 ms 155 ms ae4-10.lebr7.global-gateway.net.nz [122.56.127.78]
7 166 ms 165 ms 165 ms ae0-10.lebr8.global-gateway.net.nz [202.50.232.42]
8 * * 159 ms lag-14.ear2.lax1.sp.lumen.tech [4.68.37.89]
9 161 ms 161 ms * ae1.3505.edge9.sanjose1.net.lumen.tech [4.69.219.61]
10 158 ms 161 ms 159 ms 4.7.18.10
11 * * * Request timed out.
12 * * * Request timed out.
13 * * * Request timed out.
14 163 ms 162 ms 162 ms <my server ip>
```
8 Replies
14 days ago
Hey there! We've found the following might help you get unblocked faster:
📚 Deploy Static Sites with Zero Configuration and Custom Domains
🧵 Very high and inconsistent round trip times to backend from end user
If you find the answer from one of these, please let us know by solving the thread!
14 days ago
Hello!
We've escalated your issue to our engineering team.
We aim to provide an update within 1 business day.
Please reply to this thread if you have any questions!
Status changed to Awaiting User Response Railway • 14 days ago
14 days ago
✅ The ticket Connectivity Issue in New Zealand has been marked as completed.
14 days ago
🛠️ The ticket Connectivity Issue in New Zealand has been marked as backlog.
14 days ago
Ignore the bot, moved it to the networking engineer backlog queue.
6 days ago
This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!
Status changed to Solved Railway • 6 days ago
6 days ago
any update at all?
Status changed to Awaiting Railway Response Railway • 6 days ago
5 days ago
Are you still experiencing suboptimal routing? If so, another traceroute would be appreciated.
Status changed to Awaiting User Response Railway • 5 days ago
4 days ago
somewhat yeah
from my system in NZ:
1 2 ms 3 ms 1 ms 192.168.1.254
2 7 ms 5 ms 7 ms 222-152-80-1-fibre.sparkbb.co.nz [222.152.80.1]
3 * * * Request timed out.
4 29 ms 29 ms 29 ms 122.56.119.216
5 30 ms 40 ms 29 ms ae10-10.tkbr12.global-gateway.net.nz [202.50.232.29]
6 53 ms 53 ms 53 ms et3-0-0.sebr3.global-gateway.net.nz [122.56.127.50]
7 66 ms 59 ms 65 ms 103.13.80.121
8 54 ms 53 ms 53 ms ae-1.r23.sydnau05.au.bb.gin.ntt.net [129.250.2.56]
9 * 146 ms 145 ms ae-10.r24.sngpsi07.sg.bb.gin.ntt.net [129.250.6.149]
10 146 ms 146 ms 146 ms ae-4.a02.sngpsi07.sg.bb.gin.ntt.net [129.250.6.63]
11 146 ms 145 ms 145 ms ce-2-0-0.a02.sngpsi07.sg.ce.gin.ntt.net [116.51.16.19]
12 145 ms 145 ms 145 ms 66.33.22.111
from my SF digitalocean server:
1 138.68.34.246 (138.68.34.246) 0.609 ms 0.572 ms 138.68.34.247 (138.68.34.247) 0.491 ms
2 143.244.192.72 (143.244.192.72) 0.532 ms 143.244.192.76 (143.244.192.76) 0.473 ms 143.244.192.196 (143.244.192.196) 0.490 ms
3 143.244.227.100 (143.244.227.100) 0.991 ms 143.244.227.98 (143.244.227.98) 1.007 ms 143.244.227.100 (143.244.227.100) 1.046 ms
4 143.244.225.51 (143.244.225.51) 0.927 ms 143.244.225.49 (143.244.225.49) 0.880 ms 143.244.225.45 (143.244.225.45) 0.849 ms
5 ae-38.a04.snjsca04.us.bb.gin.ntt.net (157.238.227.40) 1.230 ms 1.167 ms *
6 ae-2.r26.snjsca04.us.bb.gin.ntt.net (129.250.5.208) 1.191 ms ae-1.r26.snjsca04.us.bb.gin.ntt.net (129.250.5.24) 1.086 ms ae-2.r26.snjsca04.us.bb.gin.ntt.net (129.250.5.208) 1.058 ms
7 ae-10.r26.asbnva02.us.bb.gin.ntt.net (129.250.6.0) 60.328 ms 60.284 ms 60.285 ms
8 ae-6.a05.asbnva02.us.bb.gin.ntt.net (129.250.3.255) 60.070 ms 59.863 ms 59.911 ms
9 ce-1-1-0.a05.asbnva02.us.ce.gin.ntt.net (157.238.225.207) 60.228 ms 60.211 ms 60.193 ms
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
using hey to send 5 requests sequentially to a small api endpoint from my DO server results in:
Summary:
Total: 0.9931 secs
Slowest: 0.4451 secs
Fastest: 0.1350 secs
Average: 0.1986 secs
Requests/sec: 5.0348
Total data: 295 bytes
Size/request: 59 bytes
doing the same but on localhost:
Summary:
Total: 0.0191 secs
Slowest: 0.0068 secs
Fastest: 0.0028 secs
Average: 0.0038 secs
Requests/sec: 261.7301
Total data: 295 bytes
Size/request: 59 bytes
i've implemented more compression into my bun server which is cutting the root document response time to 650-700ms. so assuming a render time of 50-75ms, it's inching closer to what i'd expect to be getting from NZ -> SG -> US. but this still doesn't explain a 59 byte api route taking 200ms from within the same state.
i know i can't expect railway to stand up an edge region for australia/nz so i'm probably just gonna have to put up with being routed through singapore or just move the deployment to singapore. as a workaround for those of us in smaller regions, it would be nice to see something along the lines of an option to opt out of global edge and just use a regional edge to serve the railway proxy, then we can just have cloudflare handle edge routing
Status changed to Awaiting Railway Response Railway • 4 days ago
4 days ago
New Zealand is closer to Singapore than it is to California, so it makes sense that you are hitting the Edge region that is closest to you first.
I would indeed recommend deploying to the region that is closest to you.
Status changed to Awaiting User Response Railway • 4 days ago