3 hours ago
Hey everyone. I am looking into a question and having a hard time nailing down a concrete answer. And its a fun question so I thought I would ask everyone...
For some background... As I intend to not be fined under the GDPR for storing a user uploaded picture and the EU-US Data Privacy Framework has been challenged since June it seems easiest to simply store all data in the EU region for now until this political mess is resolved. It is also fast enough for my architecture that it is generally a non issue.
So the fun question. I have:
- An IPX server on Bun that optimizes images from a few domains (e.g. google account profile pictures) as well as my internal NextJS service via the internal networking and a Railway Bucket via Bun.s3 (at
/publicand/filerespectively) - A Caddy Reverse Proxy that routes traffic based on the route between the NextJS server, PostHog (EU, Frankfurt), my API server (ElysiaJS) and my Vite SPA for the core offline-first application.
So my question is, which is faster. US-East -> EU internal service traffic, or US POP (say JFK) -> EU traffic. Having a hard time nailing it down but I am curious if this has mattered to anyone at scale?
1 Replies
3 hours ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • about 3 hours ago
an hour ago
Both options cross the Atlantic. The distance is identical, so the thing that decides this isn't which link is faster, it's how many times you pay the crossing.
Railway's edge is a real terminating proxy, not a pass-through. You can see this in responses (server: railway-hikari, x-railway-edge: atl1), and it's been observed terminating HTTP/2 itself. There's a thread on here documenting the edge cancelling an h2 stream after the origin already returned 200, and closing idle client keep-alives at 60s. Both only happen if the POP terminates the connection.
That distinction is the whole answer:
Everything in EU, US users enter via a POP. The user's TCP + TLS handshake terminates at JFK, a few ms away. Only the request itself rides the long leg to Frankfurt, on a warm, pooled connection. A page load pays the Atlantic roughly once.
Compute in US-East talking to EU internally. Every internal call pays the crossing. US-East to EU is ~80-90ms RTT, and internal calls are chatty by nature. There's another thread on this board right now where someone's pages run 40-90 queries, and at 9ms each that's already 500ms a page. Put those same queries across the Atlantic and it's 40-90 x ~85ms. That's not a slow page, that's a broken one.
So: keep compute next to the data. Since GDPR is already pinning your storage to the EU, pin the services that talk to it there too and let the anycast edge absorb the distance for US users. That's also the cheaper answer operationally, one region to reason about.
The exception, and the only case where the split wins: a US-East service that makes very few EU calls per request and batches them. One or two round trips instead of fifty. Your Vite SPA being offline-first pushes you that way a little, but your IPX server doesn't. Image optimisation pulling from a Railway Bucket is exactly the chatty, per-asset pattern that punishes a split.
To your "has this mattered at scale" question, yes, and it usually shows up exactly as you'd expect: not as one slow request, but as page latency scaling with query count. The fix is almost always co-location rather than tuning the link.