Degraded network performance from west coast / florida and other attempts
johndavi
PROOP

a month ago

My service is healthy with 1% CPU and 100% DB cache hit. Users in Atlanta see <100ms response times, but West Coast and Miami users are seeing 800ms–1.7s TTFB. My logs are showing some TCP_OVERWINDOW and NO_SOCKET.

Is anyone else experiencing poor networking from external locations?

Solved

4 Replies

johndavi
PROOP

a month ago

ex:

Attachments


Railway
BOT

a month ago

Hello!

We've escalated your issue to our engineering team.

We aim to provide an update within 1 business day.

Please reply to this thread if you have any questions!

Status changed to Awaiting User Response Railway 26 days ago


Thanks for your patience! The networking issue affecting response times has been resolved. We provisioned additional proxies and addressed the root cause, which was a blast radius issue from traffic affecting certain regions. You should now see improved TTFB across all regions. If you're still experiencing elevated latency, please let us know.


Status changed to Solved johndavi 26 days ago


johndavi
PROOP

a month ago

Man. I am sure you are getting it from all corners but this is three straight days. I can’t believe “migrate off Railway” is emerging as a todo. What do I need to do to avoid this?


Status changed to Awaiting Railway Response Railway 26 days ago


johndavi

Man. I am sure you are getting it from all corners but this is three straight days. I can’t believe “migrate off Railway” is emerging as a todo. What do I need to do to avoid this?

Apologies, we had another networking incident affecting public traffic across multiple regions including US West, which was resolved earlier today. I am happy to get into it more in as much detail but the long short of it is, we had SYN attacks to some workloads.

You may have had Cloudflare in front of your app, however, non WAF workloads could have affected the availability. As such we have implemented short terms measures to prevent this moving forward. Since then we've expanded interconnect capacity with new peering and Direct Connect, deployed per-container eBPF firewalls, hardened our edge, and built continuous connectivity monitoring between hosts. We're also rolling out a WAF for all customers, edge-level protection against malicious request patterns across all plans.

I have also gone ahead and credited your workspace $100 (about three months of cost) as a sign of goodwill. I would hate to see you migrate, but know we need to earn your business. Will be around if you have any further questions, just tag.


Status changed to Awaiting User Response Railway 25 days ago


Railway
BOT

18 days ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway 18 days ago


Loading...