Changed to public proxy to save the application for now. @railway please fix the issues with networking, I am not able to convince the clients to stay on railway for long if this just keeps happening every few days.

I flipped it back to internal network again and for now it seems to be working. but I am very afraid if this is randomly going down.

Hey there,

For you and others experiencing network issues at the time. Confirmed, we did have network impact. We're sorry. What happened was we had SYN attacks to some workloads. We have since mitigated the issues but if there was business impact, let us know, we would like to make it right.

thanks for confirming. Luckily I was online at the time and I was able to swing to public proxy and there was no business impact. Sorry that in the moment I was a bit frustrated. I love Railway's platform and you resolve our issues pretty quickly when they come, I would love if they didn't happen at all but with so much power in hackers hands these days with AI tools this is bound to happen.

We're glad you were able to avoid any business impact by switching to the public proxy, and we appreciate your patience and understanding.

The whole summary so far on the readout is we had SYN attacks to some workloads. You may have had Cloudflare in front of your app, however, non WAF workloads could have affected the availability. As such we have implemented short terms measures to prevent this moving forward. Since then we've expanded interconnect capacity with new peering and Direct Connect, deployed per-container eBPF firewalls, hardened our edge, and built continuous connectivity monitoring between hosts. We're also rolling out a WAF for all customers, edge-level protection against malicious request patterns across all plans.

We have more to be written come tomorrow. Again, sorry for the (deserved) frustration.