3 months ago
What's going on?
Attachments
35 Replies
3 months ago
same here.
3 months ago
yes getting the same
Error 503 Backend.max_conn reached
Backend.max_conn reached
Error 54113
Details: cache-ams2100114-AMS 1772808447 1303507420
Varnish cache server
I've had 4 periods just in the least week and a half. Average 98% uptime - pretty shocking.
3 months ago
Yeh, same here, happended a few times this month.
3 months ago
Same for me!
3 months ago
same
3 months ago
same
3 months ago
does it have anything todo with the new fastly implementation they have?
3 months ago
Damn. Again?
3 months ago
Same here … for my staging and production app. And even for the the Central Station!
Attachments
3 months ago
It looks like it's working fine now?
3 months ago
It's fixed on my services now, but it's troublesome even a few downtime can cause a lot of issues for my apps 🥲
3 months ago
Strange Railway status page says 'All systems operational' and has done throughout this issue. Also, no message from support clarifying the situation. Not great!
3 months ago
same here
3 months ago
Same here. We are running a service based business and have hosting on railway and things like these can cause clients to churn. I would recommend everybody setup cloudflare workers that catch these things because railway cannot be trusted it seems
3 months ago
Same here
3 months ago
Looks like it was related to the new CDN: https://status.railway.com/cmmf10gc2009qtgwc63k5cxmr
I'm all for shipping fast, but perhaps this feature was prematurely released, its caused quite a bit of pain <:cryfistsq:1238327748464873472>
3 months ago
Fastly has been around for a long time, so we are just as surprised as you are to see there have been some issues.
@Brody is there a way we can turn off CDN for our deployments at all? I'm also noticing some transient/network style issues from time to time as I access my deployment in New York from London, it enters 'railway' via Amsterdam and I'd far rather I get to hit my deployment in US-east directly
3 months ago
The CDN is not user-configurable.
3 months ago
I saw the same problem too. First, I thought it was my app somehow, but NO.
After reloading my Railway dashboard I saw the exact same error message there as well.
AND:
This is NOT the first time I saw this error.
Was at least once before (a day or so ago).
And now it's clear, the problem is NOT coming from my app.
3 months ago
Totally get it, and I'm aware Fastly has been around for a long time. I hope there are some solid SLAs in your agreement with them 😅
Either way, I cause way more reputational damage to my app (bugs/downtime etc) than Railway ever could, and this was a relatively quick incident anyway.
I know the issues in the past little while have generally been independent events (and sometimes downstream related), but a post/some acknowledgement mentioning the team's (continued) focus on stability would go a long way (for lil ol' me at least), rather than only the post-incident RCAs.
Appreciate the team, and esp you for being so responsive 🙏 <:railway:801924099960995912>
3 months ago
@brody are you saying this is Fastly's fault and not Railway's?
3 months ago
We are working together with Fastly to make sure this doesn't happen again.
2 months ago
We have put in some configuration changes that should prevent this from happening, but please let us know if you ever see this again.
CC @Jasper @matt9756y @Jalen @Pepijn
Status changed to Solved brody • 2 months ago