15 days ago
We are starting to see an increase in 524 errors from Cloudflare, saying that the origin is taking too long. Has anything changed on Railway’s side?
We’re checking on our side as well to see if it’s an issue on our end. We just found it odd that the request isn’t being registered by our observability tools, so it might be failing before that.
I also turned off Cloudflare to see if the issue is related to them.
15 Replies
15 days ago
found out that one of my redis databases are unreachable
15 days ago
(no markdown available for this content)
15 days ago
and a re-deploy doesn't take place, seems similar to: https://discord.com/channels/713503345364697088/1518697501547298997
15 days ago
also seeing that the metrics tab is totally empty but might be related to the current incident.
Attachments
15 days ago
was able to do a re-deploy (took awhile) but still same issue, I was able to connect to it via an external client but it takes forever to do any queries
15 days ago
curious which regions are timing out/524ing
15 days ago
us-east
15 days ago
(client regions, like where they are connecting from, oops)
15 days ago
redis clients are all in us-east too
15 days ago
let me see if I can restore a backup into another Redis service
15 days ago
was able to do that and yeah our outage went away
15 days ago
no idea on what happened to that Redis but it got doomed
15 days ago
ok only now I saw the email lol
Attachments
15 days ago
anyway we will implement a Redis Cluster, tbh should have done that a long time ago but busy with other stuff
https://discord.com/channels/713503345364697088/943649944072499210/1518716668140982292
15 days ago
Sorry for the interruptions!
Status changed to Solved passos • 15 days ago