2 months ago
We are starting to see an increase in 524 errors from Cloudflare, saying that the origin is taking too long. Has anything changed on Railway’s side?
We’re checking on our side as well to see if it’s an issue on our end. We just found it odd that the request isn’t being registered by our observability tools, so it might be failing before that.
I also turned off Cloudflare to see if the issue is related to them.
15 Replies
2 months ago
found out that one of my redis databases are unreachable
2 months ago
(no markdown available for this content)
2 months ago
and a re-deploy doesn't take place, seems similar to: https://discord.com/channels/713503345364697088/1518697501547298997
2 months ago
also seeing that the metrics tab is totally empty but might be related to the current incident.
Attachments
2 months ago
was able to do a re-deploy (took awhile) but still same issue, I was able to connect to it via an external client but it takes forever to do any queries
2 months ago
curious which regions are timing out/524ing
2 months ago
us-east
2 months ago
(client regions, like where they are connecting from, oops)
2 months ago
redis clients are all in us-east too
2 months ago
let me see if I can restore a backup into another Redis service
2 months ago
was able to do that and yeah our outage went away
2 months ago
no idea on what happened to that Redis but it got doomed
2 months ago
ok only now I saw the email lol
Attachments
2 months ago
anyway we will implement a Redis Cluster, tbh should have done that a long time ago but busy with other stuff
https://discord.com/channels/713503345364697088/943649944072499210/1518716668140982292
2 months ago
Sorry for the interruptions!
Status changed to Solved passos • about 2 months ago