What region is the service deployed in?

And also, can you share your logs?

yes I am west cost usa Its all red wall of bad logs nothing is coming back. II am freaking out right now. to many characters to post logs here

https://status.railway.com/incident/RL8FRJE6

this is a railway issue not me? ive been puking in stress lol

We're sorry for the trouble. We're aware of an issue affecting connectivity and deployments in our US West region, which is likely causing the errors you're seeing across your environments. We're actively working on it. You can follow updates here: https://status.railway.com/incident/RL8FRJE6

are you sure it happend right when I pushed a upate out so it was me

my testing center is still up same server side

Yup! that makes sense. Only a small percentage of traffic is affected; not every service will be impacted here.

We will be pushing updates up to the status page as linked previously by Chandrika.

i am restoring my database do i stop it what do I do? i have a ton of cients nw im freaking out . so you are 100% sure its on you? how ong?

The timing was a coincidence. This is a partial outage in US West, so some services can stay up while others are affected. Your testing center being fine doesn't mean your push caused the other services to go down. Once the incident is resolved, your affected services should come back up normally. You can track progress here: https://status.railway.com/incident/RL8FRJE6

ok im trusting you im in full blown panic attack not going to lie worst stres to imagine to much going on in my business was shocked ot see this

if i move up to 2k/m will this get me direct contact with you guys and stop this from happening

Stop the database restore. Making changes during an active incident can complicate recovery. Your data should be safer on the volume, and your services should come back on their own once the incident resolves. This is confirmed on our side, not your push. You can track progress here: https://status.railway.com/incident/RL8FRJE6

(On the support question, Business Class is available for workspaces with $5,000/month or more in spend and includes dedicated support channels with SLOs and Business Class customers get prioritized response times during them)

how do I stop database restore? I only see a button to detel volume will that delete my main?

I never mounted it to service im like realy in a panic lol this is the worst time to happen

Do NOT delete the volume. If you clicked "Restore" on a backup, that stages the change but does not apply it until you click "Deploy" - so just don't click Deploy, and you can discard the staged changes from the deploy bar at the top of your project canvas. If you already deployed the restore, your original volume is still retained (just detached), so nothing is lost either way. The safest thing right now is to leave everything as-is and wait for the incident to resolve. Your data is safe on the volume.

its just been stuckiing backing up for 30m to an hour since i di dit right before my thread trying to trouble shoot in panic. I wont deploy it if it appears.

Piling on here: all my environments in US-West (production environments) are experience 100% communications failure; redis instance, postgres instance all unreachable both to my internal services and via client connection tools. Please consider this a full outage (not something intermittent). I'm fully down.

Hey, aware, this is the same US West connectivity incident. Redis, Postgres, and inter-service communication are all affected. Services will recover once the incident is resolved, no action needed on your end.

You can both track progress here: https://status.railway.com/incident/RL8FRJE6

how long do you think it wil take

I heard that they're starting to see recovery and that it likely won't take hours. Note that this isn't a concrete ETA

should I try to restart my db?

Worth a try

it worked finally what a stressfull event thank god it was on sunday night

Glad it's working for you now!

Team is reporting that they're starting to see recovery

Glad to hear it's back up. Apologies for the stress, and thanks for your patience while we worked through it. You can see the full incident timeline here: https://status.railway.com/incident/RL8FRJE6, we're still monitoring

I had to restart my MySQL instance to get it working again.

All seems fine now.

Glad to hear it. Apologies for the disruption, the US West connectivity incident caused the database connectivity failures you were seeing. We're still monitoring for full recovery

Incident should be resolved now 🙏