Container creation failing after US West networking outage was marked resolved
brettcale312
HOBBYOP

a month ago

Region: US West

Plan: Hobby

Stack: Python web service + Postgres + persistent volume mounted at /data

Failed step: Deploy → Create container

Builds complete successfully (image pushes fine), but container creation hangs/fails consistently.

Timeline:

Earlier today: same commit deployed successfully

During the US West private-networking outage: deployments failed at container creation. Diagnose returned "Infrastructure Error — transient platform issue."

After the outage was marked resolved on the status page: tried a fresh redeploy (empty commit), still failing at the same step.

Is there a residual issue affecting individual services, or do I need to do something on my end to unstick the deployment queue? Happy to DM project/service details.

Solved

6 Replies

Railway
BOT

a month ago

Your label-agent service's latest deployment failed at container creation with no application logs produced, confirming the container never started. Both volumes (including the one at /data) are healthy and in READY state, so this is not a data or volume issue. We're aware of transient container creation issues and are actively working on improvements. Cancel any queued deployments and retry shortly; if it continues to fail at the same step, the fix will come from our side.


Status changed to Awaiting User Response Railway about 1 month ago


Railway

Your label-agent service's latest deployment failed at container creation with no application logs produced, confirming the container never started. Both volumes (including the one at /data) are healthy and in READY state, so this is not a data or volume issue. We're aware of transient container creation issues and are actively working on improvements. Cancel any queued deployments and retry shortly; if it continues to fail at the same step, the fix will come from our side.

brettcale312
HOBBYOP

a month ago

It now says this is a persistent platform issue in the US West region. Here is the full measage. "This deployment failed at the container creation stage due to a persistent platform issue in the US West region. The build completed successfully every time, but 4 consecutive deployments across different commits all failed at the same CREATE_CONTAINER step. Reaching out to Railway support is the best next step."

ErrorDeploying.png

Attachments


Status changed to Awaiting Railway Response Railway about 1 month ago


Railway
BOT

a month ago

We're aware of these transient container creation issues and are actively working on improvements. Cancel any queued deployments and avoid re-queuing for now, since additional deploys can compound the queue. The fix will come from our side.


Status changed to Awaiting User Response Railway about 1 month ago


jamesfarmer276-cmd
PRO

a month ago

Also affected on a Pro project in us-west2: Postgres service with READY attached volume, two consecutive CREATE_CONTAINER failures after Railway diagnosis reported Infrastructure Error. Before redeploy, both private postgres.railway.internal and public TCP proxy connections were failing. I can provide project/service/deployment/volume IDs privately if Railway support wants them. Avoiding more redeploys per the guidance above.


Status changed to Awaiting Railway Response Railway about 1 month ago


jamesfarmer276-cmd

Also affected on a Pro project in us-west2: Postgres service with READY attached volume, two consecutive CREATE_CONTAINER failures after Railway diagnosis reported Infrastructure Error. Before redeploy, both private postgres.railway.internal and public TCP proxy connections were failing. I can provide project/service/deployment/volume IDs privately if Railway support wants them. Avoiding more redeploys per the guidance above.

brettcale312
HOBBYOP

a month ago

Same issue, same region. Here's what fixed it for me after hours of failed deploys:

The root cause turned out to be that Railway's internal networking (postgres.railway.internal) was broken by the outage — but the error message ("password authentication failed") was misleading. The password was fine; the private network just couldn't route to Postgres.

Fix: Switch DATABASE_URL in your app service's Variables from the internal URL to the public/external URL.

  1. Go to your Postgres service → Variables tab
  2. Copy the value of DATABASE_PUBLIC_URL (looks like postgresql://postgres:...@<something>.proxy.rlwy.net:<port>/railway)
  3. Go to your app service → Variables → edit DATABASE_URL → paste the public URL
  4. Save — service restarts automatically

The public URL routes through Railway's TCP proxy instead of the broken private network. Your app will connect fine and your data is untouched.

Note: this will incur small egress fees (Railway warns about this with a yellow icon) but it's negligible — fractions of a cent per connection. You can switch back to the internal URL once Railway confirms private networking is fully restored in US West.


a month ago

Thanks for posting the fix, brettcale — your diagnosis is exactly right, and it's the recommended workaround for anyone hitting this. To confirm for everyone landing here:

Today's US West networking disruption left a residual issue where some services can't route over private networking (postgres.railway.internal), which surfaces as misleading "connection timeout" or "password authentication failed" errors even though the database is healthy and the password is fine.

Workaround: switch your app's DATABASE_URL from the internal hostname to the value of DATABASE_PUBLIC_URL (Postgres service → Variables), which routes through the TCP proxy. Small egress cost applies; switch back once private networking is confirmed restored.

A separate symptom — deploys failing at the Create container step because the service's volume is pinned to an affected host — needs platform-side recovery and can't be worked around from your end; data volumes stay intact while that's resolved. We're tracking both. Glad you're back up.


Status changed to Awaiting User Response Railway about 1 month ago


Railway
BOT

25 days ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway 25 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...