3 hours ago
Production service stopped about 8 minutes into every deploy ("network configuration step took about 7.5 minutes"), while the same image runs fine on a sibling service
Setup: Hobby plan, Dockerfile builder, runtime V2, Python (Flask) web app. Happy to share project, service and deployment IDs privately with Railway staff.
What happens
Since Oct 31, 11:40 UTC, every deployment of one service (our production service) starts, serves traffic normally for about 8 minutes, and is then stopped by Railway: "SIGTERM received", "Stopping Container", deployment FAILED. With no earlier deployment to fall back on, the domain then returns 404 "Application not found". The dashboard's own diagnosis says the build and app startup were healthy and that "the network configuration step took about 7.5 minutes before the deployment was stopped."
The same commit and image run fine on a sibling service in the same project and environment (deployed by the same push) and on three other projects. The app does not crash; Railway stops it.
Timeline (UTC, Oct 31)
- 02:32 deploy: ran about 9 hours, fine (the last good one).
- 11:40 push (application code only): stopped at 11:50, about 8m41s after start.
- 12:13 manual redeploy: stopped after about 7m54s. Railway also emailed "Build failed" at 12:21, though the build had succeeded.
- 13:16 deploy (auto-created after we removed a stuck old deployment): about 7m58s.
- 14:56 deploy after adding PORT=8080: started 14:57:22, SIGTERM 15:05:15 (7m53s).
- 15:30 deploy after resetting the domains: started 15:30:34, SIGTERM 15:38:28 (7m54s).
Until the SIGTERM the logs show normal work: the app listening on 0.0.0.0:8080, background jobs completing, requests from Railway's edge (100.64.x.x) answered. The public domain answers with our sign-in page (HTTP 401, as expected) for the whole window.
What we tried (each followed by a fresh deploy; every one was stopped the same way)
- Redeployed from the dashboard.
- Removed a stuck old deployment.
- Added the service variable PORT=8080 (the app already listened on 8080 by default).
- Deleted the service's second Railway domain (explicit target port 8080) and replaced the main domain (also target port 8080) with a fresh Railway domain with no target port, renamed back to the same subdomain. It now matches the healthy sibling's single domain.
What we ruled out
- Code or image: the same commit runs fine on the sibling service in the same environment and on three other projects.
- Build timeout: redeploy builds finish in under a minute; the stop comes about 8 minutes after the app is up and serving.
- Port or binding: the app listens on 0.0.0.0:8080 (Dockerfile EXPOSE 8080; PORT now also set), and public traffic is served throughout the window.
- Health check: no healthcheck path is configured.
- Crash or restart policy: no crash. The process runs normally until Railway's SIGTERM, and the restart policy never fires.
- Volume: Ready, 2.5 GB of 5 GB used, mounted at /data, read fine by the app.
- Network flows: the private network flow logs look normal (DNS, outbound 443, inbound 8080).
Possible clues (things unique to this service)
- It is our oldest service (created July 2026) and exists in three environments: production (failing), staging (one deployment running since July 19) and demo (running since Oct 14).
- Production's volume is also attached to this service's instance in staging. A July "railway environment new --duplicate" left the same volume ID in both environments. The healthy sibling has its own volume and lives in one environment.
- Until today the service had two Railway domains, both with an explicit target port of 8080 (now replaced, as above).
- An unattached Postgres volume sits in the production environment.
- The dashboard has shown the October 21 incident banner all day ("A fix is out and we're monitoring it").
Please don't suggest deleting the volume or the staging environment. Because the volume ID is shared, deleting it from one environment once deleted it for all of them (July 2026).
Questions
- What is the network configuration step waiting for on this service, and why does it time out at about 7.5 minutes only here?
- Could the volume being referenced by two environments, or the service's age or region, block that step?
- Can Railway reset this service's networking state from your side? If not, what is the safe way to move this volume to a fresh service, given that staging references it too?
10 Replies
Status changed to Awaiting Railway Response Railway • about 3 hours ago
2 hours ago
Same here since 14:40 UTC on Oct 31 (Pro plan). One service in our staging environment fails every deploy the same way: the container runs normally, the network configuration step takes about 7.5 minutes, then the container is stopped with no reason and the deploy is marked FAILED. Five deploys of the same commit failed. The same service and image deploy fine in our production environment, and every other service in staging deploys fine. A new private network name didn’t help.
an hour ago
Third report, same day, same pattern. Production service in a Pro project,
no volume involved.
Three consecutive deployments of the same image, all stopped at roughly
the same age:
83aefae3 started 13:11:59Z stopped 13:20:45Z (8m46s)
7acb6640 started 13:42:39Z stopped 13:50:46Z (8m07s)
81b76111 started 13:53:33Z stopped 14:01:39Z (8m06s)
Each one built, started cleanly, passed its own startup checks and served
real traffic until the moment it stopped. The container log ends
mid-stream with only "Stopping Container" — no application error, no OOM,
and no graceful shutdown sequence from the app, so the process was killed
rather than exiting.
In the deployment events the stuck step is CONFIGURE_NETWORK: 7m37s to
8m19s on the three failures, against 2-3 seconds on healthy deployments of
this same service and of every other service in the project. After it
gives up, the deployment is marked FAILED.
Two further symptoms that may be the same root cause:
-
The previous instance is never drained. DRAIN_INSTANCES does not run, so
the service ends up with two active deployments while configured for one
replica, and requests land on either — ours answered 404 on the new
routes roughly half the time.
-
railway sshto this one service is refused ("An error occurredconnecting to your service") while it works for every other service in
the same project.
Setup, for completeness: no healthcheck configured, 1 replica, sleep
disabled, restart ON_FAILURE, GitHub auto-deploy off. The identical image
deploys and runs without issue in our other environment.
Workaround that restored service for us, in case it helps anyone else
here: redeploying a PREVIOUS deployment (not a new build) comes up
immediately and stays up — that path does not run CONFIGURE_NETWORK. It
does mean you are stuck on the old image until this is fixed, but it ends
the outage.
an hour ago
Same here i posted the problem 4 hours ago still no reply, we have to deploy the service every 7 minute to keep the production alive. Unbelievable
delavegaopengl
Same here i posted the problem 4 hours ago still no reply, we have to deploy the service every 7 minute to keep the production alive. Unbelievable
an hour ago
You don't have to redeploy every 7 minutes. Redeploying a PREVIOUS deployment — not a new build — comes up and stays up: that path doesn't run CONFIGURE_NETWORK, which is the step that hangs. You're stuck on the old image until Railway fixes this, but the outage stops.
Status changed to Awaiting User Response brody • about 1 hour ago
Status changed to Awaiting Railway Response brody • about 1 hour ago
an hour ago
The same thing happen to me today , every deploy is marked failed after 7 to 8 mins .
The same thing happen to me today , every deploy is marked failed after 7 to 8 mins .
an hour ago
The same previous advice helps - just keep the previous version deployed.
f22-pro
The same previous advice helps - just keep the previous version deployed.
an hour ago
yes the previous one is still up, but the new updates that I included fall off after 8 mins,
an hour ago
Our deploying resumed now
delavegaopengl
Our deploying resumed now
42 minutes ago
Confirmed, our's also resumed now,
23 minutes ago
We're looking into the deployment failures reported in this thread. Thank you for the updates, including the reports that deployments have resumed for some of you. If you're still affected, could you try one more deployment and let us know whether it completes successfully? We're continuing to investigate the failures and will follow up here when we have more information to share. I'm sorry for the disruption.
Status changed to Awaiting User Response Railway • 23 minutes ago
Status changed to Solved owencosans • 8 minutes ago