a month ago
Hi Railway team,
We have a container in your Amsterdam region that is still running even though its service and now its entire parent project have been deleted. We have no way to stop it from our side.
Timeline:
- 1 July 2026, 21:02:52 UTC - the worker service was deleted from project kritmatta-enrichment during a migration.
Instead of terminating, the container appears to have restarted at that moment and has been running untracked ever
since (its process start time matches exactly).
- 10 July 2026, 18:31 UTC - we deleted the whole kritmatta-enrichment project (2e5dccf2-b098-4b53-92e4-f5855774845a,
workspace 460b7d63-3165-4ab6-b172-3dc3bafa3385). The project showed zero services and zero volumes at that point.
The container kept running straight through the deletion - we observed it polling our database every ~60 seconds
before, during, and after (verified until at least 18:40 UTC).
How to find it:
- Egress IP: 152.55.184.6 (your AS400940, Amsterdam)
- Traffic: continuous polling of our Supabase project yqqvecymsbiofyiripui (*.supabase.co REST API), roughly 1
request/minute
- Runtime: Node.js running as PID 1, image based on node:20-slim
- Container start: 2026-07-01 21:02:52 UTC
Please do NOT touch our healthy service: project enrichment-server (5653ed2d-ca8b-46e7-a8e4-1caada87c01e), egress 208.77.244.143 - that one must stay up.
Could you locate and terminate the orphaned container? We'd also like to understand how it escaped reaping, since it process s real jobs from our queue with outdated code and has been corrupting our enrichment data.
One note: the deleted project's permanent deletion is scheduled for 12 July 18:31 UTC. If your cleanup happens to
reap the container then, we'll confirm and close this ticket.
Thanks!
1 Replies
Status changed to Awaiting Railway Response Railway • about 1 month ago
a month ago
We found it and stopped it. The container was on one of our Amsterdam hosts, still labeled with your deleted project, and had been running since July 1 exactly as you described. It's fully terminated as of 02:32 UTC on July 11, so the polling against your Supabase project should have stopped then.
On the cause: when your service was deleted, the process ignored the shutdown signal, so it was force-killed after a grace period. The container's restart policy treated that as a crash and restarted it, and our cleanup didn't recover from that, so it kept running untracked. We've traced the exact bug and are flagging it to our platform team.
One thing on your side that would have avoided this entirely: your process runs as PID 1 in the container, and PID 1 ignores SIGTERM unless the app registers a handler. Adding a SIGTERM handler in Node (close your pollers, then process.exit(0)) means shutdowns are clean instead of force-killed, which sidesteps this bug and also gives you faster restarts on normal deploys.
Let us know if you need more help.
Railway Team
Status changed to Awaiting User Response Railway • about 1 month ago
Status changed to Solved manjitjohal • about 1 month ago