2 days ago
Since the October security incident I can't get a shell into my service, either through the CLI or through the shell in the dashboard. The service itself is online and serving traffic normally.
What I see from the CLI:
$ railway ssh --service {myservice} -- echo ok
Using SSH key from file ~/.ssh/id_{MINE}.pub
An error occurred connecting to your service. Please try again.
Connection to ssh.railway.com closed by remote host.
The same thing happens when I pass the private key explicitly (-i ~/.ssh/id_{MINE}). The dashboard console fails to connect as well.
What I've already ruled out:
- The service is Online, the latest deployment shows SUCCESS, and it serves requests normally.
railway statusshows the correct project, the production environment and the linked service.railway ssh keys listshows this key registered to my account.- The image is node:26-slim, which includes /bin/sh.
- I logged in again with
railway loginafter the forced sign-out. - Retrying a few times gives the same result.
Environment:
- Railway CLI 5.27.1 on macOS
- Region: Southeast Asia, with a volume mounted at /data
- The project is in a teammate's workspace; I'm a member, not the owner
Questions:
- Is SSH / shell access one of the "ancillary features halted for security" after the October 20–22 incident that hasn't been restored yet?
- If it has been restored, is there anything on the workspace or service side that needs to be re-enabled, e.g. a key that has to be registered at the workspace level rather than on my personal account?
8 Replies
Status changed to Awaiting Railway Response Railway • 1 day ago
a day ago
Seeing the same thing - can’t open a service shell from either the web dashboard console or railway ssh, identical error: “An error occurred connecting to your service. Please try again. The service is healthy and serving traffic normally. Region: US West (California). Notably, my other projects on the same account can open their console fine.
Status changed to Awaiting Railway Response Railway • 1 day ago
Status changed to Awaiting User Response Railway • 1 day ago
a day ago
seeing this on 3 of my services, but not on others. 3 failing ones have static egress IPs, but other than that, same image as others that are working.
Status changed to Awaiting Railway Response Railway • 1 day ago
Status changed to Awaiting User Response Railway • 1 day ago
citepulse-barret
seeing this on 3 of my services, but not on others. 3 failing ones have static egress IPs, but other than that, same image as others that are working.
a day ago
Seems like turning off static IPs allows it to work:
▎ Confirmed it's the static egress: turning off static outbound IPs on one affected service (Celery Worker General) and redeploying restored both railway ssh and the console; it had failed before the toggle and after a plain restart. ssh -vvv shows the gateway authenticates and opens the session channel, then fails after about 8.5s, so the break is the gateway-to-container hop. SSH worked on these same static-IP services on Oct 18. @r09722akabennett @mattslipari, do your affected services have static outbound IPs enabled (Settings → Networking)?
Status changed to Awaiting Railway Response Railway • 1 day ago
Status changed to Awaiting User Response Railway • 1 day ago
a day ago
Yes — our affected service has static outbound IPs enabled. We can't disable them as workaround because some service providers whitelist those egress Ip's.
Status changed to Awaiting Railway Response Railway • 1 day ago
a day ago
The reports in this thread line up on one pattern: the services that can't open a shell, from both railway ssh and the dashboard console, have Static Outbound IPs enabled, while services without static egress on the same accounts connect normally. Turning static egress off and redeploying restored shell access in one case. Since the session opens and then drops before reaching the container, this points to the static egress setup rather than SSH keys, CLI login or workspace membership.
There's currently no setting that keeps Static Outbound IPs on and restores shell access separately. If you rely on those addresses for provider allowlists, keep that in mind before using the toggle as a workaround: turning Static Outbound IPs off clears the assigned addresses, and turning them back on doesn't guarantee you'll get the same ones.
Status changed to Awaiting User Response Railway • 1 day ago
12 hours ago
Same here. Pro plan, EU West.
Our web service has Static Outbound IPs enabled (HA, three addresses), and both the dashboard console and railway ssh fail with "An error occurred connecting to your service. Please try again." In the same project, our scheduler service (built from the same repo) and Postgres have no static egress, and their consoles connect fine.
The web console last worked for us on 24 Oct and has failed since 25 Oct, through several deploys. The service itself is healthy and serving traffic normally.
We'd rather not turn static IPs off as a workaround, since the addresses can't be recovered afterwards. Is there an ETA for a fix?
Status changed to Awaiting Railway Response Railway • about 12 hours ago
Status changed to Awaiting User Response Railway • about 12 hours ago
2 hours ago
Thanks for the earlier reply. To keep this thread moving: a Railway engineer (dizzydes90) has since confirmed in another thread that this is a bug on Railway's side, not a configuration issue. The container's tunnel back to the SSH relay fails when Static Outbound IPs are enabled, and a fix is in progress:
https://station.railway.com/questions/cannot-connect-to-the-console-of-my-serv-60d55d58
Several of us here are in the same position (mattslipari, hqblind-prog and me). We can't turn off Static Outbound IPs because payment providers whitelist those exact addresses, and turning them off would release them for good.
Could someone from the Railway team please confirm on this thread:
- That this is the same tracked bug, and that you'll post an update here when the fix ships.
- A rough ETA, even a range.
- Whether there's any interim way to get a shell or run a one-off command on an affected service without disabling Static Outbound IPs, for example a support-side session or a temporary exemption.
We're running production without shell access in the meantime, so even a partial answer would help us plan. Thanks.
Status changed to Awaiting Railway Response Railway • about 2 hours ago
2 hours ago
This is the same issue described in the thread you linked: SSH and the dashboard console aren't currently compatible with Static Outbound IPs, so a service with static egress enabled fails to open a shell with exactly the error you're seeing, while services without it on the same account connect normally. We don't have a date to share for when that combination will work.
There's no support-side session or per-service exemption we can offer in the meantime. One option you can try without touching Static Outbound IPs is a jumpbox: deploy a small separate service without Static Outbound IPs, SSH into that, and then SSH from it into the affected service. We can't guarantee the affected service will be reachable that way, so treat it as something to test rather than a confirmed path.
Status changed to Awaiting User Response Railway • about 2 hours ago