a month ago
Our verified account (isVerified=true, registrationStatus=REGISTERED) runs project 04383af2-6792-4a9b-9f55-422a43a6f7f7, service 60893d19-58af-4f09-a188-66e01f61db6d, environment b70e5c0e-b64a-44b6-8522-9865411af6cf in us-west2.
On 2026-09-09, deployment bad704c8-721e-45e6-84b7-9404b02fcea9 ran bounded startup connectivity probes:
- github.com:443 connected
- our VM's HTTPS listener on TCP 8445 connected
- github.com:22 timed out
- our VM's TCP 22 / SSH status command timed out, consistently across deployments
The same restricted SSH status call succeeds from our local machine. The VM listens on TCP22; its host firewall ruleset is empty and fail2ban is inactive. A bounded VM capture during a repeated cloud probe did not show a matching successful cloud connection. We do not assume this alone identifies the packet drop location.
The management application and Supabase connection are healthy. The SSH client is OpenSSH in node:24.20.0-bookworm-slim. No proxy or tunnel is involved in the failing call. Outbound IPv6 and static outbound IP are disabled.
Is TCP22 restricted for this workspace/service despite account verification, or is there a regional outbound networking issue? Please confirm the applicable policy and supported remedy. We have not upgraded plans because current public docs do not establish that a paid upgrade resolves SSH connectivity.
No keys, credentials, native session data or private access links are included.
5 Replies
a month ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • 27 days ago
a month ago
Additional client-side evidence from Railway network flow logs:
railway logs --network --direction egress --protocol tcp --port 22 --lines 8 --json
The records for both our VM:22 and GitHub:22 have dropCause="ICMP_CSUM". The same deployment's working VM:8445 flow has dropCause=null. Example flow IDs: bd0e9286-7a99-4ce5-9202-b4971364869d (VM:22), 784a48bf-76ac-4e64-bc07-8ee6268c5bae (GitHub:22), b3a00840-7672-4528-a38e-d8eaafc8f912 (working VM:8445). These are from deployment bad704c8-721e-45e6-84b7-9404b02fcea9 at approximately 14:42 UTC on September 9.
A later independent OpenSSH probe with IPv4 forced and ConnectTimeout=15, bypassing our application adapter and HTTP timeouts, confirmed TCP was never established; no SSH banner or authentication stage was reached. The child was not killed by its outer timeout. Its network flow records also show ICMP_CSUM. Live connection configuration matches the locally working configuration.
Can you explain what ICMP_CSUM means for these TCP egress records, and whether this identifies a host-side drop, a translated kernel reason, or a policy/configuration issue? We are avoiding interpreting the label as proof of a checksum bug without confirmation.
a month ago
One additional negative-control experiment is complete (deployment 8e146e67-3076-49ab-ab7f-40b5f4e55f02, about 15:23 UTC September 9). Runtime reports kernel 6.18.5+deb13-cloud-amd64, Node v24.20.0, x64. Independent IPv4 Node TCP probes observed:
- VM:8445 connected in 177ms.
- VM:22 timed out after 10s; TcpExt.TCPTimeouts increased by 6.
- GitHub:22 timed out after 10s; TcpExt.TCPTimeouts increased by 6.
- VM:40001 (confirmed no listener) returned ECONNREFUSED in 173ms. The desktop control returned the same error in 182ms. Railway flow 5a27d7cf-a55c-4551-af6d-f8b114f5e01a for port40001 has dropCause=null.
No measured container-network-namespace ICMP counter increased during these probes. This does not exclude host-side ICMP drops. The results show that a non-HTTPS high port can reach the VM and return a refusal, while both SSH destinations time out. We have not changed the VM listener, firewall, network policy, region or plan, and have removed the temporary startup probes.
a month ago
Two things worth adding: a gap in your control test, and a precedent that
suggests this isn't specific to you.
The gap first. Your port 40001 negative control proves the path carries
control packets — a closed port returned a proper refusal, so SYN out and
RST back both traversed fine. What it doesn't prove is that an open
non-22 port completes a handshake, because 40001 never had to. That leaves
"port 22 specifically is filtered" and "something about these particular
flows fails" still tangled together.
The test that separates them: run sshd on a second port on the same VM —
2222 is fine — and connect to that same host on 2222 from the same Railway
service. Same source, same destination host, same path, same MTU, only the
port number differs. If 2222 completes and 22 times out, it's port-number
filtering and nothing about your VM, your firewall, or the network path is
relevant. If 2222 also times out, the port number is a red herring and the
ICMP_CSUM drops are pointing at something in the path instead. That single
test collapses the whole search space and takes about five minutes.
The precedent: another user hit outbound TCP 22 failing from Railway in
"Issue with outbound TCP connections to NFOServers"
(https://station.railway.com/questions/issue-with-outbound-tcp-connections-to-n-74cbff4b)
— TCP 22 to their VPS failing while UDP worked and TCP to other providers
was fine. A moderator replied about static IPs but never addressed the
outbound filtering question, and the thread died there. Two independent
reports of outbound 22 specifically, with no public documentation either
way, is at least suggestive that egress filtering on 22 exists and simply
isn't documented. Blocking outbound SSH is a common anti-abuse measure on
platforms with cheap/free tiers, since it's the port credential-stuffing
bots hammer, and "verified trial" is exactly the account class where those
rules bite hardest.
To be clear about what I can and can't tell you: I can't confirm Railway's
egress policy — only they can, and that's the thing worth pressing them for
in this thread, phrased narrowly as "is outbound TCP 22 filtered, on which
plans, and is that documented anywhere." The 2222 test is what makes that a
question they can answer with a yes or no instead of a diagnostic thread.
Meanwhile, to unblock the GitHub half today: GitHub serves SSH over 443.
Host github.com
Hostname ssh.github.com
Port 443
User gitOr test directly: ssh -T -p 443 git@ssh.github.com
That's officially supported by GitHub and sidesteps 22 entirely. It won't
help for your own VM unless you move that sshd to 443 or another port, but
it gets your git traffic working regardless of how the port 22 question
resolves.
a month ago
What to do: run a differential test that separates "port 22 is filtered"
from "something about this path fails," since your current evidence
(the 40001 negative control) proves packets flow but doesn't distinguish
those two cases — 40001 was never open, so it never had to complete a
handshake.
Add a second sshd listener on your VM on 2222, then run this from the
Railway service that's currently timing out:
#!/usr/bin/env bash
# port-diff-test.sh <host> [comparison_port] [trials]
set -uo pipefail
HOST="${1:?Usage: port-diff-test.sh <host> [comparison_port] [trials]}"
CMP_PORT="${2:-2222}"
TRIALS="${3:-3}"
TIMEOUT_S=10
test_port() {
local host="$1" port="$2" i="$3" t0 t1 dt rc
t0=$(date +%s.%N)
if timeout "$TIMEOUT_S" bash -c "exec 3<>/dev/tcp/${host}/${port}" 2>/dev/null; then
rc=0; exec 3>&- 2>/dev/null || true
else
rc=1
fi
t1=$(date +%s.%N)
dt=$(awk -v a="$t0" -v b="$t1" 'BEGIN{printf "%.2f", b-a}')
[[ "$rc" -eq 0 ]] && printf 'trial %d port %-5s CONNECTED %ss\n' "$i" "$port" "$dt" \
|| printf 'trial %d port %-5s TIMED OUT %ss\n' "$i" "$port" "$dt"
return "$rc"
}
echo "Testing $HOST — port 22 vs port $CMP_PORT, $TRIALS trials each"
p22_fail=0; pcmp_fail=0
for i in $(seq 1 "$TRIALS"); do test_port "$HOST" 22 "$i" || p22_fail=$((p22_fail+1)); done
echo
for i in $(seq 1 "$TRIALS"); do test_port "$HOST" "$CMP_PORT" "$i" || pcmp_fail=$((pcmp_fail+1)); done
echo
echo "22 failed ${p22_fail}/${TRIALS}, ${CMP_PORT} failed ${pcmp_fail}/${TRIALS}"
if [[ "$p22_fail" -eq "$TRIALS" && "$pcmp_fail" -eq 0 ]]; then
echo "RESULT: port-specific filtering on 22. Path and destination are fine."
elif [[ "$p22_fail" -eq "$TRIALS" && "$pcmp_fail" -eq "$TRIALS" ]]; then
echo "RESULT: both fail — not about port 22. Look at path/MTU/ICMP_CSUM instead."
else
echo "RESULT: mixed — rerun with more trials before concluding."
fiIf :22 fails and :2222 connects clean, that's conclusive: port-number
filtering, nothing about your VM/firewall/path. If both fail, the port
theory is dead and the ICMP_CSUM drops you already logged deserve the
attention instead.
Also relevant: another user hit outbound TCP 22 failing from Railway
(UDP and other TCP fine) in "Issue with outbound TCP connections to
NFOServers" — https://station.railway.com/questions/issue-with-outbound-tcp-connections-to-n-74cbff4b
— a moderator replied about static IPs but never actually answered the
filtering question. Two independent reports of the same specific port
failing, with nothing documented either way, is worth pressing Railway on
directly — outbound SSH is a common thing to block on cheap/free/trial
tiers since it's what credential-stuffing bots hit, and "verified trial" is
exactly the account class those rules target.
To unblock GitHub specifically today regardless of how the platform
question resolves — GitHub serves SSH over 443:
Host github.com
Hostname ssh.github.com
Port 443
User gitTest with: ssh -T -p 443 git@ssh.github.com
8 days ago
I am seeing the same issue from a verified Railway Trial account.
Production service needs outbound SFTP to an external Cloudways server on TCP/22.
From inside the deployed Railway container:
Cloudways:80 → CONNECTED
Cloudways:443 → CONNECTED
Cloudways:22 → TIMEOUT
ssh.github.com:443 → CONNECTED, SSH banner received
github.com:22 → TIMEOUT
Railway network flow logs for repeated TCP/22 attempts consistently show:
dropCause=ICMP_CSUM
flowState=partial
peerKind=internet
l4LatencyMs=0
The Cloudways server's TCP/22 is reachable successfully from an external Windows machine, and the Railway static outbound IPs have been allowlisted there.
My Railway account is verified, and Railway documentation states that verified Full Trial accounts have full network access.
Can Railway confirm whether outbound TCP/22 is currently being filtered or affected by a networking issue for verified Trial services, and what ICMP_CSUM indicates in this situation?