Outbound TCP 22 times out from verified trial service in us-west2
zjshen14
FREEOP

a month ago

Our verified account (isVerified=true, registrationStatus=REGISTERED) runs project 04383af2-6792-4a9b-9f55-422a43a6f7f7, service 60893d19-58af-4f09-a188-66e01f61db6d, environment b70e5c0e-b64a-44b6-8522-9865411af6cf in us-west2.

On 2026-09-09, deployment bad704c8-721e-45e6-84b7-9404b02fcea9 ran bounded startup connectivity probes:

  • github.com:443 connected
  • our VM's HTTPS listener on TCP 8445 connected
  • github.com:22 timed out
  • our VM's TCP 22 / SSH status command timed out, consistently across deployments

The same restricted SSH status call succeeds from our local machine. The VM listens on TCP22; its host firewall ruleset is empty and fail2ban is inactive. A bounded VM capture during a repeated cloud probe did not show a matching successful cloud connection. We do not assume this alone identifies the packet drop location.

The management application and Supabase connection are healthy. The SSH client is OpenSSH in node:24.20.0-bookworm-slim. No proxy or tunnel is involved in the failing call. Outbound IPv6 and static outbound IP are disabled.

Is TCP22 restricted for this workspace/service despite account verification, or is there a regional outbound networking issue? Please confirm the applicable policy and supported remedy. We have not upgraded plans because current public docs do not establish that a paid upgrade resolves SSH connectivity.

No keys, credentials, native session data or private access links are included.

$10 Bounty

5 Replies

Railway
BOT

a month ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway • 27 days ago


zjshen14
FREEOP

a month ago

Additional client-side evidence from Railway network flow logs:

railway logs --network --direction egress --protocol tcp --port 22 --lines 8 --json

The records for both our VM:22 and GitHub:22 have dropCause="ICMP_CSUM". The same deployment's working VM:8445 flow has dropCause=null. Example flow IDs: bd0e9286-7a99-4ce5-9202-b4971364869d (VM:22), 784a48bf-76ac-4e64-bc07-8ee6268c5bae (GitHub:22), b3a00840-7672-4528-a38e-d8eaafc8f912 (working VM:8445). These are from deployment bad704c8-721e-45e6-84b7-9404b02fcea9 at approximately 14:42 UTC on September 9.

A later independent OpenSSH probe with IPv4 forced and ConnectTimeout=15, bypassing our application adapter and HTTP timeouts, confirmed TCP was never established; no SSH banner or authentication stage was reached. The child was not killed by its outer timeout. Its network flow records also show ICMP_CSUM. Live connection configuration matches the locally working configuration.

Can you explain what ICMP_CSUM means for these TCP egress records, and whether this identifies a host-side drop, a translated kernel reason, or a policy/configuration issue? We are avoiding interpreting the label as proof of a checksum bug without confirmation.


zjshen14
FREEOP

a month ago

One additional negative-control experiment is complete (deployment 8e146e67-3076-49ab-ab7f-40b5f4e55f02, about 15:23 UTC September 9). Runtime reports kernel 6.18.5+deb13-cloud-amd64, Node v24.20.0, x64. Independent IPv4 Node TCP probes observed:

  • VM:8445 connected in 177ms.
  • VM:22 timed out after 10s; TcpExt.TCPTimeouts increased by 6.
  • GitHub:22 timed out after 10s; TcpExt.TCPTimeouts increased by 6.
  • VM:40001 (confirmed no listener) returned ECONNREFUSED in 173ms. The desktop control returned the same error in 182ms. Railway flow 5a27d7cf-a55c-4551-af6d-f8b114f5e01a for port40001 has dropCause=null.

No measured container-network-namespace ICMP counter increased during these probes. This does not exclude host-side ICMP drops. The results show that a non-HTTPS high port can reach the VM and return a refusal, while both SSH destinations time out. We have not changed the VM listener, firewall, network policy, region or plan, and have removed the temporary startup probes.


Two things worth adding: a gap in your control test, and a precedent that

suggests this isn't specific to you.

The gap first. Your port 40001 negative control proves the path carries

control packets — a closed port returned a proper refusal, so SYN out and

RST back both traversed fine. What it doesn't prove is that an open

non-22 port completes a handshake, because 40001 never had to. That leaves

"port 22 specifically is filtered" and "something about these particular

flows fails" still tangled together.

The test that separates them: run sshd on a second port on the same VM —

2222 is fine — and connect to that same host on 2222 from the same Railway

service. Same source, same destination host, same path, same MTU, only the

port number differs. If 2222 completes and 22 times out, it's port-number

filtering and nothing about your VM, your firewall, or the network path is

relevant. If 2222 also times out, the port number is a red herring and the

ICMP_CSUM drops are pointing at something in the path instead. That single

test collapses the whole search space and takes about five minutes.

The precedent: another user hit outbound TCP 22 failing from Railway in

"Issue with outbound TCP connections to NFOServers"

(https://station.railway.com/questions/issue-with-outbound-tcp-connections-to-n-74cbff4b)

— TCP 22 to their VPS failing while UDP worked and TCP to other providers

was fine. A moderator replied about static IPs but never addressed the

outbound filtering question, and the thread died there. Two independent

reports of outbound 22 specifically, with no public documentation either

way, is at least suggestive that egress filtering on 22 exists and simply

isn't documented. Blocking outbound SSH is a common anti-abuse measure on

platforms with cheap/free tiers, since it's the port credential-stuffing

bots hammer, and "verified trial" is exactly the account class where those

rules bite hardest.

To be clear about what I can and can't tell you: I can't confirm Railway's

egress policy — only they can, and that's the thing worth pressing them for

in this thread, phrased narrowly as "is outbound TCP 22 filtered, on which

plans, and is that documented anywhere." The 2222 test is what makes that a

question they can answer with a yes or no instead of a diagnostic thread.

Meanwhile, to unblock the GitHub half today: GitHub serves SSH over 443.

Host github.com

  Hostname ssh.github.com

  Port 443

  User git

Or test directly: ssh -T -p 443 git@ssh.github.com

That's officially supported by GitHub and sidesteps 22 entirely. It won't

help for your own VM unless you move that sshd to 443 or another port, but

it gets your git traffic working regardless of how the port 22 question

resolves.


What to do: run a differential test that separates "port 22 is filtered"

from "something about this path fails," since your current evidence

(the 40001 negative control) proves packets flow but doesn't distinguish

those two cases — 40001 was never open, so it never had to complete a

handshake.

Add a second sshd listener on your VM on 2222, then run this from the

Railway service that's currently timing out:

#!/usr/bin/env bash

# port-diff-test.sh <host> [comparison_port] [trials]

set -uo pipefail

HOST="${1:?Usage: port-diff-test.sh <host> [comparison_port] [trials]}"

CMP_PORT="${2:-2222}"

TRIALS="${3:-3}"

TIMEOUT_S=10

test_port() {

  local host="$1" port="$2" i="$3" t0 t1 dt rc

  t0=$(date +%s.%N)

  if timeout "$TIMEOUT_S" bash -c "exec 3<>/dev/tcp/${host}/${port}" 2>/dev/null; then

    rc=0; exec 3>&- 2>/dev/null || true

  else

    rc=1

  fi

  t1=$(date +%s.%N)

  dt=$(awk -v a="$t0" -v b="$t1" 'BEGIN{printf "%.2f", b-a}')

  [[ "$rc" -eq 0 ]] && printf 'trial %d  port %-5s  CONNECTED   %ss\n' "$i" "$port" "$dt" \

                     || printf 'trial %d  port %-5s  TIMED OUT   %ss\n' "$i" "$port" "$dt"

  return "$rc"

}

echo "Testing $HOST — port 22 vs port $CMP_PORT, $TRIALS trials each"

p22_fail=0; pcmp_fail=0

for i in $(seq 1 "$TRIALS"); do test_port "$HOST" 22 "$i" || p22_fail=$((p22_fail+1)); done

echo

for i in $(seq 1 "$TRIALS"); do test_port "$HOST" "$CMP_PORT" "$i" || pcmp_fail=$((pcmp_fail+1)); done

echo

echo "22 failed ${p22_fail}/${TRIALS}, ${CMP_PORT} failed ${pcmp_fail}/${TRIALS}"

if   [[ "$p22_fail" -eq "$TRIALS" && "$pcmp_fail" -eq 0 ]]; then

  echo "RESULT: port-specific filtering on 22. Path and destination are fine."

elif [[ "$p22_fail" -eq "$TRIALS" && "$pcmp_fail" -eq "$TRIALS" ]]; then

  echo "RESULT: both fail — not about port 22. Look at path/MTU/ICMP_CSUM instead."

else

  echo "RESULT: mixed — rerun with more trials before concluding."

fi

If :22 fails and :2222 connects clean, that's conclusive: port-number

filtering, nothing about your VM/firewall/path. If both fail, the port

theory is dead and the ICMP_CSUM drops you already logged deserve the

attention instead.

Also relevant: another user hit outbound TCP 22 failing from Railway

(UDP and other TCP fine) in "Issue with outbound TCP connections to

NFOServers" — https://station.railway.com/questions/issue-with-outbound-tcp-connections-to-n-74cbff4b

— a moderator replied about static IPs but never actually answered the

filtering question. Two independent reports of the same specific port

failing, with nothing documented either way, is worth pressing Railway on

directly — outbound SSH is a common thing to block on cheap/free/trial

tiers since it's what credential-stuffing bots hit, and "verified trial" is

exactly the account class those rules target.

To unblock GitHub specifically today regardless of how the platform

question resolves — GitHub serves SSH over 443:

Host github.com

  Hostname ssh.github.com

  Port 443

  User git

Test with: ssh -T -p 443 git@ssh.github.com


mad-chill-user-9963
PRO

8 days ago

I am seeing the same issue from a verified Railway Trial account.

Production service needs outbound SFTP to an external Cloudways server on TCP/22.

From inside the deployed Railway container:

Cloudways:80 → CONNECTED

Cloudways:443 → CONNECTED

Cloudways:22 → TIMEOUT

ssh.github.com:443 → CONNECTED, SSH banner received

github.com:22 → TIMEOUT

Railway network flow logs for repeated TCP/22 attempts consistently show:

dropCause=ICMP_CSUM

flowState=partial

peerKind=internet

l4LatencyMs=0

The Cloudways server's TCP/22 is reachable successfully from an external Windows machine, and the Railway static outbound IPs have been allowlisted there.

My Railway account is verified, and Railway documentation states that verified Full Trial accounts have full network access.

Can Railway confirm whether outbound TCP/22 is currently being filtered or affected by a networking issue for verified Trial services, and what ICMP_CSUM indicates in this situation?


Welcome!

Sign in to your Railway account to join the conversation.

Loading...