17 days ago
My Postgres service (ID: 82f30c0a-91ad-4ade-ba50-569225b92850) in project e7039ff6 is broken. The Railway Postgres DB UI shows "We are unable to connect to the database via SSH" with error "Invalid Record Length: columns length is 1, got 2 on line 4". All connections from my app fail. This started May 23, 2026. I need the service repaired or restored from backup. My app depends on this database and is down. #✋|help
41 Replies
17 days ago
Could you provide the errors you get when trying to connect from your app?
Tell them exactly this:
From the app (cozy-truth, Node.js 18 Alpine, pg@8.20.0):
With ssl: false: "Connection terminated unexpectedly"
With ssl: { rejectUnauthorized: false }: "There was an error establishing an SSL connection"
With NODE_TLS_REJECT_UNAUTHORIZED=0 set: same errors persist regardless of SSL config.
The database host resolves correctly (postgres-lwgx.railway.internal), credentials are confirmed correct via railway run environment inspection, but every connection attempt fails immediately. This started May 23, 2026.
Railway's own Postgres DB UI also shows: "We are unable to connect to the database via SSH — Failed to parse psql output: Error: Invalid Record Length: columns length is 1, got 2 on line 4"
The corruption appears to be at the PostgreSQL data file level, not an application issue.
Attachments
Attachments
17 days ago
The error visible in the console here is of your app
17 days ago
I would think that your app is crashing and then unexpectedly closing the database connection as a result of it
17 days ago
And what logs do you see in the Postgres service?
(no markdown available for this content)
Attachments
(no markdown available for this content)
Attachments
17 days ago
Postgres service
Attachments
(no markdown available for this content)
Attachments
17 days ago
Could you try suffixing ?sslmode=require&sslnegotiation=postgres to your database connection URL in your app?
17 days ago
Can do that from the variables tab
17 days ago
Also consider downgrading pg, to something like 8.15.x
17 days ago
might also fix it
17 days ago
same error on the app and postgres service?
Still getting "There was an error establishing an SSL connection". The sslnegotiation=postgres URL suffix did not fix it. The Node.js pg library is a pure JavaScript implementation — it doesn't read the sslnegotiation URL parameter from libpq. The Postgres server logs show "received direct SSL connection request without ALPN protocol negotiation extension."
Can you either:
Tell us the correct DATABASE_PUBLIC_URL for our Postgres service so we can connect through the external proxy instead of the internal hostname? or
Confirm whether the internal Postgres (postgres-lwgx.railway.internal) accepts plain non-SSL connections, and if not, what TLS configuration is required for Node.js clients?
17 days ago
Can you show me which image and version the Postgres service is on?
17 days ago
Can see this in the service settings tab
17 days ago
remove this query string as it's obvious it doesn't work
17 days ago
and consider disabling ssl ssl: false in pg as you are (should be) using railway's private networking
17 days ago
And also do confirm that your app's env looks similar to this
Attachments
We set DATABASE_URL to ${{Postgres-lwGX.DATABASE_URL}} as shown in your screenshot. With ssl: false, we still get Connection terminated unexpectedly. The injected DATABASE_URL shows postgres-lwgx.railway.internal:5432 as the host but the password appears stale. Could you verify the Postgres service credentials on your side and confirm the DATABASE_URL variable reference is resolving to current/valid credentials? We suspect the Postgres password may have rotated and the reference isn't reflecting the new valu
Done — ssl: false, query string removed. Still getting Connection terminated unexpectedly on every connection attempt. The DATABASE_URL resolves to postgres-lwgx.railway.internal:5432 with the credentials from ${{Postgres-lwGX.DATABASE_URL}}. Is it possible the Postgres password is out of sync? The Railway DB UI is also showing "unable to connect via SSH" with an invalid record length error — it looks like the Postgres service itself may have a credentials or data issue on your side.
ENOTFOUND base is fixed. Now connecting to postgres-lwgx.railway.internal correctly with ssl: false. Still getting Connection terminated unexpectedly immediately. The Railway Postgres DB UI still shows "Unable to connect via SSH — Invalid Record Length: columns length is 1, got 2 on line 4". The Postgres service itself appears to have a config or data integrity issue on your side. Can you investigate the Postgres service health directly
We've tried every SSL configuration. ssl: false → "Connection terminated unexpectedly". ssl: { rejectUnauthorized: false } → "There was an error establishing an SSL connection". ssl: { rejectUnauthorized: false, ciphers: 'DEFAULT@SECLEVEL=0' } → same SSL error. The Postgres server logs earlier showed "no shared cipher" and "bad key share" which indicates TLS cipher/version incompatibility at the infrastructure level. Can you fix the Postgres SSL configuration or confirm which port/TLS config is correct for Node.js 18 clients? Alternatively, can you provide the correct DATABASE_PUBLIC_URL (external proxy) for this service as a workaround?
BREAKTHROUGH! Decoding the hex response:
485454502f312e31203430302042616420526571756573740d0a436f6e6e656374696f6e3a20636c6f73650d0a0d0a
= HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\n
Port 5432 at postgres-lwgx.railway.internal is NOT PostgreSQL — it's an HTTP proxy. Railway updated their internal networking to route through an HTTP layer. PostgreSQL protocol gets rejected as "bad request". This is why NOTHING has worked all session — we've never been talking to the actual database.
The external proxy (turntable.proxy.rlwy.net:23306) IS real PostgreSQL. We need to fix the TLS handshake on that path. Switch to external URL and let the diagnostic tell us the actual TLS error:
17 days ago
Private networking is TCP (incl HTTP) and UDP
17 days ago
Railway did not update their internal networking at all
Found the root cause. Our cozy-truth Node.js app was accidentally deployed to the Postgres-lwGX service (deploy 206a318b). The Postgres container is running our Express server instead of PostgreSQL. The postgres-volume-0cYy volume should still have the data files intact. Can you restore the Postgres-lwGX service to a proper PostgreSQL image? The volume needs to stay attached."