railway postgres issues . need this up now or looking for new service
novummarkets
PROOP

3 months ago

My Postgres service (ID: 82f30c0a-91ad-4ade-ba50-569225b92850) in project e7039ff6 is broken. The Railway Postgres DB UI shows "We are unable to connect to the database via SSH" with error "Invalid Record Length: columns length is 1, got 2 on line 4". All connections from my app fail. This started May 23, 2026. I need the service repaired or restored from backup. My app depends on this database and is down. #✋|help

Closed

41 Replies

3 months ago

Could you provide the errors you get when trying to connect from your app?


novummarkets
PROOP

3 months ago

Tell them exactly this:

From the app (cozy-truth, Node.js 18 Alpine, pg@8.20.0):

With ssl: false: "Connection terminated unexpectedly"

With ssl: { rejectUnauthorized: false }: "There was an error establishing an SSL connection"

With NODE_TLS_REJECT_UNAUTHORIZED=0 set: same errors persist regardless of SSL config.

The database host resolves correctly (postgres-lwgx.railway.internal), credentials are confirmed correct via railway run environment inspection, but every connection attempt fails immediately. This started May 23, 2026.

Railway's own Postgres DB UI also shows: "We are unable to connect to the database via SSH — Failed to parse psql output: Error: Invalid Record Length: columns length is 1, got 2 on line 4"

The corruption appears to be at the PostgreSQL data file level, not an application issue.




novummarkets
PROOP

3 months ago

Screenshot_2026-05-02_170522.png

Attachments


novummarkets
PROOP

3 months ago

Screenshot_2026-05-27_022414.png

Attachments


novummarkets
PROOP

3 months ago

seems i need an engineer to fix this .


3 months ago

The error visible in the console here is of your app


3 months ago

I would think that your app is crashing and then unexpectedly closing the database connection as a result of it


novummarkets
PROOP

3 months ago

only the 2nd pic . i made mistake and removed it


3 months ago

And what logs do you see in the Postgres service?


novummarkets
PROOP

3 months ago

1 sec


novummarkets
PROOP

3 months ago

(no markdown available for this content)

Attachments


novummarkets
PROOP

3 months ago

(no markdown available for this content)

Attachments


3 months ago

Postgres service

1779874333_BUDdSwyyyG.png

Attachments


novummarkets
PROOP

3 months ago

(no markdown available for this content)

Attachments


3 months ago

Could you try suffixing ?sslmode=require&sslnegotiation=postgres to your database connection URL in your app?


3 months ago

Can do that from the variables tab


novummarkets
PROOP

3 months ago

i will try as soon claude is available .


3 months ago

Also consider downgrading pg, to something like 8.15.x


3 months ago

might also fix it


novummarkets
PROOP

3 months ago

ok


novummarkets
PROOP

3 months ago

on it now


novummarkets
PROOP

3 months ago

deploy still crashing


3 months ago

same error on the app and postgres service?


novummarkets
PROOP

3 months ago

we have a successful deployment but errors remain


novummarkets
PROOP

3 months ago

Still getting "There was an error establishing an SSL connection". The sslnegotiation=postgres URL suffix did not fix it. The Node.js pg library is a pure JavaScript implementation — it doesn't read the sslnegotiation URL parameter from libpq. The Postgres server logs show "received direct SSL connection request without ALPN protocol negotiation extension."

Can you either:

Tell us the correct DATABASE_PUBLIC_URL for our Postgres service so we can connect through the external proxy instead of the internal hostname? or

Confirm whether the internal Postgres (postgres-lwgx.railway.internal) accepts plain non-SSL connections, and if not, what TLS configuration is required for Node.js clients?


3 months ago

Can you show me which image and version the Postgres service is on?


3 months ago

Can see this in the service settings tab


novummarkets
PROOP

3 months ago


3 months ago

remove this query string as it's obvious it doesn't work


3 months ago

and consider disabling ssl ssl: false in pg as you are (should be) using railway's private networking


novummarkets
PROOP

3 months ago

ok


novummarkets
PROOP

3 months ago

will post results soon


3 months ago

And also do confirm that your app's env looks similar to this

1779875549_BE9tStTq4J.png

Attachments


novummarkets
PROOP

3 months ago

We set DATABASE_URL to ${{Postgres-lwGX.DATABASE_URL}} as shown in your screenshot. With ssl: false, we still get Connection terminated unexpectedly. The injected DATABASE_URL shows postgres-lwgx.railway.internal:5432 as the host but the password appears stale. Could you verify the Postgres service credentials on your side and confirm the DATABASE_URL variable reference is resolving to current/valid credentials? We suspect the Postgres password may have rotated and the reference isn't reflecting the new valu


novummarkets
PROOP

3 months ago

Done — ssl: false, query string removed. Still getting Connection terminated unexpectedly on every connection attempt. The DATABASE_URL resolves to postgres-lwgx.railway.internal:5432 with the credentials from ${{Postgres-lwGX.DATABASE_URL}}. Is it possible the Postgres password is out of sync? The Railway DB UI is also showing "unable to connect via SSH" with an invalid record length error — it looks like the Postgres service itself may have a credentials or data issue on your side.


novummarkets
PROOP

3 months ago

ENOTFOUND base is fixed. Now connecting to postgres-lwgx.railway.internal correctly with ssl: false. Still getting Connection terminated unexpectedly immediately. The Railway Postgres DB UI still shows "Unable to connect via SSH — Invalid Record Length: columns length is 1, got 2 on line 4". The Postgres service itself appears to have a config or data integrity issue on your side. Can you investigate the Postgres service health directly


novummarkets
PROOP

3 months ago

We've tried every SSL configuration. ssl: false → "Connection terminated unexpectedly". ssl: { rejectUnauthorized: false } → "There was an error establishing an SSL connection". ssl: { rejectUnauthorized: false, ciphers: 'DEFAULT@SECLEVEL=0' } → same SSL error. The Postgres server logs earlier showed "no shared cipher" and "bad key share" which indicates TLS cipher/version incompatibility at the infrastructure level. Can you fix the Postgres SSL configuration or confirm which port/TLS config is correct for Node.js 18 clients? Alternatively, can you provide the correct DATABASE_PUBLIC_URL (external proxy) for this service as a workaround?


novummarkets
PROOP

3 months ago

BREAKTHROUGH! Decoding the hex response:

485454502f312e31203430302042616420526571756573740d0a436f6e6e656374696f6e3a20636c6f73650d0a0d0a

= HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\n

Port 5432 at postgres-lwgx.railway.internal is NOT PostgreSQL — it's an HTTP proxy. Railway updated their internal networking to route through an HTTP layer. PostgreSQL protocol gets rejected as "bad request". This is why NOTHING has worked all session — we've never been talking to the actual database.

The external proxy (turntable.proxy.rlwy.net:23306) IS real PostgreSQL. We need to fix the TLS handshake on that path. Switch to external URL and let the diagnostic tell us the actual TLS error:


3 months ago

Private networking is TCP (incl HTTP) and UDP


3 months ago

Railway did not update their internal networking at all


novummarkets
PROOP

3 months ago

Found the root cause. Our cozy-truth Node.js app was accidentally deployed to the Postgres-lwGX service (deploy 206a318b). The Postgres container is running our Express server instead of PostgreSQL. The postgres-volume-0cYy volume should still have the data files intact. Can you restore the Postgres-lwGX service to a proper PostgreSQL image? The volume needs to stay attached."


Welcome!

Sign in to your Railway account to join the conversation.

Loading...