PostgreSQL logical replication (WAL streaming) connections drop with "unexpected EOF" every few minutes
mirantes
FREEOP

7 months ago

I'm running a CDC (Change Data Capture) service that uses PostgreSQL logical replication to sync data between microservices in real-time.

Problem:

The START_REPLICATION streaming protocol connections are unstable. They either fail to establish (hang indefinitely) or drop every 2-10 minutes with "unexpected EOF on standby connection".

This happens on both internal networking (*.railway.internal:5432) and the external proxy (*.proxy.rlwy.net).

Regular SQL queries to the same databases work perfectly - only the streaming replication protocol is affected.

Error messages from PostgreSQL logs:

could not receive data from client: Connection reset by peer

unexpected EOF on standby connection

These appear after the connection has been streaming for a few minutes. PostgreSQL successfully accepts the connection and starts logical decoding, but the connection drops shortly after.

Error from my application:

Failed to subscribe to slot 'lightstream_user_db_slot': Subscription timeout after 90 seconds.

$40 Bounty

5 Replies

mirantes
FREEOP

7 months ago

The subscribe sometimes takes multiple retry attempts before succeeding.

Setup:

  • Multiple PostgreSQL databases on Railway (all with wal_level = logical)
  • Bun/TypeScript service using pg-logical-replication npm package
  • Replication slots and publications are set up correctly
  • Service runs on Railway connecting via internal networking

mirantes
FREEOP

7 months ago

What works:

  • Regular SQL queries (SELECT, INSERT, UPDATE) - no issues
  • Creating replication slots via SQL - works fine
  • Logical decoding starts successfully on PostgreSQL side

mirantes
FREEOP

7 months ago

Questions:

  1. Does Railway's networking support PostgreSQL's streaming replication protocol for long-lived connections?
  2. Are there TCP keepalive or idle timeout settings that could be terminating these connections?
  3. Is there a recommended approach for services needing persistent streaming connections to PostgreSQL on Railway?

Any help would be appreciated. Happy to share more logs or details.


7 months ago

have you tried setting tcp_keepalives_idle, tcp_keepalives_interval, and

tcp_keepalives_count on your PostgreSQL connection? Something like:

tcp_keepalives_idle = 30

tcp_keepalives_interval = 10

tcp_keepalives_count = 3

Also Railway's proxy/networking has a TCP idle timeout that kills connections that appear inactive


Status changed to Open brody 7 months ago


Building on the TCP keepalive answer already here — that's the right instinct

and probably part of it, but there's a second, more likely culprit worth

checking alongside it: wal_sender_timeout on the Postgres side, and whether

your CDC client is actually sending standby status updates independently of

its own processing.

wal_sender_timeout defaults to 60s. A logical replication client has to send

feedback more often than that or Postgres kills the walsender itself —

which produces exactly your symptom pair: "unexpected EOF on standby

connection" server-side, "Subscription timeout after 90 seconds" client-side.

Many client libraries only send that feedback after finishing a batch, not on

a timer — so it survives while batches are fast and dies the moment one runs

long, which fits your 2-10 minute variability.

Check:

SHOW wal_sender_timeout;

and separately, in whatever CDC library you're using, confirm the standby

status update fires on its own interval rather than after processing.

One more thing specific to Railway, worth ruling out alongside the TCP fix

above: .railway.internal resolves IPv6-only. If your replication client (or

its driver) has any IPv4 assumption in that code path — which is often

separate from the normal query-connection path — that alone can produce the

"hangs indefinitely" half of your report even after the keepalive fix handles

the "drops after N minutes" half. Worth testing the two failure modes

separately rather than assuming one fix explains both.


Welcome!

Sign in to your Railway account to join the conversation.

Loading...