5 months ago
I'm running a CDC (Change Data Capture) service that uses PostgreSQL logical replication to sync data between microservices in real-time.
Problem:
The START_REPLICATION streaming protocol connections are unstable. They either fail to establish (hang indefinitely) or drop every 2-10 minutes with "unexpected EOF on standby connection".
This happens on both internal networking (*.railway.internal:5432) and the external proxy (*.proxy.rlwy.net).
Regular SQL queries to the same databases work perfectly - only the streaming replication protocol is affected.
Error messages from PostgreSQL logs:
could not receive data from client: Connection reset by peer
unexpected EOF on standby connection
These appear after the connection has been streaming for a few minutes. PostgreSQL successfully accepts the connection and starts logical decoding, but the connection drops shortly after.
Error from my application:
Failed to subscribe to slot 'lightstream_user_db_slot': Subscription timeout after 90 seconds.
4 Replies
5 months ago
The subscribe sometimes takes multiple retry attempts before succeeding.
Setup:
- Multiple PostgreSQL databases on Railway (all with wal_level = logical)
- Bun/TypeScript service using pg-logical-replication npm package
- Replication slots and publications are set up correctly
- Service runs on Railway connecting via internal networking
5 months ago
What works:
- Regular SQL queries (SELECT, INSERT, UPDATE) - no issues
- Creating replication slots via SQL - works fine
- Logical decoding starts successfully on PostgreSQL side
5 months ago
Questions:
- Does Railway's networking support PostgreSQL's streaming replication protocol for long-lived connections?
- Are there TCP keepalive or idle timeout settings that could be terminating these connections?
- Is there a recommended approach for services needing persistent streaming connections to PostgreSQL on Railway?
Any help would be appreciated. Happy to share more logs or details.
5 months ago
have you tried setting tcp_keepalives_idle, tcp_keepalives_interval, and
tcp_keepalives_count on your PostgreSQL connection? Something like:
tcp_keepalives_idle = 30
tcp_keepalives_interval = 10
tcp_keepalives_count = 3
Also Railway's proxy/networking has a TCP idle timeout that kills connections that appear inactive
Status changed to Open brody • 4 months ago