17 days ago
We're experiencing recurring unexpected PostgreSQL connection terminations on our service, causing cascading failures across all queue workers.
Environment:
- Database: Railway-managed PostgreSQL
- Connection library: pg-pool (node-postgres)
What's happening:
At 2026-02-19 17:14:11 UTC, all active database connections were terminated simultaneously, resulting in ECONNRESET errors across every queue worker:
- billing, connection-messages, webhooks, webhookWorker, secret-agent, linkedin-invites, emails, follow-up-actions
Error stack trace:
```
Database query error: Connection terminated unexpectedly
at /app/node_modules/pg-pool/index.js:45:11
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Object.query (/app/src/config/database.js:65:17)
at async processJobs (/app/src/services/bulkCollectionProcessor.js:78:20)
Key observations:
- All connections dropped at the exact same second, suggesting a server-side disconnection (not a client timeout)
- This is not an isolated incident — it has been recurring
- Our application uses connection pooling via pg-pool
Questions:
1. Was there any PostgreSQL maintenance, failover, or resource limit enforcement at that time?
2. Are there any known issues with the PostgreSQL instance or underlying infrastructure?
3. Is there a connection limit or idle timeout configured on the Railway side that could be causing this?
We'd appreciate any insight into what's triggering these disconnections. Happy to provide additional logs or details.
Thanks!
1 Replies
17 days ago
None except for networking weakness we have finally got a handle over the last 48 hours, which we apologize.
What happened was we had SYN attacks to some workloads. You may have had Cloudflare in front of your app, however, non WAF workloads could have affected the availability. As such we have implemented short terms measures to prevent this moving forward. Since then we've expanded interconnect capacity with new peering and Direct Connect, deployed per-container eBPF firewalls, hardened our edge, and built continuous connectivity monitoring between hosts. We're also rolling out a WAF for all customers, edge-level protection against malicious request patterns across all plans.
Status changed to Awaiting User Response Railway • 17 days ago
10 days ago
This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!
Status changed to Solved Railway • 10 days ago