Seeing intermittent errors connecting to postgres database
samkotlove
PROOP

a month ago

We saw a burst of connection failures in production today, ~15:30–15:40 UTC (Jul 18), and I believe it originated in Railway's internal networking/DNS rather than our app. We're continuing to see intermittent failures and slowness when connecting to our own postgres database

Solved

9 Replies

Status changed to Awaiting Railway Response Railway about 1 month ago


claygoddard
HOBBY

a month ago

Same here. I think there might be a Railway networking issue ongoing? I see nothing on the status page yet. But a few posts about this now. DB network connections are not working.


samkotlove
PROOP

a month ago

For color, at 15:30:08 UTC, a single dns.lookup for postgres.railway.internal (→ 10.222.217.136) took 37.1 seconds to resolve. Seeing that spike up and down currently


claygoddard
HOBBY

a month ago

Mine actually appears to have resolved for the moment (with no action on my part). After a good bit of downtime..


echohack
EMPLOYEE

a month ago

We are aware of an issue affecting Outbound Static IPs in US East. Some users relying on static egress IPs may experience intermittent connectivity issues.

We are actively investigating and working to mitigate the impact. We will provide updates as we learn more.

You can follow along here for updates: https://status.railway.com/incident/66L640AT


Status changed to Awaiting User Response Railway about 1 month ago


samkotlove
PROOP

a month ago

This seems to have popped up again at 15:30–15:34 EDT (19:30–19:34 UTC)


Status changed to Awaiting Railway Response Railway about 1 month ago


samkotlove
PROOP

a month ago

After digging further that was a spike (15:30) but seeing DNS issues continually popup today...


alex-delia
PRO

a month ago

Are you on the public proxy or private networking? I'm seeing intermittent 'Connection terminated unexpectedly' issues as well on the public proxy


samkotlove
PROOP

a month ago

Private networking


sam-a
EMPLOYEE

a month ago

Thanks for the detailed timestamps, they let us confirm this on our side. We verified elevated internal DNS resolution latency on a set of US East hosts during both windows you reported: July 18 around 15:30 UTC (the larger event, which correlated with heavy load on several hosts) and a smaller recurrence on July 19 around 19:30 UTC. This was on Railway's side, not your application, and it was separate from the Outbound Static IPs incident we linked earlier.

DNS latency across the fleet is currently back to normal. Let us know if you need more help.


Status changed to Awaiting User Response Railway about 1 month ago


Railway
BOT

24 days ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway 24 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...