Service keeps failing, but no error state or logs. Silent fail
exposuredesign
PROOP

3 months ago

I've had a few instances in the last week where my monitor service tells me this admin cms service is down. When I come to check there are no errors in the logs, the service says its still running, but when I go to the url to check in browser nothing is loaded, no errors in console, no html. Its odd.

When I restart container it rights itself, but as there are no error messages I am unsure what is happening?

Any idea how I can start to debug this?

Solved

1 Replies

sam-a
EMPLOYEE

3 months ago

Your deploy logs show the root cause. The AWS SDK's HTTP handler is hitting its default socket limit of 50 concurrent connections, with over 13,000 additional requests queued and growing. This means your application's outbound AWS calls are consuming all available sockets, so new incoming requests can't be processed. The Node.js process stays alive (no crash, no error state), but it's effectively frozen because everything is waiting for a socket. Restarting clears the queue, which is why that temporarily fixes it. The fix is on the application side - you'll need to either increase maxSockets in your NodeHttpHandler config, or investigate why so many AWS SDK requests are piling up (a connection leak or missing concurrency limit in your code).


Status changed to Awaiting User Response Railway 3 months ago


Railway
BOT

2 months ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway 2 months ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...