Internal service connection failed to Redis, how can I debug

2 months ago

Project ID: f709cfb8-baef-49d9-b8ac-fd8d1140bf26

On Jun 7, ||from 04:09:20 to 07 06:08:52|| for 2 hours there seems to be tcp connection issue according to the error logs from one of our internal service trying to connect to Redis.

The internal service was trying to run few commands but due to network issue, the output said:

INTERNAL_ERROR: redis eval error: dial tcp []:6379: i/o timeout
Failed to write server state data: UNKNOWN: redis command error: SADD failed: dial tcp 
// and couple of other redis command errors

On the redis side we didn't saw any error.

  • How can we debug and learn more about this network issue?

  • Is there any additional logging we can/should add?

  • Can railway staff help us to learn the cause of this network connection issue?

0 Replies

2 months ago

please provide timestamps in utc


2 months ago

This is the start/end time the railway logger (in the sidebar) showed:
start: 04:09:20
end: 07 06:08:52

and this is what our internal service time logged:
start: 2025/06/06 22:39:14.312325
end: 2025/06/07 00:38:52.350276

in both cases not exactly sure what's the timezone set. sorry for the extra trouble, hope this help.


2 months ago

the timezone displayed by the dashboard would be localized, what timezone are you in


2 months ago

IST


2 months ago

After combing over our internal data, I could not correlate your timestamps to anything in our data for the same region.


2 months ago

here's the screenshot of the error logs

1382264236067258400


2 months ago

I had rechecked our data, I could not correlate your timestamps to any errors in our data for the same region.


2 months ago

okay, thanks for re-checking. 🙂


Internal service connection failed to Redis, how can I debug - Railway Help Station