a year ago
Hi!
I have a serivce with Medusa.js (Node), Redis and Postgres which I've had since september. I've had no problems with this service before, but after I migrated to the new databases I have gotten a problem where the service sometimes stops working (not immediately, but after a while, seemingly at random times). It doesn't crash but the API just times out or gets 404.
I've found the logs which I think are the culprit, but I still haven't found a solution yet. It might have something to do with Redis timing out or hitting memory limits perhaps? Just a guess from my part so far. Here are the logs:
Error: read ECONNRESET
at TCP.onStreamRead (node:internal/stream_base_commons:217:20) {
errno: -104,
code: 'ECONNRESET',
syscall: 'read'
}
AbortError: Ready check failed: Redis connection lost and command aborted. It might have been processed.
at RedisClient.flush_and_error (/app/node_modules/redis/index.js:298:23)
at RedisClient.connection_gone (/app/node_modules/redis/index.js:603:14)
at Socket. (/app/node_modules/redis/index.js:227:14)
at Object.onceWrapper (node:events:632:26)
at Socket.emit (node:events:517:28)
at TCP. (node:net:350:12) {
code: 'UNCERTAIN_STATE',
command: 'INFO'
}
[ioredis] Unhandled error event: Error: read ECONNRESET0 Replies
FYI I've started experiencing this recently as well, with a PG instance - did not used to have these issues before
I'm not experiencing it on both my services, so could be an app layer issue but didn't use to happen 🤷♂️
a year ago
often times this happens when you aren't closing connections and the idle timeout is reached
On my end my websockets are maybe suspicious, i'll take a look at some websocket settings
a year ago
for postgres pooled clients specifically this is solved by setting the pool minimum to 0 so that all connections are released and marked as closed
a year ago
your issue looks to be with redis, either way you would need to reference the documentation for your database client
a year ago
not sure, you would need to reference the documentation for your database client
a year ago
medusa is not a database client, the redis npm package is
If the connections to the DB are idle that means there are no DB operations? I should be having constant traffic 🤔 maybe I'm having some other underlying issue.
Or I am missunderstanding. Nonetheless trying to put min:0 and hoping for the best! 😄
a year ago
what tech stack are you using?
a year ago
does typeorm have a pool.min setting?
They have a poolSize which is max, but they also accept extra and pass it onto the underlying driver. So that should accept min I believe
a year ago
whats the underlying driver in use?
pg and as you say that, realizing that maybe it doesn't have a min setting and I need to use the idleTimeoutMillis or allowExitOnIdle
a year ago
allowExitOnIdle seems like what we want
a year ago
let me know how that goes!
a year ago
sorry but the issue here does not lie with railway, they have internal monitoring for these kinds of things and nothing has been reported. these errors are due to how the client is handling connections
@Rasmus Lian I have the same issue with medusa, only started since we migrated. Have you found a solution?
@latrapo Yes I solved it by upgrading Medusa and Redis services I think (cache service and notification provider) and make sure the Redis config is correct. I think that was it, I am afk atm so cant give you more atm
a year ago
make sure you aren't keeping any idle connections around and are sure to close a connection when done with it