Database connection is dropping and even railway cli is lagging!!!!!
amp
PROOP

3 months ago

Hello Support,

I’ve been experiencing serious issues with my PostgreSQL database for the past 2–3 days. At certain times, the database becomes extremely slow, and even the Railway CLI is unable to connect. The application server relying on this database is also failing to connect during these periods.

This is causing significant disruptions to my service, and I need this resolved as soon as possible. If the issue is not resolved within a few hours, I will have no choice but to migrate to another database provider.

Please look into this urgently.

Thank you,

$30 Bounty

16 Replies

Railway
BOT

3 months ago

Hey there! We've found the following might help you get unblocked faster:

If you find the answer from one of these, please let us know by solving the thread!


brody
EMPLOYEE

3 months ago

This thread has been marked as public for community involvement, as it does not contain any sensitive or personal information. Any further activity in this thread will be visible to everyone.

Status changed to Open brody 3 months ago


noahd
EMPLOYEE

3 months ago

Hey there!
Could I please get a bit more info about this service?
What stack is it?
How much data are you storing?
Are you using an ORM?
If so, what ORM?
Are you doing N calls?
Do you have proper indexing?
Are you using connection pooling and are there multiple clients connecting?
Is the database marked as serverless?


amp
PROOP

3 months ago

Hey! Here’s a bit more context about the setup and what we’re seeing:

  • Stack: The service is built with NestJS on the backend, using Prisma ORM for database interactions.

  • Database: We’re using PostgreSQL, deployed in a managed environment. It’s not explicitly marked as serverless, but connections behave similarly under scaling conditions.

  • ORM: Yes, Prisma is the ORM.

  • Queries: In some cases, multiple calls (N+1 type patterns) may occur depending on the feature, but we’ve been consolidating where possible.

  • Indexing: Proper indexing exists on the main relational fields (userId, gameId, etc.), though I’m reviewing whether additional indexes are needed for the most active queries.

  • Connection pooling: Enabled via Prisma + database driver settings. Multiple clients do connect simultaneously through WebSocket sessions, which puts some load on the pool.

The error we’re seeing:

Invalid prisma.userStatistics.update() invocation:

Server has closed the connection.


amp
PROOP

3 months ago

Any update?

We fixed every N+1 pattern and server is not loading database but issue still persists!


fra
HOBBY

3 months ago

how are the metrics? usually that's the first thing to check, do you have blocking transactions or any slow query?


amp
PROOP

3 months ago

Thanks for asking! I just checked the database side — metrics look normal overall. No unusual spikes, blocking transactions, or slow queries showing up in the logs/monitoring. Everything appears healthy and there’s nothing different or concerning on the metrics side.


amp
PROOP

3 months ago

Hello,

Following up on this issue — the database is still dropping connections even under minimal load. For example, just now, only one client was connected, and a simple read + update transaction failed with the error:

Transaction API error: Unable to start a transaction in the given time.

This happened despite metrics showing no blocking transactions or slow queries. A similar issue also occurred on another database in a separate environment around the same time.

Here are the Prisma logs:

[Nest] 30024  - 09/09/2025, 10:43:47 PM   DEBUG [PrismaService] [QUERY] BEGIN -- params: [] -- duration: 264ms
[Nest] 30024  - 09/09/2025, 10:43:47 PM   DEBUG [PrismaService] [QUERY] SELECT "UserStatistics"."id", "UserStatistics"."biggestWin" FROM "UserStatistics" WHERE ("UserStatistics"."userUsername" = $1) LIMIT $2 OFFSET $3 -- params: ["<REDACTED>",1,0] -- duration: 530ms
[Nest] 30024  - 09/09/2025, 10:43:48 PM   DEBUG [PrismaService] [QUERY] UPDATE "UserStatistics" SET "totalWagered" = ("UserStatistics"."totalWagered" + $1), "coinflipsWon" = ("UserStatistics"."coinflipsWon" + $2), "biggestWin" = $3, "updated_at" = $4 WHERE ("UserStatistics"."userUsername" = $5) RETURNING "UserStatistics"."id", "UserStatistics"."userUsername", "UserStatistics"."totalWagered", "UserStatistics"."coinflipsWon", "UserStatistics"."created_at", "UserStatistics"."updated_at", "UserStatistics"."biggestWin" -- params: [301,1,602,"<REDACTED>","<REDACTED>"] -- duration: 538ms
[Nest] 30024  - 09/09/2025, 10:43:48 PM   ERROR [CoinflipService] Failed to save game in history
{"gameId":"<REDACTED>","player1Username":"<REDACTED>","player2Username":"<REDACTED>","error":"Transaction API error: Unable to start a transaction in the given time.","timestamp":"2025-09-09T18:43:48.912Z"}
[Nest] 30024  - 09/09/2025, 10:43:48 PM   ERROR [CoinflipService] Error in remaining operations processing
{"gameId":"<REDACTED>","error":"Transaction API error: Unable to start a transaction in the given time."}
PrismaClientKnownRequestError: Transaction API error: Unable to start a transaction in the given time.
  code: 'P2028'
  clientVersion: '6.15.0'

This suggests the problem may not be caused by query complexity or load, but could be related to connection handling or the managed database environment itself.

Could you please investigate this urgently? The instability is affecting production.

Thank you.


amp
PROOP

3 months ago

[Nest] 36884  - 09/09/2025, 10:48:11 PM   ERROR [PrismaService] [PRISMA ERROR] Error in PostgreSQL connection: Error { kind: Io, cause: Some(Os { code: 10054, kind: ConnectionReset, message: "An existing connection was forcibly closed by the remote host." }) }
Redis Lock Client Error Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:216:20) {
  errno: -4077,
  code: 'ECONNRESET',
  syscall: 'read'
}
Redis Client Error Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:216:20) {
  errno: -4077,
  code: 'ECONNRESET',
  syscall: 'read'
}
Redis Client Error Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:216:20) {
  errno: -4077,
  code: 'ECONNRESET',
  syscall: 'read'
}
[Nest] 36884  - 09/09/2025, 10:48:22 PM   ERROR [PrismaService] [PRISMA ERROR] Error in PostgreSQL connection: Error { kind: Io, cause: Some(Os { code: 10054, kind: ConnectionReset, message: "An existing connection was forcibly closed by the remote host." }) }

Right now database is not even reachable


fra
HOBBY

3 months ago

What's the size of the dB and what's the size of the attached volume?


amp
PROOP

3 months ago

Database has 32gb memory 2 is used right now, 32 vCPU metrics show that 0.0 is used always, max volume usage is 20gb and only 2 is used


fra
HOBBY

3 months ago

I think you might need to wait someone from railway, the only other thing I'd try would be to restart the dB (after a backup)


amp
PROOP

3 months ago

Thank you for the suggestion and for your help!

I have already restarted the development database several times when the production database experiences issues. The development database exhibits the same connection problems simultaneously, and restarting it has not resolved the issue.

Do you know when someone from Railway will be able to respond or investigate this further? The instability is affecting production, and we need urgent assistance.


fra
HOBBY

3 months ago

I think you can set a priority in your thread:
https://docs.railway.com/reference/support#pro--business-class

I'm not on pro so I don't know how to do that, but in theory you should have a higher priority


noahd

Hey there! Could I please get a bit more info about this service? What stack is it?How much data are you storing?Are you using an ORM?If so, what ORM?Are you doing N calls?Do you have proper indexing?Are you using connection pooling and are there multiple clients connecting?Is the database marked as serverless?

amp
PROOP

3 months ago

Can you check thread?


amp
PROOP

a month ago

Ty for fast response staff


fra
HOBBY

a month ago

one more thing you could check is the number of connections to the db.

ps. on railway the db is NOT managed


Loading...