PostGIS Database Freezing Unexpectedly on Railway - Requiring Manual Restarts
sgutgar
FREEOP

9 months ago

I've been experiencing issues with my PostGIS database freezing unexpectedly on Railway. This has happened twice in the last two days:

  1. Yesterday: The database froze while our data team was updating data

  2. Today: It happened again when applying migrations after a database deployment

In both cases, I was able to resolve the issue by manually restarting the PostGIS container in my Railway project. I'm using the postgis:postgis/16-master image.

Are there known configuration issues or resource constraints that might be causing this? Any suggestions for preventing these freezes would be greatly appreciated, as these interruptions are impacting our workflow.

Would adding more resources to the service help? Or are there specific configurations I should be adjusting to improve stability?

Is there a way to detect that the database container is frozen and apply an automatic restart?

Thanks in advance for any insights!

Solved

3 Replies

angelo-railway
EMPLOYEE

9 months ago

I would need to know more information about the metrics that you are seeing. My gut reaction here is that your query is causing the DB to lock. I will need more information to make that determination.

Status changed to Awaiting User Response Railway • 9 months ago

sgutgar
FREEOP

9 months ago

This is the migration that I was running the second time that the database got frozen:

await queryRunner.query(`ALTER TABLE "supplier_spare" ADD "isImageFromSpare" boolean NOT NULL DEFAULT false`);

Nothing fancy I would say... Is there a way to detect that the database is not responding to restart it automatically?

When I tried to make a deployment of my staging environment everything was running as expected up to the migrations, which got stuck in the line that I mention above. After this, I tried to access my database from my local machine through dbeaver and it wasn't loading, it didn't throw an error, it just kept on loading. Then I manually restarted my postgis container in railway and the backend (since the migrations crashed because the connection with the database was lost with the restart) and then everything worked as expected, the migration was applied correctly and it has been working fine up to now.

The first time it got frozen was because of some updates that we were performing on some data in the database.

My problem is that it just get frozen, it doesn't throw an error, an email or anything so this is hard to detect when it happens, so an automatic restart would be a nice and quick solution. Is there a way to configure this?

Status changed to Awaiting Railway Response Railway • 9 months ago

angelo-railway
EMPLOYEE

9 months ago

Not built into Railway unfortunately, you will need to configure an external service that would detect this.

Status changed to Awaiting User Response Railway • 9 months ago

Railway
BOT

4 months ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway • 4 months ago

Loading...