pgdata is Full - catch 22?
jorgelaranjo
HOBBYOP

2 years ago

pgData is Full (database is full). Now it cannot deploy or run (not expected, I'd say it should run at read only level).
In any case as it cannot run I cannot connect to DB to clean up and thus seems that only solution is to delete service, create new service and restore from backup?

Surely a better way would be to go into read only mode?

Solved

14 Replies

2 years ago

You can upgrade the plan to then grow the volume in use by the database!


Status changed to Solved Railway over 1 year ago


jorgelaranjo
HOBBYOP

2 years ago

Great, what if I don't want o grow the DB but in fact get rid of data so that it does not upgrade?
This is the staging env.


2 years ago

You might have luck with moving the volume to a filebrowser service and manually modifying the data.


jorgelaranjo
HOBBYOP

2 years ago

If I cannot connect to the DB, which by the way, is on the 500MB limit, thus not over it, how can I move the data? Seems a bug in my eyes, when the DB reaches its storage limit should go into read mode only I'd say, not offline and no way to get out of that state.


2 years ago

You are on the Hobby plan, may I ask what has stopped you from growing the volume the volume to 5GB?

Either way, whether Postgres crashes or not is far outside of Railway's control, that is the behaviour of Postgres itself when it runs out of disk space.

Please note that all databases on Railway are completely unmanaged, they are for the most part standard image deployments meaning Railway has no control over what happens when the database runs out of storage.


jorgelaranjo
HOBBYOP

2 years ago

There is no option that I can see to grow Postgresql space to 5GB.
Postgresql may well be 100% standard but if there's no way to get into that container data volume to extract the data in it, then seems to be a end state from which it is impossible to recover without upgrading?

Suspect that the 500MB limit is because it was a trial account converted into hobby and that container volumes didn't upgrade automatically.

❯ railway connect

> Select service Postgres

psql: error: connection to server at "xxx.xxx.rlwy.net" (xxx.xxx.xxx.xxx), port 41731 failed: server closed the connection unexpectedly

This probably means the server terminated abnormally

before or while processing the request.


2 years ago

There is no option that I can see to grow Postgresql space to 5GB.

This is found within the volume's settings.

Postgresql may well be 100% standard but if there's no way to get into that container data volume to extract the data in it, then seems to be a end state from which it is impossible to recover without upgrading?

As mentioned, you could try to move its volume to a filebrowser service and make any edits manually.

Suspect that the 500MB limit is because it was a trial account converted into hobby and that container volumes didn't upgrade automatically.

Correct, growing the volume is a manual process at the user's discretion because it involves downtime.


jorgelaranjo
HOBBYOP

2 years ago

As much as I look for, cannot find the Volumes settings.


2 years ago

Click on the volume, it's the attachment on the bottom of the service within the project canvas.


jorgelaranjo
HOBBYOP

2 years ago

Ok, got it... hard to find I'd say
I'd still argue that if a DB hits limit should be read only and not just plain cannot recover from this state.

Attachments


2 years ago

I'd still argue that if a DB hits limit should be read only and not just plain cannot recover from this state.

That would be very far out of Railway's control, you would need to take that feedback to Postgres themselves.


jorgelaranjo
HOBBYOP

2 years ago

A warning on docs that if one runs out of space the DB is out of action and cannot be connected to nor its volume, unless upgrade or destroying service and restoring from backup would be nice. At least one would know to prep a cron job to go into read mode after a threshold.


jorgelaranjo
HOBBYOP

2 years ago

Not in so far as that is achieved via the docker config or via cron job.

It would be something similar to if reaching 80% of total storage then go into read_only. As Railway is the one limiting the volumes, I'd say it is simple to make that an option. Otherwise should have a warning, even if just on documentation, that if running out of space the DB will be blocked from being accessed and there is no other way to reach that data volume, other than upgrade or reaching for a backup


2 years ago

I'm honestly not sure how read only mode would help you free up space, you would need the database to be read and writable to do that.


Loading...