I ran a recursive query and now there is no more space on device
themkrage
PROOP

2 years ago

I am getting this error:
2024-03-29 22:10:54.413 UTC [8] FATAL: could not write lock file "postmaster.pid": No space left on device

but i can't access the database with psql or anything to try and remove the uneccessary data. This is resulting in downtime for us.

0 Replies

brody
EMPLOYEE

2 years ago

hey @themkrage what is the current size of your volume?


themkrage
PROOP

2 years ago

50GB

It looks like the recusrive pg query i ran filled it. It's not like the database has 50GB's of content, just that one query appears to have filled the container

1223395636368375800


themkrage
PROOP

2 years ago

i don't see an option to increase the cap, and im not sure how to clear the "in-progress" query and data


themkrage
PROOP

2 years ago

i can't connect to the database either, so i can't run VACUUM, which i think would fix this


brody
EMPLOYEE

2 years ago

Okay ill do my best to get a team member here asap, in the mean time please provide your project id


themkrage
PROOP

2 years ago

/98f66160-ce86-4b15-a73b-cab912256208



Hello, working on expanding your cap now


One sec


themkrage
PROOP

2 years ago

thank you!


Okay, try now


Should be up to 100 GB


themkrage
PROOP

2 years ago

thank you! able to connect. now running VACUUM


themkrage
PROOP

2 years ago

thank you. Running vacuum brought our disk usage back to normal expected level


themkrage
PROOP

2 years ago

For future reference, is there no way to access a database that has hit the disk usage cap?


brody
EMPLOYEE

2 years ago

correct


brody
EMPLOYEE

2 years ago

postgres more or less locks itself up when it runs out of disk


themkrage
PROOP

2 years ago

is there some internal safe guard we could add to prevent this?


brody
EMPLOYEE

2 years ago

there is nothing that i am personally aware of


Loading...