pgvector service — dashboard shows Volume Size 250 GB, but mounted filesystem is only 4.6 GB
elmasri-fathallah
PROOP

22 days ago

Project ID: cfa70aaf-ab41-4cb1-86a1-bab0a7f62c07

Service: pgvector (43629917-2dcb-47d4-916a-fa74dec28634)

Volume: pgvector-volume (7f23aec6-16cb-42b2-982f-f7831bf95c5a)

Issue: Postgres has been crashing in an infinite ENOSPC PANIC loop for

several hours. The dashboard volume page reports Volume Size: 250 GB

with ~5 GB usage. The Live Resize dialog also lists 500 / 750 / 1000 GB

as the only resize options and is gated behind ">=185 GB used", which

matches the dashboard view.

But inside the running container, df -h reports the volume's actual

filesystem is only 4.6 GB and 100% full:

/dev/zd2512    4.6G    4.5G    0    100%    /var/lib/postgresql

The data directory $PGDATA = /var/lib/postgresql/data/pgdata is

consuming 4.5 GB, which is essentially the entire filesystem.

Questions:

  1. Why does the dashboard show 250 GB while the kernel sees 4.6 GB?

    Is the volume thin-provisioned with a smaller initial allocation,

    or is the 250 GB just a maximum cap?

  2. Can you grow the actual filesystem to match the configured 250 GB

    (or some larger tier) without us needing to first reach the

    =185 GB usage threshold? Postgres can't run, so we can't increase

    usage organically to qualify for the next tier.

  3. Is there an operator-side path to expand the filesystem now while

    the database is offline?

Production is down. Any priority help appreciated.

While you're investigating, can you take a fresh volume snapshot of the

CURRENT pgvector-volume (the broken one) before any resize or restore

action? I want to preserve a recoverable copy of the latest state in

case I need to fall back to the March 8 backup as an interim measure.

Solved

2 Replies

Status changed to Awaiting Railway Response Railway 22 days ago


elmasri-fathallah
PROOP

21 days ago

While you're investigating, can you take a fresh volume snapshot of the

CURRENT pgvector-volume (the broken one) before any resize or restore

action? I want to preserve a recoverable copy of the latest state in

case I need to fall back to the March 8 backup as an interim measure.


19 days ago

The filesystem on your pgvector volume has been extended to match the configured 250 GB. A fresh snapshot of the volume was taken before the resize, so you have a recoverable copy of the pre-resize state. Your service should now be able to start without the ENOSPC errors - please redeploy and confirm Postgres comes up cleanly.


Status changed to Awaiting User Response Railway 19 days ago


Railway
BOT

11 days ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway 11 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...