2 months ago
Volume vixynaserver-volume shows ~75% / ~370 MB usage in Railway Volume Metrics, but inside the mounted volume /app/data, df -h reports only 936 KB used and du shows ~812 KB of actual files. The filesystem is ext4 mounted with discard. Attempting fstrim -v /app/data returns FITRIM ioctl failed: Operation not permitted. Can you verify whether the volume backend has unreclaimed allocated blocks or whether the Volume Metrics data is inaccurate?
3 Replies
2 months ago
The volume metrics report ZFS-level allocated blocks, not the logical size of live files. When files are deleted or rewritten, freed blocks are only returned to the storage layer via TRIM/discard. Volumes are mounted with continuous discard, so reclamation happens over time, but it will never fully converge to your df/du figure because volumes use 16 KB blocks while ext4 frees space in 4 KB blocks - partially freed 16 KB blocks cannot be returned, so scattered small-file workloads can pin allocation well above guest-visible usage permanently. The fstrim failure is expected since containers lack the required capability. Billing is based on this same metric (GB-minutes at $0.15/GB-month), though the dollar impact of the gap is small. The only way to fully reset the allocation footprint is to migrate your data to a fresh volume and delete the old one.
Status changed to Awaiting User Response Railway • about 2 months ago
Railway
The volume metrics report ZFS-level allocated blocks, not the logical size of live files. When files are deleted or rewritten, freed blocks are only returned to the storage layer via TRIM/discard. Volumes are mounted with continuous `discard`, so reclamation happens over time, but it will never fully converge to your `df`/`du` figure because volumes use 16 KB blocks while ext4 frees space in 4 KB blocks - partially freed 16 KB blocks cannot be returned, so scattered small-file workloads can pin allocation well above guest-visible usage permanently. The `fstrim` failure is expected since containers lack the required capability. Billing is based on this same metric (GB-minutes at $0.15/GB-month), though the dollar impact of the gap is small. The only way to fully reset the allocation footprint is to migrate your data to a fresh volume and delete the old one.
2 months ago
Thanks, that explains why df/du differs from the Volume Metrics. However, the volume currently reports roughly 370 MB allocated while the guest filesystem contains less than 1 MB of actual data. Can Railway perform or assist with a backend migration/reclamation of this volume so I can preserve the existing persistent data while resetting the allocation footprint? If not, please confirm the recommended procedure for migrating /app/data to a fresh Railway volume without data loss.
Status changed to Awaiting Railway Response Railway • about 2 months ago
2 months ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • about 2 months ago
vixynaserver
Thanks, that explains why df/du differs from the Volume Metrics. However, the volume currently reports roughly 370 MB allocated while the guest filesystem contains less than 1 MB of actual data. Can Railway perform or assist with a backend migration/reclamation of this volume so I can preserve the existing persistent data while resetting the allocation footprint? If not, please confirm the recommended procedure for migrating /app/data to a fresh Railway volume without data loss.
2 months ago
you don't need railway to do this for you, the CLI can move volume files directly. with under 1MB of real data it should take a couple of minutes.
tar it up inside the service so permissions and structure survive:
railway ssh then tar czf /app/data/backup.tar.gz -C /app/data --exclude=backup.tar.gz .
pull it down:
railway volume files download /backup.tar.gz ./backup.tar.gz
check the tarball locally before you touch anything, tar tzf backup.tar.gz, don't skip this bit
detach the old volume and attach a fresh one at the same mount path (/app/data)
push it back and extract:
railway volume files upload ./backup.tar.gz /backup.tar.gz
then railway ssh and tar xzf /app/data/backup.tar.gz -C /app/data
once the app's happy and the data's all there, delete the old volume, that's the bit that actually clears the allocation
docs: https://docs.railway.com/cli/volume and https://docs.railway.com/cli/ssh
that said, 370MB at $0.15/GB-month is about 5 cents a month, so unless the number itself is bugging you it might not be worth the hassle.