a month ago
Hi. I've been running a small Go/SQLite program on Railway for the last month and I've noticed the volume is behaving strangely. The volume size has been increasing by a 1MB an hour for a bit less than a month now. I originally thought this might have been an application issue but the volume is reported as as only ~10MB:
~ # du -sh /main
10.4M /mainThe server binary is around 50MB if that makes a difference (I don't think it does), that's the only thing in /root.
But Railway is currently reporting the volume size as just under 800MB in the GUI, and it's increasing.
I'm not using WAL, I'm using the default synchronous mode (which does full rewrites of the database file I think) and I don't have backups enabled since I'm on the Hobby plan. I imagine my program is doing 1 or 2 database writes per minute.
I think Railway uses ZFS for volumes, which stores historical versions of files? When do the old blocks actually get deleted? Will they be deleted before I hit the 5GB default limit? I've tried restarting the service but that didn't do anything.
Thanks in advance.
4 Replies
a month ago
Hello!
We've escalated your issue to our engineering team.
We aim to provide an update within 1 business day.
Please reply to this thread if you have any questions!
Status changed to Awaiting User Response Railway • 27 days ago
a month ago
Just done so, it seems to have dropped significantly, thanks. It looks like there's been some weirdness in the last few days, I don't know if they were related to your changes.
The volume dipped slightly just before it hit 800MB (pretty much right after I posted this). It dipped slightly more an hour ago but then breifly spiked to 2GB, with my deploy running out of memory at the same time despite my RAM usage remaining way below the cap (unless the email was wrong, but something caused a restart). The volume is at 173MB which I imagine is a normal amount of overhead?
Status changed to Awaiting Railway Response Railway • 27 days ago
a month ago
We ran some cleanup commands on our end to reclaim the space. The root cause is file descriptors not being closed properly in your application. You'll need to fix this in your code - we won't be able to provide further assistance with this issue.
Status changed to Awaiting User Response Railway • 27 days ago
Status changed to Solved ray-chen • 26 days ago
a month ago
Interesting, I'll look into that, I wonder if my shutdown process isn't working properly or the SQLite driver is doing something weird with the connection pool. It's weird that redeploying normally doesn't fix it though. Thanks for your help
Status changed to Awaiting Railway Response Railway • 26 days ago
Status changed to Solved hedgehog125 • 26 days ago