4 hours ago
Service memory grows with every file written to the volume and only drops on restart
Problem and desired outcome
I run a small S3-compatible storage server (Dart, AOT-compiled, a scratch image) that writes uploaded files to a Railway volume. Locally, the process stays at ~15–45 MB of RAM while uploading large files. On Railway, the service's memory metric grows by about the same amount as the data written to the volume (1.4 GB after uploading ~1 GB of files) and stays there until the container restarts, when it drops back to ~7 MB. I would like to know what the memory metric counts (in particular, whether it includes the kernel page cache of files written to the volume), whether that memory is billed, and the recommended way to keep it from accumulating.
Resources and timing
This is a public thread, so I'm not sharing the IDs, domains or deployments of my project; the affected service is selected in the support form. The setup is one service with a volume mounted at /data (region US East), plus a static web console in a second service.
The project is open source, so anyone can reproduce it: https://github.com/Root-101/feather-io (the service is in service/, release v1.0.0-rc.3, built from its Dockerfile).
There is a Railway template for it, prepared and working, but not published yet: I'm holding it until this memory question is answered, since anyone deploying it would hit the same behaviour.
Evidence and attempts
Memory and volume usage of the service from railway metrics --since 1h --memory --volume --raw (CLI 5.64.2), times in UTC on 2026-10-12:
time memory volume
02:56 7 MB 83 MB service started
03:13 120 MB 92 MB upload of a ~200 MB file starts
03:15 246 MB 288 MB upload done, memory stays there
03:27 234 MB 288 MB idle for 12 minutes, no drop
03:28 314 MB 288 MB upload of a 780 MB file starts
03:32 1,099 MB 1,113 MB
03:36 1,430 MB 1,437 MB peak
03:37 7 MB 1,437 MB new deployment (restart)
03:45 82 MB 1,493 MB files still on the volume, memory does not come backThe 780 MB file was uploaded from a web client as multipart (8 MiB parts, joined into one file at the end), so the data is written to the volume roughly twice.
Local reproduction (performed): the same Docker image on Docker Desktop (WSL2 kernel, cgroup v2), uploading a 780 MB file with a single streaming PUT. Values from /proc/1/status of the process and memory.current / memory.stat of the container's cgroup:
process VmRSS cgroup memory.current anon file
before upload 15 MB 8 MiB 6 MiB 1 MiB
after upload 45 MB 841 MiB 35 MiB 781 MiB
30 s later 45 MB 840 MiB 35 MiB 781 MiBdocker stats (which subtracts inactive file cache) showed 59 MiB after the upload. The application streams request bodies to disk in small chunks and never holds a whole file in memory.
Attempts: none to change the behaviour yet. A restart (the new deployment at 03:37) is the only thing observed to bring the memory metric back down.
Why contact support?
I (the user) decided to contact support. I want to understand how Railway measures and bills memory for a service that writes large files to a volume, which I can't determine from the metrics alone.
Questions for Railway:
- Does the memory metric of a service include the page cache (
fileinmemory.stat) of files written to its volume? - Is that memory billed as memory usage?
- Is it reclaimed by the kernel under memory pressure or when it nears the service's limit, or does it count against the limit until a restart?
- Is there a recommended way to avoid it accumulating (for example a setting, or the application calling
posix_fadvise(POSIX_FADV_DONTNEED)after writing each file)?
Uncertainty and constraints
Observation: the memory metric tracks the data written to the volume almost MB for MB, and drops to ~7 MB after a restart even though the files remain on the volume. Hypothesis (not confirmed): the metric includes the page cache of the written files, as in my local cgroup measurement where 781 MiB of the 841 MiB were file. I have not checked whether the process heap itself grows on Railway, because the image has no shell.
A scheduled restart would bring the metric down but interrupts the service, so I'd rather avoid it.
Unrelated observation that may or may not matter: railway status reports the attached volume as 0.0 GB / 4.9 GB, and the summary of railway metrics as 4.2 MB, while the raw DISK_USAGE_GB series shows ~1.49 GB.
Prepared with Railway's support template (v1).
And fine-tuned by me.
Pinned Solution
4 hours ago
That growth is the kernel page cache of the volume writes, and Railway's memory metric (cgroup memory.current plus swap, sampled about every 30 seconds) includes it, so it is billed as memory at $10 per GB-month I believe. It stays until a restart because, with no replica limit, the ceiling rises with usage and the kernel never has pressure to evict clean pages. To help fix this, set a replica memory limit (somewhat around) 512 MB, since the process itself is only about 15 to 50 MB, and the kernel will reclaim that cache before an OOM (which I assume you know as Out of Memory). posix_fadvise(POSIX_FADV_DONTNEED) after fdatasync on the open fd drops clean pages too, but later reads pull them back, so the limit is what actually caps it.
4 Replies
Status changed to Awaiting Railway Response Railway • about 4 hours ago
4 hours ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • about 4 hours ago
4 hours ago
i would presume the memory metric contains all kind of memory so page cache included. maybe ur image is handling writes weirdly on Linux, and not flushing properly? maybe write a mem profiler into ur app to log memory usage and not just rely on railway metrics to get more in depth information?
4 hours ago
That growth is the kernel page cache of the volume writes, and Railway's memory metric (cgroup memory.current plus swap, sampled about every 30 seconds) includes it, so it is billed as memory at $10 per GB-month I believe. It stays until a restart because, with no replica limit, the ceiling rises with usage and the kernel never has pressure to evict clean pages. To help fix this, set a replica memory limit (somewhat around) 512 MB, since the process itself is only about 15 to 50 MB, and the kernel will reclaim that cache before an OOM (which I assume you know as Out of Memory). posix_fadvise(POSIX_FADV_DONTNEED) after fdatasync on the open fd drops clean pages too, but later reads pull them back, so the limit is what actually caps it.
3 hours ago
Thanks to both of you!
@milo, you were right that the metric includes the page cache. It matches what I measured locally: in the container's cgroup, 781 MiB of the 841 MiB after the upload were file (page cache) and only 35 MiB anon, while the process RSS stayed at ~45 MB. It wasn't a flushing problem, though. Unflushed data would show up as file_dirty, and the kernel writes it back within seconds; these were clean pages that simply stay cached until something needs the memory.
@gamerc0der, thanks for the explanation of why it never went down, and for the fix: with no replica memory limit the kernel never has a reason to evict that cache. The limit solved it.
I set the service's memory limit to 512 MB and uploaded a 750 MB file from the web console (multipart, 99 parts of 8 MiB, then joined into one file). From railway metrics, on 2026-10-13:
time (UTC) memory volume
15:37 9 MB 1,494 MB limit set to 512 MB, before the upload
15:39 203 MB 1,646 MB uploading the parts
15:40 405 MB 1,767 MB
15:41 500 MB 2,171 MB at the limit, parts still uploading
15:42 500 MB 2,799 MB joining the parts into the final file
15:42 281 MB 2,319 MB upload done, parts deleted
15:47 272 MB 2,319 MB idleEvery request answered 200 (99 UploadPart and the CompleteMultipartUpload, which took 21.6 s to join the parts), and there was no OOM or restart. Memory went up to the limit and stayed there while the volume kept growing by another ~1 GB, so the kernel was reclaiming clean page cache instead of killing the process. After the upload it settled at ~270 MB, which is cache still under the limit: without pressure nothing evicts it, but it can no longer grow past the limit.
For anyone who lands here with the same problem: if a service writes large files to a volume, set a replica memory limit. Without one, the page cache of those writes counts as memory until the next restart. Leave room for the process's own peak: this service uses 15–50 MB normally, but up to ~400 MB under heavy concurrent multipart uploads.
I'll recommend this limit in the project's Railway template when I publish it. Marking this as solved.
3 hours ago
A quick follow-up: the Feather IO template is now published, and it includes the recommendation from this thread to set a replica memory limit of 512 MB on the storage service: https://railway.com/deploy/feather-io
Thanks again to @milo and @gamerc0der for the help. Any feedback on the template is welcome.
Status changed to Solved medim • about 3 hours ago
