Hey there! We've found the following might help you get unblocked faster:

• 🧵 Workflow Not Executing – Worker Not Processing Jobs + MaxListenersExceededWarning
• 🧵 n8n service consuming excessive memory and unable to restart - Postgres in recovery mode
• 🧵 Memory & Kernel Settings for MongoDB Container Stability

If you find the answer from one of these, please let us know by solving the thread!

im not sure if there is a way to clean up memory with what youre describing. you can attempt to reduce how much accumulates by reducing work_mem per connection, use temp tables, auto vacuum.

your available 32 vCPU & memory is there only for your service to use it, that capacity is available for the service to use, but you aren’t actively billed for the full allocation. You’re billed for the real usage, not the total provisioned quota, having more room simply lets the service scale up when it needs it.

my thoughts are you can probably automate this though railway's graphql https://docs.railway.com/guides/manage-deployments, which might save you some time by doing that. youd need

projectId

environmentId

serviceId (the Postgres service)

to be able to automate the process. a simple node script can complete it right after your entire insert workload

I think it could be an OS level "issue" with glibc or whatever it's using where it has no reason to eagerly free up the memory since it thinks it has 32GB available. I was hoping to tune that somehow to free it. I realize I'm not billed for available memory, but I would be billed for 5GB that really is idle/not needed anymore. I'll keep investigating.

I wanted to try jemalloc but it looks like that's not installed on the server.

For now I'm going to reduce the available memory for my Postgres. I think it's the only option since others would require OS level configs that we cannot apply since it's shared.

i believe if you reduce the available memory for your postgres, it'll OOM as it won't have enough resources

No its working fine with 2GB set rather than 32GB. I believe the issue is the OS File Cache builds up and I'm getting billed for that even though it's unused and available to be reclaimed.

Running this:

SELECT sum(total_bytes) / 1024 / 1024 AS "Total_MB_Used_By_Postgres" FROM ( SELECT count(*) * 8192 AS total_bytes FROM pg_buffercache UNION ALL SELECT sum(used_bytes) FROM pg_backend_memory_contexts ) sub;

Returns 130MB, even when it says I'm being billed for 5GB+. Reducing my available memory at least limits my billing as a temporary fix.

Two things are probably going on, and the first one explains why your

MALLOC_ tweaks appeared to do nothing.

MALLOC_ARENA_MAX and MALLOC_TRIM_THRESHOLD_ are read by glibc at process

start. Postgres backends inherit their environment from the postmaster, so

setting those variables only takes effect for backends forked after the

postmaster itself restarted with them set. If you added them as service

variables and didn't restart Postgres, every existing backend kept the old

allocator config — and any backend forked from the old postmaster still does.

Set them, restart once, then re-test. That's a real possibility your earlier

test was invalid rather than ineffective.

Second, and more useful: the 5GB is almost certainly per-backend private

memory, not shared memory, and it's held by the specific backend that ran the

job. Postgres frees it internally at query end, but glibc keeps it in the

arena rather than returning it to the OS — which is exactly what you're

seeing on the RSS graph. The reliable way to get it back without restarting

the server is to end the process that owns it:

SELECT pg_terminate_backend(pid)

FROM pg_stat_activity

WHERE state = 'idle'

  AND backend_type = 'client backend'

  AND pid <> pg_backend_pid();

The OS reclaims a backend's entire private heap the moment it exits. That's

a restart of the connection, not of Postgres — no downtime, no shared buffer

cache loss, no WAL replay.

If your CDC/job client holds one long-lived connection, that single backend

hoards the peak forever. Two fixes, pick one:

• Have the job disconnect when it finishes. Simplest, works immediately.

• Put PgBouncer in front with server_lifetime set (900s or so) so server

  connections get recycled on a schedule regardless of what clients do.

Also worth doing regardless: drop work_mem globally and raise it only for the

burst, inside the job's transaction:

SET LOCAL work_mem = '512MB';

With a global work_mem, every backend that ever runs a big sort can climb to

the high-water mark and stay there. SET LOCAL confines the big allocation to

the one transaction that needs it.

One sanity check before you chase any of this — confirm how much of the 5GB

is actually reclaimable:

SELECT name, setting, unit FROM pg_settings

WHERE name IN ('shared_buffers','work_mem','maintenance_work_mem');

shared_buffers is allocated up front and its pages count toward reported

memory as they're touched. If shared_buffers is sized in the gigabytes, part

of that 5GB is normal cache warming, it is not a leak, and no amount of

trimming will or should release it. Only the delta above shared_buffers is

worth chasing.

The 130 MB SQL result does not establish that the remaining memory is filesystem cache. pg_backend_memory_contexts reports only the backend attached to the session running that query, not every Postgres process; it also isn't an OS resident-memory measurement. PostgreSQL reference

To choose a fix that avoids a server restart, could you capture the following inside the Postgres service container after a normal batch finishes, while the Railway memory graph is still high?

if [ -r /sys/fs/cgroup/memory.current ] &&
   [ -r /sys/fs/cgroup/memory.stat ]; then
  for f in memory.current memory.max memory.stat memory.events; do
    printf '\n%s\n' "$f"
    cat "/sys/fs/cgroup/$f"
  done
else
  printf 'cgroup v2 counters are unavailable at this mount\n'
fi

This only reads counters. If that mount is unavailable, report that rather than changing the container's permissions.

The useful distinction is anon versus file, with shmem considered separately: file includes shared memory/tmpfs, so it is not all disposable disk cache. The counters are in bytes. Linux cgroup reference

If the completed job's own database connection is then closed and anon falls materially, recycling that job connection is a useful way to release its private process memory without restarting the server. If filesystem cache dominates instead, disconnecting that backend does not flush the filesystem cache. I would not terminate every idle backend as a diagnostic; that disconnects unrelated clients too.

Also, the cgroup total is not proof of the exact value Railway bills. Matching these counters to the graph's timestamp would give Railway staff concrete evidence to explain its accounting. The existing information doesn't yet establish either allocator retention or cache billing as the cause.

#!/usr/bin/env bash

set -euo pipefail

URL="${DATABASE_URL:?}"

mem() { grep -E '^(anon|file) ' /sys/fs/cgroup/memory.stat | awk '{printf "%s=%.0fMB ",$1,$2/1048576}'; echo; }

echo "before: $(mem)"

psql "$URL" -At -c "SELECT pg_terminate_backend(pid) FROM pg_stat_activity

WHERE state='idle' AND pid<>pg_backend_pid();" >/dev/null

echo "after: $(mem)"

anon is real memory, file is page cache (harmless, reclaimable). Terminating idle backends returns their private anon memory to the OS — that part is fixable live. shared_buffers is not, and MALLOC_ARENA_MAX won't help because the leak is in Postgres's own memory contexts, not the C allocator.

However, Decompose the 5GB first. On cgroup v2 that's anon vs file in /sys/fs/cgroup/memory.stat. Millions of inserts leave a lot of dirty page cache (WAL + heap + index pages) counted as file. That is already reclaimed automatically under pressure and is not a leak. If most of your 5GB is file, there is nothing to fix.

If it's anon, it splits into two things:

• shared_buffers — allocated at postmaster start, cannot be shrunk without a restart on any released version. pg_resize_shared_buffers() is not real; that patch never landed. This part is a restart-only knob.
• Backend-private memory — memory contexts that grew during the burst and are cached for reuse by that backend. Reclaimable only by ending the backend (pg_terminate_backend, or a pooler in transaction pooling mode that recycles connections).

Postgres holds freed chunks in its own context freelists and rarely returns them to libc, so there is nothing for malloc to trim. Correct settings, wrong layer.

So for your actual situation — 32 vCPU / 32GB, burst peaks at 5GB, data growing over time — I'd say do nothing about the reclaim and instead cap the burst so it never climbs:

work_mem = '64MB' # per sort/hash node, not per session

max_parallel_workers_per_gather = 4hash_mem_multiplier = 1.0 # default 2, PG14+

hash_mem_multiplier is the one people miss on aggregations — a GROUP BY gets work_mem * 2 by default, per parallel worker.

Don't shrink the box. A 5GB high-water mark against 32GB with headroom for growth is fine, and the memory is reusable, not leaked. Restart only if you need the RAM for something else.

Also I should note that on Railway the container's own cgroup path isn't necessarily /sys/fs/cgroup/memory.stat — that's the root. Run it and if anon and file both read 0, you need the path from /proc/<postgres_pid>/cgroup.