Memory usage graph doesn't go down after intial spike while testing for load
shubham-complyance
PROOP

2 months ago

We have several services running right now, these are written in java, go and bun as well.

Apart from the bun known memory leakage problem, other services written in go and java, when tested under load spikes the memory usage, but after the initial memory usage spike the memory usage graph doesn't come down and stays there at peak for days

$10 Bounty

3 Replies

Railway
BOT

2 months ago

This thread has been marked as public for community involvement, as it does not contain any sensitive or personal information. Any further activity in this thread will be visible to everyone.

Status changed to Open Railway 2 months ago


ilyass012
FREE

2 months ago

this is an expected behavior from both go and java runtimes, not a real memory leak and not a railway issue

for go; the runtime uses MADV_FREE on linux, which means even after the garbage collector frees memory internally, the os won't reclaim it unless it's under pressure, so railway's graph stays high. fix this by setting the GOMEMLIMIT environment variable in your railway service to around 90% of your container's memory limit, this tells the go runtime to be more aggressive about returning memory


ilyass012
FREE

2 months ago

for java , the jvm frees objects during gc but keeps that memory inside its own heap instead of giving it back to the os, because releasing to the os after every deallocation is expensive. fix this by adding -XX:SoftMaxHeapSize to your jvm flags set it lower than your -Xmx value, and the jvm will shrink back down towards it during idle periods. if you're on java 15+ you can also switch to zgc with -XX:+UseZGC which is much better at returning memory to the os after spikes


ilyass012
FREE

2 months ago

neither srv is actually leaking, they just need to be told to hand memory back more aggressively


Welcome!

Sign in to your Railway account to join the conversation.

Loading...