Repeated Out of Memory crashes

9 months ago

Hi Team

Need help diagnosing what is causing the out of memory crashes on our API - Been pulling out our hair last 8 hours

Project: 7f34b4f7-9d09-48a8-997c-32a10e59c1eb

We have some errors from various functions but hard to pinpoint what is spiking the memory

29 Replies

9 months ago

Is this a template? Written by yourself? Language?


9 months ago

Besides the deployments logging is there anything else we can look at


9 months ago

Not a template, it's our API


9 months ago

It's an expressJS server


9 months ago

Did you make a specific change and then it started occuring?


9 months ago

we changed out our logger and it started happning around 7 hours ago


9 months ago

What is the memory spiking to? How do you know it's an OOM?


9 months ago

But we cannot determine anything further, everything points to us doing things more efficiently vs less efficiently


9 months ago

repeated Railway OOM errors and crashes


9 months ago

What's the limit you have set


9 months ago

4vCPU and 6GB memory


9 months ago

2 replicas


9 months ago

Anything that we can do to get more detailed information on Railway?


9 months ago

(thanks for the prompt reply btw)


9 months ago

Railway won't profile your application


9 months ago

I'm not sure how I would debug this on Javascript


9 months ago

yeah


9 months ago

Is this app open source?


9 months ago

no private API sorry i guess there isn't anything else we can get from railway and need to look at DD etc


9 months ago

Yeah it's not really Railway's responsibility here


9 months ago

I agree


9 months ago

Thanks for trying to help


9 months ago

Check for any added dependencies added to your project around the time the issue started happening


9 months ago

Run the Dockerfile locally to see if it's happening there too



8 months ago

It’s a super hard problem because we think the errors were happening before but the OOM started after a huge logger refactor

1414623786946531519


8 months ago

Going to try out the heap snapshot thanks @astrid(irl)*


8 months ago

That's what you get for blindly trusing AI...


8 months ago

Can also git bisect


Welcome!

Sign in to your Railway account to join the conversation.

Loading...