Repeated Out of Memory crashes

6 months ago

Hi Team

Need help diagnosing what is causing the out of memory crashes on our API - Been pulling out our hair last 8 hours

Project: 7f34b4f7-9d09-48a8-997c-32a10e59c1eb

We have some errors from various functions but hard to pinpoint what is spiking the memory

29 Replies

6 months ago

Is this a template? Written by yourself? Language?


6 months ago

Besides the deployments logging is there anything else we can look at


6 months ago

Not a template, it's our API


6 months ago

It's an expressJS server


6 months ago

Did you make a specific change and then it started occuring?


6 months ago

we changed out our logger and it started happning around 7 hours ago


6 months ago

What is the memory spiking to? How do you know it's an OOM?


6 months ago

But we cannot determine anything further, everything points to us doing things more efficiently vs less efficiently


6 months ago

repeated Railway OOM errors and crashes


6 months ago

What's the limit you have set


6 months ago

4vCPU and 6GB memory


6 months ago

2 replicas


6 months ago

Anything that we can do to get more detailed information on Railway?


6 months ago

(thanks for the prompt reply btw)


6 months ago

Railway won't profile your application


6 months ago

I'm not sure how I would debug this on Javascript


6 months ago

yeah


6 months ago

Is this app open source?


6 months ago

no private API sorry i guess there isn't anything else we can get from railway and need to look at DD etc


6 months ago

Yeah it's not really Railway's responsibility here


6 months ago

I agree


6 months ago

Thanks for trying to help


Check for any added dependencies added to your project around the time the issue started happening


Run the Dockerfile locally to see if it's happening there too



6 months ago

It’s a super hard problem because we think the errors were happening before but the OOM started after a huge logger refactor

1414623786946531600


6 months ago

Going to try out the heap snapshot thanks @astrid(irl)*


6 months ago

That's what you get for blindly trusing AI…


Can also git bisect


Loading...