High usages with new builder

2 years ago

Hi,

I was trying to move two of my services to the new builder and that was a wild ride, follow me.

I toggled the "new builder" option + new runtime "v2".
I deployed, and went on with my life.
Checked ~20 mins later and all the usage went to the roof.

The two services I toggled the new builder are cron/worker services, the memory usage has been around ~150/200MB for many months. After deploying with the new builder that what's I got (first image), that doesn't look normal. To comparison, second image is the "normal" usage before trying the new builder.

I, of course, disabled the new builder again, and usage went back to normal. I did keep runtime "v2" on because that part is good.

Is it a display/metrics issue, or am I doing something wrong to get that crazy usage?

I'm deploying docker images, with node-alpine base.

34849d22-d685-4e73-8858-fbd4fe42ea65

Closed

28 Replies

alexng353

PRO

2 years ago

very strange

alexng353

PRO

2 years ago

@Brody @Adam yoink

alexng353

PRO

2 years ago

seems related to <#1261357978469269544>

alexng353

PRO

2 years ago

When I have time I'm going to test this with the old builder and see if my reporting reduces

brody

EMPLOYEE

2 years ago

okay thank you, I'll bring this up to the team monday

2 years ago

2 years ago

This thread has been escalated to the Railway team.

Status changed to Awaiting Railway Response brody • over 1 year ago

angelo-railway

EMPLOYEE

2 years ago

Hey there,

So a few things:

1. Builds don't count towards your usage on the platform.
2. The result of the image on the new runtime results may or may not be respecting cgroup rules. Raised this to our build engineer to run an investigation.

Will keep you in the loop, keep in mind Christian is going on call, so you will see him engage in the thread as time goes on.

Status changed to Awaiting User Response Railway • over 1 year ago

2 years ago

2 years ago

Any updates on this?

brody

EMPLOYEE

2 years ago

hey, @alex and @jeremy - a fix was merged for this, let me know if you experience these issues again!

alexng353

PRO

2 years ago

@flatypus

flatypus

PRO

2 years ago

that's so real

flatypus

PRO

2 years ago

my app should not be using 32GB of ram

flatypus

PRO

2 years ago

it does like nothing

alexng353

PRO

2 years ago

lol

flatypus

PRO

2 years ago

@alex seems like for me the memory is garbage collected by python, but memory isn't freed to the OS (python apparently operates on some abstraction layer, i'm not quite confident here)

1267590648664555500