Old replicas still running?
slokh
PROOP

2 years ago

Hi! I seem to have a weird issue where old replicas are not being cleaned up it looks like.

I can see logs for the old replicas in the observability tab but when looking at the actual service those logs dont appear.

Not sure how to propely get this cleaned up. It's problematic because a good portion of my jobs are executing on old code

19 Replies

slokh
PROOP

2 years ago

22ede547-c885-40df-9562-8007a7146611


slokh
PROOP

2 years ago

also doesnt look like theres anyway i can find to kill those old replicas


slokh
PROOP

2 years ago

Removing the service doesnt remove the old replicas 😦


slokh
PROOP

2 years ago

scaling down replicas also doesnt change anything, seems like the issue probably stems from these replicas being orphaned from the service


slokh
PROOP

2 years ago

oh ok i think i figured it out, failed deployments dont get auto removed


slokh
PROOP

2 years ago

UI feedback: show failed deployments that are still running please


brody
EMPLOYEE

2 years ago

the observability tab shows you logs from past and present, so aside from seeing the logs, what makes you think they are still running?


slokh
PROOP

2 years ago

im seeing live logs


slokh
PROOP

2 years ago

i changed the log statements and the old ones from ~2 days ago were still running now


slokh
PROOP

2 years ago

ive resolved it now thoughg


brody
EMPLOYEE

2 years ago

ah so some of but not all of the replicas failed


slokh
PROOP

2 years ago

this was the issue


slokh
PROOP

2 years ago

yep


slokh
PROOP

2 years ago

and the failed one then got buried in my deployment history


slokh
PROOP

2 years ago

so i didnt know it was running


slokh
PROOP

2 years ago

i actually wasnt even able to find it in the ui, i manually constructred the URL for the deployment and then hit remove


brody
EMPLOYEE

2 years ago

gotcha, good feedback! mind posting that here?


slokh
PROOP

2 years ago

yep!


brody
EMPLOYEE

2 years ago

thanks!


Loading...