?????? Our app went down for 2 hours when we were sleeping because or auto upgrade
geongeorge
PROOP

9 months ago

Our service was auto migrated to metal and broke our app. We were sleeping and a lof of our users were pissed.

15 Replies

geongeorge
PROOP

9 months ago

6d0f799e-be59-4388-899d-f00456f30667



brody
EMPLOYEE

9 months ago

Hello,

We are really sorry about this, it is terrible timing on our part, we started a migration to Metal and and the same time the Asia networking had some issues on a subset of hosts.

You can see our incident for this here -


geongeorge
PROOP

9 months ago

I understand. Our app is a financing app in India and this is a shopify app. People leave reviews that affect us. I was so pissed. The reason why I didn't upgrade myself to metal was because you said stateful servers were still in beta.

Can I opt out of auto migration?


brody
EMPLOYEE

9 months ago

The application you linked is a stateless service, those have been GA for quite some time.

This issue was not specific to Metal workloads, it was specific to the Metal Edge.

Im terribly sorry that we migrated you at such an inopportune time, but our migration to Metal must press on.


brody
EMPLOYEE

9 months ago

Update here, we have resolved the incident.

Again, very sorry for the inconvenience caused.


geongeorge
PROOP

9 months ago

Thank you! I understand. Today was a holiday here and I missed a lot of emails. Can you tell me which all services you will be migrating?

1367434672619131000


geongeorge
PROOP

9 months ago

I know stateless is GA. but the networking between metal and GCP had latency last time we talked


geongeorge
PROOP

9 months ago

Because Postgress is stateful and I didn't want the slowdown


geongeorge
PROOP

9 months ago

hi


brody
EMPLOYEE

9 months ago

Hello,

Going to check with the team to see if we can get you this data.


Hey there KiBender,

Stateful servers have long been out of beta for around 2 months, which we messaged multiple times on the Changelog. With that said, I am very sensitive to your business needs. I remember, and I apologize.

Honestly, the best thing we can do is cut over all the services to Metal ASAP. With that said, we can try to customize your migration date.


geongeorge
PROOP

9 months ago

Thanks I will try to migrate as much as I can.

Wondering if I migrate a postgres db, should I expect data to be moved as well?


Yep- so that that was the functionality that was missing two months ago.


We’re good there


Loading...