https://railway.app/project/ce31b5f2-77b4-444c-9940-d6ef2bd33cc0

Your mongo instance definitely has stuff in it

There's 114GB of stuff there

It looks like youv'e removed all yourservices as well?

How soon will action be taken? We are losing our reputation and revenue because of your migration error. We are talking about tens of thousands of dollars and we would not want to be idle for so long. We are willing to go to a higher rate or something else to get you to restore the database

Hi, as I mentioned before, your data is there

Whatever you've done, it's on the application level

Can you tell me what error you're running into? What data is "missing"

Because you have ~200GB worth of stuff in here

We have a completely empty database. We manually looked at the volume and there is no data in it. Please rollback all changes to the project until September 28th

In the users collection (database name: test) we last saw 22K users, but the actual number of users is 14M+. Just roll it back 24 hours

Looking into what happened first

We were migrating our MongoDB database from US West to Singapore and during the migration we encountered a problem that broke all the data

Sorry, and now it's back in us west?

How did you manage that? We lock failed migrations for a reason here…

Did you make a new service and delete your old service? This Mongo instance had it's first deployment 15 hours ago

Now we're back in the US West. We have been removing MongoDB services and creating new ones by baiting them with old volume

Oh man

Can you just rollback the project as you have done before with other users?

We do not do this for other users

Railway does not manage backups for you

But we can see that our data is there, but where?

That's what we're looking into

At the time of the transfer, your dashboard showed that we were supposedly running out of space, even though we clearly had it

Also, if you want a higher level of service, we're definitely going to have to talk about that after

Having us respond and then you torch us on social media because you didn't respond here doesn't work for us

https://x.com/301Community/status/1840350767980450219

We will remove it if you really try to help us. We had no other options and your support was silent. Our app has been idle for over a day

We responded here and you did not

https://discord.com/channels/713503345364697088/1289695241716371579/1289738325510455327

We will talk about this after

after what?

Okay, we deleted the post.

After we fix your problem

We've found your volume and we're looking to roll it out. We will redeploy your instance and are working on getting it back up with the data

Please do not deploy it while we do this

Can you check if your data is restored? We think we tracked back down your original volume

Database Connection

We are unable to connect to the database over the public network.

Can you connect using this command?

mongosh "mongodb://mongo:@junction.proxy.rlwy.net:55268"

Gotchya looking into it

One more time please

I can confirm through our data UI that we see something

I want to make sure it is the correct data

i really love you bro

how can I ever thank you?

You can pay us what people normally pay us to wake us up in the middle of the night <:kekw:788259314607325204>

Starts at a 25k/mo minimum committed compute spend

(We will also fix that issue with the volume migration that messed you up, and for that, we're sorry)

https://discord.com/channels/713503345364697088/1289905951662673973/1289925191865991209

Hindsight 20/20, but how could you NOT schedule db backups? Also, I don't get the point of that (deleted) tweet, it screams attack vector & too lazy to do db backups