5fd7c146-2bc8-4c17-ad18-4d0b59c44fa0

!t

Hoping that y'all have access to some other MongoDB interface from the other region.

My guess is that you have migration scripts when moving from one region to another, but since the whole thing was down, it failed so it gave me a blank database

is this effecting a production application with a userbase?

Yes

I had to migrate data from our staging environment to our production environment, as the production database was completely wiped

We're in an OK spot now, but we lost a lot of data

@Ray - production database wiped during region migration

For context, we host websites for Funeral Homes. So people's obituary pages were down for a good hour

Thankfully our obits are stored in a separate datastore so none of that actual data was lost

https://railway.app/project/5fd7c146-2bc8-4c17-ad18-4d0b59c44fa0/service/ae145910-e686-4262-85da-9ee64354c6c7

Is this the service?

Yes that's the service

I'll defer to you guys if there is a backup… or if our data is still technically available in the east region. Open to any solution as long as we can get that data back

To confirm, was this in the production environment and a migration from US East -> US West?

Is the volume name the same before and after you performed the migration?

Correct, this was in the production environment. I migrated from US east to US west (as a possible solution to US east being down)

I did not change the volume name at all (I don't think I can since there is no volume attached)

there is no volume attached…?

Replicas
The number of instances that will be run for each deployment of this service.
Replicas
1
Replication is only available for services that do not have a volume attached.

Unless I am misunderstanding the question… it doesn't look like we configured a volume

screenshot of the database service please

is the MongoDB Settings page sufficient?

no

Thanks for the info - going to investigate this now.

(No worries on the volume name, I only asked to make sure I'm looking at the correct thing)

thank you both

@Brody do you still want that screenshot?

there is indeed a volume on the mongo service

yeah that makes sense. i just don't see anywhere to name it

I know these things take time but just getting a sense for timing—is this something I should expect a response about tonight or should I expect something more realistically tomorrow

(I also need to know so I can have our team re-create any data that was lost)

@zackword sorry, was attempting to see if I can restore your data

Do you have anything in the database linked above currently?

Can you also give me some table names so I can confirm?

We have our staging data in the DB now, I migrated it just to restore customers

But I would prefer the last production from us-east if you do have that

Some table names- lol I think we everything is under the “test” database, and the tables would look like users, venues, pages

Gotcha. Is it alright if I created a new database in your project so I can restore your previous data from US East into it?

Absolutely

You should see a new Mongo service named "MongoDB-Restored"

https://railway.app/project/5fd7c146-2bc8-4c17-ad18-4d0b59c44fa0/service/439f896d-fefd-4946-830d-0f258383cf90

Can you check if the data is accurate?

I just checked the users table and that looks right

Thank you so much

Guessing the best way to switch is to redeploy our app with the new mongodb service

Is that what you’d recommend?

Yep, redeploy with MONGO_URI set to the new one.

Got it. Thank you so much

We’re going to flip the switch after doing another quick test

Super sorry about this, it should never happen. I've sent some credits your way (average of your past 3 month's bill) that will be used to offset your next bill

Please let me know if you see anything off 🙏

Thank you for the credits

I need to run but this is super helpful

Nearly had a heart attack

Me too 😅