@Brody pls help! my app its in production

@Mahmoud @echohack please any help !?

Hey! I understand that the situation you're running into could be stressful, however please don’t direct @ mention folks from the Railway team. This can result in a ban

Now that that's out of the way. Do you mind sharing more details about the issue you're running into?

This morning I opened my Railway project and found that I cannot connect to my MySQL database. As you can see in the screenshot, the interface keeps loading.

All my production servers have stopped running because they cannot find the railway database. I haven’t touched the database in months.

This is the email account : norchenmezni@e-tafakna.com

this is my project Id : 521ea212-8af2-443a-adfe-233cfe4b2d8a

and im so sorry for the direct @ mention , im so stressed

Hmm, can you try redeploying?

Under deployments, you can click on the dot menu and choose Redeploy

yes but i was scared to do it , since u mention it to me , i will go a head and reach you out with update

Same issue even with redeploying the database

Any updates on my database issue , I’ve received a lot of complaints from users since the app is down, and I’m really hoping for guidance on next steps. Thanks in advance for any help and thanks

i am not sure if this is the right chammel to get helped with my problem unchielding my funds

Hey again, would appreciate if you can use a new #✋｜help thread for your query. Thanks!

Do you have any environment variables configured differently for the database service?

my environment variables are stored securely in the Railway environment configuration, and I haven’t made any changes to them. Everything has been the same for months, but the issue started suddenly this morning.

Interesting hmmm. And you said redeploying the database didnt fix it?

yes , it didnt fix it

can you share the db logs?

how are the metrics?

i will share screen shot

jsut pay attention to not share any secret or PII

yes dont worry

can you access the db using a client like tableplus (or similar)?

yes i can access via workbench since the db is mysql , but the schemas is empty

this is the first time that this issue happens

is it the right db? did you create a db with a different name by any chance? is the volume linked to the db? it is weird that the schema is empty….it is also weird that the disk decresed in size…

you can try also to connect via ssh in the service and try to connect to the db using the internal address to check if there are connection issues, but here it seems that something happened, otherwise in theory you should see something in the db

I didn’t create any new database or change the name. This is the same database I’ve been using for months, and everything was working fine until this morning. I also haven’t touched the volume configuration. That’s why I’m really concerned, since now the schemas are empty and the disk usage decreased suddenly

I think this need someone from the team to check, do you have backups?

can you also check the logs for the last day? maybe you can see if something happened when the db went down

maybe a migration went wrong?

i can confirm to you that 100% no one touch the database from my side , and its not shareable to any one , only me i do have the access to it

i do have backups but its for 1 month ago my db was already incremented by 2000 users already i cant get back to that backup

unfortunately I can't give you a lot of help on this, you might need to wait someone from the railway team, the only thing I can suggest is to carefully check the logs to see if you find some useful information

i already checked everything , thanks for you attention , i appreciate that

now i will wait the railway team

to help me

I hope you can understand my concern — both my mobile app and web app are completely down because of this issue, and I’m receiving a lot of complaints from users.

Hello @Fares,

I understand this is an extremely stressful situation. However, Railway's systems do not have any mechanism to delete user database schemas or data. Our platform cannot and does not automatically delete database schemas.

Since the "railway" schema deletion can only occur through user actions (via database commands, application code, or direct database access), and you are the only one with access to this project, the schema deletion must have originated from your account or someone with access to your database credentials.

Unfortunately, without recent backups, this data cannot be recovered. Our databases are unmanaged services, so data recovery depends entirely on user-maintained backups.

Going forward, I strongly recommend enabling Railway's native backup feature to prevent data loss from any future incidents.

Best,
Brody

Brody this is impossible

How this can happen no one has the environment variables

Of the database

And this action cannot be happen my application is in production and no one have access to my account

I understand this is frustrating, but Railway's infrastructure has no capability to delete user data or database schemas. The technical reality is that schema deletion requires explicit database commands or application code execution.

Since you have sole access to this project and Railway cannot delete user data, the deletion had to originate from your account access - whether through direct action, compromised credentials, or application code that executed DROP commands.

The data deletion occurred, and Railway was not the source. Without backups, the data cannot be recovered.

Brody im in a pro plan and i make sure of everything my application have more than 5000 users and its been 3 years now in railway so either way its hack to my account or something does happen in your containers

U may have snapshots to recover my database or ur container have mechanisms to recover old one

I cannot confirm whether your account was compromised, but I notice you don't have two-factor authentication enabled, which would help secure your account. You can check your logged-in devices in your account settings.

This schema deletion was not caused by Railway's containers or infrastructure. Our systems have no mechanism to delete user data.

Regarding recovery: we maintain disk-level snapshots, not individual database snapshots. However, these are not used to restore data when users delete it, as this falls outside our support scope for unmanaged databases. This policy applies regardless of plan tier or circumstances.

so brody there is no solution to recover my database ?

Since you didn't enable the automated volume backups, unfortunately not, sorry.

And this action cannot be happen my application is in production and no one have access to my account

it it possible that someone was able to achieve SQL injection against your production database? has your app undergone any security audits?

I’m suspecting the same. If the schema vanished without you touching it, a SQL injection exploit could have triggered destructive queries (like DROP or TRUNCATE). Might be worth checking recent http logs, app code that interacts with the DB, and whether parameterized queries are consistently used. Even if credentials weren’t leaked, injection via a vulnerable endpoint is another possible entry point.

can you check in your account => settings => session if there are session that you don't recognize?

i checked all the logs there is no sql injection

nope i didnt found a new session , but if some one gets there he can easly remove he is session since railway provide this action

it's infinitely more likely that your app was responsible for deleting your data than railway is

for prod critical setups you really do need to have a good backup strategy

the app ?

how

deleting the database its with command line wich Drop the data , the application is intracting with APIS and have no responsability with deleting the data

this is indeed a criminal thing