a month ago
Hello Railway Support,
I urgently need help recovering the ephemeral filesystem from a recently removed Label Studio deployment. The data was still accessible only hours before a new deployment was created after my trial ended and I upgraded to the Hobby plan.
Project ID: 20424a85-6ae3-43c6-abf1-ae8e946dcc7e
Environment ID: 97ac8e1a-474c-4a78-a461-cfdf3e8fa166
Service ID: f79ef7b4-1783-4504-a317-6051db3eb673
Removed deployment that contained the data:
e78300a3-15b6-417b-a5a5-fe73753706d2
New empty deployment:
d7dcd666-91cf-4514-b9e0-6fd6da447fdd
Evidence from Railway deploy logs:
- The removed deployment was still serving Label Studio project 3, tasks, annotations, and uploaded image files until July 11, 2026 around 01:12 KST.
- POST /api/tasks/5927/annotations returned 201 at approximately 00:53 KST.
- Uploaded files under /data/upload/3/ were returning 200.
- The new deployment started at July 11, 2026 12:08 KST.
- The new deployment now returns 404 for /api/projects/3 and the project list is empty.
There was no persistent volume attached, so the SQLite database and uploads appear to have remained on the old deployment's ephemeral filesystem. Since the old deployment was removed only recently, could you please urgently check whether its filesystem can still be recovered or snapshotted, especially /label-studio/data?
Please do not permanently purge the old deployment storage while this request is being investigated. This data contains a team labeling project's tasks, annotations, and uploaded images.
Thank you.
3 Replies
a month ago
Your label-studio service has no persistent volume attached, so all data (the SQLite database, uploads, annotations) was stored on the deployment's ephemeral filesystem. Ephemeral storage is discarded when a deployment is replaced, and that data is not recoverable. Going forward, you can attach a volume to the service so that writes to disk persist across deployments, and optionally enable volume backups for an additional safety net.
Status changed to Awaiting User Response Railway • about 1 month ago
Railway
Your label-studio service has no persistent volume attached, so all data (the SQLite database, uploads, annotations) was stored on the deployment's ephemeral filesystem. Ephemeral storage is discarded when a deployment is replaced, and that data is not recoverable. Going forward, you can [attach a volume](https://docs.railway.com/volumes) to the service so that writes to disk persist across deployments, and optionally enable volume backups for an additional safety net.
a month ago
Thank you for the automated response. I understand that ephemeral storage is not persistent and that recovery is not guaranteed.
However, this specific deployment was replaced only today, and the Railway deployment logs confirm that it was still serving the SQLite-backed Label Studio project, annotations, and uploaded files until approximately 01:12 KST.
I am not asking about the general ephemeral-storage policy. Could a Railway staff member please urgently verify whether the removed deployment's host filesystem, container disk, or any pre-removal snapshot/cache still exists before garbage collection? Please check specifically for removed deployment e78300a3-15b6-417b-a5a5-fe73753706d2 and confirm whether any best-effort recovery or snapshot of /label-studio/data is technically possible.
Please avoid purging any remaining storage associated with that deployment while this is being checked. I would appreciate confirmation from a human Railway engineer.
Status changed to Awaiting Railway Response Railway • about 1 month ago
a month ago
We've confirmed that the label-studio service has no persistent volume attached, so all data lived on the deployment's ephemeral filesystem. Ephemeral storage is tied to the lifecycle of each deployment and is not retained after a deployment is replaced, regardless of how recently the replacement occurred. There is no recovery path available for this data. For future protection, you can attach a volume to the service so data persists across deployments, and optionally enable volume backups.
Status changed to Awaiting User Response Railway • about 1 month ago
a month ago
This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!
Status changed to Solved Railway • about 1 month ago