recovery request for removed Label Studio deployment
yoojaeshin
HOBBYOP

a month ago

Hello Railway Support,

I urgently need help recovering the ephemeral filesystem from a recently removed Label Studio deployment. The data was still accessible only hours before a new deployment was created after my trial ended and I upgraded to the Hobby plan.

Project ID: 20424a85-6ae3-43c6-abf1-ae8e946dcc7e

Environment ID: 97ac8e1a-474c-4a78-a461-cfdf3e8fa166

Service ID: f79ef7b4-1783-4504-a317-6051db3eb673

Removed deployment that contained the data:

e78300a3-15b6-417b-a5a5-fe73753706d2

New empty deployment:

d7dcd666-91cf-4514-b9e0-6fd6da447fdd

Evidence from Railway deploy logs:

  • The removed deployment was still serving Label Studio project 3, tasks, annotations, and uploaded image files until July 11, 2026 around 01:12 KST.
  • POST /api/tasks/5927/annotations returned 201 at approximately 00:53 KST.
  • Uploaded files under /data/upload/3/ were returning 200.
  • The new deployment started at July 11, 2026 12:08 KST.
  • The new deployment now returns 404 for /api/projects/3 and the project list is empty.

There was no persistent volume attached, so the SQLite database and uploads appear to have remained on the old deployment's ephemeral filesystem. Since the old deployment was removed only recently, could you please urgently check whether its filesystem can still be recovered or snapshotted, especially /label-studio/data?

Please do not permanently purge the old deployment storage while this request is being investigated. This data contains a team labeling project's tasks, annotations, and uploaded images.

Thank you.

Solved

3 Replies

Railway
BOT

a month ago

Your label-studio service has no persistent volume attached, so all data (the SQLite database, uploads, annotations) was stored on the deployment's ephemeral filesystem. Ephemeral storage is discarded when a deployment is replaced, and that data is not recoverable. Going forward, you can attach a volume to the service so that writes to disk persist across deployments, and optionally enable volume backups for an additional safety net.


Status changed to Awaiting User Response Railway about 1 month ago


Railway

Your label-studio service has no persistent volume attached, so all data (the SQLite database, uploads, annotations) was stored on the deployment's ephemeral filesystem. Ephemeral storage is discarded when a deployment is replaced, and that data is not recoverable. Going forward, you can [attach a volume](https://docs.railway.com/volumes) to the service so that writes to disk persist across deployments, and optionally enable volume backups for an additional safety net.

yoojaeshin
HOBBYOP

a month ago

Thank you for the automated response. I understand that ephemeral storage is not persistent and that recovery is not guaranteed.

However, this specific deployment was replaced only today, and the Railway deployment logs confirm that it was still serving the SQLite-backed Label Studio project, annotations, and uploaded files until approximately 01:12 KST.

I am not asking about the general ephemeral-storage policy. Could a Railway staff member please urgently verify whether the removed deployment's host filesystem, container disk, or any pre-removal snapshot/cache still exists before garbage collection? Please check specifically for removed deployment e78300a3-15b6-417b-a5a5-fe73753706d2 and confirm whether any best-effort recovery or snapshot of /label-studio/data is technically possible.

Please avoid purging any remaining storage associated with that deployment while this is being checked. I would appreciate confirmation from a human Railway engineer.


Status changed to Awaiting Railway Response Railway about 1 month ago


Railway
BOT

a month ago

We've confirmed that the label-studio service has no persistent volume attached, so all data lived on the deployment's ephemeral filesystem. Ephemeral storage is tied to the lifecycle of each deployment and is not retained after a deployment is replaced, regardless of how recently the replacement occurred. There is no recovery path available for this data. For future protection, you can attach a volume to the service so data persists across deployments, and optionally enable volume backups.


Status changed to Awaiting User Response Railway about 1 month ago


Railway
BOT

a month ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway about 1 month ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...