a month ago
Hi Railway Support,
Our self-hosted n8n deployment has crashed due to a JavaScript heap out-of-memory error.
Project: friendly-presence
Environment: production
Service: n8n
Error: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed — JavaScript heap out of memory
PostgreSQL is still online, and our n8n and PostgreSQL persistent volumes remain attached.
Our current configuration includes:
n8n memory allocation: 1 GB
N8N_USER_FOLDER: /home/node
n8n volume mount: /home/node/.n8n
No explicit N8N_ENCRYPTION_KEY environment variable
No available backups shown in Railway
We are particularly concerned about preserving our existing n8n encryption key, stored credentials and workflow data.
Could you please advise:
How we can safely back up or inspect the existing n8n volume and PostgreSQL database while n8n is crashed.
Whether the current N8N_USER_FOLDER configuration places the encryption key outside the persistent volume.
The safest recovery procedure without resetting credentials, wiping storage or losing workflow data.
Whether increasing the Node.js heap limit is appropriate for our current 1 GB service allocation.
We have not deleted, wiped or detached any volumes.
Please advise on data preservation before we attempt recovery.
Thank you.
Pinned Solution
a month ago
Thank you so much for your recovery instructions!
We've installed Railway CLI 5.70.0 on a Chromebook, authenticated successfully, and linked to the correct project (friendly-presence) and production environment.
Both volumes report Ready. We've generated and registered an ED25519 SSH key and successfully loaded it into the local SSH agent.
However, railway volume files list / against n8n-volume now fails with:
Failed to initialize SFTP session — Timeout
Previously, we received an SSH authentication error, but that has changed after registering the key.
The n8n service remains crashed, while PostgreSQL is online.
Is this SFTP timeout related to the crashed n8n deployment, or is there another supported way to download the persistent volume without restarting or modifying the service?
Our priority remains preserving the original n8n configuration and encryption key before attempting recovery.
4 Replies
a month ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • about 1 month ago
a month ago
Your N8N_USER_FOLDER setting does not, by itself, put the key outside the volume. n8n appends .n8n to that path, so /home/node resolves to /home/node/.n8n. The expected settings file is /home/node/.n8n/config. Keep the current path, image version and database settings unchanged while preserving the data.
I would recover in this order:
-
Preserve the n8n volume first. Railway now documents volume-file browsing/downloads independently of the n8n application. From a trusted local machine, select the correct project, production environment and n8n volume, then use
railway volume files --volume <n8n-volume-id> list /andrailway volume files --volume <n8n-volume-id> download / ./n8n-volume-backup. Confirm the downloaded config exists and privately contains an encryptionKey; do not paste its contents here. Keep the entire backup private and encrypted. If volume-file access is unavailable for this crashed deployment, ask Railway to provide a supported recovery path before redeploying, detaching or replacing anything. -
Back up PostgreSQL separately. The n8n volume does not contain a PostgreSQL-hosted workflow database. Use pg_dump of the same major version as the server, or a compatible newer client, through an existing authorized connection. Save a custom-format dump privately; inspect it with pg_restore --list and ideally restore into an isolated test database. Do not copy a running PostgreSQL data directory as an ordinary file backup or restore over production.
-
Keep the original encryption key. Do not generate a replacement N8N_ENCRYPTION_KEY or reset credentials. Recover using the backed-up config and original database together; confirm an existing credential still decrypts before resuming workflows. Missing key recovery is different from memory recovery.
-
Address memory after preservation. --max-old-space-size controls only V8 old-space, not total container memory. A 1024/2048 MiB heap is inappropriate for a 1 GB container. Check existing NODE_OPTIONS and memory metrics first, then reduce workflow batches/concurrency or increase the service's actual memory if needed. A heap setting alone does not add RAM or guarantee recovery.
Could you confirm the n8n image tag, PostgreSQL major version, and whether the config file is present? Please share no keys, connection URLs or credential files.
References:
https://docs.railway.com/cli/volume
https://docs.railway.com/guides/postgres-backups-restores
https://nodejs.org/api/cli.html#--max-old-space-sizesize-in-mib
AI-assisted, checked against the linked documentation. I have not accessed or tested your production deployment.
a month ago
Thank you so much for your recovery instructions!
We've installed Railway CLI 5.70.0 on a Chromebook, authenticated successfully, and linked to the correct project (friendly-presence) and production environment.
Both volumes report Ready. We've generated and registered an ED25519 SSH key and successfully loaded it into the local SSH agent.
However, railway volume files list / against n8n-volume now fails with:
Failed to initialize SFTP session — Timeout
Previously, we received an SSH authentication error, but that has changed after registering the key.
The n8n service remains crashed, while PostgreSQL is online.
Is this SFTP timeout related to the crashed n8n deployment, or is there another supported way to download the persistent volume without restarting or modifying the service?
Our priority remains preserving the original n8n configuration and encryption key before attempting recovery.
a month ago
Thanks for the update. Yes—the crashed n8n deployment is a likely explanation, although the timeout alone cannot identify the exact backend failure.
I should correct the wording in my earlier reply: the volume CLI is independent of the n8n application protocol, but it is not an offline volume-export service. Railway file transfers still reach the filesystem through the attached service's running container. A volume being Ready does not establish that its service has a running instance.
I checked CLI 5.70.0: its volume SFTP implementation authenticates first, requests the SFTP subsystem, then initializes the SFTP session. Your current error comes from that last initialization step. Registering the key appears to have cleared the earlier authentication blocker; generating more keys would not address the missing running-container requirement.
Under your requirement to keep n8n stopped and leave its service/volume unchanged, I do not have a documented direct-download alternative to recommend. Railway's moderator answered this exact stopped-instance question here:
https://station.railway.com/questions/how-to-safely-back-up-volume-files-while-dbc90db4
For now:
- Keep the existing volume, mount path, image and encryption settings unchanged. Do not create a replacement N8N_ENCRYPTION_KEY, restore over the volume, or deploy a file-browser/helper as if it were a read-only operation.
- Ask Railway support whether your current plan permits a storage-level backup of this Ready volume while n8n remains crashed, without restarting/redeploying n8n. Ask them to confirm the procedure and any charges before approving it. A platform backup preserves a recovery point; it is not a downloadable archive. Restoring one stages a volume replacement and deploying it redeploys the service, so backup creation and restoration must be treated separately.
- Preserve PostgreSQL separately through an existing authorized logical-backup connection. PostgreSQL being online does not expose the n8n volume or its key.
The key file we need to preserve is still expected at /home/node/.n8n/config for your stated configuration; its actual presence remains unverified. Please keep its contents private.
References:
https://docs.railway.com/cli/ssh
https://github.com/railwayapp/cli/blob/v5.70.0/src/commands/volume/sftp.rs#L280-L308
https://docs.railway.com/volumes/backups
AI-assisted; checked against the sources above. I have not accessed your deployment or performed a recovery.
a month ago
Hi, just wanted to come back and say a big thank you for your help!
We managed to restart our n8n service successfully, access the persistent volume, recover our original encryption key configuration, and secure encrypted off-platform backups.
Your explanation about volume access requiring a running container was particularly helpful.
Really appreciate you taking the time to assist. I'll mark your response as the solution so you receive the credit you deserve.
Thanks again!
Status changed to Solved snapaarch • about 1 month ago