6 days ago
Error: s6-applyuidgid: fatal: unable to exec hermes: No such file or directory
The service uses the Docker image nousresearch/hermes-agent and has a persistent volume mounted at /opt/data.
I need help recovering service access and verifying/backing up the persistent volume BEFORE any further deployment or modification.
Please do not delete, detach, replace, or reset the volume.
Can you help me inspect the failed container's executable path and recover the service while preserving all stored files?
4 Replies
6 days ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • 6 days ago
6 days ago
Try selecting your service, press Cmd/Ctrl + K, then deploy source image.
12 hours ago
Please preserve the /opt/data volume before redeploying. That is the official Hermes data directory; the application and virtual environment live separately under /opt/hermes. This launch error does not itself establish that your stored data is damaged.
There is a related historical PATH issue: NousResearch/hermes-agent#18673, closed June 1, reports that the hermes executable was not discoverable in some Docker execution contexts. The current Dockerfile explicitly exports /opt/hermes/bin and /opt/hermes/.venv/bin on PATH. This is a useful lead, not proof that your deployed image has that exact defect: ENOENT can also mean a missing launcher interpreter or broken symlink.
Before changing the service:
- Record the currently deployed image tag/digest, Start Command and any user-defined PATH override. Do not publish environment dumps or .env contents.
- Use Railway's volume-file tooling to inspect/download the existing volume without running Hermes: select the correct project/environment and volume, then
railway volume files --volume <volume-id> list /andrailway volume files --volume <volume-id> download / ./hermes-volume-backup. Keep that backup encrypted and private; it can contain API keys and session data. If access is unavailable while crashed, ask Railway for a supported recovery/export path before any deployment change. - With the same image digest, inspect the launcher and its interpreter separately from the production volume. If the launcher exists but is absent from PATH, correct the deployment's PATH override or use a release verified to include the upstream fix after preservation. If the launcher/interpreter is missing from the image, PATH alone will not fix it.
Keep the image's supported entrypoint/s6 startup flow. Railway Start Command overrides can replace Docker ENTRYPOINT, so simply pasting an absolute hermes command into that field may create a different startup problem. Do not run a setup/reset command, reinstall into /opt/data, or start a second gateway against it.
Can you share only the image tag/digest, Start Command and whether a custom PATH is set? No secrets or backup files are needed.
Sources:
https://github.com/NousResearch/hermes-agent/issues/18673
https://github.com/NousResearch/hermes-agent/blob/main/Dockerfile
https://hermes-agent.nousresearch.com/docs/user-guide/docker
https://docs.railway.com/cli/volume
https://docs.railway.com/deployments/start-command
AI-assisted, checked against the linked sources. I have not accessed or tested your deployed container.
4 hours ago
The s6-applyuidgid failure indicates the supervisor cannot locate the hermes binary on the execution path when dropping privileges. This is typically triggered by a custom Start Command override that bypasses the image's built-in entrypoint dispatch script, or a mismatched PATH in the container environment.
To recover service access safely without risking your persistent volume at /opt/data:
Check your service settings and clear any custom Start Command override so the container uses the image's default entrypoint-dispatch script.
Ensure no custom PATH override is stripping /opt/hermes/bin or /opt/hermes/.venv/bin from the execution environment.
Trigger a clean redeploy of the exact image tag/digest to let s6 initialize the process cleanly against your existing mounted volume.
AI-assisted; checked against Railway documentation and settings.
an hour ago
AI-assisted, source-checked. One backup-first caution: current upstream Hermes startup is not read-only. Its stage2 hook changes ownership/permissions in the state directory and syncs skills before normal startup. Clearing Start Command and redeploying should not be treated as risk-free before your requested backup is verified. This is a preservation warning, not a confirmed diagnosis of your image.