a month ago
Environment:
- Project: n8n (production)
- Services: Primary, Worker, PostgreSQL, Redis (based on n8n template)
- PostgreSQL Volume: geese-volume (contains production data)
Issue Summary:
Our n8n deployment has been experiencing critical PostgreSQL connection failures since attempting to resolve timeout issues. The PostgreSQL service now fails to authenticate with our existing data volume, causing a complete service outage.
Initial Problem:
- Error: "timeout exceeded when trying to connect"
- PostgreSQL checkpoint operations taking 228+ seconds
- n8n services showing "Error 503: Database is not ready"
Current Critical Issue:
After recreating PostgreSQL service, the existing volume (geese-volume) cannot authenticate with new service credentials: 2025-09-08 21:47:37.050 UTC [7266] FATAL: password authentication failed for user "railway" 2025-09-08 21:47:37.050 UTC [7266] DETAIL: Role "railway" does not exist.
Connection matched file "/var/lib/postgresql/data/pgdata/pg_hba.conf" line 128: "host all all all scram-sha-256"
Attempted Solutions:
1. Initial timeout fix attempt:
- Added PostgreSQL tuning variables (POSTGRES_MAX_CONNECTIONS, etc.) - caused deployment failure
- Removed all custom environment variables - PostgreSQL started but timeouts persisted
2. Service recreation attempts:
- Deleted and recreated PostgreSQL service
- Reattached the existing geese-volume at /var/lib/postgresql/data
- Result: Authentication failure - new service expects user "railway", old database doesn't have this user
3. Authentication bypass attempts:
- Added POSTGRES_HOST_AUTH_METHOD=trust - PostgreSQL fails to start entirely
4. Template redeployment:
- Deployed fresh n8n template
- Mounted existing geese-volume to new PostgreSQL service
- Same authentication mismatch issue persists
Critical Information:
- The geese-volume contains production data that must be preserved
- Original PostgreSQL service ID was: cb757d5e-2311-4f08-8f3a-7981ee22ab3c
- Volume mount shows in logs: /var/lib/containers/railwayapp/bind-mounts/[id]/
vol_amf28lpo4voq7qzc
- n8n services cannot start without database connection
Error Logs:
# PostgreSQL slow checkpoint (initial issue): 2025-09-08 17:17:51.447 UTC [73] LOG: checkpoint complete: wrote 66 buffers (0.4%); write=182.098 s, sync=228.824 s, total=697.970 s
# n8n timeout errors: Error: timeout exceeded when trying to connect at /usr/local/lib/node_modules/n8n/node_modules/pnpm/pg-pool@3.6.2_pg@8.12.0/node_modules/pg-pool/index.js:45:11
# Current authentication failure:
FATAL: password authentication failed for user "railway"
DETAIL: Role "railway" does not exist.
# n8n connection refused:
Initializing n8n process. There was an error initializing DB connect ECONNREFUSED fd12:549b:ec6:0:1000:6:92dc:b607:5432
Request:
We need assistance with:
1. Recovering access to the existing PostgreSQL data in geese-volume. The original Postgres modul was cb757d5e-2311-4f08-8f3a-7981ee22ab3c.
2. Understanding why PostgreSQL fails to start with POSTGRES_HOST_AUTH_METHOD=trust
3. Properly matching authentication credentials between the new PostgreSQL service and the existing database
4. Preventing future timeout issues once connection is restored
Impact:
The production n8n instance is completely down. All workflows and automation stopped. Critical business processes affected.
Thank you for your urgent assistance
5 Replies
a month ago
Hey there! We've found the following might help you get unblocked faster:
If you find the answer from one of these, please let us know by solving the thread!
a month ago
Team, this is very urgent for us. We're loosing trust of our customers. Would really appreciate your quick assistasnce.
a month ago
Hey team Railway, any updated on this?
a month ago
After recreating PostgreSQL service, the existing volume (geese-volume) cannot authenticate with new service credentials
Re-creating and re-attaching volumes do not work in this manner because the credentials would be different. You'll need to connect your volume back to the original Postgres service it was created from.
Status changed to Awaiting User Response Railway • 29 days ago
ray-chen
After recreating PostgreSQL service, the existing volume (geese-volume) cannot authenticate with new service credentialsRe-creating and re-attaching volumes do not work in this manner because the credentials would be different. You'll need to connect your volume back to the original Postgres service it was created from.
a month ago
I deleted the original Postgres service. is there a way to restore it?
Status changed to Awaiting Railway Response Railway • 29 days ago