3 months ago
Hi Railway Support team,
I’m looking for guidance on securely using sealed variables in production. We’re trying to avoid plaintext secrets, but during production bootstrap we hit several failures that seem related to how sealed/reference vars are resolved.
### Context
- We created production by using Railway’s “Sync changes from staging” UI action.
- This cloned services/config from staging into production.
- We then updated production-specific vars and attempted to seal sensitive values (DATABASE_URL, MIGRATIONS_DATABASE_URL, storage keys, etc.).
### Failure modes we encountered
1. backend-api / backend-worker startup crash when DB vars were sealed:
- TypeError: Cannot read properties of undefined (reading 'searchParams')
- Stack pointed to pg-connection-string parse path.
- This looked like DATABASE_URL was undefined/empty at runtime.
2. Subsequent runs showed DB fallback behavior:
- ECONNREFUSED ::1:55432 and 127.0.0.1:55432
- This suggests app defaulted to local DB when env var did not resolve.
3. Redis resolution issue:
- getaddrinfo ENOTFOUND $
- We saw this when using templated/interpolated Redis URL values.
4. Practical pattern observed:
- Unsealed literal values often worked immediately.
- Equivalent sealed/reference-based values were inconsistent during bootstrap until we manually iterated vars/redeploys.
5. KEY QUESTION: do sealed shared variables work?
- This would be our preference
- But in most scenarios (database DSNs, redis addresses) it didn't work.
- Sealed shared variables did work with external API keys (example, our OpenAI provider key)
### What we need from you
Could you confirm the canonical and supported pattern for secret management in this setup, specifically:
1. Is it fully supported to use sealed vars whose value is a Railway reference, e.g.
DOCUMENT_INGESTION_STORAGE_ACCESS_KEY_ID=${{bucket.ACCESS_KEY_ID}}
MIGRATIONS_DATABASE_URL=postgresql://${{postgres.POSTGRES_USER}}:${{postgres.POSTGRES_PASSWORD}}@...
2. Are sealed refs resolved at deploy time or runtime?
Any known caveats during first deploys, especially right after “sync from staging”?
3. Are there known issues where sealed vars can appear empty/unresolved until re-saved/redeployed?
4. What is the recommended migration sequence to move from working unsealed vars to sealed vars without downtime?
5. Is there a safe way to validate the final resolved value (or at least resolution status) without exposing secrets?
6. Is there a way to make sealed shared variables work with redis urls and database DSNs, or is there another pattern that is both convenient and secure we should be using?
We now have the system mostly stable, but we want to lock everything down properly and avoid another regression. Any concrete guidance or checklist would be greatly appreciated.
Thank you! We love the platform otherwise. It's amazing.
Andrew
3 Replies
3 months ago
This thread has been marked as public for community involvement, as it does not contain any sensitive or personal information. Any further activity in this thread will be visible to everyone.
Status changed to Open Railway • 3 months ago
3 months ago
1.Yes, reference vars using ${{ ... }} are supported
2.Vars are injected into the build + the running deployment, and changes are staged until you deploy.
4.Add new sealed/shared vars -> deploy -> switch app to use them -> deploy -> remove old plaintext vars ->deploy.
5.You can’t retrieve sealed values via API/CLI...
6.Seal the secret parts like pass/keys in shared variables, and keep the URL as an unsealed ${{ ... }} reference template
check here https://docs.railway.com/variables
3 months ago
Thanks for the fast response dharmateja, but I don't think it actually answers the core of my question. I understand that's how it's supposed to work, but our experience said otherwise. Using sealed variables that were shared or had reference vars routinely failed (not 100% of the time, but about 90% of the time) and it was very difficult to understand the pattern. Clearly there is somethign that makes the combination of sealed reference vars or sealed shared vars is very unstable. Is there an authoritative guide to the best practices on this? The documentation isn't specific enough.
3 months ago
Oh, maybe I missed the answer, but is this "Seal the secret parts like pass/keys in shared variables, and keep the URL as an unsealed ${{ ... }} reference template
check here https://docs.railway.com/variables" how you recommend handling DSNs etc?