a month ago
Issue
Envoy gateway is unable to connect to internal PostgREST and Gotrue Auth services via Railway's private network, despite correct variable resolution and DNS configuration.
Diagnostic Summary
All variables resolve correctly to .railway.internal hostnames, but TCP connections from Envoy to other internal services timeout consistently.
Symptoms
-
Envoy service: Online, receiving requests correctly
- GET / returns 401 (Studio basic auth, working)
- GET /rest/v1/ returns 403 (PostgREST RBAC, working)
- GET /auth/v1/health returns 503 upstream timeout (BROKEN)
- GET /rest/v1/comercios returns 503 upstream timeout (BROKEN)
-
PostgREST service: Online, 1/1 running
-
Gotrue Auth service: Online, 1/1 running
Root Cause Evidence
-
Variables resolve correctly to
.railway.internal:REST_HOST=postgrest.railway.internal
AUTH_HOST=gotrue-auth.railway.internal
-
Envoy configuration is correct with proper cluster addresses
-
DNS resolver configured: nameserver fd12::10
-
Problem is network connectivity, not configuration:
- Envoy receives requests (401/403 responses work)
- Envoy resolves hostnames correctly
- TCP connections to other services timeout
- Affects ALL upstream services
- Error: "upstream connect error... reset reason: connection timeout"
Details
- Project: giving-empathy
- Environment: production
- Region: sfo
Conclusion
This appears to be a private network connectivity issue at the Railway infrastructure level. All services are properly configured and hostnames resolve correctly, but TCP connections timeout.
1 Replies
a month ago
Apologies for the trouble. We had an incident affecting private networking in US West that has since been resolved, which likely caused the upstream connection timeouts your Envoy service was seeing. Please retry your requests and confirm whether connectivity to your internal services has recovered. You can see the full timeline here: status.railway.com/incident/Z5Y3WO06.
Status changed to Awaiting User Response Railway • about 1 month ago
a month ago
This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!
Status changed to Solved Railway • about 1 month ago