19 days ago
Issue Type: Environment-Level DNS Failure
Quick Summary
All services in my Railway production environment cannot resolve external DNS hostnames. This is blocking all database connections to my AWS RDS production database.
Project Information
- Project ID:
e0b9be02-48a4-41cb-997c-b8884557058b - Environment ID:
0f66ea12-7899-4cf8-b0cf-a9fe1d46dd27 - Environment Name: production
- Dashboard:https://railway.com/project/e0b9be02-48a4-41cb-997c-b8884557058b?environmentId=0f66ea12-7899-4cf8-b0cf-a9fe1d46dd27
The Problem
ALL services in this environment fail DNS resolution for external hostnames.
Error message:
socket.getaddrinfo(): [Errno -2] Name or service not known
Target hostname that fails:
opsyn-db-prod.cwj08oy8u09z.us-east-1.rds.amazonaws.com
Evidence This Is a Platform Issue (Not Code/Config)
- AWS RDS is accessible - confirmed from outside Railway
- DATABASE_URL is correct - validated and working elsewhere
- Code is correct - same code works in other environments
- Dockerfile is correct - identical to working services
- ALL services fail DNS - both web and worker services
- Multiple service types affected - opsyn-backend, opsyn-sync-worker, opsyn-sync-worker-v2
- Fails at startup - before any application logic runs
What I've Already Tried
- Added DNS diagnostics to confirm failure
- Created FastAPI wrapper to run scheduler as web service
- Integrated scheduler into backend service
- All workarounds failed - DNS issue is environment-wide, not service-specific
Production Database (DO NOT CHANGE)
- Type: AWS RDS PostgreSQL (external, not Railway-managed)
- Hostname:
opsyn-db-prod.cwj08oy8u09z.us-east-1.rds.amazonaws.com - Port: 5432
- Status: Accessible from outside Railway
- This is production source of truth - cannot migrate or change
What I Need From Railway Support
- Check DNS resolver configuration for environment
0f66ea12-7899-4cf8-b0cf-a9fe1d46dd27 - Verify outbound network access for all services in this environment
- Investigate if DNS resolver changed after initial deployment
- Rebuild or migrate environment if necessary to restore DNS
- Confirm when DNS is restored so services can connect to AWS RDS
Critical Notes
- ⚠Do NOT suggest creating a Railway Postgres database - AWS RDS is production
- ⚠Do NOT suggest changing DATABASE_URL - it's correct and must point to AWS RDS
- ⚠Do NOT suggest running migrations - database is external and must not change
- ⚠This is blocking production - all services cannot start
Impact
- Cannot connect to production database
- Cannot resolve any external hostnames
- Sync scheduler cannot run
- API startup fails
- PRODUCTION IS DOWN
3 Replies
19 days ago
This thread has been marked as public for community involvement, as it does not contain any sensitive or personal information. Any further activity in this thread will be visible to everyone.
Status changed to Open Railway • 19 days ago
19 days ago
Are you sure that the database is public? I'm not getting any records when I perform a DNS lookup on the provided hostname as also suggested by the error that you are getting. https://dnschecker.org/all-dns-records-of-domain.php?query=opsyn-db-prod.cwj08oy8u09z.us-east-1.rds.amazonaws.com&rtype=ALL&dns=google
19 days ago
14 days ago
Try enabling IPv6 outbound networking in your service. Redeploy after.