3 months ago
I've encountered the following problem: I have a service running that automatically redeploys from GitHub. I also have a Nginx proxy running that rotates API requests via an /api to internal endpoint of my service. After my service redeploys, the Nginx settings stop working (requests hang for a long time and time out). When I open the Nginx web interface and simply save the settings without changing anything, everything starts working again. What could be the problem?
1 Replies
Status changed to Awaiting Railway Response Railway • 3 months ago
2 months ago
So we have a service on Railway that automatically redeploys from GitHub. An Nginx proxy sits in front of it and forwards requests from /api to the internal service endpoint.
Everything works fine initially. However, after the service redeploys:
- API requests start hanging
- Eventually they time out
- No response is returned from the upstream service
Interestingly, if we open the Nginx configuration UI and simply click “Save” (without making any changes), everything immediately starts working again.
What’s actually happening
When Railway redeploys the service, the underlying container is replaced and its internal IP address changes.
Nginx, however, resolves the service hostname only once when it starts and continues using that cached IP.
So after a redeploy:
- Nginx is still sending requests to the old (dead) IP
- Connections hang because that container no longer exists
- When we “Save” the config, Nginx reloads → DNS is resolved again → correct IP is used → things work
Solution
We need to make Nginx re-resolve DNS dynamically, instead of caching it forever.
Fix
Add a DNS resolver and force runtime resolution:
resolver 1.1.1.1 8.8.8.8 valid=10s;
resolver_timeout 5s;
set $backend "your-service.internal";
proxy_pass http://$backend;Why this works
resolvertells Nginx how to resolve DNSvalid=10sensures it refreshes periodically- using a variable (
$backend) forces Nginx to resolve the hostname at request time instead of startup
Result
After this change:
- Nginx automatically picks up the new IP after redeploys
- No more hanging requests
- No need to manually reload or “save” config