NGINX 502 Bad Gateway

thezeenx1PRO

5 months ago

Hello Railway,

I have set up 4 services (Backend, Frontend, Nodejs, NGINX) using Docker containers and each of them are working fine.

When I use NGINX service to route traffic to each services, I am getting a 502 Bad Gateway error.

If I visit mydomain.com, favicon, robots.txt and ping all respond correctly, but other routes don't work correctly.

mydomain.com is handled by Cloudflare where I have declared CNAME record for the NGINX service.

Any help will be useful.

Thanks!

Solved

9 Replies

thezeenx1PRO

5 months ago

Couldn't paste nginx config on mobile, so here's the screenshot of the config.

Attachments


5 months ago

Hello,

I see a few things wrong here -

1. Using public domains instead of private domains.

But most importantly, 2. The use of NGINX

Nginx works horribly in an environment where the upstreams have dynamic IPs, the private service domains will have a new IP every time they are deployed, that will break NGINX until it is redeployed too, instead, please use Caddy as it has no such problems, here is an example to get you started -

https://github.com/railwayapp-templates/caddy-reverse-proxy

Since the private network is IPv6, make sure all your upstreams are in the same project and binding to IPv6 -

https://docs.railway.com/guides/private-networking#listen-on-ipv6
https://docs.railway.com/overview/best-practices#deploying-related-services-into-the-same-project


Status changed to Awaiting User Response railway[bot] 5 months ago


brody

Hello,I see a few things wrong here -1. Using public domains instead of private domains.But most importantly, 2. The use of NGINXNginx works horribly in an environment where the upstreams have dynamic IPs, the private service domains will have a new IP every time they are deployed, that will break NGINX until it is redeployed too, instead, please use Caddy as it has no such problems, here is an example to get you started -https://github.com/railwayapp-templates/caddy-reverse-proxySince the private network is IPv6, make sure all your upstreams are in the same project and binding to IPv6 -https://docs.railway.com/guides/private-networking#listen-on-ipv6https://docs.railway.com/overview/best-practices#deploying-related-services-into-the-same-project

thezeenx1PRO

5 months ago

Thanks will update you after trying this out!


Status changed to Awaiting Railway Response railway[bot] 5 months ago


5 months ago

Sounds good, let me know if you run into any trouble, and if you do, please provide a project ID.


Status changed to Awaiting User Response railway[bot] 5 months ago


brody

Sounds good, let me know if you run into any trouble, and if you do, please provide a project ID.

thezeenx1PRO

5 months ago

Hi!
So I changed the server to Caddy and it fixed a lot of problems, but there still exists issue regarding /node/* routes in the deployment.

// Server for frontend (Sveltekit build)

import { handler } from './build/handler.js';
import express from 'express';

const port = process.env.PORT || 3000;

const app = express();

app.use(handler);

app.listen(port, '::', () => {
    console.log(`Server listening on [::]:${port}`);
});
// Server for nodejs

const port = process.env.PORT || 3000;
app.listen(port, '::', () => {
    logger.info(`Server listening on [::]${port}`);
});

The /* routes (for frontend) and /api/* routes (for backend (fastapi)) work perfectly fine with my current Caddy config, but /node/* routes don't work at all. It responds 403 page not found (frontend error). If I create a public link for nodejs service, it responds correctly to all requests, so there's no issue in the deployment for the nodejs service. I have verified the environment variables for the Caddy service too.

BACKEND_DOMAIN="${{backend.RAILWAY_PRIVATE_DOMAIN}}"
BACKEND_PORT="${{backend.PORT}}"
FRONTEND_DOMAIN="${{frontend.RAILWAY_PRIVATE_DOMAIN}}"
FRONTEND_PORT="${{frontend.PORT}}"
NODEJS_DOMAIN="${{nodejs.RAILWAY_PRIVATE_DOMAIN}}"
NODEJS_PORT="${{nodejs.PORT}}"

Caddyfile: (template from the github repo)

    // for fastapi
    handle /api/* {
		# proxy all requests for /api/* to the backend, configure this variable in the service settings
		reverse_proxy {
			# for private networking replicas are exposed as multiple dns results, use those dns results as the upstreams
			dynamic a {
				name {$BACKEND_DOMAIN}
				port {$BACKEND_PORT}
				refresh 1s
				dial_timeout 30s
				versions ipv4 ipv6
			}

			# configure load balancing settings
			import lb_settings

			# configure passive health checks
			import passive_health_checks

			# sets the Host header to the header to the dynamic name and port options
			header_up Host {upstream_hostport}
		}
	}
    
    // for nodejs
    handle /node/* {
		# proxy all requests for /node/* to the nodejs service, configure this variable in the service settings
		reverse_proxy {
			# for private networking replicas are exposed as multiple dns results, use those dns results as the upstreams
			dynamic a {
				name {$NODEJS_DOMAIN}
				port {$NODEJS_PORT}
				refresh 1s
				dial_timeout 30s
				versions ipv4 ipv6
			}

			# configure load balancing settings
			import lb_settings

			# configure passive health checks
			import passive_health_checks

			# sets the Host header to the header to the dynamic name and port options
			header_up Host {upstream_hostport}
		}
	}

# entrypoint.sh

set -euo pipefail

# for backwards compatibility, seperates host and port from url
export FRONTEND_DOMAIN=${FRONTEND_DOMAIN:-${FRONTEND_HOST%:*}}
export FRONTEND_PORT=${FRONTEND_PORT:-${FRONTEND_HOST##*:}}

export BACKEND_DOMAIN=${BACKEND_DOMAIN:-${BACKEND_HOST%:*}}
export BACKEND_PORT=${BACKEND_PORT:-${BACKEND_HOST##*:}}

export NODEJS_DOMAIN=${NODEJS_DOMAIN:-${NODEJS_HOST%:*}}
export NODEJS_PORT=${NODEJS_PORT:-${NODEJS_HOST##*:}}

# strip https:// or https:// from domain if necessary
FRONTEND_DOMAIN=${FRONTEND_DOMAIN##*://}
BACKEND_DOMAIN=${BACKEND_DOMAIN##*://}
NODEJS_DOMAIN=${NODEJS_DOMAIN##*://}

echo using frontend: ${FRONTEND_DOMAIN} with port: ${FRONTEND_PORT}
echo using backend: ${BACKEND_DOMAIN} with port: ${BACKEND_PORT}
echo using nodejs: ${NODEJS_DOMAIN} with port: ${NODEJS_PORT}

exec caddy run --config Caddyfile --adapter caddyfile 2>&1

Attachments


Status changed to Awaiting Railway Response railway[bot] 5 months ago


thezeenx1PRO

5 months ago

Also favicon.ico, robots.txt, /health, and /ping is working for the current Caddy server.


thezeenx1PRO

5 months ago

hmm, after a while it worked? not sure why though, I made no changes, just redeployed the nodejs and caddy service.


5 months ago

Awesome, glad Caddy worked for you!

And for what it's worth, you can omit the entrypoint.sh file as that's there for backwards compatibility.


Status changed to Awaiting User Response railway[bot] 5 months ago


brody

Awesome, glad Caddy worked for you!And for what it's worth, you can omit the entrypoint.sh file as that's there for backwards compatibility.

thezeenx1PRO

5 months ago

yeah, thanks a lot!


Status changed to Awaiting Railway Response railway[bot] 5 months ago


Status changed to Solved brody 5 months ago