Issue Summary

IPv6 private networking between services fails in PR environments but works correctly in persistent environments (development). Services cannot communicate using service.railway.internal or short hostname in PR environments, resulting in "connection refused" errors.

Environment Details

• Project ID: aa71fa2c-3f89-4ec1-ad45-2bfc9eaf1e41

• Project Name: railwayapp

• Working Environment: development (ID: 1c5ada25-2d6a-441b-8d76-d52a8025909e)

• Failing Environment: railwayapp-pr-5 (and all PR environments)

Services Involved

1. orkes (ID: 9c50bba2-b7c2-4f9f-9d96-6f8a05979b35)

   • Internal port: 5000 (Nginx) → 8080 (Spring Boot)

2. orkes-proxy (ID: 41b9f469-f5ee-491c-a689-4d093b41077b)

   • GitHub source: Naveo-Commerce-UK/railwayapp, branch: main

   • Uses Caddy to proxy to Orkes service

Problem Description

The orkes-proxy service (running Caddy) cannot connect to the orkes service on port 5000 via Railway's private networking in PR environments. The same configuration works perfectly in the development environment.

What Works

Development environment:

• Connection via orkes.railway.internal:5000 succeeds

• Connection via orkes:5000 succeeds

• IPv6 address resolution works: [fd12:9ec4:c036:1:1000:95:94b4:1b8d]:5000

• HTTP requests return 200 OK after authentication

PR environment (public access test):

• Public domain on port 5000 works correctly

• Orkes service is running and healthy

• Nginx is listening on port 5000 and serving content

What Fails

PR environment (internal networking):

• Connection via orkes.railway.internal:5000 fails with "connection refused"

• Connection via orkes:5000 fails with "connection refused"

• IPv6 address resolution succeeds but connection fails: [fd12:ac8b:e40a:0:1000:67:fa7e:72b0]:5000

• HTTP requests return 502 Bad Gateway

Error Logs

PR Environment - orkes-proxy deployment logs:

dial tcp [fd12:ac8b:e40a:0:1000:67:fa7e:72b0]:5000: connect: connection refused
2026/02/20 10:30:09.293 ERROR http.log.access.log0 handled request
  "status": 502
  "user_id": "admin"
  "duration": 2.098843541

PR Environment - Railway HTTP logs:

{
  "requestId": "6AXaW1dwRcyNZpcW2JZdWA",
  "timestamp": "2026-02-20T10:30:09Z",
  "httpStatus": 502,
  "upstreamAddress": "http://[fd12:ac8b:e40a:0:4000:72:f217:cd77]:8080",
  "upstreamErrors": ""
}

Development Environment - Working logs:

{
  "requestId": "s3mFTqDZQ7-ocKSJrJsmnA",
  "timestamp": "2026-02-20T09:25:18Z",
  "httpStatus": 200,
  "upstreamAddress": "http://[fd12:9ec4:c036:1:1000:95:94b4:1b8d]:8080",
  "upstreamErrors": ""
}

Note: Development also occasionally shows IPv4 connection refused errors, but requests succeed overall.

Reproduction Steps

1. Create a PR environment from branch with orkes-proxy configuration

2. Deploy both orkes and orkes-proxy services

3. Configure orkes-proxy Caddyfile to reverse proxy to http://orkes:5000

4. Attempt to access orkes-proxy via its public domain

5. Observe 502 errors in orkes-proxy logs with "connection refused" to IPv6 address

Configuration Files

Caddyfile (orkes-proxy service)

:{$PORT:5000}

log {
    output stdout
    format console
}

# Health check — no auth
handle /health {
    respond 200
}

# Token endpoint — return fake JWT
handle /api/token {
    header Content-Type "application/json"
    respond `{"token":"..."}` 200
}

# API requests — strip JWT before proxying
handle /api/* {
    request_header -X-Authorization
    reverse_proxy http://orkes:5000
}

# UI requests — require HTTP Basic Auth
handle {
    basicauth {
        {$ORKES_PROXY_USER:admin} {$ORKES_PROXY_PASSWORD_HASH}
    }
    reverse_proxy http://orkes:5000
}

Testing Performed

Test 1: Different hostname formats

• http://[orkes.railway.internal]:5000 - connection refused (PR)

• http://orkes.railway.internal:5000 - connection refused (PR)

• http://orkes:5000 - connection refused (PR)

• http://orkes:5000 - works (Development)

Test 2: Different ports

• http://orkes:8080 - connection refused (PR)

• http://orkes:5000 - connection refused (PR)

Test 3: Public domain (security test)

• https://orkes-railwayapp-pr-5.up.railway.app (port 5000) - works

• Confirms Orkes service is running and Nginx is listening on port 5000

Test 4: IPv6 address format

• http://[orkes.railway.internal]:5000 - connection refused (PR)

• Caddy still resolves to IPv6 and fails

Network Details

Development environment:

• Orkes IPv6: fd12:9ec4:c036:1:1000:95:94b4:1b8d

• Private domain: orkes.railway.internal

• Connection: Success

PR environment:

• Orkes IPv6: fd12:ac8b:e40a:0:1000:67:fa7e:72b0

• Private domain: orkes.railway.internal

• Connection: Refused