Skip to content
Inconsistent Third-Party API Responses Across Railway Services — Possible Egress Routing Issue
flowstatelogicinc
OP

14 days ago

Hello Railway Community,

I'm investigating inconsistent responses from a third-party AI API across six Railway services and would appreciate guidance on Railway's outbound networking behaviour.

Environment

We have two independent Railway projects (staging and production), each running three Python services:

  • FastAPI backend
  • Background worker
  • AI processing worker

Five services run in Railway's sfo region, and one runs in us-west2.

Each project's three services share the same API credentials for the external provider. Credentials have been verified and rotated.

The problem

We're calling an external AI provider's OpenAI-compatible API using the Python OpenAI SDK.

Identical authenticated requests produce inconsistent responses across containers.

On November 5, 2026, we executed direct HTTP diagnostics inside all six Railway containers.

Results:

| Service | GET /models | GET /models/{model_id} |

|---|---|---|

| Production Worker | 200, 0 models | 404 |

| Production AI Worker | 200, 0 models | 404 |

| Production API | 200, 9 models | 404 |

| Staging AI Worker | 200, 0 models | 404 |

| Staging Worker | 200, 0 models | 404 |

| Staging API | 200, 0 models | 404 |

All 404 responses were application-level model_not_found errors returned by the external provider, not connection failures.

Interesting findings

  1. Five containers reported outbound public IP addresses within the same broad address range. All five received empty model catalogs.

  2. The sixth container reported a public IP from a different address range and successfully retrieved nine models. However, requesting details for a model explicitly listed in that catalog still returned HTTP 404.

  3. During earlier testing, one worker received HTTP 404 for a model-detail request, then successfully completed an actual inference call using the same model and API key approximately 16 minutes later, without a redeployment or credential change.

  4. We've observed model availability changing following some deployments, although we haven't established a causal relationship.

  5. All six containers previously resolved the provider's hostname to identical DNS destination addresses.

  6. No HTTP_PROXY, HTTPS_PROXY, or ALL_PROXY environment variables were configured.

The observed public IP addresses were obtained through a separate external IP-check service. We recognize these are not necessarily the exact source addresses used for individual requests to the AI provider.

What we've ruled out

  • Different API keys within the same project
  • Incorrect model identifier
  • Missing credentials
  • Basic DNS resolution failures
  • An obvious application-level HTTP proxy configuration

We've also opened an engineering investigation with the external API provider.

Questions for Railway

  1. Can Railway services within the same region use different outbound NAT gateways or public IP pools?

  2. Can a service's outbound source IP or routing path change following redeployment?

  3. Could outbound routing differences cause a third-party API to return different application-level results, even when the API endpoint, credentials, and request parameters are identical?

  4. Is there a recommended way to identify or trace the actual source IP used for connections to a particular external API?

  5. Would testing static outbound IPs or changing Railway regions help isolate the issue?

We're not suggesting that Railway is necessarily responsible for the third-party API's HTTP responses. We're trying to isolate potential networking differences before continuing the investigation with the API provider.

I can provide additional non-sensitive diagnostic information if needed.

Any guidance from Railway engineers or community members who have encountered similar behaviour would be appreciated.

Thank you!

Solved

1 Reply

Railway
BOT

14 days ago

Without Static Outbound IPs, a service's outbound traffic is not tied to one fixed address, so different services, including ones in the same region, can reach the internet from different public IPs. Moving a service to another region also changes the outbound IPs it uses. If you need a predictable source address, Static Outbound IPs give a service a set of permanent outbound IPv4 addresses, and traffic is balanced across them. This feature needs the Pro plan, your workspace is on Hobby, and the addresses are not guaranteed to be dedicated to you.

All six containers connected, finished their requests, and got HTTP responses back from the provider, including the 200s and the model_not_found 404s. That means outbound traffic is reaching the provider, and what each response contains is decided on the provider's side. The fact that one container listed models and then got a 404 for a model in that list, from the same source, is a good thing to send to the provider's engineering team. Pinning a few services to Static Outbound IPs and comparing the results would help you check whether the source address makes any difference.


Status changed to Awaiting User Response Railway • 14 days ago


Railway
BOT

7 days ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway • 7 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...