Outbound connections from app container to Postgres
biloni
HOBBYOP

7 days ago

Service: tradingOS (api), project "tradingOS", service ID 11de9e83-6fc0-419f-932c-38685ea2d1c2

Database: Postgres plugin, same project, EU West (Amsterdam) — same region as the api service

Issue: The api service intermittently returns 502 "Application failed to respond" and,

when it does respond, takes 8-14+ seconds per request (confirmed via /health, which has

no logic beyond returning a static JSON body). This happens whether DATABASE_URL points

at the internal reference (${{Postgres.DATABASE_URL}}, postgres.railway.internal) or the

public proxy URL (switchyard.proxy.rlwy.net:28918) — same slowness/timeouts either way.

What I've ruled out:

  • App code: identical code, pointed at the same live Postgres database (via the public

    proxy URL), starts in under 1 second and serves requests instantly when run locally

    (outside Railway entirely) from my own machine.

  • Database health: connected directly via psql from my own machine to the same public

    proxy URL — instant connection, clean pg_stat_activity (no stuck sessions/locks), all

    122 tables present, alembic_version at the correct head revision (migrations succeeded).

  • Resources: Metrics tab shows flat ~90MB memory and ~0% CPU during the slow window — not

    crashing, OOMing, or spinning.

  • Region mismatch: both the api service and Postgres are in EU West (Amsterdam).

  • Deploy logs: the container starts, runs the pre-deploy alembic migration successfully

    (confirmed via direct DB check), but nothing further is logged even after several

    minutes, and the app is unreachable from outside for that whole window.

Evidence from the request-metrics graphs on the tradingOS service (visible in-app):

Response Time graph shows a sustained p99 of ~9-14s over a ~30 minute window

(around 20:00-20:47 in the service's local deploy-log timestamps), not a single spike —

this was the consistent baseline, not an outlier.

This points at something in Railway's own network path for outbound connections

initiated from inside an app container in this project/region, not at application code,

database health, or configuration on our end. Happy to provide direct psql access

details or run any specific diagnostic you'd want from inside the container via the

Console tab.

$10 Bounty

6 Replies

Railway
BOT

7 days ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway 7 days ago


Is the port your application is listening to the same port your URL is mapped to?


biloni
HOBBYOP

7 days ago

Thanks for the quick response. I tested that specifically:

  • Explicitly added a PORT=8000 variable to the service (previously nothing named PORT

    was set explicitly, though the app's Dockerfile CMD does read $PORT dynamically with

    an ENV PORT=8000 fallback default).

  • Confirmed the Networking target port is also set to 8000, matching.

  • Redeployed (deployment ef7aee03, 2026-08-13 21:37 PDT).

Same result: GET /health (which has no logic beyond returning a static JSON body,

no database call) still times out after 12-15s from outside, and the deploy logs show

nothing past the pre-deploy migration step's two INFO lines -- no uvicorn startup lines

at all, even though the same container image runs and logs normally within under a

second when I run it locally against the same live database.

So port mismatch looks ruled out. Given what I already sent -- database confirmed

healthy via direct psql connection (instant, no locks, correct migration state), app

code confirmed correct (runs and logs normally outside Railway against the same

database), and now port config confirmed matching -- I think this points to something

in the container's actual outbound network path or process supervision on Railway's

side for this specific deployment, rather than app config. Could someone take a look

at the raw container logs/state directly, or let me know if there's a way to get more

verbose output from the platform side than what's in the Deploy Logs tab?

Service: tradingOS (11de9e83-6fc0-419f-932c-38685ea2d1c2)

Latest affected deployment: ef7aee03 (2026-08-13 21:37 PDT)


Can you share your Dockerfile? (As well as any custom pre-deploy/deploy commands you have set up in Railway)


biloni
HOBBYOP

7 days ago

Sure — here's the Dockerfile (apps/api/Dockerfile):

FROM python:3.14-slim AS builder

COPY --from=ghcr.io/astral-sh/uv:0.7 /uv /uvx /bin/

WORKDIR /app

ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy UV_PYTHON_DOWNLOADS=never

COPY pyproject.toml uv.lock ./

RUN uv sync --frozen --no-install-project --no-dev

COPY . .

RUN uv sync --frozen --no-dev

FROM python:3.14-slim AS runtime

WORKDIR /app

RUN useradd --create-home --shell /bin/bash appuser

COPY --from=builder --chown=appuser:appuser /app /app

ENV PATH="/app/.venv/bin:$PATH"

USER appuser

EXPOSE 8000

ENV PORT=8000

HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \

CMD python -c "import os, urllib.request; urllib.request.urlopen(f'http://localhost:{os.environ.get(\"PORT\", \"8000\")}/health', timeout=3)" || exit 1

CMD uvicorn tradingos_api.main:app --host 0.0.0.0 --port "$PORT"

Railway service settings for this service (tradingOS):

  • Builder: Dockerfile (path: apps/api/Dockerfile)
  • Pre-Deploy Command: alembic upgrade head
  • Start Command: none set (uses the Dockerfile's own CMD above)
  • Root Directory: apps/api

For context, apps/api/Dockerfile's HEALTHCHECK hits GET /health on the same container

(localhost), and that endpoint has zero logic beyond returning a static JSON body -- no

database call, no external dependency. So even the container's own internal healthcheck

should be near-instant if the process is actually listening and responsive.


I'd try removing Dockerfile's healthcheck entry for now and see what happens.


biloni
HOBBYOP

7 days ago

Tried that too — removed the Dockerfile's HEALTHCHECK entirely (deployment 9d2b7407,

2026-08-13 21:53 PDT, confirmed via a fresh git push + redeploy). Same result: GET /health

still times out after 15s from outside with no response at all, and the deploy logs still

show nothing past the pre-deploy migration's two INFO lines -- no uvicorn startup output.

So to summarize what's been ruled out on our end across this thread:

  • Database: confirmed healthy via direct psql connection (instant, no locks, correct

    migration state, all tables present)

  • App code: confirmed correct -- runs and serves requests in under a second locally,

    against this exact same live database

  • Port config: explicitly pinned PORT=8000 as a service variable, matching the

    Networking target port -- no mismatch

  • Docker-level HEALTHCHECK: removed entirely -- no change in behavior

  • Region: api service and Postgres are both in EU West (Amsterdam) -- no mismatch

  • Resource usage: Metrics tab shows flat ~90MB memory, ~0% CPU during the failure

    window -- not crashing, not OOMing, not spinning

At this point I don't have another app-level or config-level lever left to pull. Could

someone look at this from the platform side -- e.g. whatever's between the edge and the

container for this specific deployment/service?

Service: tradingOS (11de9e83-6fc0-419f-932c-38685ea2d1c2)

Latest deployment: 9d2b7407 (2026-08-13 21:53 PDT)


Welcome!

Sign in to your Railway account to join the conversation.

Loading...