Procrastinate Worker Cannot Connect to Database in Staging - Connection Fails Despite Correct Configuration
rickyhopkins
PROOP

2 months ago

Hi Railway team,

We're experiencing a database connection issue with our Procrastinate worker service in staging after a recent deployment.

Problem:

- Procrastinate async worker cannot connect to PostgreSQL database

- Error: procrastinate.exceptions.ConnectorException: Database error

- Production environment is working fine with the same codebase

- Database itself is up and running - we can connect via the public URL directly

Current Configuration:

All environment variables are now correctly populated:

- DATABASE_NAME - set

- DATABASE_USER - set

- DATABASE_PASSWORD - set

- DATABASE_HOST - postgres.railway.internal

- DATABASE_PORT - 5432

Our Database Configuration:

DATABASES = {

"default": {

"ENGINE": "django.db.backends.postgresql",

"NAME": os.environ.get("DATABASE_NAME", "app"),

"USER": os.environ.get("DATABASE_USER", "postgres"),

"PASSWORD": os.environ.get("DATABASE_PASSWORD", "password"),

"HOST": os.environ.get("DATABASE_HOST", "db"),

"PORT": 5432,

}

}

Error Stack Trace:

File "/usr/local/lib/python3.12/site-packages/procrastinate/contrib/django/django_connector.py", line 105, in execute_query_all_async

File "/usr/local/lib/python3.12/site-packages/procrastinate/sync_psycopg_connector.py", line 44, in wrap_exceptions

raise exceptions.ConnectorException from exc

procrastinate.exceptions.ConnectorException: Database error.

What's Working:

- Can connect to database using public URL from external tools

- Production environment works with identical code

- Database service shows as healthy in Railway dashboard

What's Not Working:

- Staging app cannot connect to postgres.railway.internal

- Issue persists despite correct environment variables

Questions:

1. Is there a networking/service discovery issue preventing the staging app from reaching postgres.railway.internal?

2. Are there any recent changes to Railway's internal networking or private networking setup?

3. Could this be a service linking issue between the app and database?

4. Are there any firewall or network policy changes that might block the connection?

5. Should we be using a different hostname or connection method for internal database connections?

Recent Changes:

- Latest deployment contained only minor Django model changes

- No infrastructure, networking, or configuration changes were made

- Issue appeared immediately after this deployment

Can you help us diagnose why the internal hostname isn't resolving/connecting from our staging app?

$10 Bounty

1 Replies

fra
HOBBY

2 months ago

Can you share the error?


Loading...