2 months ago
I am experiencing an issue when connecting to the Railway database using the internal database URL in the production environment.
When I connect to the same database using DBeaver with the public URL, all schemas and tables are visible and accessible as expected. However, when I use the internal Railway database URL in my production application, the tables appear to be missing.
Steps to Reproduce:
- Connect to the database using DBeaver with the public URL → All schemas and tables are visible.
- Connect to the database using the internal Railway database URL (in production) → Tables are not visible.
Expected Behavior:
All schemas and tables should be visible and consistent regardless of whether the connection is made using the public or internal database URL.
Actual Behavior:
Schemas/tables are missing when using the internal Railway database URL.
Environment:
- Tool: DBeaver (for testing)
- Environment: Production
- Database: PostgreSQL
Additional Information:
This seems like the internal URL may be pointing to a different database instance or environment than the public URL.
7 Replies
2 months ago
All three services in your project (trading-platform, trading-app, and Postgres) are currently in a FAILED state, which means the private network is not operational since it only functions at runtime with running services. The internal and public URLs point to the same database instance, so when all services are healthy and running, the tables visible via the public URL will also be accessible via the internal URL. The immediate issue to resolve is getting your services back to a running state.
2 months ago
This thread has been marked as private. Any further activity in this thread will only be visible to you and Railway employees.
Status changed to Awaiting User Response Railway • about 2 months ago
a month ago
I’m referring to the HRCG Tech workspace where I’m currently signed in.
In this setup, we have a service called Backbone that connects to another service, Zoho Data, which is essentially a PostgreSQL database.
Backbone is trying to connect to this PostgreSQL database and perform an upsert operation. However, it is throwing a very strange error: it says that the table does not exist.
I have verified multiple times that the table does exist. I also connected to the database using the public URL, and everything works fine—all the tables are present.
Despite this, I’m still not sure why it is reporting that the particular table does not exist.
Status changed to Awaiting Railway Response Railway • about 1 month ago
a month ago
This thread has been marked as public for community involvement, as it does not contain any sensitive or personal information. Any further activity in this thread will be visible to everyone.
Status changed to Open Railway • about 1 month ago
a month ago
Are you sure you're accessing the right database? (IIRC it should be railway)
a month ago
Yes I made different logs.
a month ago
I verified multiple time i'm accessing right database and right schema.
a month ago
Yes, I have checked that everything related to the URL is correct. However, I have encountered another issue. When I deploy a new container, the old one still remains in a running state.
For example, if a new process starts with PID 190, the old process with PID 188 does not stop. It continues trying to connect to the database, which fails and generates many errors. This happens because I have implemented Redis and BullMQ, and the old process keeps trying to complete jobs from the queue.
Meanwhile, the new process works fine and runs in parallel with the old one. Also, whenever I deploy a new container, I get a new PID, but the old running process PID also changes and continues running.
I need a clear solution on how to stop all old processes and ensure that only a fresh new process runs.
a month ago
Here's a clear bug report for the Railway team:
---
Subject: Internal proxy routes new container connections to empty database during deploys
Project: HRCG Backend (NestJS monorepo with PM2)
people, projects) and data. The application connects via pg-zoho-data.railway.internal using the internal network.
Problem
During zero-downtime deploys, new containers' database connections are routed to what appears to be an empty database — all schemas
are missing. Old containers' existing connections continue working fine. The issue persists for the entire lifetime of the
affected container (never self-resolves).
Evidence from logs (2026-04-10, all times UTC)
Deploy at ~05:29
Old container (PID 190) — connects to pg-zoho-data.railway.internal, warmup succeeds, all schemas present.
New container (PID 178) — connects to the same pg-zoho-data.railway.internal:
05:29:55.078 Prisma $connect() succeeds (initial connection is fine)
05:30:25.053 Cache warmup query: SELECT * FROM people.employees
→ ERROR: relation "people.employees" does not exist
DB diagnostic snapshot from PID 178's connection:
{
"dbHost": "pg-zoho-data.railway.internal",
"dbNameFromUrl": "railway",
"search_path": "\"$user\", public",
"zohoDataSchemasPresent": [],
"zohoDataSchemasMissing": ["books", "crm", "mail", "meta", "people", "projects"],
"referencedSchemaTableCount": 0
}
ALL six schemas are missing. This is not a search_path issue — the schemas don't exist on whatever backend this connection reaches.
Same deploy — onModuleInit diagnostics tell a different story
The Prisma onModuleInit runs a diagnostic query (SELECT schemaname, tablename FROM pg_tables) on every service startup:
05:32:16-17 (first wave): ALL PIDs → sample_table_count = 0 ← empty DB
05:32:21-23 (5 seconds later): ALL PIDs → sample_table_count = 10 ← correct DB
The initial connections hit an empty database. Five seconds later, new connections hit the correct database. But the application's
connection pool holds onto the initial (bad) connections.
Second deploy at ~06:11
Same behavior — new container (PID 178) connects to pg-zoho-data.railway.internal, Prisma $connect() succeeds, but all subsequent
queries fail with table does not exist. The proxy never stabilizes for this container — still failing 4+ minutes after deploy.
Key observations
1. Same connection string — old and new containers use identical ZOHO_DATA_DATABASE_URL pointing to
pg-zoho-data.railway.internal/railway
2. Both PostgreSQL instances have all schemas — verified manually that pg-zoho-data and postgres-cbhs both have books, crm, mail,
meta, people, projects schemas with data
3. Other services work fine — webhooks (different service containers) connect to the same URL and write successfully at the same
time
4. Initial connection succeeds — Prisma $connect() passes. It's subsequent pool connections that hit the empty backend
5. Not transient — the issue persists for the container's entire lifetime. Only a fresh deploy/container gets correct routing
6. search_path resets — the application sets search_path to books,crm,... via onConnect, but the affected connections show default
"$user", public, suggesting the SET search_path command may have succeeded but the schemas simply don't exist on that backend
Questions for Railway team
1. Does the internal proxy (*.railway.internal) perform any connection pooling or load balancing between backends during deploys?
2. Is there a transient/temporary PostgreSQL backend that exists during the deploy window?
3. Could the proxy be caching DNS or routing state from a previous container that pointed to a different backend?
4. Is there a way to force direct connections (bypassing the internal proxy) for internal networking?
Environment
- Railway internal networking (no public URLs)
- PostgreSQL connection via pg-zoho-data.railway.internal
- ssl: false (internal network)
- Node.js pg pool with keepAlive: true, min: 1, max: 30
Can you please look into thin in priority ?