11 days ago
Service: mycpd-saas (ID: 05ae845f-a5b7-47eb-b54b-f366673dda61) Environment: production Deployment:
b167dd26-5d1a-48ba-8ffa-14782668ed72 (failed 2026-08-11 at 22:48 UTC)
Issue: Deploy failure is caused by Prisma migrations unable to reach the database. The container starts but gets
stuck in a retry loop with repeated "P1001: Can't reach database server at metro.proxy.rlwy.net:42996" errors. The
migration step never completes, so the app never finishes starting, causing the healthcheck to time out and
deployment to fail.
This is not a database connectivity issue on my end — the problem is the database proxy infrastructure
(metro.proxy.rlwy.net:42996) is not responding to connection attempts from the deployed container.
I restarted the database as suggested in a previous diagnosis, but the proxy remains unreachable. Only Railway
infrastructure can diagnose why the proxy is down or broken.
Recent deployments also failed with the same P1001 error, indicating this is an ongoing infrastructure issue
affecting all deploy attempts.
Please investigate the database proxy endpoint and restore connectivity.
18 Replies
11 days ago
Having looked into this, the issue appears to be in your application code or configuration rather than the Railway platform itself, which puts it outside what Railway support can resolve directly.
This is exactly the kind of problem the Railway community is good at, so we'd like to open your thread as a community bounty. Railway pays a bounty to the community member who solves it, and threads like this usually get picked up quickly.
Opening it makes this entire thread public, including everything already posted. Nothing becomes public until you decide. Use the buttons below.
- Open to the community - Before you click, take a moment to edit or remove anything you'd rather not share. The thread becomes publicly visible right away.
- Keep it private and close the thread - Nothing becomes public. The thread closes, since this isn't something Railway support can take further.
Status changed to Awaiting User Response Railway • 11 days ago
Railway
Having looked into this, the issue appears to be in your application code or configuration rather than the Railway platform itself, which puts it outside what Railway support can resolve directly. This is exactly the kind of problem the Railway community is good at, so we'd like to open your thread as a community [bounty](https://docs.railway.com/community/bounties). Railway pays a bounty to the community member who solves it, and threads like this usually get picked up quickly. **Opening it makes this entire thread public**, including everything already posted. Nothing becomes public until you decide. Use the buttons below. - **Open to the community** - Before you click, take a moment to edit or remove anything you'd rather not share. The thread becomes publicly visible right away. - **Keep it private and close the thread** - Nothing becomes public. The thread closes, since this isn't something Railway support can take further.
11 days ago
This conclusion doesn't match the evidence already in this thread:
- A direct psql connection to metro.proxy.rlwy.net:42996 from outside Railway entirely — no app code involved — times
out identically.
- Railway's own database browser UI, on this same project, also got stuck on "Attempting to connect to the
database..."
- An earlier diagnosis in this thread identified this as a proxy infrastructure issue, restarted the database, and
the failure persisted afterward.
- No schema, migration, or DB-related code changed in this timeframe — the same config ran unchanged for months.
A plain external psql client and Railway's own dashboard both fail the same way my app does. The common factor is the
proxy, not my code. Please have someone review this thread's diagnostic history rather than re-triaging from scratch.
Status changed to Awaiting Railway Response Railway • 11 days ago
11 days ago
This still looks like an application-level problem, so Railway support can't take it further, but the community can. The buttons below are still live: open the thread up as a public bounty after editing out anything sensitive, or keep it private and close it.
Status changed to Awaiting User Response Railway • 11 days ago
11 days ago
This thread has been opened as a public bounty so the community can help solve it. The thread and any further activity are now visible to everyone.
Status changed to Open Railway • 11 days ago
11 days ago
Have you try Redeploy Source Image of the Postgres service?
You can do it by Open the Command Palette on the Service and then Select Redeploy Source Image
mayori
Have you try Redeploy Source Image of the Postgres service? You can do it by Open the Command Palette on the Service and then Select **Redeploy Source Image**
10 days ago
Thanks for this suggestion please see where we got to below - it is still broken : Update on the P1001 issue (project zippy-smile, service Postgres, id d3f0cd18-a02e-4852-ac9e-209120cd18c5) — new
evidence that narrows this down further.
Following the suggestion to try Redeploy Source Image on the Postgres service itself, I triggered a redeploy via CLI
(railway redeploy -s Postgres) at ~10:56 UTC today (2026-08-12). New deployment id:
aa28b48d-e693-4ba4-9d3e-dc5c1c0aff9d.
What happened:
- The existing container was stopped cleanly (normal shutdown log lines beforehand).
- The new container then logged repeated Mounting volume on:
/var/lib/containers/railwayapp/bind-mounts/447ea419-abc1-4ded-b6e9-4394db4af201/vol_o5b37ak8p557isds lines — 9+
identical repeats over ~10 minutes, with no Postgres startup log ever following (no "database system is ready to
accept connections", nothing).
- Deployment status eventually moved from DEPLOYING to FAILED on its own (no manual intervention).
- Connection behavior changed as a result: before the redeploy, connections to metro.proxy.rlwy.net:42996 would hang
and time out at the Postgres protocol layer (the original P1001 pattern — TCP handshake succeeds, protocol negotiation
never completes). After the redeploy, connections now fail immediately with server closed the connection unexpectedly
— a different, faster failure mode, suggesting the new container is reachable at the network level but Postgres
itself never finishes starting.
Data integrity check (in case this is relevant to diagnosis): the volume (postgres-volume, id
db82a803-3481-4ac8-bcf2-97a51a2ae983, mount path /var/lib/postgresql/data) currently reports currentSizeMB: 230.75
against an allocated sizeMB: 5000. This closely matches the size of a known-good manual backup taken 5 days ago (~230
MB), which strongly suggests the data on the volume itself is intact — the failure looks contained to the
container/mount layer, not data loss.
Given this looks like a stuck volume-mount/container-start issue specifically (repeated mount attempts, never
progressing to Postgres startup), is there anything on Railway's infrastructure side that could explain a volume mount
hanging indefinitely for this bind-mount path? Happy to provide the full deployment logs or any other diagnostics.
Also saved to /tmp/claude-1000/.../scratchpad/railway-community-comment.md if you want to tweak before posting. I
didn't include any credentials — only IDs, timestamps, and log excerpts. Since posting to the community thread is a
public, outward-facing action, I'll leave the actual posting to you rather than doing it myself — let me know if you'd
like anything adjusted first.
10 days ago
Update on the P1001 issue (project zippy-smile, service Postgres, id d3f0cd18-a02e-4852-ac9e-209120cd18c5) — new
evidence that narrows this down further.
Following the suggestion to try Redeploy Source Image on the Postgres service itself, I triggered a redeploy via CLI
(railway redeploy -s Postgres) at ~10:56 UTC today (2026-08-12). New deployment id:
aa28b48d-e693-4ba4-9d3e-dc5c1c0aff9d.
What happened:
- The existing container was stopped cleanly (normal shutdown log lines beforehand).
- The new container then logged repeated Mounting volume on:
/var/lib/containers/railwayapp/bind-mounts/447ea419-abc1-4ded-b6e9-4394db4af201/vol_o5b37ak8p557isds lines — 9+
identical repeats over ~10 minutes, with no Postgres startup log ever following (no "database system is ready to
accept connections", nothing).
- Deployment status eventually moved from DEPLOYING to FAILED on its own (no manual intervention).
- Connection behavior changed as a result: before the redeploy, connections to metro.proxy.rlwy.net:42996 would hang
and time out at the Postgres protocol layer (the original P1001 pattern — TCP handshake succeeds, protocol negotiation
never completes). After the redeploy, connections now fail immediately with server closed the connection unexpectedly
— a different, faster failure mode, suggesting the new container is reachable at the network level but Postgres
itself never finishes starting.
Data integrity check (in case this is relevant to diagnosis): the volume (postgres-volume, id
db82a803-3481-4ac8-bcf2-97a51a2ae983, mount path /var/lib/postgresql/data) currently reports currentSizeMB: 230.75
against an allocated sizeMB: 5000. This closely matches the size of a known-good manual backup taken 5 days ago (~230
MB), which strongly suggests the data on the volume itself is intact — the failure looks contained to the
container/mount layer, not data loss.
Given this looks like a stuck volume-mount/container-start issue specifically (repeated mount attempts, never
progressing to Postgres startup), is there anything on Railway's infrastructure side that could explain a volume mount
hanging indefinitely for this bind-mount path? Happy to provide the full deployment logs or any other diagnostics.
Also saved to /tmp/claude-1000/.../scratchpad/railway-community-comment.md if you want to tweak before posting. I
didn't include any credentials — only IDs, timestamps, and log excerpts. Since posting to the community thread is a
public, outward-facing action, I'll leave the actual posting to you rather than doing it myself — let me know if you'd
like anything adjusted first.
10 days ago
from Railway Agent review of above : Let me search for this P1001 issue to see if there's an existing support thread or community post about it.
Checked your support threads
Searched support threads: P1001 Postgres volume mount × 2
Found it! You have an open support thread about this. Let me get the full details:
Opened support thread: subject-database-proxy-unreachable-p10-86351033
Got it. Your P1001 thread is at: https://station.railway.com/questions/subject-database-proxy-unreachable-p10-86351033
The last suggestion was to Redeploy Source Image on the Postgres service. It looks like you followed that and got new findings. Your update should go into that same thread as a reply — post your new evidence there to keep the investigation thread continuous.
The details you have are excellent:
New deployment ID: aa28b48d-e693-4ba4-9d3e-dc5c1c0aff9d
Volume mount hanging with repeated "Mounting volume" logs, Postgres never starts
Faster failure mode post-redeploy (connections fail immediately instead of timing out)
Data integrity check: volume is 230.75 MB (matches 5-day-old backup), data is intact
Bind-mount path issue: /var/lib/containers/railwayapp/bind-mounts/447ea419-abc1-4ded-b6e9-4394db4af201/vol_o5b37ak8p557isds
Post this as a reply to your existing P1001 thread at the link above. The diagnostic evidence is strong — it narrows the issue from "database state" to "infrastructure-level volume mount problem". The Railway team and community can see your history and build on the earlier diagnosis.
Paste your scratchpad comment directly, or refine it further if you want. Either way, replying to the existing thread keeps the context intact.
seamusmacasgaill-debug
from Railway Agent review of above : Let me search for this P1001 issue to see if there's an existing support thread or community post about it. Checked your support threads Searched support threads: P1001 Postgres volume mount × 2 Found it! You have an open support thread about this. Let me get the full details: Opened support thread: subject-database-proxy-unreachable-p10-86351033 Got it. Your P1001 thread is at: https://station.railway.com/questions/subject-database-proxy-unreachable-p10-86351033 The last suggestion was to Redeploy Source Image on the Postgres service. It looks like you followed that and got new findings. Your update should go into that same thread as a reply — post your new evidence there to keep the investigation thread continuous. The details you have are excellent: New deployment ID: aa28b48d-e693-4ba4-9d3e-dc5c1c0aff9d Volume mount hanging with repeated "Mounting volume" logs, Postgres never starts Faster failure mode post-redeploy (connections fail immediately instead of timing out) Data integrity check: volume is 230.75 MB (matches 5-day-old backup), data is intact Bind-mount path issue: /var/lib/containers/railwayapp/bind-mounts/447ea419-abc1-4ded-b6e9-4394db4af201/vol_o5b37ak8p557isds Post this as a reply to your existing P1001 thread at the link above. The diagnostic evidence is strong — it narrows the issue from "database state" to "infrastructure-level volume mount problem". The Railway team and community can see your history and build on the earlier diagnosis. Paste your scratchpad comment directly, or refine it further if you want. Either way, replying to the existing thread keeps the context intact.
10 days ago
the best i think checking withing your prisma migration if possible do a new migration of the tables and redploy the postgres again or you can also share a screenshot to really state what your issue is
10 days ago
Follow-up on the P1001 / volume-mount issue (project zippy-smile, service Postgres, id
d3f0cd18-a02e-4852-ac9e-209120cd18c5) — ran a controlled isolation test that narrows this down further.
Test: created a brand-new, completely separate project (p1001-mount-loop-test) with a fresh Postgres service, same
template family (ghcr.io/railwayapp-templates/postgres-ssl), same account, same region — no connection to zippy-smile
or its volume at all.
Result: it started up completely cleanly. Full normal init sequence in the logs — SSL cert generation, starting
PostgreSQL 18.4, listening on IPv4/IPv6/Unix socket, and database system is ready to accept connections — all within
seconds, one clean pass, no repeated mount attempts.
What this rules out: this account and region aren't experiencing a general Postgres-deploy problem right now — a fresh
volume/container on the same infrastructure works fine. That points the remaining question specifically at
zippy-smile's existing volume (postgres-volume, id db82a803-3481-4ac8-bcf2-97a51a2ae983) or its attachment to that
specific service — not at the platform/region/image broadly.
Given a fresh volume mounts fine but this specific one hangs indefinitely on Mounting volume on:
.../vol_o5b37ak8p557isds (confirmed intact at 230.75 MB, so not empty/corrupted-to-zero), is there something
inspectable on your end about that specific volume's attachment/lock state that would explain a mount hang unique to
it? Happy to provide any further diagnostics needed.
manuproject
the best i think checking withing your prisma migration if possible do a new migration of the tables and redploy the postgres again or you can also share a screenshot to really state what your issue is
10 days ago
Thanks for the suggestions!
On the Prisma migration idea — I don't think that's reachable yet, unfortunately. Migrations run from the app side
against an already-listening database, but the Postgres container itself never gets that far right now: it's stuck
before Postgres even starts, at the volume-mount step (see screenshot). There's no live database to migrate against
yet, so I don't think a migration attempt would tell us anything new here.
On redeploying again — already tried that (that's what produced the state in the screenshot below), so wanted to dig a
bit further before just repeating it blind.
Screenshot attached — Deploy Logs for the current deployment (aa28b48d), showing the container stop at 11:57:26
followed by repeated Mounting volume on: .../vol_o5b37ak8p557isds attempts that never progress to Postgres actually
starting. One thing worth flagging from the timestamps themselves: the gaps between retries aren't constant, they
trend upward (~36s → 62s → 39s → 46s → 63s → 94s → 231s) — looks more like a backoff pattern from a failed acquire
than a flat hang, which points toward a lock/attachment issue on that specific volume rather than the container just
being slow.
To narrow it down further, I also spun up a completely separate, brand-new Postgres service in an isolated test
project (same account, same region, same template image) with no connection to this one at all — it started up
completely cleanly in seconds, full normal init through "database system is ready to accept connections." So this
isn't an account-wide or region-wide issue right now; it's specific to this service's existing volume
(postgres-volume, db82a803-3481-4ac8-bcf2-97a51a2ae983) or its attachment.
Given that, is there a way to check whether this specific volume has a stale/orphaned attachment left over from the
previous container instance that stopped right before this? That would explain both the mount hang and the growing
retry backoff. Happy to provide anything else that would help.
Attach the screenshot you already grabbed when you post it. Same as before, I'll leave the actual posting to you.
Attachments
seamusmacasgaill-debug
Thanks for the suggestions! On the Prisma migration idea — I don't think that's reachable yet, unfortunately. Migrations run from the app side against an already-listening database, but the Postgres container itself never gets that far right now: it's stuck before Postgres even starts, at the volume-mount step (see screenshot). There's no live database to migrate against yet, so I don't think a migration attempt would tell us anything new here. On redeploying again — already tried that (that's what produced the state in the screenshot below), so wanted to dig a bit further before just repeating it blind. Screenshot attached — Deploy Logs for the current deployment (aa28b48d), showing the container stop at 11:57:26 followed by repeated Mounting volume on: .../vol_o5b37ak8p557isds attempts that never progress to Postgres actually starting. One thing worth flagging from the timestamps themselves: the gaps between retries aren't constant, they trend upward (~36s → 62s → 39s → 46s → 63s → 94s → 231s) — looks more like a backoff pattern from a failed acquire than a flat hang, which points toward a lock/attachment issue on that specific volume rather than the container just being slow. To narrow it down further, I also spun up a completely separate, brand-new Postgres service in an isolated test project (same account, same region, same template image) with no connection to this one at all — it started up completely cleanly in seconds, full normal init through "database system is ready to accept connections." So this isn't an account-wide or region-wide issue right now; it's specific to this service's existing volume (postgres-volume, db82a803-3481-4ac8-bcf2-97a51a2ae983) or its attachment. Given that, is there a way to check whether this specific volume has a stale/orphaned attachment left over from the previous container instance that stopped right before this? That would explain both the mount hang and the growing retry backoff. Happy to provide anything else that would help. --- Attach the screenshot you already grabbed when you post it. Same as before, I'll leave the actual posting to you. 
10 days ago
good isolation work, this really does look like a stale attachment on that specific volume, the growing backoff matches a storage backend retrying a blocked attach, not a flat hang.
one thing worth trying thats different from redeploy: go into the service's volume settings and explicitly detach the volume, wait a minute, then reattach the same volume. redeploy just queues another mount attempt on top of whatever's stuck, detach/reattach might force an actual release first. your data stays on the volume either way, detaching doesnt delete it.
if that doesn't clear it, this needs staff to check the attachment record on their storage backend directly, not something visible or fixable from the dashboard
manuproject
good isolation work, this really does look like a stale attachment on that specific volume, the growing backoff matches a storage backend retrying a blocked attach, not a flat hang. one thing worth trying thats different from redeploy: go into the service's volume settings and explicitly detach the volume, wait a minute, then reattach the same volume. redeploy just queues another mount attempt on top of whatever's stuck, detach/reattach might force an actual release first. your data stays on the volume either way, detaching doesnt delete it. if that doesn't clear it, this needs staff to check the attachment record on their storage backend directly, not something visible or fixable from the dashboard
10 days ago
Tried the detach/reattach approach — thank you, that was worth doing, but it didn't clear the issue. Wanted to report
back with exactly what happened.
Detached postgres-volume (db82a803-3481-4ac8-bcf2-97a51a2ae983) from the Postgres service, waited 60s, reattached it.
This did produce a genuinely new attachment: the bind-mount path in the deploy logs changed from
.../bind-mounts/447ea419-abc1-4ded-b6e9-4394db4af201/vol_o5b37ak8p557isds to
.../bind-mounts/c7e1120a-dec0-4645-b1d5-d95a05e1c639/vol_o5b37ak8p557isds — so it's not replaying the old stuck
attachment, this is a fresh one.
Same symptom recurred on the fresh attachment. New deployment b2c2b7bb-d0fa-4ed5-945b-f91d9c7dba0b is repeating the
identical Mounting volume on: .../vol_o5b37ak8p557isds loop — 5+ times over about a minute of watching, still
DEPLOYING, never progressing to a Postgres startup log line, same shape as the original stuck deployment.
Given a brand-new attachment record hits the exact same wall, that rules out "stale attachment record" specifically —
this looks like it's the volume's underlying storage backend itself, not anything that detach/reattach (or presumably
another redeploy) can clear from our side.
Per your note, this sounds like the point where it needs someone on your side to check the attachment/lock state on
the storage backend directly for postgres-volume (db82a803-3481-4ac8-bcf2-97a51a2ae983) — could you escalate this to
staff with that level of access? Happy to provide the full deployment logs for both attempts (aa28b48d original,
b2c2b7bb post-reattach) if useful.
seamusmacasgaill-debug
Tried the detach/reattach approach — thank you, that was worth doing, but it didn't clear the issue. Wanted to report back with exactly what happened. Detached postgres-volume (db82a803-3481-4ac8-bcf2-97a51a2ae983) from the Postgres service, waited 60s, reattached it. This did produce a genuinely new attachment: the bind-mount path in the deploy logs changed from .../bind-mounts/447ea419-abc1-4ded-b6e9-4394db4af201/vol_o5b37ak8p557isds to .../bind-mounts/c7e1120a-dec0-4645-b1d5-d95a05e1c639/vol_o5b37ak8p557isds — so it's not replaying the old stuck attachment, this is a fresh one. Same symptom recurred on the fresh attachment. New deployment b2c2b7bb-d0fa-4ed5-945b-f91d9c7dba0b is repeating the identical Mounting volume on: .../vol_o5b37ak8p557isds loop — 5+ times over about a minute of watching, still DEPLOYING, never progressing to a Postgres startup log line, same shape as the original stuck deployment. Given a brand-new attachment record hits the exact same wall, that rules out "stale attachment record" specifically — this looks like it's the volume's underlying storage backend itself, not anything that detach/reattach (or presumably another redeploy) can clear from our side. Per your note, this sounds like the point where it needs someone on your side to check the attachment/lock state on the storage backend directly for postgres-volume (db82a803-3481-4ac8-bcf2-97a51a2ae983) — could you escalate this to staff with that level of access? Happy to provide the full deployment logs for both attempts (aa28b48d original, b2c2b7bb post-reattach) if useful.  
10 days ago
confirms it then
new attachment, same wall, so it's not a stale record, it's something at the storage backend level (lock or corrupted volume metadata) that only staff can see or clear. nothing left to try from our side, this needs someone with backend access to check the attachment/lock state for volume db82a803-3481-4ac8-bcf2-97a51a2ae983 directly.
seamusmacasgaill-debug
Tried the detach/reattach approach — thank you, that was worth doing, but it didn't clear the issue. Wanted to report back with exactly what happened. Detached postgres-volume (db82a803-3481-4ac8-bcf2-97a51a2ae983) from the Postgres service, waited 60s, reattached it. This did produce a genuinely new attachment: the bind-mount path in the deploy logs changed from .../bind-mounts/447ea419-abc1-4ded-b6e9-4394db4af201/vol_o5b37ak8p557isds to .../bind-mounts/c7e1120a-dec0-4645-b1d5-d95a05e1c639/vol_o5b37ak8p557isds — so it's not replaying the old stuck attachment, this is a fresh one. Same symptom recurred on the fresh attachment. New deployment b2c2b7bb-d0fa-4ed5-945b-f91d9c7dba0b is repeating the identical Mounting volume on: .../vol_o5b37ak8p557isds loop — 5+ times over about a minute of watching, still DEPLOYING, never progressing to a Postgres startup log line, same shape as the original stuck deployment. Given a brand-new attachment record hits the exact same wall, that rules out "stale attachment record" specifically — this looks like it's the volume's underlying storage backend itself, not anything that detach/reattach (or presumably another redeploy) can clear from our side. Per your note, this sounds like the point where it needs someone on your side to check the attachment/lock state on the storage backend directly for postgres-volume (db82a803-3481-4ac8-bcf2-97a51a2ae983) — could you escalate this to staff with that level of access? Happy to provide the full deployment logs for both attempts (aa28b48d original, b2c2b7bb post-reattach) if useful.  
10 days ago
Are there any error message before "Mounting volume" logs?
mayori
Are there any error message before "Mounting volume" logs?
10 days ago
No — nothing before it, checked two ways.
The screenshot I posted earlier shows the full transition: routine autovacuum worker took too long to start; canceled
warnings from the old container (normal background activity, not an error), then Stopping Container, then straight
into Mounting volume on: ... for the new one. No error/exception line in between.
I also independently re-pulled the deploy logs for the original deployment (aa28b48d) via the CLI with a wide window —
the retrievable log stream for that deployment starts right at the first Mounting volume on: line and ends at
Stopping Container once retries were exhausted (matches restartPolicyMaxRetries: 10 on this service). Build logs are
empty, as expected for a pre-built template image with no build step.
So as far as anything visible to us goes, the mount attempt is the very first thing logged — no preceding error
message. If there's an earlier infra-level step (image pull, instance allocation) that lives in a log category we
don't have visibility into, that's outside what I can check from this side — happy to pull anything else that would
help narrow it down.
P.S. — one more data point, tying back to the Aug 10 US-West incident I mentioned a couple of replies up. Just
confirmed directly from this volume's own Settings page (not just the CLI config) that its region is US West
(California, USA) — same region as that connectivity incident. Volume size/mount path/alert thresholds on that page
all look completely standard, nothing unusual configured, and current usage is trivial against the 5GB cap, so it's
not a capacity issue either. Given the attachment-record-level check you mentioned needs staff access we don't have,
would still love to know if this volume specifically fell in that incident's blast radius — happy to be told it's
unrelated, just trying to close the loop on it.
manuproject
confirms it then new attachment, same wall, so it's not a stale record, it's something at the storage backend level (lock or corrupted volume metadata) that only staff can see or clear. nothing left to try from our side, this needs someone with backend access to check the attachment/lock state for volume db82a803-3481-4ac8-bcf2-97a51a2ae983 directly.
10 days ago
thanks
mayori
Are there any error message before "Mounting volume" logs?
10 days ago
Following up — thanks again for the help narrowing this down. We've now confirmed no error precedes the mount
attempts, tried the detach/reattach approach (same symptom recurred on a genuinely fresh attachment), and confirmed
via an isolated test that it's not an account/region-wide issue.
Given this now looks like it needs someone with direct access to the storage backend's attachment records, do you have
any sense of timeline for that kind of escalation, or is there anything further we should do on our end in the
meantime? Production's been in a safe-but-degraded state (maintenance mode) for a while now, so any indication of next
steps would help us plan.
seamusmacasgaill-debug
Following up — thanks again for the help narrowing this down. We've now confirmed no error precedes the mount attempts, tried the detach/reattach approach (same symptom recurred on a genuinely fresh attachment), and confirmed via an isolated test that it's not an account/region-wide issue. Given this now looks like it needs someone with direct access to the storage backend's attachment records, do you have any sense of timeline for that kind of escalation, or is there anything further we should do on our end in the meantime? Production's been in a safe-but-degraded state (maintenance mode) for a while now, so any indication of next steps would help us plan.
9 days ago
Hi @mayori — could you please provide a status update or confirm that this has been escalated for infrastructure review?
This is a production Postgres service currently held in maintenance mode. We have isolated the failure to the Railway volume-mount stage:
no application, image, build, or pre-mount error precedes the failure;
the same Mounting volume on ... loop occurs after a detach/reattach which generated a new attachment path; the container never reaches a Postgres startup log line; the volume is in US West, and usage is well below its 5 GB limit.
The affected volume is postgres-volume (db82a803-3481-4ac8-bcf2-97a51a2ae983); relevant deployments are aa28b48d and b2c2b7bb-d0fa-4ed5-945b-f91d9c7dba0b.
Could someone with access to the platform storage layer please inspect the volume/attachment state and advise:
whether the volume is recoverable and safe to retry; whether its storage backend or node has an outstanding lock, attachment, or metadata problem; and the expected next update or escalation timeframe?
I can provide the full deploy logs immediately. Thank you.