Builder: Metal builder "builder-wkkdbn" fails on new image builds
martinpink
PROOP

8 days ago

Service: syncora-workers (project: syncora, region: US West)

Builder: Metal builder "builder-wkkdbn"

Seven consecutive deploys of this service have failed at the

snapshot-target-unpack step with:

"Build Failed: build daemon returned an error <

failed to receive status: rpc error: code = NotFound

desc = no such job [job-id] >"

All seven routed to builder-wkkdbn over ~45 minutes. The sister

service syncora-web in the same project deploys cleanly on a

different builder.

Failed deployment IDs (most recent first):

- 7715968 (12:00 UTC) — Add test-contact-mining.ts

- 252855a (11:50 UTC) — bump build marker

- 59e7278 (11:30 UTC) — empty commit

- b475f63e (11:30 UTC)

- b79a8cf9 (11:26 UTC)

- cd306e68 (11:23 UTC)

- 47914ce2 (11:23 UTC)

- c14927aa (11:18 UTC)

builder-wkkdbn appears to be holding stuck state for this service.

Please re-route syncora-workers to a different builder, or reset

the daemon state on builder-wkkdbn.

Solved

2 Replies

Status changed to Awaiting Railway Response Railway 8 days ago


martinpink
PROOP

8 days ago

issue still persists

Issue: syncora-workers (project: syncora) cannot deploy. Eight

consecutive build failures at snapshot-target-unpack, all routed to

Metal builder "builder-wkkdbn". The sister service syncora-web in

the same project deploys cleanly on a different builder.

Just upgraded to Pro today — opening this ticket via paid support.

Error on every attempt:

"Build Failed: build daemon returned an error <

failed to receive status: rpc error: code = NotFound

desc = no such job [job-id] >"

Failed deployment IDs over ~90 minutes:

c14927aa, 47914ce2, cd306e68, b79a8cf9, b475f63e,

59e7278 (empty commit), 252855a (build-marker bump),

7715968 (added tests), 953cd3a (post-upgrade trigger)

Builder-wkkdbn appears to have stuck daemon state. Please:

(a) re-route syncora-workers to a different builder, OR

(b) reset the daemon state on builder-wkkdbn.

The currently-running active deployment (8adfa955) is healthy and

serving traffic — this is a "new deploys cannot land" problem, not

a "service is down" problem. But we have one customer-blocking

bug that the queued fix would resolve, so reasonably urgent.


sam-a
EMPLOYEE

8 days ago

Thanks for the detailed report. We had a build infrastructure issue affecting some services this morning. We've mitigated it on our side. Please trigger a fresh deploy of syncora-workers and it should go through.


Status changed to Awaiting User Response Railway 8 days ago


Status changed to Solved martinpink 8 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...