Railpack BuildKit frontend grpc server crash blocks production deploy

9 days ago

We are blocked on a production rollout today. Both API and Web deployments fail during the build image phase before our app commands run.

Project: Rigtch

Project ID: 822c34b7-cd29-4ab9-8a4e-6a8e6b48868f

Environment: production

Environment ID: 4225f711-f7f5-494b-aae2-2e71e71a0115

Failed deployments:

API:

  • 75a2d901-ae70-468d-8015-a447e2962ada
  • b5d39821-caac-4631-9867-bb9513a35a3b

Web:

  • 181bf1fa-a0ec-45cc-abe7-e0407e2ef718
  • 74f73d02-59b5-456a-a012-7ddaed5ee9fc

Builders:

  • builder-tisulz
  • builder-zbypni

Both services fail before bun install --frozen-lockfile starts. The logs stop while resolving the Railpack frontend image:

resolve image config for docker-image://ghcr.io/railwayapp/railpack-frontend:v0.30.0

docker-image://ghcr.io/railwayapp/railpack-frontend:v0.30.0@sha256:8120546d9e67be53c7edebec900ed69ef724fbde890d43cb24455dfca4aebcc0

Build Failed: build daemon returned an error < failed to solve: frontend grpc server closed unexpectedly >

This looks like Railpack/BuildKit builder infrastructure rather than our app build, because no repo command runs and it happens for two separate services/builders.

Could you please investigate urgently? We planned to run this production rollout today.

image.png

image.png

Solved

7 Replies

Status changed to Awaiting Railway Response Railway 9 days ago


9 days ago

Adding urgency/context: this blocks production rollout today. We retried API twice and Web once; both still fail before bun install while resolving railpack-frontend:v0.30.0. Current production is online, but we cannot ship the release.


9 days ago

Update: we unblocked production by temporarily switching both API and Web from Railpack to Nixpacks.

The original failure still looks like a Railpack/builder issue, because builds failed before app install/build while resolving:

ghcr.io/railwayapp/railpack-frontend:v0.30.0

with:

frontend grpc server closed unexpectedly

After switching to Nixpacks, the deploy reached the build plan. We then had to pin Node to >=22.15.0 because Nixpacks selected Node 18, which is EOL/removed from Nix. After that, deployment succeeded.

So production is unblocked, but please still investigate the Railpack failure. We would like to know whether Railpack v0.30.0 is currently safe to use again, or whether we should stay on Nixpacks for now.


8 days ago

Hey!

Not at all ideal, could you link one of the failing railpack deploys? Or even switch back to railpack and see if it was something transient? Would love to pinpoint exactly what it is


Status changed to Awaiting User Response Railway 8 days ago


8 days ago

Here is one failing Railpack deploy:

https://railway.com/project/822c34b7-cd29-4ab9-8a4e-6a8e6b48868f/service/892922a4-23d0-42b8-a4b9-8d07f70e7ac5?environmentId=4225f711-f7f5-494b-aae2-2e71e71a0115&id=75a2d901-ae70-468d-8015-a447e2962ada

It failed during Railpack image build with:

Build Failed: build daemon returned an error < failed to solve: frontend grpc server closed unexpectedly >

I’d prefer not to switch back to Railpack until you can confirm it looks stable or that the issue was transient.


Status changed to Awaiting Railway Response Railway 8 days ago


8 days ago

I attached a ticket so we can track this issue best. If I have any updates will forward them your way


Status changed to Awaiting User Response Railway 8 days ago


8 days ago

Heard back from our platform team, looks like this was a transient builder issue. You shouldnt run into it again.


noahd

Heard back from our platform team, looks like this was a transient builder issue. You shouldnt run into it again.

7 days ago

Yes it works now.


Status changed to Awaiting Railway Response Railway 7 days ago


Status changed to Solved mnigos 7 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...