Build failing on Metal builder

chickenmatty

FREEOP

3 months ago

Just wanted to report that some builds failed because the metal builder got stuck loading metadata for the docker image, I don't know if this was a temporary issue but retrying didn't work and switching off metal builder got the build to succeed

12 Replies

medim

MODERATOR

3 months ago

Mind sharing that service id?

dannyilyas

PRO

3 months ago

I'm getting the same error! Despite no code changes, even tried redeploying the succesful build from last night

medim

MODERATOR

3 months ago

Could you open your own help thread? If you could also provide the service id and error log there so I can escalate it to the team!

dannyilyas

PRO

3 months ago

Appreciate the help @Medim! thread here: https://discord.com/channels/713503345364697088/1443784102628163727

parag7823

HOBBY

3 months ago

Hi, I am also getting the same error - PROBLEM DESCRIPTION

Your Docker build on Railway is stuck indefinitely at:

[internal] load metadata for docker.io/library/python:3.9-slim
[internal] load metadata for docker.io/library/node:20-alpine

After ~30 minutes, the build times out with:

Build Failed: build daemon returned an error < failed to run Build function: Canceled: context canceled >

ROOT CAUSE

Railway's Docker build environment has slow/blocked access to Docker Hub.

The build cannot fetch metadata for base images from Docker Hub within the timeout window.

This is NOT a code issue - it's an infrastructure/network issue with Railway's build servers.

WHAT WE ALREADY TRIED

Attempt 1: Blamed SHAP Library

What we did: Removed shap==0.43.0 from

backend-requirements.txt

Why it failed: The build was stuck BEFORE pip runs, so SHAP wasn't the issue.

Result: Didn't help - build still timed out at Docker metadata fetch.

Attempt 2: Added PyPI Mirror Optimization

What we did: Added Aliyun PyPI mirror to Dockerfile:

dockerfile

RUN mkdir -p ~/.pip && echo "[global]\nindex-url = https://mirrors.aliyun.com/pypi/simple/" > ~/.pip/pip.conf

Why it failed: The build never gets to pip install - it's stuck at base image pull.

Result: Didn't help - build still times out at Docker metadata fetch.

Attempt 3: Added BuildKit Optimization

What we did: Added BuildKit syntax to Dockerfile:

dockerfile

# syntax=docker/dockerfile:1.4

Why it failed: BuildKit helps with caching, but doesn't solve the base image pull timeout.

Result: Didn't help - build still times out at Docker metadata fetch.

Attempt 4: Verified .dockerignore

What we did: Checked

.dockerignore file (already optimized)

Result: File is good, but doesn't solve the base image pull issue.

parag7823

HOBBY

3 months ago

@Medim I am also facing the same issue

parag7823

@Medim I am also facing the same issue

medim

MODERATOR

3 months ago

Hey! Can you open your own help thread? If you can, please also share the link to your service and error logs there

parag7823

HOBBY

3 months ago

@Medim I have opened my own thread. Please look into this

tomharper

HOBBY

3 months ago

I am in the same boat as of 2 days ago

tomharper

I am in the same boat as of 2 days ago

medim

MODERATOR

3 months ago

A quick fix would be to disable Metal Builder in the service settings.

If the problem persists after disabling the Metal Builder, please open your own help thread!

tomharper

HOBBY

3 months ago

I tried disabling metal build, it just turns itself back on for some unknown reason.

tomharper

I tried disabling metal build, it just turns itself back on for some unknown reason.

brody

EMPLOYEE

3 months ago

Please open your own thread!