Hi - as of about an hour ago, all builds on our Node.js web apps which use Puppeteer (and therefore Chromium) are failing.

I tried deploying a version with rolled back code to ensure the problem wasn't introduced on our end, but the builds still fail. (None of our recent updates have changed how Puppeteer is used - but I figured it was worth testing anyway, in case recent changes meant we now hit some memory limits or similar)

I did see in the logs that it was possible the version of Chromium used by Railway was upgraded under the hood? It seemed to be downloading a new version. Though maybe it does that on every deploy. It does seem to take longer to build now, beore it fails (6+ minutes instead of 1-3 minutes).

It may also be due to the Ubuntu servers being down. I can see in the logs that it says "Cannot initiate the connection to archive.ubuntu.com". (Though maybe this is a "follow-on effect" from a previous step e.g. Chromium failing)

I've attached a snippet from the logs and "linked" one of the relevant projects for admins.

Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libg/libglu/libglu1-mesa_9.0.2-1.1build1_amd64.deb Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::101). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::103). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4002:1::102). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4002:1::101). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::102). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4002:1::103). - connect (101: Network is unreachable) [IP: 185.125.190.81 80] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

✕ [stage-0 5/11] RUN sudo apt-get update && sudo apt-get install -y --no-install-recommends libnss3 libatk1.0-0 libatk-bridge2.0-0 libcups2 libgbm1 libasound2t64 libpangocairo-1.0-0 libxss1 libgtk-3-0 libxshmfence1 libglu1 chromium

process "/bin/bash -ol pipefail -c sudo apt-get update && sudo apt-get install -y --no-install-recommends libnss3 libatk1.0-0 libatk-bridge2.0-0 libcups2 libgbm1 libasound2t64 libpangocairo-1.0-0 libxss1 libgtk-3-0 libxshmfence1 libglu1 chromium" did not complete successfully: exit code: 100

Dockerfile:9

-------------------

7 | COPY .nixpacks/nixpkgs-<id>.nix .nixpacks/nixpkgs-<id>.nix

8 | RUN nix-env -if .nixpacks/nixpkgs-<id>.nix && nix-collect-garbage -d

9 | >>> RUN sudo apt-get update && sudo apt-get install -y --no-install-recommends libnss3 libatk1.0-0 libatk-bridge2.0-0 libcups2 libgbm1 libasound2t64 libpangocairo-1.0-0 libxss1 libgtk-3-0 libxshmfence1 libglu1 chromium

10 |

11 | ARG BUILDKIT_DOCKERFILE_CHECK CI <env keys>

-------------------

ERROR: failed to build: failed to solve: process "/bin/bash -ol pipefail -c sudo apt-get update && sudo apt-get install -y --no-install-recommends libnss3 libatk1.0-0 libatk-bridge2.0-0 libcups2 libgbm1 libasound2t64 libpangocairo-1.0-0 libxss1 libgtk-3-0 libxshmfence1 libglu1 chromium" did not complete successfully: exit code: 100

Error: Docker build failed