Consistently unable to apt install and connect to archive.ubuntu.com with nixpacks

vjindal0112
PRO

2 months ago

As of a few days ago, the apt install had been cached and was easily going through.

As of the last 5-6 deploys, it's no longer cached but it's hitting network errors when trying to download apt packages.

I'm getting errors like this:

Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::101). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4002:1::103). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::102). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::103). - connect (101: Network is unreachable) Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4002:1::102). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (185.125.190.82), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.81), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.82), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.83), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.83), connection timed out

I've gotten this consistently for the last several deploys so it's not a transient error.

This is blocking all deployments for our company, any help would be appreciated

Perhaps it has to do with ipv6 egress not being allowed in railway?

Solved

14 Replies

pioneerfinance
PRO

2 months ago

I'm getting this same error, network issues getting

"sudo apt-get update && sudo apt-get install -y --no-install-recommends libnss3 libatk1.0-0 libatk-bridge2.0-0 libcups2 libgbm1 libasound2t64 libpangocairo-1.0-0 libxss1 libgtk-3-0 libxshmfence1 libglu1 chromium libnss3 libatk1.0-0 libatk-bridge2.0-0 libcups2 libgbm1 libasound2t64 libpangocairo-1.0-0 libxss1 libgtk-3-0 libxshmfence1 libglu1 chromium" did not complete successfully: exit code: 100

+11

vjindal0112
PRO

2 months ago

Previous builds that had worked are now failing so this is a new issue

vjindal0112
PRO

2 months ago

Full outage as a result of this

brody
EMPLOYEE

2 months ago

Hello,

We will be looking into this, though, at this time we are not sure if this is a problem on our side or an intermittent issues with canonical's archive.

eyes1

Status changed to Awaiting User Response railway[bot] • about 2 months ago

brody

Hello,We will be looking into this, though, at this time we are not sure if this is a problem on our side or an intermittent issues with canonical's archive.

vjindal0112
PRO

2 months ago

Thanks brody.

I just checked and it looks like builds are still failing today as well. And seems like others are encountering this as well: https://station.railway.com/questions/deployment-failure-97a82e11

I'll do some looking into canonical's archive!

Status changed to Awaiting Railway Response railway[bot] • about 2 months ago

vjindal0112
PRO

2 months ago

oh I see, there's a major outage: https://status.canonical.com/

I wonder if that's the cause

brody
EMPLOYEE

2 months ago

They say archive.ubuntu.com is Operational, but that is definitely not the case, there is indeed a partial outage on their side.

Status changed to Awaiting User Response railway[bot] • about 2 months ago

vjindal0112
PRO

2 months ago

Two questions:

  1. Is there any way to force railway to install using one of the mirrors instead while still using nixpacks?

  2. Anything you'd recommend to avoid this issue in the future?

Status changed to Awaiting Railway Response railway[bot] • about 2 months ago

brody
EMPLOYEE

2 months ago

  1. You would have to find a way to modify the sources.list file, that is not functionality Nixpacks provides natively, so I am unable to provide any additional information in that regard.

  2. We don't have anything to recommend here as this issue is completely out of our control.

Status changed to Awaiting User Response railway[bot] • about 2 months ago

brody

You would have to find a way to modify the sources.list file, that is not functionality Nixpacks provides natively, so I am unable to provide any additional information in that regard.We don't have anything to recommend here as this issue is completely out of our control.

vjindal0112
PRO

2 months ago

  1. is there no way to configure a nixpacks.toml with railway? https://nixpacks.com/docs/configuration/file -- seems like I could use this to configure the sources.list file?

Status changed to Awaiting Railway Response railway[bot] • about 2 months ago

vjindal0112

is there no way to configure a nixpacks.toml with railway? https://nixpacks.com/docs/configuration/file -- seems like I could use this to configure the sources.list file?

vjindal0112
PRO

2 months ago

Actually seems like I might be able to use an environment variable here: https://nixpacks.com/docs/configuration/environment

brody
EMPLOYEE

2 months ago

Modifying the sources.list file is not a configuration that Nixpacks provides, neither via toml or environment variables.

Status changed to Awaiting User Response railway[bot] • about 2 months ago

brody

Modifying the sources.list file is not a configuration that Nixpacks provides, neither via toml or environment variables.

vjindal0112
PRO

2 months ago

could I not do something like this? 

[phases.setup]
cmds = [
  "sed -i 's|http://archive.ubuntu.com/ubuntu|mirror://mirrors.ubuntu.com/mirrors.txt|' /etc/apt/sources.list"
]

in the toml?

Status changed to Awaiting Railway Response railway[bot] • about 2 months ago

brody
EMPLOYEE

2 months ago

I'm not sure, it's not a native configuration so I would not be able to comment on whether it would work.

+11

Status changed to Awaiting User Response railway[bot] • about 2 months ago

Status changed to Solved vjindal0112 • about 2 months ago

Consistently unable to apt install and connect to archive.ubuntu.com with nixpacks - Railway Help Station