Healthcheck failed with multiple "Attempt failed with service unavailable. Continuing to retry"
chris-opendata
HOBBYOP

2 months ago

The frontend healthy check failed with mysterious messages in build logs:

==================== Starting Healthcheck ====================

Path: /

Retry window: 5m0s

Attempt #1 failed with service unavailable. Continuing to retry for 4m49s

Attempt #2 failed with service unavailable. Continuing to retry for 4m38s

Attempt #3 failed with service unavailable. Continuing to retry for 4m26s

Attempt #4 failed with service unavailable. Continuing to retry for 4m12s

Attempt #5 failed with service unavailable. Continuing to retry for 3m54s

Attempt #6 failed with service unavailable. Continuing to retry for 3m28s

Attempt #7 failed with service unavailable. Continuing to retry for 2m48s

Attempt #8 failed with service unavailable. Continuing to retry for 2m8s

Attempt #9 failed with service unavailable. Continuing to retry for 1m28s

Attempt #10 failed with service unavailable. Continuing to retry for 48s

Attempt #11 failed with service unavailable. Continuing to retry for 8s

1/1 replicas never became healthy!

Healthcheck failed!

Backend, Postgres, and Redis are online. No sure what service is about. Could anyone enlighten me?

Thanks

Solved$10 Bounty

Pinned Solution

2 months ago

Yeah, it does look like it's configured properly.
What happens if you just remove the healtcheck from your frontend service? does it deploy happily and work as expected?

6 Replies

2 months ago

Hey, this just means your front end isn't coming up after your build-

before your application deploys and replaces your existing instance, Railway runs a healthcheck if you have it configured. This ensures that your service is healthy after the new build.

It looks like you have your healthcheck configured to the path / and railway isn't able to load that page so it's marking the deploy as a failure.


mykal

Hey, this just means your front end isn't coming up after your build-before your application deploys and replaces your existing instance, Railway runs a healthcheck if you have it configured. This ensures that your service is healthy after the new build.It looks like you have your healthcheck configured to the path / and railway isn't able to load that page so it's marking the deploy as a failure.

chris-opendata
HOBBYOP

2 months ago

Thank you, mykal, for your speedy reply. The frontend Deployments Dashboard shows Initialization, build, and deploy all green-ticked so far. So, should healthcheck path for the frontend be the same as the backend one "/health"?

Also, I noticed that changing package.json didn't trigger rerunning healthcheck. Is this normal by Railway?


chris-opendata

Thank you, mykal, for your speedy reply. The frontend Deployments Dashboard shows Initialization, build, and deploy all green-ticked so far. So, should healthcheck path for the frontend be the same as the backend one "/health"?Also, I noticed that changing package.json didn't trigger rerunning healthcheck. Is this normal by Railway?

2 months ago

Nope, your front-end health-check route should be something that you expect your front-end to respond to requests on when it's in a healthy state. For most user facing web-apps, the home page (or / ) would be fine- but based on your logs it seems that your front-end service is not responding to requests on /.

As per your package.json question, no, that's not normal but it depends how you have set your frontend service up and if you actually deployed your change to GitHub. If your package.json isn't in the watch paths setup in your service settings it won't trigger a re-deploy, so it won't trigger running a healthcheck. If you're using a monorepo setup, a change in the root package.json might not trigger a redeploy of service's that rely on it without configuring watch paths.

How are you deploying your front-end? is it a static site? what frameworks are you using? that might help me get you to a solution here.


mykal

Nope, your front-end health-check route should be something that you expect your front-end to respond to requests on when it's in a healthy state. For most user facing web-apps, the home page (or / ) would be fine- but based on your logs it seems that your front-end service is not responding to requests on /.As per your package.json question, no, that's not normal but it depends how you have set your frontend service up and if you actually deployed your change to GitHub. If your package.json isn't in the watch paths setup in your service settings it won't trigger a re-deploy, so it won't trigger running a healthcheck. If you're using a monorepo setup, a change in the root package.json might not trigger a redeploy of service's that rely on it without configuring watch paths.How are you deploying your front-end? is it a static site? what frameworks are you using? that might help me get you to a solution here.

chris-opendata
HOBBYOP

2 months ago

Thank you for your explanation. It is a Next.js and Node.js application with dynamical App Router pages. It is a module-sharing monorepo project where packages/frontend and packages/backend are separate. The changes are pushed into my GitHub repo, from which the Railway is triggered and pulls files to build and deploy on its cloud platform. I am thinking if the problem is something to do with domain and port setting in the frontend.

Here is the script in the root package.json:

    "frontend:start": "cd packages/frontend/.next/standalone && HOSTNAME=0.0.0.0 PORT=$PORT node packages/frontend/server.js || HOSTNAME=0.0.0.0 PORT=$PORT node server.js"

and packages/frontend/package.json:

    "build": "next build --no-lint && mkdir -p .next/standalone/public && cp -r public/* .next/standalone/public/ && mkdir -p .next/standalone/.next/static && cp -r .next/static/* .next/standalone/.next/static/",
    "start": "node .next/standalone/server.js",

and Railway deploy logs:

Starting Container

✓ Ready in 194ms

npm warn config production Use --omit=dev instead.

> monorepo@1.0.0 frontend:start

> cd packages/frontend/.next/standalone && HOSTNAME=0.0.0.0 PORT=$PORT node packages/frontend/server.js || HOSTNAME=0.0.0.0 PORT=$PORT node server.js

▲ Next.js 15.5.3

- Local: http://localhost:8080

- Network: http://0.0.0.0:8080

✓ Starting...

Also, I configured my custom www domain with the port number 8080.

Everything seems ok to me. Please point me to where I missed or misconfigured.

Thanks


2 months ago

Yeah, it does look like it's configured properly.
What happens if you just remove the healtcheck from your frontend service? does it deploy happily and work as expected?


mykal

Yeah, it does look like it's configured properly. What happens if you just remove the healtcheck from your frontend service? does it deploy happily and work as expected?

chris-opendata
HOBBYOP

2 months ago

Thanks, with your suggestion, I did a bit search and fixed by the following steps:

  1. implemented a frontend route (/api/health) to listen to the healthcheck requests.

  2. configured the route path in the frontend toml file.

    It worked. All have been deployed successfully.

    But, the browsers reported "This site can’t be reached" when I typed my domain URL. I am investigating the new issue. As far as this problem, it is fixed.


Status changed to Solved brody about 2 months ago


Loading...