Nothing wrong with this build but healthcheck failed.
rogustcarthon
PROOP

2 years ago

Health check failed but the logs show that the server was up. The same build when redeployed is working correctly. Something wrong with the healthcheck system?

View Deploy details

ⓘ Deployment information is only viewable by project members and Railway employees.

16 Replies

brody
EMPLOYEE

2 years ago

Why did the health check fail?


rogustcarthon
PROOP

2 years ago

Looks like it just timed out. When I increased the healthcheck duration to 5 mins, a couple of the subsequent builds took too long to deploy successfully, so had to redeploy them. Avg deploy time is ~1 minute.


brody
EMPLOYEE

2 years ago

Looks like? I would appreciate it if you could tell me the reason as printed by the health check logs.


rogustcarthon
PROOP

2 years ago

Here's healthcheck status from the buildlogs

Attempt #1 failed with service unavailable. Continuing to retry for 49s
Attempt #2 failed with service unavailable. Continuing to retry for 38s
Attempt #3 failed with service unavailable. Continuing to retry for 25s
Attempt #4 failed with service unavailable. Continuing to retry for 11s

1/1 replicas never became healthy!
Healthcheck failed!

But here are the deploy logs which basically state that the server was up

loaded configuration
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env:    export GIN_MODE=release
- using code:    gin.SetMode(gin.ReleaseMode)
[GIN-debug] GET    /health                   --> reply_rocket/internal/handler.HealthCheck (6 handlers)
.
.
.
[GIN-debug] GET    /follows                  -->  (6 handlers)
2024/05/29 06:42:10 7 :8080

Also, the same commits which are facing this healthcheck issue on production are successfully deploying on staging without any redeployments.


brody
EMPLOYEE

2 years ago

Please have your GIN server listen on the PORT environment variable.


rogustcarthon
PROOP

2 years ago

I'm pretty sure its the same, haven't changed anything.
The same commit when redeployed (without any changes to env) doesn't fail again.


rogustcarthon
PROOP

2 years ago

Here's a log from a successful deployment.

2024/05/29 08:37:13 7 :8080

brody
EMPLOYEE

2 years ago

I'm pretty sure its the same

Please check, and if needed make the necessary changes.


rogustcarthon
PROOP

2 years ago

Checked, its the same.


brody
EMPLOYEE

2 years ago

Is your GIN server listening on the PORT environment variable? (y/N)


rogustcarthon
PROOP

2 years ago

y


brody
EMPLOYEE

2 years ago

Please show the applicable code.


rogustcarthon
PROOP

2 years ago

Here's the server starting:

        if err := endless.ListenAndServe(fmt.Sprintf(":%s", cfg.Port), r); err != nil {
            lg.Error().Err(err).Msg("server closed")
        }

Here's how cfg.Port references the env

type Server struct {
    Port        string            `envconfig:"PORT,optional" default:"8080"`

brody
EMPLOYEE

2 years ago

What is your PORT service variable set to?


rogustcarthon
PROOP

2 years ago

8080


diarrisso
FREE

14 days ago

ttempt #8 failed with service unavailable. Continuing to retry for 27s

1/1 replicas never became healthy!


Loading...