Requests to our API are failing intermittently
i8ramin
PROOP

2 years ago

We have a /status check on our API hosted on Railway. This status check has been failing intermittently for the past few hours. It is not clear what the issue is.

Direct to Railway (bypass custom domain)
https://api-dupe-com.up.railway.app/status

Custom domain (fails and recovers the same way)

http://api.dupe.com/status

The /status endpoint simply returns a 200 OK message and works just fine locally, so not a code issue. Seems like a network or request origin issue. We are using the new Edge proxy on Railway and have had issues with it before. Possibly related?

Solved

5 Replies

i8ramin
PROOP

2 years ago

I have duplicated our us-east API service to us-west servers and pointing our custom domain to this new instance. This seems to have fixed the issue until we know more about why the original (us-east) one is failing


i8ramin
PROOP

2 years ago

We have redeployed our service on the us-east servers (no code change), and it seems to be working better now. Let's see if it stays that way. Still unclear what the root cause of the issue was/is


i8ramin
PROOP

2 years ago

BTW, issuing a "restart" did not solve the problem for us. We had to do a "redeploy" for this issue to be resolved. We are still not sure what the root cause of the issue was (which isn't ideal). So any insight or input from the Railway team (if they have any info on their end via logs etc), would be greatly appreciated, so that we can add it to our postmortem! Thank you


Hey there Ramin,

I tagged you in Slack just in case. But these aren't insanely long requests right? Just a simple healthcheck?

Do you also have request logs you can send over so we can try chasing them down?

Thanks,
Angelo


Status changed to Awaiting User Response Railway over 1 year ago


Upon further investigation we also see that your app seems to be crash looping?


We can also help you set up Grafana or DataDog on your application to give you more traces into your application.

Attachments


Railway
BOT

7 months ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway 7 months ago


Loading...