Ongoing 499 Timeout Errors Across All Projects in US-WEST
tylerburr
PROOP

a year ago

I mentioned this previously with the 499 error, but we're still encountering it across all of our projects. All of our projects are hosted in US-WEST. I initially thought it was just the core API endpoint, but overnight we experienced the same timeout error across everything.

25 Replies

tylerburr
PROOP

a year ago

This issue has only started occurring recently and has never happened before.


a year ago

project Id is very applicable here please


tylerburr
PROOP

a year ago

1574cc15-2e31-4ab0-b5ee-ccca94ab4e7c


tylerburr
PROOP

a year ago

f32c7086-f44b-4b6c-94b8-189b0c4e1555


tylerburr
PROOP

a year ago

897abfe5-d734-41af-bc55-ee9941db3811


tylerburr
PROOP

a year ago

I currently monitor these using Uptime Kuma and a NodePing server, and both report errors at the exact same time.


tylerburr
PROOP

a year ago

1319023318267068400


a year ago

can you tell me what those IDs are for exactly?


tylerburr
PROOP

a year ago

They are the project IDs that NodePing and Uptime Kuma monitor.


a year ago

okay thank you, will look into this when I'm back at my laptop


tylerburr
PROOP

a year ago

Thanks! Let me know— I can move the projects to a different region to see if that resolves the issue. I'll wait until you let me know how you'd like to proceed.


a year ago

for the first app, it looks like you are running into limitations with how you have your app configured, blocking code, not enough uvicorn workers, etc


tylerburr
PROOP

a year ago

Ok I’ll look into that.


tylerburr
PROOP

a year ago

Did you find anything else with the other projects?


tylerburr
PROOP

a year ago

The only reason I'm asking is that I'm experiencing timeouts on all of them around the same time.


a year ago

are they still happening right now?


tylerburr
PROOP

a year ago

5:46pm was the last time


tylerburr
PROOP

a year ago

MST


a year ago

we haven't had any issues during that time, so it would have to be issues with your applications


tylerburr
PROOP

a year ago

I’ll keep looking but all projects at the same time having the same issue?


tylerburr
PROOP

a year ago

2:22am MST all projects reported timeouts again.


tylerburr
PROOP

a year ago

2:24am MST got more alerts.


a year ago

all green from our side on the monitors


tylerburr
PROOP

a year ago

Hmm, alright. I'll continue investigating and run some additional tests. The code has been running smoothly for over a year, but this issue has only started occurring in the past month and a half.


a year ago

sounds good, let me know what you find!


Loading...