Incredibly slow inference - totally randomly
adengitstuff
HOBBYOP
12 days ago
Hello,
I have a script that tests return times on a simple text classification task. I signed up for the Hobby plan yesterday. I'm aiming for 20ms - 35ms return times. I'd see occasional spikes to around 135ms, but that's okay every now and then. This is ignoring the cold start times. Most things I got were around 35ms
I'm not seeing a simple task that I'm running take 6.7+ seconds for each request. I don't know what's going on.
1 Replies
Status changed to Open Railway • 12 days ago
12 days ago
Hey, can you try changing your service region to see if it returns to the usual latency?