Serverless apps drop the first request
latifs
PROOP

a year ago

Hi guys,
I have a couple apps and obviously I'd like to enable Serverless to save some money.
One is fastAPI that runs Unstructured Pipelines to parse data from files and another one is a LangServe to serve Runnables as endpoints.

In both these apps the first requests just fail until the apps fully wake up which obviously defeats the whole purpose of serverless.

Is there something I'm not doing right?

Could you point me to a solution?

Thanks

Solved

4 Replies

Railway
BOT

a year ago

This thread has been escalated internally. We'll be in touch with you once we hear back from the team.

Status changed to In Progress Railway 11 months ago


itsrems
EMPLOYEE

a year ago

Heya, seems your server is taking too long and we consider the app as failing to respond even though it's booting. I've raised this internally to see how we can improve this.

thanks !


Status changed to Awaiting Railway Response Railway 11 months ago


itsrems
EMPLOYEE

a year ago

The maximum time we wait for is 10s. If you need to run long functions at the start of your service (ex: pulling data, etc), you should prep and cache those in a predeploy step.


Status changed to Awaiting User Response Railway 11 months ago


Status changed to Solved itsrems 11 months ago


evhaus
PRO

10 months ago

I'm having the same problem as well. The first fetch() request is simply failing with Unable to connect. Is the computer able to access the url?. I am under the Railway's Private Networking to connect to this particular service. Is there perhaps a bug where Serverless doesn't work with Private Networking? The failure happens faster than 10s. It's nearly immediate.


Status changed to Awaiting Railway Response Railway 11 months ago


evhaus

I'm having the same problem as well. The first fetch() request is simply failing with Unable to connect. Is the computer able to access the url?. I am under the Railway's Private Networking to connect to this particular service. Is there perhaps a bug where Serverless doesn't work with Private Networking? The failure happens faster than 10s. It's nearly immediate.

brody
EMPLOYEE

10 months ago

You would need to implement retries in your code for communication privately with services that go to sleep, while our public proxy will hold the request for up to 10 seconds, the same does not happen over the private network, so it is up to your code to handle that.


Status changed to Awaiting User Response Railway 10 months ago


Railway
BOT

6 months ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway 6 months ago


Loading...