Load Balancer Issue: Resource‑Based Routing Breaks High‑Concurrency Workloads (Round‑Robin Needed)
iamyegor
HOBBYOP

4 months ago

I have an issue related to how your load balancer distributes traffic across replicas.

When I scale a service to 50 replicas and start sending requests, it looks like the load balancer uses some kind of resource‑based routing. It seems to send traffic to one replica until it reaches a high CPU/RAM usage threshold, and only then switches to another replica.

For workloads like browser spawning, this behavior is extremely inefficient. Spawning a browser takes time, so when I send, for example, 50 simultaneous requests to create browser instances, almost all of them are routed to the first replica. As a result, that replica becomes overloaded and crashes. This problem combines with the resource allocation issues I described in my other post.

If the load balancer used a simple round‑robin strategy, it would distribute each incoming request evenly across all replicas regardless of their momentary resource usage. This would completely solve the issue for my case.

I tried writing my own load balancer with round‑robin logic, but I couldn’t get it to work correctly in your environment. I also found a template with a custom round‑robin load balancer, but the repository was removed.

Do you have any recommendations or supported approaches for implementing proper round‑robin request distribution on Railway? Any guidance would help a lot.

Solved

5 Replies

Railway
BOT

4 months ago

Hey there! We've found the following might help you get unblocked faster:

If you find the answer from one of these, please let us know by solving the thread!


4 months ago

Hello,

We are going to need some key information here in order to help, you mentioned sending 50 requests, are these requests via the public HTTP proxy or via the private network?


Status changed to Awaiting User Response Railway 4 months ago


brody

Hello,We are going to need some key information here in order to help, you mentioned sending 50 requests, are these requests via the public HTTP proxy or via the private network?

iamyegor
HOBBYOP

4 months ago

Hi, it's via the private network


Status changed to Awaiting Railway Response Railway 4 months ago


4 months ago

Via the private network, there is no load balancer. Instead, individual replicas are exposed via the DNS lookup your application performs. For example, if you have 5 replicas and do a DNS lookup for its private domain, you will get 5 IPs back.

The problem with this is that even though we randomize the order for every DNS lookup, some languages or libraries will still prefer a single IP from the list. In these cases, you can get around this by just doing a DNS lookup, and then randomly picking an IP to send the request to.


Status changed to Awaiting User Response Railway 4 months ago


iamyegor
HOBBYOP

4 months ago

Thank you for your answer


Status changed to Awaiting Railway Response Railway 4 months ago


Status changed to Solved iamyegor 4 months ago


Loading...