a month ago
Hey railway team, so currently we have a Postgres table which processes pending jobs using a Railway cron every 5 minutes.
We are aiming for realtime processing with a limit on spawned containers and bound on runaway cost. Our proposed design is:
->API inserts pending row into Postgres (a job)
-> upon insert, a railway worker service picks up and runs backend operations
->worker processes jobs with app-level concurrency limit
Questions:
1. Is this the recommended Railway pattern for queue workers? Should we consider redis/rabbitMQ?
2. Are worker service replicas fixed unless we explicitly configure scaling? Should we consider serverless?
3. How do we hard-cap max replicas/cost so infinite containers don't spawn when we have ghost/orphaned containers?
1 Replies
Status changed to Open Railway • about 1 month ago
a month ago
Postgres is totally fine for this.
You don't really need the overhead of RabbitMQ or Redis yet unless you're hitting massive scale like thousands of jobs a second.
On Railway, worker services are "always-on" containers, not serverless functions. Replicas won't scale unless you manually enable Autoscaling in the service settings. but keep in mind it’s based on CPU/RAM usage, not your Postgres row count.
Also you can set a hard "Max Replicas" cap in the UI or just set project-wide budget alerts in the billing settings - that’s the safest way to prevent a runaway bill from ghost containers or loops.
Hope that helps :)