21 days ago
Project: authentic-analysis (production environment) Service: web (silofm.up.railway.app) Plan: Pro ($20)
Our web service is currently rate-limited and returning the message: "Your application has reached a rate/request limit. Retry will occur after: 14641 s" in the deploy logs.
The site is fully down — pages will not load.
Root cause identified: We diagnosed the issue as runaway concurrent bot cron instances stacking due to in-loop sleep() calls combined with N+1 database queries in our follow action. We are deploying a fix now (eliminating the in-loop staggers, adding a single-instance lock, and refactoring the N+1 queries to bulk operations).
Request: Could you manually lift the current rate limit so we can deploy and verify the fix without waiting the full ~4 hours? We'll be redeploying shortly with the corrections in place.
Happy to provide additional logs or details. Thank you.
2 Replies
Status changed to Awaiting Railway Response Railway • 21 days ago
Status changed to Awaiting User Response Railway • 21 days ago
21 days ago
This thread has been marked as public for community involvement, as it does not contain any sensitive or personal information. Any further activity in this thread will be visible to everyone.
Status changed to Open Railway • 21 days ago
brody
We are not rate limiting you, this is an issue with your application.
21 days ago
Thank you for clarifying — that helps a lot. We've identified the cause as runaway concurrent bot cron instances stacking and exhausting our workers. Pausing the bots now and deploying a fix to prevent stacking. Appreciate the fast response.
Status changed to Solved Railway • 21 days ago