2 years ago
Hello,
We've been barreling towards reliability improvements recently. In fact, it's our focus for the next 3 months.
Crons, historically have been "Alright but not bulletproof". Sometimes they'd skip, sometimes they'd get stuck, etc. Not a great experience at all.
We've rolled out a couple changes this week that should abate this
Please let us know if this is not the case! These need to be bulletproof relability
NOTE: A lot of the issues were caused during periods of heavy load (aka midnight UTC). In the future, we will likely shift crons on a per project level by an offset. What this would look like is, when you put your cron schedule in as "Every hour", it would say "Your cron will run at 6 minutes past, every hour"
12 Replies
2 years ago
Our most recent occurrence of a skipped cronjob was Sunday (July 21, 24), 2 missed cron jobs out of 6 that were set that day. Yesterday and today have been stable.
2 years ago
Yup we noticed a dip in cron activity on Sunday, and have applied a fix as of about 2 hours ago. Please keep us informed!
2 years ago
Didn't notice skips but I had issues with cron reliability in that they would work perfectly for a while and then start running at random times or e.g. 100 times in a row.
Looking through log history this hasn't happened in the last 3 months now and they've been running pretty reliably - albeit I only have one a day now since I moved my business-critical crons to another place.
2 years ago
Keep us posted! Here to support any usecase especially the business critical ones :D
2 years ago
Scheduling for my project has been very unreliable to this day. I try to make a Python script run every 15 minutes. Additionally, this script checks if the time is within working hours and then executes, if not it logs "Not within working hours, exiting script" and finishes executing. For hours this script doesn't execute. It randomly does some times.
I'm appending 2 screenshots of the project and the logs, within 1 hour this script only executed once - when explicitly made it run using Railway UI
Attachments
2 years ago
Edited: moved to the proper thread
2 years ago
How can I support the bot?
2 years ago
My crons stoped running 5 days ago. Haven't change any code or config in that time. When I click "Run now" it just triggers new build, but doesn't run the start command. Panel shows that next run is scheduled, but when time comes start command is not called. I see only "Ready" states, nothing is "Active".
Did the cron run logic change or something?
Moved couple production crons today, just to find that they are not working
Also, it will be great to have "Cron:Completed" or "Cron:Error" states for all cron runs in history, since now it shows "Removed" and it's pretty strange name.
a year ago
I'm having a similar issue to @nikitattt - I've tried creating a database backup cron and it only ever gets into a "Ready" state - i.e. built but not deployed. If I deploy as a service without cron it runs successfully and exits when I push, so I think the service is set up correctly. When I push an update or click "Run now" I get another build that sits in a "READY" state, but doesn't run.
a year ago
Hey y'all. We've rolled out a new cron experience. If you're having issues with the old one, can you enable the feature flag at the bottom of the service settings and report back?
This new cron experience has another tab called "Cron runs" because we found users were very confused by the previous experience of having it all on the deployments panel (ready, deploying, completed, etc states)
a year ago
8 months since this ticket, and after our quality sprint here, I think it’s safe to say that we delivered here. Going to mark this as completed.
Status changed to Completed angelo-railway • 11 months ago
