3c0425aa-9fe4-449a-a161-8a1efb3b53ee

The schedule is 0 11 * * *, which means everyday at 11:00 AM UTC.

Logs show the last invocation was on June 1st at 6:01 AM CST (correct, although 1 minute late?).

The expected last invocation would actually be June 2nd at 6:00AM CST, but there is no activity, logs, or even an invocation on the dashboard.

I only found out about this because my cron monitor raised an issue.

The history below doesn't show anything interesting, if you're curious.

from my understanding, there are too many jobs being ran at 11am utc that some get skipped, until the team addresses this i would recommend switching to an in-code scheduler

are you serious? i literally just undid the node cron scheduler because it was consuming 50 MB memory constantly and i thought it'd be fun to move away from that

not mad at you, just… that's pretty sucky

i feel you, you could also try another time? 10:30am utc?

Yeah, I was thinking that some weird off-color time would be less likely to incur issues.

Something like XX:48

The whole job is done in 5 seconds usually.

yep you got the right idea

Gonna try 10:48 UTC and see what happens.

sounds good!

i have also sent this thread in a thread i have with cooper for gathering cron issues like yours

Alrighty; just my thought: a little detail about this being skipped, or likely to be skipped, or some transparency on the issues with cron would be nice.

I don't mind that Railway's platform is in need of improvement; but letting users explore until they hit a landmine isn't ideal.

i assume they had never thought they would be over scheduled so they never designed error handing and the ui around it

Kinda an interesting problem to think about in retrospect.

ideally the only issues that you could get out of a cron job would be an issue with the build or deploy

Maybe a check-mark that says "I'm okay with this being rescheduled slightly" would be good.
Crons that are more important could be charged at a higher rate, but they'll be prioritized on runners or whatever.

If you have 600 jobs every day at 11AM UTC, spinning up tons of machines to work on them is not exactly smart. Especially when most of them are tiny jobs.

Working in bursts would be better. And working early + late, like queueing. Start at 10:58 or even earlier to start executing.

im sure they have more than 600 at 11am utc, and if i recall correctly, its only a single schedular on their backplane

aha i really have no idea about the scale railway works at tbh

i dont really either, im just going off the crumbs they give us

i mean they do tell us a fair bit, but more info can never hurt in our position of community help

@Brody

This is pretty sucky as crons go.

I'm not sure what's going on, actually; I cannot tell if Railway is to fault here.

Nevermind, seems like something with Sentry is going wrong?
Error while running backup: AxiosError: connect EHOSTUNREACH 34.120.195.249:443

host unreachable eh? you aren't the first person to see this error even after they resolved the incident

are you on the legacy or v2 runtime? check your service settings, if legacy, switch it to v2

Got it, switched it to V2. Didn't know that was a setting lol.

just for clarity, the v2 runtime has been confirmed to fix host unreachable, but it has no impact on cron being skipped since that's a completely different system

👍

Working good!

No failures since.

they made changes to the cron scheduler too

the changes they made did not help 😦

but thank you for trying

i mean i guess at least it happened the same day?

lol

very odd

lol

back to in code