Cron job deployment logs not showing up
razvanparaschiv
PROOP

2 years ago

Project id: 8eeaf105-a438-44c0-9215-78f268bcf4a9.
I am trunning to run a cron job that has the configuration set in a railway.toml file

I have configured it so that it run from a docker file. If I build the image and run it locally everything works as expected.
Yet, when I deploy it on Railway, the cron services shows that it is about to run in x minutes but, aside from successfully building the Docker image, there are no deploy logs. How I translate that is that the container is not run in the first place.

51 Replies

razvanparaschiv
PROOP

2 years ago

Hello @Brody ! Do you reckon there might be something that I might've misconfigured? If it's the case for a bug is there a workaround for it?


2 years ago

please do not tag team or mods - <#727685388893945877> #5


2 years ago

anyway, what makes you say the container isn't being ran?


razvanparaschiv
PROOP

2 years ago

Thank you for your response. The fact that it is not displaying anything in the Deploy logs section. yet, it should since it thoretcally "has been run"


drmarshall
PRO

2 years ago

+1 – I am seeing the same. I waited up to an hour in the event there was some queue delay.

I see several posts that seem to suggest crons have stopped working or there is a behavior change: https://help.railway.app/feedback/cron-reliability-improvements-c2f2c959


2 years ago

im not seeing any cron services in any of your environments?


razvanparaschiv
PROOP

2 years ago

Um, I think there should be. development-testnet environment Cron Jobs group. I think service id of one of the cron jobs is: 17c81b48-6409-4b66-ba85-5537f35e62aa


razvanparaschiv
PROOP

a year ago

1281564108029952000


razvanparaschiv
PROOP

a year ago

1281565996368269300


razvanparaschiv
PROOP

a year ago

```[build]
builder = "DOCKERFILE"
dockerfilePath = "./cron//Dockerfile"
watchPatterns = ["cron/", "packages/", "!/*.md", "package.json"]

[deploy]
cronSchedule = "0 0 * * *"
region = "europe-west4"```


a year ago

I must be blind then, my bad, I'll give another look


razvanparaschiv
PROOP

a year ago

Please, it would be really helpful. Maybe there is something misconfigured on our end


a year ago

I'll look when I'm back at my computer <:salute:1137099685417451530>


a year ago

but in the mean time, please tell me more about this cron job.

language?
what does it achieve?
are you using a logger library?
etc


a year ago

update, found the service, something really strange is going on here for sure


razvanparaschiv
PROOP

a year ago

Great news. The service is trying to run a Dockerfile that runs a binary file created after bun env build action. Other than that it is pretty standard. Locally (on my machine Macbook M1 series) everything works fine. For some reason on the Railway env nothing works


razvanparaschiv
PROOP

a year ago

1282629787717992400


razvanparaschiv
PROOP

a year ago

Language is Typescript
Run time environment is bun.js
logger is used everywhere in our project so I doubt it's the case, we use pino

The railway.toml is provided above. Here is the Dockerfile:
```# oven/bun images dont work, have to have node underneath to build
FROM node:20.12.2 AS build

RUN npm install -g bun@1.1.26
WORKDIR /app
COPY . .
RUN bun install --frozen-lockfile --prefer-offline

ARG TURBOTEAM ENV TURBOTEAM=$TURBOTEAM ARG TURBOTOKEN
ENV TURBOTOKEN=$TURBOTOKEN
RUN bun turbo build --filter secret-cron-name

FROM gcr.io/distroless/base-debian11
WORKDIR /app
COPY --from=build /app/cron/secret-cron-name/bin /app/bin

CMD [ "/app/bin" ]```


a year ago

what are your logging settings? are you logging unbuffered to stdout / stderr?


razvanparaschiv
PROOP

a year ago

Yes. It worked for a couple of days, now they are not being run again. Seems like the cron jobs are very unreliable


a year ago

are you sure your application is exiting after it's done it's task?


razvanparaschiv
PROOP

a year ago

Yes, it even has a process.exit(0) at the end


razvanparaschiv
PROOP

a year ago

when testing locally, it always ends


a year ago

how long does it take?


razvanparaschiv
PROOP

a year ago

1 should take under 30 seconds. The other should probably take 5 minutes initially. Aftewards it will take under 30 seconds as well


a year ago

what is your cron schedule


razvanparaschiv
PROOP

a year ago

0 * * * *


razvanparaschiv
PROOP

a year ago

For example, this is the cron service that does not run: b734ede7-3e09-4b94-8699-28d74a046e30


a year ago

okay so every hour right?


razvanparaschiv
PROOP

a year ago

yes


razvanparaschiv
PROOP

a year ago

tried running it manually and it still does not log anything


a year ago

what logging library are you using?


razvanparaschiv
PROOP

a year ago


a year ago

are you the same person who reported that railway was overwriting the log level?


razvanparaschiv
PROOP

a year ago

No, this is the only thread I have opened


a year ago

okay, thank you.

do you have a custom transport?


razvanparaschiv
PROOP

a year ago

No, standard stdout/stderr


a year ago

I'm fairly confident Pino is buffering logs, if Pino is buffering logs railway can't pick the logs up to send to the frontend


razvanparaschiv
PROOP

a year ago

We are using pino for all of the services. They all log perfectly fine. The only service instances that don't print out the logs are the crons. Moreover, some of them should surely not fail, but I can't see their effects. This leads me to thinking that the cron jobs are not run in the first place


a year ago

does railway's UI show you that your job was ran?


a year ago

or does it say skipped?


razvanparaschiv
PROOP

a year ago

1285249826056962000


a year ago

can you enable the beta cron UI in the service settings


razvanparaschiv
PROOP

a year ago

Yes, I will attach a video in a bit


razvanparaschiv
PROOP

a year ago

Actually, it would take too much to record a video. I will summarize it. Pressing on run now, redeploys the cron job
Nothing else shows up. I will send you a screenshot in 3 minutes when the previous deployment shows that's supposed to run

1285253191046533000
1285253191381946400
1285253191667028000
1285253192137052400


a year ago

can you try duplicating the service? (right click -> duplicate)


razvanparaschiv
PROOP

a year ago

It should have ran but it didn't

1285254156382109700
1285254156789219300


razvanparaschiv
PROOP

a year ago

Done, duplicating it


a year ago

in our backend we have this marked as completed


razvanparaschiv
PROOP

a year ago

There's definitely something wrong. I am using this kind of dockerfile


razvanparaschiv
PROOP

a year ago

so probably you only mark this as completed because its being built?


Loading...