a month ago
For some reason, whenever I make a deployment, one of them is successful and the other one crashes. Then they both get reset by cron reset (using railway-cron) and I think the one that crashes runs an older version of the app because it's definitely calling the cronjobs at times that we had set up earlier but now no longer use - for instance there's a daily job that was at running at 1 AM, which we now run at 1:05. But I can clearly see that it simply happens twice. Though when I open the logs I can only see the latest ones from latest deployment.
Really no clue what's going on here!
In the screenshot you can kinda see the flow - it succeeds and crashes at the same time, then gets reset for both (this restart wasn't manual, it runs once per day and is supposed to only happen to one deployment, definitely shouldnt be two of them going on).
I'm not doing anything special to summon two deployments. I make the build with a dockerfile that looks like what I'll send below because it doesnt fit in this message. btw before you ask, the Puppeteer is just for making screenshots of my designs for printing. No scraping is going on.
20 Replies
FROM oven/bun:1.1.43
# Install dependencies for Puppeteer, fonts, and language support
RUN apt-get update \
&& apt-get install -y wget gnupg \
# Emoji and basic fonts
fonts-noto-color-emoji \
fonts-liberation \
# Asian language support
fonts-noto-cjk \
fonts-noto-cjk-extra \
# Extended language support
fonts-noto-core \
fonts-ipafont-gothic \
fonts-wqy-zenhei \
fonts-thai-tlwg \
fonts-kacst \
fonts-symbola \
fonts-indic \
fonts-dejavu-core \
# European language support
fonts-liberation \
fonts-liberation2 \
fonts-freefont-ttf \
# Puppeteer dependencies
libasound2 libatk-bridge2.0-0 libatk1.0-0 libc6 \
libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 \
libgbm1 libgcc1 libglib2.0-0 libgtk-3-0 libnspr4 \
libnss3 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 \
libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 \
libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 \
libxss1 libxtst6 xdg-utils \
# Install Chrome
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | gpg --dearmor -o /usr/share/keyrings/googlechrome-linux-keyring.gpg \
&& echo "deb [arch=amd64 signed-by=/usr/share/keyrings/googlechrome-linux-keyring.gpg] http://dl.google.com/linux/chrome/deb/ stable main" | tee /etc/apt/sources.list.d/google-chrome.list \
&& apt-get update \
&& apt-get install -y google-chrome-stable \
# Clean up
&& rm -rf /var/lib/apt/lists/* \
# Update font cache
&& fc-cache -fv
# Create app directory
WORKDIR /app
# Copy package files
COPY package.json bun.lockb ./
# Install dependencies
RUN bun install --frozen-lockfile
# Copy the rest of the application
COPY . .
# Build TypeScript
RUN bun run build
# Expose port 3333
EXPOSE 3333
# Start the application using Bun
CMD ["bun", "run", "start"]a month ago
I have confirmed that neither services have two deployments running. There are also no phantom deployments running.
whats running the cron twice then? definitely logged just once in the active logs and I can clearly see things happen two times
here's a shared opencode session trying to investigate it happening https://opencode.ai/s/ZIiLR4m6
summary is this:
Why Daily Profit Runs Twice (1:00 and 1:05)
The Issue:
You changed the daily profit cronjob from 0 0 * * * (1:00 AM London time) to 5 0 * * * (1:05 AM London time) on October 27, 2025 (commit 3768c06). However, the job is still running at both times.
Most Likely Causes:
Railway Cache/Old Build: Railway may be running a cached version of your application from before October 27. The change is in your current code (I verified it's at
5 0 * * *in yourmainbranch), but Railway might not have picked up the new build.Multiple Railway Services: You might have accidentally deployed two Railway services or environments (production/staging) - one running the old code (1:00) and one running the new code (1:05).
Home Environment Still Running Old Version: If your Mac mini home environment is also configured to run cronjobs, it might be running an older version of the code that still has the 1:00 schedule.
To Fix:
Trigger a fresh Railway deployment:
Make a small commit (like adding a comment) and push to trigger a rebuild
Or manually redeploy from the Railway dashboard
Check for multiple Railway services:
Log into your Railway dashboard
Check if you have multiple services or environments running
Disable/remove any old services
Check your Home environment:
If your Mac mini is also running cronjobs, verify it's using the latest code
Run
git pulland rebuild on your home environment
-NOTE: I double checked and this is not happening
Verify the deployed code:
Add a temporary log statement to identify which instance is running
Check Railway logs at both 1:00 and 1:05 to see which service is firing
The code change itself is correct - you just need to ensure Railway is running the latest version and there aren't multiple instances running.
and why does 1 deployment crash every time i push a change even though the deployment is successful and runs perfectly well
the one that runs at 1:00 which is the old deployment even fails for things that need new api keys, which then the new deployment fixes
can someone take another look what it could be? I don't have this hosted anywhere else for this to happen
@Chandrika not sure why it says you're part of this thread, could you take a look perhaps?
a month ago
Please do not ping the team, especially on weekends
a month ago
^
I mean I saw the answer but that didn't help me. There's no other place that could be running these old cronjobs. It's messing with my systems
a month ago
I'm sorry but the answer stands, we are not running two of your applications.
a month ago
Jumping in here, @Panda is the core problem that you are seeing two instances interacting with your DB and whatnot?
yeah all my cronjobs run twice and make all the changes twice - sending notifications to customers and such
you can see here, we used to run the cronjob at 1 am and I moved it to 1:05 to see if there's something fishy going on and it just continued to run both at 1 and 1:05, though the one that runs at 1 also has old api keys so some things just fail