a month ago
Project ID: 675c7104-52e7-4685-9806-2b3cbd0a9af7
Service ID: 84dc4d1a-0cea-44e8-893c-eeacbe4342fe
My cron job writes a commit hash to a volume in order to resume work on the next run. However, on every run it executes the "empty file" branch of my logic, and prints that it's "Saving current HEAD for the first time…"
I am using a WORKRDIR environment var, so I can configure where to write the "current head" txt file. In local I want it in the project root, in the Railway deployment I want it written in the volume. As far as I can tell everything is configured correctly, the WORKDIR var matches the volume mount path. Would appreciate another set of eyes on this, thank you!
0 Replies
a month ago
Hey, the only options I can think of here are:
Is your
WORKDIRenvironment variable actually being set? Perhaps a print/console.log could confirm that.Your volume might be mounted at the wrong path.
Maybe you're also able to attach a file browser template to confirm that there's nothing on the volume itself.
a month ago
Also, are you using a Dockerfile?
I suppose could check if the var is actually being set, seems farfetched that it wouldn't but stranger things have happened
a month ago
Providing that script would help the community help you.
#!/bin/bash
set -eu
if [ -f "./.env" ]; then
source "./.env"
fi
if [ -z "$WORKDIR" ]; then
echo "No WORKIR found in environment. Operation cancelled."
exit 1
fi
if [ -z "$SMTP_USER" ]; then
echo "No SMTP_USER found in environment. Operation cancelled."
exit 1
fi
if [ -z "$SMTP_PASS" ]; then
echo "No SMTP_PASS found in environment. Operation cancelled."
exit 1
fi
if [ ! -d "$WORKDIR" ]; then
mkdir -p "$WORKDIR"
fi
cd "$WORKDIR"
if [ ! -f "./previoushead.txt" ]; then
touch "./previoushead.txt"
fi
previous_head=$(< "./previoushead.txt")
current_head=$(git ls-remote https://github.com/supabase/supabase.git refs/heads/master | cut -f1)
if [ -z "$previous_head" ]; then
echo "Saving current HEAD for the first time..."
printf %s "$current_head" > "./previoushead.txt"
echo "Success!"
exit 0
fi
if [ "$current_head" = "$previous_head" ]; then
echo "Nothing has changed!"
exit 0
fi
echo "Supabase remote HEAD has updated! Checking for relevant changes locally..."
if [ -d "supabase" ]; then
cd supabase
git pull
else
git clone https://github.com/supabase/supabase
cd supabase
fi
if [ -z "$(git diff --name-only "$previous_head" HEAD -- docker)" ]; then
echo "Nothing has changed in /docker!"
else
echo "New changes in /docker! Notifying via email..."
curl --url 'smtps://smtp.gmail.com:465' \
--mail-rcpt 'ben.isenstein@gmail.com' \
--mail-from "$SMTP_USER" \
--user "$SMTP_USER:$SMTP_PASS" \
-T <(echo -e "From: \"PG On Rails\" <$SMTP_USER>\nSubject: New changes in \"/supabase/docker\"\n\nhttps://github.com/supabase/supabase/tree/master/docker\n\nSent securely via curl.")
fi
echo "Saving new HEAD \"$current_head\"..."
cd ..
printf %s "$current_head" > "./previoushead.txt"
echo "Success!"
exit 0a month ago
Also, shell scripts commonly continue execution without error even when environment variables are undefined (if not used with the correct flags).
And I just confirmed that WORKDIR and the volume mount path are both /app/volume
a month ago
Try using all absolute paths.
Some more context:
This service builds from a rootDirectory within a monorepo of cron jobs. In this directory I use a railway.json to configure the cron schedule. Now, the docs say that you can't just include a railway.json in a service's rootDirectory and expect it to work, but it's working. But could that be messing up volume behaviour? highly doubt it, but thought I would share nonetheless
Is that path not absolute to the root of the container? I tried ~/volume and that was prohibited.
a month ago
Nothing is messing up the volume behavior; the railway side of things is correct, it's the script.
a month ago
Brody said to include $WORKDIR to every path instead of using cd. Here's on how it would look like:
a month ago
#!/bin/bash
set -eu
if [ -f "./.env" ]; then
source "./.env"
fi
if [ -z "$WORKDIR" ]; then
echo "No WORKIR found in environment. Operation cancelled."
exit 1
fi
if [ -z "$SMTP_USER" ]; then
echo "No SMTP_USER found in environment. Operation cancelled."
exit 1
fi
if [ -z "$SMTP_PASS" ]; then
echo "No SMTP_PASS found in environment. Operation cancelled."
exit 1
fi
if [ ! -d "$WORKDIR" ]; then
mkdir -p "$WORKDIR"
fi
cd "$WORKDIR"
if [ ! -f "$WORKDIR/previoushead.txt" ]; then
touch "$WORKDIR/previoushead.txt"
fi
previous_head=$(< "$WORKDIR/previoushead.txt")
current_head=$(git ls-remote https://github.com/supabase/supabase.git refs/heads/master | cut -f1)
if [ -z "$previous_head" ]; then
echo "Saving current HEAD for the first time..."
printf %s "$current_head" > "$WORKDIR/previoushead.txt"
echo "Success!"
exit 0
fi
if [ "$current_head" = "$previous_head" ]; then
echo "Nothing has changed!"
exit 0
fi
echo "Supabase remote HEAD has updated! Checking for relevant changes locally..."
if [ -d "$WORKDIR/supabase" ]; then
cd "$WORKDIR/supabase"
git pull
else
git clone https://github.com/supabase/supabase "$WORKDIR/supabase"
cd "$WORKDIR/supabase"
fi
if [ -z "$(git diff --name-only "$previous_head" HEAD -- docker)" ]; then
echo "Nothing has changed in /docker!"
else
echo "New changes in /docker! Notifying via email..."
curl --url 'smtps://smtp.gmail.com:465' \
--mail-rcpt 'ben.isenstein@gmail.com' \
--mail-from "$SMTP_USER" \
--user "$SMTP_USER:$SMTP_PASS" \
-T <(echo -e "From: \"PG On Rails\" <$SMTP_USER>\nSubject: New changes in \"/supabase/docker\"\n\nhttps://github.com/supabase/supabase/tree/master/docker\n\nSent securely via curl.")
fi
echo "Saving new HEAD \"$current_head\"..."
cd "$WORKDIR"
printf %s "$current_head" > "$WORKDIR/previoushead.txt"
echo "Success!"
exit 0This path style starting with / has been the common way I've mounted every volume on Railway for years. It's the same pattern that Postgres volumes are mounted. What am I missing here?
a month ago
Yep 💯
a month ago
The volume is mounted correctly, don't worry.
a month ago
That's why I kinda hate shell scripts, hear a lot of horror stories coming from them 😅
a month ago
I was also trying your script locally and it also went correctly. MacOS user here
I'm very new to this level of sophisticated shell scripting and understanding from a systems level why this doesn't work. Can either of you really explain why my script would work on my MacOS, and not in the container on Railway?
Kay well we solved my issue; still flabbergasted that my script doesn't work as-is in the deployment XD
a month ago
I really don't know the answer, but if it actually solves it I will dig up the answer, also really curious 👀
a month ago
How are you executing your script on Railway?
railway.json
{
"$schema": "https://railway.com/railway.schema.json",
"build": {
"watchPatterns": ["/notify-docker-dir/*/"]
},
"deploy": {
"cronSchedule": "0 * * * *"
}
}
a month ago
But like the start command?
that's why I maintain crons in a monorepo. each job is a folder with a [start.sh](start.sh) and a railway.json with the cron schedule
a month ago
I remember running into a lot of issues related to POSIX shells, confirming if it's somehow causing that issue
it should always run in bash, there's a shebang at the top of the file. I guess bash has to be included in the whatever container railway spins up to execute a [start.sh](start.sh)…
a month ago
found it!
a month ago
had to fire up my ubuntu vm
a month ago
Railpack always uses sh as the start up command
Status changed to Solved brody • about 1 month ago
a month ago
Thinking now, maybe that's a bug 🤔
In my imagination, the sh program will begin executing the file, and see the #!/bin/bash on line 1, and look for /usr/bin/bash on the system in the container to continue executing. Am I wrong about how that works?
a month ago
Pretty sure the sh doesn't do that as that's only for running it as an executable (chmod +x and then ./script.sh)
a month ago
Yep, sh doesn't do that, I changed the shebang to a python one and it still continued the execution
I think you're right. That would happen if I ran ./[start.sh](start.sh) in my terminal, but it doesn't work that way when running it in Go
"
When you make a file executable and try to run it directly (for example, ./myscript), the operating system loader:
Reads the first two bytes of the file.
If they are #!, it reads the rest of that line as an interpreter directive.
The OS then executes the specified interpreter, passing the script file’s path as the first argument to that interpreter.
"
a month ago
Not specific to Go but Railpack itself
a month ago
I'll open an issue about it, I really think it should consider the shebang as the documentation itself mentions it
By the way, saw that you're Brazilian, LETS GOOO I was at Carnival In Rio in March
a month ago
a month ago
give a CPF to this man
lolol that's too good. I used a friend's cpf to get an eye exam in Floripa 😆
a month ago
Pretty common, everything in here requires a CPF pretty much
a month ago
the polluted one, São Paulo
a month ago
but im going to floripa next month for a conference 🎉
a month ago
got a nice Airbnb and all, pretty excited
a month ago
Yep, first one also
a month ago
Which one? 👀
a month ago
company is sponsoring the event
Will you have time to see some nice beaches? Campeche, Joaquina, Barra da Lagoa are so nice. I can't wait to surf in Campeche again, had the best wave of my life there in April
a month ago
I'll be in Barra da Lagoa actually, maybe I'll take some surf lessons and take a hike
Definitely hike to Praia da Galheta, I think it's about 45 mins and super wild and natural
a month ago
Thanks for the suggestion, will definitely do
#!/bin/bash
set -eu
if [ -f "./.env" ]; then
source "./.env"
fi
if [ -z "$WORKDIR" ]; then
echo "No WORKIR found in environment. Operation cancelled."
exit 1
fi
if [ -z "$SMTP_USER" ]; then
echo "No SMTP_USER found in environment. Operation cancelled."
exit 1
fi
if [ -z "$SMTP_PASS" ]; then
echo "No SMTP_PASS found in environment. Operation cancelled."
exit 1
fi
if [ ! -d "$WORKDIR" ]; then
mkdir -p "$WORKDIR"
fi
if [ ! -f "$WORKDIR/previoushead.txt" ]; then
touch "$WORKDIR/previoushead.txt"
fi
previous_head=$(< "$WORKDIR/previoushead.txt")
current_head=$(git ls-remote https://github.com/supabase/supabase.git refs/heads/master | cut -f1)
if [ -z "$previous_head" ]; then
echo "Saving current HEAD for the first time..."
printf %s "$current_head" > "$WORKDIR/previoushead.txt"
echo "Success!"
exit 0
fi
if [ "$current_head" = "$previous_head" ]; then
echo "Nothing has changed!"
exit 0
fi
echo "Supabase remote HEAD has updated! Checking for relevant changes locally..."
if [ -d "$WORKDIR/supabase" ]; then
cd "$WORKDIR/supabase"
git pull
else
git clone https://github.com/supabase/supabase "$WORKDIR/supabase"
cd "$WORKDIR/supabase"
fi
if [ -z "$(git diff --name-only "$previous_head" HEAD -- docker)" ]; then
echo "Nothing has changed in /docker!"
else
echo "New changes in /docker! Notifying via email..."
curl --url 'smtps://smtp.gmail.com:465' \
--mail-rcpt 'ben.isenstein@gmail.com' \
--mail-from "$SMTP_USER" \
--user "$SMTP_USER:$SMTP_PASS" \
-T <(echo -e "From: \"PG On Rails\" <$SMTP_USER>\nSubject: New changes in \"/supabase/docker\"\n\nhttps://github.com/supabase/supabase/tree/master/docker\n\nSent securely via curl.")
fi
echo "Saving new HEAD \"$current_head\"..."
printf %s "$current_head" > "$WORKDIR/previoushead.txt"
echo "Success!"
exit 0a month ago
at what step, sorry? Thought we fixed it
if [ -z "$previoushead" ]; then echo "Saving current HEAD for the first time…" printf %s "$currenthead" > "$WORKDIR/previoushead.txt"
echo "Success!"
exit 0
fi
a month ago
Let me see if I can see the issue on my Ubuntu VM
a month ago
Also, I don't see if Railpack supports the start command for shell scripts (as I don't see it on the code), but perhaps using ./[script.sh](script.sh) would fix the issue (make sure you commit your file with the chmod +x permission).
Kay well I'll fix it tomorrow. Gotta go to bed. Thanks for all your help brother!
a month ago
Ok, I transformed your script to be supported by POSIX compliant shells.
a month ago
#!/bin/sh
set -eu
if [ -f "./.env" ]; then
. "./.env"
fi
if [ -z "${WORKDIR:-}" ]; then
echo "No WORKDIR found in environment. Operation cancelled."
exit 1
fi
if [ -z "${SMTP_USER:-}" ]; then
echo "No SMTP_USER found in environment. Operation cancelled."
exit 1
fi
if [ -z "${SMTP_PASS:-}" ]; then
echo "No SMTP_PASS found in environment. Operation cancelled."
exit 1
fi
if [ ! -d "$WORKDIR" ]; then
mkdir -p "$WORKDIR"
fi
cd "$WORKDIR"
if [ ! -f "$WORKDIR/previoushead.txt" ]; then
touch "$WORKDIR/previoushead.txt"
fi
previous_head=$(cat "$WORKDIR/previoushead.txt")
current_head=$(git ls-remote https://github.com/supabase/supabase.git refs/heads/master | cut -f1)
if [ -z "$previous_head" ]; then
echo "Saving current HEAD for the first time..."
printf %s "$current_head" > "$WORKDIR/previoushead.txt"
echo "Success!"
exit 0
fi
if [ "$current_head" = "$previous_head" ]; then
echo "Nothing has changed!"
exit 0
fi
echo "Supabase remote HEAD has updated! Checking for relevant changes locally..."
if [ -d "$WORKDIR/supabase" ]; then
cd "$WORKDIR/supabase"
git pull
else
git clone https://github.com/supabase/supabase "$WORKDIR/supabase"
cd "$WORKDIR/supabase"
fi
if [ -z "$(git diff --name-only "$previous_head" HEAD -- docker)" ]; then
echo "Nothing has changed in /docker!"
else
echo "New changes in /docker! Notifying via email..."
printf "From: \"PG On Rails\" <%s>\nSubject: New changes in \"/supabase/docker\"\n\nhttps://github.com/supabase/supabase/tree/master/docker\n\nSent securely via curl." "$SMTP_USER" | \
curl --url 'smtps://smtp.gmail.com:465' \
--mail-rcpt 'ben.isenstein@gmail.com' \
--mail-from "$SMTP_USER" \
--user "$SMTP_USER:$SMTP_PASS" \
-T -
fi
echo "Saving new HEAD \"$current_head\"..."
cd "$WORKDIR"
printf %s "$current_head" > "$WORKDIR/previoushead.txt"
echo "Success!"
exit 0a month ago
*I used AI for it, and from my tests, everything seems to be working.
Okay ChatGPT says it's valid. I just wonder whether the <() file syntax was also posix compliant
a month ago
removed part of the script for the initial file only, that might be it
a month ago
Also, where are you seeing the <() syntax?
a month ago
hmmm i'm able to execute the full script with fake environment variables
a month ago
Got it, nevermind then
a month ago
See ya
And I guess railpack has some work to do supporting Bash? pretty please?
a month ago
Will be opening an issue on Railpack, doing it as we speak
Oh one last thing. @Brody might like this since I've been trying to do this for literally 2 years at this point - deploy Supabase on Railway with a single shell command
bash <(curl -fsSL [https://raw.githubusercontent.com/BenIsenstein/pgonrails-cli/main/start.sh](https://raw.githubusercontent.com/BenIsenstein/pgonrails-cli/main/start.sh))
Massive shell script that uses the GraphQL API to deploy the PG On Rails template
a month ago
Very nice use of our API.
a month ago
Hey, will be closing the thread, if you've any more issues feel free to create another one
a month ago
!s
a month ago
@Railway
a month ago
!s
Spent some time to improving railpack support here! https://github.com/railwayapp/railpack/pull/328 and https://github.com/railwayapp/railpack/commit/608fada19906397a011234e3116feeb346da10b0