40943c2e-d9a8-4f3c-b414-73ea73eccd8f

Nope I am wrong. I updated my question. It really isn't touching the dockerfile at all. But the builder thinks it should use dockerfile

Build isn't using Dockerfile but instead nixpacks?

do you have a root directory set?

Yes

what's your dockerfile path set to exactly?

The file it should be using (the one it even recognizes in the details) is /BBWare/Dockerfile

I found the issue

what's the issue?

It is the new builder. I had it enabled. It was messing things up.

I am guessing this is not intended behavior. But I turned it off and it worked as expected

it shouldn't, I use the v2 builder exclusively with Dockerfiles for all my own projects and I've never seen this issue

would you mind enabling the v2 builder again so we can try a few things?

There seems to be some other issue with my deployment. But it looks like a me issue at this point. The thing actually is able to build and deploy without crashing :D.

Yea, we can mess with it before I start fixing the CMD issue

well now you got me curious about the CMD issue

I think it is just that I have too many things chaned together. I really really don't think it is the cause. But I mean,,, anything is possible

okay we'll come back to that then

let me know when you're back on the v2 builder

v2 Builder bug: uses nixpacks over dockerfile

Yea, just switching it cuases it to use the wrong deployment method.

As you can see in these two photos, the builder is docker. But the actual build is being done with nixpacks?

so what method are you using to set the location of the dockerfile?

I am not. I am letting it be found by the system.

I see

This is my directory. (docker-entrypoint.sh is for local dev only)

I think the v2 builder is still looking in the root of the repo for the dockerfile, but your dockerfile is in the BBWare subfolder, right?

Yes. Sorry, the naming of this project is confussing.
The root root of this project is

gotta love that double naming of nested directories 😄

okay so if the v2 builder looked for a dockerfile in your set root directory this would work

that sound about right?

Yea. It should

cool, I'll try to spin up a repo to test that theory.

but just as a little last ditch workaround, can you try setting a service variable RAILWAY_DOCKERFILE_PATH=/BBWare/Dockerfile with the v2 builder

Didn't do anything

was worth a try, the v2 builder might not even have a concept of that variable, thanks for trying anyway

you can remove that variable and switch back to the legacy builder and I'll work on reproducing this and then reporting to the v2 builder's dev

👍 sounds good. Glad I could help. And more glad that I was able to figure out my issue

this thread is definitely a big help!

I actually have a different directory in this same project that has 2 files in it. I can switch it to the new builder and see if it does the same thing
Directory:

if you're down to try that more information would always be great

Well yea. It couldn't even begin to guess what this service is supposed to be

same results though, still tried to use nixpacks

useful information

Generated config

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "DOCKERFILE",
    "dockerfilePath": "proxy/Dockerfile",
    "buildEnvironment": "V2"
  },
  "deploy": {
    "runtime": "V2",
    "numReplicas": 1,
    "sleepApplication": false,
    "restartPolicyType": "ALWAYS"
  }
}

Yea. But becuase it had no files for nixpacks to guess what the service was, it just failed. So I guess you were right. The new builder isn't recognizing dockerfile (sub directory at least)

on paper it sounds like an easy fix, so I hope it's an easy fix in implementation, I'm afk right now so won't be able to put together a repo to reproduce, but will update you when things happen

thanks again!

was able to easily reproduce, and have reported it to the team, but the dev is offline right now

if you're curious, heres my minimal repo that can reproduce this -

if you wanna go over that CMD issue you mentioned id be happy to help

No rush (at least on my end). Not like I need the new build system for any reason.

I appreciate the willingness to help. I know exactly what the issue is and have a solution written up. Going to test it tomorrow though. It is late where I live.
But if I end up being wrong and needing help ,,,,

sounds good, I'd like to hear the problem and solution once you figure that out!

ray what lol

shh

TL;DR Django manage based actions can only be done through commands. So having a line in like RUN ./manage migrate is illegal. So, I did basically CMD ./manage migrate && ./manage runserver but it made the startup CMD enter an infinite loop. The way to get out of that, is to have the CMD be some bash script. Hence that [docker-entrypoint.sh](docker-entrypoint.sh) file. I ended up modifying it handle both local dev and deployment.

That being said, I am now seeing a different issue; my goto gunicorn config is to have workers = multiprocessing.cpu_count() * 2 + 1. But when I run with this config, it seems that railway doesn't like it:

[2024-06-14 14:35:13 +0000] [65] [INFO] Booting worker with pid: 65
[2024-06-14 14:35:13 +0000] [67] [INFO] Booting worker with pid: 67
[2024-06-14 14:35:17 +0000] [3] [ERROR] Worker (pid:28) was sent SIGKILL! Perhaps out of memory?
[2024-06-14 14:35:20 +0000] [3] [ERROR] Worker (pid:30) was sent SIGKILL! Perhaps out of memory?
[2024-06-14 14:35:20 +0000] [3] [ERROR] Worker (pid:35) was sent SIGKILL! Perhaps out of memory?
[2024-06-14 14:35:22 +0000] [68] [INFO] Booting worker with pid: 68
[2024-06-14 14:35:22 +0000] [3] [ERROR] Worker (pid:36) was sent SIGKILL! Perhaps out of memory?
[2024-06-14 14:35:24 +0000] [3] [ERROR] Worker (pid:32) was sent SIGKILL! Perhaps out of memory?
[2024-06-14 14:35:24 +0000] [69] [INFO] Booting worker with pid: 69
[2024-06-14 14:35:25 +0000] [70] [INFO] Booting worker with pid: 70
[2024-06-14 14:35:28 +0000] [3] [ERROR] Worker (pid:34) was sent SIGKILL! Perhaps out of memory?

What is the actual real way to deploy a gunicorn instance? To my understanding, to get the most of gunicorn, you should have 2-4X workers to cores. But I guess that isn't possible here? So I should just use 1 worker (default value)?
This is also not really related to this thread though; I can totally open a new one if need be

that will evaluate to 65 workers because cpu_count will return 32 despite you being on the hobby plan.

Well ,,,, I tried doing a deploy with just the default …

[2024-06-14 14:46:41 +0000] [72] [INFO] Booting worker with pid: 72
[2024-06-14 14:46:41 +0000] [73] [INFO] Booting worker with pid: 73
[2024-06-14 14:46:43 +0000] [75] [INFO] Booting worker with pid: 75
[2024-06-14 14:46:43 +0000] [30] [ERROR] Worker (pid:41) was sent SIGKILL! Perhaps out of memory?
--- Logging error ---
[2024-06-14 14:46:43 +0000] [74] [INFO] Booting worker with pid: 74
Traceback (most recent call last):
    stream.write(msg + self.terminator)
RuntimeError: reentrant call inside <_io.BufferedWriter name=''>

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1163, in emit
    self.reap_workers()
  File "/opt/venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 554, in reap_workers
  File "/opt/venv/lib/python3.12/site-packages/gunicorn/glogging.py", line 272, in error
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1568, in error
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1684, in _log
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1700, in handle
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1762, in callHandlers
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1028, in handle
    self.handleError(record)
    sys.stderr.write('--- Logging error ---\n')
RuntimeError: reentrant call inside <_io.BufferedWriter name=''>
Call stack:
  File "/opt/venv/bin/gunicorn", line 8, in 
  File "/opt/venv/lib/python3.12/site-packages/gunicorn/app/wsgiapp.py", line 67, in run
    super().run()
    Arbiter(self).run()
  File "/opt/venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 202, in run
  File "/opt/venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 571, in manage_workers
  File "/opt/venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 642, in spawn_workers
  File "/opt/venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 593, in spawn_worker
  File "/opt/venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 242, in handle_chld
Arguments: ()
[2024-06-14 14:46:43 +0000] [76] [INFO] Booting worker with pid: 76
[2024-06-14 14:46:44 +0000] [30] [ERROR] Worker (pid:42) was sent SIGKILL! Perhaps out of memory?
[2024-06-14 14:46:44 +0000] [77] [INFO] Booting worker with pid: 77

….

"default" gunicorn BBWare.wsgi -b [::]:${PORT:-8000}

you're on the trial plan? weren't you on hobby yesterday? I don't know what your app does but its very possible it's trying to use more than 500mb

This is totally possible. But then again, these numbers don't seem reasonable for a program that is idle.
I am using a trail account right now to gauge the viability of railway for my needs. A rabbit hole I really didn't need to go down. But too deep into it now :p
But once I confirm it is, I would switch this to a pro plan

Anyways, I feel like this issue is escaping the purpose of the thread. I will open a new thread

sounds good, I'll see you there!

hey joshie, I'm told this has been fixed, would you want to give the v2 builder another try?

Sure I can do that

thanks!

I am a pro when it comes to breaking things 😄
If I had to guess, it is becuase I have this attached to a repo that has been removed. Even if that is the cause, two things should happen (instead of what does). 1) the change that failed to "commit" should be reverted (it isn't) and 2) the error message should be better. Ideally.

1252274765150818416

why does it sound like you are in the middle of a data center

And this isn't a UI bug either (the setting not reverting) I refresh with no cache and it remins that way

Becuase I am
Becuase there is running water right next to me.

when the redeploy happens via deploying changes it does not grab your repo, so repo or not shouldn't matter, but maybe it doesn't like that it doesn't exist and it's breaking some flow

and it's not like it failed it just didn't trigger a redeploy, you can do that manually after all

Can confirm. I removed the repo link, without changing anything else. And it was able to build. So yea, it can still build without a repo link but having a broken one cuases a crash.

I'll try to reproduce this

Yes. The two issues is the bad error message (not informative) and the fact that it did commit my changes even if it was unable to attempt a redeploy. Idk what it should do in that case. But it just feels like it should revert the settings. But maybe other people would disagree? I guess it depends on what settings are changed. Like in this case, with the builder being changed, I want a revert. But if I was changing variables, maybe not. <:shrug:747680403778699304>

I don't think it should revert, since the settings will be applied on the next deployment

Yea. After writting it out and thinking about it, I guess that is correct.

either way, once you manually triggered a redeploy, did it work?

The website is being a little buggy / laggy. But let us ignore that part

This does work now. It was able to deploy and build properly.

awesome

then I will mark this as solved

<@&875753109501141003> please 🫴 (jk, I don't actually need it if this doesn't qualify for that :p)

why doesn't it? you are the first person to catch this dockerfile bug

Thanks 😄 (I more so didn't want to come off as pushy for it. I don't require it or anything.)

badges are fun