Running celery worker in deployed django repo
tedimcboy
PROOP

a year ago

This is my current procfile

web: waitress-serve --port=$PORT AiSalesPlatformDemo.wsgi:application
celeryworker: celery -A AiSalesPlatformDemo worker --loglevel=debug --pool=solo celerybeat: celery -A AiSalesPlatformDemo beat --loglevel=debug --scheduler djangocelerybeat.schedulers:DatabaseScheduler

which has been fine, locally without the prefixes of "celeryworker" and "celerybeat" running in seperate terminal windows, the periodic tasks executed properly. the only difference on the deployed version is that it just doesn't trigger at the specific time. Everything is set to UTC and i even se tthe CRON schedule to everything * to just trigger every minute but nothing ever happens. is there a specific prefix i need to use for railway to recognize the celery worker and celery beat? Again, locally the commands without the prefixes work perfectly (obviously the prefix will cause issues locally) assuming they are run in their own terminal window and the web waitress-serve has been working fine as well and still works. Just have a feeling the prefix is not being recognized or something and I can't see the logs of it to confirm. Any idea what im jacking up this time?

i also have a manual command to setup tasks : python manage.py setupperiodictasks

which i put as a start command in railway.json which has been working, i'd just like to see the logs for celery and make it work somehow on deployment. locally is all good I feel like i'm missing some piece of key information regarding how to handle it with railway.

47 Replies

tedimcboy
PROOP

a year ago

ba57bf2a-26c5-46ca-a9b1-8efc01fa23ab


tedimcboy
PROOP

a year ago

the periodic task and crontab are setup properly in the django db just like when it was local. dpeloyed one just doesn't want to trigger it

1331674287278657500
1331674287756546000


a year ago

hello!

in your linked project i only see your website service, two databases, and dbgate.

i dont see services for celery and celery beat?


tedimcboy
PROOP

a year ago

yo brody


tedimcboy
PROOP

a year ago

i didn't create services for celery and beat, i was under the impression that since locally i was able to run them through terminal windows i could include the commands i used in ProcFile to trigger them


tedimcboy
PROOP

a year ago

also how would i even setup services for them if thats what i have to do. all i did were commands in my project environment rather than anything external


tedimcboy
PROOP

a year ago

if possible keeping it within the same service would be ideal but ik its not exactly best practice. tbh i clearly jsut don't have a full understanding of how this should be working in a deployed env


a year ago

so you would need to translate your multiple terminal windows locally into multiple railway services.

you can just right click duplicate your main service, and then set the correct start command in its service settings


tedimcboy
PROOP

a year ago

wait thats it? so deplicate the main service, and set start command to

celery -A AiSalesPlatformDemo beat --loglevel=debug --scheduler djangocelerybeat.schedulers:DatabaseScheduler

and

celery -A AiSalesPlatformDemo worker --loglevel=debug --pool=solo
?


a year ago

yeah, duplicate twice since you need to run two more things


tedimcboy
PROOP

a year ago

the start commands are pulled from railwayjson though

1331678538306158800


a year ago

remove them from railway.json


a year ago

or have a web.railway.json, celery.railway.json, celery_beat.railway.json will the correct commands defined, and then set the applicable config file in the service settings


tedimcboy
PROOP

a year ago

oof


tedimcboy
PROOP

a year ago

that jacks up my whole automation process


a year ago

how so


tedimcboy
PROOP

a year ago

automation handles each service but just deploys from a given repo. not sure how i would modify automations to change the start commands per service


tedimcboy
PROOP

a year ago

also i don't really get how a duplicate service tied to the same repo acts aas a terminal window within a deployed env


a year ago

im not sure i understand what you mean "automations "


tedimcboy
PROOP

a year ago

so just an unexposed service with a diff start command would do that?


tedimcboy
PROOP

a year ago

i have a form setup and submitting through that form clones a base repo and sets up all teh services i need for projects and creates a new project w everything i need


a year ago

if it helps, you can think about the start command as the command you enter into the terminal


tedimcboy
PROOP

a year ago

you helped me with that a lil while back lol


tedimcboy
PROOP

a year ago

why wouldn't the proc file work for this though


a year ago

oh yeah, no it doesnt mess anything up, delete the start command from the railway.json and then just set the start command in the template


a year ago

procfile is heroku, we are railway lol


a year ago

this is better practice


tedimcboy
PROOP

a year ago

i have other start commands in there i need for the first servce tied to the repo though. its django so has to run the stupid collectstatic stuff


a year ago

then you run that as a start command


tedimcboy
PROOP

a year ago

id have to manually put in the new start command though right? for the service that guess is acting as a temrinal window


tedimcboy
PROOP

a year ago

duplicated service


a year ago

only once, in your template


tedimcboy
PROOP

a year ago

one sec, im 100% messing something up but have to handle something rq


tedimcboy
PROOP

a year ago

so i duplicated the service. youre saying to delete the start command from the railway.json file:

"startCommand": "python [manage.py](manage.py) makemigrations && python [manage.py](manage.py) migrate && python [manage.py](manage.py) setup_periodic_tasks && python [manage.py](manage.py) collectstatic --noinput && gunicorn --timeout 300 --workers 3 AiSalesPlatformDemo.wsgi",

and just leave that blank. and then manually put them into the start command of the original one.

1331690535962022000


tedimcboy
PROOP

a year ago

regarding this, i would e able to select from the applicable railway.json file if there are multiple?


tedimcboy
PROOP

a year ago

like they would show up in a dropdown or something? as of right now idisables any custom commands because it is pulling from the railway.json. was wondering if we are actually abel to pick from here or from api


tedimcboy
PROOP

a year ago

sorry dumbq uesitons ik


a year ago

you wouldnt, i made the suggestion before i remembered your usecase. please forget about it


a year ago

you want to remove the startcommand from the json file and instead set the start commands in your template


a year ago

well, right now you would want to get things running in your example project, then you can update the template


tedimcboy
PROOP

a year ago

okay cool i got it


tedimcboy
PROOP

a year ago

ty!


a year ago

also, i dont see any variables on the web service?


tedimcboy
PROOP

a year ago

ended up not using vars, temporarily using an sh script to run things under this service since i had other stuff come up i need to prioritize. I have a temp solution working for everything and you gave me all the info i need to set it up properly so i appreciate it


tedimcboy
PROOP

a year ago

my brain was a bit fried yesterday, sorry i took me a min to see what you were saying. appreciate the help as always man! this support thread is good to be closed


a year ago

nah this can't be closed until you stop hard coding your variables


a year ago

please update the template with reference variables


Loading...