2 years ago
container worked and everything was working well but container crashed…not sure why..Here's the projectid - 1b883543-ff8a-4389-9f72-1fe4b7b73668
161 Replies
2 years ago
Is there anything in the logs that might show why it crashed?
/bin/bash: line 1: web:: command not found
container event container restart
/bin/bash: line 1: web:: command not found
container event container died
Also when the app was up, it timedout on few api calls and I tried web: gunicorn app:app --timeout 120
2 years ago
web: is a Procfile thing, you should be able to set your start command to just gunicorn app:web --timeout 120
Railway hasn't supported procfiles in a long time
File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 402, in import_app
app = getattr(mod, name)
^^^^^^^^^^^^^^^^^^AttributeError: module 'app' has no attribute 'web'
Failed to find attribute 'web' in 'app'.
[2024-09-13 14:09:51 +0000] [4] [INFO] Worker exiting (pid: 4)
[2024-09-13 14:09:51 +0000] [1] [INFO] Shutting down: Master
[2024-09-13 14:09:51 +0000] [1] [INFO] Reason: App failed to load.
container event container restart
[2024-09-13 14:09:52 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2024-09-13 14:09:52 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2024-09-13 14:09:52 +0000] [1] [INFO] Using worker: sync
[2024-09-13 14:09:52 +0000] [4] [INFO] Booting worker with pid: 4
Traceback (most recent call last):
File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 402, in import_app
app = getattr(mod, name)
^^^^^^^^^^^^^^^^^^AttributeError: module 'app' has no attribute 'web'
Failed to find attribute 'web' in 'app'.
[2024-09-13 14:09:52 +0000] [4] [INFO] Worker exiting (pid: 4)
[2024-09-13 14:09:52 +0000] [1] [INFO] Shutting down: Master
[2024-09-13 14:09:52 +0000] [1] [INFO] Reason: App failed to load.
container event container died
[2024-09-13 14:09:53 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2024-09-13 14:09:53 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2024-09-13 14:09:53 +0000] [1] [INFO] Using worker: sync
[2024-09-13 14:09:53 +0000] [4] [INFO] Booting worker with pid: 4
Traceback (most recent call last):
File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 402, in import_app
app = getattr(mod, name)
^^^^^^^^^^^^^^^^^^AttributeError: module 'app' has no attribute 'web'
Failed to find attribute 'web' in 'app'.
[2024-09-13 14:09:53 +0000] [4] [INFO] Worker exiting (pid: 4)
container event container restart
[2024-09-13 14:09:54 +0000] [1] [INFO] Shutting down: Master
[2024-09-13 14:09:54 +0000] [1] [INFO] Reason: App failed to load.
[2024-09-13 14:09:55 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2024-09-13 14:09:55 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2024-09-13 14:09:55 +0000] [1] [INFO] Using worker: sync
[2024-09-13 14:09:55 +0000] [4] [INFO] Booting worker with pid: 4
Traceback (most recent call last):
File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 402, in import_app
app = getattr(mod, name)
^^^^^^^^^^^^^^^^^^AttributeError: module 'app' has no attribute 'web'
Failed to find attribute 'web' in 'app'.
[2024-09-13 14:09:55 +0000] [4] [INFO] Worker exiting (pid: 4)
[2024-09-13 14:09:55 +0000] [1] [INFO] Shutting down: Master
[2024-09-13 14:09:55 +0000] [1] [INFO] Reason: App failed to load.
container event container died
[2024-09-13 14:09:56 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2024-09-13 14:09:56 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2024-09-13 14:09:56 +0000] [1] [INFO] Using worker: sync
[2024-09-13 14:09:56 +0000] [4] [INFO] Booting worker with pid: 4
Traceback (most recent call last):
File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 402, in import_app
app = getattr(mod, name)
^^^^^^^^^^^^^^^^^^AttributeError: module 'app' has no attribute 'web'
Failed to find attribute 'web' in 'app'.
[2024-09-13 14:09:56 +0000] [4] [INFO] Worker exiting (pid: 4)
[2024-09-13 14:09:56 +0000] [1] [INFO] Shutting down: Master
[2024-09-13 14:09:56 +0000] [1] [INFO] Reason: App failed to load.
container event container died
[2024-09-13 14:09:57 +0000] [1] [INFO] Using worker: sync
2 years ago
Can you share your repo? Your application might be called app and not web, in which case, you would use app:app
2 years ago
yes we do
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): feed-api.cielo.finance:443
[2024-09-13 14<:16:1199656381486219374>51 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:4)
[2024-09-13 14<:16:1199656381486219374>51 +0000] [4] [INFO] Worker exiting (pid: 4)
[2024-09-13 14<:16:1199656381486219374>51 +0000] [5] [INFO] Booting worker with pid: 5
2 years ago
how long do you expect a single request to take
it depends on the response but should not take more than 5-7 seconds tbh
2 years ago
when testing, how many RPS are you doing?
2 years ago
requests per second
2 years ago
then you are likely running into issues with having run gunicorn with sync workers
2 years ago
start it with the uvicorn event worker
2 years ago
what's your current start command
2 years ago
no need to ping reply
2 years ago
what kind of app? fastapi, flask?
2 years ago
your start command would become gunicorn app:app --timeout 120 --worker-class eventlet
you will also need to make sure eventlet is in your requirements.txt file
Flask==2.0.3
Werkzeug==2.0.3
Jinja2==3.0.3
itsdangerous==2.0.1
click==8.0.4
gunicorn==20.1.0
requests==2.26.0
prettytable==2.5.0
Flask-Login==0.5.0
2 years ago
the latest is probably fine
Error: class uri 'eventlet' invalid or not found:
[Traceback (most recent call last):
File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 99, in load_class
mod = importlib.import_module('.'.join(components))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "/root/.nix-profile/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "", line 1204, in gcdimport
File "", line 1176, in findand_load
File "", line 1147, in findandloadunlocked
File "", line 690, in loadunlocked
File "", line 940, in exec_module
File "", line 241, in callwithframesremoved
File "/opt/venv/lib/python3.11/site-packages/gunicorn/workers/geventlet.py", line 20, in
from eventlet.wsgi import ALREADY_HANDLED as EVENTLET_ALREADY_HANDLEDImportError: cannot import name 'ALREADY_HANDLED' from 'eventlet.wsgi' (/opt/venv/lib/python3.11/site-packages/eventlet/wsgi.py)
]
container event container died
2 years ago
do you have it in your requirements.txt?
Flask==2.0.3
Werkzeug==2.0.3
Jinja2==3.0.3
itsdangerous==2.0.1
click==8.0.4
gunicorn==20.1.0
requests==2.26.0
prettytable==2.5.0
Flask-Login==0.5.0
eventlet==0.33.3
2 years ago
whats your start command
2 years ago
^
2 years ago
no, we support procfiles
2 years ago
okay try gevent
invalid type: string "gunicorn --timeout 120 --worker-class gevent app:app", expected a ma
2 years ago
try moving app:app to after gunicorn
context: 5f53c0eeaf434d6d998e9ed8ac0a2c0f
Nixpacks build failed
Error: Reading Procfile
Caused by:
invalid type: string "gunicorn app:app --timeout 120 --worker-class gevent", expected a map
gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/nix/store/wwg6376ymkhpjxy6c3w9gkbhv6l2gmqy-libxcrypt-4.4.36/include -fPIC -DLIBEVEMBED=1 -DEVCOMMON= -DEVCLEANUPENABLE=0 -DEVEMBEDENABLE=0 -DEVPERIODICENABLE=0 -DEVUSEREALTIME=1 -DEVUSEMONOTONIC=1 -DEVUSEFLOOR=1 -Isrc/gevent/libev -I/nix/store/k3701zl6gmx3la7y4dnflcvf3xfy88kh-python3-3.11.9/include/python3.11 -I/root/.nix-profile/include/python3.11 -I/tmp/pip-install-mht4g7p5/gevent7b95d12f6294466681fc5ce8f6126ea3/deps -I/tmp/pip-install-mht4g7p5/gevent7b95d12f6294466681fc5ce8f6126ea3/src/gevent/libev -I/tmp/pip-install-mht4g7p5/gevent7b95d12f6294466681fc5ce8f6126ea3/deps/libev -Isrc/gevent -Isrc/gevent/libev -Isrc/gevent/resolver -I. -I/opt/venv/include -I/nix/store/k3701zl6gmx3la7y4dnflcvf3xfy88kh-python3-3.11.9/include/python3.11 -c src/gevent/libev/callbacks.c -o build/temp.linux-x8664-cpython-311/src/gevent/libev/callbacks.o
11 103.8 gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/nix/store/wwg6376ymkhpjxy6c3w9gkbhv6l2gmqy-libxcrypt-4.4.36/include -fPIC -DLIBEVEMBED=1 -DEVCOMMON= -DEVCLEANUPENABLE=0 -DEVEMBEDENABLE=0 -DEVPERIODICENABLE=0 -DEVUSEREALTIME=1 -DEVUSEMONOTONIC=1 -DEVUSEFLOOR=1 -Isrc/gevent/libev -I/nix/store/k3701zl6gmx3la7y4dnflcvf3xfy88kh-python3-3.11.9/include/python3.11 -I/root/.nix-profile/include/python3.11 -I/tmp/pip-install-mht4g7p5/gevent7b95d12f6294466681fc5ce8f6126ea3/deps -I/tmp/pip-install-mht4g7p5/gevent7b95d12f6294466681fc5ce8f6126ea3/src/gevent/libev -I/tmp/pip-install-mht4g7p5/gevent7b95d12f6294466681fc5ce8f6126ea3/deps/libev -Isrc/gevent -Isrc/gevent/libev -Isrc/gevent/resolver -I. -I/opt/venv/include -I/nix/store/k3701zl6gmx3la7y4dnflcvf3xfy88kh-python3-3.11.9/include/python3.11 -c src/gevent/libev/corecext.c -o build/temp.linux-x8664-cpython-311/src/gevent/libev/corecext.o
11 103.8 In file included from src/gevent/libev/libev.h:9,
11 103.8 from src/gevent/libev/corecext.c:1319:
11 103.8 /tmp/pip-install-mht4g7p5/gevent_7b95d12f6294466681fc5ce8f6126ea3/deps/libev/ev.c:580:48: warning: "/*" within comment [-Wcomment]
11 103.8 580 | /#define MIN_INTERVAL 0.00000095367431640625 / 1/2**20, good till 2200 */
11 103.8 |
11 103.8 /tmp/pip-install-mht4g7p5/gevent7b95d12f6294466681fc5ce8f6126ea3/deps/libev/ev.c: In function ‘ecbbinary32tobinary16’:
File "/opt/venv/lib/python3.11/site-packages/eventlet/greenio/base.py", line 32, in
socket_timeout = eventlet.timeout.wrap_is_timeout(socket.timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "/opt/venv/lib/python3.11/site-packages/eventlet/timeout.py", line 166, in wrapistimeout
[base.is](base.is)_timeout = property(lambda _: True)
^^^^^^^^^^^^^^^TypeError: cannot set 'is_timeout' attribute of immutable type 'TimeoutError'
]
container event container died
2 years ago
yes, set it in a runtime.txt file
2 years ago
is your app now using gevent
2 years ago
you woud need to get one of them working
2 years ago
why do you say that
2 years ago
i cant verify that, you are about to delete the service
2 years ago
this wasnt an issue with railway so the timeout issue isnt going to change
2 years ago
you are not running sync workers locally
2 years ago
are you sure this is flask?
[2024-09-13 16<:16:1199656381486219374>28 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:4)
[2024-09-13 16<:16:1199656381486219374>28 +0000] [4] [INFO] Worker exiting (pid: 4)
[2024-09-13 16<:16:1199656381486219374>28 +0000] [5] [INFO] Booting worker with pid: 5
2 years ago
you need to use an async worker
2 years ago
im not sure
2 years ago
in a production environment you must use gunicorn
2 years ago
and in your case you also need to use async workers
2 years ago
try gthread then
increasaed timeout sufficiently large so that works but towards the end, the final request the response got chopped off
2 years ago
no, only a 5 minute time limit
2 years ago
increasing the time-out here is just a Band-Aid solution
2 years ago
that not not indicate an issue with railway, that indicates an issue with your configurations on railway
2 years ago
you need to use an async worker so that your app can handle more then 3 requests concurrently
yea, will need to implement celery and redis but was hoping that I can do that later.
2 years ago
you can do celery and redis later, right now you need to use an async worker
2 years ago
you have something misconfigured and will need to look into why that's happening
by importing asyncio & aiohttp and adding async with in front of all my request fetch functions
2 years ago
railway doesnt do any asset caching, is it behind cloudflare?
a year ago
please provide the error
/bin/bash: line 1: web:: command not found
container event container died
a year ago
what is the contents of your procfile
a year ago
I'll check back with this when I'm back at the computer so I can view your logs, I don't think you are sending me the correct error
a year ago
wait, aren't you deploying from a Dockerfile? why do you have a procfile
a year ago
haha slow down there
a year ago
send me your Dockerfile please
I only have the node dev environment on my end. Then I commit my changes to Github. Railway auto deploys
a year ago
you do not have a Dockerfile?
a year ago
delete your Procfile
a year ago
you should have a start script in your package.json
"scripts": {
"start": "node app.js",
"dev": "app.js",
"test": "echo \"Error: no test specified\" && exit 1"
},
a year ago
then you have no need for a procfile
a year ago
thats not valid syntax

a year ago
no problem!
when you upgrade the plan, does the app needs to be redeloyed to take advantage of the new RAM etc?
