Technical Problem - stratos
sim1693
HOBBYOP

a month ago

Hi, I have multiple services in my project (workspace "zonal-transformation", environment "production": stratos — a Python worker, stratos-backend — FastAPI/Uvicorn, stratos-frontend — Streamlit) that keep restarting repeatedly with no obvious cause on the application side. I've already ruled out several things before writing in:

  • Not memory: checked the Metrics tab several times during the restarts — max memory observed was ~250MB, no sign of OOM (no "flat ceiling then drop to zero" shape typical of an OOM kill; the graph stays irregular, consistent with normal usage).
  • Not an overlapping redeploy: checked the timestamps of my GitHub pushes — no push happened in the windows where the services restarted.
  • Not an unhandled Python exception: added PYTHONUNBUFFERED=1 on the frontend to make sure a traceback wasn't just sitting in a stdout buffer — still zero tracebacks in the logs. The logs only show a clean shutdown sequence: "Stopping..." → "Stopping Container" → "Starting Container" (frontend, Streamlit) and "Shutting down" → "Waiting for application shutdown" → "Application shutdown complete" → "Finished server process [1]" (backend, Uvicorn) — a normal SIGTERM-driven graceful shutdown, not a crash with a traceback.
  • Manually reviewed the backend's write-endpoint code (POST/DELETE on /posizioni and /alert, the ones exercised when the issue shows up) — standard pattern (open connection, query, commit, close in a finally), no plausible unhandled exception, nothing that looks like a code bug here.
  • Not tied exclusively to one specific user action: it initially looked like it only fired when clicking "Delete" in a Streamlit form (st.rerun()), but it has also happened with nobody interacting with the app at all.
  • Strongest clue: in different occurrences, stratos-frontend (Streamlit), stratos-backend (FastAPI/Uvicorn), and stratos (a third, unrelated Python scheduler process) have all restarted — three independent tech stacks with no shared code. It seems unlikely to be an isolated app bug repeating itself in all three; it points more toward something at the environment/project level.
  • The Deployments tab doesn't show an explicit reason (e.g. "Healthcheck failed") for these instances — just the stop/start cycle.

Observed frequency: several restarts on the evening of 07/10 (~22:45-22:56) and again this morning on 07/11 (two episodes, ~06:42-06:50 and ~06:56), at irregular intervals ranging from 20 seconds to a few minutes apart.

Could you check on your end (platform logs, node scheduling/migration events, or a default healthcheck I'm not seeing) what's causing these restarts in the "production" environment of this project?

Thanks

Solved

1 Replies

Railway
BOT

a month ago

The deployment history for your production environment shows that the restarts coincide with variable commit events, which are triggered when shared or environment-level variables are updated and committed. When a shared variable is modified, every service that inherits it is automatically redeployed, which explains why stratos, stratos-backend, and stratos-frontend all restart simultaneously even without a code push. Check your project's shared variables page and the activity feed in your project settings to see if any variable changes were made around those times, including changes made via API tokens.


Status changed to Awaiting User Response Railway about 1 month ago


Railway
BOT

a month ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway about 1 month ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...