Environment variables not persisting on @medialink/dossier-service (Production)
wadagni-verite
HOBBYOP

21 days ago

Environment variables not persisting on @medialink/dossier-service (Production)

Description :

Project: robust-spirit (ID: b2f4614a-696e-445f-a887-6d6faa3738a3)

Environment: production

Service: @medialink/dossier-service

Problem:

Environment variables added via Dashboard/API disappear before container runtime. Only SUPABASE_SERVICE_KEY persists. DATABASE_URL, SUPABASE_URL, SUPABASE_ANON_KEY, SUPABASE_SECRET_KEY, SUPABASE_BUCKET all vanish.

Evidence:

  • Build succeeds, runtime crashes with "PrismaClientInitializationError: error: Environment variable not found: DATABASE_URL"
  • After adding DATABASE_URL via API, getServiceConfig shows only SUPABASE_SERVICE_KEY — variable is gone
  • Pattern repeats across last 3 deployments (4f5a6311, 556de584, b93cc267)
  • Same service runs fine on staging with identical code — variables are not the issue there
  • MEDIALINK_ENV (created by user) persists fine

What we've tried:

  • Added DATABASE_URL hardcoded (Supabase URL) — disappears
  • Added SUPABASE_* variables — all disappear
  • Multiple redeployments — pattern persists
  • Verified no railway.json variables section (same as other 9 services in repo)

This is blocking Production. Need help diagnosing why variables are not reaching container at runtime.

Latest failing deployment: 4f5a6311-857c-48e4-8d11-bee00e18c7d3

$10 Bounty

3 Replies

Railway
BOT

21 days ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway 21 days ago


21 days ago

Make sure you have deployed the staged changes after editing your variables


manuproject
HOBBY

21 days ago

variables dont disappear on their own - the fact that they get deleted and its always the same one surviving means something with api access is rewriting the variable collection. railways variableCollectionUpsert with replace wipes every var you dont include in the call, so if you have a ci step or script that "syncs" env vars and its payload only has SUPABASE_SERVICE_KEY, it deletes the rest on every deploy. staging survives because whatever syncs staging has the full list (or nothing syncs it).

how to confirm: open the project activity feed in the dashboard and look at what happens right after you add DATABASE_URL - you should see a variable change event from a token/integration, not a person. also check any deploy workflow, github action, or infra script for a railway api call or railway variables --set.

fix is either include all vars in that sync payload or drop the replace behavior. adding vars by hand will keep losing this race until the writer is stopped


brezzy1337
HOBBYTop 10% Contributor

16 days ago

Hey — I was able to reproduce your error: PrismaClientInitializationError: error: Environment variable not found: DATABASE_URL

manuproject's diagnosis above is spot on. To put it plainly: something with API access is calling variableCollectionUpsert with replace semantics, and that deletes every variable left out of the payload. So do the checks posted above.

  • Checking the activity feed. Find the variable-change event just after you last added DATABASE_URL. It names the actor, so a token or integration rather than your own account confirms it.

  • Auditing the deploy pipeline. One thing that narrows this down a lot: replace defaults to false. I tested the same partial payload with the flag left off and nothing was deleted. So ordinary API writes and plain railway variables --set calls aren't suspects — you are looking for one call that passes replace: true specifically.

    Search your pipeline for replace alongside variableCollectionUpsert — GitHub Actions workflows, deploy scripts, Terraform or Pulumi, any internal config-sync tool.

  • Fix the payload, not the flag. Once you find the call, two options:

    1. Keep replace: true and send your complete variable set. If the script passes replace on purpose to prune keys deleted upstream, keep it — just make the payload complete.
    2. Otherwise drop replace and let the default merge apply.

Then re-add the missing variables and redeploy.

Hope this post clarifies the issue, appreciate manuproject for scoping this in. Looking forward to updates on your issue, hopefully a confirmation on the fix :)


Welcome!

Sign in to your Railway account to join the conversation.

Loading...