variables not setting
tattoonow
PROOP

a month ago

We're experiencing a critical deployment issue where code changes are not being applied to the running service, despite multiple deployment attempts and cache-busting strategies.

Service Details:

  • Service: tattoonow-platform (Project: tattoonow-products)
  • Repository: TattooNOW/tattoonow-business-suite
  • Branch: claude/fix-image-uploader-7Vkfj

Issue:

We've made multiple commits to update hardcoded R2 credentials in lib/r2.ts:

  • Commit: 0d9a2a1 (latest) — hardcodes R2 access key to 868f5dde39b18691c6fbfa799d079**
  • Previous attempts: Added comments, toggled Metal Build Environment, multiple redeploys

What we're observing:

  • Environment variable R2_KEY is correctly set in Railway to 868f5dde39b18691c6fbfa799d07***
  • Logs show the app is using keyId: f4269199... (wrong value)
  • New code changes are not reflected in deployments
  • Deployments complete successfully but run old compiled code

What we've tried:

  1. Cleared build cache (Metal Build Environment toggle)
  2. Multiple trivial code changes to invalidate cache
  3. Verified git commits were pushed
  4. Verified environment variables are set correctly in Railway

Conclusion:

The build system appears to be using stale compiled code and not recompiling from the latest source. This is blocking production image uploads.

Could you investigate why deployments are not picking up recent code changes?

$20 Bounty

7 Replies

Railway
BOT

a month ago

Your latest deployment completed successfully, and the runtime logs show no R2-related errors or key references, only unrelated database column errors. On our side, we confirmed the deployment is running and there is no platform-level issue preventing code from being picked up. Since you mention the app is using the wrong keyId value (f4269199...) despite your environment variable being set correctly, this strongly suggests the R2 credentials are being resolved at build time rather than runtime, likely through Next.js build-time inlining or a .env file committed to the repo. Toggling the Metal Build Environment is not the same as disabling build layer caching. To force a fully clean build with no cached layers, set the environment variable NO_CACHE=1 in your service variables, trigger a new deployment, then remove it afterward. See Disable build layer caching for details.


domehane
FREE

a month ago

Hello tattoonow,

your env var is correct but the value is being resolved at build time, not runtime, so the old key keeps getting baked in

two things to do ;

first, set NO_CACHE=1 in your railway service variables, trigger a new deploy, then remove it after , this forces a fully clean build, not just a cache toggle

second, run git grep f4269199 in your repo. if that old key shows up anywhere in a committed .env or config file, thats where its coming from and you need to remove it and rotate the key immediately

thats it, its definitely coming from your codebase or build cache

Hope this help you :)


tattoonow
PROOP

a month ago

thnaks@ almost there. This is where Im at. some of the variables are in blue, maybe thats a clue?

We've now isolated the exact issue and need escalation.

Previous Support (Addressed):

  • Railway Support suggested: "R2 credentials are being resolved at build time rather than runtime, likely through Next.js build-time inlining"
  • Domehane suggested: Set NO_CACHE=1 to force clean build (we tried this — did not resolve the issue)
  • Both recommended checking for committed .env files (we confirmed none exist via git grep)

What We've Done Since:

  1. Implemented runtime env var reading inside API route handler (not module-level)
  2. Used dynamic access patterns to prevent Next.js static analysis
  3. Forced full clean builds with NO_CACHE=1
  4. Verified fresh R2 credentials were valid by temporarily hardcoding them — upload succeeded
  5. Confirmed hardcoded values work, env var lookups return undefined

The Real Problem:

Service variables ARE in the Railway dashboard and ARE being passed to the container (we see them via diagnostic logging), but process.env.AWS_ACCESS_KEY_ID and process.env.AWS_SECRET_ACCESS_KEY return undefined at runtime.

Critical Evidence:

  • R2_ACCOUNT_ID and R2_BUCKET_NAME work fine via process.env
  • AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY do not, despite being in the dashboard
  • Hardcoding these exact values in code → upload succeeds
  • Removing hardcodes, relying on env vars → "credentials not found" error
  • This is selective to AWS_* prefixed variables only

Diagnostic Output:

🔍 R2 env check in route — 

R2_ACCOUNT_ID: true

R2_BUCKET_NAME: true

AWS_ACCESS_KEY_ID: false

AWS_SECRET_ACCESS_KEY: false

Service Details:

  • Service: tattoonow-platform (project: tattoonow-products)
  • Latest deployment: 52c7da5b-af9d-41a7-8717-080b726b6616
  • Framework: Next.js 16.2.1
  • Build command uses standard next build

Question:

Is there a Railway-specific limitation on which env var name prefixes are injected to process.env? Or is this a bug where certain variable names aren't reaching the runtime despite being in the dashboard?

Thanks again in advance!! The prompt replies are great signal

Attachments


domehane
FREE

a month ago

look at the bottom of your variables screenshot , "8 variables added by railway" railway injects its own internal variables into every container, and AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are most likely conflicting with those. thats why those two specifically fail while R2_ACCOUNT_ID and R2_BUCKET_NAME work fine

worth trying renaming your variables to something like R2_ACCESS_KEY_ID and R2_SECRET_ACCESS_KEY, update your code to match, and redeploy. avoids any possible naming conflict with railway-injected vars


tattoonow
PROOP

a month ago

doesn't look like ti.. that woulda been nice!

Attachments


domehane
FREE

a month ago

okay so those 8 variables are all RAILWAY_* prefixed so no conflict there

i have a qst to you , in your upload api route file, do you have export const runtime = 'edge' anywhere at the top of the file or in any file it imports from?


smokedmeats
HOBBY

a month ago

New user here and this will not be technical jargon. I want to post what I noticed while setting up in case it helps. There are obviously two ways to update vars: 1. delete old var - re-add same var with new value. - Railway does NOT recognize the new value when doing this, even if you deploy after just 1 var. 2. Click hamburger next to var and EDIT, and SOMETIMES it seems to recognize the new var. There also may be issues when batch added vars or batch changing already added vars with new values before deploying. I'm @XyberRun in your discord if questions.


Welcome!

Sign in to your Railway account to join the conversation.

Loading...