2 years ago
Hi there,
I'm having troubles dockerizing my next.js app, at this point I'm able to get it building and using ARG to get the environment variables on buildtime but for some reason they're not being passed through on runtime.
Here's my dockerfile:
FROM node:18-alpine AS base
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
ENV BOT_WEBHOOK_API_KEY=$BOT_WEBHOOK_API_KEY \
CLIENT_ID=$CLIENT_ID \
CLIENT_SECRET=$CLIENT_SECRET \
DATABASE_URL=$DATABASE_URL \
REDIRECT_URI=$REDIRECT_URI \
VATSIM_API_KEY=$VATSIM_API_KEY
# Install dependencies only when needed
FROM base AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
ENV BOT_WEBHOOK_API_KEY=$BOT_WEBHOOK_API_KEY \
CLIENT_ID=$CLIENT_ID \
CLIENT_SECRET=$CLIENT_SECRET \
DATABASE_URL=$DATABASE_URL \
REDIRECT_URI=$REDIRECT_URI \
VATSIM_API_KEY=$VATSIM_API_KEY
# Install dependencies based on the preferred package manager
COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
COPY prisma ./prisma/
RUN \
if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
elif [ -f package-lock.json ]; then npm ci; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm i --frozen-lockfile; \
else echo "Lockfile not found." && exit 1; \
fi
# Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
ENV BOT_WEBHOOK_API_KEY=$BOT_WEBHOOK_API_KEY \
CLIENT_ID=$CLIENT_ID \
CLIENT_SECRET=$CLIENT_SECRET \
DATABASE_URL=$DATABASE_URL \
REDIRECT_URI=$REDIRECT_URI \
VATSIM_API_KEY=$VATSIM_API_KEY
# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry during the build.
# ENV NEXT_TELEMETRY_DISABLED 1
RUN \
if [ -f yarn.lock ]; then yarn run build; \
elif [ -f package-lock.json ]; then npm run build; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm run build; \
else echo "Lockfile not found." && exit 1; \
fi
# Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
ENV BOT_WEBHOOK_API_KEY=$BOT_WEBHOOK_API_KEY \
CLIENT_ID=$CLIENT_ID \
CLIENT_SECRET=$CLIENT_SECRET \
DATABASE_URL=$DATABASE_URL \
REDIRECT_URI=$REDIRECT_URI \
VATSIM_API_KEY=$VATSIM_API_KEY
ENV NODE_ENV production
# Uncomment the following line in case you want to disable telemetry during runtime.
# ENV NEXT_TELEMETRY_DISABLED 1
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
COPY --from=builder /app/public ./public
# Set the correct permission for prerender cache
RUN mkdir .next
RUN chown nextjs:nodejs .next
# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
USER nextjs
EXPOSE 3000
ENV PORT 3000
# server.js is created by next build from the standalone output
# https://nextjs.org/docs/pages/api-reference/next-config-js/output
CMD HOSTNAME="0.0.0.0" node server.js208 Replies
Fixed it, for future reference there's no need to set the ENV everywhere, just `ARGz
Turns out what i said previously did not fix it, I've removed the solved tag for now
2 years ago
Try without the ARG to ENV assignments
I'll try and do it again but it doesn't look like it worked the first time
2 years ago
please do, regardless if it does or doesn't work, it's not needed so it's good to rule it out anyway
yeah thats the thing, I had tried that initially and it looked like it was working until i got a message from someone using the service and it turned out it wasn't so 🤷♂️ , redeploying now
2 years ago
also probably a good idea to remove the EXPOSE and ENV PORT lines too
2 years ago
does a standalone next build require variables to be prefixed with something? like NEXT_?
It's strange, the environment variables are being passed on buildtime but not runtime
2 years ago
variables are most certainly being passed at runtime, there's just something simple we are overlooking here
but in the cases where I'm using my env vars, no, as it is code being ran on the server rather than the client
2 years ago
and you're sure the environment variables that are missing aren't trying to be statically built into the code?
i mean they shouldn't be, it was working fine when i was using the default nixpacks
i just wanted to use docker so i could specify an image that isn't as RAM intensive
2 years ago
nixpacks includes every single service variable you have as an ARG in its generated dockerfile, so that may explain why it works
2 years ago
so try to specify all your environmental variables as ARGs in the correct layer of your dockerfile
but i'm specifying all of them in every layer, surely they should be available then?
2 years ago
you only have 6 service variables?
2 years ago
okay this is getting strange, have you tried a simple console log in the backend for some of those environment variables?
i know for sure that the DATABASE_URL and CLIENT_ID variables are not being passed based on my deploy logs and through my OAuth login where it's setting one of the URL params as &client_id=undefined
from that i'd say it's safe to assume that none of the environment variables are being set properly
2 years ago
would you happen to be committing an .env file to GitHub?
2 years ago
is your code trying to somehow load the .env.example?
i highly doubt it, that's never happened before and like i said the environment variables are working fine in build time where i know at least DATABASE_URL is being used
luckily for now i can just rollback to the nixpacks build but it would be nice to get a Dockerfile working
2 years ago
definitely something simple we are overlooking here
2 years ago
I'll try to reproduce
very likely, unfortunately the documentation isn't very detailed on what happens with environment variables when using Dockerfiles
2 years ago
the docs do actually cover how to use environment variables during the build, and during runtime there is nothing you need to do because service variables are automatically injected into the containers environment
2 years ago
ARG is short for build arguments, they shouldn't be put into the environment of the built image, but I've seen stranger things
however i could still be right, let me remove the ARGs from the base layer as well
2 years ago
they should only be needed in the layer that runs commands that use the environment variables
does the ARG command actually set it as an environment variable though? or as a variable inside of the docker file, because technically the DATABASE_URL isn't being used inside of the dockerfile itself, it just needs to be set in order for prisma to generate types
removing it from the base layer and only setting it in the layers where they're needed fixed it
2 years ago
ARG is short for build arguments, specifying them brings the build arguments into the build but not the runtime, railway when building from the dockerfile will specify all service variables as build arguments
I'm not quite sure here then, the environment variables are being set fine during the build because next.js is building fine, but then when it gets to runtime they no longer seem to be set
2 years ago
yes I am currently stumped here too
2 years ago
just for debugging purposes, can you change your dockerfile to only use a single FROM?
FROM node:18-alpine AS base
# Install dependencies only when needed
FROM base AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
# Install dependencies based on the preferred package manager
COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
COPY prisma ./prisma/
RUN \
if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
elif [ -f package-lock.json ]; then npm ci; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm i --frozen-lockfile; \
else echo "Lockfile not found." && exit 1; \
fi
# Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry during the build.
# ENV NEXT_TELEMETRY_DISABLED 1
RUN \
if [ -f yarn.lock ]; then yarn run build; \
elif [ -f package-lock.json ]; then npm run build; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm run build; \
else echo "Lockfile not found." && exit 1; \
fi
# Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app
ENV NODE_ENV production
# Uncomment the following line in case you want to disable telemetry during runtime.
# ENV NEXT_TELEMETRY_DISABLED 1
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
COPY --from=builder /app/public ./public
# Set the correct permission for prerender cache
RUN mkdir .next
RUN chown nextjs:nodejs .next
# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
USER nextjs
# server.js is created by next build from the standalone output
# https://nextjs.org/docs/pages/api-reference/next-config-js/output
CMD HOSTNAME="0.0.0.0" node server.jsshall i just remove all of the instances of FROM? apart from the first one of course
2 years ago
haha no you don't
2 years ago
yep that's what I meant!
2 years ago
ah gotcha, I see what you meant
2 years ago
so yeah try with only a single FROM and of course remove all the copy stuff
2 years ago
and for extra good measure, no user / chown stuff either
2 years ago
basically we want the most barebones dockerfile, then if we get that working, you can start adding the fancy stuff back for process of elimination
yeah i removed all of the copy stuff but docker has it all cached and it's a bit screwy
2 years ago
set NO_CACHE to 1
2 years ago
service variables
2 years ago
yeah thats the joys of not using a multi stage image haha, but thats too be expected
FROM node:18-alpine
RUN apk add --no-cache libc6-compat
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
COPY prisma ./prisma/
RUN \
if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
elif [ -f package-lock.json ]; then npm ci; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm i --frozen-lockfile; \
else echo "Lockfile not found." && exit 1; \
fi
COPY . .
RUN \
if [ -f yarn.lock ]; then yarn run build; \
elif [ -f package-lock.json ]; then npm run build; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm run build; \
else echo "Lockfile not found." && exit 1; \
fi
ENV NODE_ENV production
CMD HOSTNAME="0.0.0.0" node server.js2 years ago
what package manager do you use locally?
2 years ago
then youd only want to copy in package.json and pnpm-lock.yaml in the first COPY
2 years ago
haha well whats the status on the missing environment variables?
so i was getting an error that /app/server.js wasn't found, which making sense because i removed the COPY commands that copied those files over
2 years ago
simplify it more
FROM node:18-alpine
RUN apk add --no-cache libc6-compat
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
COPY package.json pnpm-lock.yaml* ./
COPY prisma ./prisma/
RUN corepack enable pnpm && pnpm i --frozen-lockfile
COPY . .
RUN pnpm run build
COPY . .
COPY /app/public ./public
COPY /app/.next/standalone ./
COPY /app/.next/static ./.next/static
ENV NODE_ENV production
CMD HOSTNAME="0.0.0.0" node server.js2 years ago
what makes you think its caching still?
2 years ago
show me how you have set that variable please
2 years ago
^
2 years ago
and now it still says CACHED?
2 years ago
hmmm odd
2 years ago
do tell!
basically i'm a noob at docker and there's a difference between what COPY and COPY --from does
2 years ago
oh haha i thought you meant you found out why the runtime envs weren't working
but i think i've got a minimal dockerfile working at this point so we can work from here
FROM node:18-alpine
RUN apk add --no-cache libc6-compat
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
COPY package.json pnpm-lock.yaml* ./
COPY prisma ./prisma/
RUN corepack enable pnpm && pnpm i --frozen-lockfile
COPY . .
RUN pnpm run build
RUN cp -r /app/.next/standalone/* ./
ENV NODE_ENV production
CMD HOSTNAME="0.0.0.0" node server.js2 years ago
can you show me how you are accessing these environment variables in code?
as i said it's worked pretty much everywhere else except using Dockerfiles
2 years ago
okay ill try to reproduce
2 years ago
works fine



2 years ago
obviously an extremely simplified test, but this working means this isnt a platform issue
2 years ago
try doing something like this
CMD echo ${variable name here}; node server.js2 years ago
do both things
2 years ago
share your repo? theres a slim chance something sticks out to me
would you mind if i added you to the repo? i don't mind you having a look at the code but i can't make the repo public right now
2 years ago
for sure, body192
2 years ago
looking
2 years ago
welp
2 years ago
i have missspelt my own name
2 years ago
my r key is going bad on my keyboard
2 years ago
my gh username is brody192
2 years ago
my bad
2 years ago
got it
2 years ago
feels

don't worry about having to blur anything out in screenshots btw i'm not that bothered, the repo just can't be publically accessible haha
2 years ago
so just to be clear, that echo in your start command does work?
2 years ago
now what would a google search for "nextjs cant read environment variables" come up with haha, since we have ruled out a platform issue
2 years ago
and for fun, can you delete the .env.example file
also this is interesting: https://github.com/vercel/next.js/issues/46296
2 years ago
try this?

2 years ago
true, doesnt work for env()
2 years ago
prisma is definitely able to read from the environment directly, something funky is going on here and I'm not sure what
2 years ago
as an ENV?
2 years ago
something real funky is going on with your project, try just running normal next start
and honestly after this if it doesn't work i'll just go back to nixpacks
2 years ago
so next start works
2 years ago
what a fun debug process!
2 years ago
now you can go back to that mutli layer dockerfile!
yep, i might file an issue on the next.js repo for this, it's a pretty weird bug
2 years ago
watch them blame the platform haha
2 years ago
happy to help!
2 years ago
lmao
2 years ago
learn javascript they said, it will be fun they said
2 years ago
haha simple fix too


