a year ago
Hi there,
I'm having troubles dockerizing my next.js app, at this point I'm able to get it building and using ARG to get the environment variables on buildtime but for some reason they're not being passed through on runtime.
Here's my dockerfile:
FROM node:18-alpine AS base
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
ENV BOT_WEBHOOK_API_KEY=$BOT_WEBHOOK_API_KEY \
CLIENT_ID=$CLIENT_ID \
CLIENT_SECRET=$CLIENT_SECRET \
DATABASE_URL=$DATABASE_URL \
REDIRECT_URI=$REDIRECT_URI \
VATSIM_API_KEY=$VATSIM_API_KEY
# Install dependencies only when needed
FROM base AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
ENV BOT_WEBHOOK_API_KEY=$BOT_WEBHOOK_API_KEY \
CLIENT_ID=$CLIENT_ID \
CLIENT_SECRET=$CLIENT_SECRET \
DATABASE_URL=$DATABASE_URL \
REDIRECT_URI=$REDIRECT_URI \
VATSIM_API_KEY=$VATSIM_API_KEY
# Install dependencies based on the preferred package manager
COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
COPY prisma ./prisma/
RUN \
if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
elif [ -f package-lock.json ]; then npm ci; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm i --frozen-lockfile; \
else echo "Lockfile not found." && exit 1; \
fi
# Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
ENV BOT_WEBHOOK_API_KEY=$BOT_WEBHOOK_API_KEY \
CLIENT_ID=$CLIENT_ID \
CLIENT_SECRET=$CLIENT_SECRET \
DATABASE_URL=$DATABASE_URL \
REDIRECT_URI=$REDIRECT_URI \
VATSIM_API_KEY=$VATSIM_API_KEY
# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry during the build.
# ENV NEXT_TELEMETRY_DISABLED 1
RUN \
if [ -f yarn.lock ]; then yarn run build; \
elif [ -f package-lock.json ]; then npm run build; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm run build; \
else echo "Lockfile not found." && exit 1; \
fi
# Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
ENV BOT_WEBHOOK_API_KEY=$BOT_WEBHOOK_API_KEY \
CLIENT_ID=$CLIENT_ID \
CLIENT_SECRET=$CLIENT_SECRET \
DATABASE_URL=$DATABASE_URL \
REDIRECT_URI=$REDIRECT_URI \
VATSIM_API_KEY=$VATSIM_API_KEY
ENV NODE_ENV production
# Uncomment the following line in case you want to disable telemetry during runtime.
# ENV NEXT_TELEMETRY_DISABLED 1
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
COPY --from=builder /app/public ./public
# Set the correct permission for prerender cache
RUN mkdir .next
RUN chown nextjs:nodejs .next
# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
USER nextjs
EXPOSE 3000
ENV PORT 3000
# server.js is created by next build from the standalone output
# https://nextjs.org/docs/pages/api-reference/next-config-js/output
CMD HOSTNAME="0.0.0.0" node server.js0 Replies
Fixed it, for future reference there's no need to set the ENV everywhere, just `ARGz
Turns out what i said previously did not fix it, I've removed the solved tag for now
a year ago
Try without the ARG to ENV assignments
a year ago
please do, regardless if it does or doesn't work, it's not needed so it's good to rule it out anyway
yeah thats the thing, I had tried that initially and it looked like it was working until i got a message from someone using the service and it turned out it wasn't so 🤷♂️ , redeploying now
a year ago
also probably a good idea to remove the EXPOSE and ENV PORT lines too
a year ago
does a standalone next build require variables to be prefixed with something? like NEXT_?
It's strange, the environment variables are being passed on buildtime but not runtime
a year ago
variables are most certainly being passed at runtime, there's just something simple we are overlooking here
but in the cases where I'm using my env vars, no, as it is code being ran on the server rather than the client
a year ago
and you're sure the environment variables that are missing aren't trying to be statically built into the code?
i mean they shouldn't be, it was working fine when i was using the default nixpacks
i just wanted to use docker so i could specify an image that isn't as RAM intensive
a year ago
nixpacks includes every single service variable you have as an ARG in its generated dockerfile, so that may explain why it works
a year ago
so try to specify all your environmental variables as ARGs in the correct layer of your dockerfile
but i'm specifying all of them in every layer, surely they should be available then?
a year ago
you only have 6 service variables?
a year ago
okay this is getting strange, have you tried a simple console log in the backend for some of those environment variables?
i know for sure that the DATABASE_URL and CLIENT_ID variables are not being passed based on my deploy logs and through my OAuth login where it's setting one of the URL params as &client_id=undefined
from that i'd say it's safe to assume that none of the environment variables are being set properly
a year ago
would you happen to be committing an .env file to GitHub?
a year ago
is your code trying to somehow load the .env.example?
i highly doubt it, that's never happened before and like i said the environment variables are working fine in build time where i know at least DATABASE_URL is being used
luckily for now i can just rollback to the nixpacks build but it would be nice to get a Dockerfile working
a year ago
definitely something simple we are overlooking here
a year ago
I'll try to reproduce
very likely, unfortunately the documentation isn't very detailed on what happens with environment variables when using Dockerfiles
a year ago
the docs do actually cover how to use environment variables during the build, and during runtime there is nothing you need to do because service variables are automatically injected into the containers environment
a year ago
ARG is short for build arguments, they shouldn't be put into the environment of the built image, but I've seen stranger things
however i could still be right, let me remove the ARGs from the base layer as well
a year ago
they should only be needed in the layer that runs commands that use the environment variables
does the ARG command actually set it as an environment variable though? or as a variable inside of the docker file, because technically the DATABASE_URL isn't being used inside of the dockerfile itself, it just needs to be set in order for prisma to generate types
removing it from the base layer and only setting it in the layers where they're needed fixed it
a year ago
ARG is short for build arguments, specifying them brings the build arguments into the build but not the runtime, railway when building from the dockerfile will specify all service variables as build arguments
I'm not quite sure here then, the environment variables are being set fine during the build because next.js is building fine, but then when it gets to runtime they no longer seem to be set
a year ago
yes I am currently stumped here too
a year ago
just for debugging purposes, can you change your dockerfile to only use a single FROM?
FROM node:18-alpine AS base
# Install dependencies only when needed
FROM base AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
# Install dependencies based on the preferred package manager
COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
COPY prisma ./prisma/
RUN \
if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
elif [ -f package-lock.json ]; then npm ci; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm i --frozen-lockfile; \
else echo "Lockfile not found." && exit 1; \
fi
# Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry during the build.
# ENV NEXT_TELEMETRY_DISABLED 1
RUN \
if [ -f yarn.lock ]; then yarn run build; \
elif [ -f package-lock.json ]; then npm run build; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm run build; \
else echo "Lockfile not found." && exit 1; \
fi
# Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app
ENV NODE_ENV production
# Uncomment the following line in case you want to disable telemetry during runtime.
# ENV NEXT_TELEMETRY_DISABLED 1
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
COPY --from=builder /app/public ./public
# Set the correct permission for prerender cache
RUN mkdir .next
RUN chown nextjs:nodejs .next
# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
USER nextjs
# server.js is created by next build from the standalone output
# https://nextjs.org/docs/pages/api-reference/next-config-js/output
CMD HOSTNAME="0.0.0.0" node server.jsshall i just remove all of the instances of FROM? apart from the first one of course
a year ago
haha no you don't
a year ago
yep that's what I meant!
a year ago
ah gotcha, I see what you meant
a year ago
so yeah try with only a single FROM and of course remove all the copy stuff
a year ago
and for extra good measure, no user / chown stuff either
a year ago
basically we want the most barebones dockerfile, then if we get that working, you can start adding the fancy stuff back for process of elimination
yeah i removed all of the copy stuff but docker has it all cached and it's a bit screwy
a year ago
set NO_CACHE to 1
a year ago
service variables
a year ago
yeah thats the joys of not using a multi stage image haha, but thats too be expected
FROM node:18-alpine
RUN apk add --no-cache libc6-compat
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
COPY prisma ./prisma/
RUN \
if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
elif [ -f package-lock.json ]; then npm ci; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm i --frozen-lockfile; \
else echo "Lockfile not found." && exit 1; \
fi
COPY . .
RUN \
if [ -f yarn.lock ]; then yarn run build; \
elif [ -f package-lock.json ]; then npm run build; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm run build; \
else echo "Lockfile not found." && exit 1; \
fi
ENV NODE_ENV production
CMD HOSTNAME="0.0.0.0" node server.jsa year ago
what package manager do you use locally?
a year ago
then youd only want to copy in package.json and pnpm-lock.yaml in the first COPY
a year ago
haha well whats the status on the missing environment variables?
so i was getting an error that /app/server.js wasn't found, which making sense because i removed the COPY commands that copied those files over
a year ago
simplify it more
FROM node:18-alpine
RUN apk add --no-cache libc6-compat
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
COPY package.json pnpm-lock.yaml* ./
COPY prisma ./prisma/
RUN corepack enable pnpm && pnpm i --frozen-lockfile
COPY . .
RUN pnpm run build
COPY . .
COPY /app/public ./public
COPY /app/.next/standalone ./
COPY /app/.next/static ./.next/static
ENV NODE_ENV production
CMD HOSTNAME="0.0.0.0" node server.jsa year ago
what makes you think its caching still?
a year ago
show me how you have set that variable please
a year ago
^
a year ago
and now it still says CACHED?
a year ago
hmmm odd
a year ago
do tell!
basically i'm a noob at docker and there's a difference between what COPY and COPY --from does
a year ago
oh haha i thought you meant you found out why the runtime envs weren't working
but i think i've got a minimal dockerfile working at this point so we can work from here
FROM node:18-alpine
RUN apk add --no-cache libc6-compat
WORKDIR /app
ARG BOT_WEBHOOK_API_KEY
ARG CLIENT_ID
ARG CLIENT_SECRET
ARG DATABASE_URL
ARG REDIRECT_URI
ARG VATSIM_API_KEY
COPY package.json pnpm-lock.yaml* ./
COPY prisma ./prisma/
RUN corepack enable pnpm && pnpm i --frozen-lockfile
COPY . .
RUN pnpm run build
RUN cp -r /app/.next/standalone/* ./
ENV NODE_ENV production
CMD HOSTNAME="0.0.0.0" node server.jsa year ago
can you show me how you are accessing these environment variables in code?
a year ago
okay ill try to reproduce
a year ago
works fine
a year ago
obviously an extremely simplified test, but this working means this isnt a platform issue
a year ago
try doing something like this
CMD echo ${variable name here}; node server.jsa year ago
do both things
a year ago
share your repo? theres a slim chance something sticks out to me
would you mind if i added you to the repo? i don't mind you having a look at the code but i can't make the repo public right now
a year ago
for sure, body192
a year ago
looking
a year ago
welp
a year ago
i have missspelt my own name
a year ago
my r key is going bad on my keyboard
a year ago
my gh username is brody192
a year ago
my bad
a year ago
got it
a year ago
feels
don't worry about having to blur anything out in screenshots btw i'm not that bothered, the repo just can't be publically accessible haha
a year ago
so just to be clear, that echo in your start command does work?
a year ago
now what would a google search for "nextjs cant read environment variables" come up with haha, since we have ruled out a platform issue
a year ago
and for fun, can you delete the .env.example file
also this is interesting: https://github.com/vercel/next.js/issues/46296
a year ago
try this?
a year ago
true, doesnt work for env()
a year ago
prisma is definitely able to read from the environment directly, something funky is going on here and I'm not sure what
a year ago
as an ENV?
a year ago
something real funky is going on with your project, try just running normal next start
a year ago
so next start works
a year ago
what a fun debug process!
a year ago
now you can go back to that mutli layer dockerfile!
yep, i might file an issue on the next.js repo for this, it's a pretty weird bug
a year ago
watch them blame the platform haha
a year ago
happy to help!
a year ago
lmao
a year ago
learn javascript they said, it will be fun they said
a year ago
haha simple fix too