2 months ago
Hi Railway Team,
My backend is deployed on Railway.
Since July 3, Firebase Admin SDK has stopped working only on Railway.
The exact same code and the same Firebase service account work perfectly on localhost.
The failure occurs when calling:
admin.app().options.credential.getAccessToken()
Error:
GaxiosError: Invalid response body while trying to fetch https://www.googleapis.com/oauth2/v4/token: Premature close
ERR_STREAM_PREMATURE_CLOSE
Restart and redeploy did not fix the issue.
Can you please check if there is any networking or runtime issue affecting outbound HTTPS requests from my deployment to Google's OAuth endpoint?
Thank you.
Pinned Solution
2 months ago
Add a variable RAILPACK_NODE_VERSION=24.18 to update node version on railpack
7 Replies
2 months ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • about 2 months ago
2 months ago
Hi,
Thanks for the detailed report. I put together a minimal repro app and deployed it to test the same code path (raw HTTPS to googleapis.com, the OAuth token exchange via plain https, and the exact admin.app().options.credential.getAccessToken() call) — all three passed cleanly with no errors, sequentially and on a fresh deploy.
That tells us the failure isn't a blanket Railway-to-Google connectivity issue, so it's likely tied to something specific about your app or deployment conditions. Could you share a few things so we can narrow it down?
- Dependency versions — output of
npm ls firebase-admin google-auth-library gaxios(or the relevant lines from your lockfile), this tells me about the versions of libraries you are using so that I can replicate this 1-1. - Traffic pattern when it fails — does it happen under concurrent/burst load, or even on a single isolated request?
- Idle time before failure — does it tend to happen on the first request after the service has been idle for a while, or can it happen anytime?
- Frequency — does every call fail now, or is it intermittent?
- Railway service details — region, whether you're on a recent redeploy with a bumped Node/base image version, and roughly what time on July 3 it started
- Full error object if you can share it (redact any token values) — specifically whether
err.causeis populated, since that often points to the underlying socket error code - Approximate request volume your service handles — does this call happen once per request, or is it cached/reused across requests?
Here's the minimal server I am using to replicate: https://github.com/mmjsmohit/fb-railway-repro
2 months ago
Premature close is known issue on node fetch a few weeks ago and has been patched. Update your node version to 24.18.0 or newer
mmjsmohit
Hi, Thanks for the detailed report. I put together a minimal repro app and deployed it to test the same code path (raw HTTPS to googleapis.com, the OAuth token exchange via plain https, and the exact `admin.app().options.credential.getAccessToken()` call) — all three passed cleanly with no errors, sequentially and on a fresh deploy. That tells us the failure isn't a blanket Railway-to-Google connectivity issue, so it's likely tied to something specific about your app or deployment conditions. Could you share a few things so we can narrow it down? 1. **Dependency versions** — output of `npm ls firebase-admin google-auth-library gaxios` (or the relevant lines from your lockfile), this tells me about the versions of libraries you are using so that I can replicate this 1-1. 2. **Traffic pattern when it fails** — does it happen under concurrent/burst load, or even on a single isolated request? 3. **Idle time before failure** — does it tend to happen on the first request after the service has been idle for a while, or can it happen anytime? 4. **Frequency** — does every call fail now, or is it intermittent? 5. **Railway service details** — region, whether you're on a recent redeploy with a bumped Node/base image version, and roughly what time on July 3 it started 6. **Full error object** if you can share it (redact any token values) — specifically whether `err.cause` is populated, since that often points to the underlying socket error code 7. **Approximate request volume** your service handles — does this call happen once per request, or is it cached/reused across requests? Here's the minimal server I am using to replicate: https://github.com/mmjsmohit/fb-railway-repro
2 months ago
Hi,
Thanks for investigating this.
Here are the requested details:
- Dependency versions
firebase-admin@13.6.0
google-auth-library@9.15.1
gaxios@6.7.1
Full output:
backend@1.0.0
+-- firebase-admin@13.6.0
| +-- @google-cloud/firestore@7.11.6
| | `-- google-auth-library@9.15.1
| +-- @google-cloud/storage@7.18.0
| | +-- gaxios@6.7.1
| | `-- google-auth-library@9.15.1
| `-- google-auth-library@9.15.1
| +-- gaxios@6.7.1
| +-- gcp-metadata@6.1.1
| `-- gtoken@7.1.0
| `-- gaxios@6.7.1
- Traffic pattern
The issue happens even on a single isolated request. No concurrent or burst traffic is required.
- Idle time
It is not related to idle time. It happens immediately after deployment as well.
- Frequency
It is 100% reproducible now. Every call to admin.app().options.credential.getAccessToken() fails.
- Railway service details
Region: Southeast Asia (Singapore)
The issue started on July 3 around 10:00 PM IST.
I don't remember intentionally changing the Node version or deployment configuration before it started.
- Error
The failure occurs inside admin.app().options.credential.getAccessToken().
The underlying error is:
❌ Firebase access token test failed: GaxiosError: Invalid response body while trying to fetch https://www.googleapis.com/oauth2/v4/token: Premature close
at Gaxios._request (/app/node_modules/gaxios/build/src/gaxios.js:149:19)
at process.processTicksAndRejections (node:internal/process/task_queues:103:5)
at async GoogleToken._GoogleToken_requestToken (/app/node_modules/gtoken/build/src/index.js:241:19)
at async GoogleToken._GoogleToken_getTokenAsync (/app/node_modules/gtoken/build/src/index.js:160:16)
at async JWT.refreshTokenNoCache (/app/node_modules/google-auth-library/build/src/auth/jwtclient.js:173:23)
at async JWT.refreshAccessTokenAsync (/app/node_modules/google-auth-library/build/src/auth/oauth2client.js:247:19)
at async JWT.getAccessTokenAsync (/app/node_modules/google-auth-library/build/src/auth/oauth2client.js:276:23)
at async ServiceAccountCredential.getAccessToken (/app/node_modules/firebase-admin/lib/app/credential-internal.js:137:9)
at async /app/server/authMiddleWare.js:13:19 {config: {
method: 'POST',
url: 'https://www.googleapis.com/oauth2/v4/token',
data: {
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'User-Agent': 'google-api-nodejs-client/9.15.1',error: FetchError: Invalid response body while trying to fetch https://www.googleapis.com/oauth2/v4/token: Premature close
'x-goog-api-client': 'gl-node/22.23.0',
Accept: 'application/json'
},
responseType: 'json',
retryConfig: {
httpMethodsToRetry: [Array],
currentRetryAttempt: 2,
retry: 3,
noResponseRetries: 2,
retryDelayMultiplier: 2,
timeOfFirstRequest: 1783106710973,
totalTimeout: 9007199254740991,
maxRetryDelay: 9007199254740991,
statusCodesToRetry: [Array]
},
paramsSerializer: [Function: paramsSerializer],
body: '<<REDACTED> - See errorRedactor option in gaxios for configuration>.',
validateStatus: [Function: validateStatus],
errorRedactor: [Function: defaultErrorRedactor]},
response: undefined,
at Gunzip.<anonymous> (/app/node_modules/node-fetch/lib/index.js:400:12)
at Gunzip.emit (node:events:531:35)
at emitErrorNT (node:internal/streams/destroy:170:8)
at emitErrorCloseNT (node:internal/streams/destroy:129:3)
at process.processTicksAndRejections (node:internal/process/task_queues:89:21) {
type: 'system',
errno: 'ERR_STREAM_PREMATURE_CLOSE',
code: 'ERR_STREAM_PREMATURE_CLOSE'},
code: 'ERR_STREAM_PREMATURE_CLOSE',
}
- Request volume
This service is not under heavy load. Firebase Admin only requests an access token during authentication and related Firebase operations.
Additional information:
The exact same codebase, package-lock, Firebase service account JSON, and environment variables work perfectly on localhost.
On localhost, admin.app().options.credential.getAccessToken() succeeds.
On Railway, the same call consistently fails with ERR_STREAM_PREMATURE_CLOSE.
One additional observation: I found another Railway Station thread where another developer reported an almost identical error (ERR_STREAM_PREMATURE_CLOSE while requesting https://oauth2.googleapis.com/token) and resolved it by replacing the internal node-fetch token request with Node's global fetch (undici). My symptoms appear to match that report very closely.
Please let me know if you'd like me to test a specific runtime, dependency version, or any additional diagnostics.
poll20
Hi, Thanks for investigating this. Here are the requested details: 1. Dependency versions firebase-admin@13.6.0 google-auth-library@9.15.1 gaxios@6.7.1 Full output: backend@1.0.0 +-- firebase-admin@13.6.0 | +-- @google-cloud/firestore@7.11.6 | | `-- google-auth-library@9.15.1 | +-- @google-cloud/storage@7.18.0 | | +-- gaxios@6.7.1 | | `-- google-auth-library@9.15.1 | `-- google-auth-library@9.15.1 | +-- gaxios@6.7.1 | +-- gcp-metadata@6.1.1 | `-- gtoken@7.1.0 | `-- gaxios@6.7.1 2. Traffic pattern The issue happens even on a single isolated request. No concurrent or burst traffic is required. 3. Idle time It is not related to idle time. It happens immediately after deployment as well. 4. Frequency It is 100% reproducible now. Every call to admin.app().options.credential.getAccessToken() fails. 5. Railway service details Region: Southeast Asia (Singapore) The issue started on July 3 around 10:00 PM IST. I don't remember intentionally changing the Node version or deployment configuration before it started. 6. Error The failure occurs inside admin.app().options.credential.getAccessToken(). The underlying error is: ❌ Firebase access token test failed: GaxiosError: Invalid response body while trying to fetch https://www.googleapis.com/oauth2/v4/token: Premature close at Gaxios._request (/app/node_modules/gaxios/build/src/gaxios.js:149:19) at process.processTicksAndRejections (node:internal/process/task_queues:103:5) at async GoogleToken._GoogleToken_requestToken (/app/node_modules/gtoken/build/src/index.js:241:19) at async GoogleToken._GoogleToken_getTokenAsync (/app/node_modules/gtoken/build/src/index.js:160:16) at async JWT.refreshTokenNoCache (/app/node_modules/google-auth-library/build/src/auth/jwtclient.js:173:23) at async JWT.refreshAccessTokenAsync (/app/node_modules/google-auth-library/build/src/auth/oauth2client.js:247:19) at async JWT.getAccessTokenAsync (/app/node_modules/google-auth-library/build/src/auth/oauth2client.js:276:23) at async ServiceAccountCredential.getAccessToken (/app/node_modules/firebase-admin/lib/app/credential-internal.js:137:9) at async /app/server/authMiddleWare.js:13:19 { config: { method: 'POST', url: 'https://www.googleapis.com/oauth2/v4/token', data: { headers: { 'Content-Type': 'application/x-www-form-urlencoded', 'User-Agent': 'google-api-nodejs-client/9.15.1', error: FetchError: Invalid response body while trying to fetch https://www.googleapis.com/oauth2/v4/token: Premature close 'x-goog-api-client': 'gl-node/22.23.0', Accept: 'application/json' }, responseType: 'json', retryConfig: { httpMethodsToRetry: [Array], currentRetryAttempt: 2, retry: 3, noResponseRetries: 2, retryDelayMultiplier: 2, timeOfFirstRequest: 1783106710973, totalTimeout: 9007199254740991, maxRetryDelay: 9007199254740991, statusCodesToRetry: [Array] }, paramsSerializer: [Function: paramsSerializer], body: '<<REDACTED> - See errorRedactor option in gaxios for configuration>.', validateStatus: [Function: validateStatus], errorRedactor: [Function: defaultErrorRedactor] }, response: undefined, at Gunzip.<anonymous> (/app/node_modules/node-fetch/lib/index.js:400:12) at Gunzip.emit (node:events:531:35) at emitErrorNT (node:internal/streams/destroy:170:8) at emitErrorCloseNT (node:internal/streams/destroy:129:3) at process.processTicksAndRejections (node:internal/process/task_queues:89:21) { type: 'system', errno: 'ERR_STREAM_PREMATURE_CLOSE', code: 'ERR_STREAM_PREMATURE_CLOSE' }, code: 'ERR_STREAM_PREMATURE_CLOSE', [Symbol(gaxios-gaxios-error)]: '6.7.1' } 7. Request volume This service is not under heavy load. Firebase Admin only requests an access token during authentication and related Firebase operations. Additional information: The exact same codebase, package-lock, Firebase service account JSON, and environment variables work perfectly on localhost. On localhost, admin.app().options.credential.getAccessToken() succeeds. On Railway, the same call consistently fails with ERR_STREAM_PREMATURE_CLOSE. One additional observation: I found another Railway Station thread where another developer reported an almost identical error (ERR_STREAM_PREMATURE_CLOSE while requesting https://oauth2.googleapis.com/token) and resolved it by replacing the internal node-fetch token request with Node's global fetch (undici). My symptoms appear to match that report very closely. Please let me know if you'd like me to test a specific runtime, dependency version, or any additional diagnostics.
2 months ago
Update node version to 24.18 or newer
mayori
Update node version to `24.18` or newer
2 months ago
Hi, thanks for the suggestion. I tested it.
Initially my Railway service was running on Node 22.23.0. Following the recommendation, I updated the project to require Node >=24.18.1. After redeploying, Railway upgraded the runtime to Node 24.17.0 (from 22.23.0), but it doesn't seem to provide Node 24.18.0 or newer even though my package.json specifies >=24.18.1.
I also confirmed the requests now use gl-node/24.17.0, so the runtime is definitely no longer Node 22. Unfortunately, the issue is still 100% reproducible with the exact same ERR_STREAM_PREMATURE_CLOSE error while fetching the Google OAuth token.
Is there any way to use Node 24.18.0+ on Railpack, or do you have any other suggestions to help narrow this down?
poll20
Hi, thanks for the suggestion. I tested it. Initially my Railway service was running on Node 22.23.0. Following the recommendation, I updated the project to require Node >=24.18.1. After redeploying, Railway upgraded the runtime to Node 24.17.0 (from 22.23.0), but it doesn't seem to provide Node 24.18.0 or newer even though my package.json specifies >=24.18.1. I also confirmed the requests now use gl-node/24.17.0, so the runtime is definitely no longer Node 22. Unfortunately, the issue is still 100% reproducible with the exact same ERR_STREAM_PREMATURE_CLOSE error while fetching the Google OAuth token. Is there any way to use Node 24.18.0+ on Railpack, or do you have any other suggestions to help narrow this down?
2 months ago
Add a variable RAILPACK_NODE_VERSION=24.18 to update node version on railpack
mayori
Add a variable `RAILPACK_NODE_VERSION=24.18` to update node version on railpack
2 months ago
Thankss! Setting RAILPACK_NODE_VERSION=24.18 fixed it. The runtime upgraded to Node 24.18 and the ERR_STREAM_PREMATURE_CLOSE error is completely gone. Really appreciate the help!
Status changed to Solved medim • about 1 month ago
