Web Push to FCM failing with ECONNREFUSED ::1:443 from Node backend
Anonymous
HOBBYOP

2 months ago

Hi everyone,

I have a Node.js backend running on Railway using the web-push library to send browser push notifications (Web Push) via FCM https://fcm.googleapis.com/fcm/send).

My production service is:

- Project / Service: enthusiastic-analysis / backend_shoglak / production

- Domains:

- api.shoglak.com

- backendshoglak-production.up.railway.app

Web Push is configured with VAPID keys via env:

- WEB_PUSH_PUBLIC_KEY

- WEB_PUSH_PRIVATE_KEY

Subscriptions are saved correctly and I see this in the logs:

```text

WebPush: saving subscription for user 81 {

endpoint: 'https://fcm.googleapis.com/fcm/send/'

}

Sending WebPush notification {

userId: 81,

subscriptionCount: 1,

type: 'comment',

url: '/social?postId=24'

}

WebPush: sending to endpoint {

userId: 81,

endpoint: 'https://fcm.googleapis.com/fcm/send/'

}

Error sending WebPush notification {

userId: 81,

endpoint: 'https://fcm.googleapis.com/fcm/send/',

statusCode: undefined,

message: 'connect ECONNREFUSED ::1:443'

}

$10 Bounty

12 Replies

ilyassbreth
FREE

2 months ago

add these environment variables to your Railway service to bypass the proxy for external HTTPS requests:

NO_PROXY=*
no_proxy=*

or more specifically:

NO_PROXY=fcm.googleapis.com,*.googleapis.com

ilyassbreth
FREE

2 months ago

then redeploy your service


ilyassbreth
FREE

2 months ago

i think this should resolve the ECONNREFUSED error and allow web-push to connect directly to FCM

i hope this help you :)


ilyassbreth

add these environment variables to your Railway service to bypass the proxy for external HTTPS requests:NO_PROXY=* no_proxy=*or more specifically:NO_PROXY=fcm.googleapis.com,*.googleapis.com

Anonymous
HOBBYOP

2 months ago

Hey, thanks again for your suggestion.

I tried both variants of NO_PROXY on my Railway Node.js service:

1) Specific domains:

NO_PROXY=fcm.googleapis.com,*.googleapis.com

no_proxy=fcm.googleapis.com,*.googleapis.com

2) Then I also tried the global version:

NO_PROXY=*

no_proxy=*

After that I redeployed the backend and added some debug logs right before calling webPush.sendNotification. Inside the Node process I now see:

WebPush: environment proxy config {

NO_PROXY: '*',

no_proxy: '*',

HTTP_PROXY: undefined,

HTTPS_PROXY: undefined

}

So the env vars are definitely being read correctly inside the container.

However, the error is still the same:

Sending WebPush notification {

userId: 72,

subscriptionCount: 1,

type: 'like',

url: '/social?postId=19'

}

WebPush: sending to endpoint {

userId: 72,

endpoint: 'https://fcm.googleapis.com/fcm/send/...'

}

Error sending WebPush notification {

userId: 72,

endpoint: 'https://fcm.googleapis.com/fcm/send/...',

statusCode: undefined,

message: 'connect ECONNREFUSED ::1:443'

}

It looks like the HTTPS request to FCM is still being redirected to ::1:443 somewhere below Node (network layer), even though NO_PROXY/* is set and HTTP_PROXY/HTTPS_PROXY are undefined.

Is there anything specific on Railway’s side that could cause outbound HTTPS traffic (port 443) to be redirected to ::1 like this?

Do I need any additional project/service configuration to allow direct connections to https://fcm.googleapis.com from a Node.js service?

Also, in the logs the endpoint gets HTML-escaped as https://fcm.googleapis.com/fcm/send&#x2F, but in code it’s the normal URL https://fcm.googleapis.com/fcm/send/, so I don’t think that’s related.

Any guidance on what to check next on Railway’s side would be really appreciated.


Hey, thanks again for your suggestion.I tried both variants of NO_PROXY on my Railway Node.js service:1) Specific domains:NO_PROXY=fcm.googleapis.com,*.googleapis.comno_proxy=fcm.googleapis.com,*.googleapis.com2) Then I also tried the global version:NO_PROXY=*no_proxy=*After that I redeployed the backend and added some debug logs right before calling webPush.sendNotification. Inside the Node process I now see:WebPush: environment proxy config {NO_PROXY: '*',no_proxy: '*',HTTP_PROXY: undefined,HTTPS_PROXY: undefined}So the env vars are definitely being read correctly inside the container.However, the error is still the same:Sending WebPush notification {userId: 72,subscriptionCount: 1,type: 'like',url: '/social?postId=19'}WebPush: sending to endpoint {userId: 72,endpoint: 'https://fcm.googleapis.com/fcm/send/...'}Error sending WebPush notification {userId: 72,endpoint: 'https://fcm.googleapis.com/fcm/send/...',statusCode: undefined,message: 'connect ECONNREFUSED ::1:443'}It looks like the HTTPS request to FCM is still being redirected to ::1:443 somewhere below Node (network layer), even though NO_PROXY/* is set and HTTP_PROXY/HTTPS_PROXY are undefined.Is there anything specific on Railway’s side that could cause outbound HTTPS traffic (port 443) to be redirected to ::1 like this?Do I need any additional project/service configuration to allow direct connections to https://fcm.googleapis.com from a Node.js service?Also, in the logs the endpoint gets HTML-escaped as https://fcm.googleapis.com/fcm/send&#x2F, but in code it’s the normal URL https://fcm.googleapis.com/fcm/send/, so I don’t think that’s related.Any guidance on what to check next on Railway’s side would be really appreciated.

2 months ago

Is there anything specific on Railway’s side that could cause outbound HTTPS traffic (port 443) to be redirected to ::1 like this?

No, absolutely not, this is solely a configuration on your end that is causing this.


Hey, thanks again for your suggestion.I tried both variants of NO_PROXY on my Railway Node.js service:1) Specific domains:NO_PROXY=fcm.googleapis.com,*.googleapis.comno_proxy=fcm.googleapis.com,*.googleapis.com2) Then I also tried the global version:NO_PROXY=*no_proxy=*After that I redeployed the backend and added some debug logs right before calling webPush.sendNotification. Inside the Node process I now see:WebPush: environment proxy config {NO_PROXY: '*',no_proxy: '*',HTTP_PROXY: undefined,HTTPS_PROXY: undefined}So the env vars are definitely being read correctly inside the container.However, the error is still the same:Sending WebPush notification {userId: 72,subscriptionCount: 1,type: 'like',url: '/social?postId=19'}WebPush: sending to endpoint {userId: 72,endpoint: 'https://fcm.googleapis.com/fcm/send/...'}Error sending WebPush notification {userId: 72,endpoint: 'https://fcm.googleapis.com/fcm/send/...',statusCode: undefined,message: 'connect ECONNREFUSED ::1:443'}It looks like the HTTPS request to FCM is still being redirected to ::1:443 somewhere below Node (network layer), even though NO_PROXY/* is set and HTTP_PROXY/HTTPS_PROXY are undefined.Is there anything specific on Railway’s side that could cause outbound HTTPS traffic (port 443) to be redirected to ::1 like this?Do I need any additional project/service configuration to allow direct connections to https://fcm.googleapis.com from a Node.js service?Also, in the logs the endpoint gets HTML-escaped as https://fcm.googleapis.com/fcm/send&#x2F, but in code it’s the normal URL https://fcm.googleapis.com/fcm/send/, so I don’t think that’s related.Any guidance on what to check next on Railway’s side would be really appreciated.

ilyassbreth
FREE

2 months ago

check if you have it installed (even if you're not using it directly):

npm list https-proxy-agent agent-base

if you see old versions, upgrade them:

npm install https-proxy-agent@latest agent-base@latest

then delete node_modules and reinstall:

rm -rf node_modules package-lock.json
npm install

old versions of https-proxy-agent monkey-patch node's https module and break all external requests by redirecting them to localhost (::1:443). it's a known issue even when you don't explicitly use the library ,just having it installed causes the problem

redeploy after this and let me know if it fixes it!


ilyassbreth

check if you have it installed (even if you're not using it directly):npm list https-proxy-agent agent-baseif you see old versions, upgrade them:npm install https-proxy-agent@latest agent-base@latestthen delete node_modules and reinstall:rm -rf node_modules package-lock.json npm installold versions of https-proxy-agent monkey-patch node's https module and break all external requests by redirecting them to localhost (::1:443). it's a known issue even when you don't explicitly use the library ,just having it installed causes the problemredeploy after this and let me know if it fixes it!

Anonymous
HOBBYOP

2 months ago

I ran npm ls https-proxy-agent agent-base inside my backend service and got:

web-push@3.6.7

https-proxy-agent@7.0.6

agent-base@7.1.4

So it looks like I'm already on the latest versions of https-proxy-agent and agent-base, not on an old buggy version.

Even with these versions and NO_PROXY set (or NO_PROXY=*), I'm still seeing:

message: 'connect ECONNREFUSED ::1:443'

Any idea what else in my setup or dependencies could cause all HTTPS requests to FCM to be redirected to ::1:443?


ilyassbreth
FREE

2 months ago

so check if you're accidentally passing proxy or agent options to webpush.sendNotification(). look for this in your code:

javascript

webpush.sendNotification(subscription, payload, {
  proxy: '...',  //  check if you have this
  agent: '...',  //  or this
  // other options...
})

also search your codebase for:

bash

grep -r "proxy.*localhost" .
grep -r "agent.*localhost" .
grep -r "https.Agent" .
grep -r "HttpsProxyAgent" .

if you're creating a custom https.Agent anywhere or if some dependency is passing proxy/agent config that points to localhost, that would cause fcm requests to redirect to ::1:443.

check your env vars too , maybe you have something like HTTPS_PROXY=http://localhost:443 set locally that's being deployed?

can you share the code where you call webpush.sendNotification()? the problem has to be in how you're configuring web-push, not in railway itself


ilyassbreth

so check if you're accidentally passing proxy or agent options to webpush.sendNotification(). look for this in your code:javascriptwebpush.sendNotification(subscription, payload, { proxy: '...', // check if you have this agent: '...', // or this // other options... })also search your codebase for:bashgrep -r "proxy.*localhost" . grep -r "agent.*localhost" . grep -r "https.Agent" . grep -r "HttpsProxyAgent" .if you're creating a custom https.Agent anywhere or if some dependency is passing proxy/agent config that points to localhost, that would cause fcm requests to redirect to ::1:443.check your env vars too , maybe you have something like HTTPS_PROXY=http://localhost:443 set locally that's being deployed?can you share the code where you call webpush.sendNotification()? the problem has to be in how you're configuring web-push, not in railway itself

Anonymous
HOBBYOP

2 months ago

Here is the code where I call webPush.sendNotification:

```js

const sendWebPushNotification = async (userId, payload) => {

if (!isWebPushConfigured) {

console.warn('WebPush not configured; skipping push notification for user', userId);

return;

}

console.log('WebPush: environment proxy config', {

NO_PROXY: process.env.NO_PROXY,

no_proxy: process.env.no_proxy,

HTTP_PROXY: process.env.HTTP_PROXY,

HTTPS_PROXY: process.env.HTTPS_PROXY

});

const subscriptions = await getUserPushSubscriptions(userId);

if (!subscriptions.length) {

console.log('No WebPush subscriptions found for user', userId);

return;

}

const data = JSON.stringify(payload);

console.log('Sending WebPush notification', {

userId,

subscriptionCount: subscriptions.length,

type: payload && payload.type,

url: payload && payload.url

});

await Promise.all(

subscriptions.map(async (sub) => {

try {

const subscription = {

endpoint: sub.endpoint,

keys: sub.keys

};

console.log('WebPush: sending to endpoint', {

userId,

endpoint: String(sub.endpoint).slice(0, 60)

});

await webPush.sendNotification(subscription, data); // no proxy/agent options here

} catch (error) {

const statusCode = error && (error.statusCode || error.status);

console.error('Error sending WebPush notification', {

userId,

endpoint: String(sub.endpoint).slice(0, 60),

statusCode,

message: error && error.message

});

}

})

);

};

```

And here is the log I added before sending:

```text

WebPush: environment proxy config {

NO_PROXY: '*',

no_proxy: '*',

HTTP_PROXY: undefined,

HTTPS_PROXY: undefined

}

```

So:

- I'm not passing any proxy or agent options to webPush.sendNotification.

- HTTP_PROXY and HTTPS_PROXY are undefined inside the Node process.

- NO_PROXY / no_proxy are set and read correctly.

Despite that, the error is still:

```text

Error sending WebPush notification {

endpoint: 'https://fcm.googleapis.com/fcm/send/...',

statusCode: undefined,

message: 'connect ECONNREFUSED ::1:443'

}

```

I also searched my codebase and I’m not using HttpsProxyAgent or a custom https.Agent anywhere.

Given this, do you have any idea what else could cause Node’s HTTPS requests to FCM to end up at ::1:443?


ilyassbreth
FREE

2 months ago

this looks like a node.js 17+ dns issue. try adding this env var to your railway service:

NODE_OPTIONS=--dns-result-order=ipv4first

node 17+ now prioritizes ipv6 by default which is making fcm.googleapis.com resolve to ::1 (ipv6 localhost) instead of the actual fcm server. this flag forces ipv4 first.

add it in your railway variables tab then redeploy. should fix the ::1:443 error


ilyassbreth

this looks like a node.js 17+ dns issue. try adding this env var to your railway service:NODE_OPTIONS=--dns-result-order=ipv4firstnode 17+ now prioritizes ipv6 by default which is making fcm.googleapis.com resolve to ::1 (ipv6 localhost) instead of the actual fcm server. this flag forces ipv4 first.add it in your railway variables tab then redeploy. should fix the ::1:443 error

Anonymous
HOBBYOP

2 months ago


brody

Is there anything specific on Railway’s side that could cause outbound HTTPS traffic (port 443) to be redirected to ::1 like this?No, absolutely not, this is solely a configuration on your end that is causing this.

Anonymous
HOBBYOP

2 months ago


Loading...