Railway Redis ioredis BullMQ Not connecting using private network

thedonmonHOBBY

4 months ago

So I know this question gets asked a lot but I cant get bullMQ queues to instantiate properly. Ive verified that I can connect to the private REDIS_URL instantiating a new Redis() object but when the queues try to connect to internal I keep getting this error in the screenshot. Ive read the docs but that only talks about instantiating the redis connection. The only way i can connect when deployed is using the public url.

For reference this is my queue class:

export class QueueService {
  private verificationQueue: Queue.Queue;
  private verificationService: VerificationService;
  private redis: Redis;
  private batchSize: number = 100;
  private rateLimitDelay: number = 500;

  constructor(verificationService: VerificationService) {
    this.verificationService = verificationService;

    // Initialize Redis connection
    this.redis = new Redis(process.env.REDIS_URL! + '?family=0', {
      maxRetriesPerRequest: null,
      enableReadyCheck: false,
    });

    // Initialize Queue with Redis client
    this.verificationQueue = new Queue('verification', {
      redis: {
        port: process.env.REDISPORT ? parseInt(process.env.REDISPORT) : 6379,
        host: process.env.REDISHOST || 'localhost',
        username: process.env.REDISUSER,
        password: process.env.REDISPASSWORD,
        lazyConnect: true,
      },
      defaultJobOptions: {
        attempts: 3,
        backoff: {
          type: 'exponential',
          delay: 1000,
        },
        removeOnComplete: true
      },
    });

The docs only speak to this piece: this.redis = new Redis(process.env.REDIS_URL! + '?family=0', {
maxRetriesPerRequest: null,
enableReadyCheck: false,
});

What do I need to do for bullmq to recognize this hostname?

Ive ran into this before and gave up and used the public urls which i would rather not do.

Solved

0 Replies

thedonmonHOBBY

4 months ago

cbaa8f69-2dcf-43d0-88bd-bc8dc65eb24f


thedonmonHOBBY

4 months ago

The error you can see is triggered from here:

private setupQueueEvents() {
    this.verificationQueue
      .on('completed', (job) => {
        if ('type' in job.data && job.data.type === 'periodic') {
          console.log('Periodic verification completed');
        } else {
          console.log(`Job ${job.id} completed for user ${(job.data as VerificationJob).userId}`);
        }
      })
      .on('failed', (job, error) => {
        console.error(`Job ${job?.id} failed:`, error);
      })
      .on('error', (error) => {
        console.error('Queue error:', error);
      })
      .on('stalled', (job) => {
        console.warn(`Job ${job?.id} stalled`);
      });
  }

Which means the redis connection is successful when instantiating at the top of the class


4 months ago

have you seen these docs? -


thedonmonHOBBY

4 months ago

Yes those docs do not help at all


thedonmonHOBBY

4 months ago

to be honest


thedonmonHOBBY

4 months ago

My issue is not the Redis connection URL


thedonmonHOBBY

4 months ago

this.verificationQueue = new Queue('verification', {
redis: {
port: process.env.REDISPORT ? parseInt(process.env.REDISPORT) : 6379,
host: process.env.REDISHOST || 'localhost',
username: process.env.REDISUSER,
password: process.env.REDISPASSWORD,
lazyConnect: true,
},

These configurations do not work with the private network hostname


thedonmonHOBBY

4 months ago

i already have the family=0 and its fine, but bullmq specifically cannot connect to the internal network hostname:

this.redis = new Redis(process.env.REDIS_URL! + '?family=0', {
maxRetriesPerRequest: null,
enableReadyCheck: false,
});


4 months ago

is redis in the same project?


thedonmonHOBBY

4 months ago

Yea

1318709522314821600


thedonmonHOBBY

4 months ago

I confirmed locally using the public url, code is working fine, internal url doesnt do anything but throw errors. Is there any other private networking i need to setup?


4 months ago

are you trying to connect during build?


thedonmonHOBBY

4 months ago

Nope, just on startup


thedonmonHOBBY

4 months ago

i added the sleep 3 also


thedonmonHOBBY

4 months ago

Ive gone through all the issues related to this nothing seems to work. Should I wipe redis and recreate it or something?


4 months ago

nope, you need to set family to 6


thedonmonHOBBY

4 months ago

i can try that


thedonmonHOBBY

4 months ago

still doesnt really affect how the hostname is used tho


4 months ago

yes it does


4 months ago

I'd recommend looking into what the family parameter represents


thedonmonHOBBY

4 months ago

but im not using the connection string directly for instantiating bullmq queues, you only get to set the host, user pass and port. Ill try 6, but the docs say 0


4 months ago

6 or 0 doesn't matter when on railway


thedonmonHOBBY

4 months ago

1318714101706391800


thedonmonHOBBY

4 months ago

still doesnt change anything


4 months ago

are you useing a very old version of bullmq?


thedonmonHOBBY

4 months ago

5.34.2


4 months ago

thats the latest, then you arent passing in family 0 / 6 correctly


thedonmonHOBBY

4 months ago

redis://default:password@redis.railway.internal:6379?family=6


thedonmonHOBBY

4 months ago

is not right?


4 months ago

maybe bullmq does not respect query params


thedonmonHOBBY

4 months ago

should family query param be on the host name? I mean theres literally no where in the docs I see you pass in a connection string:

https://docs.bullmq.io/guide/connections


thedonmonHOBBY

4 months ago

might be a family param we can pass in, looking


thedonmonHOBBY

4 months ago

const redisUrl = new URL(process.env.REDIS_URL! + '?family=6');

    const redisConfig = {
      port: parseInt(redisUrl.port),
      host: redisUrl.hostname,
      username: redisUrl.username || 'default',
      password: redisUrl.password,
      maxRetriesPerRequest: null,
      enableReadyCheck: false,
      family: 6
    };

    // Initialize Redis connection
    this.redis = new Redis(redisConfig);

    // Initialize Queue
    this.verificationQueue = new Queue('verification', {
      connection: redisConfig,
      defaultJobOptions: {
        attempts: 3,
        backoff: {
          type: 'exponential',
          delay: 1000,
        },
        removeOnComplete: true
      },
    });

Im sure the env vars work just fine now, but this is working. Might be worth adding a section to the docs about adding the family in the config. This isnt fully typed but bullmq doesnt use conn strings so its a bit confusing


4 months ago

good point, ill add a bullmq section under the ioredis section


4 months ago

but im glad you got it solved!


thedonmonHOBBY

4 months ago

please excuse my frustration lol


thedonmonHOBBY

4 months ago

thanks for the quick reply


4 months ago

no worries, im frustrated with ioredis too


thedonmonHOBBY

4 months ago

😭


4 months ago

can't imagine why the devs thought it was a good idea to do an IPv4 only dns lookup


thedonmonHOBBY

4 months ago

with the amount of help questions asking the same thing over an over i can imagine


thedonmonHOBBY

4 months ago

Yea must be outdated devs


4 months ago

they forgor about IPv6


4 months ago

we now how a section in our docs for bullmq! -


4 months ago

!s


Status changed to Solved brody 4 months ago


Railway Redis ioredis BullMQ Not connecting using private network - Railway Help Station