Railway Redis ioredis BullMQ Not connecting using private network
thedonmon
HOBBYOP

a year ago

So I know this question gets asked a lot but I cant get bullMQ queues to instantiate properly. Ive verified that I can connect to the private REDIS_URL instantiating a new Redis() object but when the queues try to connect to internal I keep getting this error in the screenshot. Ive read the docs but that only talks about instantiating the redis connection. The only way i can connect when deployed is using the public url.

For reference this is my queue class:

export class QueueService {
  private verificationQueue: Queue.Queue;
  private verificationService: VerificationService;
  private redis: Redis;
  private batchSize: number = 100;
  private rateLimitDelay: number = 500;

  constructor(verificationService: VerificationService) {
    this.verificationService = verificationService;

    // Initialize Redis connection
    this.redis = new Redis(process.env.REDIS_URL! + '?family=0', {
      maxRetriesPerRequest: null,
      enableReadyCheck: false,
    });

    // Initialize Queue with Redis client
    this.verificationQueue = new Queue('verification', {
      redis: {
        port: process.env.REDISPORT ? parseInt(process.env.REDISPORT) : 6379,
        host: process.env.REDISHOST || 'localhost',
        username: process.env.REDISUSER,
        password: process.env.REDISPASSWORD,
        lazyConnect: true,
      },
      defaultJobOptions: {
        attempts: 3,
        backoff: {
          type: 'exponential',
          delay: 1000,
        },
        removeOnComplete: true
      },
    });

The docs only speak to this piece: this.redis = new Redis(process.env.REDIS_URL! + '?family=0', {
maxRetriesPerRequest: null,
enableReadyCheck: false,
});

What do I need to do for bullmq to recognize this hostname?

Ive ran into this before and gave up and used the public urls which i would rather not do.

Solved

0 Replies

thedonmon
HOBBYOP

a year ago

cbaa8f69-2dcf-43d0-88bd-bc8dc65eb24f


thedonmon
HOBBYOP

a year ago

The error you can see is triggered from here:

private setupQueueEvents() {
    this.verificationQueue
      .on('completed', (job) => {
        if ('type' in job.data && job.data.type === 'periodic') {
          console.log('Periodic verification completed');
        } else {
          console.log(`Job ${job.id} completed for user ${(job.data as VerificationJob).userId}`);
        }
      })
      .on('failed', (job, error) => {
        console.error(`Job ${job?.id} failed:`, error);
      })
      .on('error', (error) => {
        console.error('Queue error:', error);
      })
      .on('stalled', (job) => {
        console.warn(`Job ${job?.id} stalled`);
      });
  }

Which means the redis connection is successful when instantiating at the top of the class


brody
EMPLOYEE

a year ago

have you seen these docs? -


thedonmon
HOBBYOP

a year ago

Yes those docs do not help at all


thedonmon
HOBBYOP

a year ago

to be honest


thedonmon
HOBBYOP

a year ago

My issue is not the Redis connection URL


thedonmon
HOBBYOP

a year ago

this.verificationQueue = new Queue('verification', {
redis: {
port: process.env.REDISPORT ? parseInt(process.env.REDISPORT) : 6379,
host: process.env.REDISHOST || 'localhost',
username: process.env.REDISUSER,
password: process.env.REDISPASSWORD,
lazyConnect: true,
},

These configurations do not work with the private network hostname


thedonmon
HOBBYOP

a year ago

i already have the family=0 and its fine, but bullmq specifically cannot connect to the internal network hostname:

this.redis = new Redis(process.env.REDIS_URL! + '?family=0', {
maxRetriesPerRequest: null,
enableReadyCheck: false,
});


brody
EMPLOYEE

a year ago

is redis in the same project?


thedonmon
HOBBYOP

a year ago

Yea

1318709522314821600


thedonmon
HOBBYOP

a year ago

I confirmed locally using the public url, code is working fine, internal url doesnt do anything but throw errors. Is there any other private networking i need to setup?


brody
EMPLOYEE

a year ago

are you trying to connect during build?


thedonmon
HOBBYOP

a year ago

Nope, just on startup


thedonmon
HOBBYOP

a year ago

i added the sleep 3 also


thedonmon
HOBBYOP

a year ago

Ive gone through all the issues related to this nothing seems to work. Should I wipe redis and recreate it or something?


brody
EMPLOYEE

a year ago

nope, you need to set family to 6


thedonmon
HOBBYOP

a year ago

i can try that


thedonmon
HOBBYOP

a year ago

still doesnt really affect how the hostname is used tho


brody
EMPLOYEE

a year ago

yes it does


brody
EMPLOYEE

a year ago

I'd recommend looking into what the family parameter represents


thedonmon
HOBBYOP

a year ago

but im not using the connection string directly for instantiating bullmq queues, you only get to set the host, user pass and port. Ill try 6, but the docs say 0


brody
EMPLOYEE

a year ago

6 or 0 doesn't matter when on railway


thedonmon
HOBBYOP

a year ago

1318714101706391800


thedonmon
HOBBYOP

a year ago

still doesnt change anything


brody
EMPLOYEE

a year ago

are you useing a very old version of bullmq?


thedonmon
HOBBYOP

a year ago

5.34.2


brody
EMPLOYEE

a year ago

thats the latest, then you arent passing in family 0 / 6 correctly


thedonmon
HOBBYOP

a year ago

redis://default:password@redis.railway.internal:6379?family=6


thedonmon
HOBBYOP

a year ago

is not right?


brody
EMPLOYEE

a year ago

maybe bullmq does not respect query params


thedonmon
HOBBYOP

a year ago

should family query param be on the host name? I mean theres literally no where in the docs I see you pass in a connection string:

https://docs.bullmq.io/guide/connections


thedonmon
HOBBYOP

a year ago

might be a family param we can pass in, looking


thedonmon
HOBBYOP

a year ago

const redisUrl = new URL(process.env.REDIS_URL! + '?family=6');

    const redisConfig = {
      port: parseInt(redisUrl.port),
      host: redisUrl.hostname,
      username: redisUrl.username || 'default',
      password: redisUrl.password,
      maxRetriesPerRequest: null,
      enableReadyCheck: false,
      family: 6
    };

    // Initialize Redis connection
    this.redis = new Redis(redisConfig);

    // Initialize Queue
    this.verificationQueue = new Queue('verification', {
      connection: redisConfig,
      defaultJobOptions: {
        attempts: 3,
        backoff: {
          type: 'exponential',
          delay: 1000,
        },
        removeOnComplete: true
      },
    });

Im sure the env vars work just fine now, but this is working. Might be worth adding a section to the docs about adding the family in the config. This isnt fully typed but bullmq doesnt use conn strings so its a bit confusing


brody
EMPLOYEE

a year ago

good point, ill add a bullmq section under the ioredis section


brody
EMPLOYEE

a year ago

but im glad you got it solved!


thedonmon
HOBBYOP

a year ago

please excuse my frustration lol


thedonmon
HOBBYOP

a year ago

thanks for the quick reply


brody
EMPLOYEE

a year ago

no worries, im frustrated with ioredis too


thedonmon
HOBBYOP

a year ago

😭


brody
EMPLOYEE

a year ago

can't imagine why the devs thought it was a good idea to do an IPv4 only dns lookup


thedonmon
HOBBYOP

a year ago

with the amount of help questions asking the same thing over an over i can imagine


thedonmon
HOBBYOP

a year ago

Yea must be outdated devs


brody
EMPLOYEE

a year ago

they forgor about IPv6


brody
EMPLOYEE

a year ago

we now how a section in our docs for bullmq! -


brody
EMPLOYEE

a year ago

!s


Status changed to Solved brody 12 months ago


Loading...