4 months ago
So I know this question gets asked a lot but I cant get bullMQ queues to instantiate properly. Ive verified that I can connect to the private REDIS_URL instantiating a new Redis() object but when the queues try to connect to internal I keep getting this error in the screenshot. Ive read the docs but that only talks about instantiating the redis connection. The only way i can connect when deployed is using the public url.
For reference this is my queue class:
export class QueueService {
private verificationQueue: Queue.Queue;
private verificationService: VerificationService;
private redis: Redis;
private batchSize: number = 100;
private rateLimitDelay: number = 500;
constructor(verificationService: VerificationService) {
this.verificationService = verificationService;
// Initialize Redis connection
this.redis = new Redis(process.env.REDIS_URL! + '?family=0', {
maxRetriesPerRequest: null,
enableReadyCheck: false,
});
// Initialize Queue with Redis client
this.verificationQueue = new Queue('verification', {
redis: {
port: process.env.REDISPORT ? parseInt(process.env.REDISPORT) : 6379,
host: process.env.REDISHOST || 'localhost',
username: process.env.REDISUSER,
password: process.env.REDISPASSWORD,
lazyConnect: true,
},
defaultJobOptions: {
attempts: 3,
backoff: {
type: 'exponential',
delay: 1000,
},
removeOnComplete: true
},
});
The docs only speak to this piece: this.redis = new Redis(process.env.REDIS_URL! + '?family=0', {
maxRetriesPerRequest: null,
enableReadyCheck: false,
});
What do I need to do for bullmq to recognize this hostname?
Ive ran into this before and gave up and used the public urls which i would rather not do.
0 Replies
The error you can see is triggered from here:
private setupQueueEvents() {
this.verificationQueue
.on('completed', (job) => {
if ('type' in job.data && job.data.type === 'periodic') {
console.log('Periodic verification completed');
} else {
console.log(`Job ${job.id} completed for user ${(job.data as VerificationJob).userId}`);
}
})
.on('failed', (job, error) => {
console.error(`Job ${job?.id} failed:`, error);
})
.on('error', (error) => {
console.error('Queue error:', error);
})
.on('stalled', (job) => {
console.warn(`Job ${job?.id} stalled`);
});
}
Which means the redis connection is successful when instantiating at the top of the class
4 months ago
have you seen these docs? -
this.verificationQueue = new Queue('verification', {
redis: {
port: process.env.REDISPORT ? parseInt(process.env.REDISPORT) : 6379,
host: process.env.REDISHOST || 'localhost',
username: process.env.REDISUSER,
password: process.env.REDISPASSWORD,
lazyConnect: true,
},
These configurations do not work with the private network hostname
i already have the family=0 and its fine, but bullmq specifically cannot connect to the internal network hostname:
this.redis = new Redis(process.env.REDIS_URL! + '?family=0', {
maxRetriesPerRequest: null,
enableReadyCheck: false,
});
4 months ago
is redis in the same project?
I confirmed locally using the public url, code is working fine, internal url doesnt do anything but throw errors. Is there any other private networking i need to setup?
4 months ago
are you trying to connect during build?
Ive gone through all the issues related to this nothing seems to work. Should I wipe redis and recreate it or something?
4 months ago
nope, you need to set family to 6
4 months ago
yes it does
4 months ago
I'd recommend looking into what the family parameter represents
but im not using the connection string directly for instantiating bullmq queues, you only get to set the host, user pass and port. Ill try 6, but the docs say 0
4 months ago
6 or 0 doesn't matter when on railway
4 months ago
are you useing a very old version of bullmq?
4 months ago
thats the latest, then you arent passing in family 0 / 6 correctly
4 months ago
maybe bullmq does not respect query params
should family query param be on the host name? I mean theres literally no where in the docs I see you pass in a connection string:
const redisUrl = new URL(process.env.REDIS_URL! + '?family=6');
const redisConfig = {
port: parseInt(redisUrl.port),
host: redisUrl.hostname,
username: redisUrl.username || 'default',
password: redisUrl.password,
maxRetriesPerRequest: null,
enableReadyCheck: false,
family: 6
};
// Initialize Redis connection
this.redis = new Redis(redisConfig);
// Initialize Queue
this.verificationQueue = new Queue('verification', {
connection: redisConfig,
defaultJobOptions: {
attempts: 3,
backoff: {
type: 'exponential',
delay: 1000,
},
removeOnComplete: true
},
});
Im sure the env vars work just fine now, but this is working. Might be worth adding a section to the docs about adding the family in the config. This isnt fully typed but bullmq doesnt use conn strings so its a bit confusing
4 months ago
good point, ill add a bullmq section under the ioredis section
4 months ago
but im glad you got it solved!
4 months ago
no worries, im frustrated with ioredis too
4 months ago
can't imagine why the devs thought it was a good idea to do an IPv4 only dns lookup
with the amount of help questions asking the same thing over an over i can imagine
4 months ago
they forgor about IPv6
4 months ago
we now how a section in our docs for bullmq! -
4 months ago
!s
Status changed to Solved brody • 4 months ago