Experiencing high latency to RabbitMQ

2 months ago

I might sound crazy here, but we have been getting a lot of latency to our RabbitMQ service as shown in the picture since March 17, simply for publishing messages which shouldn't take any latency at all.

The thing is, I do not see anything on Network Flow Logs that might explain this 😭

We are using private networking and not much was changed on that date, our service isn't suffering from back-pressuring or anything like that. Neither logs that might explain this.

RabbitMQ service:

Service talking to it:

Just wanted to confirm that Railway isn't at fault here (let's say we got paranoid after an incident 💀).

image.png

Attachments

Solved

6 Replies

2 months ago

Will add some tracings related to TCP and DNS to get more info


edgnkv
EMPLOYEE

2 months ago

Hey,

Can confirm the issue is IO related. Looking into this now


edgnkv
EMPLOYEE

2 months ago

Can you check now?


2 months ago

Seems like it has stabilized, will keep monitoring, can I know what happened? 👀


edgnkv
EMPLOYEE

2 months ago

the underlaying volume had high FS fragmentation


2 months ago

Big thanks for that! Will open another help thread if it comes back.


Status changed to Solved passos • 2 months ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...