Redis AOF fsync error
nevo-david
PROOP

6 months ago

For the brilliant minds here, I am getting this from Redis (on railway),

Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down Redis.

We have a lot of writing to do in Redis.
From my understanding of ChatGPT, it writes to disk, but since it takes too much time, it doesn't monitor it.

Happy know how I can solve it šŸ™‚

17 Replies

nevo-david
PROOP

6 months ago

355a123a-8f0d-473a-aa69-318226cfbe58


nevo-david
PROOP

6 months ago

I think this is the service id: 5dfaf4e9-54e6-4d5e-8125-c233a22d22f3


adam
MODERATOR

6 months ago

are you using a redis template? or attempting to roll your own


nevo-david
PROOP

6 months ago

Just the built in service

1401927119042379800


nevo-david
PROOP

6 months ago

Hi @Adam sorry to ping you šŸ™šŸ»
But I think it causes some redis data loss


nevo-david
PROOP

6 months ago

Small bump 🄲


adam
MODERATOR

6 months ago

according to this thread this is an issue that’s been around for a while with no fixes. I’ll raise it with the team


nevo-david
PROOP

6 months ago

Is this a railway problem or a redis problem?


brody
EMPLOYEE

6 months ago

A config problem more than anything, you can disable AOF by setting these variables -

REDIS_AOF_ENABLED=no
REDIS_RDB_POLICY=3600#1 300#100 60#10000

nevo-david
PROOP

6 months ago

@Brody if I deploy a new version, will it erase all my keys if I redeploy it?


brody
EMPLOYEE

6 months ago

No it will not erase all your keys


nevo-david
PROOP

6 months ago

Well, it did, but I have a backup


brody
EMPLOYEE

6 months ago

Something else would have had to wipe the keys.


nevo-david
PROOP

6 months ago

I guess there were not saved to the disk


nevo-david
PROOP

6 months ago

and my suspection is the error from before


nevo-david
PROOP

6 months ago

I am getting this message from redis, is it normal?

WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommitmemory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommitmemory=1' for this to take effect.


brody
EMPLOYEE

6 months ago

That is normal, we do not support enabling vm.overcommit_memory


Loading...