How much free space do you have on the volume?

More than 75GB, so that shouldn't be an issue?

67MB is larger than the default shm size of 64MB, can you try decreasing your shared_buffer size to say 32MB?

Lowering that will be a big hit on the performance of my DB I think, so I am a bit careful with this as there is a lot of live traffic. Currently, my shared_buffer is set to 4GB.

Just curious, why do you think, this would help potentially?

The default shm size is set to 64MB, having this increased by Railway would likely require moving to a plan above Pro because they aren't able to offer that level of customization on the Pro plan. I'm not sure if that interests you right now, so decreasing your shared_buffer size would likely be the work around, though I have yet to test it.

Just went for it as I am really trying to figure this out haha. Restarted my DB with the new setting, but getting the same error.

Have you confirmed your shared_buffer size has been updated?

Yup

Interesting, I set the maintenance_work_mem to 256MB (docs mention VACUUM here: https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-MAINTENANCE-WORK-MEM):

Now I get the error of:
ERROR: could not resize shared memory segment "/PostgreSQL.1250465576" to 140293440 bytes: No space left on device

Which is ~140MB

Not sure what I am doing wrong here.

I have a 16GB database with a lot of test rows, and I can consistently trigger the same error when running VACUUM except mine tries to use ~780 MB of shared memory.

I'm waiting for a response from the team for further actions.

Thanks!

@brody, did you hear anything back from the team?

I haven't heard anything back yet unfortunately.

Hey! The shmsize isn't something configurable at this point unfortunately. Perhaps we could try tuning Postgres to get around it, using this start cmd:

/usr/local/bin/docker-entrypoint.sh postgres --port=5432 -c 'sharedbuffers=4096MB' -c 'maintenancework_mem=512MB'

on the db service.

Without modifications, when I run VACUUM on my database, I get the same error but for over 700MB, setting maintenance_work_mem to 512MB does limit the out-of-memory error to 512MB, but considering the default shm_size on Railway is only 64MB that alone doesn't help.

Setting both shared_buffers and maintenance_work_mem to 32MB does work, I am now able to run a VACUUM on a table with over 38 million rows in 7.8 seconds.

/usr/local/bin/docker-entrypoint.sh postgres --port=5432 -c 'sharedbuffers=32MB' -c 'maintenancework_mem=32MB'

That also seems to work for me, however the performance of my DB is severely impacted when I set the shared_buffers size to only 32MB.

I'm also having the same issue - trying to run VACUUM on a large table (14M rows) fails with the same "could not resize shared memory segment" error. Setting shared_buffers and maintenance_work_mem to 32MB allowed the VACUUM to work, but now I'm worried this has performance implications for my DB.

Your VACUUM commands would run slower so yes it would have some performance implications for very heavy commands, but that's better than it erroring out.

How much memory did it try to allocate?

67MB (67145728 bytes). This is the full error message I got from running VACUUM:

could not resize shared memory segment "/PostgreSQL.3407876500" to 67145728 bytes: No space left on device

I guess I'm fine with VACUUM being slower, like you said, it's better than erroring. I would prefer if Railway's Postgres deployments were configured for this out of the box though.

Since you're Pro you can set a service variable RAILWAY_SHM_SIZE_BYTES to 536870912 (512MB) so that your container gets started with more shared memory, you can then undo the shared_buffers and maintenance_work_mem changes you have made, simply removing them might not undo it, make sure to run some queries to check their values.

Sounds good. I made that change & confirmed that shared_buffers and maintenance_work_mem are higher than before.

Awsome, let me know if you continue to get shared memory errors.