MySQL volume appears full despite only ~45 MB of table data
alexp953
PROOP

a month ago

Hello Railway Support,

I am experiencing recurring outages on a managed MySQL service and would like help investigating a possible storage-related issue.

Symptoms

My application works normally for a period of time, but eventually MySQL becomes unavailable and application workers start failing with:

SQLSTATE[HY000] [2002] Connection refused

The failures occur when attempting normal reads/writes against the database.

Database size

I inspected all application tables using information_schema and the total size is only approximately 45 MB:

  • pixels: ~30 MB
    • bot_faction_plan_pixels: ~14.6 MB
    • all remaining tables combined: ~1 MB

The largest tables are therefore relatively small.

Railway volume usage

Despite the small amount of actual table data, the Railway volume usage repeatedly reaches around 92-97% of a 1 GB volume.

When the volume approaches this level, MySQL becomes unstable and application workers start failing.

MySQL logs

The MySQL logs contain repeated InnoDB errors such as:

  • Cannot resize redo log file
    • Failed to set size
    • Retry attempts for writing partial data failed

Example messages:

[InnoDB] Cannot resize redo log file ./#innodb_redo/#ib_redoXXXX_tmp to 3 MB (Failed to set size)

1048576 bytes should have been written. Only 966656 bytes written. Retrying for the remaining bytes.

These errors repeat many times before MySQL becomes unavailable.

Additional information

  • MySQL version: 9.4
    • log_bin = OFF
    • Application tables have already been cleaned aggressively:
    • bot_events = 0 rows
  • sessions = 0 rows
    • cache = 0 rows
  • The issue persists after deleting large amounts of data.

Questions

Could you please help verify:

  1. What is actually consuming the volume space?
  2. Whether InnoDB system files, redo logs, temporary files, or fragmentation are responsible for the reported usage.
  3. Whether the volume may be in a corrupted or unreclaimed state.
  4. Whether rebuilding/recreating the volume would reclaim space.
  5. Whether there are any known issues affecting MySQL storage volumes at the moment.

From the application side, the total database content appears far smaller than the reported volume usage, so I suspect the issue may be related to MySQL internal files or the underlying storage volume.

Thank you.

Screenshot_20260605-164713.png

Screenshot_20260605-164638.png

$20 Bounty

1 Replies

Railway
BOT

a month ago

This thread has been opened as a public bounty so the community can help solve it. The thread and any further activity are now visible to everyone.

Status changed to Open Railway about 1 month ago


Even though your table data is only 45MB, MySQL requires a lot more overhead that a 1GB volume isn't really enough when relying on the default configurations. At this point, rebuilding the database from a logical dump is the best way to reclaim space. But you still need to increase your volume size, so you don't run into this problem again.


Welcome!

Sign in to your Railway account to join the conversation.

Loading...