HA Postgres creating thousands of error logs
dev-louis
HOBBYOP

9 months ago

Hi all,

As you can see above, the Patroni-based Postgres HA template on Railway seems to be generating thousands of these errors. When you look a bit closer they seem to be only INFO logs, but is there some way to make it so that they do not appear in the error logs widget, as it makes it relatively difficult to depict actual errors from these INFO ones, without further filtering etc.?

Sorry if I am doing something blatantly wrong or missing something silly.

Many thanks in advance

Attachments

$20 Bounty

1 Replies

reacx-research
FREE

3 days ago

Your screenshot shows Patroni heartbeat messages (INFO, “no action”), while Railway's widget is classifying them as errors. That follows Railway's documented logging rule: output on stderr is assigned error level. Patroni normally writes its logs to stderr, including INFO messages.

To stop those INFO messages entering the widget, set this variable on each Patroni service:

PATRONI_LOG_LEVEL=WARNING

That keeps Patroni warnings and errors but suppresses INFO output everywhere, not just in the widget. If you want to retain most INFO logging and only reduce the repeated heartbeat messages, Patroni also supports:

PATRONI_LOG_DEDUPLICATE_HEARTBEAT_LOGS=true

Deduplication suppresses repeated “no action”/lock-owner messages, but other INFO records can still appear as Railway errors. It also removes per-loop timing information that can be useful during failover diagnostics.

If PATRONI_CONFIGURATION is set, change its log section instead: that complete configuration overrides the individual PATRONI_* environment variables.

These are Patroni logging settings, not PostgreSQL server log settings. Environment changes take effect when the corresponding Patroni agent restarts. Apply them using your HA maintenance/restart procedure rather than restarting every cluster member together.

Sources:

Railway's stderr and structured-log rules: https://docs.railway.com/observability/logs#structured-logs

Patroni's supported environment settings: https://github.com/patroni/patroni/blob/master/docs/ENVIRONMENT.rst#log

Patroni's heartbeat and restart guidance: https://patroni.readthedocs.io/en/latest/faq.html#configuration

AI-assisted reply; the screenshot and linked documentation were reviewed. I have not changed or tested your HA cluster.


Welcome!

Sign in to your Railway account to join the conversation.

Loading...