2 years ago
I always have this issue in Postgresql logs
2024-04-30 02:43:15.804 UTC [66070] FATAL: sorry, too many clients already
with this database.js config on Strapi side
module.exports = ({ env }) => ({
connection: {
client: 'postgres',
connection: {
host: env('PGHOST'),
port: env('PGPORT'),
database: env('PGDATABASE'),
user: env('PGUSER'),
password: env('PGPASSWORD'),
ssl: {
rejectUnauthorized: false,
ca: env("DATABASE_CA", undefined)
},
charset: 'utf8mb4',
},
debug: false,
pool: {
min: 0,
max: 100,
acquireTimeoutMillis: 30000,
createTimeoutMillis: 30000,
destroyTimeoutMillis: 5000,
idleTimeoutMillis: 30000,
reapIntervalMillis: 500,
createRetryIntervalMillis: 100,
propagateCreateError: false,
},
},
});
The connection works well for the application, but I cannot connect to Postgresql via "Data" tab in railway or via 3rd party service (i.e. service for creating backups for AWS S3).
Am I able to configure max_connections variable? How can I debug this issue?
17 Replies
2 years ago
Postgres has a default max connection of 100 (Postgres itself, not Railway) so if you don't require 100, I would significantly lower that, Strapi has a default of max 10 fwiw.
Am I able to configure max_connections variable?
Yes, you can, but 100 is far more than you would likely need.
2 years ago
Could you please provide a way to increase this number?
Our maximum spotted online was 100 people at the same time (40k people per week). So I am wondering how to calculate how many connections we need to handle big online.
2 years ago
I was able to setup creating backups at the moment only by setting to check postgresql availability each 5 minutes, and sometimes it have success to connect. But it doesn't look good as even when online is less than 5 people, it says that "sorry, too many clients already". Probably the thing is that they send a lot of API requests, that's why all 100 connections may be used at the same time
2 years ago
Increasing the connection limit of Postgres would only be a bandaid solution, the correct solution here would be to put pgbouncer in front of your database, then have Strapi connect to pgbouncer so that connections can be reused.
We have a pgbouncer template available that you can easily deploy into your project from the + New button in the top right of your project canvas.
2 years ago
Thank you, I will try and let you know
2 years ago
I have setup pgbouncer, which I believe helps, but the problem hasn't gone
Attachments
2 years ago
Are you making sure Strapi is connecting to pgbouncer instead of directly to Postgres?
2 years ago
You are right
Strange, when I try to connect (via Strapi) to pgbouncer, I receive knex mistake
There seems to be an unexpected error, try again with --debug for more information
KnexTimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call
Attachments
2 years ago
Strapi
PGDATABASE=railway
PGHOST=monorail.proxy.rlwy.net
PGPASSWORD=real_password
PGPORT=48270
PGUSER=postgres
Bouncer
postgres://postgres:real_password@monorail.proxy.rlwy.net:48270/railway
Postgresql
postgresql://postgres:real_password@viaduct.proxy.rlwy.net:24875/railway
I was able to connect to both via PgAdmin
2 years ago
2 years ago
After all, it seems to me that Strapi has a built-in "pgbouncer" (knex pooling system), and don't need it
2 years ago
I have tried to configure 2nd pgbouncer template (by Jonathan), and the same. I am able to connect via pgAdmin, but not via Strapi (connection timeout)
2 years ago
Let go back to my original proposed solution, don't set the pool max at 100.
2 years ago
Maybe some important detail to notice: server has 6 replicas (+ 1 original server, if I am correct in counting). Not sure if it influences a lot on number of idle connections.
So, I tried to decrease max pool number from 100 to 10. And you are right - it helped. If I understand correctly, each replica creates 10 idle connections by default (that's why 66 connections opened, so it means that original server is included in replicas number and 6 servers are working, not 7). And after all, I don't see this mistake in postgresql logs "sorry, too many clients already". Now it's good.
So, my next assumption is that I need to configure max number based on number of replicas that I have. In case I would have 50 replicas, I should set max value as 2 or even 1.
Attachments
2 years ago
My only concern that I do not understand how I can, in the future, upscale database to handle bigger online using Railway tools. If I don't need pgbouncer, and don't have built-in-solutions to do it on my own, probably the only way is contacting support for some kind of horizontal scailing of postgresql.
2 years ago
Railway can not horizontal scale a database but that's not needed as increasing the connection limit only involves running a few simple queries on your database.
2 years ago
As far as I understood, in case of need I can ask Railway support to do this queries, not on my own