You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Postgres connection leak: each (re)connect on a logged-out instance leaks an unclosed sqlstore pool (evogo_auth/evogo_users) #106
Every (re)connect attempt on an instance whose WhatsApp session was logged out (pairing removed on the phone → whatsmeow LoggedOut) leaks a Postgres connection pool: a new whatsmeow client + sqlstore is created per attempt, the login fails with LoggedOut (expected), and the abandoned sql.DB pool is never Close()d. The idle connections stay parked in Postgres forever.
With any periodic reconnect source, this grows unbounded until the whole Postgres cluster hits max_connections and refuses every client. Aggravated by evolution-go connecting as the postgres superuser by default, which also exhausts superuser_reserved_connections — locking the DBA out during the incident.
We hit this twice in production in two days, through two different trigger paths:
Variant A — external POST /instance/connect (day 1)
Our supervisor retried /instance/connect every 2 min on a logged-out instance.
Postgres log: 742 × FATAL: sorry, too many clients already, origin user=postgres, db=evogo_users
Connection count of the evolution-go process (nsenter … ss -tn | grep :5432): 167 (healthy baseline for the same deployment: ~15–20)
Exact correlation: 88 authenticated POST /instance/connect (HTTP 200) since container start ≈ the accumulated leaked connections (+1 per attempt). 401 attempts (invalid token) do not leak — the leak is in the authenticated path that instantiates the client/sqlstore.
docker restart evolution-go instantly dropped the count 167 → 3 and the cluster recovered.
Variant B — internal whatsmeow auto-reconnect (day 2, no API calls at all)
After we stopped retrying via API, a logged-out instance kept cycling through whatsmeow's internal auto-reconnect:
25 × LoggedOut connection events in 6h for one instance, only 2 authenticated /instance/connect in the same window (so not API-triggered)
121 idle connections accumulated in db=evogo_auth (session store), ~7/hour, all state=idle, oldest from the moment the cycling started
Same fix: restart frees them instantly; the leak resumes while the logged-out instance keeps retrying.
Suspected root cause
On the connect path, a new whatsmeow client + sqlstore container (its own database/sql pool) is created per attempt. When login fails with LoggedOut, the client is discarded but the store/pool is not closed. An abandoned Go sql.DB keeps its idle connections open indefinitely.
Suggested fix
Close the sqlstore/sql.DB on the connect error path (or reuse the instance's existing client/pool instead of creating a new one per attempt);
Treat LoggedOut as terminal in the reconnect loop: stop auto-retrying a session that requires re-pairing (surface "re-pair required" instead);
Independently: document/support running with a non-superuser DB role — with the default superuser connection, any leak escalates to a full-cluster outage including the reserved slots.
Workarounds we use meanwhile
Supervisor treats LoggedOut as terminal and stops calling /instance/connect;
docker restart evolution-go when the count grows;
Planning a dedicated DB role with CONNECTION LIMIT as a blast-radius guard.
Happy to provide more logs/details if useful. 🇧🇷 Podemos detalhar em PT-BR também, se preferirem.
I ran into this exact same issue where connection pools were leaking during reconnect loops (especially with proxy rotations and logged-out instances) and eventually hitting the sorry, too many clients limit.
Until the PR fix is fully merged/shipped, the best workaround I found is to configure PostgreSQL to clean up these abandoned connections automatically. Since the app loses reference to the old pools, they just sit in the database as idle.
You can set idle_session_timeout to automatically close them after a few minutes.
If you're using Docker Compose, just add command: postgres -c idle_session_timeout=300000 (this sets it to 5 minutes) to your postgres service:
yaml
postgres:
image: postgres:15-alpine
container_name: sendzai_db
restart: always
command: postgres -c idle_session_timeout=300000
environment:
POSTGRES_USER: admin
POSTGRES_PASSWORD: your_password
POSTGRES_DB: evolution
If you are running Postgres natively, you can append this to your postgresql.conf and restart/reload:
idle_session_timeout = 300000
This completely stopped our DB from running out of connection slots and keeps everything running smoothly. Hopefully this helps anyone stuck on this right now!
I ran into this exact same issue where connection pools were leaking during reconnect loops (especially with proxy rotations and logged-out instances) and eventually hitting the sorry, too many clients limit.
Until the PR fix is fully merged/shipped, the best workaround I found is to configure PostgreSQL to clean up these abandoned connections automatically. Since the app loses reference to the old pools, they just sit in the database as idle.
You can set idle_session_timeout to automatically close them after a few minutes.
If you're using Docker Compose, just add command: postgres -c idle_session_timeout=300000 (this sets it to 5 minutes) to your postgres service:
yaml
postgres:
image: postgres:15-alpine
container_name: sendzai_db
restart: always
command: postgres -c idle_session_timeout=300000
environment:
POSTGRES_USER: admin
POSTGRES_PASSWORD: your_password
POSTGRES_DB: evolution
If you are running Postgres natively, you can append this to your postgresql.conf and restart/reload:
idle_session_timeout = 300000
This completely stopped our DB from running out of connection slots and keeps everything running smoothly. Hopefully this helps anyone stuck on this right now!
Hello!
Thanks for the help, @sendzai. To test this out, I ran the following on my database: ALTER DATABASE evogo_users SET idle_session_timeout = '5min'; ALTER DATABASE evogo_auth SET idle_session_timeout = '5min';
It has worked so far, and I am currently testing it.
I did this because I use the same container for other applications.
We're hitting the same leak through a third trigger path not yet documented here: the internal auto-reconnect on healthy instances — no external /instance/connect calls, no logged-out sessions involved.
Whenever a connected instance drops (ordinary WhatsApp/network blip), evolution-go logs:
[uuid] UserInfo cache cleared for token: ***
[uuid] Disconnected detected, restarting instance
[uuid] Starting reconnection process - simulating restart
and each of these internal restarts orphans one sqlstore pool (~2 idle Postgres connections, Go database/sql default MaxIdleConns).
Quantified over a 5h window on latest (2026-07), 7 instances, moderate traffic:
Component
Connections on the auth DB
Baseline: 7 instances × 2 idle per pool
14
9 internal restarts × 2 orphaned each
+18
Predicted / measured
32 / 31
pg_stat_activity shows the orphaned connections being born in pairs exactly at the restart timestamps, and never closing (one sat idle for 3+ hours). On a day with unstable connectivity this exhausted the default max_connections=100 twice in 24h — and it feeds back on itself: pool saturation makes instances drop, which triggers more internal restarts.
The diagnosis in #102 matches what we observe sqlstore.New() per (re)connect, container never closed) — that fix should cover this path too, as long as it also applies to the internal "simulating restart" flow, not only the external connect endpoints.
This should be fixed by #117 — it makes the whatsmeow auth-store use a single shared sqlstore.Container with a capped pool instead of opening a new unbounded one on every (re)connect. Reproduced the leak locally (15 reconnects: 1 → 16 connections, unbounded growth afterwards) and verified the fix keeps it at 1-2.
Confirming on 0.7.2 — the trigger is unpaired instances stuck in a QR loop
We hit this on evoapicloud/evolution-go:0.7.2 and were able to pin down a
specific trigger that doesn't seem to be documented yet: instances that have
never been paired (no JID) get stuck in an endless QR cycle, and every cycle
leaks a pool.
Environment
Evolution GO 0.7.2, Docker Swarm, dedicated postgres:15-alpine
12 instances total: 7 paired (with JID), 5 never paired (no JID)
CONNECT_ON_STARTUP=true
Mechanism
With CONNECT_ON_STARTUP=true, GO attempts to connect all instances at boot,
including logged-out ones. The 5 unpaired instances then loop forever:
[<id>] QR code generated #3 (max: 5)
[<id>] QR code generated #4 (max: 5)
[<id>] Maximum QR code count reached (5), forcing logout
[<id>] No jid found. Creating new device
[<id>] QR timeout — signaling kill channel
↳ and it starts over
Each Creating new device builds a new client and a new sqlstore pool that is
never closed. Connections then climb until:
Failed to create container: failed to upgrade database:
failed to check if version table is up to date: pq: sorry, too many clients already
Once that happens, GO can no longer create clients at all: QR generation stops
working and dropped instances can't reconnect.
Raising max_connections (300 → 500): only delays it.
idle_session_timeout=10min: verified active via SHOW idle_session_timeout,
made no measurable difference. The leak is bursty — a runaway retry loop
saturates faster than the timeout can reap.
What DID fix it
Setting CONNECT_ON_STARTUP=false, then reconnecting only the paired instances
via POST /instance/reconnect:
before: ~500 connections, saturating every ~60-90 min
after: 30 connections, stable, 0 QR codes generated
Same 7 instances connected and working in both cases. The only difference is
that the 5 unpaired instances are no longer being connected.
The QR cycle for a logged-out instance never terminates — it forces a logout
and immediately creates a new device, so it can loop indefinitely. Capping
that (or requiring an explicit connect request) would remove the amplifier
even before the pool leak is fixed.
Activity
sendzai commentedon Jul 14, 2026
Hey everyone,
I ran into this exact same issue where connection pools were leaking during reconnect loops (especially with proxy rotations and logged-out instances) and eventually hitting the sorry, too many clients limit.
Until the PR fix is fully merged/shipped, the best workaround I found is to configure PostgreSQL to clean up these abandoned connections automatically. Since the app loses reference to the old pools, they just sit in the database as idle.
You can set idle_session_timeout to automatically close them after a few minutes.
If you're using Docker Compose, just add command: postgres -c idle_session_timeout=300000 (this sets it to 5 minutes) to your postgres service:
This completely stopped our DB from running out of connection slots and keeps everything running smoothly. Hopefully this helps anyone stuck on this right now!
jonathanpog98 commentedon Jul 14, 2026
Hello!
Thanks for the help, @sendzai. To test this out, I ran the following on my database:
ALTER DATABASE evogo_users SET idle_session_timeout = '5min'; ALTER DATABASE evogo_auth SET idle_session_timeout = '5min';It has worked so far, and I am currently testing it.
I did this because I use the same container for other applications.
guilhermeCassettari commentedon Jul 14, 2026
We're hitting the same leak through a third trigger path not yet documented here: the internal auto-reconnect on healthy instances — no external /instance/connect calls, no logged-out sessions involved.
Whenever a connected instance drops (ordinary WhatsApp/network blip), evolution-go logs:
and each of these internal restarts orphans one sqlstore pool (~2 idle Postgres connections, Go database/sql default MaxIdleConns).
Quantified over a 5h window on latest (2026-07), 7 instances, moderate traffic:
pg_stat_activity shows the orphaned connections being born in pairs exactly at the restart timestamps, and never closing (one sat idle for 3+ hours). On a day with unstable connectivity this exhausted the default max_connections=100 twice in 24h — and it feeds back on itself: pool saturation makes instances drop, which triggers more internal restarts.
The diagnosis in #102 matches what we observe sqlstore.New() per (re)connect, container never closed) — that fix should cover this path too, as long as it also applies to the internal "simulating restart" flow, not only the external connect endpoints.
guilhermeCassettari commentedon Jul 15, 2026
This should be fixed by #117 — it makes the whatsmeow auth-store use a single shared sqlstore.Container with a capped pool instead of opening a new unbounded one on every (re)connect. Reproduced the leak locally (15 reconnects: 1 → 16 connections, unbounded growth afterwards) and verified the fix keeps it at 1-2.
ShowtimeProp commentedon Aug 1, 2026
Confirming on 0.7.2 — the trigger is unpaired instances stuck in a QR loop
We hit this on
evoapicloud/evolution-go:0.7.2and were able to pin down aspecific trigger that doesn't seem to be documented yet: instances that have
never been paired (no JID) get stuck in an endless QR cycle, and every cycle
leaks a pool.
Environment
postgres:15-alpineCONNECT_ON_STARTUP=trueMechanism
With
CONNECT_ON_STARTUP=true, GO attempts to connect all instances at boot,including logged-out ones. The 5 unpaired instances then loop forever:
Each
Creating new devicebuilds a new client and a new sqlstore pool that isnever closed. Connections then climb until:
Once that happens, GO can no longer create clients at all: QR generation stops
working and dropped instances can't reconnect.
Numbers
max_connections=300, few instances connectedmax_connections=500, 7 instances connected + 5 loopingWhat did NOT help
max_connections(300 → 500): only delays it.idle_session_timeout=10min: verified active viaSHOW idle_session_timeout,made no measurable difference. The leak is bursty — a runaway retry loop
saturates faster than the timeout can reap.
What DID fix it
Setting
CONNECT_ON_STARTUP=false, then reconnecting only the paired instancesvia
POST /instance/reconnect:Same 7 instances connected and working in both cases. The only difference is
that the 5 unpaired instances are no longer being connected.
Suggested angle
Two things seem worth looking at independently:
(already covered by Postgres connection leak: each (re)connect on a logged-out instance leaks an unclosed sqlstore pool (evogo_auth/evogo_users) #106 / PostgreSQL connection leak: a new sqlstore pool is created on instance reconnect #118).
and immediately creates a new device, so it can loop indefinitely. Capping
that (or requiring an explicit connect request) would remove the amplifier
even before the pool leak is fixed.
Happy to provide more logs if useful.
fix(whatsmeow): create the sqlstore container once instead of on ever…
rebuild 12/10/2026: pool do Postgres, mutex nos mapas, Disconnect pre…