[rslowinski]

Fixes #4349.

Problem

When a PoolConnection is dropped, the spawned return-to-pool task holds the connection's permit while it runs the after_release hook, pings, or closes the connection. Against a peer that vanished silently (no RST/FIN: NAT/firewall flow expiry, server restart mid-query, abandoned tokio::time::timeout) these awaits never complete. The permit is never released, and after max_connections such drops every acquire() fails with PoolTimedOut until the process restarts. Nothing logs, because nothing errors.

Fix

Per review, the whole Floating::return_to_pool call is wrapped in a 5s timeout (sibling of CLOSE_ON_DROP_TIMEOUT), which also covers the after_release and max_lifetime/pool-closed close() paths raised in the thread. On timeout the Floating is dropped: its DecrementSizeGuard releases the permit and the socket closes locally, so nothing inside (including close_hard) can pin the permit.

Test

tests/postgres/pool.rs runs a real PgPool against an in-process fake server that finishes the startup handshake and then goes silent. It drops a checked-out connection and asserts the next acquire() opens a fresh connection instead of hitting PoolTimedOut. Needs no DATABASE_URL; gated to runtime-tokio. Fails on main (PoolTimedOut at 20s), passes with the fix (~5s).

[adriangb]

Sorry about the noise, agent went a bit wild, I reigned it in and rewrote / went over this comment by hand.

We hit the same failure in production: a server restart during a query execution causes the query to hang forever. We noticed this because we have a ~ hot loop that is constantly hitting postgres, so we have a high chance of having an in-flight query where the timing of an upstream restart causes it to hang.

A gap found working on #4354 (now closed in favor of this PR): return_to_pool has three unbounded awaits, not one. The ping is the common case, but the same permit is held across:

• the after_release hook, which is arbitrary user code and, more to the point, usually runs a query on the same dead socket;
• self.close().await on the pool-closed and beyond-max_lifetime paths. close() is a graceful close: it writes Terminate and waits for the server to drop the connection, which on a silent peer never happens.

The max_lifetime one is not hypothetical for us: we set it, so every aged-out connection takes that path, and an aged-out connection on a dead socket leaks its permit exactly like the ping case.

The fix is mechanical: give each the same bound and fall back to close_hard(), which does no I/O. Cancelling close() also drops the connection and releases the permit, so a bounded close is safe:

async fn close_bounded(self) {
    let _ = crate::rt::timeout(RETURN_TO_POOL_PING_TIMEOUT, self.close()).await;
}

[abonander]

It would be better to wrap the entire return_to_pool call in the timeout just to insure against stuff like close_hard deadlocking (though it really shouldn't).

I already have this fix in #3582 but that won't be merged until 0.10 at the earliest.

[rslowinski]

Done: the timeout now wraps the whole return_to_pool call in PoolConnection::return_to_pool, so a timeout drops the Floating and releases the permit regardless of which await hung (ping, after_release, or close() on the max-lifetime / pool-closed paths). Rebased on main and squashed to one commit; test verified failing on main and passing with the fix.