feat(transport): add DoH connection self-healing (transparent retry + pool rebuild) (#42)

* feat(transport): add DoH connection self-healing (transparent retry + pool rebuild)

DohClient previously had no self-healing, unlike TcpMuxClient/DotMuxClient/
DoqMuxClient. When a proxy node flapped and DoH connections turned half-open,
reqwest reused the dead connections indefinitely (POST is non-idempotent so
hyper won't auto-retry), causing persistent 3s timeouts until restart (#41).

This adds the two safeguards mirroring the mux clients, adapted to reqwest's
constraints (no per-connection reset, no reuse signal):

- Transparent retry: on transport error (timeout/connect/IO), retry once with a
  fresh-connection budget (>=1.5s); DNS queries are semantically idempotent.
- Pool rebuild: per-upstream consecutive error counter; at threshold (default 3,
  configurable via doh_health_check_error_threshold), rebuild the reqwest::Client
  — drop evicts the dead connection pool.

Uses ArcSwap<DohHttpClient> for a hot-swappable pool (already a repo dependency)
and DashMap for per-upstream error counts. The send() signature and all callers
are unchanged.

Tests cover the healing contract (threshold rebuild, counter reset, per-upstream
isolation, error classification) plus an empirical test proving that dropping a
reqwest::Client closes its pooled keep-alive connections (the rebuild premise).

Closes #41

* style: apply rustfmt formatting to DoH self-healing tests

---------

Co-authored-by: kix <olices@9up.in>

Author: olicesx

src/config.rs

@@ -131,6 +131,10 @@ pub struct GlobalSettings {
     /// DoH 上游连接池大小（每个 upstream 的最大空闲连接数）。 / DoH upstream pool size (max idle per host)
     #[serde(default = "default_doh_pool_size")]
     pub doh_pool_size: usize,
+    /// DoH 健康检查错误阈值（连续传输错误多少次后重建连接池以驱逐死连接）。 / DoH health check error threshold (rebuild pool after N consecutive transport errors to evict dead connections)
+    /// 默认 3 次，0 表示禁用健康检查 / Default 3, 0 means disable health check
+    #[serde(default = "default_doh_health_check_error_threshold")]
+    pub doh_health_check_error_threshold: usize,
     /// DoT 上游连接池大小。 / DoT upstream connection pool size
     #[serde(default = "default_dot_pool_size")]
     pub dot_pool_size: usize,
@@ -255,6 +259,7 @@ impl Default for GlobalSettings {
             udp_pool_size: default_udp_pool_size(),
             tcp_pool_size: default_tcp_pool_size(),
             doh_pool_size: default_doh_pool_size(),
+            doh_health_check_error_threshold: default_doh_health_check_error_threshold(),
             dot_pool_size: default_dot_pool_size(),
             doq_pool_size: default_doq_pool_size(),
             tcp_health_check_error_threshold: default_tcp_health_check_error_threshold(),
@@ -911,6 +916,10 @@ fn default_doh_pool_size() -> usize {
     8
 }
 
+fn default_doh_health_check_error_threshold() -> usize {
+    3 // 连续 3 次传输错误后重建连接池 / Rebuild pool after 3 consecutive transport errors
+}
+
 /// Default DoH query path (RFC 8484 standard) / 默认 DoH 查询路径（RFC 8484 标准）
 fn default_doh_path() -> String {
     "/dns-query".to_string()

src/engine/core.rs

@@ -111,6 +111,7 @@ impl Engine {
         let udp_pool_size = cfg.settings.udp_pool_size;
         let tcp_pool_size = cfg.settings.tcp_pool_size;
         let doh_pool_size = cfg.settings.doh_pool_size;
+        let doh_health_error_threshold = cfg.settings.doh_health_check_error_threshold;
         let dot_pool_size = cfg.settings.dot_pool_size;
         let doq_pool_size = cfg.settings.doq_pool_size;
         let doq_idle_timeout_secs = cfg.settings.doq_connection_idle_timeout_seconds;
@@ -341,7 +342,10 @@ impl Engine {
             .context("initialize DoT multiplexer")?,
         );
 
-        let doh_client = Arc::new(DohClient::new(doh_pool_size).context("initialize DoH client")?);
+        let doh_client = Arc::new(
+            DohClient::new(doh_pool_size, doh_health_error_threshold)
+                .context("initialize DoH client")?,
+        );
         let doq_client = Arc::new(
             DoqClient::new(
                 doq_pool_size,