Use pipeline aggregation when data is implicitly sorted by group-by keys #19655

Activity

1. NGA-TRAN

   added the

   Feature

   issue type on Jan 5, 2026
2. NGA-TRAN

   added

   enhancementNew feature or requestNew feature or request

   on Jan 5, 2026
3. xavlee commented on Jan 5, 2026

   

   xavlee

   on Jan 5, 2026

   Contributor

   More actions

   take
4. github-actions

   assigned

   xavlee

   on Jan 5, 2026
5. NGA-TRAN commented on Jan 12, 2026

   

   NGA-TRAN

   on Jan 12, 2026

   ContributorAuthor

   More actions

   Some hints: I suggest we start by running some benchmarking or performance tests to determine whether this is worth pursuing.

   • Create two different schemas on the same dataset: one explicitly sorted on (a, b) and another sorted only on a. The first should produce a group‑by pipeline, while the second will fall back to a group‑by hash.
   • We may also want to run these tests on more realistic query such as (data is explicitly sorted on f_dkey, timestamp but implicitly sorted by f_dkey, date_bin, env)

   SELECT env, time_bin, AVG(max_bin_value) AS avg_max_value
   FROM
   (
       SELECT  f_dkey,
               date_bin(INTERVAL '30 seconds', timestamp) AS time_bin,
               env,
               MAX(value) AS max_bin_value
       FROM
           (
           SELECT
               f.f_dkey,
               d.env,
               d.service,
               d.host,
               f.timestamp,
               f.value
           FROM dimension_table d
           INNER JOIN fact_table_ordered f ON d.d_dkey = f.f_dkey
           WHERE service = 'log'
           ) AS j
       GROUP BY f_dkey, time_bin, env
   ) AS a
   GROUP BY env, time_bin
   ORDER BY env, time_bin;

   • Also, these 2 PRs have some related properties and tests:
     • feat: Preserve File Partitioning From File Scans #19124
     • feat: hash partitioning satisfies subset #19304

   @xavlee : When you start working on this, let us chat with @gene-bordegaray first.

   ❤️React with ❤️1Reacted by xavlee