Skip to content

Improve performance of aggregate expressions by implementing group accumulators #1627

Closed
#4254
Closed
Improve performance of aggregate expressions by implementing group accumulators#1627
#4254
Assignees
andygrove
Labels
enhancementNew feature or requestperformance
Milestone
0.17.0

Description

@andygrove
andygrove
opened on Apr 8, 2025

What is the problem the feature request solves?

DataFusion has a fast path for aggregates that implement group accumulators.

See #1275 (comment) for one example where we saw a 4x speedup.

We should apply this change to the following Comet aggregates.

  • stddev
  • correlation
  • covariance
  • variance

Describe the potential solution

No response

Additional context

No response

Activity

  1. andygrove
    added
    enhancementNew feature or request
    performance
    on Apr 8, 2025
  2. andygrove
    added this to the 0.9.0 milestone on Apr 8, 2025

  3. dharanad commented on May 15, 2025

    @dharanad
    dharanad
    on May 15, 2025
    Contributor

    I would like implement group accumulators variance

  4. andygrove
    removed this from the 0.9.0 milestone on Jun 5, 2025
  5. andygrove
    self-assigned this
    on May 6, 2026
  6. andygrove
    added this to the 0.17.0 milestone on May 6, 2026
  7. andygrove
    added this to Comet Developmenton May 6, 2026
  8. github-project-automation
    moved this to Todo in Comet Developmenton May 6, 2026
  9. github-project-automation
    moved this from Todo to In progress in Comet Developmenton May 6, 2026
  10. andygrove
    closed this as completedin #4254on May 19, 2026
  11. github-project-automation
    moved this from In progress to Done in Comet Developmenton May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestperformance

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions