Skip to content

Add projection to HashJoinExec to avoid unecessary output creation #6768

Closed

Description

@Dandandan

Is your feature request related to a problem or challenge?

Currently HashJoinExec returns all the columns from both sides of the join.

We can add the necessary output columns so we can save some work in gathering values (take) from batches based on matching indices.

This is similar to #5436

Describe the solution you'd like

Add projection to hashjoinexec, use it to reduce output.

Describe alternatives you've considered

No response

Additional context

No response

Activity

  1. changed the title [-]Add projection to HashJoinExec to avoid unecessary output creation[/-] [+]Add projection to `HashJoinExec` to avoid unecessary output creation[/+] on Jun 26, 2023
  2. my-vegetable-has-exploded commented on Jan 22, 2024

    @my-vegetable-has-exploded
    Contributor

    I'd like to have a try.

  3. eejbyfeldt commented on Sep 1, 2024

    @eejbyfeldt
    Contributor

    Is there something left to be done in this issue after #9236 My understanding is that it completes all the work currently described in the ticket.

  4. Dandandan commented on Sep 1, 2024

    @Dandandan
    ContributorAuthor

    It's done

  5. Dandandan commented on Sep 1, 2024

    @Dandandan
    ContributorAuthor

    Thanks @eejbyfeldt for the observation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestperformanceMake DataFusion faster

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions