Move the to_timestamp* functions to datafusion-functions #9291

Activity

1. Omega359

   added

   enhancementNew feature or requestNew feature or request

   on Feb 20, 2024
2. Omega359 commented on Feb 20, 2024

   

   Omega359

   on Feb 20, 2024

   ContributorAuthor

   More actions

   take
3. github-actions

   assigned

   Omega359

   on Feb 20, 2024
4. Omega359 commented on Feb 21, 2024

   

   Omega359

   on Feb 21, 2024

   ContributorAuthor

   More actions

   Not sure what to do about references to to_timestamp function in test code such as

   • core/tests/parquet/row_group_pruning.rs
   • core/tests/parquet/page_pruning.rs
   • optimizer/simplify_expressions/simplify_exprs.rs

   This seems mostly to be in tests however I'm unsure of the wisdom of adding a dependency to datafusion-functions in these modules (core/optimizer/etc)

   The issue really isn't limited to to_timestamp functions - any tests in those crates that use functions that we move to the new crate will have the same issue.
5. alamb commented on Feb 26, 2024

   

   alamb

   on Feb 26, 2024

   Contributor

   More actions

   Not sure what to do about references to to_timestamp function in test code such as

   • core/tests/parquet/row_group_pruning.rs
   • core/tests/parquet/page_pruning.rs

   I think the core tests will be fine as they should already have access to the (core) datafusion crate and thus to the functions in question

   • optimizer/simplify_expressions/simplify_exprs.rs

   This seems mostly to be in tests however I'm unsure of the wisdom of adding a dependency to datafusion-functions in these modules (core/optimizer/etc)

   I agree we should avoid this dependency if possible

   One potential way to fix this dependency is to move most of the actual tests out of optimizer/simplify_expressions/simplify_exprs.rs and into core/tests/ somewhere -- perhaps into https://github.com/apache/arrow-datafusion/blob/a9f0c7afe2ea3bb14bfcceedfcd33cfe4b954035/datafusion/core/tests/simplification.rs#L1-L0

   Many of those simplify tests seem to be tests that certain functions get simplified rather than a unit test of the optimizer. Thus they make more sense to be in the core crate from my perspective.

   What do you think @Omega359 ? I am sorry for the late response

   The issue really isn't limited to to_timestamp functions - any tests in those crates that use functions that we move to the new crate will have the same issue.
6. Omega359 commented on Feb 27, 2024

   

   Omega359

   on Feb 27, 2024

   ContributorAuthor

   More actions

   Thanks @alamb, I'll look at moving those tests out to the core/tests/ area.

   👍React with 👍1Reacted by Andrew Lamb
7. alamb

   closed this as completedin #9388on Mar 1, 2024