Skip to content

cargo test reports errors on the master branch. #4236

Closed
#4358
Closed
cargo test reports errors on the master branch.#4236
#4358
Labels
bugSomething isn't working

Description

@HaoYang670
HaoYang670
opened on Nov 16, 2022

Describe the bug
A clear and concise description of what the bug is.

[2022-11-16T07:11:47Z ERROR datafusion::physical_plan::file_format::parquet::page_filter] Error evaluating page index predicate values Error during planning: Can not create statistics record batch: Invalid argument error: Column 'container_min' is declared as non-nullable but contains null values
[2022-11-16T07:11:47Z ERROR datafusion::physical_plan::file_format::parquet::page_filter] Error evaluating page index predicate values Error during planning: Can not create statistics record batch: Invalid argument error: Column 'container_min' is declared as non-nullable but contains null values
[2022-11-16T07:11:48Z ERROR datafusion::physical_plan::file_format::parquet::page_filter] Error evaluating page index predicate values Error during planning: Can not create statistics record batch: Invalid argument error: Column 'container_min' is declared as non-nullable but contains null values
[2022-11-16T07:11:48Z ERROR datafusion::physical_plan::file_format::parquet::page_filter] Error evaluating page index predicate values Error during planning: Can not create statistics record batch: Invalid argument error: Column 'container_min' is declared as non-nullable but contains null values
[2022-11-16T07:11:49Z ERROR datafusion::physical_plan::file_format::parquet::page_filter] Error evaluating page index predicate values Error during planning: Can not create statistics record batch: Invalid argument error: Column 'container_min' is declared as non-nullable but contains null values
[2022-11-16T07:11:49Z ERROR datafusion::physical_plan::file_format::parquet::page_filter] Error evaluating page index predicate values Error during planning: Can not create statistics record batch: Invalid argument error: Column 'pod_min' is declared as non-nullable but contains null values
test parquet::filter_pushdown::single_file ... ok

To Reproduce
Steps to reproduce the behavior:
Run

cargo test --all

on the master branch

Or just run the test parquet::filter_pushdown::single_file

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Activity

  1. HaoYang670
    added
    bugSomething isn't working
    on Nov 16, 2022

  2. HaoYang670 commented on Nov 16, 2022

    @HaoYang670
    HaoYang670
    on Nov 16, 2022
    ContributorAuthor

    A weird thing is that the cargo test doesn't fail.

  3. HaoYang670 commented on Nov 16, 2022

    @HaoYang670
    HaoYang670
    on Nov 16, 2022
    ContributorAuthor

    @alamb, could you please take a look?

  4. Ted-Jiang commented on Nov 16, 2022

    @Ted-Jiang
    Ted-Jiang
    on Nov 16, 2022
    Member

    @HaoYang670 Thanks , this expected now.
    because these cols are string , not supported yet , after #4132 done,this will be fixed.
    Although it reports errors, will have the fall back in https://github.com/apache/arrow-datafusion/blob/509c82c6d624bb63531f67531195b562a241c854/datafusion/core/src/physical_plan/file_format/parquet/page_filter.rs#L350-L351

    --------------------------------------------------------------------------------
service:             OPTIONAL BINARY L:STRING R:0 D:1
host:                REQUIRED BINARY L:STRING R:0 D:0
pod:                 REQUIRED BINARY L:STRING R:0 D:0
container:           REQUIRED BINARY L:STRING R:0 D:0
image:               REQUIRED BINARY L:STRING R:0 D:0
time:                REQUIRED INT64 L:TIMESTAMP(NANOS,false) R:0 D:0
client_addr:         OPTIONAL BINARY L:STRING R:0 D:1
request_duration_ns: REQUIRED INT32 R:0 D:0
request_user_agent:  OPTIONAL BINARY L:STRING R:0 D:1
request_method:      OPTIONAL BINARY L:STRING R:0 D:1
request_host:        OPTIONAL BINARY L:STRING R:0 D:1
request_bytes:       OPTIONAL INT32 R:0 D:1
response_bytes:      OPTIONAL INT32 R:0 D:1
response_status:     REQUIRED INT32 L:INTEGER(16,false) R:0 D:0
decimal_price:       REQUIRED FIXED_LEN_BYTE_ARRAY L:DECIMAL(38,0) R:0 D:0

row group 1:         RC:53819 TS:6441769 OFFSET:4
``

  5. alamb commented on Nov 16, 2022

    @alamb
    alamb
    on Nov 16, 2022
    Contributor

    The strange thing is that the errors appear on stdout while running cargo test

    i wonder if some test is accidentally enabling logging always 🤔

  6. alamb commented on Nov 24, 2022

    @alamb
    alamb
    on Nov 24, 2022
    Contributor

    Here is a proposed fix: #4358

  7. alamb
    closed this as completedin #4358on Nov 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions