Skip to content

Implement high-level Parquet writer #35

Closed
Closed
Implement high-level Parquet writer#35
Labels
parquetChanges to the parquet crate

Description

@alamb
alamb
opened on Apr 26, 2021

Note: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-4392

We only have low-level parquet writer at the moment, which requires user to specify values, definition levels, repetition levels, etc. This is inconvenient. Instead, we should offer high-level Parquet writer that hide these details.

Activity

  1. alamb
    added
    arrowChanges to the arrow crate
    on Apr 26, 2021

  2. alamb commented on Apr 26, 2021

    @alamb
    alamb
    on Apr 26, 2021
    ContributorAuthor

    Comment from Andy Grove(andygrove) @ 2019-10-10T13:45:56.175+0000:

    [~csun] Are you still planning on working on this?

    Comment from Andy Grove(andygrove) @ 2019-10-10T13:51:15.064+0000:

    [~liurenjie1024] You might be interested in working on this as well?

    Comment from Renjie Liu(liurenjie1024) @ 2019-10-11T06:32:27.110+0000:

    [~andygrove] Yes it looks interesting to me. I'll take this if [~csun] is not available.
  3. alamb
    added
    parquetChanges to the parquet crate
    and removed
    arrowChanges to the arrow crate
    on Apr 26, 2021

  4. alamb commented on May 12, 2022

    @alamb
    alamb
    on May 12, 2022
    ContributorAuthor

    I think the arrow writer (that writes parquet files from arrow arrays) is a pretty good high level parquet writer

    https://docs.rs/parquet/13.0.0/parquet/arrow/index.html

  5. alamb
    closed this as completedon May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    parquetChanges to the parquet crate

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions