Skip to content

[Python] Support serialization of Arrow files on disk without the identifier "Feather" #38515

Description

@jason-s

Describe the enhancement requested

The documentation for Arrow Columnar Format suggests that the separate Feather project has been subsumed into Arrow, and that it (Feather) is really just the canonical serialization format for Arrow tables:

We recommend the “.arrow” extension for files created with this format. Note that files created with this format are sometimes called “Feather V2” or with the “.feather” extension, the name and the extension derived from “Feather (V1)”, which was a proof of concept early in the Arrow project for language-agnostic fast data frame storage for Python (pandas) and R.

The Python support of Arrow serialization still uses the identifier feather: (see the Cookbook)

Once we have a table, it can be written to a Feather File using the functions provided by the pyarrow.feather module

import pyarrow.feather as ft

ft.write_feather(table, 'example.feather')

This functionality should be kept as is, for backwards compatibility, but I wonder if the pyarrow module should just have a write() function, without requiring the need to import the pyarrow.feather package or use the term feather. This would help to reduce confusion about file extensions and the relationship between "Arrow" and "Feather".

Component(s)

Python

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions