Repository navigation
rdump to avro fails for basic use #228
Description
flow.record.version 3.21
Attempting a basic export to avro
uv run target-query /path/to/file.E01 -f users -q | uv run rdump -vvv -w users.avro
fails with
2026-05-13T15:56:35.967729Z [error ] rdump encountered a fatal error: Unsupported Avro type: path [flow.record.tools.rdump]
2026-05-13T15:56:35.967790Z [error ] Full traceback [flow.record.tools.rdump]
Traceback (most recent call last):
File "/Users/jeffbryner/development/test/.venv/lib/python3.12/site-packages/flow/record/tools/rdump.py", line 419, in main
record_writer.write(rec)
File "/Users/jeffbryner/development/test/.venv/lib/python3.12/site-packages/flow/record/adapter/avro.py", line 72, in write
self.schema = descriptor_to_schema(self.desc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jeffbryner/development/test/.venv/lib/python3.12/site-packages/flow/record/adapter/avro.py", line 159, in descriptor_to_schema
raise ValueError(f"Unsupported Avro type: {field_type}")
Or
uv run target-query /path/to/file.E01 -f users -q | uv run rdump -vvv -w avro://users.avro
fails with:
Traceback (most recent call last):
File "/Users/jeffbryner/development/test/.venv/lib/python3.12/site-packages/flow/record/tools/rdump.py", line 419, in main
record_writer.write(rec)
File "/Users/jeffbryner/development/testvil_hackathon/.venv/lib/python3.12/site-packages/flow/record/adapter/avro.py", line 72, in write
self.schema = descriptor_to_schema(self.desc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jeffbryner/development/test/.venv/lib/python3.12/site-packages/flow/record/adapter/avro.py", line 159, in descriptor_to_schema
raise ValueError(f"Unsupported Avro type: {field_type}")
ValueError: Unsupported Avro type: path
Other export formats work fine including jsonl, duckdb, etc
Activity
jeffbryner commentedon May 13, 2026
FWIW: Analysis by cline,
Overview
The failure occurs because the
avroadapter in theflow-recordlibrary (.venv/lib/python3.12/site-packages/flow/record/adapter/avro.py) lacks a mapping for thepathfield type to an Avro primitive type. Whenrdumptries to serialize the output of theusersquery, it encounters thehomefield, which is explicitly typed as apath, and throws an error.Unlike SQLite and DuckDB adapters that default unknown types to
TEXT, the Avro adapter performs a strict lookup. We can fix this by adding"path": "string"(and its system-specific variants) to theAVRO_TYPE_MAPdictionary directly within the virtual environment's package.Key Changes
.venv/lib/python3.12/site-packages/flow/record/adapter/avro.py: Update theAVRO_TYPE_MAPdictionary to mappath,posix_path, andwindows_pathto Avro's"string"type.Not sure this would work though, appears that it would happen with any unknown field type encountered?
yunzheng commentedon May 17, 2026
This is indeed due to a missed mapping in the AVRO adapter. Is this something you want to try and fix yourself? I'll be happy to review it.