Repository navigation
Arrow Flight SQL / ADBC datasource #7731
Description
Is your feature request related to a problem or challenge?
I want to fetch data from multiple remote databases.
Describe the solution you'd like
Since DataFusion is part of the Arrow ecosystem it seems to make sense to have an Arrow Flight SQL datasource.
Describe alternatives you've considered
splitgraph/seafowl's datafusion_remote_tables. However, it seems sensible to have a basic Arrow Flight SQL datasource out of the box.
Additional context
No response
Activity
backkem commentedon Oct 4, 2023
Maybe adding an ADBC backed TableProviderFactory would be a good approach. That way the actual drivers can live under the apache/arrow-adbc project.
[-]Arrow Flight SQL datasource[/-][+]Arrow Flight SQL / ADBC datasource[/+]alamb commentedon Oct 4, 2023
I think having an (optional) FlightSQL backed remote table sounds like a good idea to me.
backkem commentedon Oct 5, 2023
Yes, I don't think it needs to be enabled by default.
I was looking into the adbc direction but the rust port seems less fleshed out than the C and Go ones so far. I did find this somewhat stale PR: apache/arrow-adbc#478
alamb commentedon Aug 27, 2024
See discussion on #11938 -- we are going to try and put such table providers in other repos rather than the core datafusion one