Repository navigation
Warehouse path is always required for Glue catalog, even when there isn't one #941
Description
The warehouse config prop for Glue catalog is required, but it isn't always known or required. The Glue catalog ID is enough in order to resolve metadata locations.
Moreover - I don't think this path is used for anything but protocol (FileIO type) inference? I was able to work around this issue by passing the correct Glue catalog ID (which is the AWS account ID) and s3:// as the warehouse, with no issue.
In my opinion warehouse should be replaced with something like FileIO type and only the Glue catalog ID should be required, but for backwards compatibility not passing warehouse should at least be allowed.
Activity
CTTY commentedon May 9, 2025
I think this makes sense and making the warehouse path optional matches the iceberg-java behavior. We should probably fix this while implementing the new catalog builder: #1259
chasehaddleton commentedon Jun 10, 2025
I believe there is an entire class of bugs caused by the current behavior. For example, if you pass
s3as the warehouse, but the catalog contains tables with boths3ands3afiles, then it erroneously fails saying that protocols do not match (even though it is immaterial). Moving to aFileIOTypecould solve this, assuming it's paired with a set of "valid protocols for a type".lliangyu-lin commentedon Aug 5, 2025
Another issue is that when loading the table from glue, the table is read using the catalog file io. So if catalog file io is set to S3, but the table loaded is stored in another storage, it will continue to use the S3 file io to read the metadata. I think the file io should be inferred during create table and load table instead of initialized with the catalog...
Reference in python
github-actions commentedon Feb 2, 2026
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
blackmwk commentedon Feb 2, 2026
Don't close this, it's a but.
David-Banquet commentedon Oct 5, 2026
A status check on
main, since the code has moved since this was opened.The original concern is mostly gone: the Glue catalog no longer infers its FileIO from
warehouse, it builds it from the storage factory and the catalog properties. The only remaining use ofwarehouseis the default table location increate_table, and only when the Glue database has noLocationUri(get_default_table_locationinutils.rs).What still differs from Java is when the check happens.
GlueCatalogBuilder::loadfails with "Catalog warehouse is required" before any call to Glue. Java'sGlueCatalogaccepts a missing warehouse and only throws indefaultWarehouseLocation, when a table is created without a location in a database that has noLocationUri. So a read-only client, or one that always creates tables in databases with a location, works in Java without a warehouse and is refused in Rust.@blackmwk I see #3102 keeps the fail-fast validation on purpose. Would you take a follow-up that moves the check to
create_table, matching Java? If so I can open it once #3102 is merged.David-Banquet commentedon Oct 12, 2026
@blackmwk #3102 was closed by the stale bot last night. Do you plan to reopen it? If not, I can open the follow-up for this issue against main now: keep the
warehouseproperty optional inGlueCatalogBuilder::loadand only fail increate_tablewhen the database has noLocationUri, as Java does. It is a small change and I can rebase it on #3102 if that comes back first.blackmwk commentedon Oct 13, 2026
Hi, @David-Banquet Yes, I'll reopen it.
David-Banquet commentedon Oct 14, 2026
Thanks. I'll wait for #3102 to be back and open the follow-up on top of it: keep
warehouseoptional inGlueCatalogBuilder::loadand only fail increate_tablewhen the database has noLocationUri. If you'd rather fold that into #3102 directly, that works for me too.