Repository navigation
Conversation
logachev
requested review from
CurtHagenlocher,
birschick-bq and
davidhcoe
as code owners
October 27, 2026 22:08
davidhcoe
approved these changes
Nov 4, 2026
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
REPEATED BigQuery fields are incorrectly mapped to nullable scalars or structs instead of non-nullable lists, causing the returned schema to lose array structure.
4 open issues
What changed in this PR
This PR moves the C# BigQuery driver’s GetTableSchema from SQL queries to the REST API, enabling four-part Project.Catalog.Namespace.Table references without running a query job.
Changes:
- Build Arrow schemas from REST field definitions, including nested records and decimal precision and scale.
- Accept
catalog.namespacedataset identifiers while retaining single-segment project validation. - Add mock table registration and update schema, injection, and identifier tests.
| File | Description |
|---|---|
csharp/test/AdbcDrivers.BigQuery.Tests/SearchPatternTests.cs |
Tests two-part dataset identifier validation. |
csharp/test/AdbcDrivers.BigQuery.Tests/MockServer/MetadataInjectionTests.cs |
Updates assertions for REST requests rather than SQL queries. |
csharp/test/AdbcDrivers.BigQuery.Tests/MockServer/GetTableSchemaTests.cs |
Migrates schema fixtures to REST and adds PCNT coverage. |
csharp/test/AdbcDrivers.BigQuery.MockServer/BigQueryMockServer.cs |
Adds table registration for mock REST responses. |
csharp/src/AdbcDrivers.BigQuery/BigQueryConnection.cs |
Retrieves schemas through REST and converts fields to Arrow types. |
Review effort: Balanced
davidhcoe
self-requested a review
November 4, 2026 01:17
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Omitted-catalog schema lookups can target the billing project instead of the configured data project, returning empty or incorrect schemas, while the unauthorized-credentials test now encounters local identifier validation before authentication.
2 open issues
4 resolved since last review
Review effort: Balanced
| { | ||
| string query = string.Format("SELECT * FROM `{0}`.`{1}`.INFORMATION_SCHEMA.COLUMNS WHERE table_name = @tableName", | ||
| SanitizeIdentifier(catalog), SanitizeIdentifier(dbSchema)); | ||
| string projectId = string.IsNullOrEmpty(catalog) ? (this.ProjectId ?? string.Empty) : SanitizeCatalogIdentifier(catalog); |
| }); | ||
| Func<Task<BigQueryTable?>> func = () => Task.Run(() => | ||
| { | ||
| return Client?.GetTable(projectId, datasetId, tableName); |
- Prefer configured project_id (including bigquery-public-data) over billing_project_id when catalog is omitted in GetTableSchema, with mock tests. - Pass valid project, dataset, and table identifiers in AuthenticationTests.CanDetectUnauthorizedCredentials.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Summary
Replaces the
INFORMATION_SCHEMA.COLUMNSSQL query inBigQueryConnection.GetTableSchemawithClient.GetTablevia the BigQuery REST API. This enablesGetTableSchemato support 4-partProject.Catalog.Namespace.Table(PCNT) table references (such as BigLake Iceberg tables in namespaces), whereINFORMATION_SCHEMAviews are not supported, and avoids running a BigQuery SQL job to inspect table schemas.Changes
SELECT * FROM INFORMATION_SCHEMA.COLUMNSinGetTableSchemawithClient.GetTable(projectId, datasetId, tableName).Fielddefinitions directly fromTableFieldSchema(TableFieldToArrowField), including recursiveRECORD/STRUCTsubfields andNUMERIC/BIGNUMERICprecision and scale.catalog.namespacedataset identifiers inSanitizeIdentifierwhile enforcing single-segment validation for project/catalog identifiers inSanitizeCatalogIdentifier.AddTable(Table)helper toBigQueryMockServerand updateGetTableSchemaTests,MetadataInjectionTests, andSearchPatternTestswith REST API and PCNT coverage.Behavioral Changes from
INFORMATION_SCHEMAGetTableSchemanow calls thetables.getREST endpoint instead of submitting ajobs.insert/jobs.querySQL query againstINFORMATION_SCHEMA.COLUMNS.404 NotFoundfromtables.get),GetTableSchemareturns an empty ArrowSchema(matching the previous behavior whenWHERE table_name = @tableNamereturned 0 rows).DATA_TYPEMetadata: The field metadata entryDATA_TYPEnow reflects the BigQuery RESTTableFieldSchema.Typestring (e.g.INTEGER,FLOAT,BOOLEAN,RECORD) rather than the GoogleSQLINFORMATION_SCHEMA.COLUMNS.data_typestring (e.g.INT64,FLOAT64,BOOL,STRUCT<...>). The resulting ArrowDataTypeandXdbcDataTypemappings remain unchanged.STRUCT/RECORDFields: Nested struct fields are constructed recursively fromTableFieldSchema.Fieldsinstead of splitting theSTRUCT<...>SQL type string on spaces and commas, which also properly supports deeply nested structs.NUMERIC/BIGNUMERICPrecision and Scale: Precision and scale are read fromTableFieldSchema.PrecisionandTableFieldSchema.Scale(defaulting to(38, 9)forNUMERICand(76, 38)forBIGNUMERICwhen unparameterized).