Skip to content

Data Api Builder support for multiple databases. - #1709

Merged
rohkhann merged 57 commits into
mainfrom
rohkhann/MultipleSourceSupport
Sep 28, 2023
Merged

Data Api Builder support for multiple databases.#1709
rohkhann merged 57 commits into
mainfrom
rohkhann/MultipleSourceSupport

Conversation

@rohkhann

@rohkhann rohkhann commented Sep 11, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Why make this change?

This change brings the functionality of multiple datasource support to core engine of DAB. The multi source support is for graphql only and this pr does not address support for REST.

RFC: #1638

What is this change?

  1. changes to DI - introducting factory classes where applicable as for multiple db's we have multiple instances of various class objects
  2. Document generation handling for multiple dbs.
  3. Routing Queries/Mutations to appropriate db using db name pulled from entityName.

How was this tested?

UnitTests:

  1. Added runtimeConfig loader tests for multi-datasource loading. Both positive and negative tests.
  2. Deserialization tests for Data-Source Files converter.
  3. Query and mutation builder tests
  4. MultiSource query execution unit test.
    Integration test:
  5. Single db integration test done.
  6. multiple db integration test:
image

In the above books belongs to one mssql db and authors belongs to another mssql db.
6. Integration test also ran with 3 databases.

rohkhann and others added 23 commits September 5, 2023 22:44
@rohkhann
First commit.
62c5d03
@rohkhann
Refining logic for cases where runtimeConfig is not available in star…
1de9367 
…tup.
@rohkhann
Merge branch 'main' into rohkhann/MultiDataSourceSupport
ccf76d5
@rohkhann
dotnet format.
b565895
@rohkhann
Merge branch 'rohkhann/MultiDataSourceSupport' of https://github.com/…
e130e42 
…Azure/data-api-builder into rohkhann/MultiDataSourceSupport
@rohkhann
GqlFilterParser test fix.
ddd40d5
@rohkhann
Adding support on runtimeconfig to take in multiple datasource files …
6722dcc 
…as input.
@rohkhann
Fixing schemaCreator to handle multiple db scenario;
f2cd43b
@rohkhann
dotnet format.
5b5d808
@rohkhann
Fixing test cases for factory registration instead of instance regist…
3f69af2 
…ration.
@rohkhann
fixing test cases.
6f413ef
@rohkhann
Dotnet format.
82c227f
@rohkhann
cosmos TryAdd
4d05318
@rohkhann
DataSourceFiles converter to ensure users can not pass in datasourcef…
fefc618 
…iles.
@rohkhann
Connectionstring parsing text fix.
aa42001
@rohkhann
Adding unit tests for multi-db scenario.
aa73c59
@rohkhann
dotnet format.
2049ec2
@rohkhann
Adding QueryBuilderTests.
76319c4
@rohkhann
Adding debugging to mutationBuilder
33eaa5d
@rohkhann
Test case fix.
3d2ec4e
@rohkhann
Fixing schemaCreator dbobjects logic.
bfa5ef9
@rohkhann
Stored procedure call fix.
0ac1d45
@rohkhann
Multiple source commit.
4fa265f
seantleonard
Sean Leonard (seantleonard) reviewed Sep 25, 2023
View reviewed changes
Comment thread src/Service.Tests/Authorization/AuthorizationHelpers.cs Outdated
seantleonard
Sean Leonard (seantleonard) reviewed Sep 25, 2023
View reviewed changes
Comment thread src/Service.Tests/ModuleInitializer.cs
seantleonard
Sean Leonard (seantleonard) reviewed Sep 26, 2023
View reviewed changes
Comment thread src/Service.Tests/Unittests/MultiSourceQueryExecutionUnitTests.cs Outdated
Comment thread src/Service.Tests/Unittests/MultiSourceQueryExecutionUnitTests.cs Outdated
Comment thread src/Service.Tests/Unittests/MultiSourceQueryExecutionUnitTests.cs
Comment thread src/Service.Tests/Unittests/MultiSourceQueryExecutionUnitTests.cs
Aniruddh25
Aniruddh Munde (Aniruddh25) reviewed Sep 27, 2023
View reviewed changes

@Aniruddh25 Aniruddh Munde (Aniruddh25) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just few more comments, approving with the assumption they'll be resolved :)

Comment thread src/Cli.Tests/ModuleInitializer.cs Outdated
Comment thread src/Config/ObjectModel/RuntimeConfig.cs
Comment thread src/Core/Configurations/RuntimeConfigProvider.cs
Comment thread src/Core/Resolvers/SqlQueryEngine.cs
Comment thread src/Core/Services/OpenAPI/OpenApiDocumentor.cs
Comment thread src/Service.Tests/MultiSourceTestSchema.gql
Comment thread src/Config/ObjectModel/RuntimeConfig.cs
Aniruddh25
Aniruddh Munde (Aniruddh25) approved these changes Sep 27, 2023
View reviewed changes

@Aniruddh25 Aniruddh Munde (Aniruddh25) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just few more comments, approving with the assumption they'll be resolved :)

@rohkhann rohkhann mentioned this pull request Sep 27, 2023
Closed
seantleonard
Sean Leonard (seantleonard) reviewed Sep 27, 2023
View reviewed changes

@seantleonard Sean Leonard (seantleonard) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to go back an look at resolved comments for context. But this review covers both engine/cli/tests components.

Comment thread src/Core/Configurations/RuntimeConfigProvider.cs Outdated
Comment thread src/Core/Services/RestService.cs Outdated
Comment thread src/Service.GraphQLBuilder/GraphQLUtils.cs
Comment thread src/Service.Tests/Configuration/ConfigurationTests.cs Outdated
Comment thread src/Service.Tests/Configuration/RuntimeConfigLoaderTests.cs Outdated
Comment thread src/Service.Tests/Unittests/MultiSourceQueryExecutionUnitTests.cs Outdated
Comment thread src/Service.Tests/Multidab-config.CosmosDb_NoSql.json
Comment thread src/Service.Tests/Multidab-config.CosmosDb_NoSql.json
Comment thread src/Service.Tests/Multidab-config.CosmosDb_NoSql.json
Comment thread src/Cli.Tests/ModuleInitializer.cs Outdated
kliszaq
Kamil Kliczbor (kliszaq) reviewed Sep 28, 2023
View reviewed changes
Comment thread src/Config/ObjectModel/RuntimeConfig.cs
seantleonard
Sean Leonard (seantleonard) reviewed Sep 28, 2023
View reviewed changes

@seantleonard Sean Leonard (seantleonard) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pending one last suggestion and a comment from a OSS community member.

Comment thread schemas/dab.draft.schema.json Outdated
@rohkhann @seantleonard
Update schemas/dab.draft.schema.json
aab0b14 
Co-authored-by: Sean Leonard <sean.leonard@microsoft.com>
seantleonard
Sean Leonard (seantleonard) approved these changes Sep 28, 2023
View reviewed changes

@seantleonard Sean Leonard (seantleonard) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for all of your efforts addressing review feedback after designing and implementing this solution!

@rohkhann
rohkhann merged commit 3176820 into main Sep 28, 2023
@rohkhann
rohkhann deleted the rohkhann/MultipleSourceSupport branch September 28, 2023 17:29
@Aniruddh25 Aniruddh Munde (Aniruddh25) mentioned this pull request Sep 28, 2023
Closed
@aaronburtle aaronburtle mentioned this pull request Mar 19, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Aniruddh25 Aniruddh Munde (Aniruddh25) Aniruddh25 approved these changes
@seantleonard Sean Leonard (seantleonard) seantleonard approved these changes
@mbhaskar Bhaskar Mallapragada (mbhaskar) Awaiting requested review from mbhaskar
@JelteF Jelte Fennema-Nio (JelteF) Awaiting requested review from JelteF
@Mathos1432 Mathieu Tremblay (Mathos1432) Awaiting requested review from Mathos1432
@gledis69 Gledis Zeneli (gledis69) Awaiting requested review from gledis69
@milismsft Milis (milismsft) Awaiting requested review from milismsft
@tarazou9 tarazou9 Awaiting requested review from tarazou9
@abhishekkumams Abhishek Kumar (abhishekkumams) Awaiting requested review from abhishekkumams
@aaronpowell Aaron Powell (aaronpowell) Awaiting requested review from aaronpowell
@severussundar Shyam Sundar J (severussundar) Awaiting requested review from severussundar
@ravishetye Ravi Shetye (ravishetye) Awaiting requested review from ravishetye
@vadeveka vadeveka Awaiting requested review from vadeveka vadeveka is a code owner
@ayush3797 Ayush Agarwal (ayush3797) Awaiting requested review from ayush3797
@aaronburtle aaronburtle Awaiting requested review from aaronburtle aaronburtle is a code owner
+1 more reviewer
@kliszaq Kamil Kliczbor (kliszaq) kliszaq left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@rohkhann rohkhann

Labels

None yet

Projects

Data API builder
Status: Done

Milestone

0.10

Development

Successfully merging this pull request may close these issues.

7 participants

@rohkhann @ayush3797 @Aniruddh25 @kliszaq @seantleonard @vadeveka @aaronburtle