Skip to content

Support Dataform projects in a subdirectory of a Git repository #2028

Open

Description

@BenJackGill

Summary

Please add first-class support for pointing a Dataform repository to a subfolder within a Git repo. Many teams use monorepos. We want apps/dataform/ to be the project root, while keeping other apps and infra in the same repo.

We have a monorepo such as:

my-monorepo/
├── .git/
├── apps/
│   ├── react/
│   │   ├── package.json
│   │   └── ...
│   └── server/
│       ├── package.json
│       └── ...
└── packages/
    └── ...

And we would like to store our dataform scripts in the same git repository to make it look like this:

my-monorepo/
├── .git/
├── apps/
│   ├── react/
│   │   ├── package.json
│   │   └── ...
│   ├── server/
│   │   ├── package.json
│   │   └── ...
│   └── dataform/
│       ├── definitions/
│       ├── package.json
│       └── workflow_settings.yaml
└── packages/
    └── ...

Why this matters

  • One repo, one PR, coordinated releases across app, workflows, and warehouse.
  • Easier onboarding and code search.
  • Aligns with common monorepo layouts and code owners.
  • Avoids extra repos, submodules, subtree mirrors, or giving up the managed UI to run the CLI.

Current workarounds and drawbacks

  • Separate repo: loses atomic PRs and discoverability.
  • Git submodule or subtree: adds Git complexity and have to handle two PRs.
  • CLI only from a subfolder: loses Managed Dataform UI, assertions view, lineage.

Proposal

Allow specifying a project path when connecting a repo and when invoking runs. For example:

// Repository settings
{
  "gitRemoteSettings": {
    "url": "https://github.com/org/my-monorepo.git",
    "defaultBranch": "main",
    "projectSubpath": "apps/dataform"
  }
}
# Workflow Invocation API body
gitCommitish: main
projectSubpath: apps/dataform

Behavior: Dataform treats apps/dataform as the project root, looks for workflow_settings.yaml and definitions/ under that path, and scopes indexing to that subtree. If projectSubpath is absent, default to repo root for full backward compatibility.

Related discussion

Stack Overflow user request describing the same need: https://stackoverflow.com/questions/78086702/how-to-setup-google-cloud-dataform-git-connection-in-a-sub-folder-to-use-it-in-a

This small addition unlocks monorepo parity without sacrificing the managed experience.

Activity

  1. josegalarza commented on Dec 15, 2025

    @josegalarza

    +1 to this feature request! 🙏

  2. tbryan2 commented on Jan 5, 2026

    @tbryan2

    +1 to this from me as well

  3. meirkhan-rakhmetzhanov commented on Feb 6, 2026

    @meirkhan-rakhmetzhanov

    +1 as well

  4. NicolasChagnet commented on Feb 19, 2026

    @NicolasChagnet

    +1 as well

  5. krisjan-oldekamp commented on Mar 19, 2026

    @krisjan-oldekamp

    +1!

  6. ondfisk commented on Jun 22, 2026

    @ondfisk

    +1

  7. andresmock commented on Jun 23, 2026

    @andresmock

    +1

  8. IvanRichter commented on Jul 7, 2026

    @IvanRichter

    Google closed the matching Issue Tracker item on June 29 and said this was fixed, but I can’t find any docs or any way to set a subdirectory as the project root in the API or console.

  9. kolina commented on Jul 11, 2026

    @kolina
    Contributor

    This functionality should be available in the new Git experience in BigQuery Studio https://docs.cloud.google.com/bigquery/docs/git-repositories: you can create a Git repository connected to third-party remote host and work with (or create) pipelines in a subdirectory for this Git repository.

    But we haven't yet released documentation and API for this functionality, so I've reopened this issue until it's done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions