Merge gitdb and smmap into the GitPython repository#2179
Draft
Byron wants to merge 706 commits into
Draft
Conversation
Tell PyPI long_description is Markdown
…pped-python-2-support Remove setup.cfg to have a pure python wheel
[skip CI]
[skip CI]
…ctions/actions/checkout-6 Bump actions/checkout from 5 to 6
…es/gitdb/ext/smmap-5ec977a Bump gitdb/ext/smmap from `801bd6f` to `5ec977a`
Co-authored-by: Sebastian Thiel <sebastian.thiel@icloud.com>
…-smmap Publish updated package metadata by bumping smmap to 5.0.3
Bumps [gitdb/ext/smmap](https://github.com/gitpython-developers/smmap) from `5ec977a` to `e4ad410`. - [Release notes](https://github.com/gitpython-developers/smmap/releases) - [Commits](gitpython-developers/smmap@5ec977a...e4ad410) --- updated-dependencies: - dependency-name: gitdb/ext/smmap dependency-version: e4ad410 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
…es/gitdb/ext/smmap-e4ad410 Bump gitdb/ext/smmap from `5ec977a` to `e4ad410`
Python 3.14 emits a DeprecationWarning for codecs.open(), which gitdb
hits inside ReferenceDB._update_dbs_from_ref_file:
DeprecationWarning: codecs.open() is deprecated. Use open() instead.
The built-in open() has supported the encoding kwarg since Python 3.0
and the call site already passes encoding="utf-8", so the replacement
is byte-for-byte equivalent on every supported Python version.
Dropped the now-unused codecs import.
Verified the change with the existing test_ref.py suite.
Closes #128
fix: replace deprecated codecs.open with built-in open (#128)
Closes #120 DecompressMemMapReader.read(N) could return b'' mid-stream when zlib consumed input without producing output on a single decompress call (small N, header / dictionary frames in flight). The original `if dcompdat and ...` guard at the recursion site skipped the "refill to size" recursion in that case, so a caller using the standard idiom while chunk := stream.read(4096): yield chunk terminated at the first empty chunk -- before _br == _s. The guard exists for compressed_bytes_read(), which manipulates _br=0 and then drains the inner zip past its EOF. Recursing there would loop forever because the inner zip is already done. The fix uses zlib's own `eof` attribute (available on standard zlib.Decompress objects since Python 3.6) to distinguish: - dcompdat empty AND zip not at EOF -> still digesting, recurse - dcompdat empty AND zip at EOF -> compressed_bytes_read scrub or genuine EOF; do not recurse. `getattr(_zip, 'eof', False)` keeps the conservative behavior when running against a custom zlib object that does not expose the attribute. Adds a regression test that reads with chunk_size in {1, 4, 16, 64} from a 13 KB highly-compressible stream. With the old guard, the chunk_size <= 16 cases stopped at byte 0; the new test asserts they read all 13000 bytes. The full existing test suite (24 tests) still passes, including test_decompress_reader_special_case and test_pack which exercise the compressed_bytes_read scrub path that the original guard existed to protect.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Fix DecompressMemMapReader.read returning b'' before EOF (#120)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v6...v7) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
…tions/actions/checkout-7 Bump actions/checkout from 6 to 7
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v6...v7) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
…ctions/actions/checkout-7 Bump actions/checkout from 6 to 7
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 6 to 7. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v6...v7) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
…ctions/actions/setup-python-7 Bump actions/setup-python from 6 to 7
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
GitPython previously depended on gitdb and smmap through nested git submodules. That made a complete checkout depend on extra repository state and forced related changes to be coordinated across three repositories. Keep both projects as independently buildable distributions under their own top-level directories, while updating GitPython packaging, CI, documentation, and test fixtures to consume the in-tree sources. Remove the obsolete gitlinks and imported repository automation so a normal clone contains all code needed to build and test the three packages.
The imported gitdb and smmap histories contain their own legacy formatting and configuration. Letting GitPython's root pre-commit configuration scan those trees produced unrelated failures and would turn the subtree merge into a wholesale reformat. Exclude the imported project directories at the root so existing GitPython checks retain their previous scope; each dependency continues to own its source style.
Running pytest below gitdb or smmap still discovers GitPython's root configuration. Its coverage addopts target the git package, which is not the package under test and caused the new dependency matrix jobs to fail. Clear inherited addopts for those jobs so each imported project runs its own test suite without GitPython-specific coverage arguments.
GitPython tests used the former submodules as clonable gitdb and smmap repositories. Reconstructing those fixtures from a shared clone of the merged history initially inherited GitPython's master branch and tag namespace, causing branch and tag creation collisions. Force the fixture branch and historical smmap tag to the imported source tips so the reconstructed repositories have the same refs the existing tests expect.
The imported histories exercise tree parsing with bytearray-backed buffers. On Python 3.14 those slices remain bytearrays, while GitPython's object constructors require immutable bytes, causing random-access tests to fail. Normalize the parsed filename and object ID slices to bytes and cover the bytearray input path with a regression test.
The historical gitdb and smmap test suites contain intentional legacy uses of tempfile.mktemp. Importing them made GitPython's CodeQL workflow report new security findings even though the code is test-only and unchanged from the standalone projects. Ignore only the imported test directories while continuing to analyze all three production packages.
Creating the historical smmap tag leaves a loose, read-only Git object in the reconstructed repository. Local clones may hardlink loose objects, and Python 3.7 on Windows cannot remove those files during temporary-directory cleanup. Run Git garbage collection after constructing each fixture so its generated refs and objects use normal packed repository storage before tests clone it.
Even after packing fixture objects, Git may hardlink repository metadata such as the commit-graph during a local clone. Those read-only hardlinks made the Unicode-path test fail only when Python 3.7 on Windows cleaned its temporary directory. Pass --no-hardlinks for this local fixture clone. The test remains focused on Unicode path handling and no longer depends on platform-specific hardlink cleanup behavior.
Byron
force-pushed
the
subtree-merge-deps
branch
from
July 21, 2026 07:28
90dda28 to
f377fc7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tasks
This section is for Byron only. Models continuing this PR must not add, remove, check, uncheck, rename, or reorder checkboxes here.
Everything below this line was generated by Codex GPT-5.
Created by Codex on behalf of Byron. Byron will review before this is ready to merge.
Reported issue
Use this information to plan for a subtree merge, combining gitdb and ssmap repositories into GitPython.
The requested ssmap repository is interpreted as smmap, matching the named GitPython dependency and the source repository discussed in the linked issues.
Summary
8ce61ad5cc4016bffaf25080bc0d69b3acbe8555intosmmap/.2da3232f9d58e7761e384ac6d32f7b1ed77a74a2intogitdb/.git,gitdb, andsmmapdistributions and import APIs.Context
This implements the subtree direction discussed in GitPython issues #511, #933, and #1716, gitdb issue #116, and smmap pull request #53.
Validation
git diff --check