zstd: keep BuildDict recent-offsets positive and loadable

[zanarellidev]

Summary

BuildDict on a homogeneous corpus with default (zero) Offsets could emit a dictionary whose recent-offsets table contained 0. WithEncoderDict then rejected it with invalid offset in dictionary.

Root cause: repeat codes (offset 1/2/3) counted o.Offsets[i] even when those were still 0, so 0 ranked as a "most used" offset and was written into the dictionary.

Fix:

• skip zero targets when counting repeat-code hits
• require three positive in-range offsets (pad with 1,4,8 / sequential defaults when ranking alone is short)
• always validate the built blob with loadDict before returning

Fixes #1179

Test plan

• Fail-before / pass-after on TestBuildDictHomogeneousCorpusValidOffsets (issue repro corpus)
• go test ./zstd -run 'TestBuildDict(HomogeneousCorpusValidOffsets|LevelPathsRoundTrip)' -count=1
• go test ./zstd -count=1 (full package)

Summary by CodeRabbit

• Bug Fixes
  • Improved dictionary offset selection for more reliable compression.
  • Prevented invalid or missing offsets from being used.
  • Added validation to catch invalid dictionaries and return clear errors instead of causing crashes.
  • Improved handling of dictionaries trained on uniform HTML and JSON content.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c2c59a08-af67-4046-954d-39d4a9b15d3d

📥 Commits

Reviewing files that changed from the base of the PR and between 19daa67 and f485824.

📒 Files selected for processing (1)

• zstd/dict_test.go

🚧 Files skipped from review as they are similar to previous changes (1)

• zstd/dict_test.go

---

📝 Walkthrough

Walkthrough

BuildDict now filters and validates repeat offsets, supplies positive defaults, and returns errors when it cannot create three valid offsets. It always validates generated dictionaries. A regression test covers homogeneous randomized training data and encoder acceptance.

Changes

Dictionary validation

Layer / File(s)	Summary
Repeat-offset selection zstd/dict.go	Invalid and unset offsets are excluded. Usable offsets are ranked, checked against history, and supplemented with positive defaults.
Dictionary validation and regression coverage zstd/dict.go, zstd/dict_test.go	BuildDict returns validation errors instead of panicking. Tests verify valid offsets and encoder acceptance for homogeneous corpora.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• klauspost/compress#1147: Both PRs modify zstd/dict.go and related tests, but address different BuildDict concerns.
• klauspost/compress#1178: Both PRs improve BuildDict handling of pathological training corpora through different failure paths.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the fix for positive, loadable recent offsets in zstd dictionaries.
Linked Issues check	✅ Passed	The changes prevent non-positive offsets, validate generated dictionaries, return errors when needed, and add regression coverage for issue #1179.
Out of Scope Changes check	✅ Passed	All code and test changes directly support the linked issue and stated pull request objectives.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[klauspost]

Please apply go fix