Skip to content

fix(server): repair standalone storage contracts - #3447

Open
imbajin wants to merge 4 commits into
apache:masterfrom
hugegraph:fix/issue-103-verification
Open

imbajin wants to merge 4 commits into
apache:masterfrom
hugegraph:fix/issue-103-verification

Conversation

@imbajin

@imbajin imbajin commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Before → After

Before: standalone shard scans with an omitted bound can return HTTP 500, and a native profile unsupported by the selected community JNI can pass preparation but fail initialization.

After: validate missing scan bounds as HTTP 400 while preserving explicit empty bounds and paging contracts; validate community-compatible native profiles before runtime publication, include unzip in runtime images, and document standalone storage/configuration contracts.

Source: hugegraph#363
Paired documentation source: apache/hugegraph-doc#559

Validation boundary

The source PR records local Java/API/preparation and real standalone RocksDB/Topling checks; these are author-reported, not independently reproduced here. Published images, live HStore/Helm acceptance and Topling variant round trips remain unverified. Four existing standalone-management assertions failed in the broader suite and are not counted as passing. Coordinate the implementation and documentation merges.

Visual overview

单机 Server 分片扫描边界与 Topling 本地准备校验流程

Before → After: missing shard bounds return HTTP 400 while explicit empty bounds remain valid; Topling preparation adds actual-profile native import after the isolated EasyMigrate probe, before installing local runtime files. Profile import does not open production databases or convert existing SSTs, and it does not establish deployment or data-compatibility acceptance.

Reviewed: 2026-11-02. Source: 🔗 hugegraph/hugegraph #363 reviewed version.

Validate required shard scan bounds while retaining explicit empty bounds and paging behavior.
Use a community-compatible Topling profile and validate its plugins before runtime publication.
Include preparation tools and document standalone images, storage paths and Gremlin bindings.
Cover scan HTTP behavior and runtime preparation failures with regressions.
@codecov

codecov Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 58.42%. Comparing base (8e327b8) to head (a6d72a5).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##             master    #3447     +/-   ##
===========================================
  Coverage     58.42%   58.42%             
+ Complexity     6903     2709   -4194     
===========================================
  Files           779      779             
  Lines         69391    69395      +4     
  Branches       9567     9571      +4     
===========================================
+ Hits          40539    40545      +6     
+ Misses        25179    25176      -3     
- Partials       3673     3674      +1     
Flag Coverage Δ
commons 96.37% <ø> (ø)
hstore 36.09% <75.00%> (+<0.01%) ⬆️
pd 56.80% <ø> (+0.06%) ⬆️
server_hbase 42.55% <75.00%> (-0.04%) ⬇️
server_memory 56.58% <75.00%> (+0.05%) ⬆️
server_rocksdb 43.05% <100.00%> (+0.02%) ⬆️
store 35.93% <ø> (-0.03%) ⬇️

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Score: 9/10. Summary: Missing start or end on vertices/scan and edges/scan returns 400 while empty bounds and paging keep working, and the shipped toplingdb.yaml imports and opens with a community Topling JNI that rejects the previous CSPP profile.

orion added 2 commits November 2, 2026 18:14
Skip PD polling for local backends and pre-create writable RocksDB storage in the HStore image.
Document backend-specific shard ranges and enable missing-bound API tests on every backend.
Preserve the legacy CSPP profile and normalize preparation and launcher configuration paths.
Assert probe isolation and run Topling contracts for Server-only changes.
Read configured backend separators without depending on an environment override, and limit the local wait bypass to recognized backends.
Cover peer-only HStore startup with ready and unavailable Stores and reject incomplete template fixtures.
Resolve Topling paths physically with isolated CDPATH and add corresponding shell regressions.

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: Missing start or end on the shard scan APIs returns 400 and the new ShardScanApiTest cases run on rocksdb, memory and hbase, and the Topling profile import, path normalization and rocksdb-data ownership changes hold up. wait-storage.sh matches the backend name case-sensitively, while the Server accepts any case.

if ($0 !~ /^backend([ \t\f]*[:=]|[ \t\f]+|$)/) next
sub(/^backend[ \t\f]*([:=][ \t\f]*)?/, "")
sub(/[ \t\f]+$/, "")
backend = $0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: the Server lowercases backend (BackendProviderFactory, InitStore), but this match is case-sensitive, so backend=RocksDB with a retained pd.peers still polls PD until WAIT_STORAGE_TIMEOUT_S and exits 1. Lowercase the parsed value.

Suggested change
backend = $0
backend = tolower($0)

Match case-insensitive backend selection in BackendProviderFactory before deciding whether PD is required.
Cover configured and environment-overridden mixed-case local backends with retained PD peers and unavailable Stores.
Keep mixed-case HStore readiness checks and document the case-insensitive contract.

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Score: 9/10. Summary: wait-storage.sh lowercases the parsed backend, so mixed-case local backends with a retained pd.peers skip the PD wait, and the new cases in test-wait-storage.sh cover both the configured and environment-override paths. Shard scan bound validation, Topling profile import and rocksdb-data ownership are unchanged since the earlier review, and CI is green at this head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants