Skip to content

feat(python-sdk): SDK metadata + CDN-indexed stubs for Go and Python classes - #661

Merged
shivasurya merged 2 commits into
mainfrom
shiva/sdk-metadata-cdn-indexer
Apr 19, 2026
Merged

shivasurya merged 2 commits into
mainfrom
shiva/sdk-metadata-cdn-indexer

Conversation

@shivasurya

@shivasurya shivasurya commented Apr 19, 2026 •

Copy link
Copy Markdown
Owner

Summary

Adds the handcrafted SDK metadata and CDN indexers that drive the new /sdk/ documentation on codepathfinder.dev.

  • codepathfinder/go_rule_meta.py — 72 Go classes, 221 methods (Gin, Echo, Fiber, Chi, Gorilla Mux, GORM, sqlx, pgx, Mongo, Redis, JWT, Viper, YAML, gRPC, Resty + stdlib).
  • codepathfinder/python_rule_meta.py — 100 Python class entries, 302 methods across 9 categories (web frameworks, command exec, databases, deserialization, HTTP clients, filesystem, archives, crypto, templating).
  • scripts/generate_sdk_manifest.py — Reads each *_rule_meta.py and emits sdk-manifest.json consumed by the website. Chains the two indexers below automatically; --skip-cdn-index opts out.
  • scripts/index_python_from_cdn.py — Fills stubs for every Python module in the assets.codepathfinder.dev CDN registries that isn't already covered by hand.
  • scripts/index_go_from_cdn.py — Same pattern for Go stdlib and third-party CDN entries.

Coverage in the published manifest

Language Handcrafted CDN stubs Total classes Total methods
Go 72 132 204 1,132
Python 100 329 429 2,781

100% of 167 Go stdlib + 25 Go third-party + 211 Python stdlib + 208 Python third-party modules from the CDN are indexed. Stubs carry role: neutral with a clear banner so readers don't mistake auto-indexed metadata for verified security annotations.

Lint status

Matches the canonical lintPython task in sast-engine/build.gradle:

  • black --check codepathfinder/go_rule_meta.py codepathfinder/python_rule_meta.py scripts/ — all files left unchanged
  • ruff check codepathfinder/ scripts/ — all checks passed
  • mypy on the 5 new/modified files — no issues found

Pre-existing formatting drift in python_ir.py, go_rule.py, and tests/test_go_thirdparty_querytypes.py is out of scope and left untouched.

Test plan

  • cd python-sdk && python3 scripts/generate_sdk_manifest.py runs end-to-end and regenerates sdk-manifest.json without raising (ignored output path requires cpf-website checkout adjacent).
  • cd python-sdk && black --check codepathfinder/go_rule_meta.py codepathfinder/python_rule_meta.py scripts/ passes.
  • cd python-sdk && ruff check codepathfinder/go_rule_meta.py codepathfinder/python_rule_meta.py scripts/ passes.
  • cd python-sdk && mypy codepathfinder/go_rule_meta.py codepathfinder/python_rule_meta.py scripts/generate_sdk_manifest.py scripts/index_go_from_cdn.py scripts/index_python_from_cdn.py passes.
  • Confirm generated Go manifest totals: 204 classes / 1,132 methods.
  • Confirm generated Python manifest totals: 429 classes / 2,781 methods.

🤖 Generated with Claude Code

shivasurya and others added 2 commits April 18, 2026 23:21
Adds handcrafted SDK_META dicts that drive sdk-manifest.json generation:
- go_rule_meta.py: 72 Go classes, 221 methods covering handcrafted Go
  QueryType companions (stdlib + 3rd party: gin, echo, fiber, chi, gorilla
  mux, gorm, sqlx, pgx, mongo, redis, jwt, viper, yaml.v3, grpc, resty).
- python_rule_meta.py: 100 Python class entries, 302 methods across 9
  categories (web-frameworks, command-execution, databases, deserialization,
  http-clients, file-system, archives, crypto, templating).

Generator + CDN indexers:
- generate_sdk_manifest.py: reads *_rule_meta.py and emits sdk-manifest.json
  consumed by codepathfinder.dev. Now chains the two indexers below after
  handcrafted meta is written (--skip-cdn-index opts out).
- index_python_from_cdn.py: fills stubs for every module in
  assets.codepathfinder.dev/registries/python3.11/stdlib/v1 and
  assets.codepathfinder.dev/registries/thirdparty/v1 that isn't already
  covered by python_rule_meta.py. Auto-categorizes into 10 buckets.
- index_go_from_cdn.py: same pattern for
  assets.codepathfinder.dev/registries/go1.21/stdlib/v1 and
  assets.codepathfinder.dev/registries/go-thirdparty/v1.

Resulting coverage in sdk-manifest.json:
- Go: 204 classes / 1132 methods (100% of 167 stdlib + 25 third-party CDN).
- Python: 429 classes / 2781 methods (100% of 211 stdlib + 208 third-party).

Stubs carry role=neutral with a disclaimer in the description so they are
discoverable without making unverified security claims.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…exers

- Apply black formatting (line-length 88 from pyproject.toml) across
  go_rule_meta.py, python_rule_meta.py, and the three scripts.
- Remove unused `inspect` import flagged by ruff in generate_sdk_manifest.py.
- Fix mypy no-any-return on fetch_json() in both CDN indexers by binding
  json.loads(...) to an annotated local before returning.

Matches the canonical lintPython task in sast-engine/build.gradle:
  black --check codepathfinder/ tests/ && ruff check codepathfinder/ tests/ && mypy codepathfinder/

All five files added in the previous commit now pass black, ruff, and mypy
with no warnings. Pre-existing formatting drift in python_ir.py, go_rule.py,
and tests/test_go_thirdparty_querytypes.py is out of scope for this branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@shivasurya shivasurya added documentation Improvements or additions to documentation enhancement New feature or request python labels Apr 19, 2026
@shivasurya shivasurya self-assigned this Apr 19, 2026
@shivasurya shivasurya added documentation Improvements or additions to documentation enhancement New feature or request python labels Apr 19, 2026
@safedep

safedep Bot commented Apr 19, 2026 •

Copy link
Copy Markdown

SafeDep Report Summary

Green Malicious Packages Badge Green Vulnerable Packages Badge Green Risky License Badge

No dependency changes detected. Nothing to scan.

View complete scan results →

This report is generated by SafeDep Github App

@github-actions

Copy link
Copy Markdown

Code Pathfinder Security Scan

Pass Critical High Medium Low Info

No security issues detected.

Metric Value
Files Scanned 5
Rules 0

Powered by Code Pathfinder

@shivasurya
shivasurya merged commit 143ffdf into main Apr 19, 2026
6 checks passed
@shivasurya
shivasurya deleted the shiva/sdk-metadata-cdn-indexer branch April 19, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant