Skip to content

[api][runtime][java][python] Support cross-language skill repositories - #1214

Merged
wenjin272 merged 1 commit into
apache:mainfrom
wenjin272:codex/1194-cross-language-skills
Oct 9, 2026
Merged

wenjin272 merged 1 commit into
apache:mainfrom
wenjin272:codex/1194-cross-language-skills

Conversation

@wenjin272

Copy link
Copy Markdown
Contributor

Linked issue: Fixes #1194

Purpose of change

Python agents can load Skills from job JARs through Skills.from_classpath(...), and Java agents can load Skills from Python packages through Skills.fromPackage(...). Both combinations also work through YAML, completing source support independently of the language declaring the agent.

Runtime flow

  • Python classpath sources call JavaResourceAdapter.extractClasspathSkills with the operator's Flink user-code classloader. Java resolves the resource and copies/extracts it into the Python repository's temporary directory.
  • Java package sources cause PythonBridgeManager to initialize Python even for an otherwise Java-only agent. The resource context obtains the initialized interpreter when creating SkillManager; PackageSkillRepository invokes the Python package materializer into a Java-owned temporary directory.
  • The consuming repository discovers Skills and reads their bodies and attachments locally. Repository close releases owned temporary files, including through operator shutdown. Original JARs and package files remain intact.

Key decisions

Each SkillManager owns its source-handler map, so a handler can bind its operator's interpreter or bridge without storing runtime instances globally. Source resolution is shared with native repositories through internal materializers. Classpath fallback opens archive files without requiring a .jar suffix, covering Flink BLOB cache names and JARs without explicit directory entries.

Behavioral Semantics

Interaction decisions

Agent language Classpath source Python package source
Java Resolve with the job classloader Initialize/reuse Python; copy resources through the interpreter
Python Resolve through the operator's Java bridge Resolve through Python package resources

Cross-language use requires the Flink runtime and the corresponding dependencies in the job environment. Declaring a source does not install its JAR or package. Missing runtime bridges fail loading rather than selecting another source.

Behavioral contracts

  1. API and YAML declarations support both cross-language directions, including SKILL.md and accompanying files.
  2. Managers using the same resource names in different operator environments resolve through their own bridges/classloaders.
  3. Closing repositories, or failing during materialization/partial loading, releases owned temporary directories. Existing duplicate-Skill precedence is preserved, and displaced repositories are still closed.
  4. Missing sources or bridges fail initialization with source context. load_skill returns an error response on initialization failure in both languages, logs the exception chain, and does not misreport repeated failures as an empty configuration. URL origins in responses are redacted.
  5. Existing native Java classpath and Python package sources continue to work, including archives and zip-imported Python packages.

Failure behavior

Repository initialization errors propagate to direct manager callers; partial results are cleaned up. The manager is marked initialized only after successful creation, allowing a later call to attempt initialization again. The built-in load_skill catches initialization exceptions and returns ToolResponse.error; no configured manager, unknown Skills, and missing attachments retain their error-response behavior. Repository cleanup failures retain the existing propagation behavior.

Tests

Contract Coverage
1: Packaged sources through API/YAML PackageSkillsCrossLanguageTest installs a generated wheel; skills_cross_language_test.py submits a resource JAR. Both verify body and attachment reads. Declaration serialization tests cover the added factories.
2: Operator isolation and job classloader CrossLanguageSkillSourceTest, test_classpath_repository.py; includes reopening one manager's source after another manager is created and an extensionless JAR through a parent loader.
3: Ownership and cleanup Cross-language E2Es, package/classpath repository tests, SkillManagerTest, test_manager.py, and ResourceCacheTest.
4: Failure responses and diagnostics LoadSkillToolTest, test_load_skill.py, and missing-source/bridge tests; repeated calls and URL redaction are asserted.
5: Native source compatibility Existing classpath/package repository suites plus zip-imported package tests.

Validation: Java clean reactor build with focused API/runtime Skills tests and two Java E2Es passed (170 tests); the final Java tool-response regressions passed (12 tests). Python Skills/API/YAML/cache suites passed (312 tests), as did two packaged-JAR E2Es and the final tool/classpath regressions (17 tests). Spotless, Ruff, and git diff --check passed. Distribution JARs were clean-built and the Python wheel rebuilt for cross-language verification.

Validated with Java 11, Python 3.11, and Flink 2.3. Other supported version combinations, a remote multi-TaskManager deployment, and injected operator failover were not exercised. These runs are focused validation, not the full project test suite.

API

Adds Java Skills.fromPackage(String, String) and Python Skills.from_classpath(*resources); existing native factories remain available. YAML keeps the existing package/classpath shape and gains the corresponding runtime paths. Java/Python runtime-global source registries are removed in favor of instance-owned handlers; these are internal runtime interfaces. Python load_skill now returns an error response for initialization failures that previously escaped as exceptions; Java preserves its response-based contract with more accurate diagnostics.

Documentation

  • doc-needed
  • doc-not-needed
  • doc-included

Updates Skills usage/runtime requirements and refreshes generated YAML schemas and the bundled schema contract.

Was this patch authored or co-authored using generative AI tooling?

  • Yes
  • No

Generated-by: Codex 0.153.4 (GPT-6)

Allow Python agents to load classpath skills and Java agents to load
Python package skills through operator-scoped runtime bridges. Share
source materialization and preserve temporary resource ownership and
cleanup across both language paths.

Handle distributed JARs without filename extensions, align API and YAML
support, and cover packaged sources with cross-language end-to-end tests.

Fixes apache#1194

Generated-by: Codex 0.153.4 (GPT-6)
Co-authored-by: Codex <codex@openai.com>
@github-actions github-actions Bot added doc-included Your PR already contains the necessary documentation updates. fixVersion/0.4.0 priority/major Default priority of the PR or issue. labels Oct 9, 2026
@wenjin272
wenjin272 merged commit d6e2ecc into apache:main Oct 9, 2026
56 of 57 checks passed
weiqingy added a commit to weiqingy/flink-agents that referenced this pull request Oct 11, 2026
Brings in apache#1214, apache#1208 and apache#1138. ChatModelInvoker's two new
cancellation checks move to CancellationUtils, since main removed
ModelRoutingResolver.isCancellation. ChatModelInvokerTest keeps this
branch's tests plus main's provider cancellation test, and
test_flink_runner_context_reconcilable.py takes both sides' imports.

Generated-by: GitHub Copilot CLI 1.0.95 (Claude Opus 5.5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-included Your PR already contains the necessary documentation updates. fixVersion/0.4.0 priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Tech Debt][API][Skills] Support classpath repositories from Python and package repositories from Java

1 participant