Skip to content

[spec-enforcer] Enforce specifications for cli/data, workflow/actions, workflow/actionslanguageservice - #77137

Merged
pelikhan merged 1 commit into
mainfrom
spec-enforcer-actions-43208768309-8d40ce13529033ab
Nov 12, 2026
Merged

pelikhan merged 1 commit into
mainfrom
spec-enforcer-actions-43208768309-8d40ce13529033ab

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

Specification Test Enforcement

Three README-documented packages lacked specification tests. This PR adds 21 external-package contract tests covering the bundled provider catalogue, native workflow resolution and source preservation, and Actions expression parsing.

Package Tests Added Tests Passing Spec Sections Covered
cli/data 1 1 Public API
workflow/actions 9 9 Overview, Public API
workflow/actionslanguageservice 11 11 Overview, Public API

Test Derivation

All behavioral assertions derive from package README.md specifications. Implementation inspection was limited to package declarations, signatures, and type declarations. Tests use exported APIs, the TestSpec_ prefix, and the !integration build tag.

Validation

All 21 specification tests pass. Package builds, impacted unit tests, schema freshness, and Go lint passed. Experimental Cobertura coverage was accepted for upload with label spec-enforcer/generated-tests: generated tests cover 43.7% of statements in workflow/actions and 56.2% in workflow/actionslanguageservice; the embedded data package has no executable statements.

This PR is a draft because repository-wide workflow validation remains blocked: 357 of 365 workflows compiled, while eight unchanged workflows failed npm runtime-package validation with DEPTH_ZERO_SELF_SIGNED_CERT. The blocker was recorded separately.

Validation details and affected workflows
  • Ran make fmt and package-specific go build and go test -v -run TestSpec checks.
  • Ran make agent-report-progress once. Its build, schema, and impacted-test checks passed; Go lint was completed separately after installing the pinned linter.
  • Go lint reports zero issues. Custom lint reports 41 existing diagnostics outside the three changed test files, with no diagnostics in the changed files.
  • make check-workflow-drift completed compilation but could not establish drift status because runtime-package validation failed in daily-graft-intelligence.md, daily-reliability-review.md, daily-token-consumption-report.md, deep-report.md, go-pattern-detector.md, mcp-inspector.md, portfolio-analyst.md, and smoke-otel-backends.md.
  • The workflow check restored generated files. The commit contains only the three specification test files.

Spec-Implementation Mismatches

No implementation mismatch was confirmed for the tested contracts. SPEC_AMBIGUITY comments identify limits in the README wording, including workflow ID scope, typed-step interpolation restrictions, AST traversal granularity, expression-boundary measurement, identifier and cache-mode rules, and pattern flag/filter semantics. Pattern behavior is deferred pending a precise specification.

Round-Robin State

  • Run mode: round-robin
  • Packages processed this run: cli/data, workflow/actions, workflow/actionslanguageservice
  • Next packages in rotation: workflow/aw, workflow/cachepolicy, workflow/checkoutpolicy
  • Total eligible packages: 51 (with README.md)

Rotation advanced to index 39. Rotation JSON, enforcement records, and merged README hashes were saved and read back successfully, preserving unprocessed hashes. Nested package paths use their canonical slash-separated names; the existing PR-text recovery regex will need slash support to recover these names if cache state is lost.

The draft remains pending successful repository-wide workflow validation.

References: §43208768309

Generated by 📋 Package Specification Enforcer · codex · routed: gpt61 high · ~142 AIC · ⌖ ~2 AIC · ⊞ 459 · ◷

  • expires on Nov 15, 2026, 3:50 AM UTC-08:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The change still has two active moderate test-coverage issues: the cache-mode validator is not tested with explicit valid or invalid modes, and identifier validation lacks any invalid-input cases.

2 open issues
What changed in this PR

Adds external-package specification tests for three README-documented Go packages, covering native workflow handling, Actions expression parsing, and the bundled model catalogue.

Changes:

  • Added a bundled provider-catalogue JSON contract test for cli/data.
  • Added source-preservation, resolution, lowering, and validation tests for workflow/actions.
  • Added expression parsing, reference handling, interpolation, workflow parsing, and validation tests for workflow/actionslanguageservice.
File Description
pkg/​workflow/​actionslanguageservice/​spec_test.go Adds contract tests for Actions expression parsing, tokenization, references, interpolation, workflow parsing, identifiers, and cache-mode validation.
pkg/​workflow/​actions/​spec_test.go Adds contract tests for native workflow parsing, resolution, lowering, validation, source preservation, and editing.
pkg/​cli/​data/​spec_test.go Adds a contract test for the embedded provider catalogue JSON exposed by cli/data.

Review effort: Lite (auto)

Comment on lines +200 to +203
func TestSpec_PublicAPI_ActionsIdentifier(t *testing.T) {
for _, name := range []string{"build", "test1"} {
assert.True(t, actionslanguageservice.ActionsIdentifier(name, false), "ordinary Actions identifier %q must validate", name)
}
Comment on lines +209 to +212
func TestSpec_PublicAPI_ValidateWorkflowWithCacheMode(t *testing.T) {
source := "on: push\njobs:\n build:\n runs-on: ubuntu-latest\n steps:\n - run: echo ready\n"
err := actionslanguageservice.ValidateWorkflowWithCacheMode("spec.yml", source)
require.NoError(t, err, "a native workflow with no explicit cache policy must validate")
@pelikhan
pelikhan merged commit ccb6ce2 into main Nov 12, 2026
3 checks passed
@pelikhan
pelikhan deleted the spec-enforcer-actions-43208768309-8d40ce13529033ab branch November 12, 2026 13:03
@github-actions

Copy link
Copy Markdown
Contributor Author

🎉 This pull request is included in a new release.

Release: v1.2.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants