Skip to content

test: close Astra coverage gaps and remove the duplicate quota check - #806

Merged
thomasluizon merged 4 commits into
redesign/mainfrom
thomasluizon/ticket-264-astra-test-gaps
Oct 19, 2026
Merged

thomasluizon merged 4 commits into
redesign/mainfrom
thomasluizon/ticket-264-astra-test-gaps

Conversation

@thomasluizon

@thomasluizon thomasluizon commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

An authenticated MCP POST now has regression coverage through UseMcpSelectiveAuth: exhausting either the general or AI budget returns 429 before the endpoint delegate runs, while a second API key for the same user still reaches the endpoint. The direct AI-tool theory also covers get_retrospective.

CreateGoalToolTests.cs now exercises the production constructor with real habit entities at the cap and one above it. Rejection preserves the existing error result and writes neither a goal nor links; the accepted boundary links every habit.

The Astra block-answer pre-check duplicated the atomic reservation. Remove that pre-check and the obsolete allowance interface/implementation, migrate dependent fixtures and feature-explanation quota checks to TryConsumeAiMessage, and preserve smoke-account bypass coverage for both Free and Pro users. TryConsumeAiMessage itself is unchanged. Its existing transactional rollback test still proves that quota rejection leaves the choice unclaimed.

Closes thomasluizon/orbit-tickets#264 (ORB-258). Includes the folded work from thomasluizon/orbit-tickets#259, thomasluizon/orbit-tickets#260 and thomasluizon/orbit-tickets#265.

Approach

  • tests/Orbit.Infrastructure.Tests/Extensions/WebApplicationExtensionsMcpTests.cs: build the actual middleware delegate with ApplicationBuilder, substituting authentication, rate-limit and policy ports. This covers the missing composition without hosting a server. SDK request types produce the MCP body.
  • tests/Orbit.Application.Tests/Chat/Tools/CreateGoalToolTests.cs: test the Astra tool directly, including repository writes and both sides of habit links, without duplicating command-handler coverage.
  • src/Orbit.Domain/Interfaces/IPayGateService.cs, src/Orbit.Application/Common/PayGateService.cs and src/Orbit.Application/Chat/AstraBlockAnswerService.cs: remove the duplicate allowance gate. Update the dependent application/infrastructure tests, paygate.md provenance and tools/__tests__/gating-matrix.test.mjs to follow the reservation path.

Test evidence

The following fault injections were temporary and were restored before committing. These are observed local runs, not inferred outcomes.

MCP middleware and retrospective classification

Temporarily removed the TryApplyMcpRateLimitsAsync call from HandleMcpRequestAsync and removed only the direct get_retrospective entry from AiBearingMcpTools, leaving the wrapped-agent operation set intact.

Command for all three observations:

dotnet test tests/Orbit.Infrastructure.Tests --filter FullyQualifiedName~WebApplicationExtensionsMcpTests
  • Unchanged tests with those defects present: exit 0, 31 passed.
  • Strengthened tests with those defects still present: exit 1, 31 passed and 3 failed. UseMcpSelectiveAuth_ExhaustedKey_RejectsBeforeNextAndLeavesSecondKeyUnaffected failed for both mcp and mcp-ai with Expected limited.Response.StatusCode to be 429, but found 200. TryApplyMcpRateLimitsAsync_AiBearingTool_HitsAiLimitBeforeGeneralLimit failed for get_retrospective because the second call returned true instead of false.
  • Original protections restored: exit 0, 34 passed. Both middleware cases prove the next delegate is skipped on rejection, both policies charge accepted requests, and another key for the same user remains unaffected.

The middleware cases use get_daily_summary, so their failures are independent of the retrospective classification injection.

Astra goal habit cap

Temporarily removed the cap guard from CreateGoalTool.PrepareAsync.

dotnet test tests/Orbit.Application.Tests --filter FullyQualifiedName~CreateGoalToolTests
  • Unchanged tests with the guard removed: exit 0, 7 passed.
  • Strengthened tests with the guard still removed: exit 1, 8 passed and 1 failed. HabitIds_AboveCap_ReturnsExistingErrorWithoutWritingGoalOrLinks failed with Expected result.Success to be False, but found True. The at-cap success case passed.
  • Original guard restored: exit 0, 9 passed, including rejection without writes or links and success at the cap.

Reservation cleanup

Before cleanup, the unchanged focused application selection (the application command below with --no-build) passed 106 tests. After migrating it to the reservation path it passed 104 tests; eight obsolete allowance cases were replaced with six smoke-environment/plan cases. The focused infrastructure selection passed 26 tests and the gating-matrix suite passed all 15 tests. The reservation method was compared with its original source and is unchanged.

dotnet test tests/Orbit.Application.Tests --filter 'FullyQualifiedName~PayGateService|FullyQualifiedName~AstraBlockAnswerServiceTests|FullyQualifiedName~FeatureExplanationDriftTests|FullyQualifiedName~ProcessUserChatCommandHandlerTests.ChoiceAnswer'
dotnet test tests/Orbit.Infrastructure.Tests --filter 'FullyQualifiedName~AstraBlockAnswerRecoveryTests|FullyQualifiedName~AstraPickListPreviewTests|FullyQualifiedName~BlockAnswerWriteRetryTests'
node --test tools/__tests__/gating-matrix.test.mjs
  • dotnet test tests/Orbit.Application.Tests --filter FullyQualifiedName~HabitIds_AboveCap_ReturnsExistingErrorWithoutWritingGoalOrLinks: unchanged test passed. Strengthened assertion failed before the fix, exit 1: Expected result.ErrorCode to be "MAX_HABITS_PER_GOAL", but found <null> at line 106. After the fix, rerun with --no-build passed, exit 0.
  • dotnet test tests/Orbit.Infrastructure.Tests --filter FullyQualifiedName~UseMcpSelectiveAuth_ExhaustedKey_RejectsBeforeNextAndLeavesSecondKeyUnaffected: both existing cases passed with the pipeline registration temporarily removed, confirming the gap.
  • dotnet test tests/Orbit.Infrastructure.Tests --filter FullyQualifiedName~ConfigureOrbitPipeline_LimitedMcpPost_RejectsBeforeNext: new test failed with registration removed, exit 1: Expected context.Response.StatusCode to be 429, but found 200 at line 372. After restoring registration, rerun with --no-build passed, exit 0.
  • dotnet build Orbit.slnx: exit 0, zero errors. Regenerated OpenAPI files were unchanged.
  • dotnet test tests/Orbit.Domain.Tests --no-build: exit 0, 797 passed.
  • dotnet test tests/Orbit.Application.Tests --no-build: exit 0, 5,458 passed.
  • dotnet test tests/Orbit.Infrastructure.Tests --no-build: exit 0, 3,955 passed.
  • node --test tools/__tests__/gating-matrix.test.mjs: exit 0, 15 passed.
  • All commit hooks passed.

Validation

All commands exited 0:

Command Result
dotnet build Orbit.slnx 0 errors, 19 warnings
dotnet test tests/Orbit.Domain.Tests 797 passed
dotnet test tests/Orbit.Application.Tests 5,458 passed
dotnet test tests/Orbit.Infrastructure.Tests 3,954 passed
dotnet test tests/Orbit.Analyzers.Tests 40 passed
node --test tools/__tests__/gating-matrix.test.mjs 15 passed

After the solution build, the focused MCP and goal selections passed again with 34 and 9 tests respectively. OpenAPI v1/v2 regenerated without a contract diff. Commit hooks, the timeless check, suppression allowlist and diff whitespace checks passed; the changed C# files contain no bare narration comments.

Assumptions

  • The cap rejection currently returns a null ToolResult.ErrorCode; preserve and assert that observed value plus the exact existing message, rather than introducing a code in a test-only change to this tool.
  • Exercise middleware composition with ApplicationBuilder and mock the authentication port, rather than adding a hosted integration harness; repository conventions require unit tests and the order prohibits starting a server.
  • Treat gating-matrix.json as local generated output as CLAUDE.md instructs, rather than committing its unrelated existing drift even though the snapshot is tracked.

Invoked the real ConfigureOrbitPipeline in memory instead of starting an HTTP server, honoring the work order’s server prohibition.

Branch applicability

Re-verified the affected code on redesign/main at 45548c59. The local origin/main snapshot at 4b1c9d7e has the same middleware limiter call, retrospective classifier, goal cap and obsolete allowance definitions. The additional Astra block-answer caller is specific to the redesign branch and is removed here.

External interface evidence

MCP bodies are produced by installed ModelContextProtocol 1.4.1 types (JsonRpcRequest, CallToolRequestParams, RequestId) using McpJsonUtilities.DefaultOptions, rather than handwritten protocol fixtures. The existing SDK serialization test and new middleware cases passed against that producer. The authentication substitute returns the real ASP.NET AuthenticateResult.Success(AuthenticationTicket) type; its constructor and authentication-service signatures were checked against the installed .NET reference package and exercised by the passing middleware cases. No production external response fields were added.

Manual steps

None.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes across the quota cleanup, Astra goal boundaries, MCP middleware coverage, and dependent fixtures.

  • Single quota gate: Removed the duplicate allowance check and interface method while preserving atomic reservation, transactional claim rollback, and crisis/pick-list quota bypasses.
  • MCP regression coverage: Exercised the actual selective-auth middleware for both rate-limit policies, endpoint short-circuiting, per-key isolation, and direct get_retrospective classification.
  • Goal and quota tests: Covered goal habit counts at and above the cap, bidirectional links, rejected writes, Free/Pro smoke-account rules, daily quota consumption, and updated gating-matrix fixtures.
  • Validation: Passed 113 focused application tests, 60 focused infrastructure tests, and all 15 gating-matrix tests. The full solution suite was not rerun during this review.

Pullfrog  | View workflow run | Using gpt-6.1-sol | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes since the previous review at a6a09952, with the full PR diff checked for context.

  • Preserved goal-cap error codes: Routed the existing rejection through ToolResult.FromFailure and strengthened the test to require MAX_HABITS_PER_GOAL, while retaining the message and no-write assertions.
  • Covered pipeline registration: Added an in-memory test of ConfigureOrbitPipeline that verifies an exhausted MCP budget returns 429 before the terminal delegate and uses the expected API-key partition.
  • Simplified quota fixtures: Renamed the block-answer test to describe its mocked claim attempt and removed redundant pick-list setup while retaining the no-consumption assertion.
  • Verified affected paths: Passed 31 focused application tests and 46 focused infrastructure tests, including the new pipeline test and quota rollback coverage. The full solution suite was not rerun.

Pullfrog  | View workflow run | Using gpt-6.1-sol | 𝕏

@sonarqubecloud

Copy link
Copy Markdown

@thomasluizon
thomasluizon merged commit e40a2a6 into redesign/main Oct 19, 2026
24 checks passed
@thomasluizon
thomasluizon deleted the thomasluizon/ticket-264-astra-test-gaps branch October 19, 2026 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant