Repository navigation
Reduce Astra input and preserve background cache keys - #903
Closed
thomasluizon wants to merge 3 commits into
Closed
thomasluizon wants to merge 3 commits into
thomasluizon wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes across all 10 files, including request identity, tool selection, loader continuations, and the existing execution and budget boundaries.
- Background identity: Ambient
AiCallContextidentity supplies cache keys, safety identifiers, and usage attribution when no explicit user ID is provided. - Tool gating: Cumulative English/Portuguese keyword groups reduce initial declarations, standalone thanks omits tools, and unsupported
askis excluded from both initial and recoverable declarations. - Capability recovery: The strict
load_toolscatalog enables missing declarations without executing operations, preserves state across continuations, and keeps provider calls behind budget reservations and cancellation. - Verification: All 158 focused Infrastructure tests and 600 Application tests passed. Live end-to-end conversation replay and deployed usage measurements were not independently verified and remain the acceptance work identified in the PR description.
gpt-6.1-sol | 𝕏
|
Owner
Author
|
Closing unmerged. Its background cache-key fix landed separately as #959; its keyword tool gating stays rejected under the approved thomasluizon/orbit-tickets#1740 plan. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Tracks thomasluizon/orbit-tickets#582.
Astra still declared all 69 tools, and background calls resolved a budget user without putting that identity on the OpenAI request. This finishes the remaining request changes while preserving D147's models, prices and daily ceilings.
Intended approach
AiCompletionClient.cs:156and:296fall back toAiCallContextfor request routing and recorded usage. Explicit user IDs retain precedence. Tests cover stable/different users, HTTP/background identity, text/JSON, both tiers and legacy-model rollback.ChatToolGroups.csandProcessUserChatCommand.Ai.cs:53select cumulative English/Portuguese groups from the message plus history. The keywordless count is 16 includingload_tools, or 17 with client-supportedask.get_profilestays core. Standalone thanks carries no tools. The registry still contains 69 chat operations; no business tool was deleted.AiIntentService.cs:609exposes missing server catalog names through a strict loader enum. Loading only enables declarations. It never executes operations. Two consecutive loading rounds are allowed; existing budgets, cancellation, authorization and approvals still govern execution. Loaded state survives continuations. Unsupportedaskis excluded from both initial and recoverable declarations, with clarification in prose.AiToolModels.cscarry internal orchestration state, not endpoint DTO changes. Card guidance stays available for tools loaded later. The JSON cancellation test repeats 200 independent calls while preserving its existing fake clock and network-like stream lifetime.Remainder audit
AiSettings.cs:7,AiCompletionClient.cs:81and:150. Existing request tests remain green. No model, price or ceiling changed here.AiCompletionClient.cs:156and:296.StrictToolSchema.cs:23,AiIntentService.cs:437,StrictChatToolSchemaTestsand Application optional-null tests. No existing parameter enum changed here; #816's Enum audit lists every prior conversion and allowed values. The loader adds an enum of fixed server tool names.set_color_schemeis already absent, asserted byAgentCatalogServiceTests:121. No additional deletion was required or made; no memory tool changed.AppConstants.cs:58caps model history at 20,AiIntentService.BuildHistoryTranscriptkeeps newest whole messages, andProcessUserChatCommand.Ai.cs:44separates static/dynamic context. API/validator history stays 40.history_summarypath exists to reroute.AiUsageRecorder.cs:22,AiIntentService.CompleteObservedRoundAsync,AiCompletionClient.ExecuteResponseAsync. Recorder tests cover daily-cost agreement, shared traces, unique spans, failed calls, no-op/throwing analytics and privacy. #817 records live HTTP 200 ingestion on 2026-10-20, UUID100873a3-1abb-4761-9a05-42bcc5d7273c; unchanged fields retain that SDK evidence.External interface evidence
Verified 2026-10-24 UTC. OpenAI 2.14.0's installed manifest names commit
2e77b08828145f658ec04e49aec87abb1543c553. Read its options serializer: lines 132-135 mapSafetyIdentifiertosafety_identifier; lines 281-284 mapPromptCacheKeytoprompt_cache_key. #801/#816 establish the remaining Responses, stream, usage and strict-factory shapes. Transport tests inspect actual installed SDK serialization; responses come from its model writer.Live
GET /v1/modelsreturned these complete matching entries. Re-derive through the same authenticated endpoint and select the exact IDs fromdata:[ {"id":"gpt-4.1-mini","object":"model","created":1744318173,"owned_by":"system","shutdown_date":"2027-01-14"}, {"id":"gpt-6-luna","object":"model","created":1789406102,"owned_by":"system","shutdown_date":null} ]The official pricing page confirms Luna's standard per-million prices: input 0.10 USD, cache read 0.01, 30-minute cache write 0.125, output 0.50. The Mini page confirms 0.40 input, 0.10 cached, 1.60 output. These match the unchanged pricing configuration.
A standalone probe exported declarations through production
ConvertToSdkTool, submitted them to/v1/responsesat Luna low and received HTTP 200. All 70 declarations from the broader schema audit were accepted as strict: 69 chat tools plus the MCP-only management tool. The returned function-call excerpt was:{ "id":"resp_0b5d2a5801e69d75016add00f08b2887d29fcc743d46678ff4", "status":"completed","model":"gpt-6-luna", "output":[{"id":"fc_0b5d2a5801e69d75016add00f35e6c87d2be453affc03418c4","type":"function_call","status":"completed","arguments":"{}","call_id":"call_gatx7S0htYOidni88wypQLn6","name":"get_profile"}], "usage":{"input_tokens":26879,"input_tokens_details":{"cache_write_tokens":26876,"cached_tokens":0},"output_tokens":14,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":26893} }The repeated declarations and unrelated response fields are omitted. The submitted
get_profileparameters were{"type":"object","properties":{},"required":[],"additionalProperties":false}, withstrict:true. Gated probes also accepted the strict loader. No backend operation was executed and no app server started.Live before and after
Requests came through the real handler, prompt builder and SDK, with synthetic Run/Read habits due on 2026-10-24, the same synthetic IDs and empty history. Full uses the pre-gating catalog at
ab8547bf. Mini isgpt-4.1-miniwith temperature 0.1; Luna isgpt-6-lunaat low reasoning.Prompts: 1
obrigado; 2log my run today; 3create a habit every wednesday and sunday; 4log my run and show my progress; 5show my profile and today's summary; 6bulk log all habits today.TTFT is the first text/function-argument delta; round time ends at the terminal response. Each row is one live model round, not an executed conversation. USD uses unchanged daily-recorder rates, including cache writes. Warmth varies, so these costs are not an invoice comparison.
Luna input falls from 201,663 to 96,725 tokens, a 52.04% reduction. Every prompt falls by at least 34.7%.
ask; gated asks for the name in prose without inventing a placeholder. This is clarification, not a successful create.Gated response IDs in prompt order:
resp_08152180f3921d93016add03423ea087d28de5fa54220dd1dc,resp_07bfe78a75c20b34016add034241ac87d2805e4edf6375321b,resp_0c8095e6b259c1b4016add0343225487d296287044904f9254,resp_06909334ef50f80e016add0344017887d2832f2016d5ccd6a6,resp_0afcc6ca121e159f016add0345a7c487d28dc499049dc1aa48,resp_06ff5bcbaad83ede016add0347b9b887d2923b3192c25217e4.Final answer correctness, #1568 account replay and deployed daily-row measurements remain acceptance work. #26 is still open in the UI repository, so its promptfoo gate was unavailable here. No writes or synthetic successful tool results were used to claim completion.
Test evidence
Before edits, the unchanged identity/background/fixed-turn tests passed four cases while the cache-key defect remained:
Strengthened tests, before the implementation fix:
Exit 1: eight cases failed at the absent
prompt_cache_key; explicit identity precedence passed. The identical command passes all nine after the fix.The unchanged fixed-turn test passed with 69 tools. Before gating,
dotnet test tests/Orbit.Infrastructure.Tests --no-restore --filter FullyQualifiedName~KeywordlessTurn_DeclaresFewerThanTwentyToolsexited 1, observing 69 against the cap. It now exits 0 with exact count 16. The strengthened fixed-turn test separately enforces at least 30% serialized-request reduction against equal content with all registry schemas; this local metric is separate from live provider usage.Focused runs passed 600 Application and 170 Infrastructure tests; the final strict/input/identity selection passed 41. The JSON deadline theory now repeats its independent call 200 times.
All required commands were judged by process exit code:
dotnet build Orbit.slnx -t:RebuildAiInputBudgetTests,BudgetedBackgroundCalls,StrictChatToolSchemaTests)dotnet test tests/Orbit.Domain.Testsdotnet test tests/Orbit.Application.Testsdotnet test tests/Orbit.Infrastructure.Testsdotnet test tests/Orbit.Analyzers.Tests --no-build --no-restoredotnet test tests/Orbit.Infrastructure.Tests --no-build --no-restore --filter FullyQualifiedName~CompletionDeadline_CancelsTheWholeCallIncludingRetryWaits --collect 'XPlat Code Coverage'(with a temporary results directory)The final required-suite pass totals 14,039 passing tests plus the three environment skips. Both generated OpenAPI files remained unchanged. The final rebuild and focused/Domain/Application/Infrastructure sequence ran serially after coverage completed.
One Infrastructure attempt overlapped coverage instrumentation and was invalidated by
BadImageFormatException; it is not counted as passing. Coverage and normal runs were then separated with a rebuild. Hooks passed formatting, dash, timeless-text, root and suppression checks. No new bare narration comments were added.Assumptions
Manual steps
release.yml),environment=staging,branch=main. Verify/healthserves the merged commit. No new secret/environment value is required.AiUsageDailychat rows for model, nonzero cost and input/cached-token reduction against baseline. In PostHog > AI Observability > Generations and Traces, verify one$ai_generationper call, shared chat traces, distinct spans and agreement with daily costs. This worker did not change analytics registration or production rows.