Repository navigation
feat: migrate Astra to Responses with GPT-6.1 Sol (#582) - #801
Conversation
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes: Reviewed the complete Responses migration and its effects on Astra tool continuations and shared AI completion callers.
- Responses transport: Text, JSON and chat calls now use centralized Responses options, model-compatible temperature/reasoning settings, cancellation and per-user cache routing.
- Tool continuations: Buffered and streamed rounds preserve output items and encrypted reasoning, correlate results by
call_id, and reject failed responses, incomplete tool calls and streams without a terminal event. - Models and pricing: Defaults move to
gpt-6.1-solandgpt-6-luna, while configuration supports restoring the previous primary model through Responses. - Regression coverage: SDK-serialized fixtures cover continuation replay, cache keys, images, failure handling and cancellation during response buffering.
Validation: All 1,247 infrastructure unit tests matching FullyQualifiedName~Ai passed. No verified Critical or High regression was found. Live provider calls and full application-turn acceptance were not independently exercised in this review.
gpt-6.1-sol ｜ 𝕏
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes: Reviewed 0a39e44 against the previous Pullfrog-reviewed commit, with the complete current PR diff as context.
- Separated timeout budgets: Buffered completions now receive additional reasoning headroom while streaming retains its existing timeout limits.
- Expanded reasoning allowances: Summary and reschedule JSON calls now reserve 8,192 output tokens while retaining their user-visible text limits.
- Preserved response boundaries: Buffered text extraction and streaming deltas separate distinct assistant messages consistently.
- Refined model compatibility: Requests now use
safety_identifierwith the existing cache key, and legacy-model requests and continuations omit reasoning-specific data. - Aligned deployment configuration: Terraform API environment groups now declare the primary and subtask models, with matching documentation and regression coverage.
- Validated AI regressions: All 1,262 infrastructure tests matching
FullyQualifiedName~Aipassed. Live provider calls and full application-turn acceptance were not independently exercised in this review.
gpt-6.1-sol ｜ 𝕏
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes: Reviewed ffb62a4 against the previous Pullfrog-reviewed commit 0a39e44, with the complete current PR diff as context.
- Bounded buffered completions: Replaced the derived 450-second deadline with independently configurable
CompletionTimeoutSeconds, defaulting to 185 seconds while preserving streaming limits. - Verified cancellation behavior: Added production-constructor tests for JSON, text and buffered chat cancellation during retry waits, plus configuration tests separating the buffered deadline from network timeouts and retry counts.
- Strengthened configuration coverage: Scoped Terraform assertions to production model values and staging inheritance, and aligned
.env.examplewith the configured primary and subtask models. - Validated AI regressions: All 1,267 AI infrastructure tests passed; a separate, overlapping focused run passed all 39 completion-client and settings tests. Live provider calls and full application-turn acceptance were not independently exercised.
gpt-6.1-sol ｜ 𝕏
|
Refs thomasluizon/orbit-tickets#582. Base:
redesign/main.Astra cannot use GPT-6.1 Sol function tools through Chat Completions. This migrates chat, streaming, JSON and text calls to
/v1/responses, withgpt-6.1-solat low reasoning as primary andgpt-6-lunaat low reasoning for subtasks. Model overrides remain configuration.Intended approach and delivered layers
AiCompletionClient.cscentralizes Responses options, tier selection, compatible temperature/reasoning parameters, JSON format, retries and completion cancellation.AiIntentService.csmaps text deltas and terminal response events, keeps every output item including encrypted reasoning for tool continuations, and sends tool results with theircall_id. Usage still goes to the existing recorder once per model round. Incomplete tool calls, failed responses and streams without a terminal event fail rather than executing partial arguments.prompt_cache_keyuses the same authenticated user identifier asuser, including continuation rounds. Tests cover stable keys, different users, model rollback and pricing coverage.AiSettings.csandappsettings.jsoncarry the defaults and new prices. Existing rollback prices remain. Test HTTP fixtures now serialize the installed SDK's Responses models. The suppression allowlist only declares the two requiredOPENAI001sites: this installed SDK marks its typed Responses surface experimental.Remaining for the next round on this branch: strict schema conversion and enum audit (thomasluizon/orbit-tickets#583), input reductions (thomasluizon/orbit-tickets#16), and PostHog events (thomasluizon/orbit-tickets#584). This layered delivery does not close #582. Functions explicitly retain
strict: falseto preserve existing optional-field behavior during the transport migration. Making all fields nullable and required now would changeupdate_habit: explicit null currently clears description, emoji, recurrence, time and end date, whereas absence preserves them. That needs the separate schema and handler audit ordered in layer 3.The migration, model configuration, prices, cache key and fake-stream correction also apply semantically to
main. This PR targets and testsredesign/main;mainpredates its whole-completion deadline plumbing, so a backport needs that dependency. The generated OpenAPI documents were regenerated and have no diff. No client DTO or quota changes.Live interface evidence
Verified on 2026-10-19 against official OpenAI documentation and the account's live API:
reasoning.effort: low. Standard prices per million tokens: input $2.00, cached input $0.10, output $10.00.GET /v1/modelsreturned both exact IDs. Complete individual entries, with their original field names, are below. Re-derive usingcurl --fail https://api.openai.com/v1/models -H "Authorization: Bearer $OPENAI_API_KEY"and select those IDs fromdata.[ { "id": "gpt-6.1-sol", "object": "model", "created": 1790552874, "owned_by": "system", "shutdown_date": null }, { "id": "gpt-6-luna", "object": "model", "created": 1789406102, "owned_by": "system", "shutdown_date": null } ]The installed NuGet package is OpenAI 2.14.0, repository commit
2e77b08828145f658ec04e49aec87abb1543c553from its ownopenai.nuspec. I decompiled the installed DLL, rather than assuming SDK names. Reproduce withilspycmd -p -o /tmp/openai-installed "$HOME/.nuget/packages/openai/2.14.0/lib/net10.0/OpenAI.dll"and inspect these types:ResponsesClient.CreateResponseAsync/CreateResponseStreamingAsync/responses; streaming requiresStreamingEnabled = trueCreateResponseOptions.Model,InputItems,Toolsmodel,input,toolsCreateResponseOptions.ReasoningOptionsandResponseReasoningOptions.ReasoningEffortLevelreasoning.effort;ResponseReasoningEffortLevel.Lowserializes aslowCreateResponseOptions.PromptCacheKey,EndUserId,StoredOutputEnabled,IncludedPropertiesprompt_cache_key,user,store,includeResponseTextOptions.TextFormat/ResponseTextFormat.CreateJsonObjectFormat()text.format.type: json_objectResponseTool.CreateFunctionTool(name, parameters, strictModeEnabled, description)strictis a typed argumentResponseResult.OutputItems,Status,IncompleteStatusDetails,Usage,GetOutputText()output,status,incomplete_details,usage, combined output textFunctionCallResponseItem.CallId,FunctionName,FunctionArgumentscall_id,name, JSON stringargumentsResponseItem.CreateFunctionCallOutputItemtype: function_call_output,call_id,outputReasoningResponseItem.EncryptedContentencrypted_content, preserved in the next round withstore: falseStreamingResponseOutputTextDeltaUpdate,StreamingResponseCompletedUpdate,StreamingResponseIncompleteUpdate,StreamingResponseFailedUpdate,StreamingResponseErrorUpdateresponse.output_text.delta,response.completed,response.incomplete,response.failed,errorResponseTokenUsage/ResponseInputTokenUsageDetailsinput_tokens,output_tokens,total_tokens,input_tokens_details.cached_tokensSystem.ClientModel1.15.0 was also decompiled. ItsHttpClientPipelineTransport.HttpClientTransportResponse.Disposere-buffers aMemoryStream; cancellation disposes the stream while buffering. That explains why the original fake could replace cancellation withObjectDisposedExceptionduring cleanup. The network-shaped fake avoids that memory-only cleanup path, while still using the real SDK retry policy and cancellation.A live streaming request using the primary model, low reasoning,
store: false,include: ["reasoning.encrypted_content"]and arecord_run(date)function returned HTTP 200 and a function call. The complete terminal response follows, with encrypted reasoning content redacted. This request usedstrict: false; strict acceptance remains part of layer 3.{ "id": "resp_092d111b3dba36b1016ad689f3ce4c87d28c5fd5d48ddf7a04", "object": "response", "created_at": 1792444915, "status": "completed", "access_programs": null, "background": false, "completed_at": 1792444917, "error": null, "frequency_penalty": 0.0, "incomplete_details": null, "instructions": null, "max_output_tokens": null, "max_tool_calls": null, "model": "gpt-6.1-sol", "moderation": null, "output": [ { "id": "fc_092d111b3dba36b1016ad689f551dc87d2abab792fef0a7d96", "type": "function_call", "status": "completed", "arguments": "{\"date\":\"2026-10-19\"}", "call_id": "call_W1rQEjEq2zQlehoMkvH6rzHo", "name": "record_run" } ], "parallel_tool_calls": true, "presence_penalty": 0.0, "previous_response_id": null, "prompt_cache_key": null, "prompt_cache_retention": "24h", "reasoning": { "context": "all_turns", "effort": "low", "mode": "standard", "summary": null }, "safety_identifier": null, "service_tier": "default", "store": false, "temperature": 1.0, "text": { "format": { "type": "text" }, "verbosity": "medium" }, "tool_choice": "auto", "tool_usage": { "image_gen": { "input_tokens": 0, "input_tokens_details": { "image_tokens": 0, "text_tokens": 0 }, "output_tokens": 0, "output_tokens_details": { "image_tokens": 0, "text_tokens": 0 }, "total_tokens": 0 }, "web_search": { "num_requests": 0 } }, "tools": [ { "type": "function", "description": "Record a completed run.", "name": "record_run", "output_schema": null, "parameters": { "type": "object", "properties": { "date": { "type": "string" } }, "required": [ "date" ], "additionalProperties": false }, "strict": false } ], "top_logprobs": 0, "top_p": 0.98, "truncation": "disabled", "usage": { "input_tokens": 58, "input_tokens_details": { "cache_write_tokens": 0, "cached_tokens": 0 }, "output_tokens": 23, "output_tokens_details": { "reasoning_tokens": 0 }, "total_tokens": 81 }, "user": null, "metadata": {} }The actual migrated
AiCompletionClient+AiIntentServicewere then invoked directly, without a server or database: streamedrecord_run, supplied its synthetic success result, and streamed the final confirmation. Both rounds succeeded and recordedgpt-6.1-solusage. A real Luna JSON call returned{"value":"ok"}and recordedgpt-6-luna. Repeating the same tool/continuation exercise with onlyAI:Model = gpt-4.1-minisucceeded through Responses with temperature restored.Before and after measurements
Live measurements on 2026-10-19. These are first model rounds, not full application turns or production-account acceptance. The input used the repository's real static/dynamic prompt and tool schemas, with a synthetic habit index (
Beber água,Ler 10 minutos,Academia,Run) and local date 2026-10-19. Ask was disabled. The real keywordless declaration set is still 44 tools; no input reduction is claimed. No application writes were executed.Old:
gpt-4.1-mini, Chat Completions, temperature 0.1. New:gpt-6.1-sol, Responses, reasoning low. One sample per case/model, up to three concurrent calls, mixed warm/cold caches. TTFT measures the first text or function-argument delta; total ends at the first terminal response. Costs below use the existingAiUsageDailyformula and configured prices, not invoice reconciliation. GPT-6 responses also expose cache-write tokens; their extra write premium is not represented by the existing recorder, so these costs understate that premium. The cost schema/computation is explicitly outside this transport layer. Follow-up filing throughtools/create-ticket.mjswas unavailable because that adapter is absent in this repository.O1-O3 reproduce the quoted requests in thomasluizon/orbit-tickets#1568. O4 reconstructs its described tag request; O5 is an explicit probe for its grounding symptom. That ticket does not contain six verbatim requests, so an exact six-request owner replay remains pending. P checks the weekday intent in pt-BR.
First-round correctness observations:
frequency_unit: Week, contrary to the exact-weekday prompt ruleDayand quantity 1update_habit.sub_habits, which is absent from that schemaThese results do not prove whole-turn parity, correct previews, successful approvals, or grounded final answers for the owner account. Those acceptance checks remain open alongside thomasluizon/orbit-tickets#1568 and the remaining #582 layers. thomasluizon/orbit-tickets#26 remains open, so its promptfoo gate was unavailable.
Test evidence
Before any implementation changes:
All 31 unchanged tests passed with the missing cache key, old endpoint and temperature defect present. Strengthened existing tests then failed before the implementation change:
Both failed for the intended reasons:
/chat/completionsinstead of/responses, and missingprompt_cache_key.Three added rows failed before the fix: reasoning models in primary received temperature, and GPT-4.1 mini in a distinct subtask tier lost it. These cases pass after the capability-based fix.
The original deadline fake also survived 200 JSON iterations under focused coverage, so the sporadic timing failure was not claimed reproduced by repetition. I then strengthened the test to cancel exactly while a memory-backed response is being buffered:
Before the fake-stream fix: FAIL, expected
OperationCanceledException, gotObjectDisposedException: Cannot access a closed StreamfromHttpClientTransportResponse.BufferContentduringDispose. After the fake-stream fix: PASS. The targeted coverage run below also passed all five deadline cases, including the JSON case's 200 iterations:The initial unrestricted coverage attempt was canceled while instrumenting dependencies; the successful run scoped instrumentation to the completion client. No unrestricted-coverage success is claimed.
Final checks:
dotnet build Orbit.slnx: exit 0, zero errors, existing warnings only.dotnet test tests/Orbit.Infrastructure.Tests --no-build --no-restorewith the focused AI filter: exit 0, 194 passed. The complete projects each exited 0:dotnet test tests/Orbit.Domain.Tests --no-build --no-restore: 797 passed.dotnet test tests/Orbit.Application.Tests --no-build --no-restore: 5,441 passed.dotnet test tests/Orbit.Infrastructure.Tests --no-build --no-restore: 3,952 passed.dotnet test tests/Orbit.Analyzers.Tests --no-build --no-restore: 40 passed.Total: 10,230 passed, zero failed and zero skipped.
Additional checks: suppression allowlist, timeless rules, dash check and
git diff --checkpassed. Tests cover streamed and buffered tool continuations, reasoning-item replay, multiple call IDs, image encoding, failure/incomplete handling, usage and recorder failure, cache-key stability, old-model configuration, and pricing rows.Unchanged tests: 132 passed with defects present.
Strengthened tests: 19 failed before fixes; 147 passed afterward.
Timeout regression: three buffered paths failed before fixes; all four paths passed afterward.
Build: exit 0, zero errors.
Domain: 797 passed; Application: 5,441; Infrastructure: 3,967.
Coverage: deadline cases passed, including 200 JSON iterations.
Hooks passed; generated OpenAPI files unchanged.
Unchanged
AiSettingsTests: 8 passed with the defect present:dotnet test tests/Orbit.Infrastructure.Tests --filter FullyQualifiedName~AiSettingsTestsStrengthening
Defaults_AllowBufferedReasoningWithoutExtendingStreamingNetworkTimeoutand addingCompletionTimeout_IsIndependentOfNetworkTimeoutsAndRetryCountproduced three failures before the fix. All pass afterward.The production constructor regression failed on JSON, text, and buffered chat while the old formula remained active; all three pass afterward:
dotnet test tests/Orbit.Infrastructure.Tests --filter FullyQualifiedName~CompletionDeadline_ConfiguredBoundMoving the model values into staging made the strengthened Terraform test fail. An altered decimal separator and a staging override also failed. Restoring the configuration passed:
dotnet test tests/Orbit.Infrastructure.Tests --no-build --filter FullyQualifiedName~RenderEnvironmentGroups_UseConfiguredPrimaryAndSubTaskModelsOne focused run overlapped a deliberate mutation and failed that assertion; the restored run passed all 69 tests.
All final checks exited 0:
dotnet build Orbit.slnxdotnet test tests/Orbit.Domain.Testsdotnet test tests/Orbit.Application.Testsdotnet test tests/Orbit.Infrastructure.TestsPre-commit hooks passed. Regenerated OpenAPI files were unchanged.
Rollout
The observed staging environment has
AI__Model=gpt-4.1-mini, which overrides the new code default. Set it togpt-6.1-solwhen deploying this change;AI__SubTaskModelcan selectgpt-6-luna. Observe new model names and nonzero costs inAiUsageDaily. RestoringAI__Model=gpt-4.1-minirestores the legacy chat model and temperature behavior through Responses. A transport rollback requires reverting this PR. This worker did not deploy or merge.Assumptions
Manual steps
Release staging code first. The running base image always sends
temperature, which reasoning models reject. In GitHub Actions → Release API, dispatch frommainwithenvironment=stagingandbranch=redesign/main, after it contains this change. Confirm the carrying commit is live in Render.Then, from this repository’s configured
infra/directory, using the initialized backend, operator credentials, and ignoredlocal.tfvars, review and apply:Production: only after the production image carrying this change is live, and only with the owner’s yes:
In Render → Environment Groups →
orbit-staging-apiororbit-production-api, verifyAI__Model=gpt-6.1-solandAI__SubTaskModel=gpt-6-luna. Check service Environment overrides. Confirm subsequentAiUsageDailyrows contain the configured model names and nonzero costs.