Skip to content

fix: remove temperature from copilot_chat payload (o4-mini incompatible) - #154

Closed
don-petry wants to merge 1 commit into
mainfrom
fix/copilot-chat-temperature
Closed

don-petry wants to merge 1 commit into
mainfrom
fix/copilot-chat-temperature

Conversation

@don-petry

Copy link
Copy Markdown
Collaborator

Problem

Run job/75690347409 showed the rubber duck silently failing on every review:

[tier2] rubber duck did not produce valid JSON — continuing with deep review only
copilot_chat: HTTP 400 from GitHub Models API
{ "error": { "message": "Unsupported value: 'temperature' does not support 0 with this model. Only the default (1) value is supported." } }

Root cause

copilot_chat() sends "temperature": 0. OpenAI reasoning models (o4-mini, o1, o3) only support the default temperature of 1 and reject anything else with HTTP 400. The code degrades gracefully so the run shows green, but the duck was never actually running.

Changes

File Change
scripts/engine.sh Remove 'temperature': 0 from copilot_chat payload builder
tests/test_copilot_chat.sh Sync build_payload copy; flip Test 9 to assert temperature is absent
.github/workflows/test.yml New: run unit tests on every PR + push to main

Testing

$ bash tests/test_copilot_chat.sh
Results: 13 passed, 0 failed

The GitHub Models API rejects temperature=0 for reasoning models like
o4-mini with HTTP 400:
  'Unsupported value: temperature does not support 0 with this model.
   Only the default (1) value is supported.'

This caused the rubber duck (DUCK_ENGINE=copilot / DUCK_MODEL=o4-mini)
to fail with HTTP 400 on every claude-engine run, silently degrading
every review to deep-only with no cross-engine sanity check.

Fix: remove temperature from the copilot_chat JSON payload entirely —
the API defaults to 1, which is the only supported value for o4-mini.

Also:
- Update tests/test_copilot_chat.sh: sync build_payload to match and
  flip Test 9 to assert temperature is ABSENT (not 0).
- Add .github/workflows/test.yml: run unit tests on every PR and push
  to main so regressions are caught before merge.

Reproducer: job/75690347409 — duck failure logged as:
  copilot_chat: HTTP 400 from GitHub Models API
  {"error":{"message":"Unsupported value: 'temperature' does not
   support 0 with this model..."}}

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 13, 2026 03:02
@coderabbitai

coderabbitai Bot commented May 13, 2026

Copy link
Copy Markdown

Warning

Rate limit exceeded

@don-petry has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 56 minutes and 17 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 131bc3c7-8d04-447a-bd84-2d3db7c58320

📥 Commits

Reviewing files that changed from the base of the PR and between 47d6922 and 39ae06e.

📒 Files selected for processing (3)
  • .github/workflows/test.yml
  • scripts/engine.sh
  • tests/test_copilot_chat.sh
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/copilot-chat-temperature

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Comment on lines +10 to +14
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run copilot_chat unit tests
run: bash tests/test_copilot_chat.sh
@sonarqubecloud

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes the unsupported temperature: 0 field from the copilot_chat() payload so reasoning models (o4-mini, o1, o3) no longer reject the request with HTTP 400, and syncs the test fixture plus adds a CI workflow to run unit tests.

Changes:

  • Drop 'temperature': 0 from the JSON payload built in scripts/engine.sh.
  • Update tests/test_copilot_chat.sh to assert temperature is absent from the payload.
  • Add .github/workflows/test.yml to run tests/test_copilot_chat.sh on PRs and pushes to main.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
scripts/engine.sh Removes temperature: 0 from copilot_chat payload builder.
tests/test_copilot_chat.sh Syncs the inline payload builder and flips Test 9 to assert temperature is absent.
.github/workflows/test.yml New workflow running unit tests on PR and push to main.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request removes the 'temperature' parameter from the API request payload in 'scripts/engine.sh' and updates the test suite in 'tests/test_copilot_chat.sh' to verify that the parameter is absent. These changes are necessary for compatibility with reasoning models like 'o4-mini', which reject explicit temperature settings with an HTTP 400 error. I have no feedback to provide as no review comments were submitted.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Closing in favour of #155, which includes this fix plus the PR sort feature and CI workflow. The temperature fix was cherry-picked into that branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants