Skip to content

fix(mistralai): record prompts passed as plain dict messages - #4509

Open
breken-ai wants to merge 1 commit into
traceloop:mainfrom
breken-ai:fix/mistralai-dict-message-prompts
Open

breken-ai wants to merge 1 commit into
traceloop:mainfrom
breken-ai:fix/mistralai-dict-message-prompts

Conversation

@breken-ai

@breken-ai breken-ai commented Sep 25, 2026 •

Copy link
Copy Markdown
  • I have added tests that cover my changes.
  • If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change. (No screenshots: the change is covered by the span assertions in the new test, run against the existing VCR cassette.)
  • PR name follows conventional commits format: feat(instrumentation): ... or fix(instrumentation): ....
  • (If applicable) I have updated the documentation accordingly. (Not applicable.)

What was wrong

The Mistral client accepts chat messages as plain dicts, and the Mistral SDK README uses that form:

client.chat.complete(model="mistral-tiny", messages=[{"role": "user", "content": "..."}])

In the default (legacy attributes) mode, _set_input_attributes read message.content and message.role. A dict has neither attribute, so the first message raised AttributeError. @dont_throw swallowed it, and the span had no gen_ai.prompt.N.content at all. Only the hard-coded gen_ai.prompt.0.role = "user" was left. Prompts were recorded only when callers built UserMessage/SystemMessage objects.

Fix

_set_input_attributes now reads content and role from a dict message and falls back to the attributes for SDK models. The content goes through the existing _content_as_str, so a string is unchanged and a list of content chunks is serialised the same way completions already are.

Tests

test_mistralai_chat_legacy_with_dict_messages reuses the existing test_mistralai_chat_legacy.yaml cassette, sends a system and a user message as dicts, and asserts both prompt roles and contents.

  • Before the fix: assert None == 'You are a comedian' (1 failed, 26 passed)
  • After the fix: uv run pytest tests gives 27 passed; uv run ruff check . passes

This PR was prepared with AI assistance (Breken's agent). I reviewed the change and ran the tests above.

Summary by CodeRabbit

  • Improvements
    • Chat prompt attributes now support messages provided as dictionaries or SDK models, preserving each message’s role and content. Non-string content is converted to a string for the prompt attribute.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: f9f0c583-bd74-4c55-97b0-bf28de088eb7

📥 Commits

Reviewing files that changed from the base of the PR and between f7082c8 and 332599a.

📒 Files selected for processing (2)
  • packages/opentelemetry-instrumentation-mistralai/opentelemetry/instrumentation/mistralai/__init__.py
  • packages/opentelemetry-instrumentation-mistralai/tests/test_chat.py

Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

Chat prompt extraction now supports messages represented as dictionaries or SDK models. A synchronous test checks legacy prompt attributes for dictionary-based system and user messages.

Changes

Mistral chat prompts

Layer / File(s) Summary
Message extraction and validation
packages/opentelemetry-instrumentation-mistralai/opentelemetry/instrumentation/mistralai/__init__.py, packages/opentelemetry-instrumentation-mistralai/tests/test_chat.py
Prompt extraction reads content and role from dictionaries or SDK models and normalizes content with _content_as_str. A synchronous test checks indexed legacy prompt attributes for dictionary-based system and user messages.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 33259

The change targets dictionary-based prompts, and the supplied test summary covers system and user roles and content. No actionable merge-blocking behavior risk is evident.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 33259

Dictionary-form prompts can now appear in recorded traces where they previously did not. Existing content-capture controls still apply, but deployments handling sensitive prompts should account for the wider capture.

Retained concerns

  • Low · security · inferred: Legacy tracing now records dictionary-form prompt content that was previously absent. Because content tracing is enabled by default, deployments using dictionary messages may send more sensitive text to their configured telemetry destination.
Security review details

Security Blast Radius

  • inferred — The additional content exposure is limited to recorded spans from dictionary-form MistralAI chat calls in legacy mode when content capture is enabled. Its downstream reach depends on each deployment's telemetry configuration.

Trust Boundaries and Controls

  • observed — The new dictionary branch does not bypass the existing span-recording or prompt-content gate. No attacker privilege gain or new telemetry destination is established by the supplied evidence.

Hardening Proposals

  • proposed — For deployments that place confidential text in dictionary-form prompts, verify content-capture settings and telemetry destination access and retention before rollout.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: recording prompts supplied as plain dictionary messages in Mistral instrumentation.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants