Repository navigation
Share the metrics response shape between REST API and MCP server - #316
Merged
Merged
Conversation
…ric key migration
---
# Conversation that produced these changes
---
## User prompt
"now help me resolve these code duplications
- 4.3. mcp_server/server.py:203: Metrics flattening is duplicated in webapp/service/data_handling.py::_merge_metrics_by_type. GRADIENT_LENGTH_KEY is defined twice (mcp_server/server.py:68, monitoring/alerts/pump_pressure_alert.py:30).
- 1.1 Presenting a raw file with its metrics, three shapes. The REST API returns a list of dicts, one per metrics type. The MCP server flattens all types into one dict and prefixes a key with the type only on a collision. The webapp prefixes every key with the type. Each client has its own
loop over the same mapping to do this.
- 1.2 The metrics metadata fields are listed twice. The MCP server and the webapp each spell out settings_name, settings_version, relative_output_path and output_display_base_path to put them next to the values.
- 1.3 The display output path of a metrics entry is computed twice. The REST API and the webapp's output folder list both do "none if there is no relative path, else get_display_output_path(...)".
- 1.4 "Drop unset fields" is written four times, in the REST API (twice) and the MCP server (twice), as the same dict comprehension over vars().
- 2.1 Legacy metric names are harmonized in two places with different rules. The backend renames two legacy keys for everyone. The webapp additionally strips every msqc_ prefix and renames quanting_time_elapsed. So for old files the webapp shows evosep_... while the REST API and the MCP
server return msqc_evosep_... for every key except the one the backend knows. I left this in slice 6b on purpose, because moving it into the backend changes REST and MCP keys for old files. It is a decision, not a cleanup."
## Clarifying round 1
Q: 2.1 Legacy metric names: old MSQC metrics are stored with an 'msqc_' prefix (current calculators no longer write it), and 'quanting_time_elapsed' is the old name of 'time_elapsed'. Only the webapp strips/renames these today. Where should this harmonization live?
- Backend, for all (Recommended)
- Leave as is
- Migrate the data <-- chosen
Q: 1.1 Three output shapes of 'raw file with metrics' (REST: list of dicts per type; MCP: one flat dict, type prefix only on key collision; webapp: every column prefixed '<type>__'). The shared part (metadata + values per type) gets one helper either way. Should the shapes themselves change?
- Keep the three shapes (Recommended)
- MCP always prefixes
- MCP like REST <-- chosen
Q: 1.4 'Drop unset fields' exists four times, but the four differ (plain; with excluded keys; with a whitelist; with value formatting). One shared helper 'fields without None' could be the base of all four, saving about one line each. Worth it?
- Skip it (Recommended) <-- chosen
- Add the helper
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. |
…onstraint CI installs all requirements into one environment. fastapi==0.115.0 downgraded starlette below 0.39, the MCP requirements then upgraded it again, and fastapi 0.115.0 no longer imported: collecting rest_api/tests/test_routes.py failed and no test ran. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
--- # Conversation that produced these changes --- ## User prompt "Now create an implementation plan. Refactorings upfront, then several human-reviewable slices." ## Clarifying round 1 Q: Decision 4.4 of VALIDATION_DESIGN.md: where do the validation rules for writes live? This determines the shape of every slice. - shared.db API modules (Recommended) <-- chosen - New shared/services Q: Decision 4.1: how do clients see validation errors from the pydantic input types? - One domain error (Recommended) <-- chosen - Two error types Q: Decision 4.3: does the plan include write tools for the MCP server? Today the webapp is the only writing client. - No, leave out (Recommended) - Yes, final slice <-- chosen Q: Decision 4.5: forms and Mongo models have different max lengths (settings file names and software 64 vs 128, project description 256 vs 512, messenger id 64 vs 512). Which value becomes the single declared limit? - The form's (Recommended) <-- chosen - The model's Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- add the input type helpers (shared/domain/inputs.py): `InputModel`, `parse`, `required`, `RequiredStr`, `OptionalStr`, `max_length` - `NewSuppression` replaces `validate_suppression` of the admin page - `toast_result` lists several validation errors Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- `NewUser` and `UserUpdate` replace the cleaning on the admin page - emails are stored and looked up stripped and lower-cased - `set_status` rejects the current and unknown statuses - the initials and email uniqueness checks query the backend - the form limits are read from the input type Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- `NewProject` replaces the project id check of the projects page, the id rule is one regex - a project needs an active owner, if one is given - only active projects can be archived or deleted - the form limits are read from the input type Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…k in shared - `check_config_params` is the one check of unresolved `config_params`, used by the settings page and at job time; its helpers are private - move the runner check to shared/runner_checks.py, it also rejects an unknown runner - remove the unused `allow_absolute_paths` of `check_for_malicious_content` Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- `NewSettings` replaces the validation on submit of the settings page - `settings.create` checks the runner and the owner, reports all rule errors together and archives the previous version itself (`archive_previous`) - only active settings can be archived or deleted - the form limits are read from the input type - remove `empty_to_none` of the webapp, the input types do it Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- `NewAssignment` validates scopes and file name filters - `assign`, `unassign` and `update_to_latest` take `allow_fallback`: the caller states whether it may change the assignments of the fallback project - `assign` reports all rule errors together - `update_to_latest` points the assignment to the latest version in one write - `known_scopes` is the one list of scopes, also for the form Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- unknown scopes: neither the default scope, a vendor nor an instrument of the yaml - scopes whose instruments are all excluded - the default scope excluded, a scope both included and excluded - file name filters that always match an exclude filter `update_to_latest` does not re-check the stored scopes and filters. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A scope or file name filter that would never apply is rejected even if other entries of the assignment still apply. Messages and docstrings said "they would never apply", which read as the whole assignment. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
`fully_excluded_scopes` already rejects an excluded default scope and a scope that is both included and excluded, for every scope with an instrument in the yaml. Remove the two rules from `NewAssignment`. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A scope in both `scopes` and `excluded_scopes` was listed twice. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
`get_instrument_types` in shared/yamlsettings.py replaces the separate reads in the scope resolver and the projects, overview and admin pages. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
`KNOWN_VENDOR_NAMES` becomes `KNOWN_INSTRUMENT_TYPES`; comments, docstrings, test names, an error message and the texts of the projects page follow. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The page only knows the active versions, so "current + 1" was wrong whenever a newer version was archived or deleted. Add a test that the archive checkbox names the version `settings.create` archives, the latest active one. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…arams The gap is in that check, which every client and the processor run. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The field declares the lengths, the regex only the characters. A too short id now gets the length message, like a too long one. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Project.id and User.initials mixed a constant with a literal. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Only the projects page left it out; other clients could archive or delete it. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The record, with its possibly large file info, was loaded only to log its old status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The migration left such keys alone, and since the webapp no longer merges the duplicate columns, those documents kept an extra column. Keys with another value are still left alone and reported. Add the first tests of the migration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`_to_raw_file_dict` and `_to_dict` differed only in the excluded keys; the metrics and the backup path are added in one pass over the raw files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The error list is the same, as the project errors are collected before; the else branch and the type ignore on the version go away. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guard against an empty id list was redundant, as `id__in=[]` matches nothing, and the backend sort was repeated by the facade, whose project query returns the newest first anyway. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uest Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Status records truncated too long details in the facade function, raw files in their input type. `KrakenStatusUpdate` truncates them the same way and checks the other field limits; the limit tests compare it with the model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NaN is not equal to itself, so a duplicate key with a NaN value was kept and reported as a conflict. Also find the key holding the current name in one expression. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both were 1024 and fed the same truncation; nothing meant them to diverge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refactor validation2 xvi
Refactor validation2 xiii
Refactor validation2 xii
Refactor validation2 xi
Refactor validation2 viii
Refactor validation2 ix
reject settings assignments that can never apply
validate settings assignments with an input type in shared.db
validate new settings with an input type in shared.db
prepare the settings validation: one config_params check, runner chec…
validate new projects with an input type in shared.db
validate users with input types in shared.db
Write validation: toast_result in the webapp, drop the webapp read filter check
Write validation: regex constants to shared/domain, one function per composite write
Write validation: design and implementation plan
Refactor validation2 x
validate alert suppressions with an input type in shared.db
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #315.
🤖 Generated with Claude Code