Skip to content

DB interface: review fixes, remove design docs - #314

Open
mschwoer wants to merge 10 commits into
refactor_db_XVIfrom
refactor_db_XVII
Open

mschwoer wants to merge 10 commits into
refactor_db_XVIfrom
refactor_db_XVII

Conversation

@mschwoer

Copy link
Copy Markdown
Contributor

Stacked on #313.

  • Settings form prefills unset fields as empty text instead of "None"
  • unassign loads one assignment by id; unassign and settings delete work with a dangling assignment
  • NotFoundError when setting the status of an unknown record
  • Settings with any characters in their name are found by unique name
  • The remaining mongoengine exceptions are translated to DatabaseError
  • Projects and settings are filtered by status in the query
  • Stale docstrings and test setup removed, purge-candidate query asserted
  • The DB interface design docs are removed

🤖 Generated with Claude Code

mschwoer and others added 10 commits November 11, 2026 16:07
---
# Conversation that produced these changes
---

## User prompt

"fix

### 1.1 Settings update form prefills the string `None`

`webapp/pages_/settings.py:195-215`

- Rows now come from `vars(Settings)`, so unset optional fields are `None` and `str(...)`
  turns them into `"None"`.
- The `pd.isna` cleanup at 213-215 does not match a string.
- "Update '<name>'" then shows `None` in the inputs, and saving stores `"None"` as the file
  name.
- Main had this only in the mixed case (as `"nan"`); the branch makes it unconditional.
- Fix: map `None`/NaN to `""` before `str()`.
- `webapp/tests/pages_/test_settings.py` does not catch it: it feeds a hand-built frame
  without `None` values."

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
---
# Conversation that produced these changes
---

## User prompt

"fix

### 1.2 Purge-candidate query has no test

`airflow_src/dags/impl/remover_impl.py:183-195`

- The filter moved here from `get_purge_candidates_by_age`; its old test was deleted with
  `shared/tests/db/test_interface.py`.
- `airflow_src/tests/dags/impl/test_remover_impl.py:239` patches `find` and never asserts its
  arguments, so dropping the status exclusion, the instrument or the oldest-first order stays
  green. This code decides which files get deleted.
- Fix: assert the `RawFileFilter`, `order` and `include_file_info`, with the time frozen."

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
---
# Conversation that produced these changes
---

## User prompt

"fix 3.2. `project_settings.unassign` (`shared/db/project_settings.py:124`) loads all assignments
with their settings to find one by id. A backend `get(assignment_id)` is simpler."

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…p unused df_from_db_data parameters

---
# Conversation that produced these changes
---

## User prompt

"fix

3.4. `archive` and `set_status` (projects, settings, users) use `update_one` and silently do
nothing for an unknown id, while `delete` raises `NotFoundError`.

3.5. `df_from_db_data` (`webapp/service/db.py:228-262`) keeps `filter_dict`,
`drop_duplicates` and `drop_columns`, which only tests still use. Design 4.6 meant them gone."

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…efactoring

---
# Conversation that produced these changes
---

## User prompt

"fix

4.1. `shared/db/_mongo/models.py:3` still says the module has no dependencies on the
codebase; it now imports `shared.domain`.

4.2. `monitoring/tests/test_main.py` still names the mock `mock_connect_db` although it
patches `shared.db.ping`.

4.3. `webapp/tests/pages_/test_settings.py:55-59` has leftover `mock_ps_objects` setup.

4.4. `monitoring/tests/test_alert_manager.py:63-66`: the `no_project_owner` docstring
describes the old patch target."

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
---
# Conversation that produced these changes
---

## User prompt

"check and fix:

### 1.1 Settings with `-` or `.` in the name cannot be fetched

`shared/db/_mongo/settings.py:58`

- Lookup now parses `<name>_v<version>` with `[a-zA-Z0-9_]+`.
- `sanitize_name` in `shared/_migrations/from_0.6.0/_migrate_settings_to_standalone.py`
  keeps hyphens and dots and writes via raw `update_one`, e.g. `dia-default-v1.8`.
- `resolve_settings` returns `dia-default-v1.8_v1`; `prepare_job` -> `settings_store.get`
  raises `NotFoundError` ("is not of the form"), so quanting fails for every raw file
  resolving to such settings.
- Assign, archive and delete in the webapp fail the same way.
- `main` looked up by ObjectId and worked."

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
---
# Conversation that produced these changes
---

## User prompt

"check and fix:

### 1.3 One dangling assignment breaks all of them

`shared/db/_mongo/project_settings.py:27`

- `find` raises `NotFoundError` for the whole result if any one assignment references a
  missing settings document.
- `unassign` and `settings.delete` now call the unfiltered `find()`.
- With one ProjectSettings document pointing to a hard-deleted Settings (manual DB edit),
  `project_settings.unassign(<any id>)` and `settings.delete(<any>)` raise "Settings of
  assignment X do not exist", and the Projects and Settings pages fail.
- The broken assignment cannot be removed through the UI; `main`'s
  `ProjectSettings.objects.get(id).delete()` could."

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
---
# Conversation that produced these changes
---

## User prompt

"check and fix:

### 1.4 mongoengine exceptions still leak through `shared.db`

`shared/db/_mongo/errors.py:37`

- `translate_errors` does not cover `mongoengine.OperationError`, `MultipleObjectsReturned`,
  `InvalidQueryError` or `LookUpError`.
- Example: `save()` hits a non-duplicate OperationFailure ("not authorized" for a read-only
  DB role). mongoengine wraps it as `OperationError`, which is not a `PyMongoError`, so it
  propagates untranslated.
- A client catching `DatabaseError`, as `shared/domain/errors.py` promises, misses it.
  Today only broad `except Exception` handlers hide this."

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… callers

---
# Conversation that produced these changes
---

## User prompt

"fix:

### 3.1 "Active only" filter duplicated in Python

`airflow_src/dags/impl/watcher_impl.py:224`

- `projects.find` and `settings.find` have no status filter, so each caller filters itself:
  `watcher_impl.py:221-225`, the `get_active_project_ids` helper and the `all_settings`
  list in `webapp/pages_/settings.py` and `projects.py`, and `shared/db/settings.py:47`
  (`latest_active`).
- Each fetches inactive rows only to discard them.
- A `statuses` parameter on `find` replaces all of them."

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@claude

claude Bot commented Nov 12, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant