Repository navigation
Description
OpenSpec change: manual-entity-anonymisation
Tracking PR: #1592 (scaffolding — proposal, design, capability spec delta, tasks)
Target capability: entity-relation-grondslagen (extending)
Summary
DocuDesk operators have a recurring need: anonymise a piece of text that Presidio detection missed. The legacy "pass extra entities to the anonymise endpoint" flow no longer works post-entity-relation-grondslagen — the HTTP path is relations-driven.
This change adds the missing write path with an operator-friendly endpoint shape: one POST per "operator decides to anonymise this text in this file", server-side string matching against the file's already-extracted chunks, atomic creation of the catalogue entry (or reuse of an existing one) plus one relation row per occurrence found, with full audit-trail capture.
New endpoint: POST /api/files/{fileId}/manual-entities — body { value, type, category?, wholeWord?, caseSensitive? }.
Defaults: whole-word and case-sensitive matching. Lookup-or-create on (value, type) keeps cross-document placeholder IDs stable. Idempotent on re-call (per-position dedup). Zero-match returns HTTP 200 with the catalogue entry created and a message. RBAC: caller MUST have write access to the file (same gate as markAsAnonymized / PATCH /api/entity-relations/{id}).
Overlap-aware matcher: TextExtractionService produces overlapping chunks (default 200-char overlap). Naive concatenation would double-count. The spec mandates per-chunk preg_match_all + absolute-position dedup with deterministic canonical-chunk selection.
Specs
This change adds 6 ADDED Requirements to the entity-relation-grondslagen capability spec:
POST /api/files/{fileId}/manual-entitiesatomic create / reuse contract- Chunk-aware exact-string matching with overlap-region dedup
- Catalogue lookup-or-create on
(value, type)dedup key - Audit-trail rows:
entity_create(conditional) +entity_relations_batch_create(always) - HTTP logs and error responses MUST NOT contain operator-supplied
value(ADR-005) - Substitution pass remains value-keyed; positions on manual relations are forensic markers
Tasks
1. Value objects and constants
- 1.1 Create
lib/Service/File/ChunkTextMatcher.phputility class - 1.2 Create
lib/Exception/ManualEntityException.phpwith typed reason codes - 1.3 Add
DetectionMethodconstants (extend with'manual')
2. Mapper extensions
- 2.1
GdprEntityMapper::findOneByValueAndTypelookup helper - 2.2
EntityRelationMapper::existsForFileAtPositionidempotency probe - 2.3
EntityRelationMapper::insertBatchatomic multi-row insert
3. ManualEntityService orchestrator
- 3.1 Create
lib/Service/File/ManualEntityService.phpwith DI wiring - 3.2 Implement
addManualEntity()(transaction + lookup-or-create + match + batch insert + audit) - 3.3 Implement
writeAuditTrails()(entity_create when new + entity_relations_batch_create always) - 3.4 Implement
assertFileWriteAccess()(reuse existing helper) - 3.5 Create
ManualEntityResultDTO
4. Controller endpoint
- 4.1 Add
FileTextController::addManualEntity(int $fileId): JSONResponse - 4.2 Register
POST /api/files/{fileId}/manual-entitiesroute - 4.3 PII-redacted request logging (no
valuein logs)
5. DocumentProcessingHandler — no changes required
- 5.1 Verify
findEntitiesForAnonymizationincludes manual-method rows (add test) - 5.2 Verify
findEntityIdsByValueForFileincludes manual-method rows (add test)
6. Spec maintenance
- 6.1 Update
openspec/specs/entity-relation-grondslagen/spec.mdchange list to in-progress
7. Tests
- 7.1
ChunkTextMatcherTest(overlap dedup, whole-word, case-sensitivity, Unicode, needle-too-long, determinism) - 7.2
ManualEntityServiceTest(happy path, reuse, idempotency, zero-match, RBAC denial, audit-failure rollback) - 7.3
FileTextControllerTestextension (415 / 400 / 401 / 403 / 404 / 422 / 500 / 201 / 200 cases + log redaction) - 7.4
GdprEntityMapperTestextension (findOneByValueAndTypelookup, dedup-invariant violation warning) - 7.5
EntityRelationMapperTestextension (existsForFileAtPosition,insertBatch, manual-method infindEntitiesForAnonymization) - 7.6 Integration test against stacked OR: upload → extract → POST manual-entity → anonymise → verify
8. Documentation
- 8.1 Extend
docs/api/objects.mdwith the new endpoint contract - 8.2 CHANGELOG entry under
### Added
9. Quality and verification
- 9.1
composer check:strictclean - 9.2
openspec validate manual-entity-anonymisationclean - 9.3 Manual smoke against dev stack (Postman flow end-to-end)
- 9.4 PHPCS + Conduction named-parameters rules clean
10. Cross-app coordination
- 10.1 No DocuDesk-side code change in this PR (UI consumes the endpoint in a separate change)
- 10.2 Open DocuDesk tracking issue for the operator UI work
- 10.3 No softwarecatalog / opencatalogi changes required
Full task detail with implementation notes and acceptance criteria is in openspec/changes/manual-entity-anonymisation/tasks.md on PR #1592.
Flagged future work (out of scope here):
position-aware-substitution— substitution stays value-keyed; positions are forensic markers only.presidio-overlap-dedup— Presidio detection has the same overlap-double-count behaviour as the naive matcher; this change makes manual entities smarter but doesn't fix the detection side.
Activity
feat(manual-entity-anonymisation): implement value-objects + mappers …
test(manual-entity-anonymisation): unit tests for matcher + service +…
test(manual-entity-anonymisation): mapper unit tests for findOneByVal…
docs+chore(manual-entity-anonymisation): feature doc, CHANGELOG entry…
chore(manual-entity-anonymisation): close out remaining task book (#1593
chore(manual-entity-anonymisation): record DocuDesk PoC tracking issue (
fix(manual-entity-anonymisation): derive category + updated_at server…
feat(anonymisation-poc): Anonymisation PoC flow + manual-entity action (
fix(anonymisation-poc): drop category from manual-entity modal + store (
feat(manual-entity-anonymisation): implement value-objects + mappers …
test(manual-entity-anonymisation): unit tests for matcher + service +…
test(manual-entity-anonymisation): mapper unit tests for findOneByVal…
docs+chore(manual-entity-anonymisation): feature doc, CHANGELOG entry…
chore(manual-entity-anonymisation): close out remaining task book (#1593
chore(manual-entity-anonymisation): record DocuDesk PoC tracking issue (
fix(manual-entity-anonymisation): derive category + updated_at server…