Skip to content

feat: prioritize detected faces for automatic gravity - #11

Merged
TorstenDittmann merged 5 commits into
mainfrom
feat/face-priority
Sep 9, 2026
Merged

feat: prioritize detected faces for automatic gravity#11
TorstenDittmann merged 5 commits into
mainfrom
feat/face-priority

Conversation

@TorstenDittmann

@TorstenDittmann TorstenDittmann commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • run a pinned YuNet face detector before U²-Net and use the most prominent reliable face center as gravity
  • keep U²-Net as the fallback for images without a confident face or when face inference fails
  • share one process-wide ONNX Runtime environment safely across both reusable model sessions
  • add an additive source response field, face/source metrics, configurable FACE_SCORE_THRESHOLD, model provenance, Docker packaging, and docs
  • use a conservative 0.85 default that keeps the clear human fixture while rejecting the puppy false positive seen at 0.80
  • add 10 compact, generated regression fixtures covering centered/offset faces, a three-quarter angle, partial occlusion, low light, two-face prominence, and three no-face fallback controls\n- add a responsive docs comparison for a clear face, primary selection across two faces, and the unchanged blurred-face fallback using those real regression outputs

This performs face detection only. It does not identify people or generate/store biometric embeddings. Deliberately blurred faces, including the supplied example, remain on the saliency fallback rather than lowering the threshold enough to accept false positives.

Validation

  • mise run ci
  • make test-integration with ONNX Runtime 1.23.2
  • make test-integration-fp32 with ONNX Runtime 1.23.2
  • npm run build and npx tsc --noEmit in docs/
  • generated matrix: all 7 visible-face fixtures select source=face; landscape, back-facing person, and strongly blurred face select source=saliency
  • generated matrix: all 10 mirrored variants retain the expected source; face centers reflect consistently, including selection of the larger foreground face in the two-person scene
  • real-model clear portrait: source=face, gravity (0.5100, 0.5237), confidence 0.8814
  • supplied blurred image: source=saliency, unchanged gravity (0.5119, 0.6294), confidence 1.0

The real-model suite also covers existing non-human fixtures, concurrent model sessions, cancellation, raw/multipart equivalence, and saliency fallback behavior.

Before / after verification

Each comparison runs the same checked-in fixture through saliency-only behavior on the left and face-first behavior on the right. Orange marks the old saliency point, green marks the selected primary face, yellow marks an additional detected face, and blue marks an unchanged saliency fallback. These comparison renders are attached to this description only and are not stored in the branch.

1. Centered face — saliency (0.4760, 0.6078) → face (0.4849, 0.2705)

Centered face before and after

2. Face on the left — saliency (0.2450, 0.6374) → face (0.2625, 0.2304)

Left-positioned face before and after

3. Face on the right — saliency (0.7882, 0.6162) → face (0.7758, 0.2938)

Right-positioned face before and after

4. Three-quarter profile — saliency (0.3705, 0.6348) → face (0.4268, 0.3377)

Three-quarter profile before and after

5. Partial occlusion — saliency (0.4985, 0.6113) → face (0.4956, 0.3202)

Face with glasses, hat, and scarf before and after

6. Low light — saliency (0.5939, 0.6063) → face (0.5942, 0.3759)

Low-light face before and after

7. Two faces — saliency (0.3160, 0.6148) → primary face (0.3188, 0.3368)

Two-face primary selection before and after

8. Back-facing person — saliency fallback unchanged at (0.3789, 0.6337)

Back-facing person before and after

9. Strongly blurred face — saliency fallback unchanged at (0.4884, 0.6575)

Strongly blurred face before and after

10. No-person landscape — saliency fallback unchanged at (0.7406, 0.8016)

Landscape before and after

@greptile-apps

greptile-apps Bot commented Sep 8, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds face-first automatic gravity using YuNet, retains U²-Net saliency as the fallback, shares the ONNX Runtime environment across model sessions, and exposes the selected source through the API and telemetry. Changes since the previous review move the documentation comparison images into the public asset directory and reference them using the site’s established root-absolute URL convention.

  • Adds face detection, primary-face selection, cancellation handling, and saliency fallback.
  • Adds shared ONNX Runtime lifecycle management for both reusable sessions.
  • Packages and verifies the pinned YuNet model and documents its configuration.
  • Adds integration fixtures, API coverage, metrics, and a responsive documentation comparison.

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness, security, or repository-rule issues identified.

The current implementation consistently prioritizes reliable face detections, preserves saliency fallback on errors or absent faces, safely shares the ONNX Runtime environment, and packages the required model and documentation assets.

Important Files Changed

Filename Overview
cmd/autogravity/main.go Integrates face-first analysis, saliency fallback, source reporting, configuration, cancellation, and model lifecycle handling.
internal/facedetection/model.go Implements YuNet preprocessing, inference, filtering, non-maximum suppression, coordinate conversion, and primary-face selection.
internal/ortenv/environment.go Adds synchronized reference-counted ownership of the process-wide ONNX Runtime environment.
internal/saliency/model.go Migrates the saliency session to the shared ONNX Runtime lifecycle manager.
docs/src/components/FacePriorityDemo.tsx Presents checked-in face-priority comparison assets using URL conventions compatible with the current root deployment.
cmd/autogravity/integration_test.go Adds real-model regression coverage for face selection, fallback behavior, reflection consistency, and shared runtime ownership.

Reviews (6): Last reviewed commit: "fix(docs): package comparison images wit..." | Re-trigger Greptile

@TorstenDittmann
TorstenDittmann merged commit f1a36f0 into main Sep 9, 2026
5 checks passed
@TorstenDittmann
TorstenDittmann deleted the feat/face-priority branch September 9, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant