Add Sapiens2 Model by guarin · Pull Request #45919 · huggingface/transformers

guarin · 2026-05-12T18:20:58Z

What does this PR do?

Adds the new Sapiens2 model from Meta

There is an open PR for the original Sapiens model (v1) from 2024: #33167 I started from scratch for v2 as it supersedes the old version.

Sapiens2 repo: https://github.com/facebookresearch/sapiens2

TODO before merge

Open PRs on the hub:

Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result,
we ask that new users do not submit pure code agent PRs at this time.
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously.

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result,
this policy is likely to be updated regularly in the near future. For more information, please read CONTRIBUTING.md.

I confirm that this is not a pure code agent PR.

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline,
Pull Request section?
Was this discussed/approved via a Github issue or the forum? Please add a link
to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

HuggingFaceDocBuilderDev · 2026-05-12T18:31:41Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

…into add-sapiens2

molbap

Nice! Left a small initial review

…into add-sapiens2

guarin · 2026-06-02T16:42:55Z

run-slow: auto, sapiens2

github-actions · 2026-06-02T16:44:28Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/auto", "models/sapiens2"]
quantizations: []

github-actions · 2026-06-02T16:56:59Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	902bf705	workflow commit (merge commit)
PR	f5eb93a0	branch commit (from PR)
main	31637182	base commit (on `main`)

⚠️ Model CI failed to report results

The test failure analysis could not be completed. Please check the workflow run for details.

…into add-sapiens2

guarin · 2026-06-02T17:46:16Z

run-slow: auto, sapiens2

github-actions · 2026-06-02T17:47:44Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/auto", "models/sapiens2"]
quantizations: []

github-actions · 2026-06-02T18:05:13Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	7316ffa9	workflow commit (merge commit)
PR	54cdcacd	branch commit (from PR)
main	a46a7325	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

guarin · 2026-06-03T09:09:25Z

run-slow: auto, sapiens2

github-actions · 2026-06-03T09:10:24Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto, sapiens2

github-actions · 2026-06-03T09:10:28Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto, sapiens2

github-actions · 2026-06-03T09:10:50Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/auto", "models/sapiens2"]
quantizations: []

github-actions · 2026-06-03T09:35:42Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	0d3d550d	workflow commit (merge commit)
PR	b1a6041c	branch commit (from PR)
main	595721c4	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

* Add sapiens2 * Add attention * Add todo * Update norm * fix-repo * Amend * Fix tests * Update license * Fix docstrings * Add backbone test * Fix rescale * Add checkpoint conversion * Add expected logits * Remove unused class definitions * Default to bf16 for rope * Update layer_norm_eps * Update test * Update rope and test * Format * Handle layernorm * Add layer_types * Update image loading in test * Add docs * Update config attributes * Add semantic segmentation * Update logits for 1024x768 * Update logits for 1024x768 * Update seg conversion * Update seg inference tests * Add segmentation docs * Add masks preprocessing * Move head args to config * Add pose head * Add pose mapping * Add pose image processing * Update test inference configs * Fix test * Update test configs * Update expected heatmap size * Fix pose processing * Add to ignore_non_auto * Fix test * Format * Update kv group handling * Update conv layer * Remove pose head * Deduplicate image processing code * Fix modeling * Fix mapping * Move image processing to top * Remove todo * Add normal estimation * Fix normal * Add normal test logits * Remove original_sizes * Add pointmap estimation * Update pointmap tests * Refactor scale head * Adjust tol * Cleanup * Add matting * Update tests * Move comment * Fix head inits * Refactor heads * Fix test * Speed up tests * Fix test * More test cleanup * Add cuddn.deterministic = True * Add matting tests * Use num_key_value_heads_per_layer * Cleanup tests * Cleanup docstrings * Cleanup docstrings * Update docs * Add flip_pairs * Remove check * Remove todos * Add matting composite * Add flip test * Add matting background test * Fix test * Fix test * Simplify rope * Fix test * Try replace periods with inv_freq * Try replace periods with inv_freq * Inherit from DINOv3RopePositionEmbedding * Fix rope dtype * Fix rope * Update docs * Use revision in tests * Remove default head configs * Update automodels * Fix pose bbox output * Add target_sizes to post_process_pose_estimation * Fix docs * Format * Add flip_pairs to config * Use dinov3 rope * Fix * Fix * Remove cv2 dependency from processing * Remove np dependency from processing * Cleanup * Batch pose pre- and post-processing * Fix tests * Fix pose bbox postprocessing * Cleanup * Batch crop_and_resize * Add outputs_flipped * Return normals as dict * Return dict from pointmaps * Cleanup * Update docs * Remove cxcywh in function names * Clarify boxes shapes * Cleanup hessian computation * Cleanup processor inheritance * Cleanup processor inheritance * Rename Matting to ImageMatting following VitMatte * Inherit vitpose output * Cleanup * Use LlamaRMSNorm * Merge conv layers * Merge PixelShuffle * Merge heads * Refactor final layer * Add head_scale_final_input_size to config * Rename to model * Cleanup _init_weights * Remove post_init() * Cleanup tests * Add head_config * Speed up postprocessors * Add flip_pairs to config * Fix head config docstring * Add head config to docs * Cleanup * Add auto models * Use analytical hessian solution * Rename boxes * Update mask_token handling * Update config * Update configs * Update docs * Cleanup patch size handling * Use hfoption * Cleanup var names * Fix auto * Fix automodel * Adress review * Cleanup docs * Cleanup scale_final_input_size init * Add spacing * Cleanup * Remove layer_norm_eps and apply_layernorm from config * Update Sapiens2ConvLayer * Add decorators for task model forward again * Update docs * Cleanup boxes * Remove last_hidden_state from backbone output * Remove redundant test * Added PPOCRV5ServerDetConvBatchnormLayer again * Remove revision * Remove hopper expectation in tests * Fix DataParallel forward * Ignore missing mask token * Update publish date * fix: Upload test images to hf-internal-testing

guarin added 10 commits May 12, 2026 15:46

Add sapiens2

c3b05fd

Add attention

2df762a

Add todo

c756c60

Update norm

b375de0

fix-repo

10b9022

Amend

9f9926c

Fix tests

25f81fb

Update license

34d653a

Fix docstrings

1a93b48

Add backbone test

7c1d3bd

guarin added 5 commits May 13, 2026 08:49

Fix rescale

3673968

Add checkpoint conversion

1a19282

Merge branch 'main' into add-sapiens2

cae8877

Add expected logits

0f3ef31

Merge branch 'add-sapiens2' of https://github.com/guarin/transformers …

1a08ee5

…into add-sapiens2

molbap reviewed May 13, 2026

View reviewed changes

guarin commented May 13, 2026

View reviewed changes

Comment thread tests/models/sapiens2/test_modeling_sapiens2.py Resolved

guarin added 12 commits May 13, 2026 15:33

Remove unused class definitions

8b45e17

Merge branch 'main' into add-sapiens2

8928975

Merge branch 'add-sapiens2' of https://github.com/guarin/transformers …

0f8f772

…into add-sapiens2

Default to bf16 for rope

dbe697b

Update layer_norm_eps

08315d4

Update test

957268c

Update rope and test

05f528d

Format

07ea916

Handle layernorm

684e071

Add layer_types

fbe1558

Update image loading in test

be1d5c6

Add docs

b1d62f8

guarin added 5 commits June 2, 2026 17:59

Remove hopper expectation in tests

52211c3

Fix DataParallel forward

2d1afa5

Ignore missing mask token

6e7d61c

Merge branch 'add-sapiens2' of https://github.com/guarin/transformers …

4f44bd3

…into add-sapiens2

Merge branch 'main' into add-sapiens2

54cdcac

guarin added this pull request to the merge queue Jun 3, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 3, 2026

guarin added 3 commits June 3, 2026 09:33

Update publish date

d3db9ea

fix: Upload test images to hf-internal-testing

9f022cc

Merge branch 'main' into add-sapiens2

b1a6041

guarin added this pull request to the merge queue Jun 3, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 3, 2026

guarin added this pull request to the merge queue Jun 3, 2026

Merged via the queue into huggingface:main with commit e820678 Jun 3, 2026
119 checks passed

guarin deleted the add-sapiens2 branch June 3, 2026 10:22

LinZiyuu mentioned this pull request Jun 25, 2026

Reject path traversal in the transformers_weights config field #46890

Merged

2 of 5 tasks

Conversation

guarin commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Code Agent Policy

Before submitting

Who can review?

Uh oh!

HuggingFaceDocBuilderDev commented May 12, 2026

Uh oh!

molbap left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

guarin commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

CI Results

Commit Info

Uh oh!

guarin commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

Uh oh!

github-actions Bot commented Jun 2, 2026

CI Results

Commit Info

Uh oh!

Uh oh!

guarin commented Jun 3, 2026

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

github-actions Bot commented Jun 3, 2026

Uh oh!

github-actions Bot commented Jun 3, 2026

CI Results

Commit Info

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

guarin commented May 12, 2026 •

edited

Loading