[drbh]

this pr fixes a race when parallel ci jobs upload the first build of a new version https://github.com/huggingface/kernels-community/actions/runs/42791271906/job/129745361789

both jobs see the branch as missing and try to create it, so the slower one fails

Error: Cannot create branch `v4`
    Conflict: Reference refs/heads/v4 already exists. Use overwrite=true to force it.

a 409 from create_branch is now treated as an existing branch

that alone is not enough. a new branch starts as a copy of main, which has the same variant dirs (e.g. build/torch-rocm). previously the job that created the branch deleted every build/ file it was not uploading, but it computed those deletes before its upload and applied them after. a sibling job that committed in between would lose its variant

new branches are now cleaned up right after creation in a deletes-only commit pinned to the base commit with parent_commit. after that every upload only touches its own variant dirs. if another job commits first the cleanup is refused and the inherited files are kept

the flow for each job is now

• list refs, note if the version branch exists and the main commit (base_commit)
• if the branch is missing, create it from base_commit
  • created: commit deletes of the inherited build/ and benchmarks/ files with parent_commit = base_commit. if refused, another job already committed, so warn and keep them
  • 409: another job created it, continue as an existing branch
  • other errors fail as before
• with --create-pr, find an open upload pr to update
• list the current files on main and the version branch (or the open pr ref)
• add the files of this job's variants and only delete stale files under its own build/<variant>/
• commit in batches of up to 1000 operations, directly or as a pr

note: the hub calls are not tested since the upload tests do not mock the hub client. new_branch_cleanup_ops and the per-variant cleanup have unit tests