[Feature][AutoDeploy]: Support custom sharding config #8948

Activity

1. greg-kwasniewski1

   self-assigned this

   on Nov 5, 2025
2. greg-kwasniewski1

   added this to AutoDeploy Boardon Nov 5, 2025
3. greg-kwasniewski1

   added

   feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

   on Nov 5, 2025
4. github-project-automation

   moved this to Backlog in AutoDeploy Boardon Nov 5, 2025
5. github-actions

   added

   AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

   on Nov 5, 2025
6. coderabbitai commented on Nov 5, 2025

   

   coderabbitaiBot

   on Nov 5, 2025 – with coderabbitai

   Last edited by coderabbitai

   Contributor

   More actions

   A summary of the changes CodeRabbit can apply:

   • Implement custom sharding support by adding a CUSTOM option to ShardingConfigSource (tensorrt_llm/_torch/auto_deploy/models/factory.py), extending ShardingConfig with custom_config, sharding_priority, _validate_custom_config, get_active_config and refactored validation (tensorrt_llm/_torch/auto_deploy/utils/sharding_utils.py), adding custom_sharding_config and sharding_priority fields to ShardingTransformConfig (tensorrt_llm/_torch/auto_deploy/transform/library/sharding.py), updating Sharding._apply to accept and set custom_sharding_config into shared_config.sharding_config and use priority-based get_active_config logic, and renaming/updating detect_sharding_from_factory_config to detect_sharding_from_config to accept both HUGGINGFACE and CUSTOM — enabling user-provided sharding with configurable precedence while remaining backward compatible.

   • Update tensorrt_llm/_torch/auto_deploy/models/factory.py to document and add a CUSTOM ShardingConfigSource enum value, and modify tensorrt_llm/_torch/auto_deploy/utils/sharding_utils.py to introduce custom_config and sharding_priority fields, refactor validate_config to choose and validate configs by priority, add _validate_factory_config and _validate_custom_config helpers, and add get_active_config to return the active sharding plan (falling back to heuristics) — enabling user-provided sharding, prioritized selection, and clearer validation and logging.

   • ✅ Create PR with these edits
   • 📋 Get copyable edits
7. greg-kwasniewski1

   moved this from Backlog to In progress in AutoDeploy Boardon Nov 5, 2025
8. greg-kwasniewski1 commented on Nov 13, 2025

   

   greg-kwasniewski1

   on Nov 13, 2025

   CollaboratorAuthor

   More actions

   PR #9143
9. lucaslie

   linked a pull request that will close this issue[#8948][feat] Support custom sharding config #9143on Nov 13, 2025
10. greg-kwasniewski1

    moved this from In progress to In review in AutoDeploy Boardon Nov 14, 2025
11. greg-kwasniewski1

    added a commit that references this issue on Nov 28, 2025

    [#8948][feat] Support custom sharding config (#9143)

    ...

    cff54fc
12. greg-kwasniewski1

    closed this as completedin #9143on Nov 28, 2025
13. github-project-automation

    moved this from In review to Done in AutoDeploy Boardon Nov 28, 2025
14. MinaHuai

    added a commit that references this issue on Dec 10, 2025

    [TRTLLM-7963][feat] Cold L2 cache when doing autotune benchmarking. (N…

    ...

    3ff5534
15. codego7250

    added a commit that references this issue on Dec 11, 2025

    [NVIDIA#8948][feat] Support custom sharding config (NVIDIA#9143)

    ...

    dfe2546