Skip to content

Does **tsrt-whisper** support **tensor parallelism** and **pipeline parallelism**? I would like to distribute the model across multiple GPUs to improve inference speed. #2967

New issue
New issue
Closed
Closed
Does **tsrt-whisper** support **tensor parallelism** and **pipeline parallelism**? I would like to distribute the model across multiple GPUs to improve inference speed.#2967

Description

@Banner-Wang
Banner-Wang
opened on Mar 21, 2025
No description provided.

Activity

  1. MahmoudAshraf97 commented on Mar 24, 2025

    @MahmoudAshraf97
    MahmoudAshraf97
    on Mar 24, 2025
    Contributor

    Whisper can already fit on a single GPU, using any sort of parallelism other than data parallelism will introduce unnecessary overhead

  2. Banner-Wang
    closed this as completedon Mar 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions