Repository navigation

NOT MEANT FOR REVIEW YET [TRTLLM-6121] TRTLLM Sampler PP support - #6415

Closed

netanel-haber wants to merge 29 commits into

NVIDIA:mainfrom

netanel-haber:user/nhaber/fix/TRTLLM-6121-trtllm-sampler-pp-support

Closed

NOT MEANT FOR REVIEW YET [TRTLLM-6121] TRTLLM Sampler PP support#6415
netanel-haber wants to merge 29 commits into
NVIDIA:mainfrom
netanel-haber:user/nhaber/fix/TRTLLM-6121-trtllm-sampler-pp-support

Commits on Aug 5, 2025
- Make TRTLLM Sampler the default.
  dcampora
  committed
  0a8caed
- Fix default.
  dcampora
  committed
  2877c7c
- Fix test_return_logits.
  dcampora
  committed
  a336a00
- Wider support of executor_config.speculative_config.
  dcampora
  committed
  77d3cfe
- Set max batch size to 64 in test_llm_api_pytorch qwen
  dcampora
  committed
  d2dda76
- Fix run of tests against spec decoding)
  dcampora
  committed
  46322e1
- Fix max in max_attn_window values.
  dcampora
  committed
  829339b
- Formatting.
  dcampora
  committed
  e54b466
- Remove enable_trtllm_sampler option.
  dcampora
  committed
  47578b9
- Unwaive qwen test, adapt sampler output.
  dcampora
  committed
  08099fa
- Adapt e2e test format.
  dcampora
  committed
  225c84e
- Use Qwen3/Qwen3-0.6B-Base instead of TinyLlama for its longer max_position_embeddings.
  dcampora
  committed
  76de001
- Remove unnecessary enable_torch_sampler in test.
  dcampora
  committed
  3de720f
- Removed unnecessary setting of Torch sampler to false.
  dcampora
  committed
  30899b6
- Change enable_torch_sampler to use_torch_sampler.
  dcampora
  committed
  78b7cf8
- Fix remaining single gpu issues.
  dcampora
  committed
  957fa95
- Fix remaining failing tests.
  dcampora
  committed
  d200cd8
- Apply test correction.
  dcampora
  committed
  29e5aba
- test_disaggregated_torch_sampler
  dcampora
  committed
  b07a850
- Use torch sampler for tests that use large models.
  dcampora
  committed
  f462ce4
- refactor: Rename maxBatchSize to maxNumSequences in decoder and gptDecoder classes
  
  Funatiq
  authored and
  dcampora
  committed
  761b09e
- finalize_events: dict[str, CudaEvent]** | None = None**
  
  netanel-haber
  authored and
  dcampora
  committed
  0a3472b
- wording
  
  netanel-haber
  authored and
  dcampora
  committed
  0817776
- Adapt to use_torch_sampler.
  dcampora
  committed
  179e3a0
- Fix test_openai_chat_multimodal.
  dcampora
  committed
  35f0c17
- setup is_attention_dp_dummy=True gen requests
  netanel-haber
  committed
  dacf557
- fix dc846955207768a34a59486aad398dad995e96ae pkl5.Intracomm.Request has def wait, not def Wait
  netanel-haber
  committed
  b3411bb
Commits on Aug 6, 2025
- minimal disagg fix
  netanel-haber
  committed
  68a69bf
- Merge branch 'main' into user/nhaber/fix/TRTLLM-6121-trtllm-sampler-pp-support
  netanel-haber
  committed
  2834cb1

Navigation Menu

Repository navigation

NOT MEANT FOR REVIEW YET [TRTLLM-6121] TRTLLM Sampler PP support - #6415

NOT MEANT FOR REVIEW YET [TRTLLM-6121] TRTLLM Sampler PP support#6415
netanel-haber wants to merge 29 commits into
NVIDIA:mainfrom
netanel-haber:user/nhaber/fix/TRTLLM-6121-trtllm-sampler-pp-support

Commits on Aug 5, 2025

Make TRTLLM Sampler the default.

Fix default.

Fix test_return_logits.

Wider support of executor_config.speculative_config.

Set max batch size to 64 in test_llm_api_pytorch qwen

Fix run of tests against spec decoding)

Fix max in max_attn_window values.

Formatting.

Remove enable_trtllm_sampler option.

Unwaive qwen test, adapt sampler output.

Adapt e2e test format.

Use Qwen3/Qwen3-0.6B-Base instead of TinyLlama for its longer max_position_embeddings.

Remove unnecessary enable_torch_sampler in test.

Removed unnecessary setting of Torch sampler to false.

Change enable_torch_sampler to use_torch_sampler.

Fix remaining single gpu issues.

Fix remaining failing tests.

Apply test correction.

test_disaggregated_torch_sampler

Use torch sampler for tests that use large models.

refactor: Rename maxBatchSize to maxNumSequences in decoder and gptDecoder classes

finalize_events: dict[str, CudaEvent] | None = None

wording

Adapt to use_torch_sampler.

Fix test_openai_chat_multimodal.

setup is_attention_dp_dummy=True gen requests

fix dc846955207768a34a59486aad398dad995e96ae pkl5.Intracomm.Request has def wait, not def Wait

Commits on Aug 6, 2025

minimal disagg fix

Merge branch 'main' into user/nhaber/fix/TRTLLM-6121-trtllm-sampler-pp-support