Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
397 changes: 209 additions & 188 deletions docs/source/features/checkpoint-loading.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/source/features/sleep-wakeup.md
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,7 @@ Principal implementation locations are:
- Workers: `tensorrt_llm/executor/base_worker.py` and
`tensorrt_llm/executor/ray/gpu_worker.py`
- Allocation tagging: `tensorrt_llm/_torch/pyexecutor/py_executor_creator.py`
and `tensorrt_llm/_torch/pyexecutor/model_loader.py`
and `tensorrt_llm/_torch/models/checkpoints/memory_init.py`
- Python VMM wrapper: `tensorrt_llm/_torch/virtual_memory.py`
- CUDA VMM: `cpp/tensorrt_llm/runtime/virtualMemory.cpp` and
`cpp/include/tensorrt_llm/runtime/virtualMemory.h`
Expand Down
4 changes: 2 additions & 2 deletions docs/source/llm-api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,8 @@ The `py_executor.model_engine_creation_seconds` timing includes the total `model
| `draft_checkpoint_preparation_seconds` | Checkpoint preparation time for draft weights loaded as part of the model loader. |
| `draft_weight_population_seconds` | Weight population time for draft weights loaded as part of the model loader. |
| `draft_checkpoint_finalization_seconds` | Checkpoint finalization time for draft weights loaded as part of the model loader. |
| `post_load_processing_seconds` | Time spent in format-specific hooks and model finalization, including post-load weight transformation, quantization and memory cleanup. |
| `total_model_loading_seconds` | Overall model construction and loading interval measured after checkpoint configuration validation. It includes the named phases below. |
| `post_load_processing_seconds` | Time spent in format-specific hooks and model finalization, including post-load weight transformation, quantization, memory cleanup and other work. |
| `total_model_loading_seconds` | Overall model loading interval, from loading and validating the checkpoint configuration to the end of post-load processing. It includes the named phases above. |

`trtllm-serve` exposes the same rank-0 payload in the `startup_metrics` field of the
`GET /server_info` response:
Expand Down
332 changes: 0 additions & 332 deletions docs/source/torch/features/checkpoint_loading.md

This file was deleted.

Loading
Loading