Repository navigation
Conversation
Signed-off-by: Pengyun Lin <81065165+LinPoly@users.noreply.github.com>
|
/bot run |
|
PR_Github #276 [ run ] triggered by Bot |
| DisaggregatedParams, | ||
| ErrorResponse) | ||
| from tensorrt_llm.version import __version__ as VERSION | ||
| from ..executor import CppExecutorError |
There was a problem hiding this comment.
I think this is originally from
for handling error raised from Cpp code. By the way, I search the codebase and it seems like we don't raise such error anymore. Shall we extend the scope of this MR by removing all relative code?There was a problem hiding this comment.
Sorry I'm still confused, you mean the previous way will raise errors?
There was a problem hiding this comment.
It would handle the error from Cpp side,
There was a problem hiding this comment.
Previously LLMAPI would raise such error if it ran into an error when calling Cpp functions, and trtllm-serve caught such error, treated it as unrecoverable and then stopped the server. But looks like we don't do similar thing right now.
| else: | ||
| llm = LLM(**llm_args) | ||
|
|
||
| llm = LLM(**llm_args) |
There was a problem hiding this comment.
This is not suggested as offline discussed with @QiJune .
|
PR_Github #276 [ run ] completed with state |
|
@LinPoly As discussed offline, maybe we can close this for now? I think we don't need to worry about the relative imports for now, if it sounds good to you. |
Replace with relative import.