Add prefill method support without generation

[craftingmod]

Summary

• Add create_chat_prefill method which could be used to prompt evaluation.
  • Expose a prefill-only chat API that evaluates and returns the final next-token without generation.
• Add chat_template_kwargs support in create_chat_completion for overriding chat_template_kwargs each completion. : reverted

Maybe useful when implementing probability classification.

AI Disclose

I've used GPT 6 Luna to implementing features, but I manually reviewed and understood how to works.

[craftingmod]

• Why i made this PR: I want trying something like system-one in comfyui 😕
• Code quality might be bad so feel free to edit or close PR 😉

[JamePeng]

I believe a PR shouldn't include changes unrelated to the submission's purpose; additionally, using grammar constraints might be a simpler way to control the model's selective output.

[craftingmod]

I believe a PR shouldn't include changes unrelated to the submission's purpose; additionally, using grammar constraints might be a simpler way to control the model's selective output.

Additional modification is pushed.

• Removed unrelated changes. (Added because prefill requires non-thinking and should be a way to disable thinking without new llama context. Correctly removed.)
• Added Grammer parameter to create_chat_prefill, which could be used like LlamaGrammar.from_string('root ::= "Y" | "N"').

[JamePeng]

I can go ahead and merge this, but there are still several areas requiring fixes—such as the passing and handling of add_generation_prompt. Also, the handler already returns a PrefillResult containing an independent copy of the logits, yet create_chat_prefill() creates another PrefillResult after calculating the probabilities.

[JamePeng]

https://huggingface.co/blog/ggml-org/decision-models-in-llamacpp

ggml-org/llama.cpp#29831

It appears that work is underway on the underlying llama.cpp to adapt the execution logic for models similar to Jev.

[craftingmod]

https://huggingface.co/blog/ggml-org/decision-models-in-llamacpp

ggml-org/llama.cpp#29831

It appears that work is underway on the underlying llama.cpp to adapt the execution logic for models similar to Jev.

YAY

Seems should be wait until official API is introduced.