How to get the perplexity of the sequence #48

Activity

1. NeonBohdan commented on Jul 5, 2023

   

   NeonBohdan

   on Jul 5, 2023

   Author

   More actions

   Here is the PR that adds perplexity test over input prompt
   ggml-org/ggml#184
2. NeonBohdan commented on Jul 10, 2023

   

   NeonBohdan

   on Jul 10, 2023

   Author

   More actions

   And here one more example
   ggml-org/llama.cpp#270
3. marella commented on Jul 16, 2023

   

   marella

   on Jul 16, 2023

   Owner

   More actions

   I haven't done it myself but it should be possible to compute it using the low-level APIs llm.eval() and llm.logits.

   tokens = llm.tokenize(sentence)
   
   all_logits = []
   for token in tokens:
       llm.eval([token])
       all_logits.append(list(llm.logits))
   
   # `all_logits` can be used to compute the log probabilities of tokens.

   👍React with 👍2Reacted by NeonBohdan and Fallenhh
4. NeonBohdan

   closed this as completedon Feb 27, 2024