tcsenpai/ollama
1
0
Fork 0
mirror of https://github.com/tcsenpai/ollama.git synced 2025-06-07 11:45:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

runner: Initialize numPredict

Browse Source 
numPredict is used to enforce a limit on the number of tokens to
generate. Is it passed in from Ollama but it is never stored to
be checked.
This commit is contained in:
Jesse Gross 2024-08-13 10:38:03 -07:00 committed by jmorganca
parent ebdf781397
commit 0c2f95f3de
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
llama/runner/runner.go
View File

@ -91,6 +91,7 @@ func (s *Server) NewSequence(prompt string, numPredict int, stop []string, param
return &Sequence{ return &Sequence{
tokens: tokens, tokens: tokens,
n_prompt_tokens: len(tokens), n_prompt_tokens: len(tokens),
numPredict: numPredict,
responses: make(chan string, 1), responses: make(chan string, 1),
embedding: make(chan []float32, 1), embedding: make(chan []float32, 1),
samplingCtx: sc, samplingCtx: sc,