KeyArgo
/
smolagents
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Set ollama context lenght in example to 8192 to make it work (#252) 

* set ollama context length in example
This commit is contained in:
RolandJAAI 2025-01-17 18:41:34 +01:00 committed by GitHub
parent 58b18f5655
commit e30ec8ac91
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
docs/source/en/guided_tour.md
View File

@ -89,8 +89,9 @@ from smolagents import CodeAgent, LiteLLMModel
model = LiteLLMModel(
model_id="ollama_chat/llama3.2", # This model is a bit weak for agentic behaviours though
api_base="http://localhost:11434", # replace with remote open-ai compatible server if necessary
api_base="http://localhost:11434", # replace with 127.0.0.1:11434 or remote open-ai compatible server if necessary
api_key="YOUR_API_KEY" # replace with API key if necessary
num_ctx=8192 # ollama default is 2048 which will fail horribly. 8192 works for easy tasks, more is better. Check https://huggingface.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator to calculate how much VRAM this will need for the selected model.
)
agent = CodeAgent(tools=[], model=model, add_base_tools=True)