Further detail doc on LLM choice
This commit is contained in:
		
							parent
							
								
									2933ed30d5
								
							
						
					
					
						commit
						c98cd1543d
					
				|  | @ -30,20 +30,21 @@ To initialize a minimal agent, you need at least these two arguments: | ||||||
| 
 | 
 | ||||||
| - `tools`, A list of `Tools` that the agent can use to solve the task. It can be an empty list. You can also add the default toolbox on top of your `tools` list by defining the optional argument `add_base_tools=True`. | - `tools`, A list of `Tools` that the agent can use to solve the task. It can be an empty list. You can also add the default toolbox on top of your `tools` list by defining the optional argument `add_base_tools=True`. | ||||||
| 
 | 
 | ||||||
| Once you have these two arguments, `tools` and `model`,  you can create an agent and run it.  | Once you have these two arguments, `tools` and `model`,  you can create an agent and run it. You can use any LLM you'd like, either through [Hugging Face API](https://huggingface.co/docs/api-inference/en/index), [transformers](https://github.com/huggingface/transformers/), [ollama](https://ollama.com/), or [LiteLLM](https://www.litellm.ai/). | ||||||
| 
 | 
 | ||||||
| <hfoptions id="Pick a LLM"> | <hfoptions id="Pick a LLM"> | ||||||
| <hfoption id="Hugging Face API"> | <hfoption id="Hugging Face API"> | ||||||
| 
 | 
 | ||||||
|  | Hugging Face API is free to use without a token, but then it will have a rate limitation. | ||||||
|  | 
 | ||||||
|  | To access gated models or rise your rate limits with a PRO account, you need to set the environment variable `HF_TOKEN` or pass `token` variable upon initialization of `HfApiModel`. | ||||||
|  | 
 | ||||||
| ```python | ```python | ||||||
| from smolagents import CodeAgent, HfApiModel | from smolagents import CodeAgent, HfApiModel | ||||||
| from huggingface_hub import login |  | ||||||
| 
 |  | ||||||
| login("<YOUR_HUGGINGFACEHUB_API_TOKEN>") |  | ||||||
| 
 | 
 | ||||||
| model_id = "meta-llama/Llama-3.3-70B-Instruct" | model_id = "meta-llama/Llama-3.3-70B-Instruct" | ||||||
| 
 | 
 | ||||||
| model = HfApiModel(model_id=model_id) | model = HfApiModel(model_id=model_id, token="<YOUR_HUGGINGFACEHUB_API_TOKEN>") | ||||||
| agent = CodeAgent(tools=[], model=model, add_base_tools=True) | agent = CodeAgent(tools=[], model=model, add_base_tools=True) | ||||||
| 
 | 
 | ||||||
| agent.run( | agent.run( | ||||||
|  | @ -68,10 +69,12 @@ agent.run( | ||||||
| </hfoption> | </hfoption> | ||||||
| <hfoption id="OpenAI or Anthropic API"> | <hfoption id="OpenAI or Anthropic API"> | ||||||
| 
 | 
 | ||||||
|  | To use `LiteLLMModel`, you need to set the environment variable `ANTHROPIC_API_KEY` or `OPENAI_API_KEY`, or pass `api_key` variable upon initialization. | ||||||
|  | 
 | ||||||
| ```python | ```python | ||||||
| from smolagents import CodeAgent, LiteLLMModel | from smolagents import CodeAgent, LiteLLMModel | ||||||
| 
 | 
 | ||||||
| model = LiteLLMModel(model_id="anthropic/claude-3-5-sonnet-latest") # Could use 'gpt-4o' | model = LiteLLMModel(model_id="anthropic/claude-3-5-sonnet-latest", api_key="YOUR_ANTHROPIC_API_KEY") # Could use 'gpt-4o' | ||||||
| agent = CodeAgent(tools=[], model=model, add_base_tools=True) | agent = CodeAgent(tools=[], model=model, add_base_tools=True) | ||||||
| 
 | 
 | ||||||
| agent.run( | agent.run( | ||||||
|  | @ -87,7 +90,7 @@ from smolagents import CodeAgent, LiteLLMModel | ||||||
| model = LiteLLMModel( | model = LiteLLMModel( | ||||||
|     model_id="ollama_chat/llama3.2", # This model is a bit weak for agentic behaviours though |     model_id="ollama_chat/llama3.2", # This model is a bit weak for agentic behaviours though | ||||||
|     api_base="http://localhost:11434", # replace with remote open-ai compatible server if necessary |     api_base="http://localhost:11434", # replace with remote open-ai compatible server if necessary | ||||||
|     api_key="your-api-key" # replace with API key if necessary |     api_key="YOUR_API_KEY" # replace with API key if necessary | ||||||
| ) | ) | ||||||
| 
 | 
 | ||||||
| agent = CodeAgent(tools=[], model=model, add_base_tools=True) | agent = CodeAgent(tools=[], model=model, add_base_tools=True) | ||||||
|  | @ -99,13 +102,15 @@ agent.run( | ||||||
| </hfoption> | </hfoption> | ||||||
| </hfoptions> | </hfoptions> | ||||||
| 
 | 
 | ||||||
| #### Code execution | #### CodeAgent and ToolCallingAgent | ||||||
| 
 | 
 | ||||||
| A Python interpreter executes the code on a set of inputs passed along with your tools. | The [`CodeAgent`] is our default agent. It will write and execute python code snippets at each step. | ||||||
|  | 
 | ||||||
|  | By default, the execution is done in your local environment. | ||||||
| This should be safe because the only functions that can be called are the tools you provided (especially if it's only tools by Hugging Face) and a set of predefined safe functions like `print` or functions from the `math` module, so you're already limited in what can be executed. | This should be safe because the only functions that can be called are the tools you provided (especially if it's only tools by Hugging Face) and a set of predefined safe functions like `print` or functions from the `math` module, so you're already limited in what can be executed. | ||||||
| 
 | 
 | ||||||
| The Python interpreter also doesn't allow imports by default outside of a safe list, so all the most obvious attacks shouldn't be an issue. | The Python interpreter also doesn't allow imports by default outside of a safe list, so all the most obvious attacks shouldn't be an issue. | ||||||
| You can authorize additional imports by passing the authorized modules as a list of strings in argument `additional_authorized_imports` upon initialization of your [`CodeAgent`] or [`CodeAgent`]: | You can authorize additional imports by passing the authorized modules as a list of strings in argument `additional_authorized_imports` upon initialization of your [`CodeAgent`]: | ||||||
| 
 | 
 | ||||||
| ```py | ```py | ||||||
| from smolagents import CodeAgent | from smolagents import CodeAgent | ||||||
|  | @ -113,19 +118,25 @@ from smolagents import CodeAgent | ||||||
| agent = CodeAgent(tools=[], model=model, additional_authorized_imports=['requests', 'bs4']) | agent = CodeAgent(tools=[], model=model, additional_authorized_imports=['requests', 'bs4']) | ||||||
| agent.run("Could you get me the title of the page at url 'https://huggingface.co/blog'?") | agent.run("Could you get me the title of the page at url 'https://huggingface.co/blog'?") | ||||||
| ``` | ``` | ||||||
| This gives you at the end of the agent run: |  | ||||||
| ```text |  | ||||||
| 'Hugging Face – Blog' |  | ||||||
| ``` |  | ||||||
| 
 | 
 | ||||||
| > [!WARNING] | > [!WARNING] | ||||||
| > The LLM can generate arbitrary code that will then be executed: do not add any unsafe imports! | > The LLM can generate arbitrary code that will then be executed: do not add any unsafe imports! | ||||||
| 
 | 
 | ||||||
| The execution will stop at any code trying to perform an illegal operation or if there is a regular Python error with the code generated by the agent. You can also use [E2B code executor](https://e2b.dev/docs#what-is-e2-b) instead of a local Python interpreter by first [setting the `E2B_API_KEY` environment variable](https://e2b.dev/dashboard?tab=keys) and then passing `use_e2b_executor=True` upon agent initialization. | The execution will stop at any code trying to perform an illegal operation or if there is a regular Python error with the code generated by the agent. | ||||||
|  | 
 | ||||||
|  | You can also use [E2B code executor](https://e2b.dev/docs#what-is-e2-b) instead of a local Python interpreter by first [setting the `E2B_API_KEY` environment variable](https://e2b.dev/dashboard?tab=keys) and then passing `use_e2b_executor=True` upon agent initialization. | ||||||
| 
 | 
 | ||||||
| > [!TIP] | > [!TIP] | ||||||
| > Learn more about code execution [in this tutorial](tutorials/secure_code_execution). | > Learn more about code execution [in this tutorial](tutorials/secure_code_execution). | ||||||
| 
 | 
 | ||||||
|  | We also support the widely-used way of writing actions as JSON-like blobs: this is [`ToolCallingAgent`], it works much in the same way like [`CodeAgent`], of course without `additional_authorized_imports` since it doesn't execute code: | ||||||
|  | 
 | ||||||
|  | ```py | ||||||
|  | from smolagents import ToolCallingAgent | ||||||
|  | 
 | ||||||
|  | agent = ToolCallingAgent(tools=[], model=model) | ||||||
|  | agent.run("Could you get me the title of the page at url 'https://huggingface.co/blog'?") | ||||||
|  | ``` | ||||||
| 
 | 
 | ||||||
| ### Inspecting an agent run | ### Inspecting an agent run | ||||||
| 
 | 
 | ||||||
|  |  | ||||||
		Loading…
	
		Reference in New Issue