Further detail doc on LLM choice
This commit is contained in:
parent
2933ed30d5
commit
c98cd1543d
|
@ -30,20 +30,21 @@ To initialize a minimal agent, you need at least these two arguments:
|
||||||
|
|
||||||
- `tools`, A list of `Tools` that the agent can use to solve the task. It can be an empty list. You can also add the default toolbox on top of your `tools` list by defining the optional argument `add_base_tools=True`.
|
- `tools`, A list of `Tools` that the agent can use to solve the task. It can be an empty list. You can also add the default toolbox on top of your `tools` list by defining the optional argument `add_base_tools=True`.
|
||||||
|
|
||||||
Once you have these two arguments, `tools` and `model`, you can create an agent and run it.
|
Once you have these two arguments, `tools` and `model`, you can create an agent and run it. You can use any LLM you'd like, either through [Hugging Face API](https://huggingface.co/docs/api-inference/en/index), [transformers](https://github.com/huggingface/transformers/), [ollama](https://ollama.com/), or [LiteLLM](https://www.litellm.ai/).
|
||||||
|
|
||||||
<hfoptions id="Pick a LLM">
|
<hfoptions id="Pick a LLM">
|
||||||
<hfoption id="Hugging Face API">
|
<hfoption id="Hugging Face API">
|
||||||
|
|
||||||
|
Hugging Face API is free to use without a token, but then it will have a rate limitation.
|
||||||
|
|
||||||
|
To access gated models or rise your rate limits with a PRO account, you need to set the environment variable `HF_TOKEN` or pass `token` variable upon initialization of `HfApiModel`.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from smolagents import CodeAgent, HfApiModel
|
from smolagents import CodeAgent, HfApiModel
|
||||||
from huggingface_hub import login
|
|
||||||
|
|
||||||
login("<YOUR_HUGGINGFACEHUB_API_TOKEN>")
|
|
||||||
|
|
||||||
model_id = "meta-llama/Llama-3.3-70B-Instruct"
|
model_id = "meta-llama/Llama-3.3-70B-Instruct"
|
||||||
|
|
||||||
model = HfApiModel(model_id=model_id)
|
model = HfApiModel(model_id=model_id, token="<YOUR_HUGGINGFACEHUB_API_TOKEN>")
|
||||||
agent = CodeAgent(tools=[], model=model, add_base_tools=True)
|
agent = CodeAgent(tools=[], model=model, add_base_tools=True)
|
||||||
|
|
||||||
agent.run(
|
agent.run(
|
||||||
|
@ -68,10 +69,12 @@ agent.run(
|
||||||
</hfoption>
|
</hfoption>
|
||||||
<hfoption id="OpenAI or Anthropic API">
|
<hfoption id="OpenAI or Anthropic API">
|
||||||
|
|
||||||
|
To use `LiteLLMModel`, you need to set the environment variable `ANTHROPIC_API_KEY` or `OPENAI_API_KEY`, or pass `api_key` variable upon initialization.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from smolagents import CodeAgent, LiteLLMModel
|
from smolagents import CodeAgent, LiteLLMModel
|
||||||
|
|
||||||
model = LiteLLMModel(model_id="anthropic/claude-3-5-sonnet-latest") # Could use 'gpt-4o'
|
model = LiteLLMModel(model_id="anthropic/claude-3-5-sonnet-latest", api_key="YOUR_ANTHROPIC_API_KEY") # Could use 'gpt-4o'
|
||||||
agent = CodeAgent(tools=[], model=model, add_base_tools=True)
|
agent = CodeAgent(tools=[], model=model, add_base_tools=True)
|
||||||
|
|
||||||
agent.run(
|
agent.run(
|
||||||
|
@ -87,7 +90,7 @@ from smolagents import CodeAgent, LiteLLMModel
|
||||||
model = LiteLLMModel(
|
model = LiteLLMModel(
|
||||||
model_id="ollama_chat/llama3.2", # This model is a bit weak for agentic behaviours though
|
model_id="ollama_chat/llama3.2", # This model is a bit weak for agentic behaviours though
|
||||||
api_base="http://localhost:11434", # replace with remote open-ai compatible server if necessary
|
api_base="http://localhost:11434", # replace with remote open-ai compatible server if necessary
|
||||||
api_key="your-api-key" # replace with API key if necessary
|
api_key="YOUR_API_KEY" # replace with API key if necessary
|
||||||
)
|
)
|
||||||
|
|
||||||
agent = CodeAgent(tools=[], model=model, add_base_tools=True)
|
agent = CodeAgent(tools=[], model=model, add_base_tools=True)
|
||||||
|
@ -99,13 +102,15 @@ agent.run(
|
||||||
</hfoption>
|
</hfoption>
|
||||||
</hfoptions>
|
</hfoptions>
|
||||||
|
|
||||||
#### Code execution
|
#### CodeAgent and ToolCallingAgent
|
||||||
|
|
||||||
A Python interpreter executes the code on a set of inputs passed along with your tools.
|
The [`CodeAgent`] is our default agent. It will write and execute python code snippets at each step.
|
||||||
|
|
||||||
|
By default, the execution is done in your local environment.
|
||||||
This should be safe because the only functions that can be called are the tools you provided (especially if it's only tools by Hugging Face) and a set of predefined safe functions like `print` or functions from the `math` module, so you're already limited in what can be executed.
|
This should be safe because the only functions that can be called are the tools you provided (especially if it's only tools by Hugging Face) and a set of predefined safe functions like `print` or functions from the `math` module, so you're already limited in what can be executed.
|
||||||
|
|
||||||
The Python interpreter also doesn't allow imports by default outside of a safe list, so all the most obvious attacks shouldn't be an issue.
|
The Python interpreter also doesn't allow imports by default outside of a safe list, so all the most obvious attacks shouldn't be an issue.
|
||||||
You can authorize additional imports by passing the authorized modules as a list of strings in argument `additional_authorized_imports` upon initialization of your [`CodeAgent`] or [`CodeAgent`]:
|
You can authorize additional imports by passing the authorized modules as a list of strings in argument `additional_authorized_imports` upon initialization of your [`CodeAgent`]:
|
||||||
|
|
||||||
```py
|
```py
|
||||||
from smolagents import CodeAgent
|
from smolagents import CodeAgent
|
||||||
|
@ -113,19 +118,25 @@ from smolagents import CodeAgent
|
||||||
agent = CodeAgent(tools=[], model=model, additional_authorized_imports=['requests', 'bs4'])
|
agent = CodeAgent(tools=[], model=model, additional_authorized_imports=['requests', 'bs4'])
|
||||||
agent.run("Could you get me the title of the page at url 'https://huggingface.co/blog'?")
|
agent.run("Could you get me the title of the page at url 'https://huggingface.co/blog'?")
|
||||||
```
|
```
|
||||||
This gives you at the end of the agent run:
|
|
||||||
```text
|
|
||||||
'Hugging Face – Blog'
|
|
||||||
```
|
|
||||||
|
|
||||||
> [!WARNING]
|
> [!WARNING]
|
||||||
> The LLM can generate arbitrary code that will then be executed: do not add any unsafe imports!
|
> The LLM can generate arbitrary code that will then be executed: do not add any unsafe imports!
|
||||||
|
|
||||||
The execution will stop at any code trying to perform an illegal operation or if there is a regular Python error with the code generated by the agent. You can also use [E2B code executor](https://e2b.dev/docs#what-is-e2-b) instead of a local Python interpreter by first [setting the `E2B_API_KEY` environment variable](https://e2b.dev/dashboard?tab=keys) and then passing `use_e2b_executor=True` upon agent initialization.
|
The execution will stop at any code trying to perform an illegal operation or if there is a regular Python error with the code generated by the agent.
|
||||||
|
|
||||||
|
You can also use [E2B code executor](https://e2b.dev/docs#what-is-e2-b) instead of a local Python interpreter by first [setting the `E2B_API_KEY` environment variable](https://e2b.dev/dashboard?tab=keys) and then passing `use_e2b_executor=True` upon agent initialization.
|
||||||
|
|
||||||
> [!TIP]
|
> [!TIP]
|
||||||
> Learn more about code execution [in this tutorial](tutorials/secure_code_execution).
|
> Learn more about code execution [in this tutorial](tutorials/secure_code_execution).
|
||||||
|
|
||||||
|
We also support the widely-used way of writing actions as JSON-like blobs: this is [`ToolCallingAgent`], it works much in the same way like [`CodeAgent`], of course without `additional_authorized_imports` since it doesn't execute code:
|
||||||
|
|
||||||
|
```py
|
||||||
|
from smolagents import ToolCallingAgent
|
||||||
|
|
||||||
|
agent = ToolCallingAgent(tools=[], model=model)
|
||||||
|
agent.run("Could you get me the title of the page at url 'https://huggingface.co/blog'?")
|
||||||
|
```
|
||||||
|
|
||||||
### Inspecting an agent run
|
### Inspecting an agent run
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue