From c98cd1543d5b2283584bd066e815837549902736 Mon Sep 17 00:00:00 2001 From: Aymeric Date: Mon, 6 Jan 2025 18:18:03 +0100 Subject: [PATCH] Further detail doc on LLM choice --- docs/source/en/guided_tour.md | 41 ++++++++++++++++++++++------------- 1 file changed, 26 insertions(+), 15 deletions(-) diff --git a/docs/source/en/guided_tour.md b/docs/source/en/guided_tour.md index 17b319e..551e9c2 100644 --- a/docs/source/en/guided_tour.md +++ b/docs/source/en/guided_tour.md @@ -30,20 +30,21 @@ To initialize a minimal agent, you need at least these two arguments: - `tools`, A list of `Tools` that the agent can use to solve the task. It can be an empty list. You can also add the default toolbox on top of your `tools` list by defining the optional argument `add_base_tools=True`. -Once you have these two arguments, `tools` and `model`, you can create an agent and run it. +Once you have these two arguments, `tools` and `model`, you can create an agent and run it. You can use any LLM you'd like, either through [Hugging Face API](https://huggingface.co/docs/api-inference/en/index), [transformers](https://github.com/huggingface/transformers/), [ollama](https://ollama.com/), or [LiteLLM](https://www.litellm.ai/). +Hugging Face API is free to use without a token, but then it will have a rate limitation. + +To access gated models or rise your rate limits with a PRO account, you need to set the environment variable `HF_TOKEN` or pass `token` variable upon initialization of `HfApiModel`. + ```python from smolagents import CodeAgent, HfApiModel -from huggingface_hub import login - -login("") model_id = "meta-llama/Llama-3.3-70B-Instruct" -model = HfApiModel(model_id=model_id) +model = HfApiModel(model_id=model_id, token="") agent = CodeAgent(tools=[], model=model, add_base_tools=True) agent.run( @@ -68,10 +69,12 @@ agent.run( +To use `LiteLLMModel`, you need to set the environment variable `ANTHROPIC_API_KEY` or `OPENAI_API_KEY`, or pass `api_key` variable upon initialization. + ```python from smolagents import CodeAgent, LiteLLMModel -model = LiteLLMModel(model_id="anthropic/claude-3-5-sonnet-latest") # Could use 'gpt-4o' +model = LiteLLMModel(model_id="anthropic/claude-3-5-sonnet-latest", api_key="YOUR_ANTHROPIC_API_KEY") # Could use 'gpt-4o' agent = CodeAgent(tools=[], model=model, add_base_tools=True) agent.run( @@ -87,7 +90,7 @@ from smolagents import CodeAgent, LiteLLMModel model = LiteLLMModel( model_id="ollama_chat/llama3.2", # This model is a bit weak for agentic behaviours though api_base="http://localhost:11434", # replace with remote open-ai compatible server if necessary - api_key="your-api-key" # replace with API key if necessary + api_key="YOUR_API_KEY" # replace with API key if necessary ) agent = CodeAgent(tools=[], model=model, add_base_tools=True) @@ -99,13 +102,15 @@ agent.run( -#### Code execution +#### CodeAgent and ToolCallingAgent -A Python interpreter executes the code on a set of inputs passed along with your tools. +The [`CodeAgent`] is our default agent. It will write and execute python code snippets at each step. + +By default, the execution is done in your local environment. This should be safe because the only functions that can be called are the tools you provided (especially if it's only tools by Hugging Face) and a set of predefined safe functions like `print` or functions from the `math` module, so you're already limited in what can be executed. The Python interpreter also doesn't allow imports by default outside of a safe list, so all the most obvious attacks shouldn't be an issue. -You can authorize additional imports by passing the authorized modules as a list of strings in argument `additional_authorized_imports` upon initialization of your [`CodeAgent`] or [`CodeAgent`]: +You can authorize additional imports by passing the authorized modules as a list of strings in argument `additional_authorized_imports` upon initialization of your [`CodeAgent`]: ```py from smolagents import CodeAgent @@ -113,19 +118,25 @@ from smolagents import CodeAgent agent = CodeAgent(tools=[], model=model, additional_authorized_imports=['requests', 'bs4']) agent.run("Could you get me the title of the page at url 'https://huggingface.co/blog'?") ``` -This gives you at the end of the agent run: -```text -'Hugging Face – Blog' -``` > [!WARNING] > The LLM can generate arbitrary code that will then be executed: do not add any unsafe imports! -The execution will stop at any code trying to perform an illegal operation or if there is a regular Python error with the code generated by the agent. You can also use [E2B code executor](https://e2b.dev/docs#what-is-e2-b) instead of a local Python interpreter by first [setting the `E2B_API_KEY` environment variable](https://e2b.dev/dashboard?tab=keys) and then passing `use_e2b_executor=True` upon agent initialization. +The execution will stop at any code trying to perform an illegal operation or if there is a regular Python error with the code generated by the agent. + +You can also use [E2B code executor](https://e2b.dev/docs#what-is-e2-b) instead of a local Python interpreter by first [setting the `E2B_API_KEY` environment variable](https://e2b.dev/dashboard?tab=keys) and then passing `use_e2b_executor=True` upon agent initialization. > [!TIP] > Learn more about code execution [in this tutorial](tutorials/secure_code_execution). +We also support the widely-used way of writing actions as JSON-like blobs: this is [`ToolCallingAgent`], it works much in the same way like [`CodeAgent`], of course without `additional_authorized_imports` since it doesn't execute code: + +```py +from smolagents import ToolCallingAgent + +agent = ToolCallingAgent(tools=[], model=model) +agent.run("Could you get me the title of the page at url 'https://huggingface.co/blog'?") +``` ### Inspecting an agent run