Document OpenTelemetry (#204)

docs/source/en/_toctree.yml

@@ -8,6 +8,8 @@
   sections:
   - local: tutorials/building_good_agents
     title: ✨ Building good agents
+  - local: tutorials/inspect_runs
+    title: 📊 Inspect your agent runs using telemetry
   - local: tutorials/tools
     title: 🛠️ Tools - in-depth guide
   - local: tutorials/secure_code_execution

docs/source/en/reference/agents.md

@@ -141,3 +141,16 @@ print(model(messages, max_tokens=10))
 ```
 
 [[autodoc]] LiteLLMModel
+
+### OpenAiServerModel
+
+This class lets you call any OpenAIServer compatible model.
+Here's how you can set it:
+```py
+model = OpenAIServerModel(
+    model_id="gpt-4o",
+    base_url="https://api.openai.com/v1",
+    api_key=os.environ["OPENAI_API_KEY"],
+)
+model=LiteLLMModel("gpt-4o", api_key=os.environ["OPENAI_API_KEY"])
+```

docs/source/en/tutorials/building_good_agents.md

@@ -19,7 +19,7 @@ rendered properly in your Markdown viewer.
 
 There's a world of difference between building an agent that works and one that doesn't.
 How can we build agents that fall into the latter category?
-In this guide, we're going to see best practices for building agents.
+In this guide, we're going to talk about best practices for building agents.
 
 > [!TIP]
 > If you're new to building agents, make sure to first read the [intro to agents](../conceptual_guides/intro_agents) and the [guided tour of smolagents](../guided_tour).

docs/source/en/tutorials/inspect_runs.md

@@ -0,0 +1,103 @@
+<!--Copyright 2024 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+# Inspecting runs with OpenTelemetry
+
+[[open-in-colab]]
+
+> [!TIP]
+> If you're new to building agents, make sure to first read the [intro to agents](../conceptual_guides/intro_agents) and the [guided tour of smolagents](../guided_tour).
+
+### Why log your agent runs?
+
+Agent runs are complicated to debug.
+
+Validating that a run went properly is hard, since agent workflows are [unpredictable by design](../conceptual_guides/intro_agents) (if they were predictable, you'd just be using good old code). 
+
+And inspecting a run is hard as well: multi-step agents tend to quickly fill a console with logs, and most of the errors are just "LLM dumb" kind of errors, from which the LLM auto-corrects in the next step by writing better code or tool calls.
+
+So using instrumentation to record agent runs is necessary in production for later inspection and monitoring!
+
+We've adopted the [OpenTelemetry](https://opentelemetry.io/) standard for instrumenting agent runs.
+
+This means that you can just run some instrumentation code, then run your agents normally, and everything gets logged into your platform.
+
+Here's how it goes:
+First install the required packages. Here we install [Phoenix by Arize AI](https://github.com/Arize-ai/phoenix) because that's a good solution to collect and inspect the logs, but there are other OpenTelemetry-compatible platforms that you could use for this collection & inspection part.
+
+```shell
+pip install smolagents arize-phoenix opentelemetry-sdk opentelemetry-exporter-otlp
+```
+
+Then run the collector in the background.
+
+```shell
+python -m phoenix.server.main serve
+```
+
+Finally, set up `SmolagentsInstrumentor` to trace your agents and send the traces to Phoenix at the endpoint defined below.
+
+```python
+from opentelemetry import trace
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import BatchSpanProcessor
+
+from openinference.instrumentation.smolagents import SmolagentsInstrumentor
+from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
+from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
+
+endpoint = "http://0.0.0.0:6006/v1/traces"
+trace_provider = TracerProvider()
+trace_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint)))
+
+SmolagentsInstrumentor().instrument(tracer_provider=trace_provider)
+```
+Then you can run your agents!
+
+```py
+from smolagents import (
+    CodeAgent,
+    ToolCallingAgent,
+    ManagedAgent,
+    DuckDuckGoSearchTool,
+    VisitWebpageTool,
+    HfApiModel,
+)
+
+model = HfApiModel()
+
+agent = ToolCallingAgent(
+    tools=[DuckDuckGoSearchTool(), VisitWebpageTool()],
+    model=model,
+)
+managed_agent = ManagedAgent(
+    agent=agent,
+    name="managed_agent",
+    description="This is an agent that can do web search.",
+)
+manager_agent = CodeAgent(
+    tools=[],
+    model=model,
+    managed_agents=[managed_agent],
+)
+manager_agent.run(
+    "If the US keeps its 2024 growth rate, how many years will it take for the GDP to double?"
+)
+```
+And you can then navigate to `http://0.0.0.0:6006/projects/` to inspect your run!
+
+<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/smolagents/inspect_run_phoenix.png">
+
+You can see that the CodeAgent called its managed ToolCallingAgent (by the way, the managed agent could be have been a CodeAgent as well) to ask it to run the web search for the U.S. 2024 growth rate. Then the managed agent returned its report and the manager agent acted upon it to calculate the economy doubling time! Sweet, isn't it?

examples/inspect_runs.py

@@ -0,0 +1,42 @@
+from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import SimpleSpanProcessor
+
+from openinference.instrumentation.smolagents import SmolagentsInstrumentor
+
+from smolagents import (
+    CodeAgent,
+    DuckDuckGoSearchTool,
+    VisitWebpageTool,
+    ManagedAgent,
+    ToolCallingAgent,
+    HfApiModel,
+)
+
+# Let's setup the instrumentation first
+
+trace_provider = TracerProvider()
+trace_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter("http://0.0.0.0:6006/v1/traces")))
+
+SmolagentsInstrumentor().instrument(tracer_provider=trace_provider, skip_dep_check=True)
+
+# Then we run the agentic part!
+model = HfApiModel()
+
+agent = ToolCallingAgent(
+    tools=[DuckDuckGoSearchTool(), VisitWebpageTool()],
+    model=model,
+)
+managed_agent = ManagedAgent(
+    agent=agent,
+    name="managed_agent",
+    description="This is an agent that can do web search.",
+)
+manager_agent = CodeAgent(
+    tools=[],
+    model=model,
+    managed_agents=[managed_agent],
+)
+manager_agent.run(
+    "If the US keeps it 2024 growth rate, how many years would it take for the GDP to double?"
+)

src/smolagents/default_tools.py

@@ -293,7 +293,7 @@ class VisitWebpageTool(Tool):
             # Remove multiple line breaks
             markdown_content = re.sub(r"\n{3,}", "\n\n", markdown_content)
 
-            return truncate_content(markdown_content)
+            return truncate_content(markdown_content, 10000)
 
         except RequestException as e:
             return f"Error fetching the webpage: {str(e)}"

src/smolagents/gradio_ui.py

@@ -155,9 +155,11 @@ class GradioUI:
     def log_user_message(self, text_input, file_uploads_log):
         return (
             text_input
-            + (f"\nYou have been provided with these files, which might be helpful or not: {file_uploads_log}"
-            if len(file_uploads_log) > 0
-            else ""),
+            + (
+                f"\nYou have been provided with these files, which might be helpful or not: {file_uploads_log}"
+                if len(file_uploads_log) > 0
+                else ""
+            ),
             "",
         )