Make Your Agent Reliable and Frugal: Planning and Building Good Agents (smolagents, Module 7)

This is Module 7 of smolagents Mastery, a free 15-module course that takes you from your first code agent to a production-grade one. Start at Module 1 or browse the full syllabus.

The Quill you built in Module 6 works. Hand it a CSV and a question, and it answers. But pull up the trajectory with agent.replay() (the Module 6 skill) on a question with a few moving parts — “which category grew fastest from Q1 to Q4, and is that growth statistically meaningful?” — and watch what it actually does.

It calls load_dataset. Then profile_dataframe. Then it writes some pandas, hits a column that doesn’t exist, corrects itself. A step later it re-runs profile_dataframe on the same file — it forgot it already had the schema. It re-loads the CSV. Ten steps for what needed five. And every one of those steps is one call to the model: latency you wait for, tokens you pay for. On the free Hugging Face Inference tier (a tiny $0.10/month of included credits, as of smolagents 1.26.0) that waste is the difference between a lab you can finish and one that runs dry.

The model is fine. The problem is that Quill has no plan and vague instructions. This module makes Quill reliable and frugal — not by giving it more power, but by making it disciplined.

In this module

You’ll learn

1. Activate periodic planning with planning_interval, read the PlanningStep it inserts, and explain exactly when it fires.
2. Map the smolagents prompt system: PromptTemplates, the default YAML files, populate_template (Jinja2 + StrictUndefined), and code_block_tags.
3. Customize an agent the right way — instructions= (appended to the system prompt) vs. editing prompt_templates["system_prompt"] (which the docs call “generally not advised”).
4. Apply the six “building good agents” principles — starting with the one that matters most: reduce LLM calls.
5. Decide when to use an agent and, crucially, when not to (“regularize towards not using agentic behaviour”).

You’ll build — Quill stops winging it: you add planning_interval, sharpen its instructions and tool docstrings, then measure the drop in steps on a multi-step analysis — same task, fewer LLM calls.

Theory areas covered

• T7 — Planning & prompts — 8% of the course. This module owns it.
• T2 — Code-as-action & the theory canon — 10% (partial here). This module covers only T2.6 (when to use / avoid agents); the rest of T2 lives in Modules 1, 5, 10, and 15.

Prerequisites — Modules 1–6: the ReAct loop and steps (M2), the data tools @tool/Tool (M3), make_model() (M4), additional_authorized_imports (M5), and agent.memory.steps + replay() (M6). HF_TOKEN configured in .env (Module 1).

Where you are in the series

• ✅ Modules 1–6 — your first CodeAgent, the ReAct loop, the data toolbox, make_model(), sandboxing, memory + replay.
• 👉 Module 7 — Planning and building good agents. You are here.
• ⬜ Modules 8–15 — reliability, MCP, multi-agents, multimodal, RAG, deployment, telemetry, and the capstone.

Quill before this module: a CodeAgent that works but wings it — it reacts step by step, re-profiles and re-loads, chases missing columns, with generic instructions and vague tool docstrings. Quill after this module: the same agent, planned and sharpened — a periodic PlanningStep, a data-analyst brief appended to its system prompt, rewritten tool docstrings, and a bench that measures the drop in steps. This is a change in quality, not capability.

---

When to use an agent — and when not to

Before you reach for any of the levers in this module, ask the question almost no tutorial asks: should this even be an agent?

Anthropic’s “Building Effective Agents” (published 2024-12-19) draws the line precisely. A workflow is a system where “LLMs and tools [are] orchestrated through predefined code paths.” An agent is a system “where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.” Agentic systems is the umbrella term for both. The difference is who decides the next step: in a workflow, your code does; in an agent, the model does.

The smolagents docs are blunt about the consequence. Their advice, verbatim: “regularize towards not using any agentic behaviour.” And: “the best agentic systems will be the simplest.” The reasoning is mechanical. If a deterministic workflow covers every request you will ever get, then — in their words — “just code everything” and you get a system that is 100% reliable, with no LLM in the decision loop to drift, hallucinate a column, or loop. An agent only earns its place “when the pre-determined workflow falls short too often” — when you genuinely cannot enumerate the steps in advance.

Recall the agency spectrum from Module 1 (T1.2): agency is a continuum, from a single LLM call with no tools, up to a multi-agent system that decides its own control flow. Picking a level on that spectrum is the design decision. The rule is: choose the minimal level of agency that solves the problem.

Why “minimal”? Because every notch of agency has a hidden, recurring cost. Recall the core ReAct mechanic from Module 2: each step the agent takes is one LLM call. More agency means more steps, which means more LLM calls — more latency, more tokens, and a larger surface for things to go wrong. That cost is exactly why this module exists. Reliability and frugality are the same lever pulled from two ends.

Is Quill a good candidate for an agent?

Quill answers open-ended questions over arbitrary CSVs. You cannot enumerate the steps in advance: which columns to clean, which to group by, whether a t-test is even appropriate — all of that depends on the data and the question. That irreducible judgement is what justifies an agent. Good call.

But notice the part of Quill’s job that is not open-ended: load a CSV, then summarize its schema. That is the same three lines every single time. Letting the model “decide” to inspect column by column, re-load the file, and re-profile it is agency spent where none is needed. So Quill’s design rule — the one we encode this module — is: hard-code the deterministic part into a tool (load_dataset, profile_dataframe), and let the model decide only the genuinely open-ended analysis. That principle is the bridge to the rest of this module.

⚠️ Common misconception: “More agentic = better.”

The instinct, when an agent struggles, is to give it more — more tools, more freedom, a sub-agent to help. That almost always makes it worse. Every tool you add is one more thing the model can pick wrongly; every degree of freedom is one more LLM call and one more failure mode. The engineering skill is the opposite: regularize towards less agency. Hard-code what can be hard-coded, and leave the model only what genuinely needs its judgement. The self-test for every capability is: could a plain function do this instead of a tool the model chooses to call? This is the through-line of the entire module.

When one well-planned agent still isn’t enough, you split the work across a team of agents — but that is Module 10, and it costs even more LLM calls, not fewer.

---

Planning: the periodic step that saves steps

The first lever is planning_interval (int | None) — a constructor argument present on every agent: it lives on MultiStepAgent.__init__, so CodeAgent and ToolCallingAgent both accept it. Its default is None, which means no planning at all (this is exactly how Quill behaved in Modules 2–6).

When you set it, the agent periodically inserts a PlanningStep — a step that runs no tool and executes no code. Instead, the model pauses and takes stock: it states or revises a fact list about the task (what it knows, what it still needs) and writes a short plan for the next moves. The result is stored in memory as a PlanningStep (you saw this step type listed in Module 6; here you activate it). Its fields include the plan text, the input/output messages, timing, and token_usage. It does not produce an executed action — it is pure re-orientation.

Exactly when planning fires

This is a detail many tutorials get wrong, so let’s pin it to the source. In MultiStepAgent (smolagents 1.26.0, agents.py), the planning step runs when:

if self.planning_interval is not None and (
    self.step_number == 1 or (self.step_number - 1) % self.planning_interval == 0
):
    # ... run a PlanningStep

So planning fires at step 1, and then every planning_interval steps after that — not “every N steps starting from N.” For planning_interval=3:

step_number	(step - 1) % 3	Planning?
1	0	✅ yes (step 1 always plans)
2	1	no
3	2	no
4	0	✅ yes
7	0	✅ yes

The steps that plan, for interval=3, are 1, 4, 7, 10…. The model writes the plan up to a <end_plan> stop sequence: smolagents passes stop_sequences=["<end_plan>"] when it generates the plan. That token bounds the plan’s length and cleanly separates the plan from the action that follows.

Why a plan saves steps

Without a plan, the agent reasons locally — it reacts to the last observation, takes the next plausible action, and can drift into loops (re-profiling, re-loading, chasing a phantom column). A periodic plan re-centres it on the global goal: it re-reads its own facts, notices “I already loaded this file,” and skips the redundant work. Fewer redundant steps means fewer LLM calls.

How much fewer? Treat any number as reported, never guaranteed. The smolagents blog reports code agents take roughly ~30% fewer steps than JSON tool-calling on the same tasks — but that figure is measured against JSON tool-calling, not against planning specifically, so do not attribute it to planning alone. Planning is one of several reasons a code agent is leaner; the only honest claim is “it tends to help on multi-step jobs.”

Planning is itself an LLM call

Here is the trade-off you must hold in your head: a PlanningStep is itself one extra LLM call. It is re-orientation, not free thinking. On a long, multi-step job, that one call buys back several wasted action steps — a clear win. On a short job, it just adds a call with nothing to re-orient around.

My rule of thumb, encoded as Quill’s DEFAULT_PLANNING_INTERVAL = 3: on a 3–4 step task I leave planning off; from about 6 steps up, planning_interval=3 usually earns its extra call back. (When you measure, count action steps separately from planning steps — more on that honesty rule in the lab.)

flowchart LR
    T[Task] --> P1["PlanningStep<br/>(step 1: state facts + plan)"]
    P1 --> A1[ActionStep 1]
    A1 --> A2[ActionStep 2]
    A2 --> A3[ActionStep 3]
    A3 --> P2["PlanningStep<br/>(step 4: revise plan)"]
    P2 --> A4[ActionStep 4]
    A4 --> D{final_answer?}
    D -- no --> A5[ActionStep ...]
    D -- yes --> F[FinalAnswerStep]
    A5 -.-> P3["PlanningStep<br/>(step 7)"]
    P3 -.-> A6[...]

Planning fires at step 1, then every N=3 steps. Each diamond PlanningStep is one tool-free LLM call.

---

The prompt system: where an agent’s behaviour actually lives

To customize an agent correctly, you need to know where its behaviour comes from. In smolagents, almost all of it lives in the system prompt — and the system prompt is built, at construction time, from a set of templates.

PromptTemplates

PromptTemplates is the structure that holds every template an agent uses. It is a TypedDict (so it is just a typed dictionary — you access fields with ["..."]), with four top-level keys (smolagents 1.26.0, agents.py):

• system_prompt: str — the big one; the agent’s standing instructions.
• planning: PlanningPromptTemplate — the planning prompts, with sub-fields initial_plan, update_plan_pre_messages, update_plan_post_messages.
• final_answer: FinalAnswerPromptTemplate — sub-fields pre_messages, post_messages.
• managed_agent: ManagedAgentPromptTemplate — used only when an agent manages others (Module 10); ignore it for now.

You can read these on a live agent: agent.prompt_templates["system_prompt"] is the raw template string before rendering.

The default YAML files

Where do these templates come from? They ship as YAML files inside the package, in src/smolagents/prompts/ (smolagents 1.26.0):

• code_agent.yaml — the templates for a CodeAgent.
• toolcalling_agent.yaml — for a ToolCallingAgent.
• structured_code_agent.yaml — loaded when you turn on a structured-output mode (Module 8).

Quill is a CodeAgent, so it uses code_agent.yaml. You almost never edit these; you read them to understand what your agent is being told.

populate_template: Jinja2 with StrictUndefined

The templates are not static strings — they are Jinja2 templates, rendered at __init__ by populate_template. The source is short and worth reading (smolagents 1.26.0, agents.py):

def populate_template(template: str, variables: dict[str, Any]) -> str:
    compiled_template = Template(template, undefined=StrictUndefined)
    try:
        return compiled_template.render(**variables)
    except Exception as e:
        raise Exception(f"Error during jinja template rendering: {type(e).__name__}: {e}")

This is how the live data — the list of your tools, the authorized imports, the code-block tags via {{code_block_opening_tag}} — gets injected into the system prompt when the agent is built. Two things matter here:

1. StrictUndefined. If a template references a variable that wasn’t supplied, Jinja2 raises instead of silently rendering an empty string. smolagents chose this on purpose: a missing injection should fail loudly, not quietly leave a hole in your system prompt where the tool list was supposed to be.
2. This is why the injections are non-negotiable. Your tools, your imports, and the code-block tags are rendered into the prompt at construction. If you replace the prompt with a static string, none of that gets injected — the model literally no longer sees which tools exist.

The CodeAgent system prompt rules

The code_agent.yaml system prompt sets the contract the model must follow. Abbreviated, it tells the model to:

• Always produce a Thought: followed by a code block (the ReAct contract).
• Use only variables it has defined; pass tool arguments directly, not as a dict.
• State persists between code executions — a variable set in one step is still there in the next.
• Import only from the authorized list ({{authorized_imports}}).
• End the run by calling the final_answer tool.

Your instructions= (next section) are anchored inside this contract — they sharpen it, they do not replace it.

code_block_tags (volatile — date this one)

code_block_tags controls the delimiters the model uses around its code blocks. This is a volatile fact, so date it: as of smolagents 1.26.0, the default is ("<code>", "</code>") — the YAML uses {{code_block_opening_tag}} / {{code_block_closing_tag}}, which render to those <code> tags. Passing the string 'markdown' opts into Python-fenced blocks instead. From the source:

self.code_block_tags = (
    code_block_tags
    if isinstance(code_block_tags, tuple)
    else ("```python", "```")
    if code_block_tags == "markdown"
    else ("<code>", "</code>")
)

code_block_tags value	Code-block delimiters
default (omit it)	<code> … </code>
'markdown' (opt-in)	```python … ```

⚠️ Many tutorials still show the markdown ```py fences as if they were the default. They were, historically — but as of smolagents 1.26.0 the default is <code>, and markdown is an opt-in. If you see a tutorial relying on markdown fences without setting code_block_tags="markdown", it is out of date. Quill uses the <code> default and touches nothing.

instructions= vs editing the system prompt — the pivot of this module

Here is the single most important practical decision in this module. You want to specialize Quill for data analysis. You have two ways to do it. Only one is right.

instructions= is an argument on MultiStepAgent.__init__. The docstring is exact: “Custom instructions for the agent, will be inserted in the system prompt.” It is appended to the system prompt — your text is added on top of the rendered template, not in place of it. The Jinja2 injections (tools, imports, tags) all survive. This is the recommended way to specialize an agent.

The other way is to overwrite agent.prompt_templates["system_prompt"] with your own string. It is possible. The docs call it “generally not advised” — and for good reason: you lose the Jinja2 injection of the tool list, the authorized imports, and the code-block tags; you break the moment the library updates its prompt; and you destabilize the ReAct contract the rest of the agent assumes.

	What it does	Risk	Survives save / from_hub?	When to use	smolagents verdict
instructions=	Appends to the system prompt	Low — all injections preserved	Yes — it’s a first-class agent attribute	Specializing an agent	The recommended default
Editing prompt_templates["system_prompt"]	Replaces the prompt	High — drops the injected tool list / imports / code-block tags; breaks on library update	Only your edited string	Last resort, knowing the cost	”Generally not advised”

The takeaway: reach for instructions= by default. Touch the raw template only as a last resort, fully aware of what you are giving up.

⚠️ Pitfall: pasting a whole new system prompt to “fix” the agent is the number-one anti-pattern. You lose the injection of the tool list and the authorized imports — the model can no longer see which tools it has.

---

The six principles of building good agents

The smolagents docs distill “building good agents” into six principles. Here they are in order, each with the why and how Quill applies it. The first one is the master rule; the rest serve it.

1. Reduce the number of LLM calls — “Reduce the number of LLM calls as much as you can.” Every call is latency and money. Merge several tools into one (the docs’ example: a return_spot_information tool that fetches weather and distance in a single call rather than two), and prefer a deterministic function to an agentic decision wherever you can. Quill: profile_dataframe returns schema, dtypes, describe() statistics, and missing-value counts in one call — instead of letting the model inspect column by column across many steps.

2. Improve the information flow to the LLM. State the task very clearly, and make every tool print() what is useful — especially the detail of errors — inside its forward, so the failure lands in the next Observation: the model reads (the ReAct mechanic from Module 2). Quill: load_dataset prints a readable one-line summary (rows × cols, columns), not a multi-kilobyte raw dump that buries the signal.

3. Write better tools. This is theory entry T3.12, first touched in Module 3 and deepened here. A good tool has: a precise docstring that names the supported formats and the expected formats (e.g. a date string), gives an example, raises informative ValueErrors, and returns human-readable strings. Quill: the save_chart docstring states exactly what forward expects and what it returns (the saved file path).

4. Pass extra objects with additional_args. You can hand agent.run(...) a dict of extra objects — images, DataFrames, audio URLs — rather than making the agent fetch them. Quill: a DataFrame you already loaded in your code can be passed in via additional_args instead of being re-loaded inside the agent. (Image/vision depth is Module 11.)

5. Debugging. When an agent misbehaves, the two levers are: (a) a stronger model; and (b) more instructions — instructions= at init (appended, as you just learned), task-specific detail in the task string, and tool-specific detail in each tool’s description. Editing the system prompt directly is, again, “generally not advised.”

6. Extra planning — planning_interval=N. Planning is one of the six principles. The loop closes: the first lever in this module is also the last principle.

#	Principle	Why	Quill application
1	Reduce the number of LLM calls (master rule)	each step is one LLM call	profile_dataframe returns schema + dtypes + stats + missing in ONE call, not column-by-column
2	Improve the information flow to the LLM	failures must reach the next Observation:	load_dataset prints a readable summary, not a raw dump
3	Write better tools	the docstring is the interface the model reads	sharpened load_dataset / profile_dataframe / save_chart docstrings: formats, examples, return contract
4	Pass extra objects with additional_args	hand the agent images/DataFrames/URLs directly	a pre-loaded DataFrame can be passed in rather than reloaded
5	Debugging	a stronger model; more instructions	instructions= at init (appended), task detail in the task string, tool detail in its description
6	Extra planning	re-centre on the goal periodically	planning_interval=3 — planning IS one of the six principles

The three golden rules of tools, with a before/after

Principle 3 deserves a concrete look, because the docstring is the interface — it is injected straight into the system prompt, so it is literally what the model reads to decide whether and how to call your tool. The three rules: a precise docstring, a useful print(), and informative ValueErrors. Here is what sharpening save_chart’s docstring looks like:

before: "Save a chart."

after : "Save the CURRENT matplotlib figure (the one you just drew) to the
         outputs/ directory as a PNG and RETURN its saved file path as a
         string, e.g. 'outputs/category_revenue.png'. Draw your chart FIRST
         with matplotlib ... then call this — do NOT use plt.show() (it saves
         nothing) ... Raises ValueError if no figure has been drawn yet."

The “before” tells the model nothing: it doesn’t know it must draw first, that plt.show() is useless here, or that the return value is the path it needs. The “after” closes all three gaps. A precise docstring is the cheapest reliability win you will ever buy.

---

Build it: make Quill plan, sharpen it, and measure the drop

Goal: turn Quill from “works but wings it” into “planned and sharpened,” then measure the reduction in ActionSteps on a multi-step question. The observable result is a bench that prints baseline vs improved.

The full, tested code lives at smolagents-course-labs/module-07. It is a cumulative snapshot: only quill/agent.py and the docstrings in quill/tools/data.py change, plus a new quill/bench.py. The frozen contracts from earlier modules — make_model() (M4), the tool signatures (M3), the sandbox policy (M5), the callbacks (M6) — are untouched. The offline tests pass with no network and zero tokens.

Step 1 — Extend build_quill() by addition

build_quill() (frozen since Module 2) gains two keyword-only arguments. Both default to the smolagents defaults, so every prior call site behaves exactly as before — the signature is extended, never broken.

DEFAULT_PLANNING_INTERVAL = 3

def build_quill(
    model: Model | None = None,
    *,
    planning_interval: int | None = None,
    instructions: str | None = _DEFAULT,  # sentinel: omitted -> Quill's default brief
) -> CodeAgent:
    executor_type, authorized_imports = resolve_executor()
    resolved_instructions = QUILL_INSTRUCTIONS if instructions is _DEFAULT else instructions
    return CodeAgent(
        tools=[load_dataset, profile_dataframe, save_chart(),
               WebSearchTool(), VisitWebpageTool()],
        model=model or make_model(role="analyst"),
        executor_type=executor_type,
        additional_authorized_imports=authorized_imports,
        step_callbacks=quill_callbacks(),
        # Periodic planning (M7): None = off; an int inserts a PlanningStep at step 1, then every N.
        planning_interval=planning_interval,
        # M7: instructions are APPENDED to the system prompt — they DON'T replace it. We never do
        #   agent.prompt_templates["system_prompt"] = "..."   # <- "generally not advised"
        instructions=resolved_instructions,
        max_steps=8,
    )

A sentinel (_DEFAULT) keeps three intents distinguishable: omit instructions to get Quill’s default brief; pass "" (or None) for the bare smolagents prompt (the bench’s baseline); pass your own string to specialize.

Step 2 — The sharpened instructions (appended, never replacing)

Quill’s default instructions are a short data-analyst brief whose headline rule is profile once:

QUILL_INSTRUCTIONS = (
    "You are Quill, a meticulous data analyst. Work in this order:\n"
    "1. Profile the dataset ONCE with profile_dataframe before writing any analysis code. "
    "Never reload or re-profile a file you have already loaded — its DataFrame and your "
    "earlier results persist between code blocks.\n"
    "2. State a short plan, then execute it step by step with pandas/numpy.\n"
    "3. Only use column names you have actually seen in the profile; do not guess columns.\n"
    "4. Back a quantitative claim with a chart: draw it with matplotlib, then call save_chart.\n"
    "5. Finish with final_answer, naming the columns you used and the chart path you saved.\n"
    "Prefer the fewest steps that answer the question correctly."
)

This text is appended to the system prompt. You can prove it on a live agent: the brief shows up and the injected tool list is still there.

agent = build_quill()                         # default: QUILL_INSTRUCTIONS appended
sp = agent.system_prompt
assert "meticulous data analyst" in sp        # our brief landed
assert "load_dataset" in sp                   # ...AND the injected tool list survives

Step 4 (“Only use column names you have actually seen”) is what stops the “chase a phantom column” loop; step 1 (“profile ONCE”) is what stops the re-profiling waste.

Step 3 — Measure the baseline, then the improved run

quill/bench.py runs Quill twice on one question and prints the drop. The honest unit is the ActionStep count — the work the agent actually did — with PlanningSteps tracked separately (since each plan is its own LLM call). It reads agent.memory.steps (the supported way — never agent.logs, removed in 1.21.0):

def count_steps(agent: CodeAgent) -> StepCount:
    actions = sum(1 for s in agent.memory.steps if isinstance(s, ActionStep))
    plans = sum(1 for s in agent.memory.steps if isinstance(s, PlanningStep))
    return StepCount(action_steps=actions, planning_steps=plans)

The CLI builds the bare Quill (instructions="", no planning) and the improved Quill (planning_interval=3, default sharpened instructions), runs the same task on each, and renders the comparison:

with build_quill(instructions="") as baseline_agent:          # bare prompt, no planning
    baseline = run_and_count(baseline_agent, task)
with build_quill(planning_interval=DEFAULT_PLANNING_INTERVAL) as improved_agent:
    improved = run_and_count(improved_agent, task)
print(format_report(baseline, improved))

Run it

From inside module-07/ (so data/sales.csv, outputs/, and the quill package resolve). This makes real model calls, so set HF_TOKEN:

uv run python -m quill.bench \
  --dataset data/sales.csv \
  --question "Which category grew fastest from Q1 to Q4 2025, and is that growth statistically meaningful?"

Expected output (your numbers will vary — LLMs are non-deterministic):

===== STEP COMPARISON =====
Baseline  (no planning, bare instructions)        : 11 ActionSteps, ~11 LLM calls
Improved  (planning_interval=3, sharpened)        :  6 ActionSteps, ~8 LLM calls (2 of them planning)
Step reduction: ~45%   (your numbers will vary — LLMs are non-deterministic)

Read it honestly. The reduction (~45% here) is computed on ActionSteps — the work — not on “all steps.” The improved run’s total step count includes its two planning calls, which is exactly why you count action steps separately: a planned run has more total steps but does less redundant work. Counting “all steps” would punish planning dishonestly. And the number is a trend, not a guarantee — run it twice and you’ll get two different figures.

Test it

uv run pytest module-07/tests/                    # offline (no token, no Docker)
QUILL_LIVE_TESTS=1 uv run pytest module-07/tests/ # also the real planning + bench runs (needs HF_TOKEN)

The offline tests spend zero tokens. The clever bit: the shared FakeModel scripts both the plan (ending in <end_plan>) and the action, so a real PlanningStep lands in agent.memory.steps without a single token spent. They assert that planning_interval=1 inserts a PlanningStep at step 1 (and the default inserts none), that the trigger fires at steps 1, 4, 7 for interval 3, that instructions= is appended to agent.system_prompt (with the tool list intact), that the raw template still holds its Jinja2 placeholders, that the tool docstrings are non-empty and sharpened while their frozen signatures are unchanged, and that the bench counts steps honestly. Every Module 2–6 test still passes here.

Try it yourself

1. Find the planning break-even. Vary planning_interval over 1, 3, 5, and None on the same question; chart action steps vs total LLM calls. Find the point where planning stops paying for itself (hint: on a short task, planning_interval=1 adds a plan per step for no gain).
2. Constrain the order with instructions. Rewrite the brief to forbid any web search until the local analysis is done, and watch the effect in agent.replay().

Scope check. This lab does not validate the answer’s content — no structured output, no final_answer_checks, no response_format (that is Module 8; we use return_full_result only to count). It does not add a second agent (Module 10). And it never edits prompt_templates["system_prompt"] — that appears only as a commented anti-pattern.

---

In production

Step count is a line item. The number of steps an agent takes is a direct cost — you pay for every LLM call. In production you monitor it (the Monitor / TokenUsage accessors from Module 4) and you set a per-run budget. An unplanned agent that loops is money on fire; planning and a tighter prompt are how you cap that.

The free tier forces the issue. The Hugging Face Inference free tier (~$0.10/month of credits, as of smolagents 1.26.0 — subject to change) makes frugality non-negotiable from the lab onward. The principle “reduce LLM calls” isn’t an optimization; it’s the difference between a lab that runs and one that doesn’t.

Instructions and docstrings are code. Your instructions= string and your tool docstrings change the model’s behaviour as surely as a code path does. Version them, test them (the offline smoke tests assert the docstrings are sharpened and the instructions land), and review them in PRs — never tweak them live in a notebook and ship.

Weaker models need this more. On a smaller local model (Ollama via make_model’s local backend), explicit planning and explicit instructions matter more, not less — the model has less implicit judgement to fall back on, so the structure you provide carries more of the weight.

My default: before reaching for a second agent, I sharpen the first one’s prompt and tools. It’s cheaper, and a single agent is far easier to debug than a team. Structural reliability — validated schemas, error recovery — comes next, in Module 8.

---

Concept check

Why this matters

Planning, good prompts, and good tools are the three levers that turn an agent from “works sometimes” into “reliable and frugal.” They are the same lever seen from two angles: every step you save is a step you don’t pay for and a step that can’t go wrong. The hardest skill here is the contrarian one — knowing when not to use an agent at all (“regularize towards not using agentic behaviour”). That is engineering judgement, not a footnote: an agent where a deterministic workflow would do is slower, costlier, and less reliable by construction. It also reinforces a Module 2 idea — max_steps and the max_steps_error state: fewer, better-planned steps means you’re far less likely to hit the cap and burn your whole step budget on a broken loop.

Quiz

1. You have a job that extracts the same three fixed KPIs from CSVs that always arrive in the same known format. A deterministic pipeline covers 100% of the requests you’ll ever get. What should you build?

• A. An agent with a sub-agent to double-check the KPIs
• B. An agent with max_steps raised so it never runs out
• C. A deterministic workflow — don’t use an agent here
• D. An agent with a stronger model for reliability

2. You set planning_interval=3. At which step_numbers does the agent insert a PlanningStep?

• A. Steps 3, 6, 9… (every 3 starting at 3)
• B. Steps 1, 4, 7… (step 1, then every 3)
• C. Only step 0, once
• D. Every single step

3. A teammate wants to specialize Quill for financial analysis. What is the right way?

• A. Overwrite agent.prompt_templates["system_prompt"] with a finance prompt
• B. Pass a finance brief via instructions= (appended to the system prompt)
• C. Pass code_block_tags="markdown" so the model writes cleaner code
• D. Add five more tools so the agent has more options

4. An agent keeps looping because one of its tools returns None on failure with no message — the model never learns what went wrong. What is the fix?

• A. Raise max_steps so it has more attempts
• B. Add planning_interval so it re-plans
• C. Switch to a stronger model
• D. Make the tool print() the error detail and raise an informative ValueError in forward

5. A multi-step agent is burning too many LLM calls. Which lever do you pull first?

• A. Reduce the number of LLM calls — merge tools, hard-code the deterministic parts
• B. Add a PlanningStep on every step
• C. Switch code_block_tags to 'markdown'
• D. Edit the system prompt by hand

---

Answers

1 — C. When a deterministic workflow covers every request, “just code everything”: you get a 100%-reliable system with no LLM in the decision loop. An agent only earns its place when the pre-determined workflow falls short too often. A, B, and D all add agency to a problem that needs none — the exact anti-pattern this module warns against (T2.6).

2 — B. The trigger is step_number == 1 or (step_number - 1) % planning_interval == 0. For interval 3 that is steps 1, 4, 7… — step 1 always plans, then every 3 steps. A confuses “every 3” with “starting at 3”; C and D misread the trigger entirely (T7.1).

3 — B. instructions= is appended to the system prompt and is the recommended way to specialize an agent — the injected tool list, imports, and code-block tags all survive. A is “generally not advised”: it drops those injections and breaks on the next library update. C and D are unrelated to specialization (T7.7).

4 — D. The fix is “improve the information flow” + “write better tools”: the tool must print() the error detail and raise an informative ValueError, so the failure reaches the next Observation: and the agent self-corrects. Raising max_steps, adding planning, or swapping the model all leave the silent-None tool in place — the agent still can’t see what failed (T3.12, principles 2–3).

5 — A. “Reduce the number of LLM calls” is the master principle — merge tools, hard-code the deterministic parts. Planning every step (B) adds calls; code_block_tags (C) is cosmetic; editing the system prompt (D) is the anti-pattern (T7.8, principle 1).

Common pitfalls

• “More agency = better.” The central anti-pattern. The right reflex is to regularize toward less agency — hard-code the deterministic part. The module’s self-test: could a plain function do this instead of a tool the model chooses to call?
• Hand-editing the system_prompt. This breaks the Jinja2 injections (tools, authorized imports, code_block_tags) and shatters on the next library version. Use instructions= — it’s appended, so the injections survive.
• Trusting stale code_block_tags advice. Believing markdown ```py fences are the default. As of smolagents 1.26.0 the default is ("<code>", "</code>"); 'markdown' is an opt-in. Many tutorials are out of date on this.
• Counting “all steps” when you measure the gain. A PlanningStep is itself one LLM call. Count ActionSteps (the work) and track planning steps separately, or you’ll flatter or punish planning dishonestly.

---

Key takeaways

• “Regularize towards not using any agentic behaviour.” If a deterministic workflow covers every request, code it — an agent is only justified when you genuinely can’t enumerate the steps in advance.
• planning_interval fires at step 1, then every N steps — trigger step_number == 1 or (step_number - 1) % interval == 0. The model writes the plan up to the <end_plan> stop sequence.
• A PlanningStep is itself one LLM call. It’s a trade-off: worth it on long, multi-step jobs (~6 steps up), wasteful on short ones. Count ActionSteps and PlanningSteps separately when you measure.
• instructions= is appended to the system prompt — never replace it. Editing prompt_templates["system_prompt"] is “generally not advised”: you lose the Jinja2-injected tool list, imports, and code-block tags.
• The master principle is “reduce the number of LLM calls.” Merge tools, hard-code the deterministic parts, prefer a function over an agentic decision.
• A good tool has a precise docstring, print()s what helps (especially errors), and raises informative ValueErrors. The docstring is the interface the model reads.
• As of smolagents 1.26.0, code_block_tags defaults to ("<code>", "</code>"); 'markdown' is an opt-in, not the default many tutorials assume.

---

What’s next

Want the full smolagents concept-check bank and the next module in your inbox? Subscribe here — it’s free, like everything in this series.

Quill is frugal now — but is its output trustworthy? Right now it returns a free-text answer you have to take on faith. Module 8 — Reliable Agents: Structured Output, Validation, and Errors makes Quill’s report a validated schema and teaches it to recover from errors with final_answer_checks and the structured-output modes.

Links: Module 6 — Memory, State, and Inspecting Runs · Module 8 — Reliable Agents · Course index · Lab code for this module

---

References

• smolagents docs — Building good agents (the six principles; “regularize towards not using agentic behaviour”; “generally not advised”).
• smolagents docs — How do multi-step agents work? (the ReAct loop and planning).
• smolagents docs — Agents API reference (signatures for instructions, planning_interval, code_block_tags).
• Anthropic — Building Effective Agents (2024-12-19) — the workflow-vs-agent distinction.
• Founding papers (covered in Module 1): CodeAct (2402.01030) and ReAct (2210.03629).

Verified against smolagents 1.26.0. Version-dependent facts are dated “as of smolagents 1.26.0”; the library is experimental and subject to change, so re-check the source at the pinned tag if a detail looks off.