Skip to content

High LLM turn count due to frequent snapshot calls when using agent-browser skills #1351

Closed

Description

@nttdata-jiang

Issue: High LLM turn count due to frequent snapshot calls when using agent-browser skills (Windows 11, LangChain E2E agent)

Summary

I migrated a local E2E test agent from LangChain + MCP Playwright tools + LLM to agent-browser skills to reduce prompt/token size.

While token usage improved, I observed a significant increase in LLM tool-interaction turns, and logs show a large portion of those turns are snapshot calls.

I’d like to ask whether there is room for improvement on the agent-browser side (or recommended best practices) to reduce the number of explicit snapshot requests required by the LLM—e.g., returning an updated element reference tree automatically after UI actions.

Background

  • I have an E2E testing agent driven by an LLM.
  • Input is a test case file with natural-language steps and expected results.
  • The agent operates a real browser to execute the test.

I originally used MCP Playwright as the browser automation toolset. To reduce token consumption, I switched browser automation to agent-browser skills.

Environment

  • OS: Windows 11
  • Node.js: >= 24.14.0
  • agent-browser: ^0.26.0 (npm dependency)
  • Orchestration: LangChain (tools) + LLM (agent)
  • Baseline toolchain: MCP Playwright tools
  • New toolchain: agent-browser (skill-driven CLI usage)

What I changed

  • Before: LLM calls MCP Playwright tools for browser operations.
  • After: LLM calls a restricted tool that executes agent-browser ... commands according to agent-browser skills get core.

Metrics (same cases + same environment)

I ran ~150 test cases under the same environment and compared the total number of LLM invocations/tool-interaction turns.

Item Agent Browser Playwright Diff (absolute) Diff (%)
LLM call count 3,351 2,028 +1,323 +65.2%

Observation

From execution logs, roughly ~30% of the tool calls are snapshot during the agent-browser run.

I understand why snapshot is required:

  • The reference tree (@eN refs) can become stale after navigation or DOM updates.
  • The official skills also recommend re-snapshotting after page changes.

However, in practice this causes a large increase in the number of LLM turns, which has a direct impact on:

  • total runtime
  • cost
  • rate-limit pressure

Question / Request

Is there an approach (or a planned feature) to reduce the number of explicit snapshot calls required by the LLM?

For example:

  1. Auto-snapshot after action commands

    • After commands like open, click, fill, select, press, etc., return an updated snapshot / ref tree automatically.
    • Even returning a lightweight “delta” or “refs only” would help.
  2. Optional mode/flag to return refs

    • Provide an opt-in mode where action commands can include updated refs in their response.
  3. Best-practice guidance

    • If the intended usage is to batch actions between snapshots, what patterns are recommended to keep reliability while reducing snapshots?

Activity

  1. changed the title [-]High LLM turn count due to frequent `snapshot` calls when using `agent-browser` skills (Windows 11, LangChain E2E agent)[/-] [+]High LLM turn count due to frequent `snapshot` calls when using `agent-browser` skills[/+] on May 13, 2026
  2. added 2 commits that reference this issue on Oct 5, 2026
    00f898a
    42e0077
  3. Brandon168 commented on Oct 27, 2026

    @Brandon168
    Member

    I measured this on 0.39.4 with a same-model A/B: Claude Sonnet 5.5 (high), the same prompt and 8 read-only tasks on SPA dashboards, a CRM, docs, GitHub and Notion, 4 runs each, all against one CDP-attached Chrome with --pin-tab.

    • A: the agent drives click/fill/press, then waits, then snapshot -i, as separate calls.
    • B: a wrapper that runs the action, waits until the page is ready (URL change after a link click, then no new elements for 250 ms, capped at 5.5 s), and returns title | url, any tab the action opened, and snapshot -i in the same call.
    correct median turns median wall time median input tokens
    A 32/32 7 35.5 s 235K
    B 32/32 4 21.0 s 136K

    The gain came from fewer turns, not smaller snapshots. On SPAs the agent in A often read the page before it rendered and spent turns looking again; on static pages (docs, GitHub) A and B were the same. In my own agent sessions over two weeks, 31% of turns only observed and 1,044 fixed sleeps followed an action.

    Would an opt-in "act, settle, then snapshot" mode fit the project, e.g. --snapshot on action commands (or a --settle wait in batch), with CLI, MCP, docs and skill updated together? Happy to send a PR if that direction works for you, or to keep it as a wrapper if not.

  4. locked and limited conversation to collaborators on Nov 3, 2026
  5. converted this issue into a discussion #2373 on Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions