feat: native file drop and image paste verbs (drag carries no files, clipboard is text-only)

[oconnorjoseph]

Summary

Two gestures a QA agent is routinely asked to perform have no native verb: dropping a file from the OS onto a drop zone, and pasting an image from the clipboard. upload <sel> <files...> covers the third file gesture (setting files on an <input type="file"> through DOM.setFileInputFiles), but a drop zone with no file input, and a paste handler reading clipboardData.items, are both unreachable.

At 0.33.0:

• drag <src> <dst> dispatches pointer events between two elements and carries no DataTransfer.files, so a drop listener reading event.dataTransfer.files sees an empty list.
• clipboard write <text> takes a string only, and clipboard paste simulates Ctrl+V of that text, so a paste listener reading clipboardData.items[i].getAsFile() gets nothing.

Proposed verbs

agent-browser drop <sel|@ref> <files...>     # file drop onto an element
agent-browser paste <sel|@ref> --file <path>  # image/file paste at an element

drop could use Input.dispatchDragEvent with DragData.files so Chrome builds the DataTransfer itself (the highest-fidelity path; Input.setInterceptDrags is the related CDP surface). Paste has no CDP primitive that carries a file, so the workable implementation is the same one Playwright documents for file drops: read the file, construct a File in-page inside a DataTransfer, and dispatch a synthetic DragEvent/ClipboardEvent at the element via Runtime.evaluate. The event carries isTrusted: false, which most drop and paste handlers do not read.

Workaround in use

A wrapper script that does exactly that in-page dispatch through eval --stdin --json (stdin rather than argv, because a base64-embedded file exceeds the per-argument limit on Windows cmd.exe and Linux's 128 KiB MAX_ARG_STRLEN). Verified against Chrome for Testing 152 on Linux: a synthesized DragEvent('drop', { dataTransfer }) delivers the File to a native drop listener (defaultPrevented: true, files.length === 1), and a ClipboardEvent('paste', { clipboardData }) delivers it through items[i].getAsFile(). eval --stdin accepted a 1 MB script in ~200 ms.

Environment

• agent-browser 0.33.0 (also checked 0.34.0 through 0.36.0 release notes; no file-drop or image-paste verb)
• Chrome for Testing 152.0.7977.82
• Linux x64, Node 22

Related

• feat: File download and upload improvements #192 covers picker upload and download improvements, not drop or paste.

[holny]

Happy to pick this up. Confirmed both gaps on main: handle_drag (cli/src/native/actions.rs:10032) only dispatches Input.dispatchMouseEvent, so a drop handler sees an empty dataTransfer.files; and handle_clipboard's paste path (:8418) just sends Ctrl+V of text, so clipboardData.items stays empty for files.

Plan is roughly what you sketched: drop via Input.dispatchDragEvent with DragData.files where the zone accepts an intercepted drag, falling back to the in-page File + DataTransfer synthesis for zones that don't; paste --file <path> building a synthetic ClipboardEvent in-page through Runtime.evaluate, with the file bytes sent over stdin rather than argv for the arg-length reason you hit. I'll prototype on macOS against Chrome for Testing and flag it if the drop path really needs Input.setInterceptDrags to stay faithful.