Samurai AppDocs

Using the WebUI — the browser and the shared computer in the conversation's right panel

Any screen not listed here does not exist. Headings quote the screen's own wording.

Part of the conversations guide. Other topics: conversations, the input box, and sign-in: references/webui.md; documents, files, tables, memory, channels, and apps: references/webui-workspace.md; editing a file in place (「編集」): references/webui-file-edit.md; SIROE pages, blocks, and boards: references/webui-siroe.md; settings, administration, models, and triggers: references/webui-settings.md; external service connections: references/webui-connections.md; /office: references/webui-office.md; /meetings: references/webui-meetings.md; /calendar: references/webui-calendar.md.

The right panel — a website the agent operates, and the shared computer

  • When an agent is operating a website, that screen is shown in the right panel (you can interact with it, not just watch). 「作業中」 on its bar: the agent is still working; wait before touching. After you finish there (a sign-in), 「続けてもらう」 hands it back; ⤢ widens the panel; 「縮小」「拡大」 change only how large the screen looks to you, 50%–200% (pressing the percentage between them returns to 100%; in a narrow panel they are under 「⋯(その他の操作)」) — what the agent sees and clicks does not change
  • When a conversation has used a shared computer (「コンピュータ」), its desktop is shown live in the right panel's 「コンピュータ」 tab (shown while it is in use and for 1 minute after). 「操作する」 there lets you click and type on it yourself until you press 「返す」 (a sleeping computer shows 「起こして操作する」); while you hold it, Japanese can be typed directly with your IME and ⌘V / Ctrl+V pastes from your PC. If the screen freezes, press 「再接続」 (if it drops while shown, it reconnects once by itself; if it still says 「接続が切れています」, reconnect before downloading or picking a file). When done, close it with 「⋯(その他の操作)」 → 「ブラウザを停止…」 or ask 「ブラウザを止めて」 in chat — it keeps running about 10 minutes after the last operation (「あと M:SS」), and that time counts toward usage. The agent never closes it itself (closing loses the page and login)
  • A file you download on that screen is saved to the file list (up to 60 MiB per file, the same for a file you drop onto the screen or pick in 「ファイルを選択」 — all limits: references/webui-workspace-files.md) — a card in the browser panel says it was saved (with a link to download it; 「閉じる」 dismisses it), and it appears in the deliverables panel under 「受け取ったファイル」. The agent is told about it at your next message
  • When a page on that screen asks for a file (its 「ファイルを選択」 button), a card appears in the panel — pick from 「このデバイスから」 or 「アップロード済みから選ぶ」 and it goes into that field. A drag-and-drop area: drop the file onto that screen, or the panel's 「⋯」 → 「このデバイスから」/「アップロード済みから」ドロップ
  • Page dialogs (confirm / prompt / alert) do not show on that screen and count as cancelled; a panel card shows the text. Press 「OK で押し直す」 (or 「この値で押し直す」), then the page's button again within 1 minute. New tabs and popups: the screen and the agent move there, and back when they close
  • Not shown there: select lists (click, then ↑ ↓ and Enter; a panel card says so), date pickers (type the digits), color pickers, suggestions, the right-click menu. ⌘V pastes into plain fields only; leaving a page skips the "leave?" prompt. Japanese IME input does not work there: press the panel's 「文字入力」, type it, then 「ページへ送る」 (「Tab」/「Enter」 also press that key); it fills the field you last clicked, rich-text editors included
  • You can record your own work on that screen and turn it into a procedure (no input needed). Press the panel's 「⋯」 → 「録画を始める」 (not shown mid-reply), work as usual, and 「停止」 opens a review. It lists the values you typed and the values shown on screen; mark the values you want hidden first, then 「この内容で手順書を作成」 posts it to the conversation. It writes a procedure document (Markdown: each step's URL, action, element and value) into the conversation's files. Clicks that changed nothing are left out and noted, a run repeated per account is written once, and the sign-in step names the logins saved under 「ブラウザ認証」 that can sign in automatically. A card then offers 「スキル化する」 to make it a skill, with a box for fixes to the document (or ask 「録画をスキルにして」). While recording, the agent cannot use the browser. A closed review can be reopened from 「内容を確認する」 on the bar, or thrown away with 「破棄」. A recording that is stopped but never posted is deleted 7 days after it was stopped (or cut off). Screens that need your everyday Chrome login are recorded with record in the terminal

The Chrome extension — the agent operates a tab in your own Chrome

  • For when the samuraiapp CLI is hard to set up: the 「Samurai App」 Chrome extension (its side panel is an AI chat) lets the agent operate a tab in the Chrome you use every day, with your own logins
  • Install it from the Chrome Web Store: 設定 → 「PC と繋ぐ」 → 「インストール」 → 「Chrome 拡張で普段のタブを操作してもらう」 → 「ストアを開く」 opens its store page (https://chromewebstore.google.com/detail/bpgocdjpmafpmmgckfmgnjblfkihdnkn); press 「Chrome に追加」 there
  • Press the extension's icon to open the side panel. The first time, it shows 「Chrome 拡張を接続」 → 「接続する」, which registers it as a device (signed-in browser needed). It then appears under 設定 → 「PC と繋ぐ」 → 「登録した端末」 marked 「Chrome 拡張」, with only 「ブラウザ」 as what it allows; revoke it there like any device
  • The side panel then shows the chat. A conversation opened in the panel operates the tab that was active when the conversation was first opened there, shown at the top as 「操作するタブ」. Open another tab and 「このタブに切り替える」 moves the conversation to it. Ask in words that name the extension (「拡張のブラウザで」「サイドパネルのタブで」) — 「手元のブラウザ」 means the CLI's browser, and the two are not mixed
  • A conversation not opened in the side panel — a scheduled run, or one started elsewhere — opens its own tab in the background of that Chrome, as long as Chrome is running and the extension is connected. Each conversation keeps its own tabs; a tab a page opens joins them, and the agent can switch between them or close them. Tabs the extension opened close by themselves after the conversation has been idle for a while; your own tabs are left alone. It does not operate chrome:// pages, the Web Store, or Samurai App's own pages
  • While the agent operates a tab, the page shows a cursor marker where it points and a blue ring around the field it types into; both disappear about 30 seconds after the last operation
  • A skill script that uses your own browser runs through the extension when no CLI is connected (the CLI is used when both are connected). Through the extension it covers the same operations as the CLI, except that it does not sign in with saved logins
  • While it operates, Chrome shows a bar that the extension is debugging this browser — pressing its 「キャンセル」 stops it; the next operation attaches again. The same approval rules as 「手元の PC」 apply (an admin who closed 「手元の PC」 closes the extension too)
  • Through the extension the agent can do what it does on the CLI's browser — download, upload files, save as PDF, click by position, drag, run scripts on the page. A file it downloads also stays in that Chrome's Downloads folder. Not through the extension: automatic sign-in with saved logins (your own sign-ins are used as they are), saving into a folder on your PC, and recording
  • 「Samurai App に接続できていません」 at the top of the panel: the connection dropped or the device was revoked. Press 「再接続」 and 「接続する」 again; the new registration replaces the old one