# `SuperPi.Plugins.Web`
[🔗](https://github.com/oleksandr/super-pi-beam/blob/v0.1.0/lib/super_pi/plugins/web.ex#L1)

Web access: `web_fetch`, and `web_search` when a search provider is configured.

Used by the Assistant harness (`{Web, tools: [:fetch, :search]}`) and the Librarian (bare).

## Tools

- `web_fetch` reads one URL and returns it as markdown (default), plain text or raw text.
  Arguments: `url`, `format`, `max_chars`, `offset` (to page a long page) and `artifact` (save
  the full text under a name instead of returning it). There is no `headers` argument: an
  arbitrary header is a way to inject credentials or forge a request, and the tool refuses the
  argument if a model sends it.
- `web_search` runs a query on a provider (`SuperPi.Web.Search`). It is added to the tool list
  only when a provider resolves, so the model never sees a search tool that cannot work.

Both tools declare the effect `net_read`. `SuperPi.Plugins.Permissions` therefore allows them in
`plan` mode (unless `network_in_plan_mode` is false), `accept-edits` and `auto`, and refuses
them under `network: "deny"`. Under `network: "allowlist"` the `url` argument decides, so
`web_search`, which has none, is refused. `SuperPi.Env.Workspace` hides both from a workspace
configured with `"network" => false`. The artifact save is internal to the tool and not a
second effect.

The tools are not in the global tool catalog. The plugin adds the specs to `agent_cfg["tools"]`
in `before_agent_start` and answers the calls in `wrap_tool`, so another harness on the same
node never sees them.

## Result channels

- `output`: title, final URL and the page text, with a note that names the offset to continue
  from when the text is cut.
- `structured_output`: `final_url`, `status`, `content_type`, `bytes`, `chars`, `truncated`,
  `title`, up to 30 absolute `links` (enough to crawl without a crawl tool), `artifact`,
  `offset` and `next_offset`.
- `details`: the redirect chain, the resolved address, the duration and `cache_hit` (no cache
  exists yet, so it is always false).

A 4xx or 5xx status, a refused content type, a policy refusal, an unreadable page or a timeout
sets `is_error`, with the reason in `output`.

## Size and artifacts

The page is cut at `max_chars` (default 20,000, at most 100,000). A text longer than
`auto_artifact_chars` (200,000) is also saved in full as the artifact `web/<host>-<hash>.<ext>`
and the note names it. An artifact holds the converted text in the requested format. A PDF or
other binary file is refused unless `artifact` is given, and is then saved byte for byte.
Fetches run inline in the tool task and are not durable: a half-read socket cannot resume and a
GET is cheap to repeat. Crawling many pages goes through `delegate_subagent`.

## Options

- `tools:` (default `[:fetch, :search]`). `web_fetch` is always offered. `:search` gates
  `web_search`.
- `search:` a provider, see `SuperPi.Web.Search`.
- `fetch:` options for `SuperPi.Web.Fetch` (`max_bytes`, `timeout_ms`, `resolver`, `pin_ip`,
  `req_options`). For operators and tests, never from the model.
- `auto_artifact_chars:` and `parse_timeout_ms:` (default 10 s).

# `description`

# `id`

# `name`

# `tool_specs`

```elixir
@spec tool_specs(keyword()) :: [map()]
```

The tool specs this plugin offers for the given plugin options. `web_search` only with a usable provider.

---

*Consult [api-reference.md](api-reference.md) for complete listing*