> ## Documentation Index
> Fetch the complete documentation index at: https://sidenet.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Working with large APIs

> Paging, field projection, fan-out, caching and tool discovery — how agents handle collections too big for a conversation.

Real business APIs return thousands of rows across dozens of pages. A model
paging by hand is slow, expensive and silently incomplete: it fetches page one,
sees twenty results, and reports "I found twenty". This page covers the
platform's answer — the single most under-used capability in the product, and
the fix for most "the agent only found six of them" incidents.

## Two layers

Every tool an agent can call — API, MCP, connected app — gets a **size guard**
unconditionally. A response that would not fit the conversation is cut at an
item boundary, the full payload is kept aside for a few minutes, and the agent
receives a note explaining what it is looking at and how to get the rest.

On top of that, **looping** is opt-in per tool. Set `loop_responses` on the
tool's entry in an agent's tool list and the tool gains a small set of
reserved arguments that let the agent walk every page, fan out over a list of
ids, and project only the fields it needs — inside one tool call.

```json theme={null}
{
  "agentTools": [
    { "tool_id": "TOOL_ID", "provider_id": "PROVIDER_ID", "loop_responses": true }
  ]
}
```

In the Studio the same switch is **Allow response looping for this tool**, on
the tool card and in the agent's tool list. Turning it on for any tool also
attaches two retrieval tools, `queryToolResult` and `fetchToolResult`, that
read a stored result without calling upstream again.

<Note>
  The flag lives on the **agent's binding**, not on the tool. Two agents can
  hold the same tool with looping on for one and off for the other.
</Note>

## The reserved arguments

Looping adds these arguments to the tool's input schema. They are stripped
before the real request is built, so none of them ever reaches your API.

| Argument     | What it does                                                                                                   |
| ------------ | -------------------------------------------------------------------------------------------------------------- |
| `_fields`    | Project every item down to the listed dot-paths. Typically a 10–50× reduction in what enters the conversation. |
| `_all_pages` | Walk every page inside one step and return the merged list with a completeness verdict.                        |
| `_for_each`  | Call once per value of a parameter and merge the rows, each tagged with the value it came from.                |
| `_filter`    | Filter the merged rows server-side before they reach the model.                                                |
| `_refresh`   | Bypass a cached result and fetch again.                                                                        |

### `_fields`

```json theme={null}
{ "status": "open", "_fields": ["id", "customer.name", "total"] }
```

Paths map over arrays along the way, and the projection is always applied when
set — not only when the result would overflow. The full, unprojected payload
is still kept aside under a `_resultId`, so the agent can come back for a
field it left out.

**Wrong paths don't fail quietly.** A path that resolves to nothing comes back
as `null`, and a path that matched nothing on *any* item is reported in
`_unmatchedFields` together with the items' real top-level keys and a sample
item. The agent corrects itself in one step instead of publishing twelve empty
rows.

### `_all_pages`

Offered only when the tool's own schema declares a page-size parameter and a
page-number or offset parameter. Recognised spellings:

| Role        | Parameter names                                                                                                                |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------ |
| Page size   | `itemsPerPage`, `items_per_page`, `perPage`, `per_page`, `pageSize`, `page_size`, `maxResults`, `max_results`, `limit`, `$top` |
| Page number | `page`, `pageNumber`, `page_number`, `pageIndex`, `page_index`                                                                 |
| Offset      | `offset`, `skip`, `$skip`, `startIndex`, `start_index`                                                                         |

The parameters may sit at the top level or one level down in a request body.
There is nothing to configure: detection reads the schema you imported.

<Warning>
  Cursor-based pagination (`nextPageToken`, `Link` headers) is not walked.
  A cursor-style endpoint still gets the size guard and `_fields`, but not
  `_all_pages`.
</Warning>

Pages are fetched sequentially so upstream rate limits are respected. The walk
stops at the first short page, and on a bounded budget:

| Bound                  | Default |
| ---------------------- | ------- |
| Pages per collection   | 20      |
| Requests per tool call | 60      |
| Items                  | 5,000   |
| Raw upstream bytes     | 24 MB   |
| Wall-clock             | 60 s    |
| Values per `_for_each` | 50      |

A walk that hits a bound **says so**. The result carries `_pagingComplete:
false`, `_likelyMoreRows: true` and a note naming the reason, so the agent
reports a partial set as partial rather than passing it off as complete.
Totals, counts and averages are only presented as sound when every page was
fetched.

### `_for_each`

```json theme={null}
{ "_for_each": { "key": "property_id", "values": ["p_1", "p_2", "p_3"] } }
```

One call per value, merged into a single list. Each row is tagged with the
value it came from (an existing field of the same name is never overwritten —
the API's own answer wins). Values are de-duplicated and coerced to the
parameter's declared type. `_for_each` and `_all_pages` combine — every page of
every value — under one shared budget.

The key must be a parameter of *this* tool. Naming one that isn't returns an
`invalid_for_each_key` error that lists the tool's real parameters. Per-value
failures are reported in `_forEachErrors` alongside the rows that succeeded;
only if every value fails does the call fail.

### `_filter`

```json theme={null}
{ "_filter": [{ "field": "status", "op": "eq", "value": "open" }] }
```

Operators: `eq`, `neq`, `gt`, `gte`, `lt`, `lte`, `contains`, `in`, `exists`.
Clauses are ANDed. `_filter` applies to the merged result of the current call
— after the walk, after the fan-out — so the model never sees the rows it did
not ask for. It is honoured but **not advertised** to the model on the source
tool, because next to your API's real filter parameters a model will reach for
the reserved one and fetch everything to filter it afterwards. Use your API's
own filters first; `_filter` is for what they cannot express.

To filter rows that were already fetched, with no upstream call at all, the
agent uses `queryToolResult` on the stored `_resultId` — see below.

## Stored results

When a response is cut, or a walk completes, the full payload is kept for
about 15 minutes, scoped to the organization and the thread. The agent
receives `_resultId` and two tools:

* **`queryToolResult`** — `fields`, an optional `filter`, `offset` and `limit`
  over the stored items. Filtering runs on the full set before paging, so a
  clause may name a field that was not projected.
* **`fetchToolResult`** — raw character slices for results that have no usable
  item boundary.

If a step ends with a truncated result and the agent has not yet queried it,
the next step is restricted to the retrieval tools. That is what stops an agent
from answering off a preview.

## Caching

A **complete** walk can be cached so that the next identical call — same tool,
same arguments, same fan-out values — is served without touching upstream.
Cache entries are scoped by whose credentials fetched them: organization-level
provider credentials share across the org, per-user runtime credentials are
private to that user (or to a tenant-wide `cache_scope` your backend asserts
at mint time). Defaults are 24 hours for org credentials and 1 hour for
runtime credentials, with a per-binding override, `dataset_ttl_hours`.

A cache hit is labelled: `_cache: { hit: true, cachedAt, expiresAt }` plus a
note telling the agent that the user can get the very latest data by
re-calling with `_refresh: true`. Re-calling with different `_fields` or
`_filter` is still a hit — the stored set is unprojected and unfiltered.

<Warning>
  **Fixed arguments hit the cache every run.** A workflow's arguments are
  baked into its definition, so its cache key is identical on every scheduled
  run — exactly the runs that most need fresh data, with no human present to
  pass `_refresh`. In workflows the cache is therefore off unless the step
  opts in with a TTL, and it is worth opting in only where re-fetching is
  expensive *and* staleness up to the TTL is genuinely acceptable.
</Warning>

Caching is enabled per deployment; in shadow mode it records walks without
serving them. Experiment runs never read from the cache.

## Live progress

During a long walk the stream carries `data-tool-loop-progress` chunks, at
most one every 750 ms, with the page and request counters (`page`,
`pagesMax`, `items`, `requests`, `requestsMax`, `elapsedMs`) and, for a
fan-out, the value currently being fetched. The same chunks flow through
streamed workflow runs. Use them to render a progress line under the tool
call; nothing in them is needed for correctness.

## Tool discovery

Connected-app toolkits can be large — Gmail alone is more than sixty tools —
and a model chooses worse from a long list. Above a budget of **15 directly
injected tools**, connected-app tools move behind two meta-tools:

* `search_tools` takes a `query` and returns the best matches with short
  descriptions.
* `load_tool` takes a `toolName` (or `toolNames`) and makes those tools
  callable for the rest of the conversation.

Toolkits are kept direct **whole**, smallest first, while they fit the budget;
the first toolkit that would not fit moves entirely to the pool. That is how a
three-tool WhatsApp toolkit stays directly callable next to a sixty-tool Gmail
one. Override per entry with `discovery: true` (always pooled) or `discovery:
false` (always direct, and counted against the budget). `list_connections` and
`connect_toolkit` are always direct.

Your own API and MCP tools are never pooled. Prune them at the agent instead —
see [Connecting your API](/docs/connecting-your-api#shaping-what-the-agent-sees).

## In workflows

A `tool` step takes the same capabilities as a `loop` block, declared once
rather than chosen per call:

```json theme={null}
{
  "id": "fetch",
  "type": "tool",
  "tool": "list_orders",
  "input": { "status": "open" },
  "loop": {
    "all_pages": true,
    "fields": ["id", "total"],
    "for_each": { "key": "property_id", "values": { "ref": "$.steps.properties.ids" } },
    "filter": [{ "field": "total", "op": "gt", "value": 100 }],
    "cache": { "ttl_minutes": 60 }
  }
}
```

The step returns the tool's own payload shape with the verdict under a single
`_loop` key — `complete`, `requests`, `itemCount`, `stoppedBy`, `note` — so a
`branch` can gate on `$.steps.fetch._loop.complete`. There is no `_resultId`
in workflows: nothing downstream of a step could redeem one, so the walk's
byte budget is the only limit. See [Building a workflow](/docs/workflows).

## Guidance the agent already has

An agent holding looping tools is given the rules above as a separate system
message: walk before you count, project before you read, report a capped walk
as partial, and never name the mechanism to the user. You do not need to
repeat any of it in your own prompt. What is worth writing is *which*
collections matter: "orders are paginated; always fetch every page before
summarising" is a better instruction than an explanation of `_all_pages`.