Web web
Web pages. search and the create-* jobs take every option inside body. scrape takes url, formats and only_main_content, and map takes url, limit and include_subdomains, at the top level or inside body; their other options go inside body: web(operation="scrape", url="https://...", formats=["markdown"]).
In MCP it is listed for Kenso's own apps as web(operation="..."). Other clients call its Kenso API routes directly.
A customer's own MCP connection gets map and scrape through web_read in a workspace with web reads on, 100 calls a day.
Each operation with a route calls the Kenso API at https://api.ken.so with Authorization: Bearer <Kenso API key>, Ken-Version: 1.1 and X-Ken-Workspace-Id: <workspace id>. You can call that route directly with the same headers.
Operations: create-batch-scrape, create-crawl, create-extraction, map, scrape, search, cancel, batch-status, crawl-status, extract-status, batch-wait, crawl-wait, extract-wait
create-batch-scrape
Write. POST https://api.ken.so/v1/scraping/batch-scrapes
Create a batch scrape job. Returns 202 with a batch_scrape job. Poll and cancel it through jobs.
POST /api/v1/scraping/batch-scrapes. Sends Ken-Version 1.1 and X-Ken-Workspace-Id.
Poll GET /api/v1/jobs/{jobId}?type=batch_scrape.
Cancel with POST /api/v1/jobs/{jobId}/cancel?type=batch_scrape.
create-crawl
Write. POST https://api.ken.so/v1/scraping/crawls
Create a crawl job. Returns 202 with a crawl job. Poll and cancel it through jobs.
POST /api/v1/scraping/crawls. Sends Ken-Version 1.1 and X-Ken-Workspace-Id.
Poll GET /api/v1/jobs/{jobId}?type=crawl. Cancel with POST /api/v1/jobs/{jobId}/cancel?type=crawl.
create-extraction
Write. POST https://api.ken.so/v1/scraping/extractions
Create an extraction job. Returns 202 with an extraction job. Poll it through jobs. It cannot be canceled.
POST /api/v1/scraping/extractions. Sends Ken-Version 1.1 and X-Ken-Workspace-Id.
Poll GET /api/v1/jobs/{jobId}?type=extraction. Extraction cancel returns 409 invalid_state.
map
Read. POST https://api.ken.so/v1/scraping/map
Map links on a site. Public. body is the Firecrawl map request, forwarded as JSON.
POST /api/v1/scraping/map. Sends Ken-Version 1.1 and X-Ken-Workspace-Id. `url`, `limit` (the most links to return, 1 to 100000) and `include_subdomains` go at the top level or inside body (as `url`, `limit`, `includeSubdomains`); every other Firecrawl option, such as `search`, goes inside body. A field given both ways must have one value. On a customer's own connection each call counts against the workspace's webReadCalls quota (100 map and scrape calls a day). Past it the call fails with 429 and the time the quota resets; do not retry before then.
scrape
Read. POST https://api.ken.so/v1/scraping/scrape
Scrape one page. Public. body is the Firecrawl scrape request, forwarded as JSON.
POST /api/v1/scraping/scrape. Sends Ken-Version 1.1 and X-Ken-Workspace-Id. SSRF rejection stays in the Firecrawl service. `url`, `formats` and `only_main_content` go at the top level or inside body (as `url`, `formats`, `onlyMainContent`); every other Firecrawl option goes inside body. A field given both ways must have one value. A markdown-only request without `actions` reads Kenso's web archive first. An archive hit costs nothing and carries `metadata.kenSource: "archive"` and `metadata.capturedAt` (the copy can be up to 18 months old). Other requests, and archive misses, run on Firecrawl cloud. When the result has a top-level `source`, it names who served the page: `archive`, `workspace_key` (the workspace's own Firecrawl key) or `ken_key`. `lockdown: true` asks Firecrawl for its cached copy only and never fetches the site; a cache miss is a 404. When the archive misses and live scraping is off, the call fails with "Web scraping is off right now". Do not retry that page. A plain 402 fails with "Web scraping is out of credits". On a customer's own connection each call counts against the workspace's webReadCalls quota (100 map and scrape calls a day). Past it the call fails with 429 and the time the quota resets; do not retry before then.
search
Write. POST https://api.ken.so/v1/scraping/search
Search the web through Firecrawl. Public. body is the Firecrawl search request, forwarded as JSON.
POST /api/v1/scraping/search. Sends Ken-Version 1.1 and X-Ken-Workspace-Id.
cancel
Write.
Crawl jobs: start, status, wait, or cancel a multi-page crawl.
Crawl jobs on the canonical routes: POST `/api/v1/scraping/crawls` starts a
Kenso crawl job, GET `/api/v1/jobs/{jobId}?type=crawl` reads it and POST
`/api/v1/jobs/{jobId}/cancel?type=crawl` cancels it, all with Ken-Version 1.1
and X-Ken-Workspace-Id. `client_id` names the workspace; omit it on a
connection that already has one. `crawl_id` is the Kenso job id (a number)
that `operation=wait` or `web(operation="create-crawl")` returned. Operations:
### operation=start
Start an async multi-page crawl. Requires `url`. Returns a job envelope
with `data.id`. `prompt` asks the LLM to derive crawl parameters.
`scrape_options` applies to every crawled page, for example
`{"formats": ["markdown", "links"]}`.
`limit` is the most pages to crawl: 100 when omitted, at most 1000 (a
larger value is cut to 1000). A crawl on Kenso's key holds credits for its
whole `limit` before it starts, so pass a larger `limit` only when the
site needs more than 100 pages.
### operation=status
Poll one crawl job by `crawl_id`. `data` is Firecrawl's crawl status: terminal
statuses are `completed`, `failed`, and `cancelled`; running jobs usually
return `scraping`. Each call returns one page of results. When `next` is set,
pass it back as `cursor` to read the next page; `next` is null on the last page.
Needs Search view, as `jobs(operation="get", job_type="crawl")` does.
### operation=wait
Start a crawl, then poll until completion or timeout. Same start params
as `operation=start`. `max_wait_seconds` defaults to 120 and is capped at
600. On timeout the latest status is returned with `data.id` (the job id),
a `warning` and a `hint` naming the status poll. The wait stops at the
first "Web scraping is off right now" error instead of polling. Needs Search
edit, because it starts the crawl.
### operation=cancel
Cancel a running crawl job by `crawl_id`. Returns `data.status` `cancelled`.
A finished crawl is 409 invalid_state. Needs Search edit.
See Firecrawl docs: https://docs.firecrawl.dev/
batch-status
Read.
Batch scrape jobs: start, status, or wait for many URLs.
Batch scrape jobs on the canonical routes: POST `/api/v1/scraping/batch-scrapes`
starts a Kenso batch_scrape job and GET `/api/v1/jobs/{jobId}?type=batch_scrape`
reads it, with Ken-Version 1.1 and X-Ken-Workspace-Id. `client_id` names the
workspace; omit it on a connection that already has one. `batch_id` is the
Kenso job id (a number) that `operation=wait` or
`web(operation="create-batch-scrape")` returned; cancel one with
`jobs(operation="cancel", job_type="batch_scrape")`. Operations:
### operation=start
Start an async scrape job for many URLs. Requires `urls`. Returns a job
envelope with `data.id`. Formats match `web` operation `scrape`: `markdown`, `html`,
`rawHtml`, `links`, `summary`, `screenshot`, `json`. For JSON extraction,
pass `json_options={"schema": {"type": "object", "properties": {...}}}`;
the MCP embeds those options into Firecrawl v2's `{type: "json", ...}`
format.
### operation=status
Poll one batch scrape job by `batch_id`. `data` is Firecrawl's batch status:
terminal statuses are `completed`, `failed`, and `cancelled`. Each call returns
one page of results. When `next` is set, pass it back as `cursor` to read the
next page; `next` is null on the last page. Needs Search view, as
`jobs(operation="get", job_type="batch_scrape")` does.
### operation=wait
Start a batch scrape, then poll until completion or timeout. Same start
params as `operation=start`. `max_wait_seconds` defaults to 120 and is
capped at 600. On timeout the latest status is returned with `data.id` (the
job id), a `warning` and a `hint`. Prefer this for short batches; use
`operation=start` plus `operation=status` for background jobs. The wait stops
at the first "Web scraping is off right now" error instead of polling. Needs
Search edit, because it starts the batch scrape.
See Firecrawl docs: https://docs.firecrawl.dev/
crawl-status
Read.
Crawl jobs: start, status, wait, or cancel a multi-page crawl.
Crawl jobs on the canonical routes: POST `/api/v1/scraping/crawls` starts a
Kenso crawl job, GET `/api/v1/jobs/{jobId}?type=crawl` reads it and POST
`/api/v1/jobs/{jobId}/cancel?type=crawl` cancels it, all with Ken-Version 1.1
and X-Ken-Workspace-Id. `client_id` names the workspace; omit it on a
connection that already has one. `crawl_id` is the Kenso job id (a number)
that `operation=wait` or `web(operation="create-crawl")` returned. Operations:
### operation=start
Start an async multi-page crawl. Requires `url`. Returns a job envelope
with `data.id`. `prompt` asks the LLM to derive crawl parameters.
`scrape_options` applies to every crawled page, for example
`{"formats": ["markdown", "links"]}`.
`limit` is the most pages to crawl: 100 when omitted, at most 1000 (a
larger value is cut to 1000). A crawl on Kenso's key holds credits for its
whole `limit` before it starts, so pass a larger `limit` only when the
site needs more than 100 pages.
### operation=status
Poll one crawl job by `crawl_id`. `data` is Firecrawl's crawl status: terminal
statuses are `completed`, `failed`, and `cancelled`; running jobs usually
return `scraping`. Each call returns one page of results. When `next` is set,
pass it back as `cursor` to read the next page; `next` is null on the last page.
Needs Search view, as `jobs(operation="get", job_type="crawl")` does.
### operation=wait
Start a crawl, then poll until completion or timeout. Same start params
as `operation=start`. `max_wait_seconds` defaults to 120 and is capped at
600. On timeout the latest status is returned with `data.id` (the job id),
a `warning` and a `hint` naming the status poll. The wait stops at the
first "Web scraping is off right now" error instead of polling. Needs Search
edit, because it starts the crawl.
### operation=cancel
Cancel a running crawl job by `crawl_id`. Returns `data.status` `cancelled`.
A finished crawl is 409 invalid_state. Needs Search edit.
See Firecrawl docs: https://docs.firecrawl.dev/
extract-status
Read.
Extract jobs: start, status, or wait for LLM structured extraction.
LLM-driven structured extraction on the canonical routes: POST
`/api/v1/scraping/extractions` starts a Kenso extraction job and GET
`/api/v1/jobs/{jobId}?type=extraction` reads it, with Ken-Version 1.1 and
X-Ken-Workspace-Id. `client_id` names the workspace; omit it on a connection
that already has one. `extract_id` is the Kenso job id (a number) that
`operation=wait` or `web(operation="create-extraction")` returned. An
extraction cannot be canceled. Operations:
### operation=start
Start extraction across one or many URLs. Requires `urls`, `prompt`, and
`schema` (JSON Schema). Prefer this over chaining several `web` `scrape`
calls when the goal is a structured lead-research object. Example schema:
`{"type": "object", "properties": {"company": {"type": "string"}}}`.
Use `scrape_options` to control page fetching, `show_sources` to request
source traces, and `ignore_invalid_urls` to continue past bad inputs.
Returns a job envelope with `data.id`.
### operation=status
Poll one extraction job by `extract_id`. `data` is Firecrawl's extract status:
a running job usually returns `processing`, and a completed one returns
`completed` with its `data` matching the requested JSON Schema. A failed or
canceled extraction returns an error with `data.status` `failed` or
`cancelled` and `data.id`. Needs Search view, as
`jobs(operation="get", job_type="extraction")` does.
### operation=wait
Start an extract job, then poll until completion or timeout. Same start
params as `operation=start`. `max_wait_seconds` defaults to 120 and is
capped at 600. On timeout the latest status is returned with `data.id` (the
job id), a `warning` and a `hint` naming the status poll. The wait stops at
the first "Web scraping is off right now" error instead of polling. Needs
Search edit, because it starts the extraction.
See Firecrawl docs: https://docs.firecrawl.dev/
batch-wait
Write.
Batch scrape jobs: start, status, or wait for many URLs.
Batch scrape jobs on the canonical routes: POST `/api/v1/scraping/batch-scrapes`
starts a Kenso batch_scrape job and GET `/api/v1/jobs/{jobId}?type=batch_scrape`
reads it, with Ken-Version 1.1 and X-Ken-Workspace-Id. `client_id` names the
workspace; omit it on a connection that already has one. `batch_id` is the
Kenso job id (a number) that `operation=wait` or
`web(operation="create-batch-scrape")` returned; cancel one with
`jobs(operation="cancel", job_type="batch_scrape")`. Operations:
### operation=start
Start an async scrape job for many URLs. Requires `urls`. Returns a job
envelope with `data.id`. Formats match `web` operation `scrape`: `markdown`, `html`,
`rawHtml`, `links`, `summary`, `screenshot`, `json`. For JSON extraction,
pass `json_options={"schema": {"type": "object", "properties": {...}}}`;
the MCP embeds those options into Firecrawl v2's `{type: "json", ...}`
format.
### operation=status
Poll one batch scrape job by `batch_id`. `data` is Firecrawl's batch status:
terminal statuses are `completed`, `failed`, and `cancelled`. Each call returns
one page of results. When `next` is set, pass it back as `cursor` to read the
next page; `next` is null on the last page. Needs Search view, as
`jobs(operation="get", job_type="batch_scrape")` does.
### operation=wait
Start a batch scrape, then poll until completion or timeout. Same start
params as `operation=start`. `max_wait_seconds` defaults to 120 and is
capped at 600. On timeout the latest status is returned with `data.id` (the
job id), a `warning` and a `hint`. Prefer this for short batches; use
`operation=start` plus `operation=status` for background jobs. The wait stops
at the first "Web scraping is off right now" error instead of polling. Needs
Search edit, because it starts the batch scrape.
See Firecrawl docs: https://docs.firecrawl.dev/
crawl-wait
Write.
Crawl jobs: start, status, wait, or cancel a multi-page crawl.
Crawl jobs on the canonical routes: POST `/api/v1/scraping/crawls` starts a
Kenso crawl job, GET `/api/v1/jobs/{jobId}?type=crawl` reads it and POST
`/api/v1/jobs/{jobId}/cancel?type=crawl` cancels it, all with Ken-Version 1.1
and X-Ken-Workspace-Id. `client_id` names the workspace; omit it on a
connection that already has one. `crawl_id` is the Kenso job id (a number)
that `operation=wait` or `web(operation="create-crawl")` returned. Operations:
### operation=start
Start an async multi-page crawl. Requires `url`. Returns a job envelope
with `data.id`. `prompt` asks the LLM to derive crawl parameters.
`scrape_options` applies to every crawled page, for example
`{"formats": ["markdown", "links"]}`.
`limit` is the most pages to crawl: 100 when omitted, at most 1000 (a
larger value is cut to 1000). A crawl on Kenso's key holds credits for its
whole `limit` before it starts, so pass a larger `limit` only when the
site needs more than 100 pages.
### operation=status
Poll one crawl job by `crawl_id`. `data` is Firecrawl's crawl status: terminal
statuses are `completed`, `failed`, and `cancelled`; running jobs usually
return `scraping`. Each call returns one page of results. When `next` is set,
pass it back as `cursor` to read the next page; `next` is null on the last page.
Needs Search view, as `jobs(operation="get", job_type="crawl")` does.
### operation=wait
Start a crawl, then poll until completion or timeout. Same start params
as `operation=start`. `max_wait_seconds` defaults to 120 and is capped at
600. On timeout the latest status is returned with `data.id` (the job id),
a `warning` and a `hint` naming the status poll. The wait stops at the
first "Web scraping is off right now" error instead of polling. Needs Search
edit, because it starts the crawl.
### operation=cancel
Cancel a running crawl job by `crawl_id`. Returns `data.status` `cancelled`.
A finished crawl is 409 invalid_state. Needs Search edit.
See Firecrawl docs: https://docs.firecrawl.dev/
extract-wait
Write.
Extract jobs: start, status, or wait for LLM structured extraction.
LLM-driven structured extraction on the canonical routes: POST
`/api/v1/scraping/extractions` starts a Kenso extraction job and GET
`/api/v1/jobs/{jobId}?type=extraction` reads it, with Ken-Version 1.1 and
X-Ken-Workspace-Id. `client_id` names the workspace; omit it on a connection
that already has one. `extract_id` is the Kenso job id (a number) that
`operation=wait` or `web(operation="create-extraction")` returned. An
extraction cannot be canceled. Operations:
### operation=start
Start extraction across one or many URLs. Requires `urls`, `prompt`, and
`schema` (JSON Schema). Prefer this over chaining several `web` `scrape`
calls when the goal is a structured lead-research object. Example schema:
`{"type": "object", "properties": {"company": {"type": "string"}}}`.
Use `scrape_options` to control page fetching, `show_sources` to request
source traces, and `ignore_invalid_urls` to continue past bad inputs.
Returns a job envelope with `data.id`.
### operation=status
Poll one extraction job by `extract_id`. `data` is Firecrawl's extract status:
a running job usually returns `processing`, and a completed one returns
`completed` with its `data` matching the requested JSON Schema. A failed or
canceled extraction returns an error with `data.status` `failed` or
`cancelled` and `data.id`. Needs Search view, as
`jobs(operation="get", job_type="extraction")` does.
### operation=wait
Start an extract job, then poll until completion or timeout. Same start
params as `operation=start`. `max_wait_seconds` defaults to 120 and is
capped at 600. On timeout the latest status is returned with `data.id` (the
job id), a `warning` and a `hint` naming the status poll. The wait stops at
the first "Web scraping is off right now" error instead of polling. Needs
Search edit, because it starts the extraction.
See Firecrawl docs: https://docs.firecrawl.dev/