Extract knowledge from URL | Atlas

Extracts content from a web page and converts it to a .md knowledge file.

Single-page mode (default): scrapes one page synchronously (3-5 seconds) and returns the result directly with a 201 response.

Multi-page crawl mode (crawl: true): discovers and scrapes up to maxPages pages from the same domain asynchronously. Returns a 202 response with a jobId immediately. Poll GET /knowledgebase/knowledge-extract/jobs/{jobId} for progress.

The scraper captures visible text; JavaScript-heavy SPAs may return limited content. After extraction, attach the file to a campaign using POST /knowledgebase/{campaignId}/{id}.

Authentication

api-keystring

Your Atlas API key. Get it from the Atlas Portal under Settings > API Keys.

Request

This endpoint expects an object.

urlstringRequiredformat: "uri"

The web page URL to extract content from.

campaignIdstringOptional

Optional campaign to attach the extracted file to.

crawlbooleanOptionalDefaults to false

When true, performs a multi-page domain crawl instead of single-page extraction. The response changes to 202 with a jobId for polling.

maxPagesintegerOptional1-50Defaults to 20

Maximum number of pages to crawl when crawl is true. Ignored for single-page extraction.

keepUpToDatebooleanOptionalDefaults to false

When true, enables periodic automatic re-extraction of the URL to keep the knowledge file fresh.

Response

Knowledge extracted (single-page mode)

filePathstringOptional

filenamestringOptional

knowledgeIdstringOptional

Atlas file identifier for the extracted knowledge.

Errors

400

Bad Request Error

409

Conflict Error

500

Internal Server Error