CrawlingStart website crawl
Crawling

Start website crawl

Queues a new crawl job and returns a scanId. The crawl runs asynchronously.

curl -X POST "https://api.neostra.io/api/v1/crawl/start" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_TOKEN (JWT)" \
  -d '{
  "tenantId": "123e4567-e89b-12d3-a456-426614174000",
  "website": "example_string",
  "urlFilterRegexInclusive": "example_string",
  "urlFilterRegexExclusive": "example_string",
  "maxPages": 25
}'
{
  "success": true,
  "message": "example_string",
  "data": {
    "scanId": "123e4567-e89b-12d3-a456-426614174000"
  }
}
POST
/api/v1/crawl/start
POST
Bearer Token (JWT)
Bearer Tokenstring
Required

Bearer token (JWT) - just enter the token, "Bearer" prefix will be added automatically

Content-Typestring
Required

The media type of the request body

Options: application/json
websitestring
Required

Root URL to crawl

Format: uri
urlFilterRegexInclusivestring

Regex — only URLs matching this pattern will be crawled

urlFilterRegexExclusivestring

Regex — URLs matching this pattern will be skipped

maxPagesinteger

Maximum number of pages to crawl

Request Preview
Response

Response will appear here after sending the request

Authentication

header
Authorizationstring
Required

Bearer token (JWT). Authentication token required.

Body

application/json
websitestring
Required

Root URL to crawl

urlFilterRegexInclusivestring

Regex — only URLs matching this pattern will be crawled

urlFilterRegexExclusivestring

Regex — URLs matching this pattern will be skipped

maxPagesinteger

Maximum number of pages to crawl

Responses