Introduction

EliseBot is an HTTP API that scrapes URLs, one at a time or many in a batch scrape. A scrape answers with its page inline; a batch scrape packages its pages into one file to download. The request and response shape is borrowed from the v2 scrape API style, but it is not compatible with that API’s clients.

The API is at https://api.elisebot.dev.

Endpoint	Does
POST /v2/scrape	Scrapes one URL and answers with its document.
GET /{url}	The same with default options; the URL follows the host as is, e.g. GET /https://example.com/?q=1.
GET /v2/scrape/{id}	The status of a scrape that answered 202; once it ends, its document.
POST /v2/batch/scrape	Starts a batch scrape.
GET /v2/batch/scrape/{id}	The batch scrape’s progress; once it ends, its package URL.
GET /v2/batch/scrape/{id}/download	Redirects to the batch scrape’s results package, one gzipped NDJSON file.
DELETE /v2/batch/scrape/{id}	Cancels the batch scrape.

The fetched pages are kept in a private bucket that only the API reads. The one URL into it handed out is a batch scrape package’s download, a presigned link valid for 5 minutes.

Quickstart

1. Scrape without a key

   Scrape a page with the default options; without an API key the caller is anonymous:

   curl https://api.elisebot.dev/https://example.com

2. Scrape with an API key

   Organizations and their API keys come from the dashboard. With a key, POST /v2/scrape takes the options in a JSON body:

   curl https://api.elisebot.dev/v2/scrape \
     -H 'Authorization: Bearer <key>' \
     -H 'Content-Type: application/json' \
     -d '{"url": "https://example.com", "responseOptions": {"format": "markdown"}}'

Many URLs at once go through a batch scrape. What a key may do and what it costs is under Authentication and credits.