POST /v2/scrape and POST /v2/batch/scrape take requestOptions.headers, an ordered array sent to the target
with every fetch:
{"url": "...", "requestOptions": {"headers": [{"name": "accept", "value": "*/*"}, {"name": "cookie", "value": "a=1; b=2"}]}}GET /{url} takes none.
- The first request sends the headers in this order, exactly as given, with nothing added; redirect hops add only
the cookies the site sets (below). Only without headers does EliseBot send its own
User-AgentandAccept-Encoding: gzip. - The cookie is a raw
cookieentry; on the first request EliseBot never builds or reorders it. - A
hostentry sets the Host and takes its place in the order; it is written asHost:whatever its case. - HTTP/2 lowercases every name and splits the cookie into one field per pair, which the server joins again.
- Redirects are followed. A redirect within the first URL’s registrable domain (public suffix plus one label:
www.amazon.com,amazon.comandsmile.amazon.comare one site) sends the same headers again, cookie included, plus theRefererGo sets unless the list has one. A redirect to any other site aborts the fetch withcross_site_redirectand the redirect’sstatusCode. - A hop carries the cookies the redirect chain set. A pair of yours whose name the chain set is replaced, or dropped
when the chain deleted it or scoped it away from the hop. The chain’s cookies join your
cookieentry; with none, they go in aCookieadded at the end of the list, beforeReferer. - The body is decoded from whatever encoding was asked for: gzip, deflate, br or zstd.
- A header is refused with a 422 naming
body.requestOptions.headers[i], without repeating the value, when:- its name is not an HTTP token;
- its value holds CR, LF, NUL or another control character, or starts or ends with a space or tab;
- its name is one the transport drops or keeps for itself:
content-length,transfer-encoding,trailer,connection,proxy-connection,upgrade,keep-alive,__header_order__or__pseudo_header_order__; - its name appeared before with other headers in between, or right above it in another spelling;
- it is a second
user-agentorhost, or ahostwith an empty value.
- An adjacent repeat with the same spelling is allowed and goes out as separate lines.
- Names plus values are at most 8 KiB.
- They are part of a batch scrape’s
Idempotency-Keyrequest, order included: other headers, or the same ones in another order, under the same key are a 422.