Capturing Response

Captures

Captures are optional values that are extracted from the HTTP response and stored in a named variable. These captures may be the response status code, part of or the entire the body, and response headers.

Captured variables can be accessed through a run session; each new value of a given variable overrides the last value.

Captures can be useful for using data from one request in another request, such as when working with CSRF tokens. Variables in a Hurl file can be created from captures or injected into the session.

# An example to show how to pass a CSRF token
# from one request to another:

# First GET request to get CSRF token value:
GET https://example.org
HTTP 200
# Capture the CSRF token value from html body.
[Captures]
csrf_token: xpath "normalize-space(//meta[@name='_csrf_token']/@content)"

# Do the login !
POST https://acmecorp.net/login?user=toto&password=1234
X-CSRF-TOKEN: {{csrf_token}}
HTTP 302

Body responses can be encoded by server (see [Content-Encoding HTTP header]) but captures in Hurl files are not affected by this content compression. All body captures (body, bytes, sha256 etc...) work after content decoding.

Finally, body text captures (body, jsonpath, xpath etc...) are also decoded to strings based on [Content-Type header] so these queries can be captures as usual strings.

Structure of a capture:

A capture consists of a variable name, followed by : and a query. Captures section starts with [Captures].

Query

Queries are used to extract data from an HTTP response.

A query can extract data from

• status line:
  • status
  • version
• headers:
  • header
  • cookie
• body:
  • body
  • bytes
  • xpath
  • jsonpath
  • regex
  • sha256
  • md5
• others:
  • url
  • redirects
  • ip
  • variable
  • duration
  • certificate

Extracted data can then be further refined using filters.

Status capture

Capture the received HTTP response status code. Status capture consists of a variable name, followed by a :, and the keyword status.

GET https://example.org
HTTP 200
[Captures]
my_status: status

Version capture

Capture the received HTTP version. Version capture consists of a variable name, followed by a :, and the keyword version. The value captured is a string:

GET https://example.org
HTTP 200
[Captures]
http_version: version

Header capture

Capture a header from the received HTTP response headers. Header capture consists of a variable name, followed by a :, then the keyword header and a header name.

POST https://example.org/login
[Form]
user: toto
password: 12345678
HTTP 302
[Captures]
next_url: header "Location"

Cookie capture

Capture a Set-Cookie header from the received HTTP response headers. Cookie capture consists of a variable name, followed by a :, then the keyword cookie and a cookie name.

GET https://example.org/cookies/set
HTTP 200
[Captures]
session-id: cookie "LSID"

Cookie attributes value can also be captured by using the following format: <cookie-name>[cookie-attribute]. The following attributes are supported: Value, Expires, Max-Age, Domain, Path, Secure, HttpOnly and SameSite.

GET https://example.org/cookies/set
HTTP 200
[Captures]
value1: cookie "LSID"
value2: cookie "LSID[Value]"     # Equivalent to the previous capture
expires: cookie "LSID[Expires]"
max-age: cookie "LSID[Max-Age]"
domain: cookie "LSID[Domain]"
path: cookie "LSID[Path]"
secure: cookie "LSID[Secure]"
http-only: cookie "LSID[HttpOnly]"
same-site: cookie "LSID[SameSite]"

Body capture

Capture the entire body (decoded as text) from the received HTTP response. The encoding used to decode the body is based on the charset value in the Content-Type header response.

GET https://example.org/home
HTTP 200
[Captures]
my_body: body

If the Content-Type doesn’t include any encoding hint, a decode filter can be used to explicitly decode the body response bytes.

# Our HTML response is encoded using GB 2312.
# But, the 'Content-Type' HTTP response header doesn't precise any charset,
# so we decode explicitly the bytes.
GET https://example.org/cn
HTTP 200
[Captures]
my_body: bytes decode "gb2312"

body capture works after content encoding decompression (so the captured value is not affected by Content-Encoding response header).

Bytes capture

Capture the entire body (as a raw bytestream) from the received HTTP response

GET https://example.org/data.bin
HTTP 200
[Captures]
my_data: bytes

Like body capture, bytes capture works after content encoding decompression (so the captured value is not affected by Content-Encoding response header).

XPath capture

Capture a XPath query from the received HTTP body decoded as a string. Currently, only XPath 1.0 expression can be used.

GET https://example.org/home
# Capture the identifier from the dom node <div id="pet0">5646eaf23</div
HTTP 200
[Captures]
pet-id: xpath "normalize-space(//div[@id='pet0'])"

# Open the captured page.
GET https://example.org/home/pets/{{pet-id}}
HTTP 200

XPath captures are not limited to node values (like string, or boolean); any valid XPath can be captured and asserted with variable asserts.

# Test that the XML endpoint return 200 pets
GET https://example.org/api/pets
HTTP 200
[Captures]
pets: xpath "//pets"
[Asserts]
variable "pets" count == 200

XPath expression can also be evaluated against part of the body with a xpath filter:

GET https://example.org/home_cn
HTTP 200
[Captures]
pet-id: bytes decode "gb2312" xpath "normalize-space(//div[@id='pet0'])"

JSONPath capture

Capture a JSONPath query from the received HTTP body.

POST https://example.org/api/contact
[Form]
token: {{token}}
email: toto@rookie.net
HTTP 200
[Captures]
contact-id: jsonpath "$['id']"

Explain that the value selected by the JSONPath is coerced to a string when only one node is selected.

As with XPath captures, JSONPath captures can be anything from string, number, to object and collections. For instance, if we have a JSON endpoint that returns the following JSON:

{
  "a_null": null,
  "an_object": {
    "id": "123"
  },
  "a_list": [
    1,
    2,
    3
  ],
  "an_integer": 1,
  "a float": 1.1,
  "a_bool": true,
  "a_string": "hello"
}

We can capture the following paths:

GET https://example.org/captures-json
HTTP 200
[Captures]
an_object:  jsonpath "$['an_object']"
a_list:     jsonpath "$['a_list']"
a_null:     jsonpath "$['a_null']"
an_integer: jsonpath "$['an_integer']"
a_float:    jsonpath "$['a_float']"
a_bool:     jsonpath "$['a_bool']"
a_string:   jsonpath "$['a_string']"
all:        jsonpath "$"

Regex capture

Capture a regex pattern from the HTTP received body, decoded as text.

GET https://example.org/helloworld
HTTP 200
[Captures]
id_a: regex "id_a:([0-9]+)"
id_b: regex "id_b:(\\d+)"   # pattern using double quote 
id_c: regex /id_c:(\d+)/    # pattern using forward slash
name: regex "Hello ([a-zA-Z]+)"

The regex pattern must have at least one capture group, otherwise the capture will fail. When the pattern is a double-quoted string, metacharacters beginning with a backslash in the pattern (like \d, \s) must be escaped; literal pattern enclosed by / can also be used to avoid metacharacters escaping.

The regex syntax is documented at https://docs.rs/regex/latest/regex/#syntax. For instance, one can use flags to enable case-insensitive match:

GET https://example.org/hello
HTTP 200
[Captures]
word: regex /(?i)hello (\w+)!/

SHA-256 capture

Capture the SHA-256 hash of the response body.

GET https://example.org/data.tar.gz
HTTP 200
[Captures]
my_hash: sha256

Like body assert, sha256 capture works after content encoding decompression (so the captured value is not affected by Content-Encoding response header).

MD5 capture

Capture the MD5 hash of the response body.

GET https://example.org/data.tar.gz
HTTP 200
[Captures]
my_hash: md5

Like sha256 asserts, md5 assert works after content encoding decompression (so the predicates values are not affected by Content-Encoding response header)

URL capture

Capture the last fetched URL. This is most meaningful if you have told Hurl to follow redirection (see [Options] section or --location option). URL capture consists of a variable name, followed by a :, and the keyword url.

GET https://example.org/redirecting
[Options]
location: true
HTTP 200
[Captures]
landing_url: url

Redirects capture

Capture each step of redirection. This is most meaningful if you have told Hurl to follow redirection (see [Options]section or --location option). Redirects capture consists of a variable name, followed by a :, and the keyword redirects. Redirects query returns a collection so each step of the redirection can be capture.

GET https://example.org/redirecting/1
[Options]
location: true
HTTP 200
[Asserts]
redirects count == 3
[Captures]
step1: redirects nth 0 location
step2: redirects nth 1 location
step3: redirects nth 2 location

IP address capture

Capture the IP address of the last connection. The value of the ip query is a string.

GET https://example.org/hello
HTTP 200
[Captures]
server_ip: ip

Variable capture

Capture the value of a variable into another.

GET https://example.org/helloworld
HTTP 200
[Captures]
in: body
name: variable "in"

Duration capture

Capture the response time of the request in ms.

GET https://example.org/helloworld
HTTP 200
[Captures]
duration_in_ms: duration

SSL certificate capture

Capture the SSL certificate properties. Certificate capture consists of the keyword certificate, followed by the certificate attribute value.

The following attributes are supported: Subject, Issuer, Start-Date, Expire-Date and Serial-Number.

GET https://example.org
HTTP 200
[Captures]
cert_subject: certificate "Subject"
cert_issuer: certificate "Issuer"
cert_expire_date: certificate "Expire-Date"
cert_serial_number: certificate "Serial-Number"

Redacting Secrets

When capturing data, you may need to hide captured values from logs and report. To do this, captures can use secrets which are redacted from logs and reports, using --secret option:

$ hurl --secret pass=sesame-ouvre-toi file.hurl

If the secret value to be redacted is dynamic, or not known before execution, a capture can become a secret using redact at the end of the query’s capture:

GET https://foo.com
HTTP 200
[Captures]
pass: header "token" redact