Running Tests

Use --test Option

Hurl is run by default as an HTTP client, returning the body of the last HTTP response.

$ hurl hello.hurl
Hello World!

When multiple input files are provided, Hurl outputs the body of the last HTTP response of each file.

$ hurl hello.hurl assert_json.hurl
Hello World![
  { "id": 1, "name": "Bob"},
  { "id": 2, "name": "Bill"}
]

For testing, we are only interested in the asserts results, we don’t need the HTTP body response. To use Hurl as a test tool with an adapted output, you can use --test option:

$ hurl --test hello.hurl assert_json.hurl
hello.hurl: Success (6 request(s) in 245 ms)
assert_json.hurl: Success (8 request(s) in 308 ms)
--------------------------------------------------------------------------------
Executed files:    2
Executed requests: 10 (17.82/s)
Succeeded files:   2 (100.0%)
Failed files:      0 (0.0%)
Duration:          561 ms

Or, in case of errors:

$ hurl --test hello.hurl error_assert_status.hurl 
hello.hurl: Success (4 request(s) in 5 ms)
error: Assert status code
  --> error_assert_status.hurl:9:6
   |
   | GET http://localhost:8000/not_found
   | ...
 9 | HTTP 200
   |      ^^^ actual value is <404>
   |

error_assert_status.hurl: Failure (1 request(s) in 2 ms)
--------------------------------------------------------------------------------
Executed files:    2
Executed requests: 5 (500.0/s)
Succeeded files:   1 (50.0%)
Failed files:      1 (50.0%)
Duration:          10 ms

In test mode, files are executed in parallel to speed-ud the execution. If a sequential run is needed, you can use --jobs 1 option to execute tests one by one.

$ hurl --test --jobs 1 *.hurl

--repeat option can be used to repeat run files and do performance check. For instance, this call will run 1000 tests in parallel:

$ hurl --test --repeat 1000 stress.hurl

Selecting Tests

Hurl can take multiple files into inputs:

$ hurl --test test/integration/a.hurl test/integration/b.hurl test/integration/c.hurl

$ hurl --test test/integration/*.hurl

Or you can simply give a directory and Hurl will find files with .hurl extension recursively:

$ hurl --test test/integration/

Finally, you can use --glob option to test files that match a given pattern:

$ hurl --test --glob "test/integration/**/*.hurl"

Debugging

Debug Logs

If you need more error context, you can use --error-format long option to print HTTP bodies for failed asserts:

$ hurl --test --error-format long hello.hurl error_assert_status.hurl 
hello.hurl: Success (4 request(s) in 6 ms)
HTTP/1.1 404
Server: Werkzeug/3.0.3 Python/3.12.4
Date: Wed, 10 Jul 2024 15:42:41 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 207
Server: Flask Server
Connection: close

<!doctype html>
<html lang=en>
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>


error: Assert status code
  --> error_assert_status.hurl:9:6
   |
   | GET http://localhost:8000/not_found
   | ...
 9 | HTTP 200
   |      ^^^ actual value is <404>
   |

error_assert_status.hurl: Failure (1 request(s) in 2 ms)
--------------------------------------------------------------------------------
Executed files:    2
Executed requests: 5 (454.5/s)
Succeeded files:   1 (50.0%)
Failed files:      1 (50.0%)
Duration:          11 ms

Individual requests can be modified with [Options] section to turn on logs for a particular request, using verbose and very-verbose option.

With this Hurl file:

GET https://foo.com
HTTP 200

GET https://bar.com
[Options]
very-verbose: true
HTTP 200

GET https://baz.com
HTTP 200

Running hurl --test . will output debug logs for the request to https://bar.com.

--verbose / --very-verbose can also be enabled globally, for every requests of every tested files:

$ hurl --test --very-verbose .

In test mode, HTTP responses are not displayed. One way to get HTTP responses even in test mode is to use --output option of [Options] section: --output file allows to save a particular response to a file, while --output - allows to redirect HTTP responses to standard output.

GET http://foo.com
HTTP 200

GET https://bar.com
[Options]
output: -
HTTP 200

$ hurl --test .
<html><head><meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>301 Moved</TITLE></head><body>
<h1>301 Moved</h1>
The document has moved
<a HREF="https://www.bar.com/">here</a>.
</body></html>
/tmp/test.hurl: Success (2 request(s) in 184 ms)
--------------------------------------------------------------------------------
Executed files:    1
Executed requests: 2 (10.7/s)
Succeeded files:   1 (100.0%)
Failed files:      0 (0.0%)
Duration:          187 ms

Generating Report

In the different reports, files are always referenced in the input order (which, as tests are executed in parallel, can be different from the execution order).