hurl: HTTP, HTTPS and Gopher file grabber

Last modification on

hurl is a relatively simple HTTP, HTTPS and Gopher client/file grabber.

Why?

curl is a nice swish-army knife which handles many protocols, but the code sucks. Other tools like OpenBSD ftp are much better, but still do much.

Sometimes (or most of the time?) you just want to fetch a file via the HTTP, HTTPS or Gopher protocol.

The focus of this tool is only this.

Dependencies

  • C compiler (C99).
  • libc + some BSD functions like err() and strlcat().
  • LibreSSL(-portable)
  • libtls (part of LibreSSL).

Optional dependencies

  • POSIX make(1) (for Makefile).
  • mandoc for documentation: https://mdocml.bsd.lv/

Clone

git clone git://git.codemadness.org/hurl

Download releases

Releases are available at: https://codemadness.org/releases/hurl/.

Build and install

$ make
# make install

Features

  • Uses OpenBSD pledge(2) and unveil(2). Allow no filesystem access (writes to stdout).
  • Impose time-out and maximum size limits.
  • Use well-defined exitcodes for reliable scripting (curl sucks at this).
  • Send as little information as possible (no User-Agent etc by default).

Anti-features

  • No HTTP byte range support.
  • No HTTP User-Agent.
  • No HTTP If-Modified-Since/If-* support.
  • No HTTP auth support.
  • No HTTP/2+ support.
  • No HTTP keep-alive.
  • No HTTP chunked-encoding support.
  • No HTTP redirect support.
  • No (GZIP) compression support.
  • No cookie-jar or cookie parsing support.
  • No Gopher text handling (".\r\n").
  • ... etc...

Examples

Fetch the Atom feed from this site using a maximum filesize limit of 1MB and a time-out limit of 15 seconds:

hurl -m 1048576 -t 15 "https://codemadness.org/atom.xml"