GoAccess log format for OpenBSD httpd

Posted on

GoAccess is a neat HTTP access log reporting tool for the command-line. In this post I’ll briefly describe how to import access logs generated by OpenBSD’s built-in HTTP server.

OpenBSD’s httpd(8) supports four different log styles: The common and combined log styles are similar to the de facto standard access log formats of the Apache HTTP Server. The forwarded log style extends the combined log style by appending the widespread X-Forwarded-For and X-Forwarded-Port headers. In contrast to this, the connection log style writes a summary of all requests per connection. I won’t go into the details of the connection log style here. You can find the relevant source on the CVS and on the GitHub mirror.

A typical forwarded style log message looks like this:

www.example.com 127.0.0.1 - - [14/May/2021:13:26:56 +0000] "GET /posts/ HTTP/1.1" 200 2066 "https://www.example.com/" "Mozilla/5.0 (X11; OpenBSD amd64; rv:88.0) Gecko/20100101 Firefox/88.0" 10.146.199.139 -

It consists of twelve elments:

  1. Server name, alias virtual host
  2. Client IP address, alias remote host
  3. RFC 1413 user identity, if any
  4. HTTP authentication user identity, if any
  5. Date and time
  6. Request method, path, and protocol
  7. Response status code
  8. Response body size in bytes
  9. Referer header, if any
  10. User-Agent header, if any
  11. X-Forwarded-For header, if any
  12. X-Forwarded-Port header, if any

The corresponding format strings for GoAccess are:

For example, you can import forwarded style access logs as follows:

$ zcat -f /var/www/logs/access.log* \
    | grep -v 'logfile turned over$' \
    | awk '$8=$1$8' \
    | goaccess \
        --no-global-config \
        --log-format='%v %^ %^ %e [%d:%t] "%r" %s %b %R %u ~h{," } %^' \
        --date-format='%d/%b/%Y' \
        --time-format='%H:%M:%S %z' \
        --no-color

This pipeline uncompresses and concatenates all access logs in the default location, filters log file rotation messages, prepends the server name to the request path, and imports the result into GoAccess to show an interactive command-line report. Append --output=access.html to generate a pretty HTML report instead.