All new documentation

This patch does a lot.

- Ditch sphinx in favor of hugo. This gives us complete control of the layout
and presentation of our docs. Henceforth, docs will be hosted on our website
rather than ReadTheDocs.
- Create a simple, clean doc layout and theme.
- Remove large parts of the documentaion. I've ditched anything that was a)
woefully out of date, b) too detailed, or c) too hard to maintain in the long
term.
- Huge updates to the docs themselves: completely rewrite addons documentation,
add docs for core concepts like commands and options, and revise and tweak a
lot of the existing docs.

With this patch, we're also changing the way we publish and maintain the docs.
From now on, we don't publish docs for every release. Instead, the website will
contain ONE set of docs for each major release. The online docs will be updated
if needed as minor releases are made. Docs are free to improve during minor
releases, but anything that changes behaviour sufficiently to require a doc
change warrants a new major release. This also leaves us free to progressively
update and improve docs out of step with our release cadence.

With this new scheme, I feel CI over the docs is less important. I've removed
it for now, but won't object if someone wants to add it back in.

4 files changed, 0 insertions, 616 deletions

diff --git a/docs/pathod/intro.rst b/docs/pathod/intro.rst
deleted file mode 100644
index 1c1ad60e..00000000
--- a/docs/pathod/intro.rst
+++ /dev/null
@@ -1,307 +0,0 @@
-.. _intro:
-
-Pathology 101
-=============
-
-.. _pathod:
-
-pathod
-------
-
-Pathod is a pathological HTTP daemon designed to let you craft almost any
-conceivable HTTP response, including ones that creatively violate the
-standards. HTTP responses are specified using a :ref:`small, terse language
-<language>` which pathod shares with its evil twin :ref:`pathoc`. To start
-playing with pathod, fire up the daemon:
-
->>> pathod
-
-By default, the service listens on port 9999 of localhost, and the default
-crafting anchor point is the path **/p/**. Anything after this URL prefix is
-treated as a response specifier. So, hitting the following URL will generate an
-HTTP 200 response with 100 bytes of random data:
-
-    http://localhost:9999/p/200:b@100
-
-See the :ref:`language documentation <language>` to get (much) fancier. The
-pathod daemon also takes a range of configuration options. To view those, use
-the command-line help:
-
->>> pathod --help
-
-Mimicing a proxy
-^^^^^^^^^^^^^^^^
-
-Pathod automatically responds to both straight HTTP and proxy requests. For
-proxy requests, the upstream host is ignored, and the path portion of the URL
-is used to match anchors. This lets you test software that supports a proxy
-configuration by spoofing responses from upstream servers.
-
-By default, we treat all proxy CONNECT requests as HTTPS traffic, serving the
-response using either pathod's built-in certificates, or the cert/key pair
-specified by the user. You can over-ride this behaviour if you're testing a
-client that makes a non-SSL CONNECT request using the **-C** command-line
-option.
-
-Anchors
-^^^^^^^
-
-Anchors provide an alternative to specifying the response in the URL. Instead,
-you attach a response to a pre-configured anchor point, specified with a regex.
-When a URL matching the regex is requested, the specified response is served.
-
->>> pathod -a "/foo=200"
-
-Here, "/foo" is the regex specifying the anchor path, and the part after the "="
-is a response specifier.
-
-
-File Access
-^^^^^^^^^^^
-
-There are two operators in the :ref:`language <language>` that load contents
-from file - the **+** operator to load an entire request specification from
-file, and the **>** value specifier. In pathod, both of these operators are
-restricted to a directory specified at startup, or disabled if no directory is
-specified:
-
->>> pathod -d ~/staticdir"
-
-
-Internal Error Responses
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-Pathod uses the non-standard 800 response code to indicate internal errors, to
-distinguish them from crafted responses. For example, a request to:
-
-    http://localhost:9999/p/foo
-
-... will return an 800 response because "foo" is not a valid page specifier.
-
-
-
-
-
-.. _pathoc:
-
-pathoc
-------
-
-Pathoc is a perverse HTTP daemon designed to let you craft almost any
-conceivable HTTP request, including ones that creatively violate the standards.
-HTTP requests are specified using a :ref:`small, terse language <language>`,
-which pathoc shares with its server-side twin :ref:`pathod`. To view pathoc's
-complete range of options, use the command-line help:
-
->>> pathoc --help
-
-
-Getting Started
-^^^^^^^^^^^^^^^
-
-The basic pattern for pathoc commands is as follows:
-
-    pathoc hostname request [request ...]
-
-That is, we specify the hostname to connect to, followed by one or more
-requests. Lets start with a simple example::
-
-    > pathoc google.com get:/
-    07-06-16 12:13:43: >> 'GET':/
-    << 302 Found: 261 bytes
-
-Here, we make a GET request to the path / on port 80 of google.com. Pathoc's
-output tells us that the server responded with a 302 redirection. We can tell
-pathoc to connect using SSL, in which case the default port is changed to 443
-(you can over-ride the default port with the **-p** command-line option)::
-
-    > pathoc -s www.google.com get:/
-    07-06-16 12:14:56: >> 'GET':/
-    << 302 Found: 262 bytes
-
-
-Multiple Requests
-^^^^^^^^^^^^^^^^^
-
-There are two ways to tell pathoc to issue multiple requests. The first is to specify
-them on the command-line, like so::
-
-    > pathoc google.com get:/ get:/
-    07-06-16 12:21:04: >> 'GET':/
-    << 302 Found: 261 bytes
-    07-06-16 12:21:04: >> 'GET':/
-    << 302 Found: 261 bytes
-
-In this case, pathoc issues the specified requests over the same TCP connection -
-so in the above example only one connection is made to google.com
-
-The other way to issue multiple requests is to use the **-n** flag::
-
-    > pathoc -n 2 google.com get:/
-    07-06-16 12:21:04: >> 'GET':/
-    << 302 Found: 261 bytes
-    07-06-16 12:21:04: >> 'GET':/
-    << 302 Found: 261 bytes
-
-The output is identical, but two separate TCP connections are made to the
-upstream server. These two specification styles can be combined::
-
-    pathoc -n 2 google.com get:/ get:/
-
-
-Here, two distinct TCP connections are made, with two requests issued over
-each.
-
-
-
-Basic Fuzzing
-^^^^^^^^^^^^^
-
-The combination of pathoc's powerful request specification language and a few
-of its command-line options makes for quite a powerful basic fuzzer. Here's an
-example::
-
-    pathoc -e -I 200 -t 2 -n 1000 localhost get:/:b@10:ir,@1
-
-The request specified here is a valid GET with a body consisting of 10 random bytes,
-but with 1 random byte inserted in a random place. This could be in the headers,
-in the initial request line, or in the body itself. There are a few things
-to note here:
-
-- Corrupting the request in this way will often make the server enter a state where
-  it's awaiting more input from the client. This is where the
-  **-t** option comes in, which sets a timeout that causes pathoc to
-  disconnect after two seconds.
-- The **-n** option tells pathoc to repeat the request 1000 times.
-- The **-I** option tells pathoc to ignore HTTP 200 response codes.
-  You can use this to fine-tune what pathoc considers to be an exceptional
-  condition, and therefore log-worthy.
-- The **-e** option tells pathoc to print an explanation of each logged
-  request, in the form of an expanded pathoc specification with all random
-  portions and automatic header additions resolved. This lets you precisely
-  replay a request that triggered an error.
-
-
-Interacting with Proxies
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-Pathoc has a reasonably sophisticated suite of features for interacting with
-proxies. The proxy request syntax very closely mirrors that of straight HTTP,
-which means that it is possible to make proxy-style requests using pathoc
-without any additional syntax, by simply specifying a full URL instead of a
-simple path:
-
->>> pathoc -p 8080 localhost "get:'http://google.com'"
-
-Another common use case is to use an HTTP CONNECT request to probe remote
-servers via a proxy. This is done with the **-c** command-line option, which
-allows you to specify a remote host and port pair:
-
->>> pathoc -c google.com:80 -p 8080 localhost get:/
-
-Note that pathoc does **not** negotiate SSL without being explictly instructed
-to do so. If you're making a CONNECT request to an SSL-protected resource, you
-must also pass the **-s** flag:
-
->>> pathoc -sc google.com:443 -p 8080 localhost get:/
-
-
-
-Embedded response specification
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-One interesting feature of the Request specification language is that you can
-embed a response specification in it, which is then added to the request path.
-Here's an example:
-
->>> pathoc localhost:9999 "get:/p/:s'401:ir,@1'"
-
-This crafts a request that connects to the pathod server, and which then crafts
-a response that generates a 401, with one random byte embedded at a random
-point. The response specification is parsed and expanded by pathoc, so you see
-syntax errors immediately. This really becomes handy when combined with the
-**-e** flag to show the expanded request::
-
-    07-06-16 12:32:01: >> 'GET':/p/:s'401:i35,\x27\\x1b\x27:h\x27Content-Length\x27=\x270\x27:h\x27Content-Length\x27=\x270\x27':h'Host'='localhost'
-    << 401 Unauthorized: 0 bytes
-
-Note that the embedded response has been resolved *before* being sent to
-the server, so that "ir,@1" (embed a random byte at a random location) has
-become "i15,\'o\'" (embed the character "o" at offset 15). You now have a
-pathoc request specification that is precisely reproducible, even with random
-components. This feature comes in terribly handy when testing a proxy, since
-you can now drive the server response completely from the client, and have a
-complete log of reproducible requests to analyze afterwards.
-
-
-Request Examples
-----------------
-
-.. list-table::
-    :widths: 50 50
-    :header-rows: 0
-
-    * - get:/
-      - Get path /
-
-    * - get:/:b@100
-      - 100 random bytes as the body
-
-    * - get:/:h"Etag"="&;drop table browsers;"
-      - Add a header
-
-    * - get:/:u"&;drop table browsers;"
-      - Add a User-Agent header
-
-    * - get:/:b@100:dr
-      - Drop the connection randomly
-
-    * - get:/:b@100,ascii:ir,@1
-      - 100 ASCII bytes as the body, and randomly inject a random byte
-
-    * - ws:/
-      - Initiate a websocket handshake.
-
-
-Response Examples
------------------
-
-.. list-table::
-    :widths: 50 50
-    :header-rows: 0
-
-
-    * - 200
-      - A basic HTTP 200 response.
-
-    * - 200:r
-      - A basic HTTP 200 response with no Content-Length header. This will hang.
-
-    * - 200:da
-      - Server-side disconnect after all content has been sent.
-
-    * - 200:b\@100
-      - 100 random bytes as the body. A Content-Length header is added, so the disconnect
-        is no longer needed.
-
-    * - 200:b\@100:h"Etag"="';drop table servers;"
-      - Add a Server header
-
-    * - 200:b\@100:dr
-      - Drop the connection randomly
-
-    * - 200:b\@100,ascii:ir,@1
-      - 100 ASCII bytes as the body, and randomly inject a random byte
-
-    * - 200:b\@1k:c"text/json"
-      - 1k of random bytes, with a text/json content type
-
-    * - 200:b\@1k:p50,120
-      - 1k of random bytes, pause for 120 seconds after 50 bytes
-
-    * - 200:b\@1k:pr,f
-      - 1k of random bytes, but hang forever at a random location
-
-    * - 200:b\@100:h\@1k,ascii_letters='foo'
-      - 100 ASCII bytes as the body, randomly generated 100k header name, with the value
-        'foo'.
diff --git a/docs/pathod/language.rst b/docs/pathod/language.rst
deleted file mode 100644
index fe4ef6ca..00000000
--- a/docs/pathod/language.rst
+++ /dev/null
@@ -1,257 +0,0 @@
-.. _language:
-
-language spec
-=============
-
-************
-HTTP Request
-************
-
-    **method:path:[colon-separated list of features]**
-
-.. list-table::
-    :widths: 20 80
-    :header-rows: 0
-
-    * - method
-      - A :ref:`VALUE` specifying the HTTP method to
-        use. Standard methods do not need to be enclosed in quotes, while
-        non-standard methods can be specified as quoted strings.
-
-        The special method **ws** creates a valid websocket upgrade
-        GET request, and signals to pathoc to switch to websocket recieve
-        mode if the server responds correctly. Apart from that, websocket
-        requests are just like any other, and all aspects of the request
-        can be over-ridden.
-    * - h\ :ref:`VALUE`\ =\ :ref:`VALUE`\
-      - Set a header.
-    * - r
-      - Set the **raw** flag on this response. Pathod will not calculate a
-        *Content-Length* header if a body is set.
-    * - c\ :ref:`VALUE`
-      - A shortcut for setting the Content-Type header. Equivalent to
-        ``h"Content-Type"=VALUE``
-    * - u\ :ref:`VALUE`
-        uSHORTCUT
-      - Set a User-Agent header on this request. You can specify either a
-        complete :ref:`VALUE`, or a User-Agent shortcut: **android**,
-        **blackberry**, **bingbot**, **chrome**, **firefox**, **googlebot**,
-        **ie9**, **ipad**, **iphone**, **safari**.
-    * - b\ :ref:`VALUE`
-      - Set the body. The appropriate Content-Length header is added
-        automatically unless the **r** flag is set.
-    * - s\ :ref:`VALUE`
-      - An embedded Response specification, appended to the path of the request.
-    * - x\ :ref:`INTEGER`
-      - Repeat this message N times.
-    * - d\ :ref:`OFFSET`
-      - Disconnect after OFFSET bytes (HTTP/1 only).
-    * - i\ :ref:`OFFSET`,\ :ref:`VALUE`
-      - Inject the specified value at the offset (HTTP/1 only)
-    * - p\ :ref:`OFFSET`,SECONDS
-      - Pause for SECONDS seconds after OFFSET bytes. SECONDS can be an integer
-        or "f" to pause forever (HTTP/1 only)
-
-
-*************
-HTTP Response
-*************
-
-    **code:[colon-separated list of features]**
-
-.. list-table::
-    :widths: 20 80
-    :header-rows: 0
-
-    * - code
-      - An integer specifying the HTTP response code.
-
-        The special method **ws** creates a valid websocket upgrade
-        response (code 101), and moves pathod to websocket mode. Apart
-        from that, websocket responses are just like any other, and all
-        aspects of the response can be over-ridden.
-    * - m\ :ref:`VALUE`
-      - HTTP Reason message. Automatically chosen according to the response
-        code if not specified. (HTTP/1 only)
-    * - h\ :ref:`VALUE`\ =\ :ref:`VALUE`\
-      - Set a header.
-    * - r
-      - Set the **raw** flag on this response. Pathod will not calculate a
-        *Content-Length* header if a body is set.
-    * - l\ :ref:`VALUE`
-      - A shortcut for setting the Location header. Equivalent to
-        ``h"Location"=VALUE``
-    * - c\ :ref:`VALUE`
-      - A shortcut for setting the Content-Type header. Equivalent to
-        ``h"Content-Type"=VALUE``
-    * - b\ :ref:`VALUE`
-      - Set the body. The appropriate Content-Length header is added
-        automatically unless the **r** flag is set.
-    * - d\ :ref:`OFFSET`
-      - Disconnect after OFFSET bytes (HTTP/1 only).
-    * - i\ :ref:`OFFSET`,\ :ref:`VALUE`
-      - Inject the specified value at the offset (HTTP/1 only)
-    * - p\ :ref:`OFFSET`,SECONDS
-      - Pause for SECONDS seconds after OFFSET bytes. SECONDS can be an integer
-        or "f" to pause forever (HTTP/1 only)
-
-***************
-Websocket Frame
-***************
-
-    **wf:[colon-separated list of features]**
-
-.. list-table::
-    :widths: 20 80
-    :header-rows: 0
-
-    * - b\ :ref:`VALUE`
-      - Set the frame payload. If a masking key is present, the value is
-        encoded automatically.
-    * - c\ :ref:`INTEGER`
-      - Set the op code. This can either be an integer from 0-15, or be one of
-        the following opcode names: **text** (the default), **continue**,
-        **binary**, **close**, **ping**, **pong**.
-    * - d\ :ref:`OFFSET`
-      - Disconnect after OFFSET bytes
-    * - i\ :ref:`OFFSET`,\ :ref:`VALUE`
-      - Inject the specified value at the offset
-    * - p\ :ref:`OFFSET`,SECONDS
-      - Pause for SECONDS seconds after OFFSET bytes. SECONDS can be an integer
-        or "f" to pause forever
-    * - x\ :ref:`INTEGER`
-      - Repeat this message N times.
-    * - [-]fin
-      - Set or un-set the **fin** bit.
-    * - k\ :ref:`VALUE`
-      - Set the masking key. The resulting value must be exactly 4 bytes long.
-        The special form **knone** specifies that no key should be set, even if
-        the mask bit is on.
-    * - l\ :ref:`INTEGER`
-      - Set the payload length in the frame header, regardless of the actual
-        body length.
-    * - [-]mask
-      - Set or un-set the <b>mask</b> bit.
-    * - r\ :ref:`VALUE`
-      - Set the raw frame payload. This disables masking, even if the key is present.
-    * - [-]rsv1
-      - Set or un-set the **rsv1** bit.
-    * - [-]rsv2
-      - Set or un-set the **rsv2** bit.
-    * - [-]rsv2
-      - Set or un-set the **rsv2** bit.
-
-
-
-**********
-Data types
-**********
-
-.. _INTEGER:
-
-INTEGER
-^^^^^^^
-
-.. _OFFSET:
-
-OFFSET
-^^^^^^
-
-Offsets are calculated relative to the base message, before any injections or
-other transforms are applied. They have 3 flavors:
-
-=======                 ==========================
-integer                 An integer byte offset
-**r**                   A random location
-**a**                   The end of the message
-=======                 ==========================
-
-
-.. _VALUE:
-
-VALUE
-^^^^^
-
-Literals
-""""""""
-
-Literal values are specified as a quoted strings::
-
-    "foo"
-
-Either single or double quotes are accepted, and quotes can be escaped with
-backslashes within the string::
-
-    'fo\'o'
-
-Literal values can contain Python-style backslash escape sequences::
-
-    'foo\r\nbar'
-
-
-
-Generated
-"""""""""
-
-An @-symbol lead-in specifies that generated data should be used. There are two
-components to a generator specification - a size, and a data type. By default
-pathod assumes a data type of "bytes".
-
-Here's a value specifier for generating 100 bytes::
-
-    @100
-
-You can use standard suffixes to indicate larger values. Here, for instance, is
-a specifier for generating 100 megabytes:
-
-    @100m
-
-Data is generated and served efficiently - if you really want to send a
-terabyte of data to a client, pathod can do it. The supported suffixes are:
-
-==========          ====================
-b                   1024**0 (bytes)
-k                   1024**1 (kilobytes)
-m                   1024**2 (megabytes)
-g                   1024**3 (gigabytes)
-t                   1024**4 (terabytes)
-==========          ====================
-
-Data types are separated from the size specification by a comma. This specification
-generates 100mb of ASCII::
-
-    @100m,ascii
-
-Supported data types are:
-
-=================          ==============================================
-ascii                      All ASCII characters
-ascii_letters              A-Za-z
-ascii_lowercase            a-z
-ascii_uppercase            A-Z
-bytes                      All 256 byte values
-digits                     0-9
-hexdigits                  0-f
-octdigits                  0-7
-punctuation                !"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~ and space
-whitespace                 \\t \\n \\x0b \\x0c \\r and space
-=================          ==============================================
-
-
-
-Files
-"""""
-
-You can load a value from a specified file path. To do so, you have to specify a
-_staticdir_ option to pathod on the command-line, like so:
-
->>> pathod -d ~/myassets
-
-All paths are relative paths under this directory. File loads are indicated by
-starting the value specifier with the left angle bracket::
-
-    <my/path
-
-The path value can also be a quoted string, with the same syntax as literals::
-
-    <"my/path"
diff --git a/docs/pathod/library.rst b/docs/pathod/library.rst
deleted file mode 100644
index b055d089..00000000
--- a/docs/pathod/library.rst
+++ /dev/null
@@ -1,14 +0,0 @@
-.. _library:
-
-pathod library
-==============
-
-Behind the pathod and pathoc command-line tools lurks the **pathod** library, a
-powerful way to manipulate and serve HTTP requests and responses from code. The
-canonical documentation for the library is in the code, and can be accessed
-using pydoc.
-
-
-.. literalinclude:: ../../examples/pathod/libpathod_pathoc.py
-   :caption: examples/pathod/libpathod_pathoc.py
-   :language: python
diff --git a/docs/pathod/test.rst b/docs/pathod/test.rst
deleted file mode 100644
index b337795a..00000000
--- a/docs/pathod/test.rst
+++ /dev/null
@@ -1,38 +0,0 @@
-.. _test:
-
-pathod.test
-===========
-
-The **pathod.test** module is a light, flexible testing layer for HTTP clients.
-It works by firing up a Pathod instance in a separate thread, letting you use
-Pathod's full abilities to generate responses, and then query Pathod's internal
-logs to establish what happened. All the mechanics of startup, shutdown, finding
-free ports and so forth are taken care of for you.
-
-The canonical docs can be accessed using pydoc:
-
->>> pydoc pathod.test
-
-The remainder of this page demonstrates some common interaction patterns using
-`Nose`_. These examples are
-also applicable with only minor modification to most commonly used Python testing
-engines.
-
-
-Context Manager
----------------
-
-.. literalinclude:: ../../examples/pathod/test_context.py
-   :caption: examples/pathod/test_context.py
-   :language: python
-
-
-One instance per test
----------------------
-
-.. literalinclude:: ../../examples/pathod/test_setup.py
-   :caption: examples/pathod/test_setup.py
-   :language: python
-
-
-.. _Nose: https://nose.readthedocs.org/en/latest/