Revert "Move the doc tree out into its own repo."

This reverts commit 8f88fcedd601c0033b4469b66626a83011879baf.

12 files changed, 360 insertions, 0 deletions

diff --git a/doc-src/features/anticache.html b/doc-src/features/anticache.html
new file mode 100644
index 00000000..f42903e8
--- /dev/null
+++ b/doc-src/features/anticache.html
@@ -0,0 +1,18 @@
+
+When the __anticache__ option is passed to mitmproxy, it removes headers
+(__if-none-match__ and __if-modified-since__) that might elicit a
+304-not-modified response from the server. This is useful when you want to make
+sure you capture an HTTP exchange in its totality. It's also often used during
+[client replay](@!urlTo("clientreplay.html")!@), when you want to make sure the
+server responds with complete data.
+
+<table class="table">
+    <tbody>
+        <tr>
+            <th width="20%">command-line</th> <td>--anticache</td>
+        </tr>
+        <tr>
+            <th>mitmproxy shortcut</th> <td><b>o</b> then <b>a</b></td>
+        </tr>
+    </tbody>
+</table>
diff --git a/doc-src/features/clientreplay.html b/doc-src/features/clientreplay.html
new file mode 100644
index 00000000..6638d078
--- /dev/null
+++ b/doc-src/features/clientreplay.html
@@ -0,0 +1,22 @@
+
+Client-side replay does what it says on the tin: you provide a previously saved
+HTTP conversation, and mitmproxy replays the client requests one by one. Note
+that mitmproxy serializes the requests, waiting for a response from the server
+before starting the next request. This might differ from the recorded
+conversation, where requests may have been made concurrently.
+
+You may want to use client-side replay in conjunction with the
+[anticache](@!urlTo("anticache.html")!@) option, to make sure the server
+responds with complete data. 
+
+
+<table class="table">
+    <tbody>
+        <tr>
+            <th width="20%">command-line</th> <td>-c path</td>
+        </tr>
+        <tr>
+            <th>mitmproxy shortcut</th> <td><b>c</b></td>
+        </tr>
+    </tbody>
+</table>
diff --git a/doc-src/features/filters.html b/doc-src/features/filters.html
new file mode 100644
index 00000000..c7f0f78b
--- /dev/null
+++ b/doc-src/features/filters.html
@@ -0,0 +1,36 @@
+
+Many commands in __mitmproxy__ and __mitmdump__ take a filter expression.
+Filter expressions consist of the following operators:
+
+<table class="table">
+    <tbody>
+    <!--(for i in filt_help)-->
+        <tr>
+            <td class="filt_cmd">@!i[0]!@</td>
+            <td class="filt_help">@!i[1]!@</td>
+        </tr>
+    <!--(end)-->
+    </tbody>
+</table>
+
+- Regexes are Python-style
+- Regexes can be specified as quoted strings
+- Header matching (~h, ~hq, ~hs) is against a string of the form "name: value".
+- Strings with no operators are matched against the request URL.
+- The default binary operator is &amp;.
+
+Examples
+========
+
+URL containing "google.com":
+
+    google\.com
+
+Requests whose body contains the string "test":
+
+    ~q ~b test
+
+Anything but requests with a text/html content type:
+
+    !(~q & ~t \"text/html\")
+
diff --git a/doc-src/features/forwardproxy.html b/doc-src/features/forwardproxy.html
new file mode 100644
index 00000000..203520d5
--- /dev/null
+++ b/doc-src/features/forwardproxy.html
@@ -0,0 +1,16 @@
+
+In this mode, mitmproxy accepts proxy requests and unconditionally forwards all
+requests to a specified upstream server. This is in contrast to <a
+href="@!urlTo("reverseproxy.html")!@">reverse proxy mode</a>, in which
+mitmproxy forwards ordinary HTTP requests to an upstream server.
+
+<table class="table">
+    <tbody>
+        <tr>
+            <th width="20%">command-line</th> <td>-F http[s]://hostname[:port]</td>
+        </tr>
+        <tr>
+            <th>mitmproxy shortcut</th> <td><b>F</b></td>
+        </tr>
+    </tbody>
+</table>
diff --git a/doc-src/features/index.py b/doc-src/features/index.py
new file mode 100644
index 00000000..e15f3311
--- /dev/null
+++ b/doc-src/features/index.py
@@ -0,0 +1,15 @@
+from countershape import Page
+
+pages = [
+    Page("anticache.html", "Anticache"),
+    Page("clientreplay.html", "Client-side replay"),
+    Page("filters.html", "Filter expressions"),
+    Page("forwardproxy.html", "Forward proxy mode"),
+    Page("setheaders.html", "Set Headers"),
+    Page("serverreplay.html", "Server-side replay"),
+    Page("sticky.html", "Sticky cookies and auth"),
+    Page("proxyauth.html", "Proxy Authentication"),
+    Page("replacements.html", "Replacements"),
+    Page("reverseproxy.html", "Reverse proxy mode"),
+    Page("upstreamcerts.html", "Upstream Certs"),
+]
diff --git a/doc-src/features/proxyauth.html b/doc-src/features/proxyauth.html
new file mode 100644
index 00000000..c22d50f3
--- /dev/null
+++ b/doc-src/features/proxyauth.html
@@ -0,0 +1,26 @@
+
+Asks the user for authentication before they are permitted to use the proxy.
+Authentication headers are stripped from the flows, so they are not passed to
+upstream servers. For now, only HTTP Basic authentication is supported. The
+proxy auth options are ignored if the proxy is in transparent or reverse proxy
+mode.
+
+<table class="table">
+    <tbody>
+        <tr>
+            <th width="20%">command-line</th>
+            <td>
+                <ul>
+                    <li>--nonanonymous</li>
+
+                    <li>--singleuser USER</li>
+
+                    <li>--htpasswd PATH</li>
+                </ul>
+            </td>
+        </tr>
+    </tbody>
+</table>
+
+
+
diff --git a/doc-src/features/replacements.html b/doc-src/features/replacements.html
new file mode 100644
index 00000000..c10fe2c3
--- /dev/null
+++ b/doc-src/features/replacements.html
@@ -0,0 +1,74 @@
+Mitmproxy lets you specify an arbitrary number of patterns that define text
+replacements within flows. Each pattern has 3 components: a filter that defines
+which flows a replacement applies to, a regular expression that defines what
+gets replaced, and a target value that defines what is substituted in.
+
+Replace hooks fire when either a client request or a server response is
+received. Only the matching flow component is affected: so, for example, if a
+replace hook is triggered on server response, the replacement is only run on
+the Response object leaving the Request intact. You control whether the hook
+triggers on the request, response or both using the filter pattern. If you need
+finer-grained control than this, it's simple to create a script using the
+replacement API on Flow components. 
+
+Replacement hooks are extremely handy in interactive testing of applications.
+For instance you can use a replace hook to replace the text "XSS" with a
+complicated XSS exploit, and then "inject" the exploit simply by interacting
+with the application through the browser. When used with tools like Firebug and
+mitmproxy's own interception abilities, replacement hooks can be an amazingly
+flexible and powerful feature. 
+
+
+## On the command-line
+
+The replacement hook command-line options use a compact syntax to make it easy
+to specify all three components at once. The general form is as follows:
+
+    /patt/regex/replacement
+
+Here, __patt__ is a mitmproxy filter expression, __regex__ is a valid Python
+regular expression, and __replacement__ is a string literal. The first
+character in the expression (__/__ in this case) defines what the separation
+character is. Here's an example of a valid expression that replaces "foo" with
+"bar" in all requests:
+
+    :~q:foo:bar
+
+In practice, it's pretty common for the replacement literal to be long and
+complex. For instance, it might be an XSS exploit that weighs in at hundreds or
+thousands of characters. To cope with this, there's a variation of the
+replacement hook specifier that lets you load the replacement text from a file.
+So, you might start __mitmdump__ as follows:
+
+<pre class="terminal">
+mitmdump --replace-from-file :~q:foo:~/xss-exploit
+</pre>
+
+This will load the replacement text from the file __~/xss-exploit__.
+
+Both the _--replace_ and _--replace-from-file_ flags can be passed multiple
+times.
+
+
+## Interactively
+
+The _R_ shortcut key in mitmproxy lets you add and edit replacement hooks using
+a built-in editor. The context-sensitive help (_h_) has complete usage
+information.
+
+<table class="table">
+    <tbody>
+        <tr>
+            <th width="20%">command-line</th>
+            <td>
+                <ul>
+                    <li>--replace</li>
+                    <li>--replace-from-file</li>
+                </ul>
+            </td>
+        </tr>
+        <tr>
+            <th>mitmproxy shortcut</th> <td><b>R</b></td>
+        </tr>
+    </tbody>
+</table>
diff --git a/doc-src/features/reverseproxy.html b/doc-src/features/reverseproxy.html
new file mode 100644
index 00000000..d399cdc0
--- /dev/null
+++ b/doc-src/features/reverseproxy.html
@@ -0,0 +1,19 @@
+
+In reverse proxy mode, mitmproxy accepts standard HTTP requests and forwards
+them to the specified upstream server. This is in contrast to <a
+href="@!urlTo("forwardproxy.html")!@">forward proxy mode</a>, in which
+mitmproxy forwards HTTP proxy requests to an upstream server.
+
+Note that the displayed URL for flows in this mode will use the value of the
+__Host__ header field from the request, not the reverse proxy server. 
+
+<table class="table">
+    <tbody>
+        <tr>
+            <th width="20%">command-line</th> <td>-P http[s]://hostname[:port]</td>
+        </tr>
+        <tr>
+            <th>mitmproxy shortcut</th> <td><b>P</b></td>
+        </tr>
+    </tbody>
+</table>
diff --git a/doc-src/features/serverreplay.html b/doc-src/features/serverreplay.html
new file mode 100644
index 00000000..1282be06
--- /dev/null
+++ b/doc-src/features/serverreplay.html
@@ -0,0 +1,35 @@
+
+- command-line: _-S path_ 
+- mitmproxy shortcut: _S_
+
+Server-side replay lets us replay server responses from a saved HTTP
+conversation.
+
+Matching requests with responses
+--------------------------------
+
+By default, __mitmproxy__ excludes request headers when matching incoming
+requests with responses from the replay file. This works in most circumstances,
+and makes it possible to replay server responses in situations where request
+headers would naturally vary, e.g. using a different user agent. The _--rheader
+headername_ command-line option allows you to override this behaviour by
+specifying individual headers that should be included in matching.
+
+
+Response refreshing
+-------------------
+
+Simply replaying server responses without modification will often result in
+unexpected behaviour. For example cookie timeouts that were in the future at
+the time a conversation was recorded might be in the past at the time it is
+replayed. By default, __mitmproxy__ refreshes server responses before sending
+them to the client. The __date__, __expires__ and __last-modified__ headers are
+all updated to have the same relative time offset as they had at the time of
+recording. So, if they were in the past at the time of recording, they will be
+in the past at the time of replay, and vice versa. Cookie expiry times are
+updated in a similar way.
+
+You can turn off response refreshing using the _--norefresh_ argument, or using
+the _o_ options shortcut within __mitmproxy__.
+
+
diff --git a/doc-src/features/setheaders.html b/doc-src/features/setheaders.html
new file mode 100644
index 00000000..b74525df
--- /dev/null
+++ b/doc-src/features/setheaders.html
@@ -0,0 +1,18 @@
+
+This feature lets you specify a set of headers to be added to requests or
+responses, based on a filter pattern. You can specify these either on the
+command-line, or through an interactive editor in mitmproxy.
+
+<table class="table">
+    <tbody>
+        <tr>
+            <th width="20%">command-line</th>
+            <td>
+                --setheader PATTERN
+            </td>
+        </tr>
+        <tr>
+            <th>mitmproxy shortcut</th> <td><b>H</b></td>
+        </tr>
+    </tbody>
+</table>
diff --git a/doc-src/features/sticky.html b/doc-src/features/sticky.html
new file mode 100644
index 00000000..59116067
--- /dev/null
+++ b/doc-src/features/sticky.html
@@ -0,0 +1,60 @@
+
+## Sticky cookies
+
+When the sticky cookie option is set, __mitmproxy__ will add the cookie most
+recently set by the server to any cookie-less request. Consider a service that
+sets a cookie to track the session after authentication. Using sticky cookies,
+you can fire up mitmproxy, and authenticate to a service as you usually would
+using a browser. After authentication, you can request authenticated resources
+through mitmproxy as if they were unauthenticated, because mitmproxy will
+automatically add the session tracking cookie to requests. Among other things,
+this lets you script interactions with authenticated resources (using tools
+like wget or curl) without having to worry about authentication. 
+
+Sticky cookies are especially powerful when used in conjunction with [client
+replay](@!urlTo("clientreplay.html")!@) - you can record the authentication
+process once, and simply replay it on startup every time you need to interact
+with the secured resources.
+
+<table class="table">
+    <tbody>
+        <tr>
+            <th width="20%">command-line</th>
+            <td>
+                <ul>
+                    <li>-t FILTER</li>
+                </ul>
+            </td>
+        </tr>
+        <tr>
+            <th>mitmproxy shortcut</th> <td><b>t</b></td>
+        </tr>
+    </tbody>
+</table>
+
+
+## Sticky auth
+
+The sticky auth option is analogous to the sticky cookie option, in that HTTP
+__Authorization__ headers are simply replayed to the server once they have been
+seen. This is enough to allow you to access a server resource using HTTP Basic
+authentication through the proxy. Note that __mitmproxy__ doesn't (yet) support
+replay of HTTP Digest authentication. 
+
+<table class="table">
+    <tbody>
+        <tr>
+            <th width="20%">command-line</th>
+            <td>
+                <ul>
+                    <li>-u FILTER</li>
+                </ul>
+            </td>
+        </tr>
+        <tr>
+            <th>mitmproxy shortcut</th> <td><b>u</b></td>
+        </tr>
+    </tbody>
+</table>
+
+
diff --git a/doc-src/features/upstreamcerts.html b/doc-src/features/upstreamcerts.html
new file mode 100644
index 00000000..8de75ee3
--- /dev/null
+++ b/doc-src/features/upstreamcerts.html
@@ -0,0 +1,21 @@
+When mitmproxy receives a connection destined for an SSL-protected service, it
+freezes the connection before reading its request data, and makes a connection
+to the upstream server to "sniff" the contents of its SSL certificate. The
+information gained - the __Common Name__ and __Subject Alternative Names__ - is
+then used to generate the interception certificate, which is sent to the client
+so the connection can continue.
+
+This rather intricate little dance lets us seamlessly generate correct
+certificates even if the client has specifed only an IP address rather than the
+hostname. It also means that we don't need to sniff additional data to generate
+certs in transparent mode.
+
+Upstream cert sniffing is on by default, and can optionally be turned off.
+
+<table class="table">
+    <tbody>
+        <tr>
+            <th width="20%">command-line</th> <td>--no-upstream-cert</td>
+        </tr>
+    </tbody>
+</table>