From 74b3b842fefec6a05d17bbdf365cd92c82fd3503 Mon Sep 17 00:00:00 2001 From: Maximilian Hils Date: Fri, 4 Sep 2015 16:17:55 +0200 Subject: rewrite basic docs for readthedocs --- docs/features/upstreamcerts.rst | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 docs/features/upstreamcerts.rst (limited to 'docs/features') diff --git a/docs/features/upstreamcerts.rst b/docs/features/upstreamcerts.rst new file mode 100644 index 00000000..a287daef --- /dev/null +++ b/docs/features/upstreamcerts.rst @@ -0,0 +1,4 @@ +.. _upstreamcerts: + +Upstream Certificates +===================== \ No newline at end of file -- cgit v1.2.3 From 853cd810757a02252402da22ae77cab77c5a3fb6 Mon Sep 17 00:00:00 2001 From: Maximilian Hils Date: Sun, 6 Sep 2015 01:37:15 +0200 Subject: docs++ --- docs/features/anticache.rst | 15 +++++++++ docs/features/clientreplay.rst | 18 +++++++++++ docs/features/filters.rst | 38 ++++++++++++++++++++++ docs/features/replacements.rst | 72 ++++++++++++++++++++++++++++++++++++++++++ docs/features/serverreplay.rst | 39 +++++++++++++++++++++++ docs/features/setheaders.rst | 18 +++++++++++ 6 files changed, 200 insertions(+) create mode 100644 docs/features/anticache.rst create mode 100644 docs/features/clientreplay.rst create mode 100644 docs/features/filters.rst create mode 100644 docs/features/replacements.rst create mode 100644 docs/features/serverreplay.rst create mode 100644 docs/features/setheaders.rst (limited to 'docs/features') diff --git a/docs/features/anticache.rst b/docs/features/anticache.rst new file mode 100644 index 00000000..5244587a --- /dev/null +++ b/docs/features/anticache.rst @@ -0,0 +1,15 @@ +.. _anticache: + +Anticache +========= +When the :option:`--anticache` option is passed to mitmproxy, it removes headers +(``if-none-match`` and ``if-modified-since``) that might elicit a +``304 not modified`` response from the server. This is useful when you want to make +sure you capture an HTTP exchange in its totality. It's also often used during +:ref:`clientreplay`, when you want to make sure the server responds with complete data. + + +================== ====================== +command-line :option:`--anticache` +mitmproxy shortcut :kbd:`o` then :kbd:`a` +================== ====================== \ No newline at end of file diff --git a/docs/features/clientreplay.rst b/docs/features/clientreplay.rst new file mode 100644 index 00000000..b8ca989e --- /dev/null +++ b/docs/features/clientreplay.rst @@ -0,0 +1,18 @@ +.. _clientreplay: + +Client-side replay +================== + +Client-side replay does what it says on the tin: you provide a previously saved +HTTP conversation, and mitmproxy replays the client requests one by one. Note +that mitmproxy serializes the requests, waiting for a response from the server +before starting the next request. This might differ from the recorded +conversation, where requests may have been made concurrently. + +You may want to use client-side replay in conjunction with the +:ref:`anticache` option, to make sure the server responds with complete data. + +================== ================= +command-line :option:`-c path` +mitmproxy shortcut :kbd:`c` +================== ================= \ No newline at end of file diff --git a/docs/features/filters.rst b/docs/features/filters.rst new file mode 100644 index 00000000..5b22376c --- /dev/null +++ b/docs/features/filters.rst @@ -0,0 +1,38 @@ +.. _filters: + +Filter expressions +================== + +Many commands in :program:`mitmproxy` and :program:`mitmdump` take a filter expression. +Filter expressions consist of the following operators: + +.. documentedlist:: + :listobject: libmproxy.filt.help + +- Regexes are Python-style +- Regexes can be specified as quoted strings +- Header matching (~h, ~hq, ~hs) is against a string of the form "name: value". +- Strings with no operators are matched against the request URL. +- The default binary operator is &. + +Examples +-------- + +URL containing "google.com": + +.. code-block:: none + + google\.com + +Requests whose body contains the string "test": + +.. code-block:: none + + ~q ~b test + +Anything but requests with a text/html content type: + +.. code-block:: none + + !(~q & ~t "text/html") + diff --git a/docs/features/replacements.rst b/docs/features/replacements.rst new file mode 100644 index 00000000..8f760866 --- /dev/null +++ b/docs/features/replacements.rst @@ -0,0 +1,72 @@ +.. _replacements: + +Replacements +============ + +Mitmproxy lets you specify an arbitrary number of patterns that define text +replacements within flows. Each pattern has 3 components: a filter that defines +which flows a replacement applies to, a regular expression that defines what +gets replaced, and a target value that defines what is substituted in. + +Replace hooks fire when either a client request or a server response is +received. Only the matching flow component is affected: so, for example, if a +replace hook is triggered on server response, the replacement is only run on +the Response object leaving the Request intact. You control whether the hook +triggers on the request, response or both using the filter pattern. If you need +finer-grained control than this, it's simple to create a script using the +replacement API on Flow components. + +Replacement hooks are extremely handy in interactive testing of applications. +For instance you can use a replace hook to replace the text "XSS" with a +complicated XSS exploit, and then "inject" the exploit simply by interacting +with the application through the browser. When used with tools like Firebug and +mitmproxy's own interception abilities, replacement hooks can be an amazingly +flexible and powerful feature. + + +On the command-line +------------------- + +The replacement hook command-line options use a compact syntax to make it easy +to specify all three components at once. The general form is as follows: + +.. code-block:: none + + /patt/regex/replacement + +Here, **patt** is a mitmproxy filter expression, **regex** is a valid Python +regular expression, and **replacement** is a string literal. The first +character in the expression (``/`` in this case) defines what the separation +character is. Here's an example of a valid expression that replaces "foo" with +"bar" in all requests: + +.. code-block:: none + + :~q:foo:bar + +In practice, it's pretty common for the replacement literal to be long and +complex. For instance, it might be an XSS exploit that weighs in at hundreds or +thousands of characters. To cope with this, there's a variation of the +replacement hook specifier that lets you load the replacement text from a file. +So, you might start **mitmdump** as follows: + +>>> mitmdump --replace-from-file :~q:foo:~/xss-exploit + +This will load the replacement text from the file ``~/xss-exploit``. + +Both the :option:`--replace` and :option:`--replace-from-file` flags can be passed multiple +times. + + +Interactively +------------- + +The :kbd:`R` shortcut key in the mitmproxy options menu (:kbd:`o`) lets you add and edit +replacement hooks using a built-in editor. The context-sensitive help (:kbd:`?`) has +complete usage information. + +================== ============================= +command-line :option:`--replace`, + :option:`--replace-from-file` +mitmproxy shortcut :kbd:`o` then :kbd:`R` +================== ============================= diff --git a/docs/features/serverreplay.rst b/docs/features/serverreplay.rst new file mode 100644 index 00000000..3b4af4e8 --- /dev/null +++ b/docs/features/serverreplay.rst @@ -0,0 +1,39 @@ +.. _serverreplay: + +Server-side replay +================== + +Server-side replay lets us replay server responses from a saved HTTP +conversation. + +Matching requests with responses +-------------------------------- + +By default, :program:`mitmproxy` excludes request headers when matching incoming +requests with responses from the replay file. This works in most circumstances, +and makes it possible to replay server responses in situations where request +headers would naturally vary, e.g. using a different user agent. +The :option:`--rheader headername` command-line option allows you to override +this behaviour by specifying individual headers that should be included in matching. + + +Response refreshing +------------------- + +Simply replaying server responses without modification will often result in +unexpected behaviour. For example cookie timeouts that were in the future at +the time a conversation was recorded might be in the past at the time it is +replayed. By default, :program:`mitmproxy` refreshes server responses before sending +them to the client. The **date**, **expires** and **last-modified** headers are +all updated to have the same relative time offset as they had at the time of +recording. So, if they were in the past at the time of recording, they will be +in the past at the time of replay, and vice versa. Cookie expiry times are +updated in a similar way. + +You can turn off response refreshing using the :option:`--norefresh` argument, or using +the :kbd:`o` options shortcut within :program:`mitmproxy`. + +================== ================= +command-line :option:`-S path` +mitmproxy shortcut :kbd:`S` +================== ================= \ No newline at end of file diff --git a/docs/features/setheaders.rst b/docs/features/setheaders.rst new file mode 100644 index 00000000..0a6c2296 --- /dev/null +++ b/docs/features/setheaders.rst @@ -0,0 +1,18 @@ +.. _setheaders: + +Set Headers +=========== + +This feature lets you specify a set of headers to be added to requests or +responses, based on a filter pattern. You can specify these either on the +command-line, or through an interactive editor in mitmproxy. + +Example: +.. code-block:: none + + mitmdump -R http://example.com --setheader :~q:Host:example.com + +================== ============================= +command-line :option:`--setheader PATTERN` +mitmproxy shortcut :kbd:`o` then :kbd:`H` +================== ============================= \ No newline at end of file -- cgit v1.2.3 From b4013659a81a48908eb3e060f04143ba1f9689bb Mon Sep 17 00:00:00 2001 From: Maximilian Hils Date: Sun, 6 Sep 2015 03:20:58 +0200 Subject: docs :tada: --- docs/features/passthrough.rst | 94 +++++++++++++++++++++++++++++++++++++ docs/features/proxyauth.rst | 17 +++++++ docs/features/responsestreaming.rst | 68 +++++++++++++++++++++++++++ docs/features/reverseproxy.rst | 56 ++++++++++++++++++++++ docs/features/setheaders.rst | 3 +- docs/features/socksproxy.rst | 10 ++++ docs/features/sticky.rst | 41 ++++++++++++++++ docs/features/tcpproxy.rst | 30 ++++++++++++ docs/features/upstreamcerts.rst | 21 ++++++++- docs/features/upstreamproxy.rst | 12 +++++ 10 files changed, 350 insertions(+), 2 deletions(-) create mode 100644 docs/features/passthrough.rst create mode 100644 docs/features/proxyauth.rst create mode 100644 docs/features/responsestreaming.rst create mode 100644 docs/features/reverseproxy.rst create mode 100644 docs/features/socksproxy.rst create mode 100644 docs/features/sticky.rst create mode 100644 docs/features/tcpproxy.rst create mode 100644 docs/features/upstreamproxy.rst (limited to 'docs/features') diff --git a/docs/features/passthrough.rst b/docs/features/passthrough.rst new file mode 100644 index 00000000..83374955 --- /dev/null +++ b/docs/features/passthrough.rst @@ -0,0 +1,94 @@ +.. _passthrough: + +Ignore Domains +============== + +There are two main reasons why you may want to exempt some traffic from mitmproxy's interception mechanism: + +- **Certificate pinning:** Some traffic is is protected using `Certificate Pinning`_ and + mitmproxy's interception leads to errors. For example, the Twitter app, Windows Update or + the Apple App Store fail to work if mitmproxy is active. +- **Convenience:** You really don't care about some parts of the traffic and just want them to go away. + +If you want to peek into (SSL-protected) non-HTTP connections, check out the :ref:`tcpproxy` feature. +If you want to ignore traffic from mitmproxy's processing because of large response bodies, +take a look at the :ref:`responsestreaming` feature. + +How it works +------------ + +================== ============================= +command-line :option:`--ignore regex` +mitmproxy shortcut :kbd:`o` then :kbd:`I` +================== ============================= + + +mitmproxy allows you to specify a regex which is matched against a ``host:port`` string +(e.g. "example.com:443") to determine hosts that should be excluded. + +There are two important quirks to consider: + +- **In transparent mode, the ignore pattern is matched against the IP.** While we usually infer the + hostname from the Host header if the :option:`--host` argument is passed to mitmproxy, we do not + have access to this information before the SSL handshake. +- In regular mode, explicit HTTP requests are never ignored. [#explicithttp]_ The ignore pattern is + applied on CONNECT requests, which initiate HTTPS or clear-text WebSocket connections. + +Tutorial +-------- + +If you just want to ignore one specific domain, there's usually a bulletproof method to do so: + +1. Run mitmproxy or mitmdump in verbose mode (:option:`-v`) and observe the ``host:port`` + information in the serverconnect messages. mitmproxy will filter on these. +2. Take the ``host:port`` string, surround it with ^ and $, escape all dots (. becomes \\.) + and use this as your ignore pattern: + +.. code-block:: none + :emphasize-lines: 6,7,9 + + >>> mitmdump -v + 127.0.0.1:50588: clientconnect + 127.0.0.1:50588: request + -> CONNECT example.com:443 HTTP/1.1 + 127.0.0.1:50588: Set new server address: example.com:443 + 127.0.0.1:50588: serverconnect + -> example.com:443 + ^C + >>> mitmproxy --ignore ^example\.com:443$ + + +Here are some other examples for ignore patterns: + +.. code-block:: none + + # Exempt traffic from the iOS App Store (the regex is lax, but usually just works): + --ignore apple.com:443 + # "Correct" version without false-positives: + --ignore '^(.+\.)?apple\.com:443$' + + # Ignore example.com, but not its subdomains: + --ignore '^example.com:' + + # Ignore everything but example.com and mitmproxy.org: + --ignore '^(?!example\.com)(?!mitmproxy\.org)' + + # Transparent mode: + --ignore 17\.178\.96\.59:443 + # IP address range: + --ignore 17\.178\.\d+\.\d+:443 + + +.. seealso:: + + - :ref:`tcpproxy` + - :ref:`responsestreaming` + +.. rubric:: Footnotes + +.. [#explicithttp] This stems from an limitation of explicit HTTP proxying: + A single connection can be re-used for multiple target domains - a + ``GET http://example.com/`` request may be followed by a ``GET http://evil.com/`` request on the + same connection. If we start to ignore the connection after the first request, + we would miss the relevant second one. +.. _Certificate Pinning: https://security.stackexchange.com/questions/29988/what-is-certificate-pinning \ No newline at end of file diff --git a/docs/features/proxyauth.rst b/docs/features/proxyauth.rst new file mode 100644 index 00000000..edc428a7 --- /dev/null +++ b/docs/features/proxyauth.rst @@ -0,0 +1,17 @@ +.. _proxyauth: + +Proxy Authentication +==================== + + +Asks the user for authentication before they are permitted to use the proxy. +Authentication headers are stripped from the flows, so they are not passed to +upstream servers. For now, only HTTP Basic authentication is supported. The +proxy auth options are not compatible with the transparent, socks or reverse proxy +mode. + +================== ============================= +command-line :option:`--nonanonymous`, + :option:`--singleuser USER`, + :option:`--htpasswd PATH` +================== ============================= \ No newline at end of file diff --git a/docs/features/responsestreaming.rst b/docs/features/responsestreaming.rst new file mode 100644 index 00000000..50fd0614 --- /dev/null +++ b/docs/features/responsestreaming.rst @@ -0,0 +1,68 @@ +.. _responsestreaming: + +Response Streaming +================== + +By using mitmproxy's streaming feature, response contents can be passed to the client incrementally +before they have been fully received by the proxy. This is especially useful for large binary files +such as videos, where buffering the whole file slows down the client's browser. + +By default, mitmproxy will read the entire response, perform any indicated +manipulations on it and then send the (possibly modified) response to +the client. In some cases this is undesirable and you may wish to "stream" +the reponse back to the client. When streaming is enabled, the response is +not buffered on the proxy but directly sent back to the client instead. + +On the command-line +------------------- + +Streaming can be enabled on the command line for all response bodies exceeding a certain size. +The SIZE argument understands k/m/g suffixes, e.g. 3m for 3 megabytes. + +================== ============================= +command-line :option:`--stream SIZE` +================== ============================= + +.. warning:: + + When response streaming is enabled, **streamed response contents will not be + recorded or preserved in any way.** + +.. note:: + + When response streaming is enabled, the response body cannot be modified by the usual means. + +Customizing Response Streaming +------------------------------ + +You can also use an :ref:`inlinescripts` to customize exactly +which responses are streamed. + +Responses that should be tagged for streaming by setting their ``.stream`` attribute to ``True``: + +.. literalinclude:: ../../examples/stream.py + :caption: examples/stream.py + :language: python + +Implementation Details +---------------------- + +When response streaming is enabled, portions of the code which would have otherwise performed changes +on the response body will see an empty response body instead (:py:data:`netlib.http.CONTENT_MISSING`). +Any modifications will be ignored. + +Streamed responses are usually sent in chunks of 4096 bytes. If the response is sent with a +``Transfer-Encoding: chunked`` header, the response will be streamed one chunk at a time. + +Modifying streamed data +----------------------- + +If the ``.stream`` attribute is callable, ``.stream`` will wrap the generator that yields all chunks. + +.. literalinclude:: ../../examples/stream_modify.py + :caption: examples/stream_modify.py + :language: python + +.. seealso:: + + - :ref:`passthrough` \ No newline at end of file diff --git a/docs/features/reverseproxy.rst b/docs/features/reverseproxy.rst new file mode 100644 index 00000000..87a598ff --- /dev/null +++ b/docs/features/reverseproxy.rst @@ -0,0 +1,56 @@ +.. _reverseproxy: + +Reverse Proxy +============= + +In reverse proxy mode, mitmproxy accepts standard HTTP requests and forwards +them to the specified upstream server. This is in contrast to :ref:`upstreamproxy`, in which +mitmproxy forwards HTTP proxy requests to an upstream proxy server. + +================== ===================================== +command-line :option:`-R http[s]://hostname[:port]` +================== ===================================== + +Here, **scheme** signifies if the proxy should use TLS to connect to the server. +mitmproxy always accepts both encrypted and unencrypted requests and transforms +them to what the server expects. + +.. code-block:: none + + >>> mitmdump -R https://httpbin.org -p 80 + >>> curl http://localhost/ + # requests will be transparently upgraded to TLS by mitmproxy + + >>> mitmdump -R https://httpbin.org -p 443 + >>> curl https://localhost/ + # mitmproxy will use TLS on both ends. + + +Host Header +----------- + +In reverse proxy mode, mitmproxy does not rewrite the host header. While often useful, this +may lead to issues with public web servers. For example, consider the following scenario: + +.. code-block:: none + :emphasize-lines: 5 + + >>> mitmdump -d -R http://example.com/ + >>> curl http://localhost:8080/ + + >> GET https://example.com/ + Host: localhost:8080 + User-Agent: curl/7.35.0 + [...] + + << 404 Not Found 345B + +Since the Host header doesn't match "example.com", an error is returned. +There are two ways to solve this: + +1. Modify the hosts file of your OS so that "example.com" resolves to your proxy's IP. + Then, access example.com directly. Make sure that your proxy can still resolve the original IP + or specify an IP in mitmproxy. +2. Use mitmproxy's :ref:`setheaders` feature to rewrite the host header: ``--setheader :~q:Host:example.com``. + However, keep in mind that absolute URLs within the returned document or HTTP redirects will + cause the client application to bypass the proxy. diff --git a/docs/features/setheaders.rst b/docs/features/setheaders.rst index 0a6c2296..f118e6f8 100644 --- a/docs/features/setheaders.rst +++ b/docs/features/setheaders.rst @@ -7,7 +7,8 @@ This feature lets you specify a set of headers to be added to requests or responses, based on a filter pattern. You can specify these either on the command-line, or through an interactive editor in mitmproxy. -Example: +Example: Set the **Host** header to "example.com" for all requests. + .. code-block:: none mitmdump -R http://example.com --setheader :~q:Host:example.com diff --git a/docs/features/socksproxy.rst b/docs/features/socksproxy.rst new file mode 100644 index 00000000..fb9117f2 --- /dev/null +++ b/docs/features/socksproxy.rst @@ -0,0 +1,10 @@ +.. _socksproxy: + +SOCKS Mode +========== + +In this mode, mitmproxy acts as a SOCKS5 proxy server. + +================== ================= +command-line :option:`--socks` +================== ================= \ No newline at end of file diff --git a/docs/features/sticky.rst b/docs/features/sticky.rst new file mode 100644 index 00000000..e155fb9b --- /dev/null +++ b/docs/features/sticky.rst @@ -0,0 +1,41 @@ +.. _sticky: + +Sticky cookies and auth +======================= + +Sticky cookies +-------------- + +When the sticky cookie option is set, __mitmproxy__ will add the cookie most +recently set by the server to any cookie-less request. Consider a service that +sets a cookie to track the session after authentication. Using sticky cookies, +you can fire up mitmproxy, and authenticate to a service as you usually would +using a browser. After authentication, you can request authenticated resources +through mitmproxy as if they were unauthenticated, because mitmproxy will +automatically add the session tracking cookie to requests. Among other things, +this lets you script interactions with authenticated resources (using tools +like wget or curl) without having to worry about authentication. + +Sticky cookies are especially powerful when used in conjunction with :ref:`clientreplay` - you can +record the authentication process once, and simply replay it on startup every time you need +to interact with the secured resources. + +================== ====================== +command-line :option:`-t FILTER` +mitmproxy shortcut :kbd:`o` then :kbd:`t` +================== ====================== + + +Sticky auth +----------- + +The sticky auth option is analogous to the sticky cookie option, in that HTTP +**Authorization** headers are simply replayed to the server once they have been +seen. This is enough to allow you to access a server resource using HTTP Basic +authentication through the proxy. Note that :program:`mitmproxy` doesn't (yet) support +replay of HTTP Digest authentication. + +================== ====================== +command-line :option:`-u FILTER` +mitmproxy shortcut :kbd:`o` then :kbd:`A` +================== ====================== \ No newline at end of file diff --git a/docs/features/tcpproxy.rst b/docs/features/tcpproxy.rst new file mode 100644 index 00000000..53df8ed6 --- /dev/null +++ b/docs/features/tcpproxy.rst @@ -0,0 +1,30 @@ +.. _tcpproxy: + +TCP Proxy +========= + +WebSockets or other non-HTTP protocols are not supported by mitmproxy yet. However, you can exempt +hostnames from processing, so that mitmproxy acts as a generic TCP forwarder. +This feature is closely related to the :ref:`passthrough` functionality, +but differs in two important aspects: + +- The raw TCP messages are printed to the event log. +- SSL connections will be intercepted. + +Please note that message interception or modification are not possible yet. +If you are not interested in the raw TCP messages, you should use the ignore domains feature. + +How it works +------------ + +================== ====================== +command-line :option:`--tcp HOST` +mitmproxy shortcut :kbd:`o` then :kbd:`T` +================== ====================== + +For a detailed description how the hostname pattern works, please look at the :ref:`passthrough` feature. + +.. seealso:: + + - :ref:`passthrough` + - :ref:`responsestreaming` diff --git a/docs/features/upstreamcerts.rst b/docs/features/upstreamcerts.rst index a287daef..84cfb84e 100644 --- a/docs/features/upstreamcerts.rst +++ b/docs/features/upstreamcerts.rst @@ -1,4 +1,23 @@ .. _upstreamcerts: Upstream Certificates -===================== \ No newline at end of file +===================== + +When mitmproxy receives a connection destined for an SSL-protected service, it +freezes the connection before reading its request data, and makes a connection +to the upstream server to "sniff" the contents of its SSL certificate. The +information gained - the **Common Name** and **Subject Alternative Names** - is +then used to generate the interception certificate, which is sent to the client +so the connection can continue. + +This rather intricate little dance lets us seamlessly generate correct +certificates even if the client has specifed only an IP address rather than the +hostname. It also means that we don't need to sniff additional data to generate +certs in transparent mode. + +Upstream cert sniffing is on by default, and can optionally be turned off. + +================== ============================= +command-line :option:`--no-upstream-cert` +mitmproxy shortcut :kbd:`o` then :kbd:`U` +================== ============================= \ No newline at end of file diff --git a/docs/features/upstreamproxy.rst b/docs/features/upstreamproxy.rst new file mode 100644 index 00000000..e06833c2 --- /dev/null +++ b/docs/features/upstreamproxy.rst @@ -0,0 +1,12 @@ +.. _upstreamproxy: + +Upstream proxy mode +=================== + +In this mode, mitmproxy accepts proxy requests and unconditionally forwards all +requests to a specified upstream proxy server. This is in contrast to :ref:`reverseproxy`, +in which mitmproxy forwards ordinary HTTP requests to an upstream server. + +================== =================================== +command-line :option:`-U http://hostname[:port]` +================== =================================== -- cgit v1.2.3 From c4286b15dc3d95f52b7ce5b5292796109fa77f3f Mon Sep 17 00:00:00 2001 From: Maximilian Hils Date: Mon, 7 Sep 2015 10:52:18 +0200 Subject: docs: minor fixes --- docs/features/anticache.rst | 2 +- docs/features/clientreplay.rst | 2 +- docs/features/passthrough.rst | 11 +++++++---- docs/features/proxyauth.rst | 2 +- docs/features/responsestreaming.rst | 11 ++++++----- docs/features/reverseproxy.rst | 3 ++- docs/features/serverreplay.rst | 2 +- docs/features/setheaders.rst | 2 +- docs/features/socksproxy.rst | 2 +- docs/features/sticky.rst | 2 +- docs/features/tcpproxy.rst | 3 ++- docs/features/upstreamcerts.rst | 2 +- 12 files changed, 25 insertions(+), 19 deletions(-) (limited to 'docs/features') diff --git a/docs/features/anticache.rst b/docs/features/anticache.rst index 5244587a..65d22bab 100644 --- a/docs/features/anticache.rst +++ b/docs/features/anticache.rst @@ -12,4 +12,4 @@ sure you capture an HTTP exchange in its totality. It's also often used during ================== ====================== command-line :option:`--anticache` mitmproxy shortcut :kbd:`o` then :kbd:`a` -================== ====================== \ No newline at end of file +================== ====================== diff --git a/docs/features/clientreplay.rst b/docs/features/clientreplay.rst index b8ca989e..b0eb6792 100644 --- a/docs/features/clientreplay.rst +++ b/docs/features/clientreplay.rst @@ -15,4 +15,4 @@ You may want to use client-side replay in conjunction with the ================== ================= command-line :option:`-c path` mitmproxy shortcut :kbd:`c` -================== ================= \ No newline at end of file +================== ================= diff --git a/docs/features/passthrough.rst b/docs/features/passthrough.rst index 83374955..80521393 100644 --- a/docs/features/passthrough.rst +++ b/docs/features/passthrough.rst @@ -3,14 +3,17 @@ Ignore Domains ============== -There are two main reasons why you may want to exempt some traffic from mitmproxy's interception mechanism: +There are two main reasons why you may want to exempt some traffic from mitmproxy's interception +mechanism: - **Certificate pinning:** Some traffic is is protected using `Certificate Pinning`_ and mitmproxy's interception leads to errors. For example, the Twitter app, Windows Update or the Apple App Store fail to work if mitmproxy is active. -- **Convenience:** You really don't care about some parts of the traffic and just want them to go away. +- **Convenience:** You really don't care about some parts of the traffic and just want them to go + away. -If you want to peek into (SSL-protected) non-HTTP connections, check out the :ref:`tcpproxy` feature. +If you want to peek into (SSL-protected) non-HTTP connections, check out the :ref:`tcpproxy` +feature. If you want to ignore traffic from mitmproxy's processing because of large response bodies, take a look at the :ref:`responsestreaming` feature. @@ -91,4 +94,4 @@ Here are some other examples for ignore patterns: ``GET http://example.com/`` request may be followed by a ``GET http://evil.com/`` request on the same connection. If we start to ignore the connection after the first request, we would miss the relevant second one. -.. _Certificate Pinning: https://security.stackexchange.com/questions/29988/what-is-certificate-pinning \ No newline at end of file +.. _Certificate Pinning: https://security.stackexchange.com/questions/29988/what-is-certificate-pinning diff --git a/docs/features/proxyauth.rst b/docs/features/proxyauth.rst index edc428a7..bfd32fbd 100644 --- a/docs/features/proxyauth.rst +++ b/docs/features/proxyauth.rst @@ -14,4 +14,4 @@ mode. command-line :option:`--nonanonymous`, :option:`--singleuser USER`, :option:`--htpasswd PATH` -================== ============================= \ No newline at end of file +================== ============================= diff --git a/docs/features/responsestreaming.rst b/docs/features/responsestreaming.rst index 50fd0614..af3d889a 100644 --- a/docs/features/responsestreaming.rst +++ b/docs/features/responsestreaming.rst @@ -47,9 +47,9 @@ Responses that should be tagged for streaming by setting their ``.stream`` attri Implementation Details ---------------------- -When response streaming is enabled, portions of the code which would have otherwise performed changes -on the response body will see an empty response body instead (:py:data:`netlib.http.CONTENT_MISSING`). -Any modifications will be ignored. +When response streaming is enabled, portions of the code which would have otherwise performed +changes on the response body will see an empty response body instead +(:py:data:`netlib.http.CONTENT_MISSING`). Any modifications will be ignored. Streamed responses are usually sent in chunks of 4096 bytes. If the response is sent with a ``Transfer-Encoding: chunked`` header, the response will be streamed one chunk at a time. @@ -57,7 +57,8 @@ Streamed responses are usually sent in chunks of 4096 bytes. If the response is Modifying streamed data ----------------------- -If the ``.stream`` attribute is callable, ``.stream`` will wrap the generator that yields all chunks. +If the ``.stream`` attribute is callable, ``.stream`` will wrap the generator that yields all +chunks. .. literalinclude:: ../../examples/stream_modify.py :caption: examples/stream_modify.py @@ -65,4 +66,4 @@ If the ``.stream`` attribute is callable, ``.stream`` will wrap the generator th .. seealso:: - - :ref:`passthrough` \ No newline at end of file + - :ref:`passthrough` diff --git a/docs/features/reverseproxy.rst b/docs/features/reverseproxy.rst index 87a598ff..77e9327c 100644 --- a/docs/features/reverseproxy.rst +++ b/docs/features/reverseproxy.rst @@ -51,6 +51,7 @@ There are two ways to solve this: 1. Modify the hosts file of your OS so that "example.com" resolves to your proxy's IP. Then, access example.com directly. Make sure that your proxy can still resolve the original IP or specify an IP in mitmproxy. -2. Use mitmproxy's :ref:`setheaders` feature to rewrite the host header: ``--setheader :~q:Host:example.com``. +2. Use mitmproxy's :ref:`setheaders` feature to rewrite the host header: + ``--setheader :~q:Host:example.com``. However, keep in mind that absolute URLs within the returned document or HTTP redirects will cause the client application to bypass the proxy. diff --git a/docs/features/serverreplay.rst b/docs/features/serverreplay.rst index 3b4af4e8..261a1bd6 100644 --- a/docs/features/serverreplay.rst +++ b/docs/features/serverreplay.rst @@ -36,4 +36,4 @@ the :kbd:`o` options shortcut within :program:`mitmproxy`. ================== ================= command-line :option:`-S path` mitmproxy shortcut :kbd:`S` -================== ================= \ No newline at end of file +================== ================= diff --git a/docs/features/setheaders.rst b/docs/features/setheaders.rst index f118e6f8..cbc8b6a5 100644 --- a/docs/features/setheaders.rst +++ b/docs/features/setheaders.rst @@ -16,4 +16,4 @@ Example: Set the **Host** header to "example.com" for all requests. ================== ============================= command-line :option:`--setheader PATTERN` mitmproxy shortcut :kbd:`o` then :kbd:`H` -================== ============================= \ No newline at end of file +================== ============================= diff --git a/docs/features/socksproxy.rst b/docs/features/socksproxy.rst index fb9117f2..76d4cda9 100644 --- a/docs/features/socksproxy.rst +++ b/docs/features/socksproxy.rst @@ -7,4 +7,4 @@ In this mode, mitmproxy acts as a SOCKS5 proxy server. ================== ================= command-line :option:`--socks` -================== ================= \ No newline at end of file +================== ================= diff --git a/docs/features/sticky.rst b/docs/features/sticky.rst index e155fb9b..a79cbe8d 100644 --- a/docs/features/sticky.rst +++ b/docs/features/sticky.rst @@ -38,4 +38,4 @@ replay of HTTP Digest authentication. ================== ====================== command-line :option:`-u FILTER` mitmproxy shortcut :kbd:`o` then :kbd:`A` -================== ====================== \ No newline at end of file +================== ====================== diff --git a/docs/features/tcpproxy.rst b/docs/features/tcpproxy.rst index 53df8ed6..fd0746a2 100644 --- a/docs/features/tcpproxy.rst +++ b/docs/features/tcpproxy.rst @@ -22,7 +22,8 @@ command-line :option:`--tcp HOST` mitmproxy shortcut :kbd:`o` then :kbd:`T` ================== ====================== -For a detailed description how the hostname pattern works, please look at the :ref:`passthrough` feature. +For a detailed description how the hostname pattern works, please look at the :ref:`passthrough` +feature. .. seealso:: diff --git a/docs/features/upstreamcerts.rst b/docs/features/upstreamcerts.rst index 84cfb84e..c687aec7 100644 --- a/docs/features/upstreamcerts.rst +++ b/docs/features/upstreamcerts.rst @@ -20,4 +20,4 @@ Upstream cert sniffing is on by default, and can optionally be turned off. ================== ============================= command-line :option:`--no-upstream-cert` mitmproxy shortcut :kbd:`o` then :kbd:`U` -================== ============================= \ No newline at end of file +================== ============================= -- cgit v1.2.3 From c3ba98b68127b1a0d974e564c0c9af97b97f2a06 Mon Sep 17 00:00:00 2001 From: Maximilian Hils Date: Tue, 8 Sep 2015 15:36:24 +0200 Subject: docs: update documentedlist, add filter table headers --- docs/features/filters.rst | 1 + 1 file changed, 1 insertion(+) (limited to 'docs/features') diff --git a/docs/features/filters.rst b/docs/features/filters.rst index 5b22376c..2adcfb70 100644 --- a/docs/features/filters.rst +++ b/docs/features/filters.rst @@ -7,6 +7,7 @@ Many commands in :program:`mitmproxy` and :program:`mitmdump` take a filter expr Filter expressions consist of the following operators: .. documentedlist:: + :header: "Expression" "Description" :listobject: libmproxy.filt.help - Regexes are Python-style -- cgit v1.2.3