diff options
Diffstat (limited to 'docs/features')
| -rw-r--r-- | docs/features/anticache.rst | 15 | ||||
| -rw-r--r-- | docs/features/clientreplay.rst | 18 | ||||
| -rw-r--r-- | docs/features/filters.rst | 39 | ||||
| -rw-r--r-- | docs/features/passthrough.rst | 97 | ||||
| -rw-r--r-- | docs/features/proxyauth.rst | 17 | ||||
| -rw-r--r-- | docs/features/replacements.rst | 72 | ||||
| -rw-r--r-- | docs/features/responsestreaming.rst | 69 | ||||
| -rw-r--r-- | docs/features/reverseproxy.rst | 57 | ||||
| -rw-r--r-- | docs/features/serverreplay.rst | 39 | ||||
| -rw-r--r-- | docs/features/setheaders.rst | 19 | ||||
| -rw-r--r-- | docs/features/socksproxy.rst | 10 | ||||
| -rw-r--r-- | docs/features/sticky.rst | 41 | ||||
| -rw-r--r-- | docs/features/tcpproxy.rst | 31 | ||||
| -rw-r--r-- | docs/features/upstreamcerts.rst | 23 | ||||
| -rw-r--r-- | docs/features/upstreamproxy.rst | 12 |
15 files changed, 0 insertions, 559 deletions
diff --git a/docs/features/anticache.rst b/docs/features/anticache.rst deleted file mode 100644 index 65d22bab..00000000 --- a/docs/features/anticache.rst +++ /dev/null @@ -1,15 +0,0 @@ -.. _anticache: - -Anticache -========= -When the :option:`--anticache` option is passed to mitmproxy, it removes headers -(``if-none-match`` and ``if-modified-since``) that might elicit a -``304 not modified`` response from the server. This is useful when you want to make -sure you capture an HTTP exchange in its totality. It's also often used during -:ref:`clientreplay`, when you want to make sure the server responds with complete data. - - -================== ====================== -command-line :option:`--anticache` -mitmproxy shortcut :kbd:`o` then :kbd:`a` -================== ====================== diff --git a/docs/features/clientreplay.rst b/docs/features/clientreplay.rst deleted file mode 100644 index b0eb6792..00000000 --- a/docs/features/clientreplay.rst +++ /dev/null @@ -1,18 +0,0 @@ -.. _clientreplay: - -Client-side replay -================== - -Client-side replay does what it says on the tin: you provide a previously saved -HTTP conversation, and mitmproxy replays the client requests one by one. Note -that mitmproxy serializes the requests, waiting for a response from the server -before starting the next request. This might differ from the recorded -conversation, where requests may have been made concurrently. - -You may want to use client-side replay in conjunction with the -:ref:`anticache` option, to make sure the server responds with complete data. - -================== ================= -command-line :option:`-c path` -mitmproxy shortcut :kbd:`c` -================== ================= diff --git a/docs/features/filters.rst b/docs/features/filters.rst deleted file mode 100644 index 2adcfb70..00000000 --- a/docs/features/filters.rst +++ /dev/null @@ -1,39 +0,0 @@ -.. _filters: - -Filter expressions -================== - -Many commands in :program:`mitmproxy` and :program:`mitmdump` take a filter expression. -Filter expressions consist of the following operators: - -.. documentedlist:: - :header: "Expression" "Description" - :listobject: libmproxy.filt.help - -- Regexes are Python-style -- Regexes can be specified as quoted strings -- Header matching (~h, ~hq, ~hs) is against a string of the form "name: value". -- Strings with no operators are matched against the request URL. -- The default binary operator is &. - -Examples --------- - -URL containing "google.com": - -.. code-block:: none - - google\.com - -Requests whose body contains the string "test": - -.. code-block:: none - - ~q ~b test - -Anything but requests with a text/html content type: - -.. code-block:: none - - !(~q & ~t "text/html") - diff --git a/docs/features/passthrough.rst b/docs/features/passthrough.rst deleted file mode 100644 index b7b5df84..00000000 --- a/docs/features/passthrough.rst +++ /dev/null @@ -1,97 +0,0 @@ -.. _passthrough: - -Ignore Domains -============== - -There are two main reasons why you may want to exempt some traffic from mitmproxy's interception -mechanism: - -- **Certificate pinning:** Some traffic is is protected using `Certificate Pinning`_ and - mitmproxy's interception leads to errors. For example, the Twitter app, Windows Update or - the Apple App Store fail to work if mitmproxy is active. -- **Convenience:** You really don't care about some parts of the traffic and just want them to go - away. - -If you want to peek into (SSL-protected) non-HTTP connections, check out the :ref:`tcpproxy` -feature. -If you want to ignore traffic from mitmproxy's processing because of large response bodies, -take a look at the :ref:`responsestreaming` feature. - -How it works ------------- - -================== ============================= -command-line :option:`--ignore regex` -mitmproxy shortcut :kbd:`o` then :kbd:`I` -================== ============================= - - -mitmproxy allows you to specify a regex which is matched against a ``host:port`` string -(e.g. "example.com:443") to determine hosts that should be excluded. - -There are two important quirks to consider: - -- **In transparent mode, the ignore pattern is matched against the IP and ClientHello SNI host.** While we usually infer the - hostname from the Host header if the :option:`--host` argument is passed to mitmproxy, we do not - have access to this information before the SSL handshake. If the client uses SNI however, then we treat the SNI host as an ignore target. -- In regular mode, explicit HTTP requests are never ignored. [#explicithttp]_ The ignore pattern is - applied on CONNECT requests, which initiate HTTPS or clear-text WebSocket connections. - -Tutorial --------- - -If you just want to ignore one specific domain, there's usually a bulletproof method to do so: - -1. Run mitmproxy or mitmdump in verbose mode (:option:`-v`) and observe the ``host:port`` - information in the serverconnect messages. mitmproxy will filter on these. -2. Take the ``host:port`` string, surround it with ^ and $, escape all dots (. becomes \\.) - and use this as your ignore pattern: - -.. code-block:: none - :emphasize-lines: 6,7,9 - - >>> mitmdump -v - 127.0.0.1:50588: clientconnect - 127.0.0.1:50588: request - -> CONNECT example.com:443 HTTP/1.1 - 127.0.0.1:50588: Set new server address: example.com:443 - 127.0.0.1:50588: serverconnect - -> example.com:443 - ^C - >>> mitmproxy --ignore ^example\.com:443$ - - -Here are some other examples for ignore patterns: - -.. code-block:: none - - # Exempt traffic from the iOS App Store (the regex is lax, but usually just works): - --ignore apple.com:443 - # "Correct" version without false-positives: - --ignore '^(.+\.)?apple\.com:443$' - - # Ignore example.com, but not its subdomains: - --ignore '^example.com:' - - # Ignore everything but example.com and mitmproxy.org: - --ignore '^(?!example\.com)(?!mitmproxy\.org)' - - # Transparent mode: - --ignore 17\.178\.96\.59:443 - # IP address range: - --ignore 17\.178\.\d+\.\d+:443 - - -.. seealso:: - - - :ref:`tcpproxy` - - :ref:`responsestreaming` - -.. rubric:: Footnotes - -.. [#explicithttp] This stems from an limitation of explicit HTTP proxying: - A single connection can be re-used for multiple target domains - a - ``GET http://example.com/`` request may be followed by a ``GET http://evil.com/`` request on the - same connection. If we start to ignore the connection after the first request, - we would miss the relevant second one. -.. _Certificate Pinning: https://security.stackexchange.com/questions/29988/what-is-certificate-pinning diff --git a/docs/features/proxyauth.rst b/docs/features/proxyauth.rst deleted file mode 100644 index bfd32fbd..00000000 --- a/docs/features/proxyauth.rst +++ /dev/null @@ -1,17 +0,0 @@ -.. _proxyauth: - -Proxy Authentication -==================== - - -Asks the user for authentication before they are permitted to use the proxy. -Authentication headers are stripped from the flows, so they are not passed to -upstream servers. For now, only HTTP Basic authentication is supported. The -proxy auth options are not compatible with the transparent, socks or reverse proxy -mode. - -================== ============================= -command-line :option:`--nonanonymous`, - :option:`--singleuser USER`, - :option:`--htpasswd PATH` -================== ============================= diff --git a/docs/features/replacements.rst b/docs/features/replacements.rst deleted file mode 100644 index 8f760866..00000000 --- a/docs/features/replacements.rst +++ /dev/null @@ -1,72 +0,0 @@ -.. _replacements: - -Replacements -============ - -Mitmproxy lets you specify an arbitrary number of patterns that define text -replacements within flows. Each pattern has 3 components: a filter that defines -which flows a replacement applies to, a regular expression that defines what -gets replaced, and a target value that defines what is substituted in. - -Replace hooks fire when either a client request or a server response is -received. Only the matching flow component is affected: so, for example, if a -replace hook is triggered on server response, the replacement is only run on -the Response object leaving the Request intact. You control whether the hook -triggers on the request, response or both using the filter pattern. If you need -finer-grained control than this, it's simple to create a script using the -replacement API on Flow components. - -Replacement hooks are extremely handy in interactive testing of applications. -For instance you can use a replace hook to replace the text "XSS" with a -complicated XSS exploit, and then "inject" the exploit simply by interacting -with the application through the browser. When used with tools like Firebug and -mitmproxy's own interception abilities, replacement hooks can be an amazingly -flexible and powerful feature. - - -On the command-line -------------------- - -The replacement hook command-line options use a compact syntax to make it easy -to specify all three components at once. The general form is as follows: - -.. code-block:: none - - /patt/regex/replacement - -Here, **patt** is a mitmproxy filter expression, **regex** is a valid Python -regular expression, and **replacement** is a string literal. The first -character in the expression (``/`` in this case) defines what the separation -character is. Here's an example of a valid expression that replaces "foo" with -"bar" in all requests: - -.. code-block:: none - - :~q:foo:bar - -In practice, it's pretty common for the replacement literal to be long and -complex. For instance, it might be an XSS exploit that weighs in at hundreds or -thousands of characters. To cope with this, there's a variation of the -replacement hook specifier that lets you load the replacement text from a file. -So, you might start **mitmdump** as follows: - ->>> mitmdump --replace-from-file :~q:foo:~/xss-exploit - -This will load the replacement text from the file ``~/xss-exploit``. - -Both the :option:`--replace` and :option:`--replace-from-file` flags can be passed multiple -times. - - -Interactively -------------- - -The :kbd:`R` shortcut key in the mitmproxy options menu (:kbd:`o`) lets you add and edit -replacement hooks using a built-in editor. The context-sensitive help (:kbd:`?`) has -complete usage information. - -================== ============================= -command-line :option:`--replace`, - :option:`--replace-from-file` -mitmproxy shortcut :kbd:`o` then :kbd:`R` -================== ============================= diff --git a/docs/features/responsestreaming.rst b/docs/features/responsestreaming.rst deleted file mode 100644 index 8975c1f8..00000000 --- a/docs/features/responsestreaming.rst +++ /dev/null @@ -1,69 +0,0 @@ -.. _responsestreaming: - -Response Streaming -================== - -By using mitmproxy's streaming feature, response contents can be passed to the client incrementally -before they have been fully received by the proxy. This is especially useful for large binary files -such as videos, where buffering the whole file slows down the client's browser. - -By default, mitmproxy will read the entire response, perform any indicated -manipulations on it and then send the (possibly modified) response to -the client. In some cases this is undesirable and you may wish to "stream" -the response back to the client. When streaming is enabled, the response is -not buffered on the proxy but directly sent back to the client instead. - -On the command-line -------------------- - -Streaming can be enabled on the command line for all response bodies exceeding a certain size. -The SIZE argument understands k/m/g suffixes, e.g. 3m for 3 megabytes. - -================== ============================= -command-line :option:`--stream SIZE` -================== ============================= - -.. warning:: - - When response streaming is enabled, **streamed response contents will not be - recorded or preserved in any way.** - -.. note:: - - When response streaming is enabled, the response body cannot be modified by the usual means. - -Customizing Response Streaming ------------------------------- - -You can also use an :ref:`inlinescripts` to customize exactly -which responses are streamed. - -Responses that should be tagged for streaming by setting their ``.stream`` attribute to ``True``: - -.. literalinclude:: ../../examples/stream.py - :caption: examples/stream.py - :language: python - -Implementation Details ----------------------- - -When response streaming is enabled, portions of the code which would have otherwise performed -changes on the response body will see an empty response body instead -(:py:data:`netlib.http.CONTENT_MISSING`). Any modifications will be ignored. - -Streamed responses are usually sent in chunks of 4096 bytes. If the response is sent with a -``Transfer-Encoding: chunked`` header, the response will be streamed one chunk at a time. - -Modifying streamed data ------------------------ - -If the ``.stream`` attribute is callable, ``.stream`` will wrap the generator that yields all -chunks. - -.. literalinclude:: ../../examples/stream_modify.py - :caption: examples/stream_modify.py - :language: python - -.. seealso:: - - - :ref:`passthrough` diff --git a/docs/features/reverseproxy.rst b/docs/features/reverseproxy.rst deleted file mode 100644 index 87065e73..00000000 --- a/docs/features/reverseproxy.rst +++ /dev/null @@ -1,57 +0,0 @@ -.. _reverseproxy: - -Reverse Proxy -============= - -In reverse proxy mode, mitmproxy accepts standard HTTP(S) requests and forwards -them to the specified upstream server. This is in contrast to :ref:`upstreamproxy`, in which -mitmproxy forwards HTTP(S) proxy requests to an upstream proxy server. - -================== ===================================== -command-line :option:`-R http[s]://hostname[:port]` -================== ===================================== - -Here, **http[s]** signifies if the proxy should use TLS to connect to the server. -mitmproxy always accepts both encrypted and unencrypted requests and transforms -them to what the server expects. - -.. code-block:: none - - >>> mitmdump -R https://httpbin.org -p 80 - >>> curl http://localhost/ - # requests will be transparently upgraded to TLS by mitmproxy - - >>> mitmdump -R https://httpbin.org -p 443 - >>> curl https://localhost/ - # mitmproxy will use TLS on both ends. - - -Host Header ------------ - -In reverse proxy mode, mitmproxy does not rewrite the host header. While often useful, this -may lead to issues with public web servers. For example, consider the following scenario: - -.. code-block:: none - :emphasize-lines: 5 - - >>> mitmdump -d -R http://example.com/ - >>> curl http://localhost:8080/ - - >> GET https://example.com/ - Host: localhost:8080 - User-Agent: curl/7.35.0 - [...] - - << 404 Not Found 345B - -Since the Host header doesn't match "example.com", an error is returned. -There are two ways to solve this: - -1. Modify the hosts file of your OS so that "example.com" resolves to your proxy's IP. - Then, access example.com directly. Make sure that your proxy can still resolve the original IP - or specify an IP in mitmproxy. -2. Use mitmproxy's :ref:`setheaders` feature to rewrite the host header: - ``--setheader :~q:Host:example.com``. - However, keep in mind that absolute URLs within the returned document or HTTP redirects will - cause the client application to bypass the proxy. diff --git a/docs/features/serverreplay.rst b/docs/features/serverreplay.rst deleted file mode 100644 index 261a1bd6..00000000 --- a/docs/features/serverreplay.rst +++ /dev/null @@ -1,39 +0,0 @@ -.. _serverreplay: - -Server-side replay -================== - -Server-side replay lets us replay server responses from a saved HTTP -conversation. - -Matching requests with responses --------------------------------- - -By default, :program:`mitmproxy` excludes request headers when matching incoming -requests with responses from the replay file. This works in most circumstances, -and makes it possible to replay server responses in situations where request -headers would naturally vary, e.g. using a different user agent. -The :option:`--rheader headername` command-line option allows you to override -this behaviour by specifying individual headers that should be included in matching. - - -Response refreshing -------------------- - -Simply replaying server responses without modification will often result in -unexpected behaviour. For example cookie timeouts that were in the future at -the time a conversation was recorded might be in the past at the time it is -replayed. By default, :program:`mitmproxy` refreshes server responses before sending -them to the client. The **date**, **expires** and **last-modified** headers are -all updated to have the same relative time offset as they had at the time of -recording. So, if they were in the past at the time of recording, they will be -in the past at the time of replay, and vice versa. Cookie expiry times are -updated in a similar way. - -You can turn off response refreshing using the :option:`--norefresh` argument, or using -the :kbd:`o` options shortcut within :program:`mitmproxy`. - -================== ================= -command-line :option:`-S path` -mitmproxy shortcut :kbd:`S` -================== ================= diff --git a/docs/features/setheaders.rst b/docs/features/setheaders.rst deleted file mode 100644 index cbc8b6a5..00000000 --- a/docs/features/setheaders.rst +++ /dev/null @@ -1,19 +0,0 @@ -.. _setheaders: - -Set Headers -=========== - -This feature lets you specify a set of headers to be added to requests or -responses, based on a filter pattern. You can specify these either on the -command-line, or through an interactive editor in mitmproxy. - -Example: Set the **Host** header to "example.com" for all requests. - -.. code-block:: none - - mitmdump -R http://example.com --setheader :~q:Host:example.com - -================== ============================= -command-line :option:`--setheader PATTERN` -mitmproxy shortcut :kbd:`o` then :kbd:`H` -================== ============================= diff --git a/docs/features/socksproxy.rst b/docs/features/socksproxy.rst deleted file mode 100644 index 76d4cda9..00000000 --- a/docs/features/socksproxy.rst +++ /dev/null @@ -1,10 +0,0 @@ -.. _socksproxy: - -SOCKS Mode -========== - -In this mode, mitmproxy acts as a SOCKS5 proxy server. - -================== ================= -command-line :option:`--socks` -================== ================= diff --git a/docs/features/sticky.rst b/docs/features/sticky.rst deleted file mode 100644 index a79cbe8d..00000000 --- a/docs/features/sticky.rst +++ /dev/null @@ -1,41 +0,0 @@ -.. _sticky: - -Sticky cookies and auth -======================= - -Sticky cookies --------------- - -When the sticky cookie option is set, __mitmproxy__ will add the cookie most -recently set by the server to any cookie-less request. Consider a service that -sets a cookie to track the session after authentication. Using sticky cookies, -you can fire up mitmproxy, and authenticate to a service as you usually would -using a browser. After authentication, you can request authenticated resources -through mitmproxy as if they were unauthenticated, because mitmproxy will -automatically add the session tracking cookie to requests. Among other things, -this lets you script interactions with authenticated resources (using tools -like wget or curl) without having to worry about authentication. - -Sticky cookies are especially powerful when used in conjunction with :ref:`clientreplay` - you can -record the authentication process once, and simply replay it on startup every time you need -to interact with the secured resources. - -================== ====================== -command-line :option:`-t FILTER` -mitmproxy shortcut :kbd:`o` then :kbd:`t` -================== ====================== - - -Sticky auth ------------ - -The sticky auth option is analogous to the sticky cookie option, in that HTTP -**Authorization** headers are simply replayed to the server once they have been -seen. This is enough to allow you to access a server resource using HTTP Basic -authentication through the proxy. Note that :program:`mitmproxy` doesn't (yet) support -replay of HTTP Digest authentication. - -================== ====================== -command-line :option:`-u FILTER` -mitmproxy shortcut :kbd:`o` then :kbd:`A` -================== ====================== diff --git a/docs/features/tcpproxy.rst b/docs/features/tcpproxy.rst deleted file mode 100644 index fd0746a2..00000000 --- a/docs/features/tcpproxy.rst +++ /dev/null @@ -1,31 +0,0 @@ -.. _tcpproxy: - -TCP Proxy -========= - -WebSockets or other non-HTTP protocols are not supported by mitmproxy yet. However, you can exempt -hostnames from processing, so that mitmproxy acts as a generic TCP forwarder. -This feature is closely related to the :ref:`passthrough` functionality, -but differs in two important aspects: - -- The raw TCP messages are printed to the event log. -- SSL connections will be intercepted. - -Please note that message interception or modification are not possible yet. -If you are not interested in the raw TCP messages, you should use the ignore domains feature. - -How it works ------------- - -================== ====================== -command-line :option:`--tcp HOST` -mitmproxy shortcut :kbd:`o` then :kbd:`T` -================== ====================== - -For a detailed description how the hostname pattern works, please look at the :ref:`passthrough` -feature. - -.. seealso:: - - - :ref:`passthrough` - - :ref:`responsestreaming` diff --git a/docs/features/upstreamcerts.rst b/docs/features/upstreamcerts.rst deleted file mode 100644 index af2e2226..00000000 --- a/docs/features/upstreamcerts.rst +++ /dev/null @@ -1,23 +0,0 @@ -.. _upstreamcerts: - -Upstream Certificates -===================== - -When mitmproxy receives a connection destined for an SSL-protected service, it -freezes the connection before reading its request data, and makes a connection -to the upstream server to "sniff" the contents of its SSL certificate. The -information gained - the **Common Name** and **Subject Alternative Names** - is -then used to generate the interception certificate, which is sent to the client -so the connection can continue. - -This rather intricate little dance lets us seamlessly generate correct -certificates even if the client has specified only an IP address rather than the -hostname. It also means that we don't need to sniff additional data to generate -certs in transparent mode. - -Upstream cert sniffing is on by default, and can optionally be turned off. - -================== ============================= -command-line :option:`--no-upstream-cert` -mitmproxy shortcut :kbd:`o` then :kbd:`U` -================== ============================= diff --git a/docs/features/upstreamproxy.rst b/docs/features/upstreamproxy.rst deleted file mode 100644 index e06833c2..00000000 --- a/docs/features/upstreamproxy.rst +++ /dev/null @@ -1,12 +0,0 @@ -.. _upstreamproxy: - -Upstream proxy mode -=================== - -In this mode, mitmproxy accepts proxy requests and unconditionally forwards all -requests to a specified upstream proxy server. This is in contrast to :ref:`reverseproxy`, -in which mitmproxy forwards ordinary HTTP requests to an upstream server. - -================== =================================== -command-line :option:`-U http://hostname[:port]` -================== =================================== |