diff options
| -rw-r--r-- | doc-src/dev-docs/Makefile | 2 | ||||
| -rw-r--r-- | doc-src/dev-docs/_static/favicon.ico | bin | 0 -> 5430 bytes | |||
| -rw-r--r-- | doc-src/dev-docs/_static/mitmproxy-long.png | bin | 0 -> 123829 bytes | |||
| -rw-r--r-- | doc-src/dev-docs/conf.py | 14 | ||||
| -rw-r--r-- | doc-src/dev-docs/index.rst | 21 | ||||
| -rw-r--r-- | doc-src/dev-docs/inlinescripts.rst | 211 | ||||
| -rw-r--r-- | doc-src/dev-docs/models.rst | 19 | ||||
| -rw-r--r-- | doc-src/dev-docs/proxy.rst | 2 | ||||
| -rw-r--r-- | libmproxy/script.py | 4 |
9 files changed, 254 insertions, 19 deletions
diff --git a/doc-src/dev-docs/Makefile b/doc-src/dev-docs/Makefile index 4d3b2046..666a3a86 100644 --- a/doc-src/dev-docs/Makefile +++ b/doc-src/dev-docs/Makefile @@ -192,4 +192,4 @@ pseudoxml: @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." livehtml: - sphinx-autobuild -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html + sphinx-autobuild -b html -z '../../libmproxy' -r '___jb_(old|bak)___$$' $(ALLSPHINXOPTS) $(BUILDDIR)/html
\ No newline at end of file diff --git a/doc-src/dev-docs/_static/favicon.ico b/doc-src/dev-docs/_static/favicon.ico Binary files differnew file mode 100644 index 00000000..3c3b891c --- /dev/null +++ b/doc-src/dev-docs/_static/favicon.ico diff --git a/doc-src/dev-docs/_static/mitmproxy-long.png b/doc-src/dev-docs/_static/mitmproxy-long.png Binary files differnew file mode 100644 index 00000000..f9397d1e --- /dev/null +++ b/doc-src/dev-docs/_static/mitmproxy-long.png diff --git a/doc-src/dev-docs/conf.py b/doc-src/dev-docs/conf.py index af7f7b9d..23db112c 100644 --- a/doc-src/dev-docs/conf.py +++ b/doc-src/dev-docs/conf.py @@ -105,7 +105,7 @@ exclude_patterns = ['_build'] pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. -#modindex_common_prefix = [] +modindex_common_prefix = ['libmproxy.'] # If true, keep warnings as "system message" paragraphs in the built documents. #keep_warnings = False @@ -123,31 +123,33 @@ html_theme = 'sphinx_rtd_theme' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. -#html_theme_options = {} +html_theme_options = { + # 'logo_only': True, +} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # "<project> v<release> documentation". -#html_title = None +html_title = "mitmproxy %s documentation" % version # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. -#html_logo = None +html_logo = "mitmproxy-long.png" # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. -#html_favicon = None +html_favicon = "favicon.ico" # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -#html_static_path = ['_static'] +html_static_path = ['_static'] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied diff --git a/doc-src/dev-docs/index.rst b/doc-src/dev-docs/index.rst index 6d335c17..15ec3617 100644 --- a/doc-src/dev-docs/index.rst +++ b/doc-src/dev-docs/index.rst @@ -1,19 +1,19 @@ -.. mitmproxy documentation master file, created by - sphinx-quickstart on Thu Sep 03 14:04:13 2015. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. +Welcome to mitmproxy's dev documentation! +========================================= -Welcome to mitmproxy's documentation! -===================================== Contents: .. toctree:: - :maxdepth: 2 + :maxdepth: 2 - protocols - proxy - exceptions + End User Documentation <https://mitmproxy.org/doc/> + + inlinescripts + protocols + proxy + exceptions + models Indices and tables @@ -21,5 +21,4 @@ Indices and tables * :ref:`genindex` * :ref:`modindex` -* :ref:`search` diff --git a/doc-src/dev-docs/inlinescripts.rst b/doc-src/dev-docs/inlinescripts.rst new file mode 100644 index 00000000..0c53b1a4 --- /dev/null +++ b/doc-src/dev-docs/inlinescripts.rst @@ -0,0 +1,211 @@ +Inline Scripts +============== + +**mitmproxy** has a powerful scripting API that allows you to modify flows +on-the-fly or rewrite previously saved flows locally. + +The mitmproxy scripting API is event driven - a script is simply a Python +module that exposes a set of event methods. Here's a complete mitmproxy script +that adds a new header to every HTTP response before it is returned to the +client: + +.. literalinclude:: ../../examples/add_header.py + :caption: examples/add_header.py + :language: python + +The first argument to each event method is an instance of +:py:class:`~libmproxy.script.ScriptContext` that lets the script interact with the global mitmproxy +state. The **response** event also gets an instance of :py:class:`~libmproxy.script.ScriptContext`, +which we can use to manipulate the response itself. + +We can now run this script using mitmdump or mitmproxy as follows: + +>>> mitmdump -s add_header.py + +The new header will be added to all responses passing through the proxy. + +Examples +-------- + +mitmproxy comes with a variety of example inline scripts, which demonstrate many basic tasks. +We encourage you to either browse them locally or on `GitHub`_. + + +Events +------ + +.. TODO: Split this into Connection, HTTP and TCP events once we have TCP events. + +The ``context`` argument passed to each event method is always a +:py:class:`~libmproxy.script.ScriptContext` instance. It is guaranteed to be the same object +for the scripts lifetime and is not shared between multiple inline scripts. You can safely use it +to store any form of state you require. + +Events are listed in the order they usually occur. + +.. py:function:: start(context, argv) + + Called once on startup, before any other events. + + :param List[str] argv: The inline scripts' arguments. + For example, ``mitmproxy -s 'example.py --foo 42'`` sets argv to ``["--foo", "42"]``. + +.. py:function:: clientconnect(context, root_layer) + + Called when a client initiates a connection to the proxy. Note that + a connection can correspond to multiple HTTP requests. + + .. versionchanged:: 0.14 + :param Layer root_layer: The root layer (see :ref:`protocols` for an explanation what the root + layer is), which provides transparent access to all attributes of the + :py:class:`~libmproxy.proxy.RootContext`. For example, ``root_layer.client_conn.address`` + gives the remote address of the connecting client. + + +.. py:function:: request(context, flow) + + Called when a client request has been received. The ``flow`` object is + guaranteed to have a non-None ``request`` attribute. + + :param HTTPFlow flow: The flow containing the request which has been received. + The object is guaranteed to have a non-None ``request`` attribute. + +.. py:function:: serverconnect(context, server_conn) + + Called before the proxy initiates a connection to the target server. Note that + a connection can correspond to multiple HTTP requests. + + :param ServerConnection server_conn: The server connection object. It is guaranteed to have a + non-None ``address`` attribute. + +.. py:function:: responseheaders(context, flow) + + Called when the headers of a server response have been received. + This will always be called before the response hook. + + :param HTTPFlow flow: The flow containing the request and response. + The object is guaranteed to have non-None ``request`` and + ``response`` attributes. ``response.content`` will be ``None``, + as the response body has not been read yet. + +.. py:function:: response(context, flow) + + Called when a server response has been received. + + :param HTTPFlow flow: The flow containing the request and response. + The object is guaranteed to have non-None ``request`` and + ``response`` attributes. ``response.body`` will contain the raw response body, + unless response streaming has been enabled. + +.. py:function:: error(context, flow) + + Called when a flow error has occurred, e.g. invalid server responses, or + interrupted connections. This is distinct from a valid server HTTP error + response, which is simply a response with an HTTP error code. + + :param HTTPFlow flow: The flow containing the error. + It is guaranteed to have non-None ``error`` attribute. + +.. py:function:: serverdisconnect(context, server_conn) + + Called when the proxy has closed the server connection. + + .. versionadded:: 0.14 + :param ServerConnection server_conn: see :py:func:`serverconnect` + +.. py:function:: clientdisconnect(context, root_layer) + + Called when a client disconnects from the proxy. + + .. versionchanged:: 0.14 + :param Layer root_layer: see :py:func:`clientconnect` + +.. py:function:: done(context) + + Called once on script shutdown, after any other events. + + +API +--- + +The canonical API documentation is the code, which you can browse here, locally or on `GitHub`_. +*Use the Source, Luke!* + +The main classes you will deal with in writing mitmproxy scripts are: + +:py:class:`~libmproxy.script.ScriptContext` + - A handle for interacting with mitmproxy's Flow Master from within scripts. +:py:class:`~libmproxy.models.ClientConnection` + - Describes a client connection. +:py:class:`~libmproxy.models.ServerConnection` + - Describes a server connection. +:py:class:`~libmproxy.models.HTTPFlow` + - A collection of objects representing a single HTTP transaction. +:py:class:`~libmproxy.models.HTTPRequest` + - An HTTP request. +:py:class:`~libmproxy.models.HTTPResponse` + - An HTTP response. +:py:class:`~libmproxy.models.Error` + - A communications error. +:py:class:`netlib.odict.ODDict` + - A dictionary-like object for managing sets of key/value data. There + is also a variant called ODictCaseless that ignores key case for some + calls (used mainly for headers). +:py:class:`netlib.certutils.SSLCert` + - Exposes information SSL certificates. +:py:class:`libmproxy.flow.FlowMaster` + - The "heart" of libmproxy, usually subclassed as :py:class:`libmproxy.dump.DumpMaster` or + :py:class:`libmproxy.console.ConsoleMaster`. + +Script Context +-------------- + +.. autoclass:: libmproxy.script.ScriptContext + :members: + :undoc-members: + +Running scripts in parallel +--------------------------- + +We have a single flow primitive, so when a script is blocking, other requests are not processed. +While that's usually a very desirable behaviour, blocking scripts can be run threaded by using the +:py:obj:`libmproxy.script.concurrent` decorator. +**If your script does not block, you should avoid the overhead of the decorator.** + +.. literalinclude:: ../../examples/nonblocking.py + :caption: examples/nonblocking.py + :language: python + +Make scripts configurable with arguments +---------------------------------------- + +Sometimes, you want to pass runtime arguments to the inline script. This can be simply done by +surrounding the script call with quotes, e.g. ```mitmdump -s 'script.py --foo 42'``. +The arguments are then exposed in the start event: + +.. literalinclude:: ../../examples/modify_response_body.py + :caption: examples/modify_response_body.py + :language: python + +Running scripts on saved flows +------------------------------ + +Sometimes, we want to run a script on :py:class:`~libmproxy.models.Flow` objects that are already +complete. This happens when you start a script, and then load a saved set of flows from a file +(see the "scripted data transformation" example `here <https://mitmproxy.org/doc/mitmdump.html>`_). +It also happens when you run a one-shot script on a single flow through the ``|`` (pipe) shortcut +in mitmproxy. + +In this case, there are no client connections, and the events are run in the following order: +**start**, **request**, **responseheaders**, **response**, **error**, **done**. +If the flow doesn't have a **response** or **error** associated with it, the matching events will +be skipped. + +Spaces in the script path +------------------------- + +By default, spaces are interpreted as a separator between the inline script and its arguments +(e.g. ``-s 'foo.py 42'``). Consequently, the script path needs to be wrapped in a separate pair of +quotes if it contains spaces: ``-s '\'./foo bar/baz.py\' 42'``. + +.. _GitHub: https://github.com/mitmproxy/mitmproxy diff --git a/doc-src/dev-docs/models.rst b/doc-src/dev-docs/models.rst new file mode 100644 index 00000000..1ac7d042 --- /dev/null +++ b/doc-src/dev-docs/models.rst @@ -0,0 +1,19 @@ +.. _models: + +Models +====== + +.. warning:: + The documentation for models has not been converted to rst yet and **many attributes/features + are missing**. + Please read the source code instead. + +.. automodule:: libmproxy.models + :show-inheritance: + :members: + :undoc-members: + + +.. automodule:: netlib.http.semantics + :members: Request, Response + :undoc-members:
\ No newline at end of file diff --git a/doc-src/dev-docs/proxy.rst b/doc-src/dev-docs/proxy.rst index 258b904c..e772e5b9 100644 --- a/doc-src/dev-docs/proxy.rst +++ b/doc-src/dev-docs/proxy.rst @@ -9,4 +9,4 @@ Proxy .. autoclass:: DummyServer .. autoclass:: ProxyConfig .. autoclass:: RootContext - :members: + :members:
\ No newline at end of file diff --git a/libmproxy/script.py b/libmproxy/script.py index e13f0e2b..fb04f8c3 100644 --- a/libmproxy/script.py +++ b/libmproxy/script.py @@ -11,6 +11,10 @@ class ScriptError(Exception): class ScriptContext: + """ + The script context should be used to interact with the global mitmproxy state from within a + script. + """ def __init__(self, master): self._master = master |
