1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
|
Mitmproxy has four modes of operation that allow you to use mitmproxy in a
variety of scenarios:
- **Regular** (the default)
- **Transparent**
- **Reverse Proxy**
- **Upstream Proxy**
Now, which one should you pick? Use this flow chart:
<img class="img-responsive" src="@!urlTo('schematics/proxy-modes-flowchart.png')!@"/>
<div class="page-header">
<h1>Regular Proxy</h1>
</div>
Mitmproxy's regular mode is the simplest and the easiest to set up.
1. Start mitmproxy.
2. Configure your client to use mitmproxy. For instance on IOS, the settings might look like <a href="@!urlTo('screenshots/ios-manual.png')!@">this</a>.
3. Quick Check: You should already be able to visit an unencrypted HTTP site
through the proxy.
4. Open the magic domain <strong>mitm.it</strong> and install the certificate for your device.
<div class="well">
<strong>Heads Up:</strong> Unfortunately, some applications bypass the
system HTTP proxy settings - Android applications are a common example. In
these cases, you need to use mitmproxy's transparent mode.
</div>
If you are proxying an external device, your network will probably look like this:
<img class="img-responsive" src="@!urlTo('schematics/proxy-modes-regular.png')!@">
The square brackets signify the source and destination IP addresses. Your
client explicitly connects to mitmproxy and mitmproxy explicitly connects
to the target server.
<div class="page-header">
<h1>Transparent Proxy</h1>
</div>
In transparent mode, traffic is directed into a proxy at the network layer,
without any client configuration required. This makes transparent proxying
ideal for situations where you can't change client behaviour. In the graphic
below, a machine running mitmproxy has been inserted between the router and
the internet:
<a href="@!urlTo('schematics/proxy-modes-transparent-1.png')!@">
<img class="img-responsive" src="@!urlTo('schematics/proxy-modes-transparent-1.png')!@">
</a>
The square brackets signify the source and destination IP addresses. Round
brackets mark the next hop on the *Ethernet/data link* layer. This distinction
is important: when the packet arrives at the mitmproxy machine, it must still
be addressed to the target server. This means that Network Address Translation
should not be applied before the traffic reaches mitmproxy, since this would
remove the target information, leaving mitmproxy unable to determine the real
destination.
<a href="@!urlTo('schematics/proxy-modes-transparent-wrong.png')!@">
<img class="img-responsive" src="@!urlTo('schematics/proxy-modes-transparent-wrong.png')!@"></a>
<h2>Common Configurations</h2>
There are many ways to configure your network for transparent proxying. We'll
look at three common scenarios:
1. Configuring the client to use a custom gateway/router/"next hop"
2. Implementing custom routing on the router
In most cases, the first option is recommended due to its ease of use.
<h3>(a) Custom Gateway</h3>
One simple way to get traffic to the mitmproxy machine with the destination IP
intact, is to simply configure the client with the mitmproxy box as the
default gateway.
<a href="@!urlTo('schematics/proxy-modes-transparent-2.png')!@">
<img class="img-responsive" src="@!urlTo('schematics/proxy-modes-transparent-2.png')!@"></a>
In this scenario, we would:
- Configure the proxy machine for transparent mode. You can find instructions
in the <em>Transparent Proxying</em> section of the mitmproxy docs.
- Configure the client to use the proxy machine's IP as the default gateway.
<a href="@!urlTo('screenshots/ios-gateway.png')!@">Here</a> is what this would
look like on IOS.
- Quick Check: At this point, you should already be able to visit an
unencrypted HTTP site over the proxy.
- Open the magic domain <strong>mitm.it</strong> and install the certificate
for your device.
Setting the custom gateway on clients can be automated by serving the settings
out to clients over DHCP. This lets set up an interception network where all
clients are proxied automatically, which can save time and effort.
<div class="well">
<strong style="text-align: center; display: block">Troubleshooting Transparent Mode</strong>
<p>Incorrect transparent mode configurations are a frequent source of
error. If it doesn't work for you, try the following things:</p>
<ul>
<li>
Open mitmproxy's event log (press `e`) - do you see clientconnect
messages? If not, the packets are not arriving at the proxy. One common
cause is the occurrence of ICMP redirects, which means that your
machine is telling the client that there's a faster way to the
internet by contacting your router directly (see the
<em>Transparent Proxying</em> section on how to disable them). If in
doubt, <a href="https://wireshark.org/">Wireshark</a> may help you
to see whether something arrives at your machine or not.
</li>
<li>
Make sure you have not explicitly configured an HTTP proxy on the
client. This is not needed in transparent mode.
</li>
<li>
Re-check the instructions in the <em>Transparent Proxying</em> section. Anything you missed?
</li>
</ul>
If you encounter any other pitfalls that should be listed here, please let us know!
</div>
<h3>(b) Custom Routing</h3>
In some cases, you may need more fine-grained control of which traffic reaches
the mitmproxy instance, and which doesn't. You may, for instance, choose only
to divert traffic to some hosts into the transparent proxy. There are a huge
number of ways to accomplish this, and much will depend on the router or
packet filter you're using. In most cases, the configuration will look like
this:
<a href="@!urlTo('schematics/proxy-modes-transparent-3.png')!@">
<img class="img-responsive" src="@!urlTo('schematics/proxy-modes-transparent-3.png')!@">
</a>
<div class="page-header">
<h1>Reverse Proxy</h1>
</div>
Mitmproxy is usually used with a client that uses the proxy to access the
Internet. Using reverse proxy mode, you can use mitmproxy to act like a normal
HTTP server:
<a href="@!urlTo('schematics/proxy-modes-reverse.png')!@">
<img class="img-responsive" src="@!urlTo('schematics/proxy-modes-reverse.png')!@">
</a>
There are various use-cases:
- Say you have an internal API running at http://example.local/. You could now
set up mitmproxy in reverse proxy mode at http://debug.example.local/ and
dynamically point clients to this new API endpoint, which provides clients
with the same data and you with debug information. Similarly, you could move
your real server to a different IP/port and set up mitmproxy at the original
place to debug all sessions.
- Say you're a web developer working on example.com (with a development
version running on localhost:8000). You can modify your hosts file so that
example.com points to 127.0.0.1 and then run mitmproxy in reverse proxy mode
on port 80. You can test your app on the example.com domain and get all
requests recorded in mitmproxy.
- Say you have some toy project that should get SSL support. Simply set up
mitmproxy with SSL termination and you're done (<code>mitmdump -p 443 -R
https2http://localhost:80/</code>). There are better tools for this specific
task, but mitmproxy is very quick and simple way to set up an SSL-speaking
server.
- Want to add a non-SSL-capable compression proxy in front of your server? You
could even spawn a mitmproxy instance that terminates SSL (https2http://...),
point it to the compression proxy and let the compression proxy point to a
SSL-initiating mitmproxy (http2https://...), which then points to the real
server. As you see, it's a fairly flexible thing.
Note that mitmproxy supports either an HTTP or an HTTPS upstream server, not
both at the same time. You can work around this by spawning a second mitmproxy
instance.
<div class="well">
<strong style="text-align: center; display: block">Caveat: Interactive Use</strong>
One caveat is that reverse proxy mode is often not sufficient for interactive
browsing. Consider trying to clone Google by using:
<code>mitmproxy -R http://google.com/</code>
This works for the initial request, but the HTML served to the client remains
unchanged. As soon as the user clicks on an non-relative URL (or downloads a
non-relative image resource), traffic no longer passes through mitmproxy, and
the client connects to Google directly again.
</div>
<div class="page-header">
<h1>Upstream Proxy</h1>
</div>
If you want to chain proxies by adding mitmproxy in front of a different proxy
appliance, you can use mitmproxy's upstream mode. In upstream mode, all
requests are unconditionally transferred to an upstream proxy of your choice.
<a href="@!urlTo('schematics/proxy-modes-upstream.png')!@">
<img class="img-responsive" src="@!urlTo('schematics/proxy-modes-upstream.png')!@"></a>
mitmproxy supports both explicit HTTP and explicit HTTPS in upstream proxy
mode. You could in theory chain multiple mitmproxy instances in a row, but
that doesn't make any sense in practice (i.e. outside of our tests).
|
