Virtualenv is the one and only recommended dev environment

- Create a "dev" script to create a virtualenv-based dev environment
- Update the docs to remove non-virtualenv install recommendations
- Update osx-binaries generation to use virtualenv

TODO:
- The dev script or an equivalent should be made to work on Windows
- We still can't remove the annoying top-level command scripts, because
pyinstaller doesn't support entry points. Once it does, they can go.

3 files changed, 88 insertions, 55 deletions

diff --git a/README.mkd b/README.mkd
index 0fdf3715..ce276764 100644
--- a/README.mkd
+++ b/README.mkd
@@ -16,8 +16,7 @@ mitmproxy.org website:
 
 [mitmproxy.org](http://mitmproxy.org).
 
-
-You can find complete directions for installing mitmproxy [here](http://mitmproxy.org/doc/install.html). 
+You can find complete directions for installing mitmproxy [here](http://mitmproxy.org/doc/install.html).
 
 
 Features
@@ -41,43 +40,49 @@ only mitmdump is supported, which does not have a graphical user interface.
 Hacking
 -------
 
+To get started hacking on mitmproxy, make sure you have
+[Python](http://www.python.org) 2.7.x. with
+[virtualenv](https://virtualenv.pypa.io/en/latest/) installed (you can find
+installation instructions for virtualenv
+[here](https://virtualenv.pypa.io/en/latest/installation.html)). Then do the
+following:
 
-### Requirements
-
-
-* [Python](http://www.python.org) 2.7.x.
-* [netlib](http://pypi.python.org/pypi/netlib), version matching mitmproxy.
-* Third-party packages listed in [setup.py](https://github.com/mitmproxy/mitmproxy/blob/master/setup.py)
-
-Optional packages for extended content decoding:
-
-* [PyAMF](http://www.pyamf.org/) version 0.6.1 or newer.
-* [protobuf](https://code.google.com/p/protobuf/) version 2.5.0 or newer.
-* [cssutils](http://cthedot.de/cssutils/) version 1.0 or newer.
-
-For convenience, all optional dependencies can be installed with 
-
-`pip install "mitmproxy[contentviews]"`
+```
+$ git clone https://github.com/mitmproxy/mitmproxy.git
+$ git clone https://github.com/mitmproxy/netlib.git
+$ git clone https://github.com/mitmproxy/pathod.git
+$ cd mitmproxy
+$ ./dev
+```
 
-### Setting up a dev environment
+The *dev* script will create a virtualenv environment in a directory called
+"venv.mitmproxy", and install all of mitmproxy's development requirements, plus
+all optional modules. The primary mitmproxy components - mitmproxy, netlib and
+pathod - are all installed "editable", so any changes to the source in the git
+checkouts will be reflected live in the virtualenv.
 
-The following procedure is recommended to set up your dev environment:
+To confirm that you're up and running, activate the virtualenv, and run the
+mitmproxy test suite:
 
 ```
-$ git clone https://github.com/mitmproxy/mitmproxy.git
-$ cd mitmproxy
-$ pip install --src . -r requirements.txt
+$ source ../venv.mitmproxy/bin/activate
+$ nosetests ./test
 ```
+Note that the main executables for the project - **mitmdump**, **mitmproxy** and
+**mitmweb** - are all created within the virtualenv. After activating the
+virtualenv, they will be on your $PATH, and you can run them like any other
+command:
 
-This installs the latest GitHub versions of mitmproxy, netlib and pathod into `mitmproxy/`. All other development dependencies save countershape are installed into their usual locations. 
+```$ mitmdump --version```
 
 
-### Testing
 
-The test suite requires the `dev` extra requirements listed in [setup.py](https://github.com/mitmproxy/mitmproxy/blob/master/setup.py). Install these with:
+### Testing
 
-`pip install "mitmproxy[dev]"`
+If you've followed the procedure above, you already have all the development
+requirements installed, and you can simply run the test suite:
 
+```nosetests ./test```
 
 Please ensure that all patches are accompanied by matching changes in the test
 suite. The project maintains 100% test coverage.
diff --git a/dev b/dev
new file mode 100755
index 00000000..c8346af1
--- /dev/null
+++ b/dev
@@ -0,0 +1,16 @@
+#!/bin/sh
+VENV=../venv.mitmproxy
+PIP="$VENV/bin/pip --cache-dir ~/.pipcache"
+
+echo "This script sets up the following:"
+echo "\t~/.pipcache - A pip cache directory"
+echo "\t$VENV - A development virtualenv"
+
+mkdir -p ~/.pipcache
+
+virtualenv $VENV
+source $VENV/bin/activate
+$PIP install -r ./requirements.txt
+# Re-install these to make them editable
+$PIP install --editable ../netlib
+$PIP install --editable ../pathod
diff --git a/release/osx-binaries b/release/osx-binaries
index d2af5e18..990d047d 100755
--- a/release/osx-binaries
+++ b/release/osx-binaries
@@ -2,32 +2,44 @@
 
 # Quick and dangerous script for building OSX binaries.
 
-# A few quirks to note, which should be re-checked every release:
-# - We require the latest development version of PyInstaller.
-
-# To run, first install netlib and mitmproxy, then run
-#
-#	./release/osx-binaries
-#
-# From the top-level mitmproxy directory.
-
-usage ()
-{
-  echo 'Usage : ./release/osx-binaries /path/to/pyinstaller.py'
-  echo 'Run from the top-level mitmproxy directory'
-  exit
-}
-
-if [ "$1" = "" ]
+# At the moment, pyinstaller has no support for entry points, except for this
+# hideous hack on the wiki:
+#   https://github.com/pyinstaller/pyinstaller/wiki/Recipe-Setuptools-Entry-Point
+# Once this is fixed, we can ditch the redundant command scripts.
+
+VENV=../venv.mitmproxy
+PYINST_CMD="$VENV/bin/pyinstaller -F --clean"
+TMPDIR=./tmp
+CACHE="~/Library/Application Support/pyinstaller"
+
+
+if [ ! -d $VENV ]
 then
-    usage
+    echo "Failed: set up a dev environment as described in the README"
+    echo "and run from the top-level mitmproxy directory."
+    exit
 fi
 
+source $VENV/bin/activate
 
-TMPDIR=./tmp
-PYINST_CMD=$1" -F --clean"
-CACHE="~/Library/Application Support/pyinstaller"
+if [ ! -f $VENV/bin/pyinstaller ]
+    then
+    echo "Installing pyinstaller..."
+    $VENV/bin/pip install \
+        --force-reinstall \
+        --upgrade \
+        https://github.com/pyinstaller/pyinstaller/archive/develop.zip
+    $VENV/bin/pip install --upgrade macholib
+fi
+
+# readline.so is actually a symlink to a Python file, which breaks PyInstaller
+# (and readline itself). Why? Who knows. Re-address this when this stupidity
+# ceases to be.
+echo "Removing broken readline..."
+rm -f $VENV/lib/python2.7/readline.so
 
+
+echo "Clearing caches..."
 rm -f dist/*
 rm -rf $TMPDIR
 rm -rf $CACHE
@@ -36,14 +48,14 @@ $PYINST_CMD ./release/mitmdump.spec
 echo "Running mitmdump..."
 ./dist/mitmdump --version || exit 1
 
-$PYINST_CMD ./release/mitmproxy.spec
-echo "Running mitmproxy..."
-./dist/mitmproxy --version || exit 1
-
-$PYINST_CMD ./release/mitmweb.spec
-echo "Running mitmweb..."
-./dist/mitmweb --version || exit 1
+#$PYINST_CMD ./release/mitmproxy.spec
+#echo "Running mitmproxy..."
+#./dist/mitmproxy --version || exit 1
 
+#$PYINST_CMD ./release/mitmweb.spec
+#echo "Running mitmweb..."
+#./dist/mitmweb --version || exit 1
+exit
 DST=osx-mitmproxy-`./dist/mitmdump --shortversion 2>&1`
 mkdir -p $TMPDIR/$DST
 cp ./dist/mitmproxy $TMPDIR/$DST