Friday, June 10, 2016

Telling Python to prefer your user's private directory over the global library directory

Just a quick note since this caused quite a bit of frustration for me over the last couple of days:

I usually install Python modules using 'pip install --user <module>' since this is nice and safe, and I don't need to sudo to do it. These modules are installed, by default, to ${HOME}/.local/lib/python<version>/site-packages. This has served me well over the years, and it's something that worked before pip going back to easy_install and even before when running 'python setup.py install --user' manually.

Recently, trying to debug problems I am having installing TensorFlow, I noticed that the global packages were taking priority over my user-local modules in the cases where the module was installed in both. eg if I
    >>> import six
    >>> six.__file__
    /usr/lib/python2.7/dist-packages/six.pyc
even though I have six installed locally. Note, this is the debian-custom global location for Python modules. I don't think the fact that I am using a Debian-derivative modifies this story, but I am not completely sure.

After a lot of Googling and looking through site.py, this ended up being due to:

  1. Having ENABLE_USER_SITE = None in /usr/lib/python2.7/site.py although I don't remember ever having changed this before and this used to work ok. Anyway, set this to True
  2. More importantly, there was a row in ${HOME}/.local/lib/python2.7/easy-install.pth saying '/usr/lib/python2.7/dist-packages'. All these directories get pre-pended to sys.path, so this sneaks the global path above my user-local. Removing this line fixes the problem.
/usr/lib/python2.7/dist-packages gets added to sys.path anyway later in site.py, but it gets appended instead of pre-prended, so all works out well.

Friday, March 4, 2016

Deep Q network trained to play Galaxian

I think DeepMind was a bit unlucky in not choosing Galaxian as one of their original set of games to train on. It's a very popular game, much more interesting than its Space Invaders ancestor, and, more importantly, their algorithm learns great on it.

Here are the usual graphs for it:


Note that, unlike in the Space Invaders case, these scores are very much human-level. Those scores in the high 20,000 range are really hard to get unless you play the game a lot. Even the 10,000 average is very respectable. It also very quickly reaches perfectly-reasonable and quite human-styled levels of play (less than 10 epochs).

I made a learning-progression video for it:




I used run_nature.py from Nathan Sprague's code (https://github.com/spragunr/deep_q_rl). You'll need a newish version of ALE (https://github.com/mgbellemare/Arcade-Learning-Environment) to train this since code to support Galaxian was only incorporated recently (commit c62378aece70cfb119d0d4111d15b4b830a23d6c or later).