Database

Create / update database

python manage.py syncdb

Models

Models define how data is stored. Create a model by inheriting from django.db.models.Model

from django.db import models
 
class Choice(models.Model):
    poll = models.ForeignKey(Poll)
    choice = models.CharField(max_length=200)
    votes = models.IntegerField()
    def __unicode__(self):
        return self.choice

App

An “app” is simply a directory with an __init__.py file and other python files. (An “app” is more commonly known in other languages as a “package”.)

The INSTALLED_APPS variable in settings.py tells Django which apps to use.

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'polls'
)

Make sure your app directory is on your Python path.

Re-run python manage.py syncdb after making changes to the INSTALLED_APPS.

URLs

The variable urlpatterns tells Django which Python code to run (which view to use) based on the url path.

from django.conf.urls.defaults import patterns, include, url
from django.contrib import admin
admin.autodiscover()
 
urlpatterns = patterns('',
   (r'^polls/$', 'polls.views.index'),
   (r'^polls/(?P
<poll_id>\d+)/$', 'polls.views.detail'),
   url(r'^admin/', include(admin.site.urls)),
)

The ROOT_URLCONF variable in settings.py tells Django where to find the module-level variable urlpatterns. E.g. if urlpatterns is defined in mysite/urls.py, the settings.py file should have

ROOT_URLCONF = 'mysite.urls'

View

The code used to generate a page is called a view. A view is a Python function which returns a web page. Example index() view in polls/views.py

from django.template import Context, loader
from polls.models import Poll
from django.http import HttpResponse
 
def index(request):
    latest_poll_list = Poll.objects.all().order_by('-pub_date')[:5]
    t = loader.get_template('polls/index.html')
    c = Context({
        'latest_poll_list': latest_poll_list,
    })
    return HttpResponse(t.render(c))

The urlpatterns tells Django which view to use based on the url. For example, the urlpatterns line

   (r'^polls/$', 'polls.views.index'),

tells Django that the url http://site.com/polls is created by the index() function in polls/views.py.

render_to_response is a common shortcut for creating views. Equivalent view using render_to_response

from django.shortcuts import render_to_response
from polls.models import Poll
 
def index(request):
    latest_poll_list = Poll.objects.all().order_by('-pub_date')[:5]
    return render_to_response('polls/index.html', {'latest_poll_list': latest_poll_list})

Often views need to respond to a request. Templates use the {% csrf_token %} to prevent “Cross Site Request Forgeries”, and the view uses a django.template.RequestContext

from django.template import RequestContext
# ...
def detail(request, poll_id):
    p = get_object_or_404(Poll, pk=poll_id)
    return render_to_response('polls/detail.html', {'poll': p},
                               context_instance=RequestContext(request))

Template

A Django template is a HTML skeleton with special Django tags which tell the Django view where to fill in data. Example template polls/index.html

{% if latest_poll_list %}
<ul>
    {% for poll in latest_poll_list %}
<li><a href="/polls/{{ poll.id }}/">{{ poll.question }}</a></li>
 
    {% endfor %}
    </ul>
 
{% else %}
    <p>No polls are available.</p>
{% endif %}

Admin Interface

Register apps with Django’s admin interface

from polls.models import Poll
from django.contrib import admin
 
class PollAdmin(admin.ModelAdmin):
    fields = ['pub_date', 'question']
 
admin.site.register(Poll, PollAdmin)

Generic Views

There’s generic views which means that you don’t have to write a view function; you use django.views.generic to generate a simple view.

from django.conf.urls.defaults import patterns, include, url
from django.views.generic import DetailView, ListView
from polls.models import Poll
 
urlpatterns = patterns('',
    (r'^$',
        ListView.as_view(
            queryset=Poll.objects.order_by('-pub_date')[:5],
            context_object_name='latest_poll_list',
            template_name='polls/index.html')),
    (r'^(?P
<pk>\d+)/$',
        DetailView.as_view(
            model=Poll,
            template_name='polls/detail.html')),
    url(r'^(?P
<pk>\d+)/results/$',
        DetailView.as_view(
            model=Poll,
            template_name='polls/results.html'),
        name='poll_results'),
    (r'^(?P
<poll_id>\d+)/vote/$', 'polls.views.vote'),
)

In Dreamhost panel, setup site for use with Passenger & ssh

Ssh into your Dreamhost server

Setup a Python virtualenv so you can install your own Python packages. Check the virtualenv pypi page for the latest virtualenv version (currently 1.6.3).

wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.6.3.tar.gz
tar xzf virtualenv-1.6.3.tar.gz
python virtualenv-1.6.3/virtualenv.py $HOME/local
rm -rf virtualenv*
export PATH=$HOME/local/bin:$PATH

You probably want to add $HOME/local/bin to your path permanently. Add the export line to your ~/.bashrc file:

export PATH=$HOME/local/bin:$PATH

To make sure that you are using the virtualenv python, check that which python returns /home/youruser/local/bin/python and NOT /usr/bin/python.

Now you can install some Python packages

pip install --upgrade django mezzanine south paste

Create a mezzanine project

cd ~/site.com
# Name this project whatever you like.  I will use "mez"
mezzanine-project mez

Edit your settings in mez/local_settings.py. I will leave the sqlite database for testing. You could should probably create a mysql database for a real site though. Add these lines so Django knows where files are.

TIME_ZONE = 'America/Los_Angeles'
APP_URL = 'http://site.com'
## Note the leading and trailing slash here
#ADMIN_MEDIA_PREFIX = '/admin_media/'
#MEDIA_ROOT= '/home/youruser/site.com/mez/site_media/'
#MEDIA_URL = 'http://site.com/site_media'

Setup your site with Passenger by creating a ~/site.com/passenger_wsgi.py file with the following:

import sys,os
# Force Passenger to run our virtualenv python
INTERP = "/home/youruser/local/bin/python"
if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)
# Setup paths and environment variables
sys.path.append("/home/youruser/site.com")
sys.path.append("/home/youruser/site.com/mez")
os.environ['DJANGO_SETTINGS_MODULE'] = 'mez.settings'
# Set the application
import django.core.handlers.wsgi
from paste.exceptions.errormiddleware import ErrorMiddleware
application = django.core.handlers.wsgi.WSGIHandler()
# Use paste to display errors
application = ErrorMiddleware(application, debug=True)

Now setup the static files

ln -s ~/site.com/mez/site_media ~/site.com/public/site_media

Setup the database

cd ~/site.com/mez
python manage.py syncdb
# I think I initially had some errors running migrate, 
# but it worked the second time running migrate
python manage.py migrate

Restart the web server. Ordinarily you might do sudo /etc/init.d/apache2 restart. On Dreamhost you do this by touching tmp/restart.txt. (If the tmp directory doesn’t exist, mkdir ~/site.com/tmp)

touch ~/site.com/tmp/restart.txt

That should be it. Check to see if your site is up and running.

TODO’s: setup mysql, customize theme, etc

There’s alternative ways to handle the static files. The package django-staticfiles might be better? I haven’t tried it.

Edit Config Files

The sprinkle readme doesn’t mention it, but you need to edit the config files before running. Add your mysql password to config/database.yml and change your hostname and email in config/gitorious.yml.
Beware that if the script already installed the files to /var/www/gitorious/config it won’t overwrite the existing files, so you will need to make your changes to the files at /var/www/gitorious/config (or delete the files and re-run the sprinkle script).

Sprinkle Script Overview

The sprinkle script will

• Install required gems
• Download and configure gitorious
• Install and start startup scripts (stomp, git-poller, git-daemon)
• Setup gitorious mysql database
• Create a user named “git” (whose home dir is /var/www/gitorious)
• Setup apache for use with gitorious

Deciphering Errors

I had a bunch of problems running

bundle exec sprinkle -s gitorious.rb

Ruby/capistrano spits out some long error messages, which you have to decipher to figure out which line in gitorious.rb failed to run.

Bundle couldn’t install the latest version of echoe (0.4.6) because Ubuntu’s rubygems (1.3.7) isn’t recent enough, so I installed an older version of echoe.

sudo gem install echoe -v="0.4.5"

I then removed echoe from the list of gitorious_dependencies in gitorious.rb. In hindsight, I think I could have added an entry to Gemfile specifying the echoe version, but I’m a Ruby noob, and just hacked the script to get it to work.

Then Rake couldn’t setup my database (some error about having activated rake 1.9.1 but the script required rake 1.8.7). Turns out I need to call bundle exec rake instead of /var/lib/gems/1.8/bin/rake. I modified the lines in gitorious.rb:

    #post :install, 'su - git -c "/var/lib/gems/1.8/bin/rake db:setup RAILS_ENV=production"'
    post :install, 'su - git -c "bundle exec rake db:setup RAILS_ENV=production"'

Miscellaneous notes

Make sure your server is getting ports 80 and 443 (for ssl).

I also had to set up my server to properly route email so that Gitorious could send emails (e.g. registration confirmation). I used this guide to setup postfix to route mail through Gmail’s smtp.

Dependencies

• Python (I have the 32-bit Python 2.7.1 installed. I’m not sure if bedtools-python is 3.x compliant) Be sure that you’ve added C:/Python27/bin to your PATH environment variable so you can run python from the command line.
• easy_install (not technically required, but makes installing many Python modules easy)

1. Install MinGW32 for your compiler. This is as close to GCC as you get for Windows and will be much easier to compile than trying to use Visual Studio. Download the suggested file (currently mingw-get-inst-20110211.exe).

2. Install the cython package

easy_install cython

3. Download bedtools-python and extract the folder into C:/MinGW/msys/1.0/home/USERNAME

The hairy part #1: try to compile

4. Open the MinGW shell. You should see the bedtools directory you extracted. Go to the folder and try to build bedtools-python.

ls
cd arq5x-bedtools-python-6671082
python setup.py build --compiler=mingw32

I get an error “src/gzstream.h:35:18: fatal error: zlib.h: No such file or directory”

You need to install zlib. Download the precompiled DLL and unzip into the bedtools-python folder (or install it permanently into the c:/Mingw folder).

Change the name of lib/zdll.lib to lib/libzdll.a

Edit setup.py to tell it about zlib. You will need to modify the libraries and include_dirs line, and add a library_dirs line in the Extension section. It should look like this

    Extension("bedtools.cbedtools",
              sources=["bedtools/cbedtools.pyx"] + glob.glob("src/*.cpp"),
              libraries=["stdc++", "zdll"],
              library_dirs=["lib/"],
              include_dirs=["src/", "include/"],
              depends = glob.glob("src/*.h"),
              language="c++"),

Hopefully you can build bedtools-python now (ignore the ugly stream of compiler commands and warning messages). Now install bedtools-python.

python setup.py install

Hairy part #2:

Now see if you can use bedtools.

python
>>> import bedtools

If you get errors like

from cbedtools import IntervalFile
ImportError: DLL load failed: The specified procedure could not be found.

then bedtools can’t find some dlls that it needs on python’s path. To figure out what dll files it’s looking for, run depends.exe from dependencywalker.com and open up build/lib.win32-2.7/bedtools/cbedtools.pyd. This hopefully gives you a list of DLL files that it’s looking for. You will need to either find and copy these files to C:/Windows/system32. I was missing libgcc_s_dw2-1.dll and libstdc++-6.dll (found in C:/Ming32/bin) and MSVCR90.DLL.

Now use bedtools.

python
>>> import bedtools

name = 'moose'
adjective = 'squishy'
 
print '{name} is {adjective}'.format(**locals())
# moose is squishy
 
# vars() with no args is same as locals()
print '{name} is {adjective}'.format(**vars())
# moose is squishy

Using vars to access object properties

class Cat:
    def __init__(self, name, adj):
        self.name = name
        self.adjective = adj
 
kitty = Cat('Garfield', 'hungry')
 
print '{name} is {adjective}'.format(**vars(kitty))
# Garfield is hungry

Now do some hacks to get rid of that .format(**locals())

import inspect
 
def printhack(s):
    caller = inspect.currentframe().f_back
    print s.format(**caller.f_locals)
 
applecolor = 'red'
 
printhack('apples are {applecolor}')
# apples are red
 
 
def dbgvar(varname):
    caller = inspect.currentframe().f_back
    print 'variable "{varname}" is {value}'.format(varname=varname, value=caller.f_locals[varname])
 
duck = 'soup'
 
dbgvar('duck')
# variable "duck" is soup

sudo tasksel install lamp-server
sudo apt-get install redmine

Select sqlite for the database (redmine-sqlite is installed as a dependency by apt-get and it’s simpler to setup than mysql).

Now to configure the Apache bits.

sudo ln -s /usr/share/redmine/public /var/www/redmine
sudo apt-get install libapache2-mod-passenger
sudo vim /etc/apache2/mods-available/passenger.conf

Add the PassengerDefaultUser www-data line to /etc/apache2/mods-available/passenger.conf. It should look like this:

<IfModule mod_passenger.c>
  PassengerDefaultUser www-data
  PassengerRoot /usr
  PassengerRuby /usr/bin/ruby
</IfModule>

Configure apache by adding these lines to /etc/apache2/sites-available/default

<Directory /var/www/redmine>
    RailsBaseURI /redmine
    PassengerResolveSymlinksInDocumentRoot on
</Directory>

Configure email

cd /etc/redmine/default
sudo cp /usr/share/redmine/config/email.yml.example email.yml
sudo vim email.yml

If you want to use gmail, delete the default settings and uncomment the gmail section settings. Add the line

    enable_starttls_auto: true

after the tls: true line. (see Redmine Email Config for more info.)

Restart Apache (you will have to restart Apache after you make any redmine config file changes)

sudo service apache2 restart

Now see if your install worked. Try http://localhost/redmine or http://server.com/redmine. Login with username admin and password admin.

Update: Until Ubuntu updates to Redmine 1.0.4 (currently it installs 1.0.0), there is a bug where dates get displayed as {{count}} if you have the Ruby gem i18 v0.5.0 installed instead of v0.4.2. Quick fix:

sudo gem install i18n -v=0.4.2
sudo gem uninstall i18n -v=0.5.0

#Must inherit from object
#class C: won't work.
class C(object):
    # Define getter for x
    #@x.getter doesn't work because self.x is not defined yet  :(
    @property
    def x(self):
        return self._x
 
    # Define setter for x
    @x.setter
    def x(self, value):
        self._x = value
 
    # Define deleter for x
    @x.deleter
    def x(self):
        del self._x
 
class D(C):
    @C.x.getter
    def x(self):
        return self._x * 2
 
    @x.setter
    def x(self, value):
        self._x = value / 2

code stolen from the What’s new in 2.6 doc

Editorializing: I don’t like the asymmetry of defining the getter and the setter. It would be better if @x.getter would work for the setter instead of using @property for the getter. But I guess it’s better than using x=property(x_getter, x_setter). Work on it, Guido.

SparkleShare is dropbox-like software that can sync to your own private server. The SparkleShare documentation recommends syncing to GitHub or Gitorius, but all you need is a server with ssh and git. By using your own private server, your files won’t be publicly viewable and you can use as much storage space as you have on your server. Here’s how to setup SparkleShare with a private server on Ubuntu.

First setup the server. You will need to have ssh access to the server because
SparkleShare will sync to an existing git repository using ssh public key login. To setup ssh public key login (passwordless login) run this on your local box

ssh-copy-id user@server.com

If you get an error, you may need to create a ssh key first. Run ssh-keygen and create a key without a password. Then run ssh-copy-id again.

If you don’t have a server, you can test on the same machine. Just use localhost for the server address.

ssh-copy-id user@localhost

Now create a git repository on your server.

#If your public key setup worked, you should
#be able to login to ssh without entering a password
ssh user@server.com
git init --bare sparkle.git
exit

Then set up SparkleShare. You will need sudo privileges for this.

# There may be more missing dependencies here
sudo apt-get install git build-essential intltool mono-devel gtk-sharp2 nant python-nautilus
mkdir ~/install
cd ~/install
git clone https://github.com/hbons/SparkleShare.git
cd SparkleShare
./autogen.sh --prefix=/usr
make
sudo make install
sparkleshare start

SparkleShare pops up a welcome dialog with some setup. Fill in user@server.com for the server and use /home/user/sparkle.git for the folder location. Then click Sync. When SparkleShare finishes syncing you should have a SparkleShare folder at ~/SparkleShare/sparkle where you can put files.

To redo the setup, stop sparkleshare sparkleshare stop then remove the sparkleshare config directory rm -rf ~/.config/sparkleshare and remove sparkleshare’s synced directory rm -rf ~/SparkleShare and restart sparkleshare sparkleshare start

To uninstall

sparkleshare stop
cd ~/install/SparkleShare
sudo make uninstall
sudo rm -rf /usr/{lib,share/gnome/help,share}/sparkleshare
rm -rf ~/SparkleShare
rm -rf ~/.config/sparkleshare

I couldn’t get numpy to work with the 64 bit version of Python 2.7 for Windows, so I uninstalled the 64 bit version and installed the 32 bit Python. Now I need Cython, but Cython gives me some vcvarsall.bat error. The Cython FAQ mentions some this vcvarsall error, but none of the workarounds there work. (Interestingly, the patch they have seems to have been applied twice to pyxbuild.py.) Getting desperate, I tried installing ActivePython, but it gets stuck on “determining disk space requirements” forever. =P

The trick to installing ActivePython is invoking the installer from a command prompt using msiexec. Set the INSTALLDIR property if you want to change the directory that ActivePython is installed to (default is C:\python27)

msiexec /package ActivePython-2.7.1.3-win64-x64.msi INSTALLDIR="C:\activepython27" /qr

I used the ORCA MSI Editor to find the INSTALLDIR property. Thanks to Harold van de Kamp’s blog for the info on using msiexec.

map <F5> <Esc>:w<CR>:!%:p<CR>

This maps the F5 key to run two things: :w, which saves your file, and :!%:p, which will run your current script (! runs a shell command and %:p expands to the path of your current file).

A test example

echo "#!/bin/bash" > test.sh
echo "echo hello world" >> test.sh
chmod +x test.sh
vim test.sh

Now pressing F5 while in Vim will save test.sh and run the file in Vim’s internal shell. Just press enter when the script is done to return to Vim.

code based on discussion at stackoverflow