When trying to run Mathematica over SSH using Cygwin, you will probably get some error message like

ssh -Y user@host
mathematica
xset: bad font path element (#23), possible causes are:
   Directory does not exist or has wrong permissions
   Directory missing fonts.dir
   Incorrect font server address or syntax

This is because you need to install the Mathematica fonts into Cygwin’s X server. This solution worked for me using the latest Cygwin 1.7.2:

  1. Download the linux fonts from Wolfram’s latest Mathematica fonts into your Cygwin home C:\cygwin\home\username 
    wget http://support.wolfram.com/technotes/MathematicaV7FontsLinux.tar.gz
  2. Unzip the fonts 
    tar xvfz MathematicaV7FontsLinux.tar.gz
  3. You should have a Fonts folder now with subfolders TTF, SVG, etc inside. All the subfolders need to be copied into /usr/share/fonts so you end up with /usr/share/fonts/TTF/, /usr/share/fonts/SVG/, etc. 
    cd Fonts
cp * /usr/share/fonts
  4. Exit and restart your Cygwin/X server.

References:
https://wiki.lepp.cornell.edu/lepp/bin/view/Computing/CygwinXorg
http://x.cygwin.com/docs/faq/cygwin-x-faq.html#q-error-font-eof
http://handyfloss.wordpress.com/2006/06/20/mathematica-fonts-error