Installing Python 2.7 on the HPCVL
|This page was developed by the Queen's University Applied Sustainability Research Group.|
The default Python installation on the HPCVL is has some issues which may prevent you from installing some packages. It thinks that the GNU C compiler is available, when really we want the SUN OS C compiler. So first we need to install python the right way.
Since we don't have admin access to install things globally, we'll install Python 2.7 to our own account
 Log in over SSH
First you'll need an HPCVL account. It takes a while to go through all the steps, so try to start this at least a week before you actually need it.
Assuming you've gotten everything up and running account-wise (you've got an hpcXXXX account and a password), it's time to log in.
If you're running linux or mac OSX, open up a terminal. If you're on windows (poor you), fire up (or get and then fire up) PuTTY. On Ubuntu linux there's a handy shortcut
ctrl+shift+T to get a terminal.
I'm not sure about PuTTY, but on linux and mac you can connect over ssh with a command something like this:
hpcXXXX is your HPCVL username.
You may need to type "yes" to add the ssh key, and then it should prompt for your password.Once inside you should get a prompt like this:
Great, you're in!
 Setting Up
First create a working directory for the python installation:
and then enter that directory.
To verify you're in the right place, type
pwd (print working directory). You should get something like this
 Getting Python
Head over to the python download page, and look for the first "bizipped source tarball". At the time of writing, that source is for python 2.7.3. You may find a more recent version is available now. At this time of writing, version 3.2.3 is actually also available, but for better compatibility I stuck with 2.7.3.
Right-click on the link for the bzipped tarball and choose copy link address (that address is http://python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2 for python 2.7.3). Now go back to your terminal and use wget to download the link
tip: middle-click in a terminal window in Ubuntu (and possibly other versions of linux) to paste.
Ctrl+v won't work.
You should get a progress indicator of the python download, which should take about 30 seconds.
It's compressed, so we need to decompress it
bzip2 -d Python-2.7.3.tar.bz2
should take a couple seconds, and then
tar -xf Python-2.7.3.tar
should take a couple more.
Now if you enter
ls to show the contents of the current directory, you should see an entry for the python
.tar file, and a directory containing all the uncompressed files.
We need to install our Python without root access, so we'll have to tell it that. I found a very simple guide in this stackoverflow answer, which I'm copying out here.
First create a
.local directory in our home folder
Don't worry if it complains about the directory already existing, we just need to make sure it does exist if it didn't before.
Next we need to enter the python directory that we uncompressed in the last steps:
(remember we're already inside the mypython directory we created earlier, so now we should be at
Now we run the configuration script making sure we specify that we want to install it to our local account.
Make sure you replace
hpcXXXX with your username!
This script takes a while to run, and writes a lot of output to the screen.
Now we can install:
(this one also takes a while)
(guess what? this one takes a while too!)
I got a lot of warnings when running
make install, but as long as you don't see any errors, you should be fine.
Yay we're done!
Your new version of python still doesn't get used when you type in
python at the prompt, at this point. If you try it, you should see that it's still the hpc version 2.6.2:
hpcXXXX@sflogin0$ python Python 2.6.2 (r262:71600, Aug 11 2009, 07:00:48) [GCC 4.0.2] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>>
To run our new version, you need to use this command:
Which, for me, shows me that I'm running 2.7.3:
hpcXXXX@sflogin0$ ~/.local/bin/python Python 2.7.3 (default, Jul 4 2012, 16:19:45) [C] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>>
You can make sure you are using this version by adding a "shebang" to the top of your python files:
So for example I created this file
whatpython.py with these contents:
#!/home/hpcXXXX/.local/bin/python import sys print sys.version_info
and then made it executable with
chmod +x whatpython.py
and then ran it with
and it produced this output:
sys.version_info(major=2, minor=7, micro=3, releaselevel='final', serial=0)
Alternatively, to avoid the
#! line, you could simply call our new Python directly, like this:
hpcXXXX@sflogin0$ ~/.local/bin/python whatpython.py
which gives me the same output.
 Making it Default
If you'd rather not worry about the calling the right Python every time you use it, you can tell bash to use your new python by default instead. You just need to add it to the beginning of your system
PATH, by executing this command:
PATH=/home/hpcXXXX/.local/bin:$PATH; export PATH
note that this change is not permanent.
You should now see
Python 2.7.X when you type
python into the command line.
If that worked and you'd like to make the change permanent, you need to add it to your
and then add the line to the end of the file. My new file looks like this:
if [ -r ~/.bashrc ] then . ~/.bashrc fi # put your own environment settings here, e.g. # export PAGER=less # put "use" commands here to extend standard-user-settings # # e.g. uncomment next line to select specific package # use fluent PATH=/home/hpcXXXX/.local/bin:$PATH export PATH
Now whenever you log in, you should be able to use the new version of python by default.
phil@ubuntu:~$ ssh hpcXXXX@sfnode0.hpcvl.queensu.ca |-----------------------------------------------------------------| | This system is for the use of authorized users only. | | Individuals using this computer system without authority, or in | | excess of their authority, are subject to having all of their | | activities on this system monitored and recorded by system | | personnel. | | | | In the course of monitoring individuals improperly using this | | system, or in the course of system maintenance, the activities | | of authorized users may also be monitored. | | | | Anyone using this system expressly consents to such monitoring | | and is advised that if such monitoring reveals possible | | evidence of criminal activity, system personnel may provide the | | evidence of such monitoring to law enforcement officials. | |-----------------------------------------------------------------| Password: Last login: Wed Jul 4 16:43:32 2012 from d249.n116.queen This is sflogin0, a dedicated login host on the HPCVL grid SunFire E2900, 24 x 1.8 GHz UltraSPARC-IV+, 192 GB RAM ------------------------------------------------------------------ * Please contact HPCVL User Support with questions about usage http://www.hpcvl.org/contactus.html Email firstname.lastname@example.org * For system status updates, see https://www.hpcvl.org/members/ * Available packages can be listed with "use -l" hpcXXXX@sflogin0$ python Python 2.7.3 (default, Jul 4 2012, 16:19:45) [C] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>>
 Cleaning Up
If python installed successfully, you can delete the python source directory:
cd ~/ rm -rf mypython
 Going Forward
If you need to install more packages, you'll probably have a better time with pip. See my instructions on [install pip on the HPCVL].
I needed to go through all this in order to get mpi4py working on the HPCVL. I've got a futher guide for [installing mpi4py on the HPCVL], which you might want to follow after getting pip up and running.