Installing Python 2.7 on the HPCVL
We had a need for the HPCVL, but had a lot of trouble getting mpi4py working.
The default Python installation on the HPCVL is has some issues which may prevent you from installing some packages. It thinks that the GNU C compiler is available, when really we want the SUN OS C compiler. So first we need to install python the right way.
Since we don't have admin access to install things globally, we'll install Python 2.7 to our own account
Log in over SSH[edit | edit source]
First you'll need an HPCVL account. It takes a while to go through all the steps, so try to start this at least a week before you actually need it.
Assuming you've gotten everything up and running account-wise (you've got an hpcXXXX account and a password), it's time to log in.
If you're running linux or mac OSX, open up a terminal. If you're on windows (poor you), fire up (or get and then fire up) PuTTY. On Ubuntu linux there's a handy shortcut ctrl+shift+T
to get a terminal.
I'm not sure about PuTTY, but on linux and mac you can connect over ssh with a command something like this:
ssh hpcXXXX@sfnode0.hpcvl.queensu.ca
where hpcXXXX
is your HPCVL username.
You may need to type "yes" to add the ssh key, and then it should prompt for your password.Once inside you should get a prompt like this:
hpcXXXX@sflogin0$
Great, you're in!
Setting Up[edit | edit source]
First create a working directory for the python installation:
mkdir mypython
and then enter that directory.
cd mypython
To verify you're in the right place, type pwd
(print working directory). You should get something like this
/home/hpcXXXX/mypython
cool.
Getting Python[edit | edit source]
Head over to the python download page, and look for the first "bizipped source tarball". At the time of writing, that source is for python 2.7.3. You may find a more recent version is available now. At this time of writing, version 3.2.3 is actually also available, but for better compatibility I stuck with 2.7.3.
Right-click on the link for the bzipped tarball and choose copy link address (that address is http://python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2 for python 2.7.3). Now go back to your terminal and use wget to download the link
wget http://python.org/ftp/python/2.7.3/Python-2.7.3.tar.bz2
tip: middle-click in a terminal window in Ubuntu (and possibly other versions of linux) to paste. Ctrl+v
won't work.
You should get a progress indicator of the python download, which should take about 30 seconds.
It's compressed, so we need to decompress it
bzip2 -d Python-2.7.3.tar.bz2
should take a couple seconds, and then
tar -xf Python-2.7.3.tar
should take a couple more.
Now if you enter ls
to show the contents of the current directory, you should see an entry for the python .tar
file, and a directory containing all the uncompressed files.
Python-2.7.3 Python-2.7.3.tar
Installing[edit | edit source]
We need to install our Python without root access, so we'll have to tell it that. I found a very simple guide in this stackoverflow answer, which I'm copying out here.
First create a .local
directory in our home folder
mkdir ~/.local
Don't worry if it complains about the directory already existing, we just need to make sure it does exist if it didn't before.
Next we need to enter the python directory that we uncompressed in the last steps:
cd Python-2.7.3
(remember we're already inside the mypython directory we created earlier, so now we should be at ~/mypython/Python-X.X.X
)
Now we run the configuration script making sure we specify that we want to install it to our local account.
./configure --prefix=/home/hpcXXXX/.local
Make sure you replace hpcXXXX
with your username!
This script takes a while to run, and writes a lot of output to the screen.
Now we can install:
make
(this one also takes a while)
make install
(guess what? this one takes a while too!)
I got a lot of warnings when running make
and make install
, but as long as you don't see any errors, you should be fine.
Yay we're done!
Running[edit | edit source]
Your new version of python still doesn't get used when you type in python
at the prompt, at this point. If you try it, you should see that it's still the hpc version 2.6.2:
hpcXXXX@sflogin0$ python Python 2.6.2 (r262:71600, Aug 11 2009, 07:00:48) [GCC 4.0.2] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>>
To run our new version, you need to use this command:
~/.local/bin/python
Which, for me, shows me that I'm running 2.7.3:
hpcXXXX@sflogin0$ ~/.local/bin/python Python 2.7.3 (default, Jul 4 2012, 16:19:45) [C] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>>
Hooray!
You can make sure you are using this version by adding a "shebang" to the top of your python files:
#!/home/hpcXXXX/.local/bin/python
So for example I created this file whatpython.py
with these contents:
#!/home/hpcXXXX/.local/bin/python import sys print sys.version_info
and then made it executable with
chmod +x whatpython.py
and then ran it with
./whatpython.py
and it produced this output:
sys.version_info(major=2, minor=7, micro=3, releaselevel='final', serial=0)
2.7.3
. Excellent.
Alternatively, to avoid the #!
line, you could simply call our new Python directly, like this:
hpcXXXX@sflogin0$ ~/.local/bin/python whatpython.py
which gives me the same output.
Making it Default[edit | edit source]
If you'd rather not worry about the calling the right Python every time you use it, you can tell bash to use your new python by default instead. You just need to add it to the beginning of your system PATH
, by executing this command:
PATH=/home/hpcXXXX/.local/bin:$PATH; export PATH
note that this change is not permanent.
You should now see Python 2.7.X
when you type python
into the command line.
If that worked and you'd like to make the change permanent, you need to add it to your .bash_profile
.
nano ~/.bash_profile
and then add the line to the end of the file. My new file looks like this:
if [ -r ~/.bashrc ] then . ~/.bashrc fi # put your own environment settings here, e.g. # export PAGER=less # put "use" commands here to extend standard-user-settings # # e.g. uncomment next line to select specific package # use fluent PATH=/home/hpcXXXX/.local/bin:$PATH export PATH
Now whenever you log in, you should be able to use the new version of python by default.
phil@ubuntu:~$ ssh hpcXXXX@sfnode0.hpcvl.queensu.ca |-----------------------------------------------------------------| | This system is for the use of authorized users only. | | Individuals using this computer system without authority, or in | | excess of their authority, are subject to having all of their | | activities on this system monitored and recorded by system | | personnel. | | | | In the course of monitoring individuals improperly using this | | system, or in the course of system maintenance, the activities | | of authorized users may also be monitored. | | | | Anyone using this system expressly consents to such monitoring | | and is advised that if such monitoring reveals possible | | evidence of criminal activity, system personnel may provide the | | evidence of such monitoring to law enforcement officials. | |-----------------------------------------------------------------| Password: Last login: Wed Jul 4 16:43:32 2012 from d249.n116.queen This is sflogin0, a dedicated login host on the HPCVL grid SunFire E2900, 24 x 1.8 GHz UltraSPARC-IV+, 192 GB RAM ------------------------------------------------------------------ * Please contact HPCVL User Support with questions about usage http://web.archive.org/web/20110818023616/http://hpcvl.org/contactus.html Email help@hpcvl.org * For system status updates, see http://www.hpcvl.org/members/ * Available packages can be listed with "use -l" hpcXXXX@sflogin0$ python Python 2.7.3 (default, Jul 4 2012, 16:19:45) [C] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>>
Cleaning Up[edit | edit source]
If python installed successfully, you can delete the python source directory:
cd ~/ rm -rf mypython
Ta daa.
Going Forward[edit | edit source]
If you need to install more packages, you'll probably have a better time with pip. See my instructions on [install pip on the HPCVL].
I needed to go through all this in order to get mpi4py working on the HPCVL. I've got a futher guide for [installing mpi4py on the HPCVL], which you might want to follow after getting pip up and running.