use MPICH bulid a sample cluster

Updated at: February 23, 2017


parallel program interface

e.g. MPI MPICH (MPI implementations for FORTRAN, C, and C++)

Setup Details:

(1) all you need to install is mpich:(e.g. ubuntu) – simple..

sudo apt-get install mpich

(2) launch a bunch of instances

(3) edit hosts file on master&slave add all ip into master hosts file

geluo@facebook-core-7868:~$ sudo cat /etc/hosts
[sudo] password for geluo: localhost facebook-core-7868 master backup slave1 slave2
10.148.248.xx6 slave3 slave4 slave5

add master ip into slave hosts file..

(4) add pubkey of master node to each slave

geluo@facebook-core-7868:~$ ssh geluo@slave1 'cat /home/geluo/.ssh/authorized_keys'

(5) have a try with mpirun on master

geluo@facebook-core-7868:~$ mpirun -np 7 -host master,backup,slave1,slave2,slave3,slave4,slave5 hostname

(6) share folder between master/slave install NFS server on master

sudo apt-get install nfs-kernel-server

export your share foler

geluo@facebook-core-7868:~$ cat /etc/exports
/mnt/shared *(rw,sync,no_root_squash,no_subtree_check)
$ exportfs -a

install NFS client on slave

$ sudo apt-get install nfs-common
$ sudo mount -t nfs master:/mnt/shared  /mnt/shared
geluo@facebook-salve-std1-7635:~$ df -hT
Filesystem Type Size Used Avail Use% Mounted on
udev devtmpfs 5.9G 12K 5.9G 1% /dev
tmpfs tmpfs 1.2G 392K 1.2G 1% /run
/dev/disk/by-label/cloudimg-rootfs ext4 57G 1.5G 53G 3% /
none tmpfs 4.0K 0 4.0K 0% /sys/fs/cgroup
none tmpfs 5.0M 0 5.0M 0% /run/lock
none tmpfs 5.9G 0 5.9G 0% /run/shm
none tmpfs 100M 0 100M 0% /run/user
/dev/vdb vfat 64M 34K 64M 1% /mnt
master:/mnt/shared nfs4 591G 19G 542G 4% /mnt/shared