ANNOUNCEMENTS:
We will meet in the lab thursday at 3:30 for our final discussion.
Here is a link to the research handout from last week:
research problems for
consideration
raven cluster is partly up – raven0,
raven1, raven3 and raven6 are working. Your accounts are on raven1.
The instructions in “running on raven” link at the bottom of this page work, except that you should substitute home1 for home0 when referring to your working directory.
raven0 is somewhat flaky – in case it goes down, the mpi installation has been copied to /home/mpich on raven1 which also appears as /home1/mpich.
The link “Collected Notes with Index” has been mildly updated to correct errors in the performance section
Latest version of PC compiler - PC-2.9.1-1.7-src.tgz source
can be found at:
- from http://www.csci.csusb.edu/egomez/
-
select link: PC Research Group
- select link: files
This
version has be re-installed and re-compiled on raven0 as of start of
term
Your syllabus is this page plus the links:
Class
Schedule/outline (preliminary)
Grading
and assessment
If you are in need of an accomodation for
a disability in order to participate in this class, please see the
instructor and contact Services to Students with Disabilities at
(909)537-5238.
Professor Ernesto Gomez
phone: (909)537-5429
office:
jb337
home page: www.csci.csusb.edu/egomez/
email:
ernesto@csusb.edu
Class:
Tuesday-Thursday 3:30-5:20PM, College of Education 113
office
hours: TBA (or by appointment)
text: L. Ridgway Scott, Terry Clark, Babak Bagheri, "Scientific Parallel Computing", Princeton 2005
Description:
This course will focus on programming
distributed computer systems. The course will include theoretical
models of distributed processing, their application to real systems,
programming paradigms and algorithms for distributed processing.
Specific objectives of the course are:
To study the main theoretical models of distributed processing
To study the major programming paradigms of distributed
processing
To learn how distributed systems are programmed
through some examples of distributed algorithms
To learn how
program a distributed computer system in the message passing model,
using MPI
Other topics will be covered as resources and time
permit; added topics may be explored depending on student interest.
This class will include advanced topics of current research that are not present in the text (or in any text). Where possible, additional references will be cited, and notes will be provided, but some topics will, of necessity only be covered in class discussion. It is your responsibility to take notes of such topics.
Class
Schedule/outline (preliminary)
Grading
and assessment
Introduction
- models - (pdf)
Performance
Networks
Notes
1
Notes
2
Control
flow graph
Parallel
state and execution - notes on dependences - updated
The
semantics of point-point messages
Collective
communications
Old
deadlock notes - include the basics
Paralell
prefix - process sets sample code
Collected
notes with index
Wednesday's discussion of 'The Loop' is on
pg 34-36.
Recommended references:
1. Designing and
Building Parallel Programs , Ian Foster (Full text online)
2.
Harry F. Jordan and Gita Alaghband, "Fundamentals of Parallel
Processing", Prentice Hall 2003
3. Hagit Attiya and Jennifer
Welch, "Distributed Computing - Fundamentals, Simulations and
Advanced Topics", McGraw-Hill 1998
4. J. Blazaewicz, K.
Ecker, B. Plateaus and D. Trystram, Eds. "Handbook on Parallel
and Distributed Processing", Springer 2000
5. Raymond
Greenlaw, H.James Hoover, Walter L. Ruzzo; "Limits to Parallel
Computation", Oxford University Press
6. Andrew S. Tanenbaum
and Maarten van Steen, " Distributed Systems - Principles and
Paradigms", Prentice-Hall 2002
7. Krzystof R. Apt and
Ernst-Rudinger Olderog, "Verification of Sequential and
Concurrent Programs", Second Edition, Springer 1997
Web references:
Parallel computing timeline (from Gregory V. Wilson, University
of Toronto): Text
Cornell Theory
Center
MPI
documentation and downloads -- MPICH Download site
PVM
Home Page and downloads
CRPC
- Rice University
Analysis
of Algorithms
BSP
home page
MIT
parallel+distributed systems
MIT
DSM download page
Planguages references:
Pfortran manual: Postcript
/ PDF
PC manual: Postcript
/ PDF
User
Guides: PC
/ Pfortran
PC Code Downloads:
Latest
PC source and notes
Planguages main site:
University
of Chicago CS
Also see item 2,in Recommended references,
above.
Postcript viewer (necessary to read notes, online papers:) Ghostview
Assignments:
Suggestions for individual project
-
Theory
-
Programming
Programming
and theory problems - skip 3,4 - do the others.
Target dates
for each project/problem are stated in the link. Late work will be
accepted without penalty up to the last class day of the term which
is a final deadline.
More notes:
Notes
on programming
Visualization of parallel and distributed
execution: Postcript
/ PDF
The
Loop!
SPMD
execution, synchronization, fusions (old)
Loop
transformations and message passing
Leader
election
Barrier implementation: hypercube
reduction/broadcast algorithm : sample
code
MPI
1.1 standard
MPI
2 standard
MPICH manual: Postcript
/ PDF
The PC compiler:
PC compiler is installed on raven, in /home0/ernesto/apar/PC. The
file README.PC in that directory explains which files you need to
copy, how to test and use the PC compiler.
The latest version of
PC is not yet installed on jb359 - watch this space for news.
If
you want to install a copy of the compiler on your own system, email
me to request instructions on where and how to download it.
If you get '-laio not found' error when running MPI
programs, you need to place the mpich/bin directory at the start of
your PATH environment variable - the error is due to calling the
wrong version of mpirun (see me for details on this).
The
directory /u/faculty/egomez/mpi-examples contains cpi.c and all the
header files, also a compiled version of same that you can test, also
file machines.4 that you can use in your tests. You can copy this
directory to your account, then cd into it and try
"mpirun
-np 4 -machinefile machines.4 cpi", and
"mpicc -lm cpi.c
-o cpi"
Examples of distributed programs in MPI and
Planguages may be found in ~egomez/mpich/examples and in
~egomez/plang/install/examples
Programming note: if
programming in MPI and C, printf from different processes may appear
out of order, even if the processes are ordered, due to buffered
standard output. When using printf to standard output, follow it with
"fflush(stdio);" to force the buffers to empty. Only do
this if output order from multiple processes is important, because it
slows things down.