Contents:
Parallel Computing Courses in Universities
Carnegie Mellon
- Course Name: Parallel Computer Architecture and Programming
- Course Number: 15-418
- Course Overview:
- Optimization
- Memory management
- Multiprocessor design
- GPU vs CPU applications
- Scaling
- The entire syllabus along with readings and lecture notes can be found here.
Colorado School of Mines
- Course Name: Parallel Computing for Scientists and Engineers
- Since Parallel Computing is considered a highly specialized field it is usually only offered advanced computer science students. The point of this course is to expose Parallel Computing not only to highly advanced programmers but also to non computer scientists.
- Course Overview:
- Unix workstations.
- Design and analysis of sequential and parallel algorithms, performance analysis and optimization.
- Parallelizing compilers, parallel debuggers, graphical user interface and their effect on program development.
- Writing parallel programs for shared versus distributed memory machines.
- Taxonomy of parallel computers.
- Interconnection networks, parallel computing architecture issues and their effect on program development.
- The architecture of the 3 machines used in the course.
- Information about the course can be found here.
Columbia University
- Course Name: Principles and Practice of Parallel Programming
- Course Number: COMS 4130
- Pre-reqs: Experience with Java; knowledge of algorithm analysis, COMS W1004 (Intro to CS, = CS111) and COMS W3137 ("Data Structures and Algorithms", = CS 112)
- Course Overview:
- Uses X10 (an IBM parallel programming language)
- Covers the basics of parallel program design
- Theory: granularity, data-level parallelism, load-balancing, deadlock, safe methods for parallelism (e.g. locks, barriers, etc.)
- Study already-made algorithms from various types of fields (games, machine learning, high-performance computing), as well as frameworks like Map/Reduce
- Homework assignments are parallel programs
- Main part of the course is implementing a challenge program on a cluster of multi-core processors; to be done throughout the semester
- Course website can be found here.
Georgia Institute of Technology
- Course Name: Intro to Parallel Computing
- Course Number: ECE 4893
- Course Overview:
- Parallel computing systems.
- Parallel programming.
- Parallel algorithms.
- Upon the completion of the course, the students are expected to:
- Understand parallel computing systems.
- Be able to design algorithms.
- Be able to implement programs for such systems.
- Article about the course can be found here.
Harvard University
- Course Name: Massively Parallel Computing
- Course Number: CS 264
- Course Overview:
- Parallel programming models
- GPU programming with CUDA
- Cluster computing with MPI and ZeroMQ
- Cloud computing with MapReduce using Hadoop on Amazons EC2 (with Cluster GPU Instances and MIT StarCluster)
- Course website can be found here.
Massachusetts Institute of Technology (MIT)
- Course Name: Applied Parallel Computing
- Course Number: SMA 5505 (Math Department)
- Course Overview:
- Linear Algebra
- N-body problems
- Multigrid
- Fast-Multipole
- Wavelets
- Fourier Transforms
- Mesh Generation
- Much more math topics than other schools, big focus on mathematic applications.
- Lecture notes can be found here.
Purdue University
- Course Name: Parallel Computing
- Course Number: CS 525
- The course deals with emerging trends in the use of large scale computing platforms ranging from desktop multicore processors, tightly coupled SMPs, message passing platforms, and state-of-the-art virtualized cloud computing environments.
- Course Overview:
- Parallel and distributed platforms.
- Parallel Programming.
- Parallel and Distributed Algorithms.
- Applications of Parallel Computing.
- Course website can be found here.
Rice University
- Course Name: Parallel Computing
- Course Number: CS 422
- Goal of course is to introduce students to:
- The foundations of parallel computing
- Principles of parallel algorithm design
- Analytical modeling of parallel programs
- OpenMP and MPI programming models for shared- and distributed-memory systems
- Parallel computer architectures
- Numerical and non-numerical algorithms for parallel systems
- A key aim of the course is for the student to gain a hands-on knowledge of the fundamentals of parallel programming by writing efficient parallel programs in some of the programming models that you learn in class.
- Course website can be found here.
Rochester Institute of Technology (RIT)
- Course Name: Parallel Computing I
- Course Number: CS 531/735
- Prereqs: Knowledge of Java, knowledge of concurrent programming (threads, synchronization, etc.), Operating Systems I, Computer Organization, and Computer Science I-IV
- Course overview:
- Architectures: SMPs and clusters, along with the software used for them
- Calculate metrics: speedup, running time, startup
- Theory: explain what different parallel programming models are and when to use them
- Programs: assignments are based on writing programs for SMP and cluster parallel computers
- Course website can be found here.
Stanford University
- Course Name: Programming Massively Parallel Processors with CUDA
- Course Number: CS 193
- Course Overview:
- The aim of this course is to provide students with knowledge and hands-on experience in developing applications software for processors with massively parallel computing resources.
- NVIDIA processors and the CUDA programming tools are used in the lab section of the course
- Syllabus is offered here.
- Course website can be found here.
Stony Brook University
- Course Name: Parallel Programming.
- Course Number: CSE 613.
- Course explores algorithms and techniques for programming state-of-the-art shared-memory (e.g., multicores) and distributed-memory parallel computers. The course will include both theoretical and programming components.
- Course Overview:
- Analytical modeling of parallel programs.
- Bounds on parallel performance, scheduling, synchronization.
- Programming using the message-passing paradigm and for shared address-space platforms.
- Parallel algorithms for dense matrix operations, sorting, searching, graphs, computational geometry, and dynamic programming, concurrent data structures, and transactional memory.
- Course website can be found here.
University of California, Berkeley
- Course Name: Parallel Programming for Multicore.
- Course Number: SC 194.
- This course provides an introduction to parallel programming with a focus on the techniques that are most appropriate for multicore architectures and the use of parallelism to improve program performance.
- The purpose of this course is to teach the next generation of programmers to develop exciting new applications that can take advantage of on-chip parallelism.
- Course Overview:
- Performance analysis and tuning
- Data and task parallelism
- Synchronization techniques
- Shared data structures
- Load balancing
- Course website can be found here.
University of Illinois
- Course Name: Applied Parallel Programming.
- Course Number: ECE 498. (ECE-> Department of Electrical and Computer Engineering)
- This course and the Stanford course seem to be sharing the exact same material and syllabus, both courses are identical.
Virginia Tech
- Course Name: Parallel Computing and Visualization on GPU
- Course Number: CS 5984
- Objectives of course:
- Understand the basic ray-tracing algorithm and some advanced real-time visualization techniques.
- Understand the computational profile and performance bottleneck for these visualization algorithms.
- Understand the parallel architecture and programming framework for Graphics Processing Units.
- Learn the standard optimization processes for GPU-based parallel programs.
- Learn and design the GPU-based parallel implementations for real-time visualization algorithms.
- Understand the state-of-art researches in real-time visualization and GPU-based performance acceleration.
- Understand the concept of In-Situ visualization, and design a simple GPU-based In-Situ application.
- Course website can be found here.
Comparison Spread Sheets
Less Detailed
More Detailed
Textbook and Papers Used in Courses
Compiled List