Parallel computing
Undergraduate · CS / Programming
Syllabus focus
Topics typically covered
Standard syllabus
Parallel models
- Flynn's taxonomy; shared vs distributed memory
- Amdahl's and Gustafson's laws
- Threads, locks, and race conditions
- Synchronization primitives and lock-free ideas (intro)
- Parallel algorithm design: divide-and-conquer, data parallelism
Programming interfaces
- Pthreads or std::thread programming (intro)
- OpenMP directives for loop parallelism
- MPI basics: send, receive, collective ops (intro)
- GPU computing with CUDA or OpenCL (survey)
- Deterministic debugging of concurrent programs
Correctness under concurrency
- Happens-before and memory models (intro)
- Deadlocks, livelocks, and starvation
- Deterministic replay challenges
- Transactional memory survey
- Message-passing vs shared-memory design choices
- Testing concurrent code systematically
STEM / applied
Performance engineering
- Cache coherence and false sharing
- Scheduling and work-stealing (intro)
- Benchmarking parallel speedup and efficiency
- Domain decomposition for PDEs/grids (intro)
- Pipeline parallelism in data processing frameworks
Applications
- Parallel sorting and graph algorithms (intro)
- MapReduce/Hadoop/Spark programming model (survey)
- Scientific computing workloads on clusters
- Fault tolerance in distributed jobs (intro)
- Ethics and energy costs of large-scale compute
Scaling workloads
- Profiling parallel bottlenecks
- Load balancing strategies
- NUMA awareness at introductory level
- Batch vs stream parallel frameworks
- Energy and thermal limits of scale-up/out
- Capstone: speedup study with correctness checks
Notes
Mix of theory and programming varies; some courses emphasize HPC, others data-parallel frameworks.