Indian Institute of Information Technology, Allahabad
Department of Information Technology
Course Syllabus
a) Understand the basics of various parallel and distributed computing platforms
b) Identify the models and frameworks best suited to various workloads.
b) Provide solutions to parallel and distributed computing problems.
Unit | Topics for Coverage |
Unit 1 | Introduction to PDC: Latency vs. Bandwidth, Applications and Challenges, Types of architecture, Flynn’s taxonomy, Basic concepts: cores, nodes, threads, processes, speedup, efficiency, overhead, strong and weak scaling (Amdahl’s law, Gustafson’s law), Cache, Principle of Locality, Programming Models. |
Unit 2 | Distributed Computing: Distributed Memory, Message Passing Interface, Asynchronous/Synchronous computation/communication, concurrency control, fault tolerance, Distributed Programming with OpenMPI. |
Unit 3 | Parallel Computing: shared memory, data and task parallelism, Synchronization, Concurrent Data Structures, Shared Memory Programming with available APIs: PThreads, OpenMP, TBB. |
Unit 4 | GPU Programming: GPU Architecture, Programming Models: CUDA/OpenCL, Basic Concepts: Threads, Blocks, Grids, GPU memory hierarchy, Thread Scheduling, Warps and Control divergence, Memory Coalescing, Programming with CUDA, Using CUDA Libraries: CuBLAS, CuFFT. |