An Introduction to Parallel Programming
A course for PhD students, Department of Computer Science and Engineering, University of Bologna, 2018/2019
Skip to the teaching material
Multi- and many-core processors, once found on expensive
supercomputers, are now ubiquitous; however, parallel programming
paradigms must be used to benefit from the processing power of multi-
and many-core processors, but those paradigms are little known to the
majority of programmers. In this course we introduce the basic
techniques for programming shared-memory multi-core processors (CPUs)
and graphics processing units (GPUs). For the former we will use
OpenMP/C, i.e., the OpenMP extensions to the C programming language.
For the latter, we will introduce programming general-purpose graphics
processing units (GPGPUs) using CUDA/C, a proprietary extension of the
C programming language developed by NVidia corp. for GPGPU
programming.
Syllabus
- Introduction to parallel programming
- Parallel architectures; Flynn's taxonomy
- Programming models for parallel applications
- Speedup and scalability of parallel programs
- OpenMP programming
- Basic concepts
- The
#pragma omp parallel
and #pragma omp for
directives
- Variables scoping
- OpenMP constructs for reduction and synchronization
- CUDA/C programming
- Basic concepts: kernels, threads, thread blocks
- CUDA memory hierarchy
References
-
Peter Pacheco, An
Introduction to Parallel Programming, Morgan Kaufmann,
2011, ISBN 978-0123742605
-
CUDA C programming guide
-
Georg Hager, Gerhard Wellein, Introduction to High Performance Computing for Scientists and Engineers, CRC Press,
2010, ISBN 9781439811924
-
I. Foster. Designing and Building Parallel Programs. Addison-Wesley, 1995, ISBN 978-0201575941
-
Jason Sanders, Edward Kandrot, CUDA by Example: An Introduction to General-Purpose GPU Programming, Addison-Wesley, 2010, ISBN 978-0131387683
Back to top.
Sample code
HPC1819-phd.zip contains the programs
and other sample code discussed during the lectures.
Lectures
All lectures will take place at DISI, Mura Anteo Zamboni 7, Bologna.
Unless stated otherwise, the slides below can be distributed under the terms of the Creative
Commons Attribution-Share Alike 4.0 International (CC BY-SA 4.0) license.
- 2019-01-25, 11:00, Aula seminari 2
- Introduction
[ODP] [PDF]
- 2019-01-30, 11:00, Aula Busi
- Parallel Programming Patterns
[ODP] [PDF]
- 2019-02-06, 11:00, Aula Busi
- Shared-Memory Programming with OpenMP
[ODP] [PDF]
- 2019-02-13, 11:00, Aula Busi
- CUDA Programming
[ODP] [PDF]
Back to top.