Architectures and Platforms for Artificial Intelligence (2nd module)
Department of Computer Science and Engineering, University of Bologna, 2020/2021
About the course
This is the Web page of the Architectures and platforms for Artificial Intelligence (2nd module) for the second cycle degree in Artificial Intelligence, University of Bologna.
Multi- and many-core processors, once found on expensive supercomputers, are now ubiquitous; however, parallel programming paradigms must be used to benefit from the processing power of multi- and many-core processors, but those paradigms are little known to the majority of programmers. In this module we introduce the basic techniques for programming shared-memory multi-core processors (CPUs) and graphics processing units (GPUs). For the former we will use OpenMP/C, i.e., the OpenMP extensions to the C programming language. For the latter, we will introduce programming general-purpose graphics processing units (GPGPUs) using CUDA/C, a proprietary extension of the C programming language for GPGPU programming.
Syllabus
- Parallel architectures and Flynn's taxonomy
- Parallel programming patterns (embarassingly parallel, decomposition, scatter/gather, scan, reduce, ...)
- Shared-Memory programming with OpenMP
- OpenMP programming model
- The "omp parallel" costruct
- Scoping costructs
- Other work-sharing costructs
- Some examples of applications
- GPU programming with CUDA
- CUDA architecture and terminology
- CUDA programming model
- CUDA memory hierarchy
- CUDA/C programming costructs
- Some examples of applications
References
- Peter Pacheco, An Introduction to Parallel Programming, Morgan Kaufmann, 2011, ISBN 978-0123742605
- CUDA C programming guide
- Georg Hager, Gerhard Wellein, Introduction to High Performance Computing for Scientists and Engineers, CRC Press, 2010, ISBN 9781439811924
- I. Foster. Designing and Building Parallel Programs. Addison-Wesley, 1995, ISBN 978-0201575941
- Jason Sanders, Edward Kandrot, CUDA by Example: An Introduction to General-Purpose GPU Programming, Addison-Wesley, 2010, ISBN 978-0131387683
Online resources on C programming:
-
Brian W. Kernighan and Dennis M. Ritchie, The C Programming
Language, Second Edition Prentice Hall, Inc., 1988. ISBN
0-13-110362-8 (paperback), 0-13-110370-9 (hardback). Also available
online at the
Internet Archive
This is the definitive guide to the C programming language, written by its inventors. However, this book is not appropriate for absolute beginners, since it is considered a bit terse (I disagree: this is not intended to be a bookfor dummies
, and the C language is itself quite terse). - Mike Banahan, Declan Brady and Mark Doran: The C Book, 2nd edition, Addison Wesley
- A Little C Primer
- [In Italian] Alessandro Rubini, Dalla A alla X passando per C: un distillato del linguaggio C e Approfondimenti sul linguaggio C
Tutorials on the Linux shell
Final exam
The final exam of this module consists of a programming project using OpenMP and/or CUDA. Depending on the number of students enrolled, the project might be assigned by the instructor or proposed by each student. The project must include a written report at most six pages long. The project can be handed in at any time during the academic year, up to september 2021 (when the new edition of this course will begin).
The final grade of the Architectures and Platforms for
Artificial Intelligence
course will be determined as the weighted
average of the grades of modules 1+3 and module 2, rounded to the
nearest integer. Honors (lode
) will be
awarded at the discretion of the instructors for outstanding
exams.
Teaching material
Sample code
Sample code discussed during the lectures is available here.
Lectures
Unless stated otherwise, the slides below can be distributed under the terms of the Creative
Commons Attribution-Share Alike 4.0 International (CC BY-SA 4.0) license.
- 2020-09-21
- Introduction
[ODP] [PDF]Additional material- Herb Sutter, The Free Lunch is Over: A Fundamental Turn Toward Concurrency in Software, Dr. Dobb's Journal, 30(3), March 2005.
- 2020-09-28
- Parallel architectures
[ODP] [PDF]
Per approfondire- Blaise Barney, Introduction to Parallel Computing, Lawrence Livermore National Laboratory
- Latency numbers every programmer should know
- Introduction to Hyper-Threading technology, Intel
- Parallel Programming Patterns
Additional material - Shared-Memory Programming with OpenMP
Additional material- OpenMP.org; provides the official specifications of all OpenMP versions
- Blaise Barney, OpenMP Tutorial, Lawrence Livermore National Laboratory.
- Performance Evaluation of Parallel Programs
[ODP] [PDF]
Per approfondire- Sarah Leon, Mistakes, we've drawn a few: Learning from our errors in data visualisation
- CUDA Programming
