Architectures and Platforms for Artificial Intelligence (2nd module)
Department of Computer Science and Engineering, University of Bologna, 2020/2021
News
- 8/11/2020
- The lecture on monday november 9 is rescheduled at 2pm—4:30pm; the lecture will be held online and will be recorded as usual. Apologies for the late notice.
- 28/10/2020
- The lecture scheduled on monday, november 2nd is cancelled. All future lectures of module 2 (prof. Marzolla) will be held online only. Sorry for the inconvenience.
- 10/10/2020
- Instructions on how to get an account on the server are available on virtuale.unibo.it
- 5/10/2020
- There are issues accessing the server that will be used for OpenMP/CUDA programming. New instructions will be available soon.
- 12/09/2020
- Page created.
About the course
This is the Web page of the Architectures and platforms for Artificial Intelligence (2nd module) for the second cycle degree in Artificial Intelligence, University of Bologna.
Multi- and many-core processors, once found on expensive supercomputers, are now ubiquitous; however, parallel programming paradigms must be used to benefit from the processing power of multi- and many-core processors, but those paradigms are little known to the majority of programmers. In this module we introduce the basic techniques for programming shared-memory multi-core processors (CPUs) and graphics processing units (GPUs). For the former we will use OpenMP/C, i.e., the OpenMP extensions to the C programming language. For the latter, we will introduce programming general-purpose graphics processing units (GPGPUs) using CUDA/C, a proprietary extension of the C programming language for GPGPU programming.
Syllabus
- Parallel architectures and Flynn's taxonomy
- Parallel programming patterns (embarassingly parallel, decomposition, scatter/gather, scan, reduce, ...)
- Shared-Memory programming with OpenMP
- OpenMP programming model
- The "omp parallel" costruct
- Scoping costructs
- Other work-sharing costructs
- Some examples of applications
- GPU programming with CUDA
- CUDA architecture and terminology
- CUDA programming model
- CUDA memory hierarchy
- CUDA/C programming costructs
- Some examples of applications
References
- Peter Pacheco, An Introduction to Parallel Programming, Morgan Kaufmann, 2011, ISBN 978-0123742605
- CUDA C programming guide
- Georg Hager, Gerhard Wellein, Introduction to High Performance Computing for Scientists and Engineers, CRC Press, 2010, ISBN 9781439811924
- I. Foster. Designing and Building Parallel Programs. Addison-Wesley, 1995, ISBN 978-0201575941
- Jason Sanders, Edward Kandrot, CUDA by Example: An Introduction to General-Purpose GPU Programming, Addison-Wesley, 2010, ISBN 978-0131387683
Online resources on C programming:
-
Brian W. Kernighan and Dennis M. Ritchie, The C Programming
Language, Second Edition Prentice Hall, Inc., 1988. ISBN
0-13-110362-8 (paperback), 0-13-110370-9 (hardback). Also available
online at the
Internet Archive
This is the definitive guide to the C programming language, written by its inventors. However, this book is not appropriate for absolute beginners, since it is considered a bit terse (I disagree: this is not intended to be a bookfor dummies
, and the C language is itself quite terse). - Mike Banahan, Declan Brady and Mark Doran: The C Book, 2nd edition, Addison Wesley
- A Little C Primer
- [In Italian] Alessandro Rubini, Dalla A alla X passando per C: un distillato del linguaggio C e Approfondimenti sul linguaggio C
Tutorials on the Linux shell
Final exam
The final exam of this module consists of a programming assignment using OpenMP and CUDA. The project must include a written report at most six pages long. The project can be handed in at any time during the academic year, up to september 2021 (when the new edition of this course will begin).
The specification of the assignment are available on virtuale.
The final grade of the Architectures and Platforms for
Artificial Intelligence
course will be determined as the weighted
average of the grades of modules 1+3 and module 2, rounded to the
nearest integer. Honors (lode
) will be
awarded at the discretion of the instructors for outstanding
exams.
Teaching material
Sample code
Sample code discussed during the lectures is available here.
Lectures
Unless stated otherwise, the slides below can be distributed under the terms of the Creative
Commons Attribution-Share Alike 4.0 International (CC BY-SA 4.0) license.
- 2020-09-21
- Introduction (Pacheco ch. 1)
[ODP] [PDF] [video]Additional material- Herb Sutter, The Free Lunch is Over: A Fundamental Turn Toward Concurrency in Software, Dr. Dobb's Journal, 30(3), March 2005.
- 2020-09-28
- Parallel architectures (Pacheco ch. 2, excluding 2.3.3)
[ODP] [PDF] [video]
Per approfondire- Blaise Barney, Introduction to Parallel Computing, Lawrence Livermore National Laboratory
- Latency numbers every programmer should know
- Introduction to Hyper-Threading technology, Intel
- 2020-10-05
- Parallel Programming Patterns
[ODP] [PDF] [video]Additional material- Michael McCool, Arch D. Robinson, James Reinders, Structured Parallel Programming, Morgan Kaufmann, 2012, ISBN 978-0-12-415993-8; see Chapter 3 (the book can be downloaded for free from the University network)
- Guy L. Blelloch, Prefix Sums and Their Applications
- 2020-10-12
- Shared-Memory Programming with OpenMP (Pacheco ch. 5)
[ODP] [PDF] [video]
[Instructions on how to get an account on the server are on virtuale.unibo.it]Additional material- OpenMP.org; provides the official specifications of all OpenMP versions
- Blaise Barney, OpenMP Tutorial, Lawrence Livermore National Laboratory.
- 2020-10-19
- Shared-Memory Programming with OpenMP (cont.) [video]
- 2020-10-26
- Performance Evaluation of Parallel Programs (Pacheco ch. 2.6)
[ODP] [PDF] [video]
Per approfondire- Sarah Leon, Mistakes, we've drawn a few: Learning from our errors in data visualisation
- 2020-11-09
- CUDA Programming
[ODP] [PDF] [video] - 2020-11-16
- CUDA Programming (cont.) [video]
- 2020-11-23
- CUDA Programming (cont.) [video]
- 2020-11-30
- Conclusions and wrap-up
