Parallelization Programming

There are many parallelization programming libraries and languages out there. In our group, we mainly use MPI and OpenMP (or hybrid) to deal with distributed and shared memory, respectively. Both libraries are mainly targeting CPU. Here we will not cover programming languages or libraries for GPU. If you would like to learn a programming tool for GPU, we would recommend OpenACC. We would recommend learning OpenMP and OpenACC first, because they are easy to understand. Below, we provide some online tutorials. Please feel free to find more tutorials by your own.

• OpenMP (shared memory on CPU)
  • XSEDETraining OpenMP
  • Introduction to OpenMP - Tim Mattson (Intel)
  • OpenMP Webinars (Mastering OpenMP)
  • Eurepean OpenMP Users Conference 2020
• MPI (distributed memory among different nodes)
  • XSEDETraining MPI
  • Argonne ATPESC2016 Programming Models and Languages
• OpenACC (GPU)
  • XSEDETraining OpenACC

Also, you can consider taking Marco’s course (MS 141), which covers a few lectures on MPI and OpenMP.

---