Threaded Programming via OpenMP

CS 441 Lecture, Dr. Lawlor

Because threaded programming is so ugly and tricky, there's a newish (mainstream in 2007) language extension out there called OpenMP, designed to make it substantially easier to write multithreaded code.

The basic idea is you take what looks like an ordinary sequential loop, like:

    for (int i=0;i<n;i++) do_fn(i);

And you add a little note to the compiler saying it's a parallel forloop, so if you've got six CPUs, the iterations should be spread across the CPUs. The particular syntax they chose is a "#pragma" statement, with the "omp" prefix:

#pragma omp parallel for
    for (int i=0;i<n;i++) do_fn(i);

Granted, this line has like a 10us/thread overhead, so it won't help tiny loops, but it can really help long loops.

You can also add a variety of interesting options to the "parallel for" line:

"private(n)" would give each CPU a separate read-write copy of the variable "n" (or any other variable).
"reduction(+:k)" would total up each CPU's (private) copy of the variable "k" (or any other variable) using the "+" operation (or any other operation). The compiler generates atomic ("lock" prefix) instructions to do this.
"schedule(dynamic,10)" tells each thread to do only ten iterations of the loop, instead of n/num_cores iterations.

In addition to "pragma omp parallel for", there are other pragma lines:

"barrier" tells each thread to not to continue until all threads have reached this point.
"critical" only allows one thread at a time to execute the following block (wraps a lock around the block).

Note that this is still shared-memory threaded programming, so global variables are still (dangerously) shared by default!

Here's how you enable OpenMP in various compilers. Visual C++ 2005 (but NOT express!), Intel C++ version 9.0, and gcc version 4.2 all support OpenMP, although earlier versions do not!

Here's the idiomatic OpenMP program: slap "#pragma parallel for" in front of your main loop. You're done!

Here's a more complex "who am I"-style OpenMP program from Lawrence Livermore National Labs. Note the compiler "#pragma" statements!

On the powerwall, you can compile and run OpenMP code as follows:
g++-4.2 get_info.c -o get_info -fopenmp
./get_info

Chris Granade has a nice page on OpenMP on the Cell Broadband Engine.