This is the mail archive of the gsl-discuss@sourceware.org mailing list for the GSL project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Adding OpenMP support for some of the GSL functions


Hi Rhys,
I did the comparison you requested. Here is the load balancing information for my test run :
These were all obtained with gcc 4.7.2. The total times for the runs were :
- Without vectorization or OpenMP : 1m21s
- With vectorization : 1m14s
- With OpenMP : 1m01s


Here is load-balance information obtained with OpenSpeedshop.
Without vectorization or OpenMP :
Max Posix ThreadId Min Posix ThreadId Average Function (defining location)
Exclusive of Max Exclusive of Min Exclusive
Time Across Time Across Time Across
Posix Posix Posix
ThreadIds(s) ThreadIds(s) ThreadIds(s)


28.085714 140384673677232 28.085714 140384673677232 28.085714 rkf45_apply (libgsl.so.0.16.0)

With OpenMP :
Max Posix ThreadId Min Posix ThreadId Average Function (defining location)
Exclusive of Max Exclusive of Min Exclusive
Time Across Time Across Time Across
Posix Posix Posix
ThreadIds(s) ThreadIds(s) ThreadIds(s)


[...]
2.800000 1146448192 1.228571 1088166208 1.778571 rkf45_apply._omp_fn.4 (libgsl.so.0.16.0)
2.171429 1146448192 0.971429 1129662784 1.460714 rkf45_apply._omp_fn.6 (libgsl.so.0.16.0)
2.142857 1146448192 1.400000 140073673240496 1.671429 rkf45_apply._omp_fn.3 (libgsl.so.0.16.0)
2.085714 1146448192 1.257143 1112877376 1.725000 rkf45_apply._omp_fn.5 (libgsl.so.0.16.0)
2.085714 1146448192 1.171429 1088166208 1.578571 rkf45_apply._omp_fn.2 (libgsl.so.0.16.0)
1.600000 1146448192 0.885714 140073673240496 1.139286 rkf45_apply._omp_fn.1 (libgsl.so.0.16.0: rkf45.c,233)
0.714286 1146448192 0.457143 1129662784 0.557143 rkf45_apply._omp_fn.0 (libgsl.so.0.16.0)
[...]
The 7 loop make a total of about ~12s.



With vectorization :
Max Posix ThreadId Min Posix ThreadId Average Function (defining location)
Exclusive of Max Exclusive of Min Exclusive
Time Across Time Across Time Across
Posix Posix Posix
ThreadIds(s) ThreadIds(s) ThreadIds(s)


24.542857 140440801677232 24.542857 140440801677232 24.542857 rkf45_apply (libgsl.so.0.16.0)


This was obtained with gcc 4.7.2, with 2 quad-core CPUs, for a total of 8 threads. To summarize :
- Vectorization makes a difference, but it is minor (gained 4s out of the 28s used by the normal rkf45_apply ).
- OpenMP is the clear winner (gained 20s out of 28s).



Maxime






Le 2012-12-13 10:51, Rhys Ulerich a écrit :
I am doing that too, but any gain we can get is an important one, and it
turns out that by parallelizing rkf45_apply, my simulation runs 30% faster
on 8 cores.
That's a parallel efficiency of 0.18% ( = 1 time unit / (8 cores *
0.70 time units)).  This feels like you're getting a small
memory/cache bandwidth increase for the rkf45_apply level-1-BLAS-like
operations by using multiple cores but the cores are otherwise not
being used effectively.  I say this because a state vector 1e6 doubles
long will not generally fit in cache.  Adding more cores increases the
amount of cache available.

I will have a deeper look at vectorization of GSL, but in my understanding,
vectorizing can only be done with simple operations, while algorithms like
RKF45 involve about 10 operations per loop iterations.
The compilers are generally very good.  Intel's icc 11.1 has to be
told that the last four loops you annotated are vectorizable.  GCC
nails it out of the box.

On GCC 4.4.3 with something like
     CFLAGS="-g -O2 -march=native -mtune=native
-ftree-vectorizer-verbose=2 -ftree-vectorize" ../gsl/configure && make
shows every one of those 6 loops vectorizing.  You can check this by
configuring with those options, running make and waiting for the build
to finish, and then cd-ing into ode-initval2 and running
     rm rkf45*o && make
and observing all those beautiful
     LOOP VECTORIZED
messages.  Better yet, with those options, 'make check' passes for me
on the 'ode-initval2' subdirectory.

Try ripping out your OpenMP pragmas in GSL, building with
vectorization against stock GSL as I suggested, and then seeing how
fast your code runs with GSL vectorized on 1 core versus GSL's
unvectorized rkf45_apply parallelized over 8 cores.  I suspect it will
be comparable.

- Rhys


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]