mpi performance/settings issue

Hi,

I am using intel mpi 4.1.3 with different process managers, mpd/hydra and got very different behavior

when i use mpd (mpiexec -perhost 32 -nolocal -n 384 -env I_MPI_FABRICS shm:dapl ./wrf.exe ) i've got all 32 cores on 100% on each nodes.  

when i use hydra (mpirun -perhost 32 -nolocal -n 384 -env I_MPI_FABRICS shm:dapl ./wrf.exe) i've got only 26 cores on 100% the rest are about 0% cpu time. and the performance decrease about two times.

What is the explanation? how to fix this?

 

Regards,

SK