Hi, I am not a computer science major, so I am very much confused by lots of the user guide online in this relevant topic, but I have decent knowledge in matlab and fortran coding, just have not done parallel computing with fortran before. I would appreciate if you can provide your replies in layman fashion. 

Here is my situation: I have installed Microsoft Visual Studio 2015 and recently installed Intel Parallel Studio XE Cluster Edition for Windows*, I have windows 10 x64 system and multiple cores in my CPU. I would like to harvest my parallel computer power using fortran. So I got this Hello World sample code for a try and clicked build button on my MS VS: 

    program Console3

    implicit none

    include "mpif.h"
    integer ierr, my_rank, nprocs

    call MPI_INIT(ierr)
    call MPI_COMM_RANK(MPI_COMM_WORLD,my_rank,ierr)
    call MPI_COMM_SIZE(MPI_COMM_WORLD,nprocs,ierr)

    write(*,*) 'Hello from processor ',my_rank,' out of ', nprocs

    call MPI_FINALIZE(ierr)
    stop

    end program Console3

But the following errors appeared in the building process:

Severity    Code    Description    Project    File    Line    Suppression State
Error        Compilation Aborted (code 1)        c:\users\wg7\documents\visual studio 2015\Projects\Console3\Console3\Console3.f90    1    
Error        error #7002: Error in opening the compiled module file.  Check INCLUDE paths.   [MPI]        c:\users\wg7\documents\visual studio 2015\Projects\Console3\Console3\Console3.f90    17    
Error        error #6404: This name does not have a type, and must have an explicit type.   [MPI_COMM_WORLD]        c:\users\wg7\documents\visual studio 2015\Projects\Console3\Console3\Console3.f90    23    

So I figured maybe I need to do something extra to use this MPI library. I tried to read some user guide: https://software.intel.com/en-us/node/535525, but the language there is all very confusing and unclear to me, I have also tried in command prompt:

C:\Program Files (x86)\IntelSWTools\mpi\5.1.3.207\intel64\bin>mpivars

Intel(R) MPI Library 5.1 Update 3 for Windows* Target Build Environment for Intel(R) 64 applications
Copyright (C) 2007-2015 Intel Corporation. All rights reserved.

C:\Program Files (x86)\IntelSWTools\mpi\5.1.3.207\intel64\bin>hydra_service -install
OpenSCManager failed:
Access is denied. (error 5)
Unable to remove the previous installation, install failed.

C:\Program Files (x86)\IntelSWTools\mpi\5.1.3.207\intel64\bin>hydra_service -start
OpenSCManager failed:
Access is denied. (error 5)

But I am not sure what it did nor whether it helped. Here is what I know and what I do not know:

1) I know the intel cluster edition should already include MPI library which includes everything I need. 

2) I know I need to link MPI library with MS VStudio but I do not know how or whether it is already linked, or how to set the parameters in property. 

3) I know the fortran code needs to include some special lines for MPI, but I do not know how to compile and build it to become an exe application.

4) I know where is the installation folder with the mpi applications and where is my fortran project folder, they are separate folders. But sometimes I do not know which folder to direct to in command prompt, or whether I should add which path to which. 

Searching online always just gets me little pieces for the above problem, but what I really would like is to see a comprehensive step by step guide and examples. If you would help me put things together, I would really appreciate it. Thank you! 

I am new to using mpi. I just followed the steps provided in 'Configuring a Visual Studio Project' in intel resources. However, I am getting this output where it says mpiexec.exe has exited. Here is the outputs from debug. Please help me figure out what is the problem here. 

I have a cluster made up of 15 nodes with 8 Intel Xeon E3-1230 V2 processors per node. The cluster is running Rocks v6.2. I recently installed Intel Parallel Studio XE Cluster Edition for Linux on the cluster. Now when I try to run a parallel program through the SGE scheduler using Intel MPI I get the error

[2] DAPL startup: RLIMIT_MEMLOCK too small
[3] DAPL startup: RLIMIT_MEMLOCK too small
[5] DAPL startup: RLIMIT_MEMLOCK too small
[6] DAPL startup: RLIMIT_MEMLOCK too small
[7] DAPL startup: RLIMIT_MEMLOCK too small
[1] DAPL startup: RLIMIT_MEMLOCK too small
[4] DAPL startup: RLIMIT_MEMLOCK too small
[0] DAPL startup: RLIMIT_MEMLOCK too small

I already tried increasing the locked memory by editing the /etc/security/limits.conf file on the frontend and adding the lines

*            hard   memlock           unlimited
*            soft    memlock           unlimited

then copying the limits.conf file to all of the compute nodes, but the error persists.The parallel program still runs despite the error.

Let me know if you need any other info. Thanks in advance for any help.

-Brian

Hello,

I wanted to ask if somebody could explain how i can calculate the memory usage for the intel optimized HPCG from grid size nx ny nz ?

Thank you for any help :D

Hi guys,

I have a SLURM cluster setup with intelmpi and Ansys CFX.

Here are my settings for the jobs:

export I_MPI_DEBUG=5
export PSM_SHAREDCONTEXTS=1
export PSM_RANKS_PER_CONTEXT=4
export TMI_CONFIG=/etc/tmi.conf
export IPATH_NO_CPUAFFINITY=1
export I_MPI_DEVICE=rddsm
export I_MPI_FALLBACK_DEVICE=disable
export I_MPI_PLATFORM=bdw
export SLURM_CPU_BIND=none
export I_MPI_FABRICS=shm:tmi
export I_MPI_TMI_PROVIDER=psm
export I_MPI_FALLBACK=1

I have also the intelmpi 5.0.3 module loaded under Centos 7

And also the simulation starts but the traffic does not go trought ib0 interfaces.

This is the output from the debug:

[0] MPI startup(): Multi-threaded optimized library
[0] MPI startup(): shm and tmi data transfer modes
[8] MPI startup(): shm and tmi data transfer modes
[2] MPI startup(): shm and tmi data transfer modes
[10] MPI startup(): shm and tmi data transfer modes
[4] MPI startup(): shm and tmi data transfer modes
[12] MPI startup(): shm and tmi data transfer modes
[1] MPI startup(): shm and tmi data transfer modes
[9] MPI startup(): shm and tmi data transfer modes
[3] MPI startup(): shm and tmi data transfer modes
[15] MPI startup(): shm and tmi data transfer modes
[6] MPI startup(): shm and tmi data transfer modes
[14] MPI startup(): shm and tmi data transfer modes
[5] MPI startup(): shm and tmi data transfer modes
[11] MPI startup(): shm and tmi data transfer modes
[7] MPI startup(): shm and tmi data transfer modes
[13] MPI startup(): shm and tmi data transfer modes
[0] MPI startup(): Rank    Pid      Node name                 Pin cpu
[0] MPI startup(): 0       12614    qingclinf-01.hpc.cluster  {0,1,2,20,21}
[0] MPI startup(): 1       12615    qingclinf-01.hpc.cluster  {3,4,22,23,24}
[0] MPI startup(): 2       12616    qingclinf-01.hpc.cluster  {5,6,7,25,26}
[0] MPI startup(): 3       12617    qingclinf-01.hpc.cluster  {8,9,27,28,29}
[0] MPI startup(): 4       12618    qingclinf-01.hpc.cluster  {10,11,12,30,31}
[0] MPI startup(): 5       12619    qingclinf-01.hpc.cluster  {13,14,32,33,34}
[0] MPI startup(): 6       12620    qingclinf-01.hpc.cluster  {15,16,17,35,36}
[0] MPI startup(): 7       12621    qingclinf-01.hpc.cluster  {18,19,37,38,39}
[0] MPI startup(): 8       12441    qingclinf-02.hpc.cluster  {0,1,2,20,21}
[0] MPI startup(): 9       12442    qingclinf-02.hpc.cluster  {3,4,22,23,24}
[0] MPI startup(): 10      12443    qingclinf-02.hpc.cluster  {5,6,7,25,26}
[0] MPI startup(): 11      12444    qingclinf-02.hpc.cluster  {8,9,27,28,29}
[0] MPI startup(): 12      12445    qingclinf-02.hpc.cluster  {10,11,12,30,31}
[0] MPI startup(): 13      12446    qingclinf-02.hpc.cluster  {13,14,32,33,34}
[0] MPI startup(): 14      12447    qingclinf-02.hpc.cluster  {15,16,17,35,36}
[0] MPI startup(): 15      12448    qingclinf-02.hpc.cluster  {18,19,37,38,39}
[0] MPI startup(): I_MPI_DEBUG=5
[0] MPI startup(): I_MPI_FABRICS=shm:tmi
[0] MPI startup(): I_MPI_FALLBACK=1
[0] MPI startup(): I_MPI_INFO_NUMA_NODE_DIST=10,21,21,10
[0] MPI startup(): I_MPI_INFO_NUMA_NODE_MAP=qib0:0
[0] MPI startup(): I_MPI_INFO_NUMA_NODE_NUM=2
[0] MPI startup(): I_MPI_PIN_MAPPING=8:0 0,1 3,2 5,3 8,4 10,5 13,6 15,7 18
[0] MPI startup(): I_MPI_PLATFORM=auto
[0] MPI startup(): I_MPI_TMI_PROVIDER=psm

But there is not traffic over infinabd

        inet 10.0.2.1  netmask 255.255.255.0  broadcast 10.0.2.255
        inet6 fe80::211:7500:6e:de10  prefixlen 64  scopeid 0x20<link>
Infiniband hardware address can be incorrect! Please read BUGS section in ifconfig(8).
        infiniband 80:00:00:03:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00  txqueuelen 256  (InfiniBand)
        RX packets 121  bytes 23835 (23.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 118  bytes 22643 (22.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I did chmod 666 on the /dev/ipath and /dev/infiniband* on the compute nodes.

the /etc/tmi.conf has the library.

Why are the sims running ok and will not over infinband. I can ping and ssh onver infinband but canno use it.

Thanks in advance.

According to part 4.3 of release notes for Parallel studio 2016 Update 3: "The operating systems listed below are supported by all components on Intel® 64 Architecture." But if I try to install this on Debian 8, I get:

Intel(R) Trace Analyzer and Collector 9.1 Update 2 for Linux* OS: Unsupported OS
Intel(R) Cluster Checker 3.1 Update 2 for Linux* OS: Unsupported OS

Why is that, are the release notes incorrect?

Hi all,

### The problem at hand ###

I am trying to model a workflow where a manager process and multiple worker processes have multiple "conversations" simultaneously.

The manager process had a "main thread" that sends messages to each workers, waits for them to do something and picks up the results (this entire pattern being repeated many times).
It also has a secondary thread that listens on messages from the workers and processes them independently of the main computations (sending back a status to the workers).
The secondary thread is very simple: it looks as follows (pseudo-code):

while (true) {

    status = probe(comm)
    if (status.tag == relatedToThisThread) {
        msg = receive(comm, status.tag, status.source)
        status = process(msg)
        send(comm, status)
    }
}

From the worker's point of view, they may issue calls to the secondary thread at any time in the middle of their work or between two pieces of work.

I tried 2 different approaches so far to model this workflow:

### Case 1: 1 Manager( 1 communicator, 2 threads); N Workers (1 communicator, 1 thread each) ###

The Manager has one communicator with which it can communicate with all Workers.
Both threads share a reference to the same communicator that is hooked to the N Worker processes.

I sometimes get failures where even though I am calling "receive" with a given tag and source; messages addressed to different threads seem to get intermingled occasionally (can be made reproducible by having a large number of worker processes).

### Case 2: 1 Manager ( N communicators, N + 1 threads); N Workers (1 communicator, 1 thread each) ###

The Manager has one communicator per worker (mostly to improve fault tolerance). Each communicator is referenced to by both the main thread and one of the N secondary threads.

This seems more reliable. In particular, it scales better with a large number of workers. However, I cannot convince myself that the same problem I observed in case 1 cannot happen here.

### My question ###

I was wondering if anyone had had to deal with a similar situation. What strategy did you use to solve these issues?

From my initial readings, it seems that MPI_Mprobe and MPI_Mreceive may be the key to solving this problem; but I would welcome any suggestion while I'm experimenting with this situation.

I am performing several benchmarks with Intel MPI and SGI MPT. Intel MPI has been 1.2 times faster them MPT. However,  i have three specifications who is need to follow:

 1 - Use the faster MPI - is this case is Intel MPI
 2 - Prevent the user to access to the processing node through ssh
 3 - Use Torque queue system

 With MPT i was  able to prevent the SSH access and use torque queue system, but, MPT is not the faster MPI.

 Have Intel MPI  any way to integrate with torque queue system as to a be abble to block SSH access (like OpenMPI has --with-tm ) ?

  Best regards.

I have a couple of users that are experiencing intermittent failures using Intel MPI (typically versions 4.1.0 or 4.1.3) on RedHat 6.4 and 6.7 systems using Mellanox OFED 2.0 and 3.1 respectively.

The error messages being seen are as follows:

[80:node1] unexpected reject event from 16:node2
Assertion failed in file ../../dapl_conn_rc.c at line 992: 0

or

[0:node1] unexpected DAPL event 0x4003

Assertion failed in file ../../dapl_init_rc.c at line 1332: 0

These errors are happening extremely intermittently on both systems. I believe that the jobs are relying on default values for  I_MPI_FABRICS (shm:dapl) and I_MPI_DAPL_PROVIDER (should be ofa-v2-mlx4_0-1 on both systems).

It seems like these are DAPL layer errors.  Any ideas on what might cause these sorts of intermittent failures?

Thanks!

I am experimenting with MPI (world) across different systems. One system is a Xeon Phi KNL (64 cores, 256 logical processors) called KNL, and the other system is an E5-2620v2 (6 cores, 12 logical processors) called Thor.

What I intended to do is do is to partition the KNL into 2 ranks, and then run the 3rd rank on the E5-2620v2.

mpirun -n 3 -ppn 1 -hosts KNL,KNL,Thor program arg arg arg

This launches 2 ranks on KNL and 1 rank on Thor as expected

So far, so good.

Rank 0 was located on KNL
Rank 1 on Thor
Rank 2 on KNL

Aren't the nodes associated in the -hosts order?

What is the recommended way to control rank placement?

When I try 8 on KNL and 1 on Thor, the distribution is quite goofy. (likely my issue)

I tried using the : separator to sequester the KNL on left side and Thor on right side of :, but mpirun choked on the comman line.

(I am a noob at this)

Jim Dempsey

Hi, I'm having some troubles with the attached code SendReceive.c.

The idea is to open a dataset with process p-1 and then to distribute it to the remaining processes. This solution works when the variable ln (local number of elements) is less than 8182. When I increase the number of elements I've the following error:

mpiexec -np 2 ./sendreceive 16366
Process 0 is receiving 8183 elements from process 1
Process 1 is sending 8183 elements to process 0
Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(224)...................: MPI_Recv(buf=0x2000590, count=8183, MPI_DOUBLE, src=1, tag=MPI_ANY_TAG, MPI_COMM_WORLD, status=0x1) failed
PMPIDI_CH3I_Progress(623).......: fail failed
pkt_RTS_handler(317)............: fail failed
do_cts(662).....................: fail failed
MPID_nem_lmt_dcp_start_recv(288): fail failed
dcp_recv(154)...................: Internal MPI error! cannot read from remote process

I'm using the student license of the intel implementation of mpi (obtained by installing Intel® Parallel Studio XE Cluster Edition (includes Fortran and C/C++)).

Is this a limitation of the licence? Otherwise, what I'm doing wrong?

Thank you for using Intel® software development tools. We are committed to making the best possible software and platforms to meet your development needs. Your personal experience with our products is extremely valuable to us and we want to know how we can do better.

Click here to share your thoughts by completing a 10-min survey on Intel® Parallel Studio XE, as well as your general tool usage. We value your opinion and look forward to your feedback. If you have any questions, post a comment below.

If you reside outside of the United States and decide to participate in this survey, you are agreeing to have your personal data transferred to and processed in the United States. Refer to Privacy for more details.

I find this parameter I_MPI_WAIT_MODE  on Intel MPI 2017 reference guide. It said that "Use the Native POSIX Thread Library* with the wait mode for shm communications". My application is MPI + Pthread.  MPI is intel MPI2017

So I add “export I_MPI_WAIT_MODE=1” into my scripts,  it print the following error. Do you have some suggestion? thanks

Error in system call pthread_mutex_unlock: Operation not permitted

    ../../src/mpid/ch3/channels/nemesis/netmod/tmi/tmi_poll.c:629

Error in system call pthread_mutex_unlock: Operation not permitted

    ../../src/mpid/ch3/channels/nemesis/netmod/tmi/tmi_poll.c:629

Error in system call pthread_mutex_unlock: Operation not permitted

    ../../src/mpid/ch3/channels/nemesis/netmod/tmi/tmi_poll.c:629

Error in system call pthread_mutex_unlock: Operation not permitted

    ../../src/mpid/ch3/channels/nemesis/netmod/tmi/tmi_poll.c:629

 << OLE Object: Picture (Device Independent Bitmap) >>

If I run with IntelMPI with a forked process ( & ), it leaves mpiexe.hydra:

30545 pts/25   00:00:00 Job1
30546 pts/25   00:00:00 Job2
30646 pts/25   00:00:00 mpirun
30651 pts/25   00:00:00 mpiexec.hydra <defunct>

Details:
IntelMPI 5.1.2.150
JOB1 is run with "&", which creates JOB2 which runs mpirun with "-configfile"
If I_MPI_PROCESS_MANAGER=mpd is used, mpiexec.hydra is not left.
If JOB1 is run without "&", mpiexec.hydra is not left.
If I set "-v" I see at the end:

[proxy:0:0@sudev604] got pmi command (from 10): finalize
[proxy:0:0@sudev604] PMI response: cmd=finalize_ack
[proxy:0:0@sudev604] got pmi command (from 12): finalize
[proxy:0:0@sudev604] PMI response: cmd=finalize_ack

My results are fine, the issue is that mpiexe.hydra is left.

I have not been able to find this issue anyplace else.

I am trying to setup a heterogeneous MPI configuration and need some assistance. I've followed the instructions in the Intel(R) MPI Library for Windows* OS Developer Guide, section 5.5.

This is what I see

C:\Downloads\SANRAL\engineMPIOpenMPwithCrames\x64\Release>mpiexec -demux select -bootstrap service -genv I_MPIFABRICS shm:tcp -hostos linux -n 1 -host KNL hostname
Error connecting to the Service
[mpiexec@i72600K] ..\hydra\utils\sock\sock.c (270): unable to connect from "i72600K" to "KNL" (No error)

The Windows machine (i72600K is Windows 7, and it is configured as a WORKGROUP. The Linux machine (KNL) is configured without Domain controller. I also have a second Linux machine (Thor). I can run mpiexec as

Multiple ranks on i72600K
Multiple ranks on KNL
Multiple ranks on Thor
Multiple ranks on KNL + Thor

My issue is getting KNL + Thor + i72600K to work together.

Yes, I can ping cross systems And the MPI app, Windows variant, running as 1 or 2 ranks on i72600K (Windows) can connect to a MySQL database running on Thor (Linux).

The Window document (named above) may not be listing enough information as to what is require on Linux or Windows.

In a different documentation it states that the hydra service should be started with "-bootstrap ssh"?

Your help would be appreciated.

Jim Dempsey

I have install Intel MPI 5.1.0.031 on two different Windows 7 machines with ip addresses, 192.168.2.144 and 192.168.2.128. 

I am unable to launch an MPI process across these two machines using the following test,

mpiexec.exe -delegate  -ppn 1 -n 2 -hosts 192.168.2.144,192.168.2.128 hostname

It lists the hostname of the first computer where I am launching the process from and then the command just hangs.  No error message or warning...

From each machine I can launch a local mpiexe

From 192.168.2.144
mpiexec.exe -delegate  -ppn 1 -n 1 -hosts 192.168.2.144 hostname

From 192.168.2.128
mpiexec.exe -delegate  -ppn 1 -n 1 -hosts 192.168.2.128 hostname

Output from both machines for these commands says success,

From  both machines I can check status of hydra_service and smpd on the other machine for example from 192.168.2.144 I can run,

C:\Windows\system32>hydra_service -status 192.168.2.128
hydra service running on 192.168.2.128

C:\Windows\system32>smpd.exe -status 192.168.2.128
smpd running on 192.168.2.128

C:\Windows\system32>mpiexec -validate
SUCCESS

I have disabled the Symantec firewall on both machines. 

Thanks Andy

Hello,

So I downloaded the Intel MPI Library 2017 for Linux, and after it finished installing, I noticed mpiic is not there, although installation finished successfully. Shouldn't it be? 

I have from before Open MPI working, but I couldn't manage to build HPL that way, so I turned to Intel's MPI. I don't have a lot of experience on this, so I'm not sure what to try next, if someone has any suggestions...

Hi there, 

I have two identical computers. Each with 4-dual cores and Intel Parallel Studio Cluster edition installed. I have a Fortran code, which on a single machine takes 9 hours to run if not parallelized and 1.37 hours to run once parallelized (OpenMP). I would like to connect the two computers and take advantage of both machines at the same time. Adding the MPI layer to the code should be straightforward, my problem is much more basic. 

How can I connect the two computers? They are on the same LAN network and I can easily remote login from one into other and vice-versa, but how to I pool the two computers for MPI computation?

Thanks,

I'm currently running MPI (5.03.048) on Windows 10 (64 bit) 8 core machine with 32GB RAM. I am using MPI_COMM_SPAWN from a C++ app (that is launched using mpiexec.exe -localonly -n 1) to spawn N MPI workers - actually, I call MPI_COMM_SPAWN N times each for a single worker (FT pattern). If I try to spawn 21 or more workers, I often get a crash from the MPI library itself. This is not consistent i.e. sometimes I can spawn 32 workers with no problems, sometimes I get a problem with 21. Has anyone else come across such a problem? Can anyone suggest what the issue might be?

Hello,

   When I run the VASP calculation software, the calculation job was "okay" during one day. After that, some jobs show the following error:

[mpiexec@node46.chess.com] HYD_pmcd_pmiserv_send_signal (./pm/pmiserv/pmiserv_cb.c:221): assert (!closed) failed

[mpiexec@node46.chess.com] ui_cmd_cb (./pm/pmiserv/pmiserv_pmci.c:128): unable to send SIGUSR1 downstream
[mpiexec@node46.chess.com] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status
[mpiexec@node46.chess.com] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:388): error waiting for event
[mpiexec@node46.chess.com] main (./ui/mpich/mpiexec.c:745): process manager error waiting for completion

     IMPORTANT: sometimes some of jobs are finished without error. some of them cannot be completed with above error. What should I do?