and OpenSees using the new interpreters OpenSeesSP and OpenSeesMP
I am trying to build OpenSees with MPI for use on on Linux system with SGE parallel job submission:
The OS is Red Hat Linux Enterprise 5.
I'm using the Intel Compiler Suite, version 11.1
Our MPI package is mvapich2 version 1.6
TCL & TK 8.4 librariries are available in /usr/lib64
I chose the the RANGER Makedef file, which met most of my criteria, and modified it. the modification is here:
The libraries build, but I get link errors:
There's two groups of them- the frist set want "Machinebrokers" & "The Channels"; the second (dmavtec, dlsolve)
are maybe missing or mismatched numerical libararies.
I am trying to link to a locally -build version of the MKL libraries for Intel, but I'm willling to give that up for now if I can get it built.
Any help is approeciated.
these lines in SRC/tcl/command.cpp:
extern MachineBroker *theMachineBroker;
extern Channel **theChannels;
extern int numChannels;
extern int rank;
extern int np;
There are member functions in MachineBroker to return channel, rank and NP- not sure why it does this.
I'd be interested in knowing if anyone else building OpenSees in _PARALLEL_INTERPRETERS mode got past these errors- looks like one of the posters right before me had the same problem.
Thanks, I will try that. Do I need to download everything under SRC to pick up all the changes
since the 2.2.2 Release? (If this is a silly question, it's because I'm still a CVS guy, and my subversion
is pretty minimal )
This machine has infinband network cards on it . If I use mvapich2 as my MPI package,
OpenSessMP fails without ever getting to a shell prompt. (mvapich2 on my system is
configured to use Infniband).
With the lastest version of mavpich2 (1.6), all i get is a floating point exception.
If I try it with version 1.4, it sits fro 30 seconds, then gives me these errors:
dqcneh033.crc.nd.edu.19268ipath_wait_for_device: The /dev/ipath device failed to appear after 30.0 seconds: Connection timed out
dqcneh033.crc.nd.edu.19268PSM Could not find an InfiniPath Unit on device /dev/ipath (30s elapsed) (err=21)
psm_ep_open failed with error PSM Could not find an InfiniPath Unit
Fatal error in MPI_Init: Internal MPI error!, error stack:
MPIR_Init_thread(311): Initialization failed
MPID_Init(191).......: channel initialization failed
(unknown)(): Internal MPI error![
I decide to try building it without infinband, and used openMPI - this works fine. I can get a OpenSees shell prompt, and source in and run
I need to run down the above error to figure out what's going on.
and not the OpenSees software. I'll be able to verify this after we reload the infiniband stuff this weekend.
Thanks for all the help,
> After quite a bit of hair pulling, in appears that out problems with
> infiniband are in the infiniband drivers,
> and not the OpenSees software. I'll be able to verify this after we reload
> the infiniband stuff this weekend.
> Thanks for all the help,
i wonder what's the verdict? It's a bit complicated issue to deal with.