Hello, again.
I have another (hopefully simple) question regarding rosetta, specifically RosettaDock.
I have compiled rosetta3.4 with MPI support (thanks smlewis!), and am trying to run a test docking job to ensure it is working. This cluster uses PBS as the scheduler.
My PBS script is as follows:
------------------------
#!/bin/sh
#PBS -l select=2:mpiprocs=4:ncpus=4
#PBS -q nopreempt
#PBS -l walltime=1:00:00
#PBS -j oe
module load mpich2-intel/1.3
cd $PBS_O_WORKDIR
rbin=/home/eep3m/rosetta_source/bin
rdb=/home/eep3m/rosetta_database
pdb=1brs.pdb
outfile=scores_mpi
flags="-s $pdb -dock_pert 3 8 -spin -ex1 -ex2aro -nstruct 25 -mute core.util.prof -out:file:scorefile $outfile -out:file:fullatom -mute core.io.database"
mpiexec -np 8 $rbin/docking_protocol.mpi.linuxiccrelease $flags --database $rdb/ > rosetta_output
-----------------
When I submit this to the queue, I get the following error in return:
-----------------
Job is running on node(s):
------------------------
lc5-compute-3-43 lc5-compute-4-45
------------------------
ERROR: Option matching -p4wd not found in command line top-level context
mpiexec: Warning: task 0 exited before completing MPI startup.
mpiexec: Warning: task 1 was never spawned due to earlier errors.
--------------------
I have looked for the past few hours, and have not seen any instance of the flag -p4wd anywhere. I also tried running:
mpiexec -np 8 $rbin/docking_protocol.mpi.linuxiccrelease -help
and get the same error.
I am forgetting / overlooking something simple?
Thanks again for all of your help!
--Eddie
Apparently PBS (or mpiexec) is injecting -p4wd into Rosetta's options list. The ERROR line is the standard message for non-existent options.
Very brief googling suggests that PBS/mpiexec/MPI are not playing well together (in other words, it's probably not something I can fix).
Here's one thing to try: try running docking_protocol.mpi.linuxiccrelease with no options and no mpiexec. You should get some sort of error about an uninitialized MPI (I think).
Also try this: try mpirun instead of mpiexec. I don't know the difference, but I do know that I always use the former and not the latter. (I don't know if this is even possible or sensible to suggest).
Thanks for the quick reply (again).
Your statement about mpiexec injecting the -p4wd flag into the options list made me double check that the MPI libraries were correct. They weren't. I had to add one additional flag and now everything is playing nice, and running smoothly.
In the event that this may help someone in the future (although I think it is pretty specific to this cluster) here is the mpiexec command that worked:
mpiexec -np 8 -comm mpich2-pmi $rbin/docking_protocol.mpi.linuxiccrelease $flags --database $rdb/ > rosetta_output
Thanks again for all of your help. As someone relativly new to Rosetta, this is a fantastic resource.
Just for comparison, my job submissions look something like this:
bsub -q day -n 10 -J $cwd -o log.%J -B -N -a mvapich mpirun /path/to/rosetta/bin/mpi_msd.mpi.linuxgccrelease @options
So, yeah, different formats for different clusters.