Our clusters are currently computing more active hours that were processed in the first two years in total. Today we exceeded 300,000 CPU hours of work, that's more than 34 computational years being processed currently.
There are several MPI issues which users need to keep in mind. Nodes 604 and 605 are not completely infiniband enabled. This means that some selections of servers will not work with infiniband. We would recommend that advanced MPI users steer clear of these nodes by making use of the -l directives to select specific servers:
#PBS -l nodes=srvslshpc600:ppn=2+srvslshpc601:ppn=2+srvslshpc602:ppn=2
will start a 6 core job over 3 servers with working IB cables. It's recommended that users monitor the dashboard to determine which nodescores are free prior to starting an MPI job. We apologize for this inconvenience and we will have this sorted out during the June maintenance slot.
There are several MPI issues which users need to keep in mind. Nodes 604 and 605 are not completely infiniband enabled. This means that some selections of servers will not work with infiniband. We would recommend that advanced MPI users steer clear of these nodes by making use of the -l directives to select specific servers:
#PBS -l nodes=srvslshpc600:ppn=2+srvslshpc601:ppn=2+srvslshpc602:ppn=2
will start a 6 core job over 3 servers with working IB cables. It's recommended that users monitor the dashboard to determine which nodescores are free prior to starting an MPI job. We apologize for this inconvenience and we will have this sorted out during the June maintenance slot.