Co-Design is a new HPC infrastructure design, essentially the next level of performance and scale. Co-design in a nutshell is the ability to reduce as much of the CPU cycle operations to a design whereby there are synergies created between the software and hardware. The software will execute on every active component that exists in the data center, part of the application will execute on the CPU, part of it will execute on the network and the same for storage.
The intelligence of the interconnect is becoming more and more valuable in building tomorrow’s clusters using Co-Design. Offloading technologies which we are already making use of at UCT to deliver some of our services such as RDMA, GPU Direct, Direct Communications ensures the delivery of an intelligent interconnect.
The Mellanox Switch IB2 will be released soon is a switch which uses a technology called SHArP ( Scalable Hierarchical Aggregation Protocol). This SHArP technology enables the switch silicon to manage and execute MPI operations on the network. The biggest benefit is the reduction of network traffic which a typical MPI job generates between nodes which is initiated by the workers. The Switch-IB2 enables the switch network to operate as a Co-Processor ( similar to that of a Intel PHI, Nvidia GPU ) and since the switch connects all the worker nodes together it would make sense for it to handle the collective operations of a MPI job. Programmers / application developers and end-users would not need to change any of their existing MPI code to make use of the network offloading abilities. The network becomes a co-processor which will be built into the MPI libraries which is referenced when running a MPI workload. Mellanox are working with other mpi companies to take advantage of this. At UCT we make use of the MLNX_OFED distribution so the MPI libraries will be updated in time.
There are currently two features of the Mellanox Switch-IB2 which really stood out for me. The first is that all network related operations which are executed on the CPU are now being executed on the network switch itself. This ofcourse will free up the CPU cycles on the host to be used by other more important tasks. The second feature is that the network now becomes a co-processor.
It would be great to have this as part of our next cluster.