We've started to test the new BackBlaze disk unit. It's presenting an iSCSI mount but we don't have sufficient switch ports to wire up all our worker nodes. As a compromise we've made the iSCSI mount available to nodes 408 (active) and 409 (passive) and exported a 2TB lun from node 408 via NFS to all other nodes, including the head node.
Test results show that for optimal usage NFS should be avoided. The best we could get via the NFS mount was a peak of 20MB/s and 10Mb/s sustained whereas the direct iSCSI data transfer direct from node 408 hit a peak of 60MB/s and wrote at 20MB/s sustained (see lower graph). The upper graph is CPU utilization. As the transfer was from an existing NFS mount on /home the CPU took a double whammy. Ideally all user disk IO should take place on one disk mount.
