{"id":3513,"date":"2018-10-05T13:08:37","date_gmt":"2018-10-05T11:08:37","guid":{"rendered":"http:\/\/ucthpc.uct.ac.za\/?p=3513"},"modified":"2018-10-06T20:34:09","modified_gmt":"2018-10-06T18:34:09","slug":"infiniband-cabling","status":"publish","type":"post","link":"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/","title":{"rendered":"Infiniband cabling"},"content":{"rendered":"<p>Thermal regulation and air flow are two important parts of any HPC installation. Adding Infiniband increases the amount of physical material at the back of the servers which can obstruct fans,\u00a0reduce airflow and lead to overheating and damaged equipment. EDR cabling is significantly thicker than FDR which adds to the difficulty.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2018\/10\/rack.jpg\" \/><\/p>\n<p>Many thanks to Moegemat Alexander from Workplace Services for sorting out the cabling in our racks. The work is not quite finished as we still need to add four more GPU servers and do some additional cable work but it&#8217;s looking a lot neater than it did a week ago. The power cables have two colours as we run each dual power supply from a different DB phase, each one being powered from one of two UPS. The number of unused plug points is due to the fact that we keep the amp rating of each PDU below 50% of its rated capacity so that in the event we lose a UPS or DB board the remaining PDU can power the entire stack.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Thermal regulation and air flow are two important parts of any HPC installation. Adding Infiniband increases the amount of physical material at the back of the servers which can obstruct fans,\u00a0reduce airflow and lead to overheating and damaged equipment. EDR cabling is significantly thicker than FDR which adds to the difficulty. Many thanks to Moegemat&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[6,4,24],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Infiniband cabling - UCT HPC<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Infiniband cabling - UCT HPC\" \/>\n<meta property=\"og:description\" content=\"Thermal regulation and air flow are two important parts of any HPC installation. Adding Infiniband increases the amount of physical material at the back of the servers which can obstruct fans,\u00a0reduce airflow and lead to overheating and damaged equipment. EDR cabling is significantly thicker than FDR which adds to the difficulty. Many thanks to Moegemat...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/\" \/>\n<meta property=\"og:site_name\" content=\"UCT HPC\" \/>\n<meta property=\"article:published_time\" content=\"2018-10-05T11:08:37+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-10-06T18:34:09+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2018\/10\/rack.jpg\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/bd981a3acf3b5495041e7884626c1157\"},\"headline\":\"Infiniband cabling\",\"datePublished\":\"2018-10-05T11:08:37+00:00\",\"dateModified\":\"2018-10-06T18:34:09+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/\"},\"wordCount\":176,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"articleSection\":[\"hardware\",\"hpc\",\"Infiniband\"],\"inLanguage\":\"en-ZA\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/\",\"name\":\"Infiniband cabling - UCT HPC\",\"isPartOf\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\"},\"datePublished\":\"2018-10-05T11:08:37+00:00\",\"dateModified\":\"2018-10-06T18:34:09+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/#breadcrumb\"},\"inLanguage\":\"en-ZA\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ucthpc.uct.ac.za\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Infiniband cabling\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#website\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"name\":\"UCT HPC\",\"description\":\"University of Cape Town High Performance Computing\",\"publisher\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-ZA\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#organization\",\"name\":\"University of Cape Town High Performance Computing\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"contentUrl\":\"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png\",\"width\":450,\"height\":423,\"caption\":\"University of Cape Town High Performance Computing\"},\"image\":{\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/bd981a3acf3b5495041e7884626c1157\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-ZA\",\"@id\":\"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/925fcc8bb19120dc6cb9841c0ace3e14?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/925fcc8bb19120dc6cb9841c0ace3e14?s=96&d=mm&r=g\",\"caption\":\"admin\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Infiniband cabling - UCT HPC","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/","og_locale":"en_US","og_type":"article","og_title":"Infiniband cabling - UCT HPC","og_description":"Thermal regulation and air flow are two important parts of any HPC installation. Adding Infiniband increases the amount of physical material at the back of the servers which can obstruct fans,\u00a0reduce airflow and lead to overheating and damaged equipment. EDR cabling is significantly thicker than FDR which adds to the difficulty. Many thanks to Moegemat...","og_url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/","og_site_name":"UCT HPC","article_published_time":"2018-10-05T11:08:37+00:00","article_modified_time":"2018-10-06T18:34:09+00:00","og_image":[{"url":"http:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2018\/10\/rack.jpg"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/#article","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/"},"author":{"name":"admin","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/bd981a3acf3b5495041e7884626c1157"},"headline":"Infiniband cabling","datePublished":"2018-10-05T11:08:37+00:00","dateModified":"2018-10-06T18:34:09+00:00","mainEntityOfPage":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/"},"wordCount":176,"commentCount":0,"publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"articleSection":["hardware","hpc","Infiniband"],"inLanguage":"en-ZA","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/","url":"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/","name":"Infiniband cabling - UCT HPC","isPartOf":{"@id":"https:\/\/ucthpc.uct.ac.za\/#website"},"datePublished":"2018-10-05T11:08:37+00:00","dateModified":"2018-10-06T18:34:09+00:00","breadcrumb":{"@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/#breadcrumb"},"inLanguage":"en-ZA","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ucthpc.uct.ac.za\/index.php\/2018\/10\/05\/infiniband-cabling\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ucthpc.uct.ac.za\/"},{"@type":"ListItem","position":2,"name":"Infiniband cabling"}]},{"@type":"WebSite","@id":"https:\/\/ucthpc.uct.ac.za\/#website","url":"https:\/\/ucthpc.uct.ac.za\/","name":"UCT HPC","description":"University of Cape Town High Performance Computing","publisher":{"@id":"https:\/\/ucthpc.uct.ac.za\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ucthpc.uct.ac.za\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-ZA"},{"@type":"Organization","@id":"https:\/\/ucthpc.uct.ac.za\/#organization","name":"University of Cape Town High Performance Computing","url":"https:\/\/ucthpc.uct.ac.za\/","logo":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/","url":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","contentUrl":"https:\/\/ucthpc.uct.ac.za\/wp-content\/uploads\/2015\/09\/logocircless.png","width":450,"height":423,"caption":"University of Cape Town High Performance Computing"},"image":{"@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/bd981a3acf3b5495041e7884626c1157","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-ZA","@id":"https:\/\/ucthpc.uct.ac.za\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/925fcc8bb19120dc6cb9841c0ace3e14?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/925fcc8bb19120dc6cb9841c0ace3e14?s=96&d=mm&r=g","caption":"admin"}}]}},"_links":{"self":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/3513"}],"collection":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/comments?post=3513"}],"version-history":[{"count":5,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/3513\/revisions"}],"predecessor-version":[{"id":3521,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/posts\/3513\/revisions\/3521"}],"wp:attachment":[{"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/media?parent=3513"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/categories?post=3513"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ucthpc.uct.ac.za\/index.php\/wp-json\/wp\/v2\/tags?post=3513"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}